Can we stop Out of Office operating in a space?
Yes, but not from inside the app: block the app's access to that space in your organization's data security policy at admin.atlassian.com, after which nothing there is reassigned or commented on.
That takes organization administrator rights, and naming this one app rather than all eligible apps at once also takes an Atlassian Guard Standard or Premium subscription.
Related pages
Restricting Out of Office to Specific Spaces - where this is covered in full.
Security and Privacy - more questions on this topic.