Hide flagged items of other mailboxes in To-Do Bar

I’ve got an additional account in Outlook which I share with someone else. We often flag emails for each other which then show up in the To-Do Bar as well were it mixes with my own tasks and flagged mails.

I tried to disable this mailbox from displaying reminders in the To-Do Bar but that option appears not to be available.

Is there a way to only show tasks and reminders from emails in my personal mailbox and not from another?