I've moved around my Toolbars and buttons but when I restart Outlook, they are back in their original position.
How can I make them stick in their location?
The issue depends a bit on which Toolbars you have moved around. The default Toolbars of Outlook and the ones you have created yourself should stick in their location.
However, Outlook does not control the location of 3rd party add-in toolbars. The add-in itself determines or should remember the location of the toolbar. If it doesn't you can contact the developer of your add-in to submit your request.
If they state that their add-in should remember it, you might want to do a reset on the Toolbar and your add-in cache. Click here for more information on how to do that.
If they state that their add-in won’t remember it and/or they cannot implement the feature for whatever reason, then you can also use the tool FixMyBars from Outlook-Stuff to lock the toolbars in place.
Main dialog of FixMyBars to force a location of a Toolbar.