No, only the server components were installed, agent and dev tools on separate machines. It wasn't present on the machine with only the dev tools installed either.
I ran the installer again now on the server, selecting agent and dev tools as well, and that added the Datastore migration utility. Seems a bit strange, but at least it's present now.
Thanks for pointing me in the right direction!