How to mass revoke app licenses?

If your new MDM solution is following Apple's guidelines (and if they're not, are you sure you want to use them? 😉), you should only need to install the new VPP token on the new MDM server. The new server should detect that the account was not managed by it and either revoke all the licenses and retire all the users from VPP.

You might be able to get Profile Manager to revoke all your licenses with the following command (it's all one line):

sudo -u _devicemgr psql -U _devicemgr -d devicemgr_v2m0 -h /Library/Server/ProfileManager/Config/var/PostgreSQL -c "DELETE FROM library_items_assets; SELECT dm_begin_vpp_sync();"

You'll need to wait for the VPP sync to finish, which may take some time. Also note that book licenses can't be revoked in VPP.

Finally, this may delete other non-VPP assignments you've made (i.e., enterprise apps), too, but if you're decommissioning your Profile Manager server, that's probably not much of a concern to you. YMMV.

Well, if they're following Apple guidelines, once you install the VPP token on the new MDM server, it will mark the VPP account as owned by that server and Profile Manager will no longer attempt to use the account. You can also uncheck the Volume Purchase Program checkbox in Server.app to make Profile Manager stop using the VPP token.