So I got impatient and took the plunge despite warnings to hold off and it's been fine so far. The only snag for your library is you need to export the Music library to XML every time you make changes but that's not much effort:
Everything else is working fine for me, all drivers and rekordbox seems ok. The Music app definitely doesn't max out CPU anymore which is good but the simplification does annoy me a little bit. They've made some small changes which are stupidly annoying and if you use list view in iTunes and have lots of playlists be prepared for all your field options to be reset. Also if you use Doug Scripts be aware most of those won't work anymore although every day he's remaking them for the Music app.