Expecting that you have the same user on both computers (so the paths in the rekordbox library are the same on both computers), I'd try this:
1) place the rekordbox library and your music directories on a Dropbox or Google drive (this will ensure the content and files are the same on both computers)
2) Symlink the original locations of the directories so they are pointing to the Dropbox or GDrive directories (so the REkordbox is accessing the shared content but still thinks its using the local one)
This is pretty easy to do BUT you must always ensure that the files are NEVER accessed from the Rekordbox on both computers at the same time. Also make sure the absolute path to the files remains the same on both computers and you should be fine..
Haven't tried this with RB yet but I am using it with other apps regularly and it works pretty flawless and I see no reason why this could not work with the RB... - the symlink is a magic how to do such things :-)
Make a dummy library you can mess with and give it few tries.