Comentário oficial
If, within rekordbox, you see the update manager, you can use that to download the new version. If you do that, you will be prompted if you'd like to remove the previous version (which you can do - on Windows, each is installed into its own version-named folder, so you can run multiple versions in parallel).
By downloading a new version straight from the rekordbox.com website, you're not doing anything different - it simply won't prompt you to remove the previous installation.
You can safely go to the uninstaller for the old version, or through the Windows program removal process to get rid of the 2nd one.