Actually most USB's are OK, but not all ideal. Try a few different that you have lying around and if the problem persists you can rule out the USB factor. Personally I use Supertalent Pico since 7 years and never had any problems or long load times. My emergency stick is an old Verbatim and it actually performs as good as the Supertalent's.
Make sure you re-format them and do a fresh Rekordbox export with just a few songs.