Rekordbox USB Transfer Speed

@pfaffian > Good luck with that. As @tasos says, you'd have to change the bundle every time you update ONE file.

@tasos > The database and accompanying file system is quite complex, all I know is a) it's not likely to change (I've been asking since day 1), and b) get the fastest drive you can afford.

@tasos @pioneer

You wouldn't have to copy the whole bundle.  Properly written software would simply write the changeset to, say, a sqlite database that is stored on the key.  Yes, it would require a firmware update on the CDJs.  The point is, Pioneer should fix the problem rather than trying to say it's too complicated.

One might expect that Pioneer has employed software developers who know the various database options available to them...

I know Pulse, i already have the fastest think out there :)

It's a nice thought from pfaffian, but who knows if it's even possible to build, or what all the cost would be to do all the changes that are required to get there (IF it would be possible).

I am happy with my usb3.0 ssd. No errors, and for example the update 4.06 did "cost" me 20 minutes to sync 2 drives. I can live with that.

then why are they using side by side analysis files?

why not just write all the data needed to the tracks itself like other companies do? then you only have to export the track, maybe the engineers can look into that option

I see that problem still persist .. been using sandisk ultra fit 128gb for almost 1 year now, and because of some freezing on xdj, i decided to buy a new usb flash drive to try it out .. i bought kingston hyperX savage 256, and when trying to export my library (around 80gb) it took ages .. on older drive, it took about 4 hours to export everything (which is still slow), now i'm exporting to the new drive, same library, and it did 3% in 1 hour (about 9gb) .. wtf Pioneer?

Copying in finder same files took about 1minute! .. I read about creating directories and that bullsh*, but this is too much

@Ziga > rekordbox uses the same transfer mechanism as the finder does, so the speed is not because of rekordbox itself.

When you did your transfer via finder, did you copy only the audio files? Or did you also copy the heaps of smaller support files?

What is the file system format of the HyperX drive?

I copied everything that rekorbox copy in 1 hour back to internal hdd and then back to flash via finder, it took 1 minute. its hfs+ mbr

Did you include the hidden folders?

I think i know the problem.
It is rescanning of the tracks before writing to the device.
For me rescanning the whole library made that transfering the tracks was in an normal time done.

@Mark > It only does that if the tracks had been imported/analysed prior to the 4.0.8 version (I think it was that update) where the CDJ-2000NXS2 was added and additional information was made available with the tracks. After you've done that once, or re-analysed the tracks, it would export at a much faster speed.

I enabled hidden folders and it took < 4 minutes to copy all content from internal drive back to usb, this was about 20gigs of stuff, of which rekordbox took 4 hours to export to flash drive, and all music was reanalized after 4.0.8

Edit: It transfered 60% in 7 hours, still not finished .. it must be something broken in rkdb

Without being too sarcastic...

This is starting to sound very much like a company is in denial that something is wrong with their software.

Look how long this thread is and people are still having problems.

I still experience export problems, I recently bought a new 128GB USB- drive and I'm never quite confident that this will cause and error during live playback at gigs.

I'm running some export samples to give you all a better idea of results, but @Ziga, the fact it's taking you that long to export 20GB tells me your USB drive is a low-speed / low-performance drive.

The drive is fastest you can buy, but the bus is usb 2.0, but still .. same files in finder less than 4mins, in rekordbox 4 hours.. this is definetly not hardware related.. maybe rkdb have some problems if database is on one drive and actual mp3s on other (i have 256gb samsung ssd pro for system and that is where database is stored, and another 1tb wd black 7200 hard drive for music, both internal, both sata3)

20gb = 20000mb and you say you transfered that in under 4 mins? That means in under 240sec, so you get 20000mb / 240sec = 83mb/sec? How is that possible when usb2.0 has maximum speed transfer about 30mb/sec?

thats what finder said, maybe it was 5 minutes and size was 19gb, not exactly 20.. you didn get the point.. i dont care if it take 5 or 10 minutes, but if it take 3 hours, then i care.. btw whole collection (about 80gb) i started to exporting around 4pm yesterday, now is 2:30am, and i'm on 75% .. thats when i'm worried that something ist allright

Sorry mate, no offense, but under the best conditions, you need about 12 minutes to get 20gb in a usb2.0 drive (or via usb2.0 bus). Best conditions means: just 1 file, avi or zip or something, and not all that small analyze files rekordbox has, you drive must be usb3.0 so you get the maximum from the usb2.0 bus, connect the drive directly in a USB on the pc/laptop/Mac and do not use a USB splitter.
I get the point and I hope you find a solution quickly, but there are some of us who don't have any of these problems. Did you try re-installing rb?

I'm sorry to misinform you, i double checked, and it is usb 3.0 port .. i have mid 2012 macbook pro, and i was almost sure that usb ports are all 2.0, but on apple site it says, that it is 3.0, so from there the speeds. Btw, rekordbox is still exporting, its now 18hours of exporting 80gig library and still not finished, its on 89%

@BriChi

Please check try this and say if this helps:

-get an (fast) empty usb stick USB3.0 

-make an playlist with many tracks, like 500

-Re-scan/analyse all that files in that specific playlist

-write the whole playlist to your device (after rescanning al that 500 tracks)

for me i think that was my problem. before rekordbox wrote the track to the USB stick it re-scanned  that specific track and after that it will write the track down to the stick.

after re-scanning my whole library (more than 1 day at 100% CPU) the transfer speed was much more faster.

it will always be some slower than transferring it via Windows/Mac but that reason is said several times above.

Please try my tutorial and give us some feedback. when this is the problem Pioneer can make an protocol or something to get away this frustration. (because it's always 1 hour before an gig :) )

@Pulse, do you agree?

I have tested it all, for me I have a few USB 3 flash drives and ssd externals

on my Patriot Magnum 512gb it takes about 8 hours to write 41,000 tracks (roughly 300gb) which I feel is decent for a flash drive

my Samsung USB3 SSD external drive takes under an hour

BUT If I format them FAT32 then it takes literally days but OS X journaled takes and hour to 8 hours depending on the drive I use

I'm having the same issues. The scenario that Žiga described is quite accurate. When copying music directly from finder to USB stick (FAT32) it's quite fast. A few GBs in few minutes. A few days ago I exported about 2 GB of music in Rekordbox and it took about 3 hours (same USB drive).

I don't see how this difference in speed can be a USB stick problem, as copying from finder is fast and when using Rekordbox it's super slow. I have tried different USB sticks as well. The result is the same.

Of course I could buy a USB 3.0 stick but as my mac has USB 2.0 I can't see how this would make my Rekordbox transfer speeds quicker?

So Guys, i try one last time to explain about that usb speeds... When rekordbox exports it's list's etc, it does export also very small files which are the analyse files from each track. The reason the speed is slow, is exactly this. The very small files. You have to choose very carefully what usb stick/drive or hdd you buy, because not every drive has good transfer speeds on very small files.

Example: i recently bought that super new super fast usb drive from kingston called hyperX savage. It's total crap! It is very fast for copying a movie 1-2gb, but on rekordbox (small files) it sux. Using the tool CrystalDiskMark you can test any drive and see if it's ok for rekordbox. On this picture i have tested the hyperx savage:

As you can see the 4k writes SUX! And that is what matters to get fast export speed on rekordbox. Another example is my ssd intel 535 256gb, in a fantec usb3.1 (type-c) case:

See the 4k Writes?? That matters.

So it's not only the usb 2 or 3 drive/port. You have to do a litle research to get the right drive. 

"BUT If I format them FAT32 then it takes literally days but OS X journaled takes and hour to 8 hours depending on the drive I use"

Can anyone at Pioneer comment on this? Is it better to format USB Flashdrives in OSX Journalled? I am on OSX and play on CDJ2KNXS's

Transfer speeds are entirely based on the ability for your operating system to read/write. Your mileage may vary depending on your system and the drive you use.

@Chovanpeter > I see you deleted your post, but I have the email and can provide some insight...

The Kingston HyperX has been tested and is VERY POOR at transferring small files, which rekordbox does a lot of (database, waveforms, cue points, etc.), and as such, it's not a suitable drive for use with rekordbox.

I have shared this same information with our team on the ticket you have open.