track and folder organization

Here's my organisational method, hopefully it helps build a method of your own!

Alright, this is how I manage my digital music collection, first of all, I let iTunes manage the folder structure behind my music collection. Here's why:

1. I always look at music through iTunes or RekordBox, I've never looked for music through the file system.
   
2. The OS X finder (when opening/importing anything) has a really nifty media browser (bottom left of the Finder Window) which lets you browse your iTunes library (including playlists) without ever looking directly at the file system.
   
3. If something changes in iTunes and so the filename changes, RekordBox has a really nifty tool that will scan for missing media, and will then let you select it's new location. And guess what, it will let you use the media browser mentioned in point 2.
   

So now lets get down to business with it, what I do is create two playlist folders within iTunes, call one of them Purchases and the other one Promos, I've also started a third folder recently called Mashups & Bootlegs, but that's option. Now in each folder create playlists for each month my suggested format is like so: 2011/08 August. This way they're in chronological order, and you can clearly see the year and month, if you like you can create one for each week (eg. 2011/08 August Week 1, 2011/08 August Week 2), I tend to only do this for big purchase months though, tends to be overkill for most months.

Now as you import music, drag and drop to the relevant playlist under your Purchases/Promos folder and bob's your uncle.

Now all you need to now is import your two folders Promos and Purchases using the Bridge function in RekordBox, and it will keep the very same structure we've just setup.

Few other things:

What I also do (And this is optional) is stick the label into the Grouping Field, 3 reasons:

1. I've always been one to search for music via label rather than the artist

2. It allows me to create smart playlists in iTunes for each label (I have a folder called Labels full of smart playlists, this doesn't get imported into RekordBox though)

But most importantly

3. Under the RekordBox Preferences Bridge Tab, you'll find another nifty feature: Convert iTunes "Grouping" To Rekordbox "Label"

Key mixing:

I still say anyone can paint by numbers, and DJs should be able to use their ears and not rely on their eyes, but 10 years ago we all said the same thing about BPM counters, and now they're on every CDJ and player out there, so moving on.

If you do use Mixed In Key, there's the option in Preferences to 'Update custom "Initial Key" mp3 tag". Enable this option and let it scan your library (Again, to import media you can use the OS X media browser in point 2... Or drag and drop directly from iTunes), once done in RekordBox if you click on the Info pane (you can select as many tracks as you want, but I recommend one or two to start with, just to test) and hit the reload button (Looks like a browser's refresh button, two arrows in a circle), which will reload all the info from the ID3 tags.

And there you have it.

I don't see the need to create playlists based on key or any other field than the Purchases/Promos, because in RekordBox Preferences, under the Export tab, you can select exactly which info you want to display and sort by on the CDJ, and all this gets exported to your USB drives as well.

I did a lot of work on that guide… and then got sidetracked by life, of course.  Ironically, just today, I had a long chat with someone about this very issue (and just saw this post tonight).  Time to revisit this project, I guess.

To be honest—and I think I mentioned this once before—one of the reasons I've been sitting on it instead of releasing it is because I was hoping Rekordbox would improve to the point where I could remove all the parts that are essentially workarounds for its deficiencies.  I'd much rather be supportive of brand Pioneer if I can.  Even without taking an editorial position on the application, there's simply no way to avoid negative inferences because of the umpteen steps that are required only because of quirks, bugs and lack of professional development.

In the interim, I'll offer a much less thorough outline of the process here, in case it helps anyone.

1. Buy some tracks
2. Import a folder of tracks into iTunes by dragging the folder into the sidebar (automatically creates a playlist)
3. Strip the "Purchased from…" or other comments from those tracks
4. Drag tracks directly from iTunes into the MIK window, and analyze them (MIK setting to write key in "Comments" field)
5. Once analyzed, with those tracks still selected in iTunes, right click and select "Convert ID3 Tags" (Select latest version, then "OK")
6. Review tracks:  confirm/edit "Genre"; add a number 1–5 in "Composer" (based on where a track is most likely to fit within a night/set); add a subjective label in the "Grouping" field (e.g. "Deep," "Driving," "Summery," etc.)

That pretty much takes care of the pre-Rekordbox part of the process.

Note:  Steps 3, 4 and 5 could be condensed into a single step if Mixed in Key's "overwrite" function actually worked.  (It used to work in MIK 3).  Since it doesn't, you have to first strip comments, then analyze, then refresh the tag info via the "convert ID3…"

Then the fun begins.  :-/   In Rekordbox...

1. Import tracks via Bridge.  The best way is to drag a folder (including multiple playlists, potentially) from the iTunes side to the RB playlists section
2. Once tracks have been imported and have finished analyzing, reload the tags.  This captures certain fields which don't load at first for some reason.  One of the fields that often gains data at this stage is "Label," which is a field that is unavailable in iTunes.  That can be very handy, depending on how significant record labels are to you.
3. Sort your tracks by "Comment" which will group them by MIK-style keys.  Select each group (e.g. all your 4A tracks), and using the Info panel, type that value into the "Key" field (in this case, 4A).  Now you have the key in "Key" and in "Comments."
4. Sort your tracks by "Composer" which are the 1–5 values you added back in iTunes.  Repeat step 3, but this time, instead of adding the MIK-key to the "Key" field, you're adding a color using the "Color" field.  I use red for 5's, orange for 4's, and so on…
5. I like to copy paste the (record) label information from the "Label" field to the "Album Artist" field.  That way, when I refresh/reload my tags back in iTunes, I have access to record label info (which was present before, but is not accessible in iTunes).  ***Please see note of caution below***

That's basically it.  Like I alluded to—once you're done in RB, you can go back into iTunes and "convert ID3…" again to refresh/reload your tag info, which will update it according to any changes you made in RB.

CAUTION:  if you populate your "Album Artist" field with record label information, AND if you have iTunes set to automatically organize your library (as you should, in my opinion)… then step #5 causes a bit of an inconvenience (HA!).  Actually—it's more than an inconvenience…  Because iTunes prioritizes "Album Artist" above "Artist," when it manages the underlying files and folders of your iTunes library, refreshing the tags will cause iTunes to reorganize (MOVE) any files to which you added "Album Artist" data.  iTunes doesn't skip a beat… but RB, of course, is completely lost.  The only effective way to remedy this situation (other than avoiding it, which is certainly an option) is to go into RB and remove all the tracks that you just imported… and then import them again.  Elegant or convenient, it is not.  Effective, it is, however.  You'll have to wait for the tracks to reanalyze of course… but this time, the data on them is already in place, so there shouldn't be any reason for files to move around on you...

Unless you change, oh, say the Artist, or the Name field… who does that anyway? right?… :-/  yeah, right.  In any case, that's it in a nutshell.  The full version goes into a lot more detail about the concepts involved in organizing and sorting a big and diverse collection, as well as a lot of tips on how to use iTunes to manage it all (very effectively and efficiently).  Hopefully it will get posted at some point… fingers crossed.

Hope that helps...

-Warth

BTW… forgot to mention the important role that Smart Playlists and the Browser function in iTunes play in this system.

Smart Playlists:

— I've made one for each MIK variable (e.g. 7A).  So… that's 24 Smart Playlists (which I keep together in a single folder, which usually remains closed/collapsed in the iTunes sidebar).

—I've also made 24 "Compatible-With…" Smart Playlists (e.g. "Compatible with 7A," which would include 6A, 7A, 7B, 8A, and a few others, depending on how you like to implement that system).  That group of Smart Playlists is in another folder, which usually remains closed/collapsed as well.

—I've got a few other Smart Playlists that I use in a compound manner (i.e. one Smart Playlist might reference one or more other Smart Playlists).  This means that by changing the criteria for a single Smart Playlist, I can change the contents of several others.  It's really handy when you're trying to program a certain style and you want to be able to browse through your playlists without seeing styles that you're not using at that point.  e.g. If I'm making a Tech-House mix, I can change one variable on one smart playlist, and all my "Key" playlists will reflect only Tech-House tracks… or only Deep House… or only 2's and 3's… or whatever….)

The iTunes Column Browser

Most people are aware of iTunes' 3-column browser… but most don't realize that you can add more columns to it (up to 5).  So… my columns are (from left to right):

• Genre
• Composers (1, 2, 3, 4, 5)
• Groupings (Deep, Driving, Dark, Light, etc.)
• Artists
• Albums

With all those columns above (which are reductive in the way they limit what is seen below in the main library list) is awesome.  With all this in place, you literally have dozens of ways to think about, search for and find tracks as you program a mix, a set or whatever…  It's great…

The only downside (other than the frustrations with RB) is that it takes a LOT of commitment and dedication to organize your library like this.  Once it's done (as it is for me), it's cake… but getting there can be a long and lonely experience… haha!  Once you're on the other side of it though, you'll feel like a superhero… everything at your fingertips, no matter how inebriated you are, or how little time you had to prepare, you're in a relatively solid position.  All that's required from that point is keeping at it whenever you add tracks to your library.

-J

Jason - Did you read my post above? Have a read, I think it'll save you a bit of confusion and work from your method

Couple of things I do different to you:

• First of all, I only ever update the Key ID3 tag using Mixed In Key, I never update comments or song name or anything like that, it doens't seem to work for M4A files, but mp3 files it workds fine (you do need to hit Reload track first for the key field to show in RekordBox however), I can always see that field in RekordBox and have never had to look at it in iTunes, don't forget on the CDJ900/2000 you can actually browse by the actual Key field and don't have to have a specific playlist setup for each one

• Instead of putting in the label into the Album Artist field, put it in the Grouping Field in iTunes, and then enable the option in RekordBox to import from iTunes

• When reloading tracks, the first time you reload, the imported label field from iTunes will be overwritten with the ID3 tag of the track, subsequent reloads are fine.  What I do is reload them in groups so select a group of tracks from the same label, Copy the label field before reloading, paste it back straight away, problem solved.

Thanks to you both! I will comment later in more detail on your strategies. My strategy  differs with both of you because I try to avoid itunes touching my songs (I import my songs directly from t&r into rekordbox) and I try to keep my folder organization as clean as possible instead of letting itunes do the work (it's too much of a black box for me).

Howerver, my problems are inside rekordbox: i find it hard to find what to play for a gig. Do you guys prepare a gig by making a playlist beforehand so you know what to play before the gig ? Or WHATdo you guys put into your playlists (i ask this because i use e.g. "2010-02" but I find it hard to rememer what I bought when, so I clearly need an other organizational system e.g "hot tracks", "warming up music",...???).

Thanks Angus… Just read it… skipped over it before, actually.  Your method(s) are a bit different than mine.  I have MIK put the key info in the "Comments" field because I want to be able to view key info while in iTunes (i.e. before my music ever sees RB).  I do have MIK set to write that key info into the "Initial Key" field as well… so maybe with that, you can avoid part of my process above.

I use the Grouping field for subjective criteria (Deep, Driving, Grooved, etc.), so it doesn't work for me to allow Rekordbox to use that for the record label field, unfortunately.  On the old forums, I suggested that the user should be able to choose the field to use for that feature… but right now, it appears as though it's "Grouping" only.  Also, "Convert iTunes 'Grouping' to Rekordbox 'Label'" isn't very clear to me… if anything, shouldn't it be the other way around?  I don't want iTunes 'Grouping' info to populate the 'Label' field… I'd want 'Label' to populate 'Grouping"… that is, if I weren't already using the Grouping field.  It seems that for you, it makes sense because you've already put the Record label info into the 'Grouping' field (while in iTunes.)  For me, I'm actually using Rekordbox to reveal record label information, and capturing it by copying it to another field.

Your idea about loading groups of tracks from the same labels is a good one.  Unfortunately, when I buy 15-20 tracks, there are only so many groupings to be made (i.e. 15 tracks probably equates to 8–10 labels).  If/when I'm importing larger groups of tracks, I reload the tags (at which point the label field appears), and then sort by that field.  Then I can select groups of tracks and put the label in the Album Artist field.

I've made this point a million times, but I'm going to keep making it until I die, or the Rekordbox developers make it irrelevant:  If we were working with a single set of data here, instead of 2 distinct sets with overlapping information, none of this would matter.  Rekordbox insists on trying to reinvent the wheel, however—duplicating functionality that everyone (Mac and PC) already has.  In the past, this was frustrating simply for the way it interfered with iTunes.  Now it's not just that, but the fact that the core functions of the RB application are completely unusable to some of us.  (I can't even plug in my hard drive without it crashing!).  And you know what—I am going to say it:  I TOLD YOU SO!  (Not you, Angus, but Pioneeer / MixVibes, of course).  If you guys would give up trying to be something that you're obviously never going to be (and which we don't need anyway), and instead focus on the unique functions that Pioneer hardware users need (and which Rekordbox used to be pretty good at), then everything would be hunky dory.  Instead, we have disaster… and I'm not kidding when I say that I literally spoke to someone at length TODAY who was trying to make the decision whether to switch to Traktor or Serato, or buy another Pioneer CDJ.  Guess what?  Rekordbox is his main sticking point (of course).  This application matters Pioneer.  Wake up!  You're losing sales and long-term customers because you won't drop a few thousand bucks on developing a decent piece of software.

/commence tumbleweeds and crickets…

@ s_decoster:  You're welcome… hope it helps!  To answer your question (which I should have addressed in my original post)—the entire point of my method of operation is so I can find / program tracks on the fly in as many different ways as possible.  Using the categories and sorting on the CDJ-2000s makes the whole thing worthwhile.  When your tracks have all those different data points attached (key, genre, subjective value, 1–5, etc.) you suddenly have a dozen ways to arrive at "this is the next track I want to play."  I do keep a separate folder of playlists called "Live" in which I will keep a nother folder for each gig I play.  In that folder, I keep a couple playlists—one containing "maybe I'll play these," and another containing "this is what I wound up playing," which obviously gets populated after I play.

I'll try to post some screenshots of my library with all the columns and the sidebar full of playlists so you can see what it looks like...

Here's a quick screen-cap, which hopefully gives you an idea of the iTunes part of my setup.

@s_decoster: Mate, the best advice I can give you is to find your own style of organisation and work on it from there, typically I tend to play my most latest stuff out, so I tend to work back through my chronological playlists (starting from the latest ones) and dropping in the odd older tune here or there,  If I do have a specific warm up set (I'm supporting Above and Beyond in a few weeks, and they're very specific about what I can and can't play before them) then I'll go ahead and create a playlist of specific tunes for that gig and usually play from that playlist.

Part of the reason I use iTunes is so I can listen to my music on my iPhone throughout my day.  Try not to over think it too much and in my opinion, No amount of organisation though can make up for actually knowing the tunes in your current rotation, and then using your playlists strictly as a guide (remember, you're meant to be creative and thinking outside the box here).

@Warth Whoa man, yours is obviously a lot more detailed and thoroughly thought out than mine (which is obviously why mine's a bit easier to manage)

I'm kind of lucky, I'm very OCD and have ended up using literally every field (with the exception of Grouping in iTunes) for it's intended purpose, what you use for Grouping though I tend to use replicate that with Colour in RekordBox (tend not to worry about it in iTunes, but I probably would if it existed in iTunes/I could edit tracks on the go on my iPhone)

I keep a gig history too, but I don't recreate it in iTunes (too much work, I just keep a playlist folder in RekordBox and leave it there)

I completely agree with you about RekordBox as well, one of my most common complaints about it is it feels like a 0.7 or 0.8 rather than a 1.5 release.  It's brilliant software, but it feels like it's absolutely full of buggy and legacy code (I mean come on, you can't even Tab to the next field in the Info Pane!)

Here's a quick screen cap of my Playlist sidebar,

Labels is full of Smart Playlists based on the Grouping Field, Missing Data is full of smart playlists on data that isn't filled in, (handy if you're OCD like me), My Mixes is playlists of my promo/demo mixes.

Basically I import Bootlegs & Mashups, Promos & Purchases into RekordBox using Bridge, occasionally I'll setup a specific playlist for a specific gig but I tend to just work mainly off the latest playlists in those three (mainly Purchases and Promos)

Recently upgraded to 900s and have spent more time with rekordbox and attempting to get all my music organized than I have actually using the CDJs.

A few oddities that I need help with.

Rekordbox seems to pick and choose when it will copy files over into Artist/Album/Track directory format.  This has worked inconsistently and I thought I had it fixed until I bought new music yesterday and imported the new tracks into my collection.

• In preferences verify that the directory is set to /Volumes/geeves hd
• Copy latest BP-created folder to my external HD
• File -> Import -> Folder
• Highlight added tracks Right-click -> Analyze
• Copy into All Tracks into a new Playlist
• Different tracks are actually added to the playlist see: http://www.youtube.com/watch?v=QD_upB8Msn0

After all of this, it turns out the new tracks were never analyzed (no BPM and no waveform) and when the HD is connected to the CDJ they do not show up in the playlist and I have to navigate to  the Folder itself.

Am I missing that "?" step between step 1 and profit?

Also another question or two that might belong somewhere else:

• When a track is played in full on the CDJ and the waveform is complete - why is this not copied to the HD then readable by Rekordbox?
• Why does the track have to play completely for the CDJ to read the waveform unlike its CD counterpart when it scans ahead?

@s_decoster: I have all my tracks in folders just like you. Year and month 2011-01-2011-02  and so on.

Every time I drag a new folder into Rekordbox I chosse "date added" so the latest tracks appears on the top of the collection. The I mark the latest import, click on -Info- tab so i can see both window, and write the folder name (2011-01 etc) in -Album-. Then all the tracks are collected with the same album/folder name and you can scroll through the album/folders in the album area over your collection or just sort your collection by album.

Hope this made sense:)

@geeves > Please start your own thread when you have your own issues.

You are putting your music onto your external hard drive but then you are importing the music over to your laptop. You are, therefore, creating a duplicate of files, one set is on your external HD and the other is on your internal HD (i.e. on the laptop). What you need to do is subsequently export the music from Rekordbox back to your external hard drive (highlight the files and right click for the export option / or creating a playlist and then export that).

Again, it makes total sense that the files do not have the BPM or waveform information as you are not analysing the files on your harddrive - you are transferring them onto your laptop.

I'm not 100% clear on what you mean here: "When a track is played in full on the CDJ and the waveform is complete - why is this not copied to the HD then readable by Rekordbox?"

However, to address your second issue, this depends on the files you are using. If you are downloading MP3s, this is a compressed media format and the player has to uncompress the file. As a result, the waveform takes time to build. If you analyse the files in Rekordbox prior to this, the waveform data will build instantly. If you do not wish to use Rekordbox, you can use WAV or another uncompressed media format. It is likely, when you burn a CD, you are burning as a CD-Audio rather than a MP3 CD. This means that the files are uncompressed and then burned to the CD and the player does not have to do any 'uncompressing'.

I hope this addresses your issues and you achieve Step 4. Profit ;)

When I have created local playlists on my laptop HD - it still does not work.  Also it does not work via The Bridge.

I get the same error in the end of incorrect files being listed in the playlist

Take for instance this import from iTunes playlist to my external HD (I have tried this from both my laptop HD and external HD with the same exact result)

http://www.youtube.com/watch?v=-EZVGslVR4A

One thing I do wish RekordBox had was a way to batch edit File paths :(

I'm in the process of relocation the music (Music only, no Movies/TV Shows) to the Internal HDD on my Mac using iTunes's Consolidate Files feature.

I'm doing it one playlist at a time, but all I really want to do with my RekordBox Database is find entries where the Location starts with /Volumes/iTunes/Music/* and replace with /Users/angusgibbins/iTunes/iTunes Media/Music/* all in one hit Using the Find Missing Files feature in RekordBox at the moment but it's very time consuming :(

hey warth!

question with your step 3 "

• Sort your tracks by "Comment" which will group them by MIK-style keys.  Select each group (e.g. all your 4A tracks), and using the Info panel, type that value into the "Key" field (in this case, 4A).  Now you have the key in "Key" and in "Comments."

Why don't you let MIK directly write it into the key field instead of the comments field (I don't use my comments field for keys because i use it to type in what songs "fit" with the current song)

thanks!!

^^ The reason being is iTunes doesn't support the 'Key' field, putting it into comments allows you to create smart playlists in iTunes.

I feel your pain guys. I have the same problems using folders to organize my music. I don't want to use playlists (more detailed comments in: https://forums.pioneerdj.com/hc/en-us/community/posts/115018095303/comments/360000392646)

A large part of this complex setup is that RB doesn't have a search box on the explorer nodes.