The yellow marker shows you the current Cue Point, this is not saved with the track. So depanding of the setting of the CDJ (is auto-cue on or off), the CDJ will cue up at that point.
to deactive this behavior, just disable Auto Cue. this does mean the track will start to play after loading, but it avoid the Cue point being made.
If all your tracks have a red marked cue point, you have activated a feature inthe setting, this way Rekordbox will use the Auto Cue function to determine the location of a Cue point that it sets and remembers. you can deactive this is the preferences
Preferences > Analisys tab > uncheck the box labeled 'Set Memory Cue at the first beat...'