DJM 900NXS Recording Issues

Hi I upgraded from a djm 800 to djm 900 NXS 1 for better recording quality as the NXS uses printer cable for recording and 800 is only AUX.

Since upgrading, my recording audio is worse than the or just about on par with the DJM 800.

The past week I've tried all different configurations corresponding from the mixer to the DJM utility settings driver and Audacity.

Configuration:

Usb output: -5 (djm setting utility on pc)

Dout LV: -5 (utility menu on the mixer)

Dout FS: 48k (utility menu on the mixer)

Peak limiter: OFF (utility menu on the mixer)

Then the FS is 48k in audicity
And the sample rate is 24bit on audicity

---------------------------------------------------------
I've tried different solutions with levels such as: Playing a song by itself, adjust the trim to the same volume as the original track. Referencing by recording and playing the original track file compared to the DJM recording.

After finding the original trim. When playing two songs at once, feels like there fighting for headroom.

After trying that solution, I've also tried lowing all the trims and then it all sounds flat, no bass and also muddy at points.

It seems I won't be able to get a studio quality mix, from this setup?

2 CDJ 850's and 2 CDJ 900 + DJM 900 NXS. I have a feeling its because there out dated players? or is it cause I'm using basic red and white cables from the decks to djm?

I've also brought a new printer cable to check if its that and that wasn't the problem.

Also the DJM, CDJS and PC all have up to date pioneer drivers/firmware.