Hard to say with so many variables
Windows or MAC?
I've had overloading issues because I was watching my OBS meters and not the meter displaying the incoming signal (Win10, Sound, Recording, check the little meter next to your input: 
I have also seen issues with OBS and ASIO drivers that don't play nice together.
When you're playing are you monitoring the XDJ, Computer, Audio Interface or OBS? If it's any of the above except OBS you probably wouldn't notice any issues, but if your monitoring your OBS output (not recommended when DJing because of the slight delay) you may well hear your bad sound.
What are your OBS Output settings set to? I usually set the audio bitrate to 320, video bitrate around 2500, you could try changing the encoder settings just to see what happens? Beneath Streaming in the "Output" settings there's the Recording settings try mp4 which I have found to be a bit more stable than mkv
How are you connecting XDJ to the Audio interface? The XLRs on the interface may well be only for microphones which are more sensitive and could easily be more prone to distortion? Try the unbalanced RCA/phono outs from the XDJ to the interface inputs
Good luck mate
George
