Official comment
The input on your Mac is a microphone input - we don't recommend connecting the RCA master output from your mixer to a mic input (either on the Mac or to your phone) as it's a line level signal and will easily distort through the mic pre-amplifier and sound terrible.
There are instructions on the forums already for both the DJM-900NXS2 and XDJ-RX2 for streaming with OBS.