Both Canon M6 and Nikon D5600 have APS-C sized 24.0 MP resolution sensors so sensor size and resolution is not a differentiator between these two cameras.
Another difference between these two cameras is that Nikon D5600's sensor lacks anti-alias (Low-Pass) filter. Removing anti-alias filter increases the sharpness and level of detail but at the same time, it increases the chance of moire occurring in certain scenes.
Below you can see the M6 and D5600 sensor size comparison.
As seen above, Nikon D5600 has a 1.1x Larger sensor area than Canon M6. Larger sensors give photographer more control on the depth of field and blurry background compared to smaller sensor when shot in same focal length and aperture.