Both Fujifilm XF10 and Nikon D5300 have APS-C sized 24.0 MP resolution sensors so sensor size and resolution is not a differentiator between these two cameras.
One other difference between these two cameras that is worth mentioning is that Nikon D5300's sensor doesn't have an anti-alias (Low-Pass) filter. Removing anti-alias filter increases the sharpness and level of detail but at the same time, it increases the chance of moiré occurring in certain scenes.
Below you can see the XF10 and D5300 sensor size comparison.
Sensor Size and Resolution Comparison image of Fujifilm XF10 and Nikon D5300 Cameras
Fujifilm XF10 and Nikon D5300 have almost the same sensor size, so neither of them has any significant advantage over the other in terms of providing control over depth of field when used with the same focal length and aperture.
Sensor size sets the circle of confusion, so it drives how much depth of field each
of these bodies gives you at a given aperture. You can calculate it exactly for the
Fujifilm XF10
or the
Nikon D5300 ,
including hyperfocal distance.