Both Nikon D5300 and Nikon D3500 have 24.0 MP resolution sensors but Nikon D5300's sensor is APS-C (23.5 x 15.6 mm ) and Nikon D3500's sensor is APS-C (23.5 x 15.6 mm ).
Since Nikon D3500's has a larger sensor area with the same resolution, this means that it also has a larger pixel area hence better light collecting capacity for a given aperture compared to Nikon D5300.
Another similarity between these two cameras is that both Nikon D3500 and Nikon D5300 sensors lack anti-alias (Low-Pass) filters. Removing anti-alias filter increases the sharpness and level of detail but at the same time, it increases the chance of moiré occurring in certain scenes.
Below you can see the D5300 and D3500 sensor size comparison.
Nikon D5300 and Nikon D3500 have the same sensor sizes so they will provide same level of control over the depth of field when used with same focal length and aperture.