Canon 77D has a 24.0MP APS-C (22.3 x 14.9 mm ) sized CMOS sensor and features DIGIC 7 processor. On the other hand, Canon 5DS R has a 51.0MP Full frame (36 x 24 mm ) sized CMOS sensor and features Dual DIGIC 6 processor. Canon 5DS R's sensor provides 27MP more than Canon 77D's sensor, which gives a significant advantage in real life. You can print your images larger or crop more freely.
On the other hand, please keep in mind that Max sensor resolution is not the only determinant of resolving power. Factors such as the optical elements, low pass filter, pixel size and sensor technology also affects the final resolution of the captured image.
Another difference between these two cameras is that Canon 5DS R's sensor lacks anti-alias (Low-Pass) filter. Removing anti-alias filter increases the sharpness and level of detail but at the same time, it increases the chance of moire occurring in certain scenes.
Below you can see the 77D and 5DS R sensor size comparison.
As seen above, Canon 5DS R has a 2.6x Larger sensor area than Canon 77D. Larger sensors give photographer more control on the depth of field and blurry background compared to smaller sensor when shot in same focal length and aperture.