Emulationstation accepts the --screensize <width> <height> parameters so it might work to start it in a viewport without mocking with the overscan bands on the display. This, in combination with the RetroArch viewport settings, might enable you to run everything in a custom viewport. Of course, you might have trouble with console script (i.e RetroPie Setup or any other scripts) and with non-Libretro emulators. EDIT: you might need to also add --screenoffset <offsetX> <offsetY> in addition to --screensize to position the viewport.

Re. speed gains : What about running games at low resolutions, but then using shaders to make it look acceptable? Or do the shaders always run at the emulated resolution?

I installed Xfce. Scaled fonts, icons and cursor. Looks a bit odd, but it works. Emulationstation, kodi and the desktop are all accesible from the couch :)