@exarkuniv Ok now I feel extremely stupid for not thinking about this before. But because I'm probably not the only one, I figured I'd post this info in case others run into the same thing. After wasting more than half a day trying different HDMI configurations and editing the config file a million times, I realized that by plugging in HDMI 1 into the lcd controller and using HDMI 0 as the output source for a bigger TV, the pi will always check HDMI 0 first for signal and if it doesn't find one, then it goes to HDMI 1. So in other words, no need for an HDMI switch or splitter. Man I feel so stupid but at least the problem is solved now.

sure! didn't want to imply that, it just was a major roadblock for me yesterday and today and a lot of searching around just showed that a bunch of people had the same problem. the solutions with manually editing the config file sure help but it would have been just so much easier if I could have set it in the menue. not because editing the file is difficult, more like you don't think of it because everything else just works like it should (well, beside the playstation classic controller not working despite being a simple USB device) and the config tool just blanking out is confusing. honestly if the whole system wouldn't crawl around at around 10 - 15fps at 4k you wouldn't even notice the problem is there to begin with. I'll search if the problem is known at Raspbian.

@mistersouji No, desoldering them will not have any power benefits. These connector ports are "passive" components and they do not draw power from the power supply until you physically install/connect a device into them.