Just wanted to leave this last message here for anyone that finds themselves in a similar position.
I ended up using the below github to get this situation sorted out. I used the "remove orphans" feature after using notepad ++ to change all of the paths of the directory that I wanted to get rid of to "remove.zip"
The gameslist editor linked below then saw these xml entries as "orphans" and allowed for easy removal of the entries without messing with anything else in the gameslist.xml file.
https://github.com/NeeeeB/GameList_Editor/releases
Thank you @ getupor for leading my mind down this path.