Boxart for roms in their own folders a WIP

[Bilgus]

I for one like to have my roms organized in their own separate folders
Emulationstation however does not like this,
the following few posts are an attempt to rectify this.

First we have a script that takes a gamelist.xml and adds folder entries with image and description so you can have Boxart in the main romlist

Next, I have made a couple patches to the ES source that allows games within a folder to display boxart without waiting for a cursor change to update the screen and the other clears the unused tags like playcount, date, publisher from the display of folder boxart

#Emulation Station RetroPie BoxArt Metadata Folder Generation Script
#ES doesn't show box art when roms are inside their own folder
#this script parses gamelist.xml and adds a folder entry with path, name,
#description, image, etc of the game node, Warning while the old game list is saved existing folder entries will be deleted and re-generated

#REQUIRES XMLStarlet Toolkit: Command line utilities for XML
#sudo apt-get update
#sudo apt-get install xmlstarlet

#USAGE
#copy to .emulationstation/gamelist/<system> directory
#at directory in command line
#type chmod +x es_fix_folders.sh
#type ./es_fix_folders.sh

#sample record
#<?xml version="1.0"?>
#<gameList>
#	<game>
#		<path></path>
#		<name></name>
#		<desc></desc>
#		<image></image>
#		<rating></rating>
#		<releasedate></releasedate>
#		<developer></developer>
#		<publisher></publisher>
#		<genre></genre>
#		<players></players>
#		<playcount></playcount>
#		<lastplayed></lastplayed>
#       <thumbnail></thumbnail>
#       <marquee></marquee>
#       <video></video>
#	</game>
#
#We add this
#
#	<folder>
#		<path></path>
#		<name></name>
#		<desc></desc>
#		<image></image>
#		<rating></rating>
#		<releasedate></releasedate>
#		<developer></developer>
#		<publisher></publisher>
#		<genre></genre>
#		<players></players>
#       <thumbnail></thumbnail>
#       <marquee></marquee>
#       <video></video>
#	</folder>
#</gamelist>

FIELDSEP=$'\034'
ROOTNODE="gameList"
GAMENODE="game"
FOLDERNODE="folder"
TMP_NODE="FTmP"
FILENAME="gamelist"
GPATH=0
GDESC=1
GL_ELEMS=([$GPATH]="path" [$GDESC]="desc" "name" "image" "releasedate" "publisher" "developer" "rating" "players" "thumbnail" "genre" "marquee" "video")

#backup old gamelist
if [ -e ./$FILENAME.old ]
then
    echo backup of $FILENAME.xml already exists
    while true; do
        read -r -p "Parse backup instead (Y/N/Q)? " result

        case "$result" in
            Y*|y*) echo Parsing $FILENAME.old
                   break ;;
            N*|n*) echo Parsing $FILENAME.xml
                   cp ./$FILENAME.old ./$FILENAME.bak;
                   echo $FILENAME.old copied to $FILENAME.bak
                   cp ./$FILENAME.xml ./$FILENAME.old;
                   echo $FILENAME.xml copied to $FILENAME.old
                   break ;;
            Q*|q*) exit 1; break ;;
        esac
    done
else
    cp ./$FILENAME.xml ./$FILENAME.old
    echo $FILENAME.xml copied to $FILENAME.old
fi

#copy game records
xmlstarlet sel -t -c "$ROOTNODE" ./$FILENAME.old > ./$FILENAME.tmp

#delete all folder nodes so we can rebuild them
xmlstarlet ed -L -P -d /$ROOTNODE/$FOLDERNODE ./$FILENAME.tmp

start_time=$(date +%s)
path_list="$FIELDSEP"

exec {FD}<>"./$FILENAME.tmp" #open for R/W with file descriptor

#get count of game records
count_var=$(xmlstarlet sel -t -v "count(/$ROOTNODE/$GAMENODE)" /dev/fd/${FD})
echo $count_var records copied to $FILENAME.tmp

set -f #Disable filename expansion (globbing).
for ((i=1; i <  $count_var+1; i+=1)); do #index of current GAMENODE -> $((i))

    #record values of GAMENODE
    rec_in=$(xmlstarlet sel -t -m "//$ROOTNODE/$GAMENODE[$((i))]" \
                            -v "${GL_ELEMS[$GPATH]}" -o "$FIELDSEP" \
                            -v "${GL_ELEMS[$GDESC]}" -o "$FIELDSEP" \
                            -v "${GL_ELEMS[2]}" -o "$FIELDSEP" \
                            -v "${GL_ELEMS[3]}" -o "$FIELDSEP" \
                            -v "${GL_ELEMS[4]}" -o "$FIELDSEP" \
                            -v "${GL_ELEMS[5]}" -o "$FIELDSEP" \
                            -v "${GL_ELEMS[6]}" -o "$FIELDSEP" \
                            -v "${GL_ELEMS[7]}" -o "$FIELDSEP" \
                            -v "${GL_ELEMS[8]}" -o "$FIELDSEP" \
                            -v "${GL_ELEMS[9]}" -o "$FIELDSEP" \
                            -v "${GL_ELEMS[10]}" -o "$FIELDSEP" \
                            -v "${GL_ELEMS[11]}" -o "$FIELDSEP" \
                            -v "${GL_ELEMS[12]}" -o "$FIELDSEP" \
			                /dev/fd/${FD})

    #split values into array; first substitute all \n with \a
    # then substitute FIELDSEP with \n
    mapfile -n ${#GL_ELEMS[@]} -t <<< "$(tr $FIELDSEP $'\n' <<< "${rec_in//$'\n'/$'\a'}")"
    #remove all but dirpath of the current path(discard basename)
    current_path="${MAPFILE[$GPATH]%/*}"

    #create a new node <folder> and fill with data gathered from GAMENODE
    if (( ${#current_path} > 1 )) && [[ $path_list != *"$current_path$FIELDSEP"* ]]; then
        #only create one folder entry per unique path && ignore paths that are not in a subfolder
        path_list="$path_list$current_path$FIELDSEP"

        #create new FOLDERNODE with subnodes [path], [name], [desc], [image]
        (xmlstarlet ed -L -P -a /$ROOTNODE/$GAMENODE[$((i))] -t elem -n "$TMP_NODE" -v "" \
            -s //"$TMP_NODE" -t elem -n "${GL_ELEMS[$GPATH]}" -v  "$current_path" \
            -s //"$TMP_NODE" -t elem -n "${GL_ELEMS[2]}" -v  "${MAPFILE[2]}" \
            -s //"$TMP_NODE" -t elem -n "${GL_ELEMS[$DESC]}" -v  "${MAPFILE[$GDESC]//$'\a'/$'\n'}" \
            -s //"$TMP_NODE" -t elem -n "${GL_ELEMS[3]}" -v "${MAPFILE[3]}" \
            -s //"$TMP_NODE" -t elem -n "${GL_ELEMS[4]}" -v "${MAPFILE[4]}" \
            -s //"$TMP_NODE" -t elem -n "${GL_ELEMS[5]}" -v "${MAPFILE[5]}" \
            -s //"$TMP_NODE" -t elem -n "${GL_ELEMS[6]}" -v "${MAPFILE[6]}" \
            -s //"$TMP_NODE" -t elem -n "${GL_ELEMS[7]}" -v "${MAPFILE[7]}" \
            -s //"$TMP_NODE" -t elem -n "${GL_ELEMS[8]}" -v "${MAPFILE[8]}" \
            -s //"$TMP_NODE" -t elem -n "${GL_ELEMS[9]}" -v "${MAPFILE[9]}" \
            -s //"$TMP_NODE" -t elem -n "${GL_ELEMS[10]}" -v "${MAPFILE[10]}" \
            -s //"$TMP_NODE" -t elem -n "${GL_ELEMS[11]}" -v "${MAPFILE[11]}" \
            -s //"$TMP_NODE" -t elem -n "${GL_ELEMS[12]}" -v "${MAPFILE[12]}" \
            -r //"$TMP_NODE" -v $FOLDERNODE \
            /dev/fd/${FD})
    fi

    echo -ne "Parsing node $i of $count_var "'\r'
done

exec {FD}>&- #close file descriptor
set +f

rm ./$FILENAME.xml

if [ -e ./$FILENAME.xml ]; then
    echo unable to delete $FILENAME.xml
    echo unformated xml saved to $FILENAME.tmp
    xmlstarlet val -e -w ./$FILENAME.tmp
else
    #fix formatting of the new nodes
    xmlstarlet fo -t ./$FILENAME.tmp > ./$FILENAME.xml
    if [ -e ./$FILENAME.xml ]; then
        rm ./$FILENAME.tmp
    fi
fi


echo ""

echo "Game Entries:" $(xmlstarlet sel -t -v "count(/$ROOTNODE/$GAMENODE)" ./$FILENAME.xml)

echo "Folder Entries:" $(xmlstarlet sel -t -v "count(/$ROOTNODE/$FOLDERNODE)" ./$FILENAME.xml)

echo "Completed in $(( $(date +%s) - $start_time )) seconds"
#validate new gamelist
xmlstarlet val -e -w ./$FILENAME.xml
sleep 10
exit 1```

[pjft]

Hi!

Apologies: what exactly isn't supported in the current EmulationStation, in regards to your use case?

[Bilgus]

@pjft when roms are in their own folders you enter the folder and the boxart and descriptions are blank until you scroll within the list, that could be either up/down or left/right, I'm still working on the patches for that but the script above pulls the image and description from the rom and puts a folder entry in so you get boxart and game description in the main list

[Bilgus]

FIXED IN 2.63ok and now for the patch
this patch makes the gamelist update anytime it is rendered and also hides the unneeded labels for folders

from the root of the sourcefiles apply with patch -p1 <espatch.diff

/*espatch.diff*/

diff --git a/es-app/src/components/TextListComponent.h b/es-app/src/components/TextListComponent.h
--- a/es-app/src/components/TextListComponent.h
+++ b/es-app/src/components/TextListComponent.h
@@ -143,6 +143,10 @@
 	if(size() == 0)
 		return;
 
+        /* force update info before we render if cursor is on first element */
+ 	if(mCursor == 0)
+                stopScrolling();
+
 	const float entrySize = std::max(font->getHeight(1.0), (float)font->getSize()) * mLineSpacing;
 
 	int startEntry = 0;

[pjft]

@bilgus thanks for clarifying. Thanks for sharing!

A question: do we really need to have a separate gamelist element to fix this, though?

It would be great if that would not be the case, for several reasons.

One is that people who customize their gamelists right now to avail of this may find that their gamelists will break when going to a regular ES build in the future. I don't know, but that element will either be ignored and overwritten in the best case, or it'll fail to load in the worst case.

The second is that, if it'd be able to be fixed without the additional element, you could easily submit a PR to ES for it to be available to all.

Consider it - it would certainly be more than welcome!

[Bilgus]

@pjft , The folder element is a feature already in the standard ES I've been using the above script for a few months now with no issues, or are you referring to something I'm unaware of?

I do admit it would be nice if we could have the description refer back to the <game><desc> entry as it would save a lot of space in the gamelist.xml

I'm open to suggestions but barring a breaking change I don't see another way to do it..

I haven't figured out a way to make the folders act like <game> entries to keep from having the <folder> element; However, I've tried naming folders like the rom which just results in a folder that launches the emulator when selected but just returns to the list
I have also tried making simlinks back to the games which just results in a rom that acts just like it was in the root of the <system> folder (dumping cfg and save files in the root of the <system> folder)

Also Do note that the patch fixes a short coming in the ES source code the script and patch are independent of each other..

The script allows you to have box art and descriptions show for every game folder

Whereas the patch fixes an issue of roms within a <system>/subfolder not showing the boxart and description without first triggering a scroll change (up/dn, Left/Right)

[pjft]

@bilgus oh. I wasn't aware of that folder element!

Would you see if anything on this thread addresses part of your expected behavior, though?

https://retropie.org.uk/forum/topic/8942/video-preview-for-roms-in-folders

Either way, a fix for the metadata for the first element not showing would be very much appreciated - will you submit a PR?

Thanks!

[Bilgus]

@pjft I see in this fork that you all have added a lot more folder elements along with video!! I'm still not sure how we could get away with not having a folder entry to get 'my' expected behaviour though but I will surely be trying out more elements to see if they show up.

As for the patch on the first element not showing that is still a WIP but I'd be glad to submit once I have a chance to really test it, I have a sneaking suspicion that I'm doing a lot of extra processing updating on every render rather than just on first load although part of that was because of those tags missing from <folder> elements

[Bilgus]

I updated the es_patch.diff file above and also the folder generation script to take in to account the expanded folder tags that are available, I also made the script easier to expand with more tags for future expansion