Gamelist Excel export script

MrSco

Ok, I made a couple edits https://pastebin.com/uT9TjqGR to fix the errors but now I'm hitting this while writing the excel file. I get the file but it errors...

2018-02-18 17:46:16,871 INFO Total games after parsing gamelist files - 13005
2018-02-18 17:46:16,872 INFO Exporting to file /home/pi/RetroPie/roms/export_list.xlsx
/usr/local/lib/python2.7/dist-packages/XlsxWriter-1.0.2-py2.7.egg/xlsxwriter/worksheet.py:2348: UserWarning: Name looks like a cell name in add_table(): 'c64'
/usr/local/lib/python2.7/dist-packages/XlsxWriter-1.0.2-py2.7.egg/xlsxwriter/worksheet.py:2348: UserWarning: Name looks like a cell name in add_table(): 'n64'
/usr/local/lib/python2.7/dist-packages/XlsxWriter-1.0.2-py2.7.egg/xlsxwriter/worksheet.py:2342: UserWarning: Invalid Excel characters in add_table(): 'sg-1000'
Traceback (most recent call last):
File "export.py", line 468, in <module>
xlsx_export_workbook(systems, output)
File "export.py", line 392, in xlsx_export_workbook
xlsx_export_system_row(wb, sheet, j+1, g, g.info["system"])
File "export.py", line 434, in xlsx_export_system_row
game.info['desc'], {'x_scale': 4, 'y_scale': 4})
File "build/bdist.linux-armv7l/egg/xlsxwriter/worksheet.py", line 65, in cell_wrapper
File "build/bdist.linux-armv7l/egg/xlsxwriter/worksheet.py", line 1150, in write_comment
TypeError: object of type 'NoneType' has no len()

mitu

@mrsco I've found the error, can you re-download the script and try again ?

mitu

@cyperghost Sorry, I didn't see your update (probably edited). I've corrected the link, so the : is removed.
I've thought about adding the custom collections part, I just don't use any and didn't know they're saved. I'll see about this next week and give it a try.

cyperghost

@mitu No problem
custom collections are stored in /opt/retropie/configs/all/emulationstation/collections

and their naming is always custom-AaBbCcDdEe.cfg
AaBbCcDdEe represents the name of the collection ;)

Inside is just plain text with Romname and complete path.

Take a look here

PokeEngineer

@mitu

Huh, neat.

detron

I get the same error as @MrSco , but I will keep a close eye on this thread, since i think this looks amazing! so excited to play with this in the future

@mrsco said in Gamelist Excel export script:

Traceback (most recent call last):
File "export.py", line 460, in <module>
systems = parse_systems()
File "export.py", line 241, in parse_systems
rom = Game(game)
File "export.py", line 146, in init
self.info[attr] = get_xml_element_date(obj, attr)
File "export.py", line 69, in get_xml_element_date
if len(date_text) < len('19860101T000000'):
TypeError: object of type 'NoneType' has no len()

mitu

@cyperghost Shoudn't be so hard to add, I just need to compute the fullpath (which I already do anyway to check if the Rom exists) and then parse the .cfg file.

mitu

@detron Have you tried after my last update (1h ago ?) to download the script and try again ? If you still get the error - run with the -d flag and upload the gamelist.xml for the system with the error on pastebin.com so I can take a look.

detron

@mitu said in Gamelist Excel export script:

@detron Have you tried after my last update (1h ago ?) to download the script and try again ? If you still get the error - run with the -d flag and upload the gamelist.xml for the system with the error on pastebin.com so I can take a look.

I got the error about 2 minutes before my post, which was after your update. but I will try again from scratch.

thank you for the assistance. I would love this to be a retropie standard, allowing you to create a report with ease

detron

@mitu said in Gamelist Excel export script:

@detron Have you tried after my last update (1h ago ?) to download the script and try again ? If you still get the error - run with the -d flag and upload the gamelist.xml for the system with the error on pastebin.com so I can take a look.

I still get the same error, if there is any other info that I could provide besides the info below, I am happy to grab any info that might help.

here is the pastebin

link to pastebin

mitu

@detron @MrSco Beginner's error - gist.github.com changes the URL after an edit, so my new version was at a different URL. Try again now, I have updated the URL in the initial post. Overwrite the initial export.py script by downloading it again

wget -O export.py https://gist.githubusercontent.com/cmitu/73cf0783e9e1e4e02d8ce5c44f7a8984/raw/7f61f714bdf9fd49a690cf75cc719b8ffcdf404f/gamelist.py

detron

a lot more activity now, but a different error happened.

again, thank you for your help.

pastebin version

Traceback (most recent call last):
File "export.py", line 464, in <module>
systems = parse_systems()
File "export.py", line 251, in parse_systems
rom.info['name'], s.info['path']))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 3: ordinal not in range(128)

mitu

@detron Please give me the gamelist.xml file where the error occurs. When running with -d parameter, it will show you which system is trying to parse. Get the system's gamelist.xml file and upload it to pastebin.com.

detron

I got it to work. the problem was with a pre-built gamelist file (for a system I have zero roms for.)

this is really cool, thank you for the program, and the support

the prebuilt gamelist has some foreign letters that may be the issue. the file is too big for the free pastebin, but if you really need it, I am sure I can upload it latter to google drive or something for you

mitu

@detron Please do, I'd like to have a look at it. Normally, the gamelist is generated by ES and should be valid UTF8 (which the script can handle), if it's generated externally it might have a different encoding. If you can upload the file as-is (no pasting), that would allow me to try and find a fix/solution for the error.

detron

@mitu

here you go.

link

let me know if it fails, I will fix it.

mitu

@detron Thank you for the file. The problem is seems was in the logging part, which was tripping on the game's game. It should be fixed now (URL updated in the 1st post), so you can re-download the script and try again with your initial gamelist.

wget -O export.py https://gist.githubusercontent.com/cmitu/73cf0783e9e1e4e02d8ce5c44f7a8984/raw/0ee908ff097fca55c4db5911bd78d794666af59c/gamelist.py

detron

@mitu

thank you for the help others may be saved by your edit, but I have zero roms for that system. I just thought the idea of a gamelist with all images was brilliant. add your few roms, and the art would show up.

meleu

I have a strong conviction that @Used2BeRX will love this tool! :)

Used2BeRX

This looks AWESOME!!!!! @mitu
I can't wait to try it out. I actually asked a few months ago if somebody could make something EXACTLY like this.

If this works the way that I think it will from your OP, I think this is going to end up saving me countless hours on my work in the future, and I should still be able to get a ton of work out of it for the NES project I'm currently working on.

Is there any chance that you could write a reverse script that would then convert the existing spreadsheet back into a gamelist.xml? That would make this absolutely perfect then and I would be able to do all of my editing inside of a single spreadsheet (assuming it wasn't insanely taxing on my ancient PC to do so).

Thanks for the heads up @meleu.

If this is something that I can use like I think it is, I was hoping that when you get time to do some more things with our script that you might be able to make that reverse script we were talking about that would then take the gamelist.xml and convert it back to the initial synopsis.txt files.

That way when all of the work was done I could re-generate the edited synopsis files for the XBox ready release as well as the Pi Zero and Pi 3 release without having to edit all of them by hand.

Amazing. Thanks for sharing mitu!