RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login
    Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

    Compining results from two scrappers.

    Scheduled Pinned Locked Moved Help and Support
    two scraperses scraperskyscraper
    3 Posts 2 Posters 440 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • abjA
      abj
      last edited by

      RetroPie 4.8.6 (Bookworm lite) manually installed.
      Emulastation scrapper
      Skyscraper 3.10

      I have successfully scrape my roms with the Skyscrapper but i have few games with strange names or romhacks that it can't find them.

      I tested with ES scrapper and i finded the missing one, with the use of input fuction and a lot of try and error.

      It's safe, for gamelist.xml to add info from a different scrapper or i will break the file in a long run ? Thanks.

      `Please be patient, my English is not very good.`

      1 Reply Last reply Reply Quote 0
      • LolonoisL
        Lolonois
        last edited by Lolonois

        It might be ok in the long run, but the gamelist you now have may easily lose the information from another source when it is rewritten with Skyscraper. Although Skyscraper does a great deal not to overwrite information that is already present in a gamelist. However, if Skyscraper has updated data in its cache or from its scraping process this newer data is put into the gamelist. Same may happen if you (accidentially) remove the gamelist.xml file: Anything not in the Skyscraper cache can not be reconstructed then.

        That said, I see these options to make your approach more robust: They all have in common that they use Skyscraper as single source of authority for a gamelist file.

        Option 1: IIRC ES-Scraper uses Screenscraper.fr under the hood, thus you may re-scrape with Skyscraper, a la Skyscraper -p <platform> -s screenscraper --query="romnom=<your search term here>" ~/RetroPie/roms/<platform>/<romfilename+extenstion>. If you know a variant of a rom exists in the screenscraper.fr data you may also use --query="md5sum=<md5sum of romfile>" for example. Run md5sum ~/RetroPie/roms/<platform>/<romfilename+extenstion> to get the MD5 checksum. See the Skyscraper documentation for more details. https://gemba.github.io/skyscraper/CLIHELP/#-query-string

        Option 2: If you don't want to do the re-scraping again. Use Skyscraper to import your current up-to-date gamelist.xml file Skyscraper -p <platform> -s esgamelist. Then adjust the priorities.xml file (see section "Resource and scraping module priorities" in https://gemba.github.io/skyscraper/CACHE/#resource-cache) and have <esgamelist> put after <screenscraper>.

        Option 3: If you still want to make edits/updates to your scraped data for the special roms (=those which were tricky to find in the first place). Copy the relevant <game> entries from your current gamelist into the import folder for that platform (~/.skyscraper/import/<platform>/textual/). See here for details: https://gemba.github.io/skyscraper/IMPORT/ and the XML format to use: https://gemba.github.io/skyscraper/IMPORT/#textual-data-definitions-file. Eventually the XML-import format can be the a full <game> section from the gamelist (=no need to remove <lastplayed>, <favorite>, ... elements). For the import file use the same name as the rom file but change the extension to .xml. Also copy the screenshot files and other binary data to the respective import folder for each platform (e.g. ~/.skyscraper/import/<platform>/screenshots). When you are done run a Skyscraper -p <platform> -s import. You can edit the import files later on and rerun the Skyscraper -p <platform> -s import command to have it updated in the gamelist.

        1 Reply Last reply Reply Quote 1
        • abjA
          abj
          last edited by abj

          Thank you very much for the detail answer! i will check all options carefully.

          `Please be patient, my English is not very good.`

          1 Reply Last reply Reply Quote 0
          • First post
            Last post

          Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

          Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.