RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login
    Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

    Skyscraper two ways: games beginning with "The", and config.ini settings not working

    Scheduled Pinned Locked Moved Help and Support
    skyscraper
    12 Posts 2 Posters 548 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • muldjordM
      muldjord @sleve_mcdichael
      last edited by muldjord

      @sleve_mcdichael

      Is this a setting that can be disabled somewhere, or just something I'll have to cope with?

      It can't currently be disabled, but I'll add a --flags theinfront to the next release.

      It (1) still asks if I want to skip already existing game list entries

      It can currently not be used under a scraping module specific section. But there's no reason it shouldn't be able to, so I'll add that to the next release. Thanks for noticing.

      and cacheRefresh="true" don't seem to be working, in the [esgamelist] section, at least.

      This is working, so I'm thinking it's something else.

      (unless -s esgamelist never caches artwork or videos? In any case, there are none in the cache for the esgamelist module.)

      According to your settings, you've disabled them, so that's the reason it doesn't cache them.

      1 Reply Last reply Reply Quote 0
      • muldjordM
        muldjord @sleve_mcdichael
        last edited by

        @sleve_mcdichael Skyscraper 3.6.8 out now. Please update.

        S 1 Reply Last reply Reply Quote 0
        • S
          sleve_mcdichael @muldjord
          last edited by sleve_mcdichael

          @muldjord thank you so much! Re: cacheRefresh in the module section, here is a lightly-edited log from this morning:

          # Skyscraper installed from binary (built 23 Feb 2021):
          
          pi@retropie:~ $ cat /home/pi/.emulationstation/gamelists/arcade/gamelist.xml | grep \<name\> | grep Simpsons
                          <name>The Simpsons</name>
          pi@retropie:~ $ Skyscraper -p arcade --cache edit /home/pi/RetroPie/roms/arcade/simpsons2p.zip | grep Title
          Title:          'Simpsons, The' (esgamelist)
          q
          pi@retropie:~ $ cat /home/pi/.skyscraper/config.ini
          [main]
          
          # define paths
          gameListFolder="/home/pi/.emulationstation/gamelists"
          mediaFolder="/home/pi/.emulationstation/downloaded_media"
          #inputFolder="/home/pi/RetroPie/roms"
          #cacheFolder="/home/pi/.skyscraper/cache"
          #importFolder="/home/pi/.skyscraper/import"
          
          # video settings
          videos="true"
          videoSizeLimit="42"
          videoConvertCommand="videoconvert.sh %i %o"
          videoConvertExtension="mp4"
          symlink="true"
          
          # generic
          brackets="false"
          
          [screenscraper]
          # userCreds="user:pass"
          userCreds="REDACTED:REDACTED"
          
          [esgamelist]
          # import customized ES textual data
          cacheRefresh="true"
          #unattend="true"
          cacheCovers="false"
          cacheScreenshots="false"
          cacheWheels="false"
          cacheMarquees="false"
          videos="false"
          pi@retropie:~ $ Skyscraper -p arcade -s esgamelist --flags unattend
          
          # scroll scroll scroll
          
          ---- Resource gathering run completed! YAY! ----
          Writing quick id xml, please wait... Done!
          Writing 1705 (0 new) resources to cache, please wait... Done!
          
          ---- And here are some neat stats :) ----
          Total completion time: 00:00:00
          
          Average search match: 100%
          Average entry completeness: 60%
          
          Total number of games: 73
          Successfully processed games: 73
          Skipped games: 0 (Filenames saved to '/home/USER/.skyscraper/skipped-arcade-esgamelist.txt')
          
          pi@retropie:~ $ Skyscraper -p arcade --cache edit /home/pi/RetroPie/roms/arcade/simpsons2p.zip | grep Title
          Title:          'Simpsons, The' (esgamelist)
          q
          pi@retropie:~ $ sudo /home/pi/RetroPie-Setup/retropie_setup.sh 
          
          # ...Installed Skyscraper from source (built: 31 Mar 2021)
          
          pi@retropie:~ $ Skyscraper -p arcade -s esgamelist --flags unattend
          
          # scroll scroll scroll
          
          ---- Resource gathering run completed! YAY! ----
          Writing quick id xml, please wait... Done!
          Writing 1705 (0 new) resources to cache, please wait... Done!
          
          ---- And here are some neat stats :) ----
          Total completion time: 00:00:00
          
          Average search match: 100%
          Average entry completeness: 75%
          
          Total number of games: 73
          Successfully processed games: 73
          Skipped games: 0 (Filenames saved to '/home/USER/.skyscraper/skipped-arcade-esgamelist.txt')
          
          pi@retropie:~ $ Skyscraper -p arcade --cache edit /home/pi/RetroPie/roms/arcade/simpsons2p.zip | grep Title
          Title:          'The Simpsons' (esgamelist)
          q
          pi@retropie:~ $
          

          Play-by-play:

          • Gamelist: The Simpsons
          • Cache: Simpsons, The (esgamelist)
          • Refresh enabled in module via config.ini
          • Scrape gamelist:
          • Cache: Simpsons, The (esgamelist)
          • Update from source, scrape gamelist:
          • Cache: The Simpsons (esgamelist)

          Something seems to have changed between 23 Feb and today (n.b., before today, I was installed from source built ca. 7 Mar, still getting the "before" behavior.) In any case, it works now, so hooray!

          unattend="true" in the module section is working now as well, thank you!

          Re: flag theinfront -- as I understand it, when building a gamelist, this flag if enabled will try and manipulate the title to put "The" in front, and if disabled it will try and manipulate the title to put "The" in back. In both cases, it's manipulating the cached title before writing the gamelist. Would it make sense to have an option that says "don't manipulate anything, just write it exactly how it says in the cache"?

          (Aside: what about other articles like "a/an," "in," "to," "of," or "from"? I'm not too sure how often these would come up, but I can think of at least one instance in "A Boy and His Blob.")

          One suggestion:

          articleFix="true" - attempts to fix article placement in game titles. If disabled, returns title as-is.

          articleFirst="true" - puts the article first, only if articleFix is also true. When disabled, puts the article last, only if articleFix is true. No effect if articleFix is false.

          (Edit: does it have to be "true/false" or would articleFix=<front|back|none> be a way to get it done in one go?)

          What do you think?

          muldjordM 1 Reply Last reply Reply Quote 0
          • muldjordM
            muldjord @sleve_mcdichael
            last edited by

            @sleve_mcdichael said in Skyscraper two ways: games beginning with "The", and config.ini settings not working:

            In both cases, it's manipulating the cached title before writing the gamelist. Would it make sense to have an option that says "don't manipulate anything, just write it exactly how it says in the cache"?

            Screenscraper tends to always return "Game, The" while TheGamesDb tends to return "The Game". Sometimes they will do some with one and some with the other. I can't think of instances where you wouldn't want to align those when creating a gamelist. So unless you have a strong argument for it, I'll leave it as it is.

            (Aside: what about other articles like "a/an," "in," "to," "of," or "from"? I'm not too sure how often these would come up, but I can think of at least one instance in "A Boy and His Blob.")

            Not sure about this. I don't think you would normally move those around for sorting or presentation. So I'll leave it as is. Thanks for the suggestion though.

            S 1 Reply Last reply Reply Quote 0
            • S
              sleve_mcdichael @muldjord
              last edited by

              @muldjord said in Skyscraper two ways: games beginning with "The", and config.ini settings not working:

              I can't think of instances where you wouldn't want to align those when creating a gamelist. So unless you have a strong argument for it, I'll leave it as it is.

              I suppose the "for" argument is just that it could be frustrating to have your meticulously-edited cached data be subjected to manipulation before writing to the gamelist. If you then re-scrape that manipulated gamelist, those changes have now wormed their way back into your cache, as well (though I do admit that with theinfront available, I see this being less of an issue.)

              I guess it's just the control freak in me believes that since you can manually edit the cache to say whatever you want, it should then be able to pass what you've manually entered, at least with the option of no further manipulation. As customizable and controllable as it is, and then there's this one instance where it's just like "nah, we do this one thing my way instead." It's like having an incredible endless smorgasbord with everything you can imagine and then at the end they're like, "Coke or Pepsi? No tea."

              muldjordM 1 Reply Last reply Reply Quote 0
              • muldjordM
                muldjord @sleve_mcdichael
                last edited by

                @sleve_mcdichael

                I suppose the "for" argument is just that it could be frustrating to have your meticulously-edited cached data be subjected to manipulation before writing to the gamelist.

                I feel like that frustration you mention is a thought up example. We are just talking about "The" which is a normal thing to move around when sorting or generating alphabetical lists. I'm not a fan of adding functionality for the sake of it. The CLI option is a flag which can be enabled or disabled. Supporting tristate would mess with that and complicate things needlessly.

                So, please bear with me, thank you for your suggestions, but you will have to live with this one. :)

                S 1 Reply Last reply Reply Quote 1
                • S
                  sleve_mcdichael @muldjord
                  last edited by sleve_mcdichael

                  @muldjord yes, you are correct. Now that theinfront is available, the remaining frustration is very much imagined, not actual. I don't really expect it to be an issue in implementation, just in principle ;)

                  However, I've now got another issue.

                  Today I am getting "video conversion: failed" errors when scraping a new game. I tried refreshing one that worked previously, and got the same error. Reverted to binary 23 Feb 2021 and both games worked fine ("video conversion: success!"). Reinstalled today's source, and:

                  pi@retropie:~ $ Skyscraper -p arcade -s screenscraper /home/pi/RetroPie/roms/arcade/1942.zip --verbosity 3
                  ------------------------------------------
                  Running Skyscraper v3.6.8 by Lars Muldjord
                  ------------------------------------------
                  Platform:           'arcade'
                  Scraping module:    'screenscraper'
                  Input folder:       '/home/pi/RetroPie/roms/arcade'
                  Game list folder:   '/home/pi/.emulationstation/gamelists/arcade'
                  Covers folder:      '/home/pi/.emulationstation/downloaded_media/arcade/covers'
                  Screenshots folder: '/home/pi/.emulationstation/downloaded_media/arcade/screenshots'
                  Wheels folder:      '/home/pi/.emulationstation/downloaded_media/arcade/wheels'
                  Marquees folder:    '/home/pi/.emulationstation/downloaded_media/arcade/marquees'
                  Videos folder:      '/home/pi/.emulationstation/downloaded_media/arcade/videos'
                  Cache folder:       'cache/arcade'
                  
                  DID YOU KNOW: You can create reports for all files missing certain resources in the cache using the '--cache report:missing=<GROUP or RESOURCE1,RESOURCE2,...> option. Great ways to add missing resources that the online sources are having trouble providing are the '--cache edit' option or the '-s import' module.
                  
                  Fetching limits for user 'REDACTED', just a sec...
                  Setting threads to 2 as allowed for the supplied user credentials.
                  
                  Reading and parsing quick id xml, please wait... Done!
                  Reading and parsing resource cache, please wait... Done!
                  Successfully parsed 1718 resources!
                  
                  Resource cache stats for selected platform:
                  'esgamelist' module
                    Titles       : 73
                    Platforms    : 73
                    Descriptions : 73
                    Publishers   : 73
                    Developers   : 72
                    Ages         : 0
                    Tags         : 73
                    Ratings      : 72
                    ReleaseDates : 73
                    Covers       : 0
                    Screenshots  : 0
                    Wheels       : 0
                    Marquees     : 0
                    Videos       : 0
                  'import' module
                    Titles       : 0
                    Platforms    : 6
                    Descriptions : 0
                    Publishers   : 0
                    Developers   : 0
                    Ages         : 0
                    Tags         : 0
                    Ratings      : 0
                    ReleaseDates : 0
                    Covers       : 2
                    Screenshots  : 2
                    Wheels       : 2
                    Marquees     : 2
                    Videos       : 6
                  'screenscraper' module
                    Titles       : 74
                    Platforms    : 74
                    Descriptions : 74
                    Publishers   : 74
                    Developers   : 73
                    Ages         : 11
                    Tags         : 74
                    Ratings      : 73
                    ReleaseDates : 74
                    Covers       : 73
                    Screenshots  : 74
                    Wheels       : 74
                    Marquees     : 74
                    Videos       : 73
                  
                  Looking for optional 'priorities.xml' file in cache folder... Found!
                  Priorities loaded successfully!
                  
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/fbneo'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/memcard'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/nvram'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/ctrlr'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/hi'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/diff'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/comment'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/ini'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/input'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/image'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/cfg'
                  
                  Trying to parse and load existing game list metadata... Success!
                  
                  Starting scraping run on 1 files using 2 threads.
                  Sit back, relax and let me do the work! :)
                  
                  Warning! Couldn't add resource to cache. Have you run out of disk space?
                  cover: '2149208' > '323270', choosing resize for optimal result!
                  marquee: '158970' > '107347', choosing resize for optimal result!
                  #1/1 (T1) Pass 1 ---- Game '1942' found! :) ----
                  Scraper:        screenscraper
                  From cache:     NO
                  Search match:   100 %
                  Compare title:  '1942'
                  Result title:   '1942' ()
                  Platform:       'Capcom Classics' ()
                  Release Date:   '1984-01-01' ()
                  Developer:      'Capcom' ()
                  Publisher:      'Capcom' ()
                  Players:        '2' ()
                  Ages:           '1+' ()
                  Tags:           'Shoot'em Up, Shoot'em up / Vertical' ()
                  Rating (0-1):   '0.7' ()
                  Cover:          YES ()
                  Screenshot:     YES ()
                  Wheel:          YES ()
                  Marquee:        YES ()
                  Video:          YES ()
                  
                  Description: ()
                  '"1942" is a vertically-scrolling shoot-em-up set in the Pacific theatre during World War II, in which the goal is to reach Tokyo and destroy the entire Japanese air fleet. 
                  
                  The player pilots a plane dubbed the "Super Ace" (its appearance is that of a Lockheed P-38 Lightning) and has to shoot down waves of enemy planes while avoiding incoming enemy fire. The 'Super Ace' can perform a limited number of rolls or 'loop-the-loops' to evade enemy planes and shots. 
                  
                  During the game, waves of red enemy planes periodically appear. If the player manages to destroy a red wave, a power-up, in the form of a 'POW' symbol, will be dropped and can be picked up. The power-up will award either increased fire-power (doubling the player's guns from two to four), a smart bomb that destroys all on-screen enemy planes the instant it's collected, two wingmen planes that flank the Super Ace and increase fire power, or simply extra points.
                  
                  At the end of each stage the 'Super Ace' lands on an aircraft carrier and bonus points are awarded based on player performance. "1942" differs from other games in that its levels are numbered in reverse order, so the game begins at stage 32 and ends at stage 1.'
                  
                  Cache output:
                  Video conversion: 
                  %i: '/opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4'
                  %o: '/opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/tmpfile_769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4'
                  Running command: 'videoconvert.sh /opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4 /opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/tmpfile_769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4'
                  
                  
                  Failed! (set higher '--verbosity N' level for more info)
                  
                  Debug output:
                  Tried with: 'crc=F52D7D3C&md5=89625F073887A5915C3CE2E2E4DA8875&sha1=BF332AC22C7FDB5470AFF4172A9051555E6E1266&romnom=1942.zip&romtaille=106788'
                  Platform: arcade
                  
                  #1/1, (1/0)
                  Elapsed time   : 00:00:22
                  Est. time left : 00:00:00
                  
                  ---- Resource gathering run completed! YAY! ----
                  Writing quick id xml, please wait... Done!
                  Writing 1717 (-1 new) resources to cache, please wait... Done!
                  
                  ---- And here are some neat stats :) ----
                  Total completion time: 00:00:22
                  
                  Average search match: 100%
                  Average entry completeness: 100%
                  
                  Total number of games: 1
                  Successfully processed games: 1
                  Skipped games: 0 (Filenames saved to '/home/USER/.skyscraper/skipped-arcade-screenscraper.txt')
                  
                  pi@retropie:~ $ 
                  

                  I see where it asks if I might be out of disk space. I am not; I have 20G available:

                  pi@retropie:~ $ df -h
                  Filesystem      Size  Used Avail Use% Mounted on
                  /dev/root        29G  8.1G   20G  30% /
                  devtmpfs        805M     0  805M   0% /dev
                  tmpfs           934M   12K  934M   1% /dev/shm
                  tmpfs           934M  9.6M  925M   2% /run
                  tmpfs           5.0M  4.0K  5.0M   1% /run/lock
                  tmpfs           934M     0  934M   0% /sys/fs/cgroup
                  /dev/mmcblk0p1  256M   46M  210M  18% /boot
                  tmpfs           187M     0  187M   0% /run/user/1000
                  pi@retropie:~ $
                  

                  Here is my conversion script:

                  pi@retropie:~ $ ls -l /home/pi/.skyscraper/videoconvert.sh 
                  -rwxr-xr-x 1 pi pi 437 Mar 12 22:52 /home/pi/.skyscraper/videoconvert.sh
                  pi@retropie:~ $ cat /home/pi/.skyscraper/videoconvert.sh 
                  #! /bin/bash
                  # Script provided by jgcobra
                  
                  echo Converting video...
                  echo Source: $1
                  echo Dest: $2
                  
                  # Check if the video is already in 420p format
                  c=$(ffprobe "$1" 2>&1 | grep -c yuv420p)
                  
                  if [ "$c" -eq 0 ]
                  then
                          echo Incorrect source video format detected. Converting video to 420p format...
                          ffmpeg -i "$1" -y -c:v libx264 -pix_fmt yuv420p -acodec copy "$2"
                  else
                          echo Video is in correct 420p format. Copying as is...
                          cp "$1" "$2"
                  fi
                  

                  ...and current config.ini:

                  pi@retropie:~ $ cat /home/pi/.skyscraper/config.ini
                  [main]
                  
                  # define paths
                  gameListFolder="/home/pi/.emulationstation/gamelists"
                  mediaFolder="/home/pi/.emulationstation/downloaded_media"
                  #inputFolder="/home/pi/RetroPie/roms"
                  #cacheFolder="/home/pi/.skyscraper/cache"
                  #importFolder="/home/pi/.skyscraper/import"
                  
                  # video settings
                  videos="true"
                  videoSizeLimit="42"
                  videoConvertCommand="videoconvert.sh %i %o"
                  videoConvertExtension="mp4"
                  symlink="true"
                  
                  # generic
                  brackets="false"
                  theInFront="true"
                  
                  [screenscraper]
                  # userCreds="user:pass"
                  userCreds="REDACTED:REDACTED"
                  
                  [esgamelist]
                  # import customized ES textual data
                  cacheRefresh="true"
                  unattend="true"
                  cacheCovers="false"
                  cacheScreenshots="false"
                  cacheWheels="false"
                  cacheMarquees="false"
                  videos="false"
                  

                  Edit: here's the log from the working binary version:

                  pi@retropie:~ $ Skyscraper -p arcade -s screenscraper /home/pi/RetroPie/roms/arcade/1942.zip --verbosity 3
                  ------------------------------------------
                  Running Skyscraper v3.6.7 by Lars Muldjord
                  ------------------------------------------
                  Platform:           'arcade'
                  Scraping module:    'screenscraper'
                  Input folder:       '/home/pi/RetroPie/roms/arcade'
                  Game list folder:   '/home/pi/.emulationstation/gamelists/arcade'
                  Covers folder:      '/home/pi/.emulationstation/downloaded_media/arcade/covers'
                  Screenshots folder: '/home/pi/.emulationstation/downloaded_media/arcade/screenshots'
                  Wheels folder:      '/home/pi/.emulationstation/downloaded_media/arcade/wheels'
                  Marquees folder:    '/home/pi/.emulationstation/downloaded_media/arcade/marquees'
                  Videos folder:      '/home/pi/.emulationstation/downloaded_media/arcade/videos'
                  Cache folder:       'cache/arcade'
                  
                  DID YOU KNOW: You can turn off these hints using the '--flags nohints' command line flag.
                  
                  Fetching limits for user 'REDACTED', just a sec...
                  Setting threads to 2 as allowed for the supplied user credentials.
                  
                  Reading and parsing quick id xml, please wait... Done!
                  Reading and parsing resource cache, please wait... Done!
                  Successfully parsed 1680 resources!
                  
                  Resource cache stats for selected platform:
                  'esgamelist' module
                    Titles       : 72
                    Platforms    : 72
                    Descriptions : 72
                    Publishers   : 72
                    Developers   : 71
                    Ages         : 0
                    Tags         : 72
                    Ratings      : 71
                    ReleaseDates : 72
                    Covers       : 0
                    Screenshots  : 0
                    Wheels       : 0
                    Marquees     : 0
                    Videos       : 0
                  'import' module
                    Titles       : 0
                    Platforms    : 6
                    Descriptions : 0
                    Publishers   : 0
                    Developers   : 0
                    Ages         : 0
                    Tags         : 0
                    Ratings      : 0
                    ReleaseDates : 0
                    Covers       : 2
                    Screenshots  : 2
                    Wheels       : 2
                    Marquees     : 2
                    Videos       : 6
                  'screenscraper' module
                    Titles       : 72
                    Platforms    : 72
                    Descriptions : 72
                    Publishers   : 72
                    Developers   : 71
                    Ages         : 11
                    Tags         : 72
                    Ratings      : 71
                    ReleaseDates : 72
                    Covers       : 71
                    Screenshots  : 72
                    Wheels       : 72
                    Marquees     : 72
                    Videos       : 70
                  
                  Looking for optional 'priorities.xml' file in cache folder... Found!
                  Priorities loaded successfully!
                  
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/fbneo'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/memcard'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/nvram'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/ctrlr'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/hi'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/diff'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/comment'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/ini'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/input'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/image'
                  Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/cfg'
                  
                  Trying to parse and load existing game list metadata... Success!
                  
                  Starting scraping run on 1 files using 2 threads.
                  Sit back, relax and let me do the work! :)
                  
                  cover: '2149208' > '323270', choosing resize for optimal result!
                  marquee: '158970' > '107347', choosing resize for optimal result!
                  #1/1 (T1) Pass 1 ---- Game '1942' found! :) ----
                  Scraper:        screenscraper
                  From cache:     NO
                  Search match:   100 %
                  Compare title:  '1942'
                  Result title:   '1942' ()
                  Platform:       'Capcom Classics' ()
                  Release Date:   '1984-01-01' ()
                  Developer:      'Capcom' ()
                  Publisher:      'Capcom' ()
                  Players:        '2' ()
                  Ages:           '1+' ()
                  Tags:           'Shoot'em Up, Shoot'em up / Vertical' ()
                  Rating (0-1):   '0.7' ()
                  Cover:          YES ()
                  Screenshot:     YES ()
                  Wheel:          YES ()
                  Marquee:        YES ()
                  Video:          YES ()
                  
                  Description: ()
                  '"1942" is a vertically-scrolling shoot-em-up set in the Pacific theatre during World War II, in which the goal is to reach Tokyo and destroy the entire Japanese air fleet. 
                  
                  The player pilots a plane dubbed the "Super Ace" (its appearance is that of a Lockheed P-38 Lightning) and has to shoot down waves of enemy planes while avoiding incoming enemy fire. The 'Super Ace' can perform a limited number of rolls or 'loop-the-loops' to evade enemy planes and shots. 
                  
                  During the game, waves of red enemy planes periodically appear. If the player manages to destroy a red wave, a power-up, in the form of a 'POW' symbol, will be dropped and can be picked up. The power-up will award either increased fire-power (doubling the player's guns from two to four), a smart bomb that destroys all on-screen enemy planes the instant it's collected, two wingmen planes that flank the Super Ace and increase fire power, or simply extra points.
                  
                  At the end of each stage the 'Super Ace' lands on an aircraft carrier and bonus points are awarded based on player performance. "1942" differs from other games in that its levels are numbered in reverse order, so the game begins at stage 32 and ends at stage 1.'
                  
                  Cache output:
                  Video conversion: 
                  %i: '/opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4'
                  %o: '/opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/tmpfile_769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4'
                  Running command: 'videoconvert.sh /opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4 /opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/tmpfile_769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4'
                  Converting video...
                  Source: /opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4
                  Dest: /opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/tmpfile_769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4
                  Video is in correct 420p format. Copying as is...
                  
                  
                  Success!
                  
                  Debug output:
                  Tried with: 'crc=F52D7D3C&md5=89625F073887A5915C3CE2E2E4DA8875&sha1=BF332AC22C7FDB5470AFF4172A9051555E6E1266&romnom=1942.zip&romtaille=106788'
                  Platform: arcade
                  
                  #1/1, (1/0)
                  Elapsed time   : 00:00:11
                  Est. time left : 00:00:00
                  
                  ---- Resource gathering run completed! YAY! ----
                  Writing quick id xml, please wait... Done!
                  Writing 1681 (1 new) resources to cache, please wait... Done!
                  
                  ---- And here are some neat stats :) ----
                  Total completion time: 00:00:11
                  
                  Average search match: 100%
                  Average entry completeness: 100%
                  
                  Total number of games: 1
                  Successfully processed games: 1
                  Skipped games: 0 (Filenames saved to '/home/USER/.skyscraper/skipped-arcade-screenscraper.txt')
                  
                  pi@retropie:~ $ 
                  

                  I notice I actually see output from the videoconvert script, which seems to be missing entirely from the first one, with the error.

                  muldjordM 2 Replies Last reply Reply Quote 0
                  • muldjordM
                    muldjord @sleve_mcdichael
                    last edited by muldjord

                    @sleve_mcdichael

                    Dammit, I was afraid of that. I had to convert the processes due to deprecated functions in Qt (the framework I use) that run those commands and tested some of them, but not all. And that's probably why this doesn't work. Don't know why it wouldn't save the data to the cache though, I haven't touched that code.

                    I'll have to look into it. Thanks.

                    1 Reply Last reply Reply Quote 0
                    • muldjordM
                      muldjord @sleve_mcdichael
                      last edited by muldjord

                      @sleve_mcdichael said in Skyscraper two ways: games beginning with "The", and config.ini settings not working:

                      I see where it asks if I might be out of disk space. I am not; I have 20G available:

                      I can't reproduce this error. Did you perhaps accidentally run Skyscraper as root at some point (with sudo Skyscraper ...)? In that case some of the cache files might be owned by root and Skyscraper can't overwrite them. If that is the case you need to fix permissions of the /home/pi/.skyscraper folder for it to work again (owner of all files should be pi:pi).

                      muldjordM 1 Reply Last reply Reply Quote 0
                      • muldjordM
                        muldjord @muldjord
                        last edited by muldjord

                        @muldjord 3.6.9 released. Video conversion should be fixed. I tested it with quite an elaborate command line and it worked. Also fixed --flags unpack which also broke with 3.6.8. This is also tested and now works again.

                        The cache saving issue I believe to be a problem on your end. As mentioned I haven't touched that code, and it works fine here.

                        S 1 Reply Last reply Reply Quote 1
                        • S
                          sleve_mcdichael @muldjord
                          last edited by

                          @muldjord said in Skyscraper two ways: games beginning with "The", and config.ini settings not working:

                          The cache saving issue I believe to be a problem on your end. As mentioned I haven't touched that code, and it works fine here.

                          I believe the cache-saving error was closely related to the video-conversion error. The resource that couldn't be saved to the cache was the video that didn't get converted/copied. I only saw them together, and only in 3.6.8. Neither were present in 3.6.7, nor after today's 3.6.9 update, thank you!

                          pi@retropie:~ $ Skyscraper -p arcade -s screenscraper /home/pi/RetroPie/roms/arcade/1942.zip --verbosity 3
                          ------------------------------------------
                          Running Skyscraper v3.6.9 by Lars Muldjord
                          ------------------------------------------
                          Platform:           'arcade'
                          Scraping module:    'screenscraper'
                          Input folder:       '/home/pi/RetroPie/roms/arcade'
                          Game list folder:   '/home/pi/.emulationstation/gamelists/arcade'
                          Covers folder:      '/home/pi/.emulationstation/downloaded_media/arcade/covers'
                          Screenshots folder: '/home/pi/.emulationstation/downloaded_media/arcade/screenshots'
                          Wheels folder:      '/home/pi/.emulationstation/downloaded_media/arcade/wheels'
                          Marquees folder:    '/home/pi/.emulationstation/downloaded_media/arcade/marquees'
                          Videos folder:      '/home/pi/.emulationstation/downloaded_media/arcade/videos'
                          Cache folder:       'cache/arcade'
                          
                          DID YOU KNOW: With the '--cache show' command line option, you can check the status of the resources that are cached for the platform you are scraping.
                          
                          Fetching limits for user 'REDACTED', just a sec...
                          Setting threads to 2 as allowed for the supplied user credentials.
                          
                          Reading and parsing quick id xml, please wait... Done!
                          Reading and parsing resource cache, please wait... Done!
                          Successfully parsed 1700 resources!
                          
                          Resource cache stats for selected platform:
                          'esgamelist' module
                            Titles       : 72
                            Platforms    : 72
                            Descriptions : 72
                            Publishers   : 72
                            Developers   : 71
                            Ages         : 0
                            Tags         : 72
                            Ratings      : 71
                            ReleaseDates : 72
                            Covers       : 0
                            Screenshots  : 0
                            Wheels       : 0
                            Marquees     : 0
                            Videos       : 0
                          'import' module
                            Titles       : 0
                            Platforms    : 7
                            Descriptions : 0
                            Publishers   : 0
                            Developers   : 0
                            Ages         : 0
                            Tags         : 0
                            Ratings      : 0
                            ReleaseDates : 0
                            Covers       : 3
                            Screenshots  : 3
                            Wheels       : 3
                            Marquees     : 2
                            Videos       : 7
                          'screenscraper' module
                            Titles       : 73
                            Platforms    : 73
                            Descriptions : 73
                            Publishers   : 73
                            Developers   : 72
                            Ages         : 11
                            Tags         : 73
                            Ratings      : 72
                            ReleaseDates : 73
                            Covers       : 72
                            Screenshots  : 73
                            Wheels       : 73
                            Marquees     : 73
                            Videos       : 72
                          
                          Looking for optional 'priorities.xml' file in cache folder... Found!
                          Priorities loaded successfully!
                          
                          Adding files from subdir: '/home/pi/RetroPie/roms/arcade/fbneo'
                          Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010'
                          Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/memcard'
                          Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/nvram'
                          Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/ctrlr'
                          Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/hi'
                          Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/diff'
                          Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/comment'
                          Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/ini'
                          Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/input'
                          Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/image'
                          Adding files from subdir: '/home/pi/RetroPie/roms/arcade/mame2010/cfg'
                          
                          Trying to parse and load existing game list metadata... Success!
                          
                          Starting scraping run on 1 files using 2 threads.
                          Sit back, relax and let me do the work! :)
                          
                          cover: '2149208' > '323270', choosing resize for optimal result!
                          marquee: '158970' > '107347', choosing resize for optimal result!
                          #1/1 (T1) Pass 1 ---- Game '1942' found! :) ----
                          Scraper:        screenscraper
                          From cache:     NO
                          Search match:   100 %
                          Compare title:  '1942'
                          Result title:   '1942' ()
                          Platform:       'Capcom Classics' ()
                          Release Date:   '1984-01-01' ()
                          Developer:      'Capcom' ()
                          Publisher:      'Capcom' ()
                          Players:        '2' ()
                          Ages:           '1+' ()
                          Tags:           'Shoot'em Up, Shoot'em up / Vertical' ()
                          Rating (0-1):   '0.7' ()
                          Cover:          YES ()
                          Screenshot:     YES ()
                          Wheel:          YES ()
                          Marquee:        YES ()
                          Video:          YES ()
                          
                          Description: ()
                          '"1942" is a vertically-scrolling shoot-em-up set in the Pacific theatre during World War II, in which the goal is to reach Tokyo and destroy the entire Japanese air fleet. 
                          
                          The player pilots a plane dubbed the "Super Ace" (its appearance is that of a Lockheed P-38 Lightning) and has to shoot down waves of enemy planes while avoiding incoming enemy fire. The 'Super Ace' can perform a limited number of rolls or 'loop-the-loops' to evade enemy planes and shots. 
                          
                          During the game, waves of red enemy planes periodically appear. If the player manages to destroy a red wave, a power-up, in the form of a 'POW' symbol, will be dropped and can be picked up. The power-up will award either increased fire-power (doubling the player's guns from two to four), a smart bomb that destroys all on-screen enemy planes the instant it's collected, two wingmen planes that flank the Super Ace and increase fire power, or simply extra points.
                          
                          At the end of each stage the 'Super Ace' lands on an aircraft carrier and bonus points are awarded based on player performance. "1942" differs from other games in that its levels are numbered in reverse order, so the game begins at stage 32 and ends at stage 1.'
                          
                          Cache output:
                          Video conversion: 
                          %i: '/opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4'
                          %o: '/opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/tmpfile_769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4'
                          Running command: 'videoconvert.sh /opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4 /opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/tmpfile_769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4'
                          Converting video...
                          Source: /opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4
                          Dest: /opt/retropie/configs/all/skyscraper/cache/arcade/videos/screenscraper/tmpfile_769ccf601a52d89cec19168a8bf3df6351a9a0f5.mp4
                          Video is in correct 420p format. Copying as is...
                          
                          
                          Success!
                          
                          Debug output:
                          Tried with: 'crc=F52D7D3C&md5=89625F073887A5915C3CE2E2E4DA8875&sha1=BF332AC22C7FDB5470AFF4172A9051555E6E1266&romnom=1942.zip&romtaille=106788'
                          Platform: arcade
                          
                          #1/1, (1/0)
                          Elapsed time   : 00:00:11
                          Est. time left : 00:00:00
                          
                          ---- Resource gathering run completed! YAY! ----
                          Writing quick id xml, please wait... Done!
                          Writing 1700 (0 new) resources to cache, please wait... Done!
                          
                          ---- And here are some neat stats :) ----
                          Total completion time: 00:00:11
                          
                          Average search match: 100%
                          Average entry completeness: 100%
                          
                          Total number of games: 1
                          Successfully processed games: 1
                          Skipped games: 0 (Filenames saved to '/home/USER/.skyscraper/skipped-arcade-screenscraper.txt')
                          
                          pi@retropie:~ $
                          

                          No apparent errors. Thanks again!

                          1 Reply Last reply Reply Quote 1
                          • First post
                            Last post

                          Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                          Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.