sselph scraper "-convert_videos" not working on Windows 10

Scorpio588

I am currently trying to keep all my ROM's, gamelists.xml's, and images on an external usb drive. This is helping me save the memory on the SD card. The stand alone sselph scraper is working great except for one problem.

When I try to run the command:

"scraper -workers 8 -console_src ss -download_vidoes -convert_videos"

I get this on the command prompt (for each ROM):

2018/03/07 07:33:21 INFO: Starting: Bomber-Man Tournament.gba
2018/03/07 07:33:21 ERR: error processing Bomber-Man Tournament.gba: exit status 1
2018/03/07 07:33:27 INFO: Starting: Bomberman Max 2 - Blue (E).gba
2018/03/07 07:33:27 ERR: error processing Bomberman Max 2 - Blue (E).gba: exit status 1

The scraper IS downloading the images and videos, but not converting them into a suitable raspberry pi format (i.e. the "-convert_videos" tag is really working) I already have ffmpeg installed and accessible in the cmd prompt.

I posted this issue on GitHub for sselph's scraper, but wondered if anyone on here has been able to accomplish the setup I'm trying to (all ROM's, gamelists, metadata, images, and videos on external usb), or if anybody has some nuggets of wisdom they'd be willing to share. Thanks!

scraping on Windows 10 Home x64
scraper version = 1.4.5
ffmpeg version = N-90173-gfa0c9d69d3

(P.S. Besides the videos, this DOES work. I can scrape from windows on the USB, and all images and metadata load just fine when I'm put the USB back into the RetroPie, but it I download videos, the pi struggles to keep up with some of them. I'm hoping "-convert_videos" fixes this.)

mitu

@scorpio588 Is the ffmpeg.exe executable in your $path (or in the scraper's foler) when you execute the scraper ? The convert just call ffmpeg directly, so I guess the error it's related to the convert process failing to start the program.

Scorpio588

@mitu Yes, the ffmpeg.exe is in my $path. I was getting "ffpmeg not in $PATH" errors before I added it. I'm not familiar enough with ffmpeg yet, do you know how I could convert all videos in a directory to 320x240@30fps with ffmpeg?

mitu

@scorpio588 Looking at the sselph's scraper source, the command for encoding the videos is

ffmpeg -i <input_file> -c:v libx264 -preset fast -crf 23 -vfscale=w=320:h=240:force_original_aspect_ratio=decrease -c:aa ac -b:a 80k <outputFile>

You could run it from the command line and see it works. If it fails, then you'll know why ffmpeg refuses to convert the video.

Scorpio588

@mitu I'll give that a try! I couldn't find that in the source before. Thanks!

Scorpio588

Ok, I just tried running:

ffmpeg -i . -c:v libx264 -preset fast -crf 23 -vfscale=w=320:h=240:force_original_aspect_ratio=decrease -c:aa ac -b:a 80k .

and the cmd prompt gave me:

ffmpeg version N-90173-gfa0c9d69d3 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 7.3.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libmfx --enable-amf --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth
libavutil 56. 7.101 / 56. 7.101
libavcodec 58. 13.100 / 58. 13.100
libavformat 58. 10.100 / 58. 10.100
libavdevice 58. 2.100 / 58. 2.100
libavfilter 7. 12.100 / 7. 12.100
libswscale 5. 0.101 / 5. 0.101
libswresample 3. 0.101 / 3. 0.101
libpostproc 55. 0.100 / 55. 0.100
Unrecognized option 'vfscale=w=320:h=240:force_original_aspect_ratio=decrease'.
Error splitting the argument list: Option not found

So it's not liking the "vfscale=w=320:h=240:force_original_aspect_ratio=decrease" argument. I'm guessing that ffmpeg was updated but the scraper hasn't changed to reflect that, or the scraper just wasn't tested for this on windows.

Scorpio588

@scorpio588 said in sselph scraper "-convert_videos" not working on Windows 10:

Unrecognized option 'vfscale=w=320:h=240:force_original_aspect_ratio=decrease'.

Ok, I had to split "vfscale=w=320:h=240:force_original_aspect_ratio=decrease" into "vf scale=w=320:h=240:force_original_aspect_ratio=decrease".

Scorpio588

Now when I run:

ffmpeg -i %CD% -c:v libx264 -preset fast -crf 23 -vf scale=w=320:h=240:force_original_aspect_ratio=decrease -c:aa ac -b:a 80k %CD%

I get a "G:\retropie-mount\roms\gba: Permission denied" error.

mitu

@scorpio588 %CD% in Windows (cmd.exe) is the current folder, you cannot convert a folder, you need to have an input file and a - different - output file as arguments

Scorpio588

Oh, ok. I mistook "<input_file>" above as the directory. In that case, would I have to run this command for each video then? If possible, I'd like to be able to convert all the videos in a directory.

mitu

@scorpio588 Something like (borrowed from https://superuser.com/questions/808302/use-ffmpeg-to-batch-process-m4a-to-m4a):

forfiles /P "<My_Videos_folder>" /C "cmd /c ffmpeg -i @file <lots of arguments here> @fname-converted.@ext"

Scorpio588

@mitu said in sselph scraper "-convert_videos" not working on Windows 10:

@scorpio588 Looking at the sselph's scraper source, the command for encoding the videos is
ffmpeg -i <input_file> -c:v libx264 -preset fast -crf 23 -vfscale=w=320:h=240:force_original_aspect_ratio=decrease -c:aa ac -b:a 80k <outputFile>
You could run it from the command line and see it works. If it fails, then you'll know why ffmpeg refuses to convert the video.

Where did you find that command in the source code? I got some flags in my last test (still not working but getting closer I think) saying "Unknown encoder 'ac' ".

Scorpio588

I tried running:

forfiles /P ".\images" /C "cmd /c ffmpeg -i @file -c:v libx264 -preset fast -crf 23 -vf scale=w=320:h=240:force_original_aspect_ratio=decrease -c:aa ac -b:a 80k .\images\converted\\@file.mp4"

and the output for each file looked something like this:

ffmpeg version N-90173-gfa0c9d69d3 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7.3.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libmfx --enable-amf --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth
  libavutil      56.  7.101 / 56.  7.101
  libavcodec     58. 13.100 / 58. 13.100
  libavformat    58. 10.100 / 58. 10.100
  libavdevice    58.  2.100 / 58.  2.100
  libavfilter     7. 12.100 /  7. 12.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, image2, from 'Sonic Advance-image.jpg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 8147 kb/s
    Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 400x397, 25 tbr, 25 tbn, 25 tbc
Codec AVOption b (set bitrate (in bits/s)) specified for output file #0 (.\images\converted\Sonic Advance-image.jpg.mp4) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
.\images\converted\Sonic Advance-image.jpg.mp4: No such file or directory

I realize that I'm doing something wrong with the file name, but before I fix that, is there a way to run this so it only tries to convert .mp4 files? The directory I'm using contains .mp4 and .jpg files. I tried a command:

for %a in ("*.mp4") do ffmpeg -i "%a" -c:v libx264 -preset fast -crf 23 -vf scale=w=320:h=240:force_original_aspect_ratio=decrease -c:aa ac -b:a 80k "converted\%a.mp4"

and that seemingly did nothing.

mitu

You can use the /m *.mp4 argument to forfiles to filter through the files and only get the .mp4 files.

Scorpio588

Ok, I ran:

for %a in ("*.mp4") do ffmpeg -i "%a" -c:v libx264 -preset fast -crf 23 -vf scale=w=320:h=240:force_original_aspect_ratio=decrease -c:aa -b:a 80k "converted\%a.mp4"

and got output:

G:\retropie-mount\roms\gba\images>ffmpeg -i "Zelda - the Minish Cap-video.mp4" -c:v libx264 -preset fast -crf 23 -vf scale=w=320:h=240:force_original_aspect_ratio=decrease -c:aa -b:a 80k "converted\Zelda - the Minish Cap-video.mp4.mp4"
ffmpeg version N-90173-gfa0c9d69d3 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7.3.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libmfx --enable-amf --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth
  libavutil      56.  7.101 / 56.  7.101
  libavcodec     58. 13.100 / 58. 13.100
  libavformat    58. 10.100 / 58. 10.100
  libavdevice    58.  2.100 / 58.  2.100
  libavfilter     7. 12.100 /  7. 12.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Zelda - the Minish Cap-video.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.20.100
  Duration: 00:00:36.75, start: 0.000000, bitrate: 2271 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 720x480, 2073 kb/s, 60 fps, 60 tbr, 15360 tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
[NULL @ 0000013d46bf72c0] Unable to find a suitable output format for '80k'
80k: Invalid argument

with the last couple lines:

[NULL @ 0000013d46bf72c0] Unable to find a suitable output format for '80k'
80k: Invalid argument

being in red.

Scorpio588

The "80k" error was due to a missing parameter, which was "aac". After figuring that out, I tried this:

for %a in ("*.mp4") do ffmpeg -i "%a" -c:v libx264 -preset fast -crf 23 -vf scale=w=320:h=240:force_original_aspect_ratio=decrease -c:aa aac -b:a 80k "converted\%a.mp4"

and got:

G:\retropie-mount\roms\gba\images>ffmpeg -i "Zelda - the Minish Cap-video.mp4" -c:v libx264 -preset fast -crf 23 -vf scale=w=320:h=240:force_original_aspect_ratio=decrease -c:aa aac -b:a 80k "converted\Zelda - the Minish Cap-video.mp4.mp4"
ffmpeg version N-90173-gfa0c9d69d3 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7.3.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libmfx --enable-amf --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth
  libavutil      56.  7.101 / 56.  7.101
  libavcodec     58. 13.100 / 58. 13.100
  libavformat    58. 10.100 / 58. 10.100
  libavdevice    58.  2.100 / 58.  2.100
  libavfilter     7. 12.100 /  7. 12.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Zelda - the Minish Cap-video.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.20.100
  Duration: 00:00:36.75, start: 0.000000, bitrate: 2271 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 720x480, 2073 kb/s, 60 fps, 60 tbr, 15360 tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0000026d517800c0] height not divisible by 2 (320x213)
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0000026d5136c280] Qavg: 59253.898
[aac @ 0000026d5136c280] 2 frames left in the queue on closing
Conversion failed!

This actually produces .mp4 files in the converted directory! But they are unplayable (corrupted file errors when trying to play).

Scorpio588

I just realized that I have to use "converted.%~na.mp4" instead of "converted.%a.mp4" to avoid .mp4.mp4 files. So the command looks like this:

for %a in ("*.mp4") do ffmpeg -i "%a" -c:v libx264 -preset fast -crf 23 -vf scale=w=320:h=240:force_original_aspect_ratio=decrease -c:aa aac -b:a 80k "converted\%~na.mp4"

and I get this:

G:\retropie-mount\roms\gba\images>ffmpeg -i "Zelda - the Minish Cap-video.mp4" -c:v libx264 -preset fast -crf 23 -vf scale=w=320:h=240:force_original_aspect_ratio=decrease -c:aa aac -b:a 80k "converted\Zelda - the Minish Cap-video.mp4"
ffmpeg version N-90173-gfa0c9d69d3 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7.3.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libmfx --enable-amf --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth
  libavutil      56.  7.101 / 56.  7.101
  libavcodec     58. 13.100 / 58. 13.100
  libavformat    58. 10.100 / 58. 10.100
  libavdevice    58.  2.100 / 58.  2.100
  libavfilter     7. 12.100 /  7. 12.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Zelda - the Minish Cap-video.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.20.100
  Duration: 00:00:36.75, start: 0.000000, bitrate: 2271 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 720x480, 2073 kb/s, 60 fps, 60 tbr, 15360 tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0000020422f600c0] height not divisible by 2 (320x213)
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0000020422b4c240] Qavg: 59253.898
[aac @ 0000020422b4c240] 2 frames left in the queue on closing
Conversion failed!

Scorpio588

Ok! I got it to work (mostly)! I used the command:

for %a in ("*.mp4") do ffmpeg -i "%a" -c:v libx264 -preset fast -crf 23 -vf scale=320:240: -c:aa aac -b:a 80k -r 30 "converted\%~na.mp4"

to convert all the .mp4 files in the images folder and store them in images\converted. I'll just have to copy-replace the converted images into the images directory when the conversions are done! There is a chance that some videos will look funny because I'm not preserving the aspect ratio, but they look OK so far to me. The converted videos are all 320x240@30fps!

Thank you for your help @mitu!

Scorpio588

Sorry, there's a typo in the last comment. Use this command instead:

for %a in ("*.mp4") do ffmpeg -i "%a" -c:v libx264 -preset fast -crf 23 -vf scale=320:240 -c:aa aac -b:a 80k -r 30 "converted\%~na.mp4"