scrape videos and reencode them directly on the Raspberry Pi with sselph's scraper

blubbblubb

Foreword:
After setting up a new Retropie system i wanted to scrape some coverart and videos with sselph's scraper but the video files were too big for my SD Card so i wanted to use the integrated "-convert_videos" flag to reduce the file size a bit. However there are no readily available HandBrake builds for the Raspberry Pi so i scraped and converted everything on my desktop system, after i was done i thought that it would be nice to be able to do all of this on the RPi itself so i tried to compile HandBrake on my RPi2

Make sure to read the whole post before you start compiling everything, there is a way to save some time which i will mention at the end of the post

For now this seems to only work on a Raspberry Pi 2 or 3, it is possible to build the HandBrakeCli executable on a Raspberry Pi 1 while following the steps described below, however the resulting binary can not be used on the Raspberry Pi 1, it just crashes with "illegal instruction" as of now i was not able to figure out why this is happening

How to compile HandBrakeCli on your Raspberry Pi

# First update your apt repositories
sudo apt update
# then install some dependencies
sudo apt install libxml2-dev libjansson-dev libvorbis-dev libx264-dev libass-dev libmp3lame-dev libopus-dev libbz2-dev libtheora-dev
# export some variables so that all dependencies will be found during compilation
export PKG_CONFIG=/usr/bin/pkg-config
export PKG_CONFIG_PATH=/usr/lib/pkgconfig
# to be sure change to your Home directory, so we know where all the files will be stored
cd ~
# now clone the HandBrake git repository
git clone https://github.com/HandBrake/HandBrake.git
# and change to the newly created directory
cd HandBrake
# now we need to configure how we want to build HandBrake, as our main objective is to use the command line interface for sselph's scraper we disable the GUI
./configure --disable-gtk
# download this patch file
wget -O patch.diff "https://drive.google.com/uc?export=download&id=0By2gyQDysGXKdXBIRWI0RlpYZEk"
# now we apply the patch to disable some encoders and decoders so we dont need to install a ton of dependencies
# take a look at the file with "cat patch.txt" or similar to make sure im not trying anything funny ;)
patch -p0 < patch.diff
# now we change to the build directory
cd build/
# and start building HandBrake (i did not pay that much attention to the time it took to completely build everything but i would guess it was around 3 hours on a stock RPi2
make
# after this completes successfully we can install HandBrake systemwide
sudo make install

Now you are able to directly reencode the videos scraped by sselph's scraper

Example command to scrape and reencode videos:

/opt/retropie/supplementary/scraper/scraper -console_src=ss,gdb -max_height=540 -max_width=505 -download_videos=true -download_marquees=true -image_dir=media -image_path=media -video_dir=media -video_path=media -marquee_dir=media -marquee_path=media -use_nointro_name=false -convert_videos -lang "en" -region "eu,wor,us,jp,xx"

To give you some idea of the time needed to reencode videos on the RPi2 i ran a small test with 10 NES roms:

scraping without the -convert_videos flag took 34.453s
scraping with the flag took 14m22.360s
the size of the media folder was 33MB without reencoding and 17MB with reencoded videos, reencoding saves roughly 50% of the space needed

precompiled deb package

compiling your own software does not give you a rush of excitement?
You can think of a better way to spend a couple of hours?
I created a deb package (RPi2/3 only at the moment) which you can easily install on your system and save some time, you just need to install one dependency beforehand

# update apt repositories
sudo apt update
# install dependency
sudo apt install libjansson-dev
# download deb package, note this will only work on a RPi2 or RPi3
wget -O handbrakecli_rpi2_rpi3-20170926-1_armhf.deb "https://drive.google.com/uc?export=download&id=0By2gyQDysGXKZWhXQ0g4UGt4Z28"
sudo dpkg -i handbrakecli_rpi2_rpi3-20170926-1_armhf.deb

I hope this is helpful to some of you ;)

If you have any suggestions for improving this little guide or find a mistake feel free to tell me :)

Trailjacker

@blubbblubb said in scrape videos and reencode them directly on the Raspberry Pi with sselph's scraper:

update apt repositories

sudo apt update

install dependency

sudo apt install libjansson-dev

download deb package, note this will only work on a RPi2 or RPi3

wget -O handbrakecli_rpi2_rpi3-20170926-1_armhf.deb "https://drive.google.com/uc?export=download&id=0By2gyQDysGXKZWhXQ0g4UGt4Z28"
sudo dpkg -i handbrakecli_rpi2_rpi3-20170926-1_armhf.deb

Thank you very much. This is a HUGE help.

Jaxel

I get an error about missing "FFMPEG" when I try to do this...

Jaxel

The exact error I get:

exec: "ffmpeg": executable file not found in $PATH

Jaxel

Anyone?