[SCRIPT] Batch convert YUV 4:4:4 videos to YUV 4:2:0 in Retropie/Linux

Quackwalks · 18 Apr 2019, 11:58

mkdir -p converted                         # create the folder "converted" if it doesn't exist
for f in *-video.mp4                       # process all videos
  do
    c=$(avprobe $f 2>&1 | grep -c yuv444p) # (-c)ount no. of "444p"

    if [ "$c" -gt 0 ]                      # if 444p is present,
      then				   # convert into 420p
        avconv -i $f -y -pix_fmt yuv420p -strict experimental converted/$f
    fi
done

I highlighted the text from your code block then copy and pasted it into notepad++. Saved it with UTF-8 encoding as a .sh file. Used WinSCP to copy it into home/pi. On the raspberry pi 3b+ running retropie 4.4, I quit emulationstation and used the chmod command with no problem. Then entered cd home/pi/RetroPie/roms/fba/media/videos, then sh/home/pi/444p-to-420p.sh, which gives me the syntax error. The converted video folder was created just fine.
The system I tried it for is fba. I used Skraper and the Screen Scraper database to scrape my roms, which left me with .mp4 files that had names identical to the roms (not having -video at the end of the file names). Don't go to any great trouble for me. I realized last night I had backup videos from a previous image. I used XML Scraper V2 back then, so last night used a bulk rename application to rename them without "-video" at the end. The funny thing is that I used this script about a year ago for the videos I had backed up, so it worked for me then. It's a real noodle scratcher for me.

mitu · 18 Apr 2019, 12:05

@Quackwalks said in [SCRIPT] Batch convert YUV 4:4:4 videos to YUV 4:2:0 in Retropie/Linux:

I highlighted the text from your code block then copy and pasted it into notepad++. Saved it with utc-8 encoding as a .sh file. Used WinSCP to copy it into home/pi. On the raspberry pi 3b+ running retropie 4.4, I quit emulationstation and used the chmod command with no problem. Then entered cd home/pi/RetroPie/roms/fba/media/videos, then sh/home/pi/444p-to-420p.sh, which gives me the syntax error. The converted video folder was created just fine.

It's not the character encoding, but the line endings settings of the file - should be Unix (LF) in Notepad++.

Quackwalks · 18 Apr 2019, 12:14

@mitu Ah, I see now. Thank you to you two for the help. I'll never gloss over line ending settings again.

Clyde · 25 Apr 2019, 08:01

@mitu @Quackwalks Sorry, for the late answer. I was occupied over Easter and in the days after. Thanks @mitu for stepping in.

Gogol · 29 Apr 2019, 13:43

Hello, I wonder if someone could help me with this batch converter as I am having some trouble.

These are the steps I have taken...

Copied the text from the code block posted by clyde into Notepad++, I saved it as a Unix script file, with Unix (LF) and UTF-8 encoding, the file appears as 444p-to-420p.sh on my PC desktop.
Using WinSCP, I moved the 444p-to-420p.sh file to /home/pi on my Raspberry Pi 3B.
From emulationstation, I press F4 on my keyboard to get to the command line.
I type in chmod u+x 444p-to-420p.sh and press enter and I get an error message "chmod: changing permissions of '444p-to-420p.sh': Operation not permitted"

I have obviously made a mistake somewhere and would be grateful for any assistance.

mitu · 29 Apr 2019, 19:21

@Gogol Are you using WinScp as the root ? It's not advisable and it's probably the cause of this error. Before running chmod, run:

sudo chown pi 444p-to-420p.sh

Clyde · 30 Apr 2019, 11:24

@Gogol If @mitu's tip doesn't help, please show us the output of

ls -l 444p-to-420.sh

to see the owner and permissions of the file. Both "l" are small "L" and not ones.

As a side note, if you save the file in /home/pi/bin you can invoke it from any other directory without writing its full path (/home/pi/444p-to-420p.sh in your case), i.e. from your videos directory. See my Hint in my opening post.

From your current location of the file, these commands will to that:

cd /home/pi             # make sure you are in pi's home directory
mkdir bin               # create the directory "bin"
mv 444p-to-420p.sh bin  # move the script to bin

Gogol · 2 May 2019, 10:50

Hi, sorry for the late reply.

I was not able to get this to work, but I think the problem lies with the .mp4 videos I am trying to convert, I downloaded them from the internet from sites like emumovies and arcadepunks.

They do not display on the preview, so I tried converting one of the video snaps via VLC Media Player and was able to get the video to display in emulation station, however it was extremely glitchy. I have also tried with handbrake but with no success, the video does not display.

Do you know of another way to convert these videos, or possibly the correct settings in VLC or handbrake to get the videos to display without any glitches?

thelostsoul · 2 May 2019, 11:01

@Gogol I converted most problematic videos by completely recoding with Avidemux on my Ubuntu pc. That was before I saw this script. I don't remember the exact settings:

Video Codec=Mpeg4 AVC (x264)
change quality setting to something like 26 or so (really don't know anymore)
Audio Codec=AAC or MP3, not sure
Output Format=MP4v2 Muxer

Gogol · 2 May 2019, 15:49

@thelostsoul I tried Avidemux for Windows with those settings but was unsuccessful in getting them to work.

I do not want to derail this thread and I could not find a PM function, so I wanted to ask if you would be willing to convert the eight .mp4 video snaps I have using the batch converter created by Clyde if I sent them to you, to test if they work after conversion and then send them back to me if they do?

thelostsoul · 2 May 2019, 15:55

@Gogol You are right. Create a new thread and use my nickname so I get ping. Upload the files all at once in a zip file to https://uploadfiles.io/ and I try.

bluebrews · 2 Feb 2020, 20:49

Just came across this and it worked like a champ for me. Thanks!

iainjh · 9 Jun 2020, 18:05

thank you also. My .mp4's werent named -video.mp4 so I whipped that out and the script has run perfectly. thanks!

toto2000 · 8 Jul 2020, 06:32

hello everyone, I'm using Retropie 4.6 right now, and I encountered lots of scrapped videos that only had sound but no display in ES, with OMX player on or off.

I'm sorry, but aren't we all taking the wrong way to find a solution? The short term solution is to convert the videos, but the long term solution is to be able to play them as is, without converting. What other distribs (Batocera, Recalbox, Regamebox) do that Retropie doesn't? Why all these videos do play on other images and not on Retropie?
I got some scrapped romsets and I had to convert all videos, but when I spoke to the authors they said (what? They do play perfectly with our images.... update Retropie, we're using VLC behind the scenes, there is no problem....)

So... what would be the solution to play these videos WITHOUT converting?

Clyde · 8 Jul 2020, 11:30

@toto2000 said in [SCRIPT] Batch convert YUV 4:4:4 videos to YUV 4:2:0 in Retropie/Linux:

So... what would be the solution to play these videos WITHOUT converting?

You could switch from OMX player to VLC by disabling the option "Other Settings" > "Use OMX Player (HW Accelerated)" in Emulation Station. VLC should be able to play almost any video format you throw at it, but it is slower than OMX on Raspberry Pis.

toto2000 · 12 Jul 2020, 07:58

@Clyde no, it doesn't change anything. I tried to switch this option off in ES Menu, and the video have sound but do not display at all + I also tried to restart ES. Nothing seems to have an effect on these videos.
Here is such a video: https://mega.nz/file/DhRyGYiJ#O_HlRb3Dptb-NW-ZyW9GQPdVl8I_VRhlwI0bPx2KkJs

Clyde · 7 Dec 2020, 16:01

Yeah well, this video is in the aforementioned yuv444 format, and as I learned recently, VLC now shares some limitations of OMX because its now also hardware accelerated.

So on a Pi, you may only have the option to convert the videos to yuv420.

bmulligan · 3 Oct 2020, 04:01

Thank you for this, Clyde. I've been pulling my hair out for days trying to figure out what I have done wrong scraping video snaps from the ArcadeItalia DB. It seems insane that the video player in ES can't be updated to play 4:4:4 video by default, or that there's no 'official' documentation for this at least.

Clyde · 21 Nov 2020, 09:59

Update 4:

Removed "-video" suffix to support videos without it, e.g. those scraped with Skyscraper.
Added quotation marks to the variable $f to support file names with spaces and special characters.
Added a simple progress indicator by showing the currently checked file's path and name in the console.
Moved the update history here to declutter the opening post.

Update 3: Changed avprobe to ffprobe and avconv to ffmpeg for compatibility with Debian Buster and above, which doesn't have the transitional package libav-tools anymore . See Update #1.

Update 2: Deleted some outdated lines and edit: remarks.

Update 1: From Retropie 4.4 on, avprobe was replaced by ffprobe, and avconv was replaced by ffmpeg. But at least in RP 4.4, the old designations are still usable, and so is the script.

Colombia20102018 · 27 Dec 2020, 13:58

@clyde hello, I hate to do this but I’m completely new to this. Is there a way to summarize the steps to follow in a way that a kid would understand it? I’m reading all the posts and I get the idea but not sure how to implement it. I am working with a keyboard connected to my raspberry pi 3B and I can hear the sound of my videos but there is no picture. Can I fix them directly on raspberry pi or do I need to connect to a terminal? Can I fix them all at once or I have to fix one by one? Thank you and sorry for so many questions.