Translate games on the fly

Folly

While being busy on the old-Japanese-computer-appreciation-thread,
I got an idea for translating Japanese games on the fly.

So that's why I made this separate thread.

I was a bit curious if I was able to translate Japanese text from a screenshot of a game.
First I was tinkering with english and later with Japanese.
I was able to get some results.
Although with Japanese it's hard to tell, but the characters recognised, looked quite the same.
Putting it through an online translator gave me somewhat mixed results.
It's hard to tell if the Japanese translations are correct.

Anyway I proceeded because the results were looking quite good.

So I automated this in a simple script and made a repository for it.

My script looks, on the fly, for pictures, recognises them, translates them and then talks the translations back over audio.

I experimented with OSD-overlays but these only work when running Retroarch from a desktop environment.
OSD-overlays really distracts you.
Also, it was really difficult to get some good results.

For the original Retropie OS this is also not an option and text to speech is quite nice.
So when I found that there were text to speech options, I implemented that.

For Bing I had to add a seperate text to speech program.
Google did not need a seperate text to speech program.

While testing I found that google-translate has a limit.
So, alternatively, I made a script for Bing-translate on the way.
Seems that Bing has also limits. But I think it can be used quite often.

Along the way I also made a nice discovery.
I was curious if I could make several screenshots with the same checksum of a game.
Then you could combine files with a particular image.
Did a test with 3 screenshots and all md5sums were exactly the same :)
WOW. That made me wonder, how much we could do with this knowledge ?

For example, then I could combine a screenshot with a textfile containing improved translations. Or add help files.

So I made a seperate script, that checks for screenshots, check the checksum, compares that to a textfile, and turns the text into speech.
And, WOW, that works.
(this is not in the repository yet)

In theory, everyone can then add translations from a particular screenshot, that can be shared.

Here is my repository :
https://github.com/FollyMaddy/retropie-translator

(beware : it can have mistakes)

For the adventurers :
If you want to try, Try it on your own risk !
Read the scripts, so you know what it does.
Don't directly expect too much of it.
It is all in very early stages and perhaps will never become a reality.
But I think it is important to share this idea.

I will not be able to always work on it.
But if I have some time I will add/improve some stuff.

If anyone has suggestions, please let me know.

mitu

RetroArch has a translation system built-in, I think you only need to bind one key/button to activate it for the current screen - https://www.libretro.com/index.php/upcoming-retroarch-1-7-8-ai-service-machine-translation-ocr-text-to-speech/.

See also https://retropie.org.uk/forum/topic/25614 .

Folly

@mitu said in Translate games on the fly:

RetroArch has a translation system built-in, I think you only need to bind one key/button to activate it for the current screen - https://www.libretro.com/index.php/upcoming-retroarch-1-7-8-ai-service-machine-translation-ocr-text-to-speech/.

See also https://retropie.org.uk/forum/topic/25614 .

Thanks for letting met know.
Actually I never knew that. Well that's really nice !
Will try out your suggestion.

Next time, before I start, I will have a better look in the forums. :-)

But, along the way I learned quite some stuff. ;-)

AdamBeGood

@folly Have you tried out their translation system yet? Interested to know what you think of it.

Clyde

@folly said in Translate games on the fly:

Next time, before I start, I will have a better look in the forums. :-)

But, along the way I learned quite some stuff. ;-)

That's what I like about hobbies like this – fun and learning in one go. 😁 Kudos for sharing your work with us. 👍

Folly

@adambegood said in Translate games on the fly:

@folly Have you tried out their translation system yet? Interested to know what you think of it.

I have to read a lot of stuff.

configs
translator links
accounts
API's
etc.

First impression looks like a real nightmare :(

Will take quite a lot of time to figure that out, I guess.

Folly

@clyde said in Translate games on the fly:

@folly said in Translate games on the fly:

Next time, before I start, I will have a better look in the forums. :-)

But, along the way I learned quite some stuff. ;-)

That's what I like about hobbies like this – fun and learning in one go. 😁 Kudos for sharing your work with us. 👍

Thanks for your Kudos.

I agree, we have to think in ideas.
That can brings us much further.

I will try some other stuff, with my idea.
On the long run, you could make quite a simple offline translator.

Another idea I have is :
If you have Japanese filenames, these can be translated with the translation-shell and then be renamed.
That's also very nice.

mitu

@folly said in Translate games on the fly:

First impression looks like a real nightmare :(

I think all I needed to add to retroarch.cfg was:

ai_service_enable = "true"
ai_service_mode = "0"
ai_service_pause = "true"
ai_service_source_lang = "0"
ai_service_target_lang = "1"
ai_service_url = "http://ztranslate.net/service?api_key=HEREISMYKEY"
input_ai_service = "p"

and then pressing P on the keyboard triggered the translation.

AdamBeGood

@mitu said in Translate games on the fly:

@folly said in Translate games on the fly:

First impression looks like a real nightmare :(

I think all I needed to add to retroarch.cfg was:
ai_service_enable = "true"
ai_service_mode = "0"
ai_service_pause = "true"
ai_service_source_lang = "0"
ai_service_target_lang = "1"
ai_service_url = "http://ztranslate.net/service?api_key=HEREISMYKEY"
input_ai_service = "p"
and then pressing P on the keyboard triggered the translation.

And that is the only step that needs to be followed? I'll definitely try this out then!

Folly

@mitu @AdamBeGood @Clyde

Tested it, and it works. :)
WOW. Thats awesome. Never thought this would be possible.

Still have somewhat a problem with my hotkey.
But I will test a bit more.

edit (issue) :

It seems the AI "p" key is in conflict with the "pause" function.
Athough it's commented in the retroarch.cfg like this :

# input_pause_toggle = p

When I remove the AI_..... lines, translation does not work, but the pause hotbutton does (while commented ?)

Anyone have an idea how this is possible ?
This config is edited :
/opt/retropie/configs/all/retroarch.cfg

edit (fix) :

Changed (input_ai_service = "p") into this (input_ai_service = "t")

No conficts anymore ! :-)

Now we use the "t" to "translate", isn't that convenient !

ai_service_enable = "true"
ai_service_mode = "0"
ai_service_pause = "true"
ai_service_source_lang = "0"
ai_service_target_lang = "1"
ai_service_url = "http://ztranslate.net/service?api_key=HEREISMYKEY"
input_ai_service = "t"

Folly

And now .....

A few pictures ! (Ys (pc88 version))

before and after :

Zering

@folly That is AWESOME.

Are there still limits on how much translating we can do?

Folly

@zering said in Translate games on the fly:

@folly That is AWESOME.

Are there still limits on how much translating we can do?

It still seems to work fine. No limits encountered yet.

With PC98 I had a few crashes though, when I was in DOS.
But I use the "tab" key for enabeling the hotkeys.
That key also responds as keyboard input in some emulators.
Perhaps it conflicts a bit.
Don't actually know.

Overall it seems to work great :)

Are you going to try this ?

Zering

@folly Eventually, yes, but as I've said on a previous thread I've had a disk failure so I'm focusing on that (I hate myself for having 2 TB of Roms...).
At the moment I don't have any untranslated games I can play ^^

Folly

@zering said in Translate games on the fly:

@folly Eventually, yes, but as I've said on a previous thread I've had a disk failure so I'm focusing on that (I hate myself for having 2 TB of Roms...).
At the moment I don't have any untranslated games I can play ^^

Yes, I am aware of that. Good luck with that.
If your ready, let me know what you think of it.
Would like to hear your opinion.

Zering

@folly I'll have to find an untranslated game that I don't have too many expectations for, so I can gauge the quality of the translation and make sure I can actually get the gist of things ;)

Folly

@zering said in Translate games on the fly:

@folly I'll have to find an untranslated game that I don't have too many expectations for, so I can gauge the quality of the translation and make sure I can actually get the gist of things ;)

Well, It already makes a big difference for menu's.
That I know.

AdamBeGood

Mine doesn't seem to work - I've tried putting it into the /all/ retroarch.cfg, and then I tried the pc98 and pc88 ones as well, just in case that works.

It is literally just putting the text into that file isn't it? Or are there other steps also?

Here is my retroarch.cfg:

https://pastebin.com/1VzG5U4w

saccublenda

@adambegood I think you have to replace HEREISMYKEY with the actual key that you get when you register to the website.

AdamBeGood

@saccublenda said in Translate games on the fly:

@adambegood I think you have to replace HEREISMYKEY with the actual key that you get when you register to the website.

Great shout! I've amended that now, and still no joy. I am focusing on PC98 so far as that is where I have a few games I'd like to try. I tried putting the code into /pc98/retroarch.cfg also, but that didn't do it.

Here's my PC98 config: https://pastebin.com/GNisWHPH

I am pressing the keyboard hotkey and the 't' key.

Edit: let me have a look at this in the morning, I have had a read around the issue now and hopefully can try some things out.