Help adding a library to ES build

[sselph]

Hi,

I've noticed a lot of people experimenting working on ES so thought I would ask. I want to compile my scraper in to a C library and integrate it in to ES. When I compile, I get a C archive or shared library and a header file but I'm not sure how to add this in to ES's build. If anyone has some pointers that would be great. I know very little about building C++.

[grahamken]

@sselph Huge fan of your Scraper mate and I know this is off topic, but...where's the love for the ZX Spectrum?

[sselph]

So I looked a little more in to what I need to do:

To actually build my code I have to run commands like:
go get -u github.com/sselph/scraper/libscraper
go build -buildmode=c-shared -o libscraper.a github.com/sselph/scraper/libscraper

That downloads and builds a C shared library and a header file. It assumes that Go is installed and it is at least version 1.5. It looks like CMake has find_program and execute_process that could be used to test if go exists that go version has an expected output. Once those commands are run then something like include_directories and link_directories could be used to include the header and shared library.

Let me know if this is on the right track. I'm going to sleep and will hopefully have time to test some of this tomorrow evening or next week.

[herb_fargus]

Just posting to tag @fieldofcows just in case he didn't see this post as he may have the best idea of integrating it if he's up to it

[sselph]

I think I might be trying to be too fancy. I think I could just make my own make file that runs the correct build command then copies the files to /usr/lib and /usr/include and use the find_package to check for it.

[fieldofcows]

Thanks @herb_fargus - I hadn't seen this post.

I'll certainly help where I can although ideally I guess libscraper would be built into some sort of repository and ES would just be updated to include the headers and link against it (e.g. run apt-get install libscraper-dev before building ES).

I'm not sure how the retropie ES is built and whether there is an opportunity to build dependencies of ES from source before building ES.

But yes, you are correct. You should have your own makefile (or cmake or whatever build system you want to use) that produces the header and libraries and installs them into /usr/lib and /usr/include. The ES CMakeLists.txt would then be updated with a find_package to locate your library and will add the include path and link libraries.

If you want, I'll try building your library as per your instructions then I can make changes to the ES build files and show you how to do this.

[sselph]

@fieldofcows Thanks. I think going down the wrong path a few times has taught me enough about cmake to get it working. Once I have some working code, I'd love to have you look at the pull request to see if it looks okay. But I probably won't have time to work on it again until this weekend.

[fluffypillow]

ES uses CMake, so it's not too hard to add a new lib into the build. Integrating it into the source code will likely cause more troubles, I think. I've never used Go, so feel free to correct these steps; here's the general process:

• After cloning ES, create a new directory under external, and add it to the build system in the external/CMakeLists.txt file
• Create a CMakeLists.txtfile in your new directory, something like this:

project(sselph_scraper)

execute_process(COMMAND
    go get -u github.com/sselph/scraper/libscraper)
execute_process(COMMAND
    go build -buildmode=c-shared -o libscraper.a github.com/sselph/scraper/libscraper)

Of course, there's a LOT of stuff you can do here, this is just a very basic CMake file based on your commands. You'll probably want to do error checking, proper paths, cross compile support, and such. Also I don't know where will go get and go build place its files -- you'll have to tell that to CMake too if they're in the wrong place. You will also have to tell where are the C or C++ headers, with eg. include_directories, or target_include_directories. Variables like CMAKE_BUILD_DIR or PROJECT_BUILD_DIR can help to set proper paths.

• Since the archive is created by Go, I think you'll need to link that directly; like this, in the es-app/CMakeLists.txt:
  target_link_libraries(es-app path/to/yourlib.a)
  or set the the library path, and just
  target_link_libraries(es-app yourlib)

• After that, you'll have to actually integrate it into ES, you'll probably have to create a wrapper class to match the scraper interface, like the current scrapers underes-app/src/scrapers/.

[sselph]

Thanks. I went ahead and copied my library and header to the lib and include folders and wrote a Find file that picks them up and it appears to work. This will at least let me start testing the interaction between the ES and my code.

I actually already wrote Go code to start a local web server that when passed a file path would return the information ES wants in XML. I also had the code for ES to query and parse that XML. So the first pass at this is just replacing the web handler with an exported C function that returns the same XML. If that works, I'll start trying to optimize it and make it more asynchronous.

[sselph]

An update: I had time to play with the code this morning. It compiled and linked and I'm currently scraping my roms. It is slow but probably no slower than if you use a single thread and don't pass -thumb_only with my scraper standalone. Since it is downloading the full image then resizing and bandwidth is my bottle neck right now.

The code needs a ton of clean up and review from people who know how to write C but looks promising.