diff --git a/wiki-rtd/docs/wiki_development/general/add-system.md b/wiki-rtd/docs/wiki_development/general/add-system.md index 47a0929..e6de4a7 100644 --- a/wiki-rtd/docs/wiki_development/general/add-system.md +++ b/wiki-rtd/docs/wiki_development/general/add-system.md @@ -18,25 +18,18 @@ How well do you know the emulator you are looking to add to RetroDECK. * What libraries and assets does it need. * Does it need bios files * Controller support - * Command line arguments for redirecting configs or saves etc. + * Command line arguments for redirecting configs or saves etc ## Tasks Involved ### Clone RetroDeck -Fork and clone the main [RetroDeck project.](https://github.com/XargonWan/RetroDECK/fork) - -Adding an emulator would be classed as a new feature so create a branch based of the label feat - -ie feat/new_emulator_name - -An example for ruffle can be seen [here.](https://github.com/monkeyx-net/RetroDECK_UK/tree/feat/ruffle) - -Initial testing can be done via the [debug mode](/wiki_development/general/debug-mode/#retrodeck-in-debug-mode) - -You will be able to manually run via emulator and test functionality. - -To get the emulator added to the manifest an example of ruffle is shown below as simple example manifest. +- Fork and clone the main [RetroDeck project.](https://github.com/XargonWan/RetroDECK/fork) +- Adding an emulator would be classed as a **new feature** so create a branch based of the label feat, ie: `feat/new_emulator_name` +- An example for **ruffle** can be seen [here](https://github.com/monkeyx-net/RetroDECK_UK/tree/feat/ruffle) +- Initial testing can be done via the [debug mode](/wiki_development/general/debug-mode/#retrodeck-in-debug-mode) +- You will be able to manually run via emulator and test functionality +- To get the emulator added to the manifest an example of ruffle is shown below as simple example manifest ### Editing the manifest file @@ -46,86 +39,31 @@ To get the emulator added to the manifest an example of ruffle is shown below as - name: ruffle buildsystem: simple build-commands: - - mkdir -p "${FLATPAK_DEST}/share/ruffle" - - tar -zxvf *.tar.gz -C "${FLATPAK_DEST}/share/ruffle" - chmod +x "${FLATPAK_DEST}/share/ruffle/ruffle" - ln -s ${FLATPAK_DEST}/share/ruffle/ruffle ${FLATPAK_DEST}/bin/ruffle sources: - - type: file - url: RUFFLEURLPLACEHOLDER - sha256: RUFFLESHAPLACEHOLDER + - type: archive + url: https://github.com/ruffle-rs/ruffle/releases/download/nightly-2024-07-02/ruffle-nightly-2024_07_02-linux-x86_64.tar.gz + sha256: a410ce8956723a0043d1744ef9b519debc978faa2b5868953acc548e44586d15 # Ruffle - END ``` - * Please note lines 1 and 15 as remarks/labels having these help with logging and fund the emulator within the file. - * Line 3 is a label - * Lines 4 indicates a simple build type ie downloading and copying files rather than compiling and building the project. The preference is to compile and build emulators to support the Flatpak process in producing increased compatibly and customisation. Ruffle is a rust project and under constant development. It is hoped to make this a compiled project in the future. - * A simple project is also a good starting point to flatpak builds. - * Lines5 to 9 shows the build process kinked to the downloaded ruffle file. - * Line 10 sources has some place holders url: and sha256: See below + - Please note lines 1 and 15 as remarks/labels having these help with logging and fund the emulator within the file. + - Line 3 is a label + - Lines 4 indicates a simple build type, the simple build type is just executing the given `build-commands`, ie downloading and copying files rather than compiling and building the project. The preference is to compile and build emulators to support the Flatpak process in producing increased compatibly and customisation. Ruffle is a rust project and under constant development. It is hoped to make this a compiled project in the future. + - A simple project is also a good starting point to Flatpak builds. + - Lines 5 to 9 shows the build process kinked to the downloaded ruffle file. + - Lines 11-13 are defining the sources: + - type: `archive`, this means that the archive is download and extracted without any explicit manifest action, `file` type source instead would just download the file without extracting it. + - url: obviously the url to download + - sha256: the sha256sum of the file, this is mandatory -### Add/Change Placeholders +> **NOTE:** In some rare caes we might don't know the sha (or even the url) in advance, for example when an emulator is having only one release on GitHub that is being kept updated without a release history. +[This guide](wiki-rtd/docs/wiki_development/general/automate-emulator-with-placeholders.md) is explaining how to use the placeholders instead of urls or sha in case the this data cannot be forseen. +Fort his purpose we have implemented an automation to get that data from the repository itself to being used only as a final resource as we prefer to control the version of each module for proper troubleshooting and version control. -The placeholders are referenced in the file automation_tools/automation_task_list.cfg - -``` bash linenums="1" -# The proper format for this file is -# ACTION^PLACEHOLDERTEXT^URL^REPO(Optional) -# hash^DOOMSHAPLACEHOLDER^https://buildbot.libretro.com/assets/cores/DOOM/Doom%20%28Shareware%29.zip -hash^DUCKSTATIONSHAPLACEHOLDER^https://github.com/stenzek/duckstation/releases/download/preview/DuckStation-x64.AppImage -hash^SAMEDUCKSHAPLACEHOLDER^https://buildbot.libretro.com/nightly/linux/x86_64/latest/sameduck_libretro.so.zip -hash^PPSSPPBIOSHASHPLACEHOLDER^https://github.com/hrydgard/ppsspp/archive/refs/heads/master.zip -hash^MSXBIOSHASHPLACEHOLDER^http://bluemsx.msxblue.com/rel_download/blueMSXv282full.zip -hash^XEMUHDDHASHPLACEHOLDER^https://github.com/mborgerson/xemu-hdd-image/releases/latest/download/xbox_hdd.qcow2.zip -hash^VITA3KSHAPLACEHOLDER^https://github.com/Vita3K/Vita3K/releases/download/continuous/ubuntu-latest.zip -hash^RANIGHTLYCORESPLACEHOLDER^https://buildbot.libretro.com/nightly/linux/x86_64/RetroArch_cores.7z -url^RUFFLEURLPLACEHOLDER^"https://github.com/ruffle-rs/ruffle/releases/download/$(git ls-remote --tags https://github.com/ruffle-rs/ruffle.git | tail -n 1 | cut -f2 | sed 's|refs/tags/||')/ruffle-$(git ls-remote https://github.com/ruffle-rs/ruffle.git | tail -n 1 | cut -f2 | sed -E 's|refs/tags/||; s/-/_/2; s/-/_/2')-linux-x86_64.tar.gz" -hash^RUFFLESHAPLACEHOLDER^"https://github.com/ruffle-rs/ruffle/releases/download/$(git ls-remote --tags https://github.com/ruffle-rs/ruffle.git | tail -n 1 | cut -f2 | sed 's|refs/tags/||')/ruffle-$(git ls-remote https://github.com/ruffle-rs/ruffle.git | tail -n 1 | cut -f2 | sed -E 's|refs/tags/||; s/-/_/2; s/-/_/2')-linux-x86_64.tar.gz" -hash^RETRODECKMAMEPLACEHOLDER^"https://github.com/XargonWan/RetroDECK-MAME/releases/download/$(curl -s https://api.github.com/repos/XargonWan/RetroDECK-MAME/releases/latest | grep -oP '"tag_name": "\K(.*?)(?=")')/RetroDECK-MAME-Artifact.tar.gz" -url^RETRODECKMAMEURLPLACEHOLDER^"https://github.com/XargonWan/RetroDECK-MAME/releases/download/$(curl -s https://api.github.com/repos/XargonWan/RetroDECK-MAME/releases/latest | grep -oP '"tag_name": "\K(.*?)(?=")')/RetroDECK-MAME-Artifact.tar.gz" -latestcommit^UNIVERSALDYNAMICINPUTCOMMITPLACEHOLDER^https://github.com/Venomalia/UniversalDynamicInput^main -outside_file^VERSIONPLACEHOLDER^${GITHUB_WORKSPACE}/buildid -branch^THISBRANCH -repo^RDREPO -``` - -The example above shows the relevant Ruffle entries at lines line 11 and 12. - - * Line 11 creates the URL based on date/time. - * Line 12 created a sha256 hash. The Flatpak build gave a sha error build without passing a sha256 hash. - -The automation_tools/automation_task_list.cfg is processed by the automation_tools/pre_build_automation.sh script and replaces the placeholders with the calculated values. - -You can edit and run the developer_toolbox/build_retrodeck_locally.sh below to check how it changes the manifest file. By using the file below and comment all .sh scripts apart from line 13. It restores the default manifest file after the test run. - -```bash linenums="1" -#!/bin/bash - -# WARNING: run this script from the project root folder, not from here!! - -git submodule update --init --recursive - -export GITHUB_WORKSPACE="." -cp net.retrodeck.retrodeck.appdata.xml net.retrodeck.retrodeck.appdata.xml.bak -cp net.retrodeck.retrodeck.yml net.retrodeck.retrodeck.yml.bak - -#automation_tools/install_dependencies.sh -#automation_tools/cooker_build_id.sh -automation_tools/pre_build_automation.sh -#automation_tools/cooker_flatpak_portal_add.sh -#automation_tools/appdata_management.sh -#automation_tools/flatpak_build_download_only.sh -#automation_tools/flatpak_build_only.sh -#automation_tools/flatpak_build_bundle.sh - -rm -f net.retrodeck.retrodeck.appdata.xml -rm -f net.retrodeck.retrodeck.yml -cp net.retrodeck.retrodeck.appdata.xml.bak net.retrodeck.retrodeck.appdata.xml -cp net.retrodeck.retrodeck.yml.bak net.retrodeck.retrodeck.yml -``` - -### Build the RetroDECK flatpak +### Build the RetroDECK Flatpak If you have added the necessary files to the manifest @@ -143,21 +81,23 @@ If the build completes without errors it should produce an artifact that you can ## Backup your key data and configs -Make a copy of ~/.var/app/net.retrodeck.retrodeck/ folder and name it old.net.retrodeck.retrodeck -Make a full back up or partial backups of the retrodeck folder normally under ~/retrodeck or sdcard/other drive. +Make a copy of `~/.var/app/net.retrodeck.retrodeck/` folder and name it `old.net.retrodeck.retrodeck` +Make a full back up or partial backups of the retrodeck folder normally under `~/retrodeck` or sdcard/other drive. On retrodeck folder backups: Generally, very few things would target the roms folder, but the other folders could be targeted for various scripts. Our recommendation would be to back up the full ~/retrodeck folder, but as a tester you can decide how much you want to risk. -## Install and Test the built flatpak +## Install and Test the built Flatpak -Remove current version of RetroDECK +Remove all the installed versions of RetroDECK, to be sure uninstall both in the system and userland: -`flatpak uninstall net.retrodeck.retrodeck` and press (y) yes to remove RetroDECK. If you have more then one version installed for some reason choose to remove all versions. +`flatpak uninstall --user net.retrodeck.retrodeck` and press (y) yes to remove RetroDECK. If you have more then one version installed for some reason choose to remove all versions. + +`flatpak uninstall --system net.retrodeck.retrodeck` and press (y) yes to remove RetroDECK. If you have more then one version installed for some reason choose to remove all versions. Download and unzip the artifact to a folder of your choice and then run this command in the same folder and the unarchived file. -`flatpak install --user --bundle --noninteractive -y RetroDECK-cooker.flatpak` +`flatpak install --user --bundle --noninteractive -y RetroDECK-cooker.Flatpak` On first run it will ask you to upgrade diff --git a/wiki-rtd/docs/wiki_development/general/automate-emulator-with-placeholders.md b/wiki-rtd/docs/wiki_development/general/automate-emulator-with-placeholders.md new file mode 100644 index 0000000..8fc8382 --- /dev/null +++ b/wiki-rtd/docs/wiki_development/general/automate-emulator-with-placeholders.md @@ -0,0 +1,67 @@ +# WIP + +# Automate emulator updates with placeholders +This guide is explaining how to use the placeholders in the situation where the emulator release cannot be forseen, for example if an emulator is having only one release on GitHub that is being kept updated without a release history. +In this case we might don't know the final filename nor the sha256. + +Fort his purpose we have implemented an automation to get that data from the repository itself. + +### Add/Change Placeholders + +The placeholders are referenced in the file automation_tools/automation_task_list.cfg + +``` bash linenums="1" +# The proper format for this file is +# ACTION^PLACEHOLDERTEXT^URL^REPO(Optional) +# hash^DOOMSHAPLACEHOLDER^https://buildbot.libretro.com/assets/cores/DOOM/Doom%20%28Shareware%29.zip +hash^DUCKSTATIONSHAPLACEHOLDER^https://github.com/stenzek/duckstation/releases/download/preview/DuckStation-x64.AppImage +hash^SAMEDUCKSHAPLACEHOLDER^https://buildbot.libretro.com/nightly/linux/x86_64/latest/sameduck_libretro.so.zip +hash^PPSSPPBIOSHASHPLACEHOLDER^https://github.com/hrydgard/ppsspp/archive/refs/heads/master.zip +hash^MSXBIOSHASHPLACEHOLDER^http://bluemsx.msxblue.com/rel_download/blueMSXv282full.zip +hash^XEMUHDDHASHPLACEHOLDER^https://github.com/mborgerson/xemu-hdd-image/releases/latest/download/xbox_hdd.qcow2.zip +hash^VITA3KSHAPLACEHOLDER^https://github.com/Vita3K/Vita3K/releases/download/continuous/ubuntu-latest.zip +hash^RANIGHTLYCORESPLACEHOLDER^https://buildbot.libretro.com/nightly/linux/x86_64/RetroArch_cores.7z +url^RUFFLEURLPLACEHOLDER^"https://github.com/ruffle-rs/ruffle/releases/download/$(git ls-remote --tags https://github.com/ruffle-rs/ruffle.git | tail -n 1 | cut -f2 | sed 's|refs/tags/||')/ruffle-$(git ls-remote https://github.com/ruffle-rs/ruffle.git | tail -n 1 | cut -f2 | sed -E 's|refs/tags/||; s/-/_/2; s/-/_/2')-linux-x86_64.tar.gz" +hash^RUFFLESHAPLACEHOLDER^"https://github.com/ruffle-rs/ruffle/releases/download/$(git ls-remote --tags https://github.com/ruffle-rs/ruffle.git | tail -n 1 | cut -f2 | sed 's|refs/tags/||')/ruffle-$(git ls-remote https://github.com/ruffle-rs/ruffle.git | tail -n 1 | cut -f2 | sed -E 's|refs/tags/||; s/-/_/2; s/-/_/2')-linux-x86_64.tar.gz" +hash^RETRODECKMAMEPLACEHOLDER^"https://github.com/XargonWan/RetroDECK-MAME/releases/download/$(curl -s https://api.github.com/repos/XargonWan/RetroDECK-MAME/releases/latest | grep -oP '"tag_name": "\K(.*?)(?=")')/RetroDECK-MAME-Artifact.tar.gz" +url^RETRODECKMAMEURLPLACEHOLDER^"https://github.com/XargonWan/RetroDECK-MAME/releases/download/$(curl -s https://api.github.com/repos/XargonWan/RetroDECK-MAME/releases/latest | grep -oP '"tag_name": "\K(.*?)(?=")')/RetroDECK-MAME-Artifact.tar.gz" +latestcommit^UNIVERSALDYNAMICINPUTCOMMITPLACEHOLDER^https://github.com/Venomalia/UniversalDynamicInput^main +outside_file^VERSIONPLACEHOLDER^${GITHUB_WORKSPACE}/buildid +branch^THISBRANCH +repo^RDREPO +``` + +The example above shows the relevant Ruffle entries at lines line 11 and 12. + + * Line 11 creates the URL based on date/time. + * Line 12 created a sha256 hash. The Flatpak build gave a sha error build without passing a sha256 hash. + +The automation_tools/automation_task_list.cfg is processed by the automation_tools/pre_build_automation.sh script and replaces the placeholders with the calculated values. + +You can edit and run the developer_toolbox/build_retrodeck_locally.sh below to check how it changes the manifest file. By using the file below and comment all .sh scripts apart from line 13. It restores the default manifest file after the test run. + +```bash linenums="1" +#!/bin/bash + +# WARNING: run this script from the project root folder, not from here!! + +git submodule update --init --recursive + +export GITHUB_WORKSPACE="." +cp net.retrodeck.retrodeck.appdata.xml net.retrodeck.retrodeck.appdata.xml.bak +cp net.retrodeck.retrodeck.yml net.retrodeck.retrodeck.yml.bak + +#automation_tools/install_dependencies.sh +#automation_tools/cooker_build_id.sh +automation_tools/pre_build_automation.sh +#automation_tools/cooker_flatpak_portal_add.sh +#automation_tools/appdata_management.sh +#automation_tools/flatpak_build_download_only.sh +#automation_tools/flatpak_build_only.sh +#automation_tools/flatpak_build_bundle.sh + +rm -f net.retrodeck.retrodeck.appdata.xml +rm -f net.retrodeck.retrodeck.yml +cp net.retrodeck.retrodeck.appdata.xml.bak net.retrodeck.retrodeck.appdata.xml +cp net.retrodeck.retrodeck.yml.bak net.retrodeck.retrodeck.yml +``` \ No newline at end of file