I think the default should be "No Metadata" for scraping. When you first start out, all games have no metadata anyway. The "All Games" option should only be used if you want to rescrape. I did that my first time because I had my preferred region wrong.
Appreciate the efforts, the scrapers and things, but is it possible to just load existing covers and screenshots from the folder the game binary is in? I don't want to scrape, I already have all the artwork and in-game-movies to be used at hand. I also don't want to scrape things again and again. Whenever you use or import your game collection in another emulator or "menu system", things need to be rescraped from scratch, what a bad idea. Isn't there a standard for game-metadata now? Something that describes a game in all its details and with local/external/relative links to required files? A standardized game-info.xml or json or something, which can be used by as many different emulators out there? Scraping is good and helpful in the beginning, no question, but if your collection is rather large, you don't want to re-scrape, it takes ages and it may even overwrite details you put together manually (backgrounds, move files etc.). We really need a metadata handling standard for this. If you load your mp3 collection into any player, it is also capable of pulling tags from the file to show artist and song name, even get cover art from the file. This should work for game binaries as well. Create a sidecar archive maybe where all necessary information is available and NEXT to the game binary, not on the other end of your operating system or file server. Maybe some will agree?! Have a nice day! o)
Appreciate your feedback! I can hope its understandable that creating a consistent and shared metadata format across all the available frontends would be fairly complex (and coordinating changes on that shared structure would also add complexity that I am not sure all frontends would want to adopt). That said, we would be glad to contribute to that conversation for sure. For now though, it is possible to use existing artwork and metadata that has been scraped for a different frontend. You just need to move it into a directory structure ES-DE expects. For gamelist.xml files (which store metadata for games): - as long as they utilize the EmulationStation format that was defined by the original ES they will work with ES-DE - I know offhand that Retropie and Batocera use the gamelist.xml format and have tested using those files as is in ES-DE - to get gamelist.xml files to show up in ES-DE simply move them into the ES-DE/gamelists/{system} folder that they relate to - for example for genesis you would move the gamelist.xml into ES-DE/gamelists/genesis For media files (i.e. the artwork for each game) - You would just need to move those into into the ES-DE/downloaded_media/{system} folder that they relate to - And in that folder follow the instructions laid out here: gitlab.com/es-de/emulationstation-de/-/blob/master/USERGUIDE.md#manually-copying-game-media-files In short, it is possible to reuse your existing files using the notes above. And if you have questions on that process we'll be glad to help. You may ask "why does ES-DE place the gamelists and downloaded_media separately from games"? - First reason is portability of the files between different platforms that ES-DE is supported on (i.e. you can easily use/sync the same gamelists and downloaded media across PC, MacOS, Linux and Android) - Second reason is we believe in a separation of concerns when it comes to your games and metadata/media files. The prime example of this is if you wanted to use another frontend in the future, we do not want to pollute your games directory with files that ES-DE looks for (we feel thats not a good practice) Hope that helps at least explain our perspective and thank you for checking this out.
@@ES-DE_Frontend Kodi / Jellyfin have solved this for Movie/Show libraries, using xml ".nfo" files. But sadly no gaming library frontend has attempted this yet. Online scrappers will always have mistakes, they can't account for all new fanmade patched versions of ROMs, obscure/recent Doom pwads made by fans, or rare/custom/recent DOS games. Not to mention you need to be "online" for that to work, which isn't something all devices might be able to do. Syncing a monolithic file makes the games less portable, because you'd need to sync the entire gamelist.xml of each platform from your desktop to your smaller device, where you might not want to have the same games. You can't simply pick and choose games to copy over. And if you ever manually change the name of a file from your file browser, you have to edit a huge gamelist.xml and try to find out which file was changed (or worse, regenerate/rescrap the game and edit it yet again), instead of simply renaming the corresponding nfo file next to it. IMHO, scrapping from online sources should be the "fallback" behavior when no nfo file with metadata is found locally. Having "nfo" files next to the game can be optional thing, no "polluting", just use it as a READ-ONLY source of metadata. The frontend doesn't need to write the nfo file, but whenever the file is already present, use it; otherwise just scrap online. This way you can even decide to only add a ".nfo" file for specific games that are hard to detect. The frontend could then have the path of the "nfo" added to the gamelist.xml so it can be dynamically imported and perhaps override some values if writting a change of it is necessary. The only reason people have been OK with the current state of things is because most only care about emulating static sets of roms from old platforms that don't evolve as quickly as other forms of media.. but the minute you start thinking about platforms with a lot of obscure new indie content every month, it starts falling apart. Already with DOOM and DOSBox systems in ES-DE you can start to see problems.
@@ES-DE_Frontend The auto scraper either just skips all my ISOs or flat out pulls the wrong data in. How do I manually target data for a game using the scrapper? Like instead of having the scraper figure it out and pull it in, how do I tell it what game I have and pull in data for that game?
hello, im hoping someone might be able to offer some guidance. i am having trouble getting ES-DE to scrape using either source, whether im signed in to screenscraper or not. it stays "working" for upwards of 10 minutes, then it has a network timeout. for the record, it does the same thing when trying to download themes. ive done everything i can think of from moving files, to scraping games individually, and trying it on different networks.
Sorry I don't have a 353v to compare. The device in the video is a Retroid Pocket 4 Pro with the following specs: www.goretroid.com/cdn/shop/files/1_e1658dfb-4505-4926-8d98-8d978d3ad4c5_1024x1024@2x.jpg?v=1702977514
One thing i would find useful (but couldn't find) - would be a way to log games that it searched for during scraping but came back with nothing. I have a lot of roms which are not technically games e.g ."Acorn Electron" (mame rom "electron") and it keeps attempting to re-scrape these, using up my scraping request count. If i could get a list of these failures i could write a little python script to fake the meta data for them and therefore let the scraper skip them on the next scraping run
for those types of file i wonder if it would be ok to use the hide option? if you dont need those files to display in your gamelists they can be hidden and then scraper will ignore them (they will still be in the directory so they can be used by emulators). what do you think?
@@ES-DE_Frontend its more a problem with scraping - lets say that the quota is 1000scrapes per day and i have 100,000 roms to scrape. Scrape Run 1 (No meta data filter option): 1000 games scraped, 800 successfully, 200 not found - quota reached Scrape Run 2 (No meta data filter option): 1000 games scraped, 600 successfully, 200 not found from scrape run 1 (still no meta data), 200 newly not found games - quota reached Scrape Run 3 (No meta data filter option): 1000 games scraped, 400 successfully, 200 failed from day1, 200 failed from day 2, 200 failed from day 3 Because the failures don't update the meta data (as they shouldn't) - it means, you eventually only scrape roms that have no meta data available and you constantly run out of quota by scrape run 5 i will have scraped about 5000 roms but will be fail on 1000 roms - these failures will constantly hit the quota limit and i will never be able to scrape past these initial 5000 and roms 5000-100,000 will never be scraped I can't tell, easily which roms are going to fail. The Acorn Electron was an example i could easily identify - but there were lots of others that are not so obvious
Hola increíble me encanta muchas opciones justo me preguntaba si tengo el sistema en la pc y en android puedo pasar lo que descargue en pc de imágenes y vídeos para pasarlo a android sin tener que hacerlo de nuevo gracias
we will be looking at how multilanguage support in the future. we will need translation help as well so if you are interested please feel free to join our discord discord.gg/JPE9ZT6b
Are you serious using a open source project and make pay 5€ a month to use the android version ??? ( if it was at least one payment ) but a subscription to use a frontend in android ????thats so scummy man !!!!!!i guess im gonna use retroarch alone !!! and you lose a customer that would gladly pay 5€ for the frontend in google play or sideload !
for clarification ES-DE on Android is a one time purchase and the other versions (Windows, MacOS and Linux) are all free. For the android version you can choose to stay a long time subscriber if you want but its not required, just a one time purchase is all thats asked - you will get updates for life from that one time purchase. Hope that helps.
I think the default should be "No Metadata" for scraping. When you first start out, all games have no metadata anyway. The "All Games" option should only be used if you want to rescrape. I did that my first time because I had my preferred region wrong.
thank you for your suggestion. will consider it!
Did you make this front end yourself if you did it is amazing and you did a great job thankyou
Appreciate the efforts, the scrapers and things, but is it possible to just load existing covers and screenshots from the folder the game binary is in?
I don't want to scrape, I already have all the artwork and in-game-movies to be used at hand. I also don't want to scrape things again and again.
Whenever you use or import your game collection in another emulator or "menu system", things need to be rescraped from scratch, what a bad idea. Isn't there a standard for game-metadata now? Something that describes a game in all its details and with local/external/relative links to required files? A standardized game-info.xml or json or something, which can be used by as many different emulators out there?
Scraping is good and helpful in the beginning, no question, but if your collection is rather large, you don't want to re-scrape, it takes ages and it may even overwrite details you put together manually (backgrounds, move files etc.). We really need a metadata handling standard for this.
If you load your mp3 collection into any player, it is also capable of pulling tags from the file to show artist and song name, even get cover art from the file. This should work for game binaries as well. Create a sidecar archive maybe where all necessary information is available and NEXT to the game binary, not on the other end of your operating system or file server.
Maybe some will agree?! Have a nice day! o)
Appreciate your feedback! I can hope its understandable that creating a consistent and shared metadata format across all the available frontends would be fairly complex (and coordinating changes on that shared structure would also add complexity that I am not sure all frontends would want to adopt). That said, we would be glad to contribute to that conversation for sure.
For now though, it is possible to use existing artwork and metadata that has been scraped for a different frontend. You just need to move it into a directory structure ES-DE expects.
For gamelist.xml files (which store metadata for games):
- as long as they utilize the EmulationStation format that was defined by the original ES they will work with ES-DE
- I know offhand that Retropie and Batocera use the gamelist.xml format and have tested using those files as is in ES-DE
- to get gamelist.xml files to show up in ES-DE simply move them into the ES-DE/gamelists/{system} folder that they relate to
- for example for genesis you would move the gamelist.xml into ES-DE/gamelists/genesis
For media files (i.e. the artwork for each game)
- You would just need to move those into into the ES-DE/downloaded_media/{system} folder that they relate to
- And in that folder follow the instructions laid out here: gitlab.com/es-de/emulationstation-de/-/blob/master/USERGUIDE.md#manually-copying-game-media-files
In short, it is possible to reuse your existing files using the notes above. And if you have questions on that process we'll be glad to help.
You may ask "why does ES-DE place the gamelists and downloaded_media separately from games"?
- First reason is portability of the files between different platforms that ES-DE is supported on (i.e. you can easily use/sync the same gamelists and downloaded media across PC, MacOS, Linux and Android)
- Second reason is we believe in a separation of concerns when it comes to your games and metadata/media files. The prime example of this is if you wanted to use another frontend in the future, we do not want to pollute your games directory with files that ES-DE looks for (we feel thats not a good practice)
Hope that helps at least explain our perspective and thank you for checking this out.
@@ES-DE_Frontend Kodi / Jellyfin have solved this for Movie/Show libraries, using xml ".nfo" files. But sadly no gaming library frontend has attempted this yet.
Online scrappers will always have mistakes, they can't account for all new fanmade patched versions of ROMs, obscure/recent Doom pwads made by fans, or rare/custom/recent DOS games. Not to mention you need to be "online" for that to work, which isn't something all devices might be able to do.
Syncing a monolithic file makes the games less portable, because you'd need to sync the entire gamelist.xml of each platform from your desktop to your smaller device, where you might not want to have the same games. You can't simply pick and choose games to copy over.
And if you ever manually change the name of a file from your file browser, you have to edit a huge gamelist.xml and try to find out which file was changed (or worse, regenerate/rescrap the game and edit it yet again), instead of simply renaming the corresponding nfo file next to it.
IMHO, scrapping from online sources should be the "fallback" behavior when no nfo file with metadata is found locally. Having "nfo" files next to the game can be optional thing, no "polluting", just use it as a READ-ONLY source of metadata. The frontend doesn't need to write the nfo file, but whenever the file is already present, use it; otherwise just scrap online. This way you can even decide to only add a ".nfo" file for specific games that are hard to detect. The frontend could then have the path of the "nfo" added to the gamelist.xml so it can be dynamically imported and perhaps override some values if writting a change of it is necessary.
The only reason people have been OK with the current state of things is because most only care about emulating static sets of roms from old platforms that don't evolve as quickly as other forms of media.. but the minute you start thinking about platforms with a lot of obscure new indie content every month, it starts falling apart. Already with DOOM and DOSBox systems in ES-DE you can start to see problems.
@@ES-DE_Frontend The auto scraper either just skips all my ISOs or flat out pulls the wrong data in. How do I manually target data for a game using the scrapper? Like instead of having the scraper figure it out and pull it in, how do I tell it what game I have and pull in data for that game?
@@ES-DE_Frontend I kept watching and I found my answer. Thank you so much. Saved me.
hello, im hoping someone might be able to offer some guidance.
i am having trouble getting ES-DE to scrape using either source, whether im signed in to screenscraper or not. it stays "working" for upwards of 10 minutes, then it has a network timeout.
for the record, it does the same thing when trying to download themes.
ive done everything i can think of from moving files, to scraping games individually, and trying it on different networks.
hmm that sounds like you have an overall network issue on the device. if both theme download and scraper are not working
Why mouse or touch screen screen support, trying to navigate these menus are very time consuming with the D-Pad.
answer is here: gitlab.com/es-de/emulationstation-de/-/blob/master/FAQ-ANDROID.md?ref_type=heads#will-touch-gesture-support-get-added
I own the rg353v I really like how this one looks. Is the performance between this one and the 353v the same?
Sorry I don't have a 353v to compare. The device in the video is a Retroid Pocket 4 Pro with the following specs: www.goretroid.com/cdn/shop/files/1_e1658dfb-4505-4926-8d98-8d978d3ad4c5_1024x1024@2x.jpg?v=1702977514
Can I ask what theme you’re using and what theme config?
its a theme called “colorful (simplified)” using the “list: metadata & boxart” variant
Fantastic, thanks, just found it
One thing i would find useful (but couldn't find) - would be a way to log games that it searched for during scraping but came back with nothing. I have a lot of roms which are not technically games e.g ."Acorn Electron" (mame rom "electron") and it keeps attempting to re-scrape these, using up my scraping request count. If i could get a list of these failures i could write a little python script to fake the meta data for them and therefore let the scraper skip them on the next scraping run
for those types of file i wonder if it would be ok to use the hide option? if you dont need those files to display in your gamelists they can be hidden and then scraper will ignore them (they will still be in the directory so they can be used by emulators). what do you think?
@@ES-DE_Frontend its more a problem with scraping - lets say that the quota is 1000scrapes per day and i have 100,000 roms to scrape.
Scrape Run 1 (No meta data filter option): 1000 games scraped, 800 successfully, 200 not found - quota reached
Scrape Run 2 (No meta data filter option): 1000 games scraped, 600 successfully, 200 not found from scrape run 1 (still no meta data), 200 newly not found games - quota reached
Scrape Run 3 (No meta data filter option): 1000 games scraped, 400 successfully, 200 failed from day1, 200 failed from day 2, 200 failed from day 3
Because the failures don't update the meta data (as they shouldn't) - it means, you eventually only scrape roms that have no meta data available and you constantly run out of quota
by scrape run 5 i will have scraped about 5000 roms but will be fail on 1000 roms - these failures will constantly hit the quota limit and i will never be able to scrape past these initial 5000 and roms 5000-100,000 will never be scraped
I can't tell, easily which roms are going to fail. The Acorn Electron was an example i could easily identify - but there were lots of others that are not so obvious
It scraps the wrong games and data and skips most of my games. How do I manually add information or at least point the scrapper to the game myself?
Thank you! Oh man this video saved me. Lol.
glad it helped!
Hello! How can I add artworks to the games skipped by es de?
check out this video ES-DE: How to edit and manage game metadata
ua-cam.com/video/RWA4wK7pEko/v-deo.html
@@ES-DE_Frontend Thank you!
"Can't you choose RetroArch 32-bit and 64-bit separately? I need both versions."😩
sorry no that isn’t possible in ES-DE. but if you would like to discuss details please join us on discord here: discord.gg/EVVX4DqWAP
sorry not directly. there may be a way to handle this through custom systems but best to check our discord for details.
Which device are you using here?
its a Retroid Pocket 4 Pro
Hola increíble me encanta muchas opciones justo me preguntaba si tengo el sistema en la pc y en android puedo pasar lo que descargue en pc de imágenes y vídeos para pasarlo a android sin tener que hacerlo de nuevo gracias
yes the gamelists and downloaded_media folders (and their contents) are useable on any ES-DE install on any device
I really like it, but it can't edit game names in Chinese🥴
we will be looking at how multilanguage support in the future. we will need translation help as well so if you are interested please feel free to join our discord discord.gg/JPE9ZT6b
Are you serious using a open source project and make pay 5€ a month to use the android version ??? ( if it was at least one payment ) but a subscription to use a frontend in android ????thats so scummy man !!!!!!i guess im gonna use retroarch alone !!! and you lose a customer that would gladly pay 5€ for the frontend in google play or sideload !
for clarification ES-DE on Android is a one time purchase and the other versions (Windows, MacOS and Linux) are all free. For the android version you can choose to stay a long time subscriber if you want but its not required, just a one time purchase is all thats asked - you will get updates for life from that one time purchase. Hope that helps.