Android 2.7.0 Beta

[Krampus]

Hello,
I recently installed the 2.7.0 beta version on my Android phone, replacing my 2.6.0 version.

The directory structure and settings were also all transferred, but the game paths are no longer correct.

A message at the start of the new version pointed out significant changes in the file access system.

Do I now have to change the path "manually" for every single game?

Is there a trick to adjust all game directories to the new file structure?

Glad about any help!
Niki

[Praetorian]

Hello,
I recently installed the 2.7.0 beta version on my Android phone, replacing my 2.6.0 version.

The directory structure and settings were also all transferred, but the game paths are no longer correct.

A message at the start of the new version pointed out significant changes in the file access system.

Do I now have to change the path "manually" for every single game?

Is there a trick to adjust all game directories to the new file structure?

Glad about any help!
Niki

Hello!

At the moment there is no automated way to migrate to the new system. We considered a few ideas but none would be ellegant to implement and cover all potential scenarios. Also we're still missing dedicated Android developers to focus on, implement and test any potential solution, which kind of why we're releasing 2.7.0 on Android on the beta channel.

Anyway, if you have a standard setup where every game folder is under a common, say, "games" folder then you will have to do the first recommended steps once to setup access for the app to the parent folder (here: "games") and then change the path for every game you've added to use this new way of folder access.

For posterity, I'm pasting here our short instructions that are in the popup:

In this new version of ScummVM Android, significant changes were made to the file access system to allow support for modern versions of the Android Operating System.

If you find that your existing added games or custom paths no longer work, please edit those paths and this time use the SAF system to browse to the desired locations.

To do that:

1. For each game whose data is not found, go to the "Paths" tab in the "Game Options" and change the "Game path"
2. Inside the ScummVM file browser, use "Go Up" until you reach the "root" folder where you will see the "<Add a new folder>" option.
3. Choose that, then browse and select the "parent" folder for your games subfolders, e.g. "SD Card > myGames". Click on "Use this folder".
4. Then, a new folder "myGames" will appear on the "root" folder of the ScummVM browser.
5. Browse through this folder to your game data.

Steps 2 and 3 need to be done only once for all of your games.

Let us know if there's a specific point in the instructions that needs further clarification.

I suppose that if you can access the "scummvm.ini" configuration file on your Android, using the ScummVM LAN server feature, *and* all your game folders are under one common parent folder, you could theoretically (I haven't tested it):

• Launch ScummVM on your phone and do the above steps for one game and note the game's ID.
• Close scummvm on your phone.
• Get your scummvm.ini using ScummVM LAN, on your PC, keep a backup and then edit it with a text editor (eg. Notepad++)
• Find the new path prefix used for the game whose path you just updated in ScummVM (matching the game id you noted in a previous step) and replace the old prefix on your other game entries with the new prefix.
• Also edit the "versioninfo" key and set its value to a bit higher than the existing one (so for the case of "2.7.0", set it to "2.7.1")
• Save the edited "scummvm.ini" file as "scummvmrc" (no extension, no ".txt" nor ".ini" etc)
• Copy this "scummvmrc" file in the root of the external application folder for ScummVM (we have a shortcut for it named "ScummVM data (ext)" in the ScummVM browser.
• Launch ScummVM again. It should auto-replace the previous scummvm.ini with the new version, and will delete the "scummvmrc" file (so it won't reuse it again).

If you have your game folders split under more than one common parent folders, you'll have to adjust the above process accordingly, ie. you'll need a new path prefix for each of the common parent folders.

Edit: Just copying back the edited scummvm.ini file in its original folder won't work. For one, when trying to copy it back (after editing it) to overwrite the old one on Android, ScummVM will refuse to do it -- it does not allow copying over a file with a name identical to an existing file. And for a second issue, I think it may be a bit unorthodox to change the scummvm.ini file while ScummVM LAN is running -- it'd probably be fine, but if something unexpected happens I wouldn't be surprised either.

Edit 2: I've updated the last steps with the mechanism we currently have to update the scummvm.ini with a new one inherited from the system we implemented for migrating from old-old versions of ScummVM.
Discussion about this method came from these threads:
- (solved) scummvm.ini change path?
- Editing scummvm.ini externally

[Krampus]

Thanks for the feedback!

I have a scummvmrc file saved as a backup for my android phone!

I edited this and replaced the old file paths with the new folder prefix.

For example:
OLD: path=/storage/3839-6431/app-data/org.scummvm.sdl/Maniac Mansion

NEW:
path=/sat/3839-6431%3Aapp-data%2Forg.scummvm.sdl%2FManiac%20Mansion

After that I copied the file to the corresponding directory of my Android phone. This was also recognized and the changes were applied.

The new file path is correctly displayed under the game options, but still not recognized. Only a manual selection with the "Add new folder" option works.

[Praetorian]

Thanks for the feedback!

I have a scummvmrc file saved as a backup for my android phone!

I edited this and replaced the old file paths with the new folder prefix.

For example:
OLD: path=/storage/3839-6431/app-data/org.scummvm.sdl/Maniac Mansion

NEW:
path=/sat/3839-6431%3Aapp-data%2Forg.scummvm.sdl%2FManiac%20Mansion

After that I copied the file to the corresponding directory of my Android phone. This was also recognized and the changes were applied.

The new file path is correctly displayed under the game options, but still not recognized. Only a manual selection with the "Add new folder" option works.

If it's alright could you attach your scummvm.ini before and after the edit?

Off the top of my head, and comparing with my own SAF paths, I can see these issues:
- Your new path starts with "/sat" but it should start with "/saf"
- On my new paths, neither the path separator (slash) character nor the blank space character are shown as their hex (HTML URL encoded) value, ie. my new paths look like this (notice that there's no "%2F" for the slash separator nor "%20" for the space):

Code: Select all

/saf/5C5B-B66B%3Agames/Blade Runner

I am unsure as of yet, if it's okay to have the hex values. Maybe the code "translates" and parses them as it should. Also unsure if your "/sat" prefix is a typo in the post or it's what you pasted in your new scummvm.ini. So that's why I am asking for an attachement of the full file.

[Krampus]

Hello!
The "sat" prefix was a typo!

In the revised version I tried it with path separator (slash) and space. Unfortunately this does not work!

Alternatively I changed the slashes and spaces to the hexadecimal values. Also no success.

The only working method is to change the game path "manually" for every single game!

In the meantime I have "downgraded" the version to 2.6.1.

[Praetorian]

Hello!
The "sat" prefix was a typo!

In the revised version I tried it with path separator (slash) and space. Unfortunately this does not work!

Alternatively I changed the slashes and spaces to the hexadecimal values. Also no success.

The only working method is to change the game path "manually" for every single game!

In the meantime I have "downgraded" the version to 2.6.1.

According to developer lephilousophe who worked on and is more familiar with the update to SAF paths, you need to:

1. Add "/storage/3839-6431/app-data/org.scummvm.sdl" as new SAF, which is basically do the first SAF steps process for one game, setting the SAF "root" (the folder you select from the Android file browser UI) to be the "org.scummvm.sdl".
2. For your Maniac Mansion game entry, you'd then set its new path from the ScummVM interface, by browsing into the "Maniac Mansion" directory using the SAF shortcut from the previous step (likely shown as "org.scummvm.sdl" I guess). The path should result showing like this:

   Code: Select all

   /saf/3839-6431%3Aapp-data%2Forg.scummvm.sdl/Maniac Mansion

   Notice that the slash between "app-data" and "org.scummvm.sdl" is encoded since it's part of the SAF root prefix, but what follows *after* "org.scummvm.sdl" is not encoded!
   So basically in your case it's a bit tricky and you need to be careful on what's encoded and what's not, for it to work.
3. Edit the rest of the paths for your games as above, provided that they are all in subfolders under "org.scummvm.sdl"

[Krampus]

I have now defined the "game path" in the root directory as SAF.

Did I understand correctly that the game paths can only be changed manually within the Scumm VM interface?

Because every attempt to provide the scummvm.ini (scummvmrc) with the correct SAF leads to a total crash, where only a reinstallation helps!

[Praetorian]

I have now defined the "game path" in the root directory as SAF.

I am not sure I understand what you did here. Please paste the scummvm.ini or at least the part of it that shows the path for the game after you set the path from the ScummVM GUI.

Did I understand correctly that the game paths can only be changed manually within the Scumm VM interface?

The purpose of this thread is to test if it can be further automated, via editing the scummvm.ini file.
However, since I didn't have time to test this myself, we are pretty much investigating whether it's possible or not via your efforts. In theory I believe it should work, and so does lephilousophe.

[Krampus]

Hello,

I will list below all the steps in chronological order and hope that this will make things more transparent.

1) Initial situation: ScummVM version 2.6.1 installed on my Android smartphone.
2.) The scummvm.ini (scummvmrc) I initially used from my Windows version and adapted all paths to the Android directory structure. This worked very well and without any restrictions.
3.) An example for the game path (excerpt from the ini file) under version 2.6.1:
path=/storage/3839-6431/Spiele/Black Cauldron Amiga
(please don't be surprised - the game path is now different from the last posts, because I simplified the file structure on my smartphone in the meantime)
4.) I have successfully installed the beta version 2.7.0 from the Playstore on my smartphone
5.) All games are also displayed correctly in the user interface, also the settings are transferred correctly from the old version!
6.) In the user interface I used "Add Game...." / <Add a new folder> to set the directory /storage/3839-6431/Games as new SAF!
7.) The above mentioned directory is now displayed under "Add Game..." and can also be selected.
8.) After that I set the game path for "Black Cauldron Amiga" under "Game Options...." like in the example above. This looks now as follows:
/saf/3839-6431%3ASpiele/Black Cauldron Amiga
9.) For the further procedure I downloaded the scummvm.ini via the LAN function, adapted it and changed the game paths to the new prefix (/saf/3839-6431%3ASpiele/+Gametitle like above)
10.) And with this all efforts end, because ScummVM basically starts, but the screen stays black. Also further start attempts remain unsuccessful.
11.) I uninstalled the app and then reinstalled it.
12.) I put the scummvmrc file back in and then started the program.
13.) All games are displayed correctly again, also the game paths are displayed correctly with the new prefix under "Game-Options....:"
14.) However, not a single game can be started - "Game data not found".
15.) Out of interest I did step 6.) again with the result (see step 10.)

Now I do not know any further!
Please look over my presentation if I did something wrong.
The ini files, I wanted to attach, but are not allowed as an attachment!

[Praetorian]

Thank you for writing an analytical description of your steps.

I have now (as of an hour ago) tested this on mine and, in my case, the process worked.

I will list my steps and specific examples of paths.

(I actually wrote up the whole explanatory thing, then the forums logged me out and kept none of the text, so I will re-write it when I get some free time later in the day ).

[Praetorian]

So these are my steps:

1. Clean install of 2.6.1 (arm64-v8a). Basically I uninstalled my 2.7.0 and then installed 2.6.1.
2. Launched ScummVM (v2.6.1)
3. (optional) From the "Global Options" / "Backend", I set mouse control to "touchpad" for "in menus" and "in 2d games". This is a personal preference.
4. Added a few games that reside on my SD Card and are under a parent "games" folder. Some of these games' data folders are direct subfolders of "games", and some are a few levels deeper. Like there's games with paths like:

   Code: Select all

   /storage/5C5B-B66B/games/7days

   and games with paths like:

   Code: Select all

   /storage/5C5B-B66B/games/Amiga LOOM/Loom_Disks_Extr/Rooms

5. Tested launching a couple of these games to check if they run. They did run ok.
6. (optional) I ran the ScummVM LAN Server. (Global Options -> LAN). I set the "/root/ Path" to ScummVM's internal folder using the "ScummVM data (Int)" shortcut on the top level of the ScummVM file browser.
7. (optional) From my desktop PC's browser I navigated to the ScummVM's LAN server URL, and grabbed the "scummvm.ini" file. These last two steps are optional, mostly for me to have a backup of the original 2.6.1 "scummvm.ini" and also have a working reference to compare with the updated scummvm.ini on 2.7.0.
8. On my smartphone, I exited ScummVM.
9. I installed ScummVM 2.7.0 (arm64-v8a), which updated the existing installation.
10. I launched ScummVM (v2.7.0). All previous settings and already added games were kept. But, as expected, the games do not launch now.
11. I selected a game from the list (in my case "7 Days A Skeptic", an AGS game), chose "Game Options...", then "Paths" and "Game Path:". Using the ScummVM file browser, I navigated to the top level (pressing the "Go Up" button until it went no further) and then selected "<Add a new folder>/". This brought up Android's file browser, and I navigated to the SD card, and then inside the "games" folder, and then selected "Use this folder", and allowed ScummVM to access it. After that the focus returned to ScummVM file browser, where the new shortcut "games/" had appeared in the top level. I chose that and then selected the game data folder for "7 Days A Skeptic" (in my case "7days/"). Clicked on "Choose", then pressed "Ok" and tested launching the game. It now launched fine, using the new SAF path. The new SAF path for it looked like this:

    Code: Select all

    /saf/5C5B-B66B%3Agames/7days

12. I ran the ScummVM LAN Server from Global Options -> LAN. I set the "/root/ Path" to ScummVM's internal folder using the "ScummVM data (Int)" shortcut on the top level of the ScummVM file browser.
13. From my desktop PC's browser I navigated to the ScummVM's LAN server URL, and grabbed the "scummvm.ini" file.
14. On my desktop PC, I edited the scummvm.ini file with Notepad++ to:
    • Change the "versioninfo" key's value to "2.7.1" (from 2.7.0)

      Code: Select all

      versioninfo=2.7.1

    • Replace all substring instances of "/storage/5C5B-B66B/games/" with "/saf/5C5B-B66B%3Agames/". Obviously, these strings will be different for each user, but should be easy to figure out simply comparing the old (classic paths) and new value (SAF path) for the game paths. The replace was done using Notepad++'s Ctrl+H shortcut to bring up and use the "replace" feature.

      

      NotepadReplace.png (24.09 KiB) Viewed 9343 times

    • Saved the final result as "scummvmrc" with no extensions. Actually, Notepad++ automatically added a ".ini" extension which I then had to remove manually by renaming the file.
15. Back to my phone, ScummVM is still running with the LAN server tab still open. I set the "/root/ Path" for the LAN server to the app's external folder now using the "ScummVM data (Ext)" shortcut on the top level of the ScummVM file browser. I clicked on "Apply".
16. From my desktop PC's browser I navigated to the ScummVM's LAN server URL, and refreshed to get the external folder's contents (it was empty). Then, I uploaded the "scummvmrc" file to the external app folder.
17. On my phone I exited ScummVM.
18. Re-launched ScummVM. It launched ok, and all the added games were still there, but now all were launching ok as well.

I hope I didn't forget anything.

If you did the same steps (your description is very similar to what I did, I think), and got different results, then I guess it's an Android thing, or maybe some side effect from a rogue setting we overlooked? Or a localized text thing? Although I don't see any special characters on your paths and also I wouldn't really expect this to be an issue. Anyway, it's probably why we didn't go with automating this process. Too many parameters or user-specific cases that we could not test and guarantee would work

If you can spot something that you did different and could correct, please try it and report back.

I am attaching (via pastebin) the full versions of my scummvm.ini from the above process, for 2.6.1, 2.7.0 (after changing one game to SAF path), and the edited scummvmrc that worked.

scummvm.ini (2.6.1)
scummvm.ini (2.7.0)
scummvmrc

[Krampus]

First attempt:
I followed all steps exactly and added a few games to the "empty" 2.6.1. version!
The "conversion" to version 2.7.0 worked fine and the games could also be started.

Second attempt:
I added the new prefixes to my old scummvm.ini of version 2.6.1 (with about 250 entries) and then added them.
Also in this case the games can be started under 2.7.0 without problems.
The only catch: The app takes almost 5 minutes to load!

I guess that in this case I have to "clean up" the scummvm.ini a bit.

Finally, thank you very much for your efforts - please forward this to lephilousophe as well. You are simply the best!

[Krampus]

Hello,
now I have to be annoying again in this matter.

As already mentioned in the last post, the loading time of the latest ScummVM version is now up to 5 minutes. The games that are started within ScummVM also work fine - they run smoothly and without any delays!

I tried to remove "superfluous" entries from scummvm.ini - without success - the loading time of the app did not decrease!

After that I completely uninstalled ScummVM and then installed the latest version (from the Playstore) on my smartphone, but did not add any existing scummvm.ini.
I added some of my games in the process. I noticed that the loading time of ScummVM increases by a few seconds with each added game.

Has this happened to others, or does this just seem to be a problem on my smartphone.
For comparison: ScummVM 2.6.1 loads within a few seconds!

[Praetorian]

The loading time of the ScummVM launcher will be decreased significantly in the next beta update.
Internal testing with 75 games added had the launch time at less than 2 seconds.

However, there will be an issue that will persist:
- A few games may take a more than 5 seconds just to be added (for me the offensive game is Gabriel Knight 2: The Beast Within (with subtitles patch)).
- Some games may take longer to launch, like 1 minute or sometimes up to 3 minutes plus (examples for such games for me are again The Beast Within and also Little Big Adventure, The Longest Journey with the HD mod, RAMA and Grim Fandango (this is less than 1 minute thankfully). Here are the results the last time I've checked (with the current development build in "release" mode)

Code: Select all

Beast Within: launches in 3min 14secs
The Longest Journey HD: launches in 1min 4secs
Grim Fandango: launches in 39secs
RAMA (Sierra): launches in 2min 24secs
Little Big Adventure: launches in 1min 20secs

Other games still launch fast, so I guess some of our users may not even notice this.

Unfortunately, there may be very little we can do to further improve performance at this point. Google requires to use latest SDK for the app to remain on the store *and* target modern devices. And the latest SDK removes functionality and a compatibility mode that, up to 2.6.1, we were able to use so that games could launch faster; note that even 2.6.0 and 2.6.1 could not be on the Play Store because by then Google had raised the requirements for SDK and target API. So we *have* to use SAF for file access (...).

Of course that should not be an issue, but SAF access is reportedly multiple times slower than the previous File I/O methods (I guess tighter security is preferrable than decent performance/responsiveness) (1, 2, 3, 4, 5, 6 and so on). I don't know how Google has allowed this as the better and, arguably, the only option for certain apps, but also I do know that ScummVM is *not* the typical app Google has in mind for their Play Store.

[Krampus]

Hello!
I have now tested the latest branch master!
Within seconds the launcher has started!

I have not stopped the time, but also the games load all very fast