EMI PS2 -> PC

[Gannet]

EMI on ResidualVM has been steadily improving and though it still has a way to go before all the issues are ironed out, I thought it might be time to revisit an old plan: Putting the improvements from the PS2 version of EMI into the PC version. You can read about my previous (unsuccessful) endeavours here.

Since we now have the option of running the PS2 version directly in ResidualVM, I'll start by listing some of its disadvantages that make the PC version preferable (all glitches and stability issues aside):

• Lower quality backgrounds (they show signs of dithering, implying a reduction to 16-bit colour).
• Some backgrounds have been lightened and have a washed-out look to them.
• Lower quality music (24kHz vs 44kHz in the PC version).
• Pre-rendered giant monkey head is replaced with a 3D model.
• No "main menu", no menu hotkeys, and a very limited save/load system.

With that out of the way, here are the 5 improvements I have identified in the PS2 version...

Smoother Models (status: not yet)
The PS2 renders models more smoothly than the PC version by using "smoothing groups". I don't know much else about this. ResidualVM doesn't currently do this and I don't know if it would be feasible to implement, or if it could be applied to the PC version too.

Higher Quality Video (status: not worth it)
The PS2 video suffers from less compression artifacts than the PC video. Sadly, this comes at a cost: the video is only 640x448 rather than 640x480, dropping every 15th line. It's questionable as to which version is actually better. Various techniques could be tried to reconstruct the missing lines, but the end result still may not be any better than the current video. (I've seen the PAL version too, it's the same resolution - only difference is lower frame rate).

Higher Quality Audio (status: success)
The PS2 videos have 48kHz uncompressed audio tracks, while the PC version has 22kHz bink audio.
This is relatively straight-forward to add to the PC version, though it requires some Windows-exclusive tools:

1. Demux all the PS2 videos using PSS Demux.
2. Cut the SMUSH headers out of the PC .m4b videos.
3. Use RAD Video Tools to mix in the wav files to the corresponding m4b files. I recommend quality = 3.
4. Add the SMUSH headers back in.

Note: Skip the "sory" video as the PS2 version of this is a few seconds shorter.
This only works in ResidualVM. If you want to do this for the original game you'll need to find a version of the RAD Video Tools that uses the old audio format (I used v1.0 with quality = 2).

Extra Dialog (status: success)
The PS2 version shows a few extra lines of dialog in various places, such as a line between Guybrush and the catapult operator about why he's having such a hard time hitting the house. These lines don't appear in the PC version because it doesn't show the scroll arrows for some reason.
This can be fixed by changing Dialog.scroll_mode (local.m4b/_dialog.lua, line 20) from FALSE to TRUE. I'm hoping this fix can be included with ResidualVM at some point but in the meantime you can install the patch file below.

Monkey Kombat Chart (status: success)
The PS2 version keeps track of moves you learn for Monkey Kombat so you can easily review them.
In another small change, the PS2 version will kick you out of a duel if you cause 3 draws, leaving Guybrush slightly puzzled. I guess this is supposed to be a hint for how to win the game.
I've succeeded in porting this to the PC version by copying some lua code and assets, you can install the patch file below and hold shift during your turn to display the chart. (Note it doesn't get removed properly when you release shift, I think this is a glitch in ResidualVM).

Monkey Invaders (status: not yet)
The PS2 version includes a new easter egg mini-game called Monkey Invaders, accessible from the Bonus menu once you learn all the moves in Monkey Kombat. Could be fun to try porting this to the PC version, though I'm not sure how you would activate it (the PC version doesn't have a Bonus menu).


Patch file
Download and extract the RVM or PC Dialog+Kombat patch from here.
RVM: Place the datausr.m4b file in your EMI install folder and check the "Load user patch" option in the ResidualVM game settings.
PC: Place the patch.m4b file in your EMI install folder, replacing the existing one. (This patch also includes a bugfix for Guybrush on the raft).



Any comments, corrections or further info are most welcome.

[Nitrus]

Regarding the models, has anyone actually tried loading the meshes in a 3D editor and tried counting the faces? "9 times more polygons" seems highly unlikely.
The fact that smoothing groups are mentioned however could indicate that there is additional data on how the mesh normals are calculated.
It's a very basic/simple principle, I think anyone who's done 3D modelling would be familiar with it.
I've talked about this in the Grim deluxe thread before, though sadly the links to the images are down and I don't keep them locally...

A good examle of applied smoothing groups can be seen here.

Normals can be calculated on many levels, per edge, per face, per vertex... I believe vertex normals are already loaded in ResidualVM as I can see it here: https://git.io/vz77Z

This could simply mean that: (this is only speculation without actually looking at the model data, as I'm not sure I can find a copy again of the PS2 anytime soon)
- PS2 models have better or any normal data on the vertex level. I can't remember whether the PC models have it, though I expect they must. If this is the case, and the "smoothing effect" that they're boasting has already been applied on the vertex normals themselves when creating the models, I'm not sure if there's anything more to be done engine side
- they have additional grouping data, and therefore normals may have been computed based on that. E.G. whether vertices belong to the same group or share a face. There's a simple example of some shared-face computing done in this post: https://goo.gl/521JfW

In the thread you posted there are some windows vs ps2 comparison links, but they're down. I think checking between how ResidualVM would render the PS2 models and how they're natively rendered (maybe even ResidualVM comparisons between the PC and PS2 versions) would prove or disprove what I've said above...

EDIT: Having looked at the best quality gameplay video on youtube that I could find, I'd only make this suggestion:
Enable antialiasing on in the *.ini file, something upwards of 16...
E.G. If you're on Windows, the file would be in: C:\Users\[USERNAME]\AppData\Roaming\ResidualVM\residualvm.ini
Then find the relevant EFMI entry, like [monkey4-win] or [monkey4-ps2] (probably), and somewhere in that section add:
antialiasing=16

To me what I'm seeing from the video and hat I'm seeing in ResidualVM don't seem to have much if any difference :/

DOUBLE EDIT: Actually I think I can see some subtle differences which don't seem to be related only to the shader. In that video I think I notice that Otis' arms are bulkier/more rounded...

[Gannet]

The only reference I have to smoothing groups is a post on the forums here.

I've uploaded some screenshots, follow the link to dialog fix in my first post. On the PS2 things appear "bulkier/more rounded", as you mentioned. You can also see the lighting in RVM is different (I'm tempted to say better?) than the original game.

[Nitrus]

I've taken a look at the images (especially at the PS2 comparisons in native environment and residual), an idea occurs to me...
In the other forum you mention that the PS2 models need to be converted, you also say that you've managed to load the PS2 models in the Windows version:

The format of the PS2 .meshb files is almost identical to the Windows ones. Once converted they can be put into the Windows data and the game will load them correctly.

So I guess the question is, were you able to see the model's differences with the PS2 versions on Windows? Presumably not on ResidualVM, but natively-run.
If yes (as in, once the PS2 model was loaded on Windows and indeed had the better look), ResidualVM isn't handling the extra data.
If no, they've done something interesting engine-side on the PS2.

And again, if yes, I have a couple of speculations. I can't check them out myself as I don't have the PS2 discs right now (I have looked at them though once before), I'll try to get them in the next few days, but in the meantime:
- The Windows version has all the models etc. bundled up in the root art*.m4b files. I remember that the PS2 actually had some island-named folders with the files in them as well, in addition to the art*.m4b ones. One speculation is that one of those (duplicate) files might be higher-res than the others. But instead of getting that one, ResidualVM is (maybe) somehow loading the default one from the art*.m4b file.
If such a file indeed exists, placing it in the datausr.m4b will most likely suffice, as it takes precedence. Probably not for the Windows version though as I can see some platform specific loading...

- What did converting them from PS2 to PC entail? On the other forum I saw that you wrote that the PS2 uses longs instead of words in the facedata. Do you strip out the extra bytes? Is it empty padding?
If it's not padding, it might hold some relevant data. But again this is only a speculation...

As for the lighting, yes, it's different.
IIRC the original game was done with Direct3D while ResidualVM uses OpenGL, so I guess it can't be perfectly identical... And AFAIK the PS2 didn't support OpenGL (with the exclusion of PSGL I think it was).

You can also see the lighting in RVM is different (I'm tempted to say better?) than the original game.

Bah, I think I like the PS2 lighting better myself

[Gannet]

Unfortunately the answer is no. The only observed differences were subtle variations in shading, just as can be seen between the two RVM screenshots (also see post #7 in the other forum).

I checked through all the subfolders etc when I was doing this originally and found only duplicate copies of the meshes - all identical.

Unfortunately I don't have the script I made to convert the meshes anymore. It did indeed strip out extra bytes but whether there was more to it or not I don't remember. The PS2 meshes could well contain extra data about smoothing groups or something but the Windows version isn't designed for that, and for the moment it seems ResidualVM isn't either. Of course, the devs could probably weigh on what they know?

[Zaxx]

The fix could also be applied to the original game.

Can I ask how? I see lots of bugs in ResidualVM when playing MI4 and I can get the original game working with the fanmade installer and launcher just fine so I'd rather play that one.

[Gannet]

Can I ask how? I see lots of bugs in ResidualVM when playing MI4 and I can get the original game working with the fanmade installer and launcher just fine so I'd rather play that one.

It's been a while since I did this but it should be something like this:
1. Use Resource File Creator and Dumper to extract "_dialog.lua" from "local.m4b".
2. Download and build the ResidualVM Tools.
3. Use the luac command to decompile _dialog.lua.
4. Change Dialog.scroll_mode from FALSE to TRUE on line 20.
5. Use luac to recompile _dialog.lua.
6. Pack it back into local.m4b.

I may have patch.m4b file kept somewhere containing the fix, if you have any trouble I'll see if I can find it.

[Zaxx]

2. Download and build the ResidualVM Tools.

Sorry, that's where my knowledge ends, I'm just a mere gamer.

[Gannet]

If you follow the link to the dialog fix in the first post, I've added a PC version in there. That is, I *think* it contains the dialog fix but not 100% sure :)

[Zaxx]

If you follow the link to the dialog fix in the first post, I've added a PC version in there. That is, I *think* it contains the dialog fix but not 100% sure

Overwritten the original file with the one in your fix, started up the game went to talk to the catapult operator... there were no extra dialog options. I don't know any other way to test it.

Anyway is there no release for those tools, one you don't have to freakin' build yourself?

[Gannet]

Okay, try downloading it again. I've added local.m4b instead of patch.m4b this time.

[Zaxx]

Okay, try downloading it again. I've added local.m4b instead of patch.m4b this time.

It works! Thank you very much for your help!

One side note though: the fix doesn't affect saved games you made before applying it. Because of this I almost thought the fix wasn't working but I checked it by starting a new game too and voila, the dialog options appeared. I'm pretty sure I did the same with your first solution too but I'm not 100% sure so who knows, maybe that would have worked too.

[Gannet]

I got the Monkey Kombat Chart to work! See first post for RVM patch file and screenshot. I'll get a patch file up for the original game soon...

[TarkinMX]

Gannet, I want to thank you for your continuing work on this. I appreciate it very much and look forward to seeing more.

[Wesker]

Higher Quality Audio (status: success)
The PS2 videos have 48kHz uncompressed audio tracks, while the PC version has 22kHz bink audio.
This is relatively straight-forward to add to the PC version, though it requires some Windows-exclusive tools:

1. Demux all the PS2 videos using PSS Demux.
2. Cut the SMUSH headers out of the PC .m4b videos.
3. Use RAD Video Tools to mix in the wav files to the corresponding m4b files. I recommend quality = 3.
4. Add the SMUSH headers back in.

Note: Skip the "sory" video as the PS2 version of this is a few seconds shorter.
This only works in ResidualVM. If you want to do this for the original game you'll need to find a version of the RAD Video Tools that uses the old audio format (I used v1.0 with quality = 2).

Does anyone knows if this applies to languages other than English too? I suppose it's the case, but I can't notice the difference that much and so I would like a confirmation before I check and do all that tedious process myself.

And what's that RVM Audio Patch in the Dropbox? Is it related to this or to something else? I suppose those are for the English version anyway.