I would like to contribute Might and Magic III

[eje211]

Hello. I bought Might and Magic III: Isles of Terra in 1992 (and again on GOG) and I've never really known how to play it until now. It works fine on Dosbox, but that's true of many games on here. In addition to wanting to make it easier for me to play Might and Magic III, I'd like to contribute to the project. I see that ScummVM supports Might and Magic IV, V, and World of Xeen, but not Might and Magic III. Is there a specific reason for that? Would it be helpful if I tried Might and Magic III to the Xeen engine? Or is it deliberately excluded?

If it would be helpful is there a guide for people making their first engine support for ScummVM? Just asking.

I intend to fully conform to all style guides for the project.

[eje211]

And if I manage to get a somewhat usable implementation of Might and Magic III working (stranger things have happened), I'd like to try to implement the engine for Countdown, the 1990 game from Access. It's not supported by ScummVM either. If there is a specific reason why it's not, again, please let me know. It makes sense it wouldn't be supported: it's not exactly famous. The point is less to play Countdown than to learn how to implement the game engine for an old game the way ScummVM does.

[dreammaster]

The simple reason is the amount of time it takes to support each game. In many cases, such as the Might and Magic series, we don't have access to the original sources, and are forced to gradually disassemble the game executables to figure out how they work, and then implement code to support them. Even the games being in the same "series" doesn't help much, since frequently companies write each new game entirely from scratch.

In the case of the Might and Magic series, we were lucky that they implemented Clouds of Xeen and Darkside of Xeen using the same engine, with World of Xeen being the two combined. Plus Swords of Xeen originally being a fan-made modification of the game meant we could support all of them with a single engine. Then recently-ish, I actually finished implementing support for Might and Magic Book 1, and I even added in an enhanced mode that uses the Xeen automap and has enhanced graphics. However this took me about 6 months? or so so just for that game. M&M II and III were also complicated by the fact that the way the game executables were built made it hard to properly disassemble. There was a forum user that managed to figure out a way to disassemble M&M 3 a while back and was planning to work on it; no idea how much progress they've made.

If you'd like to contribute, we'd certainly welcome it. See https://wiki.scummvm.org/index.php?title=HOWTO-Engines. There's a tool called create_engine that can aid you in creating a new skeleton game engine. Plus, the Developer Central section of wiki.scummvm.org has a lot of helpful information, such as the HOWTO Reverse Engineering. If you are able to get to the point of familiarizing yourself with creating engines and basic disassembly skills, I could put you in touch with the developer who was previously experimenting with M&M3, and maybe the two of you could collaborate.

[dreammaster]

If there is a specific reason why it's not, again, please let me know

Again, the question of time. We're lucky in the case of the Access games that a lot of the core engine was shared between games. We already support Amazon Guardians of Eden, and we have a disassembly of Mean Streets that could, with enough effort, be used to support it as well. Countdown could likely also be disassembled with enough effort, using the existing Access engine as a guide.

[eje211]

See? That's not how I would have started. I would have started looking through the data files, see how they were structured, and try to figure out how to make those structures work. That, before you explained this, seemed simpler in my mind. But disassembling would be a great tool to have in my kit.

Again, I was not requesting anything. I didn't want to post an early draft of my implementation and hear, "Actually, we know for a fact that we can't support MM3 for X legal reason, or for Y technical reason."

This is great! I'll try decompiling one game and the other! And if I miserably fail, I at least won't have made anything worse.

I also wanted to do more C++ programming. And if there is some C in there, that's good practice too.

Now, here I go, walking tall and confidently into a wall. Or maybe not? Who knows?

[Mok]

M&M II and III were also complicated by the fact that the way the game executables were built made it hard to properly disassemble. There was a forum user that managed to figure out a way to disassemble M&M 3 a while back and was planning to work on it; no idea how much progress they've made.

I'm not sure if I remember it correctly, but it wasn't that hard to get MM3 executable to load into the commonly used disassembler. AFAIR Borland C++ overlays were/are supported already, the problematic part was that main chunk was compressed and the overlay part expect segment table at specific offset from the start of the file. What I did, was to use UNP twice on the exe, to get it decompressed (first "Compressor v1.01" and then "Exepack"), this will remove the overlay completely. Then I cut the overlay part from the original exe (starts with "FBOV" id). Then you need to find the offset of the segment table in the unpacked exe (search for "VDISK FAKE" string and the table is at 0x3E from there). Put that offset into dword at offset 8 in the overlay and then join the unpacked exe and overlay into one file. It won't work properly when run (there are likely more hardcoded values that need to be changed or the unpacked exe structure is just incompatible) but it's enough to load into debugger and get overlayed code fully visible. Not sure if it's enough to try to RE the whole game, I don't even remember why I tried that years ago, probably to rip the music player out.

[dreammaster]

My recollections of complexity may well have been just as simple as not being aware of tool like unp at the time I first looked at the game. I've come a long way from the green reverse engineer amateur trying to figure out how Lure of the Temptress worked some twenty years ago. Man, time sure does fly. So many games worked on since then..

[dreammaster]

I checked with the developer in question, and they haven't made any further progress. They also mentioned something interesting:

IIRC I also find out that German and French versions shipped unpacked executables with removed copy protection whose happily load into IDA without any EXE/IDAdosloader modifications and some functions were less inlined there compared to English EXE.

So that may be the way to go if you plan to experiment with disassembling the game, since it'd be a lot simpler initially than dealing with the English version.

[eje211]

I did not expect this discussion to continue. That's really encouraging. Thanks to both of you!

In the meantime, I tried Ghidra, because I'm on Linux and I was not sure about running IDA through Wine. It did not go as smoothly as I tought it would. Then, I tried to look into the CC files and they made no sense, until I found a blog post that explained that the CC files for Clouds of Xeen and Darkside of Xeen were encrypted. The encryption and decryption looked pretty straightforward, so I made a little Python app to reverse it, but THEN I found out that the CC files for MM3 were encrypted AND compressed. It seems that the CC file for MM3 is much more complicated to crack. But once I've done that, the rest might simpler.

I also found a website that provided an IPS file to decrypt the CC file. I'm not sure what that is. I looked it up, but all that came up were old utilities and a website that offers to apply a provided IPS file to a provided game data file. It looks like a modding thing.

I'm not giving up, but I really did not expect that much of struggle. The main point for the whole thing for me, was for it to be educational, and, so far, it definitely is that!

Although, if the CC file for MM3 was beyond the ability of the person who wrote the ScummVM engine for Xeen, I may try something else to begin with.

[NoSFeRaTU]

It won't work properly when run (there are likely more hardcoded values that need to be changed or the unpacked exe structure is just incompatible) but it's enough to load into debugger and get overlayed code fully visible.

IIRC I unpacked english executable with some dos-era unpacker called something like "Universal Program Cracker". Produced exe was playable but required small patch to either fix crash in copy protection dialog (the one where you enter words from specific page and line from game manual) or to bypass it entirely. And to load it properly in IDA there is also need to patch incorrect overlay offset in exe directly or slightly hack IDA dos loader. Loader already contains autofixing procedure for this case but it won't trigger for this executable (either add si.flags=|SI_OVR in CppInfoBase or fbov.exeinfo -= fbov.segnum*sizeof(seginfo_t) in LoadCppOverlays).

In the meantime, I tried Ghidra, because I'm on Linux and I was not sure about running IDA through Wine. It did not go as smoothly as I tought it would. Then, I tried to look into the CC files and they made no sense, until I found a blog post that explained that the CC files for Clouds of Xeen and Darkside of Xeen were encrypted. The encryption and decryption looked pretty straightforward, so I made a little Python app to reverse it, but THEN I found out that the CC files for MM3 were encrypted AND compressed. It seems that the CC file for MM3 is much more complicated to crack. But once I've done that, the rest might simpler.

CC format in MM3 is mostly the same as in Xeen with the difference that it use different compression format - LZHUF with non-standard dictionary value. There is someone made a write-up and provided unpacker source code for it if you are interested:
http://blog.rewolf.pl/blog/?p=1202
There is also exists mm3 mod, which fixes a couple of game bugs in vanilla mm3 executable and also replaces MM3.CC compression algorithm to the one used in xeen:
https://jeffludwig.com/mm3/download.php

[eje211]

I had come across that web page but not focused on that one specifically. I'll give it another look. I'm really very thankful at how much help this community is providing! Earlier today, I really thought that beginning with MM3 was probably not reasonable, now I think I might be able to do it!

[eje211]

Actually, I meant that I had spent more time on the second page mentioned here rather than the first. Anyway, I'll look at both more closely.

[NoSFeRaTU]

I also found a website that provided an IPS file to decrypt the CC file. I'm not sure what that is. I looked it up, but all that came up were old utilities and a website that offers to apply a provided IPS file to a provided game data file. It looks like a modding thing.
...
I had come across that web page but not focused on that one specifically.

Heh, I only now realised that you were talking about Ludwig's mod I mentioned regarding IPS. That's DOS-era patching format of the samely named tool IPS. You can find tools for it in the internet, for example there:
https://www.zophar.net/utilities/patchutil.html
You need top one which is a DOS version as at least WinXP variant won't work correctly with this mod.
But as I wrote earlier that mod will convert MM3.CC into Xeen format and patch executable so it will be able to use it. So rewolf's packer/repacker is more useful in regards to researching actual MM3.CC format.