Legend Entertainment's Games

[dreammaster]

I've updated my too to allow selecting sub palettes. I also added a zoom slider. Code is on GitHub

It'll likely be difficult to come up with a general purpose solution without reversing the scene display code from the actual games, which I'll get around to eventually. But in the interim, every little bit you find out will help me eventually. So thanks

[Risca]

I've come back to this issue tonight to dig for some more information, but I can't wrap my head around it.
I added extra log outputs to dosbox-0.74 so that it would print out the palette when it's changed:

Code: Select all

--- dosbox-0.74/src/gui/render.cpp.old	2017-11-05 23:23:45.508654441 +0100
+++ dosbox-0.74/src/gui/render.cpp	2017-11-05 23:24:34.987851237 +0100
@@ -68,11 +68,16 @@
 		break;
 	case scalerMode32:
 	default:
+		if (render.pal.first == 0) {
+			LOG_MSG("const uint32_t g_palette[256] = {\n");
+		}
+
 		for &#40;i=render.pal.first;i<=render.pal.last;i++&#41; &#123;
 			Bit8u r=render.pal.rgb&#91;i&#93;.red;
 			Bit8u g=render.pal.rgb&#91;i&#93;.green;
 			Bit8u b=render.pal.rgb&#91;i&#93;.blue;
 			Bit32u newPal = GFX_GetRGB&#40;r,g,b&#41;;
+			LOG_MSG&#40;"\t0x%08X,\n", newPal&#41;;
 			if &#40;newPal != render.pal.lut.b32&#91;i&#93;&#41; &#123;
 				render.pal.changed = true;
 				render.pal.modified&#91;i&#93; = 1;

I also added extra printouts for other system calls like file open, read, and close. Here's an example of the log output from a run of the game:

Code: Select all

   8281289: FILES:file open command 0 file DGATE010.FNT
SPEL\DGATE\DGATE010.FNT: reading 4096 bytes @ 0x00000000
   8303866: FILES:file close command: SPEL\DGATE\DGATE010.FNT
   8367343: FILES:file close command: SPEL\DGATE\DGATE000.PIC
   8372084: FILES:file open command 0 file C:\SPEL\DGATE\DGATE001.PIC
SPEL\DGATE\DGATE001.PIC: reading 4096 bytes @ 0x00000000
SPEL\DGATE\DGATE001.PIC: reading 4096 bytes @ 0x00001000
SPEL\DGATE\DGATE001.PIC: reading 5120 bytes @ 0x00002000
SPEL\DGATE\DGATE001.PIC: reading 4096 bytes @ 0x00003400
   8593830: FILES:file close command: SPEL\DGATE\DGATE001.PIC
   8598443: FILES:file open command 0 file C:\SPEL\DGATE\DGATE000.PIC
SPEL\DGATE\DGATE000.PIC: reading 4096 bytes @ 0x00000000
SPEL\DGATE\DGATE000.PIC: reading 4096 bytes @ 0x0000196D
   8702771: FILES:file close command: SPEL\DGATE\DGATE000.PIC
   8706810: FILES:file open command 0 file C:\SPEL\DGATE\DGATE001.PIC
SPEL\DGATE\DGATE001.PIC: reading 4096 bytes @ 0x00000000
SPEL\DGATE\DGATE001.PIC: reading 4096 bytes @ 0x0001311A
SPEL\DGATE\DGATE001.PIC: reading 4096 bytes @ 0x000062AC
   8779698: FILES:file open command 0 file C:\SPEL\DGATE\DGATE019.RGN
SPEL\DGATE\DGATE019.RGN: reading 4096 bytes @ 0x00000000
   8836227: FILES:file close command: SPEL\DGATE\DGATE001.PIC
   8840350: FILES:file open command 0 file C:\SPEL\DGATE\DGATE019.PIC
SPEL\DGATE\DGATE019.PIC: reading 4096 bytes @ 0x00000000
SPEL\DGATE\DGATE019.PIC: reading 4096 bytes @ 0x0001EA98
SPEL\DGATE\DGATE019.PIC: reading 4608 bytes @ 0x0001FA98
SPEL\DGATE\DGATE019.PIC: reading 4096 bytes @ 0x00020C98
SPEL\DGATE\DGATE019.PIC: reading 4096 bytes @ 0x00021C98
SPEL\DGATE\DGATE019.PIC: reading 4096 bytes @ 0x00022C98
SPEL\DGATE\DGATE019.PIC: reading 4096 bytes @ 0x00023C98
SPEL\DGATE\DGATE019.PIC: reading 4096 bytes @ 0x00024C98
SPEL\DGATE\DGATE019.PIC: reading 4096 bytes @ 0x00025C98
   8894818: FILES:file close command: SPEL\DGATE\DGATE019.PIC
   8899704: FILES:file open command 0 file C:\SPEL\DGATE\DGATE057.PIC
SPEL\DGATE\DGATE057.PIC: reading 4096 bytes @ 0x00000000
SPEL\DGATE\DGATE057.PIC: reading 4096 bytes @ 0x0006FDE4
   8928523: FILES:file close command: SPEL\DGATE\DGATE057.PIC
   8933374: FILES:file open command 0 file C:\SPEL\DGATE\DGATE019.PIC
SPEL\DGATE\DGATE019.PIC: reading 4096 bytes @ 0x00000000
SPEL\DGATE\DGATE019.PIC: reading 4096 bytes @ 0x00028891
SPEL\DGATE\DGATE019.PIC: reading 4096 bytes @ 0x00026B68
SPEL\DGATE\DGATE019.PIC: reading 4096 bytes @ 0x0002865C
SPEL\DGATE\DGATE019.PIC: reading 4096 bytes @ 0x0002788A
const uint32_t g_palette[256] = {
	0x00000000,
	0x00101010,
...
	0x00414141,
	0x00FFFFFF,

There is much more in the log but I thought it best to keep it short. There is also a fading effect going on after the intro. This seems to be done with palette cycling; a lot of new palettes are getting applied while after the intro is done and the first scene fades in.

I've setup dosbox to automatically run the game and I hit the ESC button to skip any intro, which takes me directly to the first interactive screen. This is how that screen looks like:



By inspecting the log, and using my tool to open the resource files, I can see that the game extracts a few different images and layer them on top of each other. For example, DGATE019.PIC contains this image at offset 0x0001EA98



Notice how the right lamp post, Xar, and the Nexus seal piece above his head are missing. These are all also located in the DGATE019.PIC resource file, at the offsets indicated by the log file. However, and this is what really bothers me, if I take the exact palette that dosbox dumped in the log for me and apply it to those overlayed images, it shows the completely wrong colors (the "big" picture is fine). For example, here's what I believe is the lamp post and the nexus seal piece:



I feel like I'm missing something crucial in understanding either how palettes work or how death gate is handling these overlayed images. Maybe there is some kind of index translation going on?
There might be some clues in the DGATE019.RGN file that is opened just before DGATE019.PIC.

Test code is, as usual, available on GitHub

EDIT:
I feel that figuring out how the scene display code works is the major blocker for allowing me (and others) to work on an actual game engine for Death Gate. I'm running out of ideas and that's when the progress slows down. If only I could figure this out, it would be so much easier to start hacking away at getting an actual game running. It would definitely feel like it's progressing faster once you can actually interact with the game.

[dreammaster]

A bit of good news. Barring any other bugs turning up, I've finished development on the Starship Titanic engine, which now supports both the English and German versions. I'm now returning to work finishing up the Xeen engine, with the anticipation that I'll spend some more solid time working on the Legend games when I've done with it.

[Dark-Star]

A bit of good news [...] I'm now returning to work finishing up the Xeen engine...

Cool! Looking forward to saving Crodo once again

[Risca]

Okay, I updated my resource manage to visually show me the palette of the "big" images. I also fixed a bug where the transparency of overlayed images where not working properly.

From what I can see, it seems quite common for the palette at index 17-80 to be all green and not used. It's only the smaller images that references that part of the palette, and they only reference index 17 (see attached picture).



The code is, as usual, available on GitHub

[Risca]

Okay, so I solved some indexing bugs in my code and I was wrong when I said that smaller images only use index 17 for green; that was a conversion error in my code because I was doing image conversions and alpha blending. I rewrote the code to do the image overlay manually and it turned out to be much simpler than converting the images to RGBA for alpha blending. Now I get correct indexes for all the green pixels. I haven't pushed it to GitHub yet, but I will do that soon.

Talking about green pixels, I figured out what that part of the palette is from: it's the menu interface! That part of the palette can be read from DGATE000.PIC. It still looks like crap (as before), but I made some progress at least.

I still haven't figured out how pixel indexes of smaller images are translated. I've made some experiments with the dosbox ability to capture video. It turns out that you can actually dump the whole screen in palette+image by activating video capture. Dosbox saves it in Zip Motion Blocks Video (ZMBV) format and it was fairly simple to extract stuff from it using dd and zlib-flate. The first frame of a video always contains the palette and an image. I'll do some more experiments and maybe write an extractor that I can put on github.

Stay tuned!

EDIT: code uploaded to github:

• https://github.com/Risca/first_zmbv_vid ... _extractor (terrible name, I know)
• https://github.com/Risca/dgate_resource_manager

EDIT2:
By comparing a dump from dosbox with an actual image+overlay, I can conclude that the very same pixel value in a smaller image might actually be translated to different pixel values in the dosbox dump. This might indicate some kind of translation based on offset, or it might just be some kind of fading effect by the game engine.

[Jorpho]

Wow, it's the thread I started twelve years ago! (Oh, to be young again.)

I saw the news on the front page about Star Trek and immediately came looking to see if there was renewed interest in this other seemingly hopeless cause.

I wish I had something useful to add. Rumor has it that GoG actually had Death Gate up for sale for a brief moment before they realized they'd made a horrible mistake. (Apparently the authors of the Death Gate books have been badly burned by licensing in the past.)

[dreammaster]

That's a coincidence; only just this morning I added a wiki engine stub page for my WIP Legend engine. Well, some good news.. as I've recounted earlier, I started making some progress disassembling Gateway and Companions of Xanth last year. However, I then temporarily put work on them aside whilst I finished Starship Titanic and then Xeen (which itself had already been on hiatus for a while). Now that both of them are done, I'm currently in a bit of an R&R phase, and taking things a bit easy for a bit.

Moving forward, I've also kind of gotten interested in adding support for the earlier Ultima games, particularly seeing whether I can create an enhanced mode for them that uses the Ultima 6 graphics for the overworld/cities/etc. I'm not sure how much time I'll spend working on Legend vs Ultima, but even if I end up spending the bulk of the time on Ultima 1, I'll likely switch back afterwards to working exclusively on Legend for a while before I worry about any of the other Ultima games in the series.

Of course, considering each Legend game's logic is hardcoded into their executables, don't expect rapid progress supporting all the games in the system. It'd be nice if original source code be located for any of the games, even if I'm not holding my breath for it.

[enderandrew]

That's a coincidence; only just this morning I added a wiki engine stub page for my WIP Legend engine. Well, some good news.. as I've recounted earlier, I started making some progress disassembling Gateway and Companions of Xanth last year. However, I then temporarily put work on them aside whilst I finished Starship Titanic and then Xeen (which itself had already been on hiatus for a while). Now that both of them are done, I'm currently in a bit of an R&R phase, and taking things a bit easy for a bit.

Moving forward, I've also kind of gotten interested in adding support for the earlier Ultima games, particularly seeing whether I can create an enhanced mode for them that uses the Ultima 6 graphics for the overworld/cities/etc. I'm not sure how much time I'll spend working on Legend vs Ultima, but even if I end up spending the bulk of the time on Ultima 1, I'll likely switch back afterwards to working exclusively on Legend for a while before I worry about any of the other Ultima games in the series.

Of course, considering each Legend game's logic is hardcoded into their executables, don't expect rapid progress supporting all the games in the system. It'd be nice if original source code be located for any of the games, even if I'm not holding my breath for it.

I believe there is some existing work done by the Ultima community in pulling data out of the classic Ultima games.

[dreammaster]

I believe there is some existing work done by the Ultima community in pulling data out of the classic Ultima games.

Yes indeed, I've been provided some good links to commented disassemblies in the Exult forums

[Risca]

Talking about commented disassemblies: do anyone know the legality of it?

I've started commenting some of my disassembly now. I managed to figure out how to unbind DOS/4G from DGATE.EXE and load it into IDA Pro (the freeware version).[1][2] With some help from the dosbox debugger, I was able to match the entrypoint in my "unbound" DGATE.LE to the running version in dosbox.[3]

I now wonder if I am allowed to share my work?

[1] https://blog.torh.net/2015/10/30/disassemble-dos4gw/
[2] https://www.hex-rays.com/products/ida/s ... _freeware/
[3] https://www.vogons.org/viewtopic.php?t=24744

[dreammaster]

Talking about commented disassemblies: do anyone know the legality of it?

There aren't any legal problems that I'm aware of, so sharing them is fine. Just an FWIW, last year Strangerke was able to locate a copy of the Shannara demo with all the original debug symbols left in it. I was then able to use the free BinDiff to apply a lot of the symbols to the full Shannara game, and am likely to be working on it more towards the end of the year. You can likely do similar to Death Gate as well, giving you a major headstart on figuring it out. This would help a lot - my original plans after Shannara was to maybe work on Companions of Xanth or one of the earlier games like Gateway, so finishing reversing Death Gate would be a big help. I'll upload a copy of the demo IDB somewhere and post a link if you're interested.

[Risca]

Oh, I would love to take a look into that! Can't promise anything, since this is very much an on-and-off thing for me

Learning x86 assembly as I go along is a bit daunting.

[dreammaster]

Oh, I would love to take a look into that! Can't promise anything, since this is very much an on-and-off thing for me

Learning x86 assembly as I go along is a bit daunting.

I had trouble trying to produce a 64-bit disassembly of the demo - the current IDB is using the 32-bit IDA version, which obviously can't be opened in the IDA freeware versions. Because of this, the best I can do is provide a HTML dump of the Shannara demo disassembly. If you're able to manually find any identical methods in Death Gate, it may help to identify lots of other methods:
download from here.

[Risca]

Is there a compatibility issue between IDA Pro and IDA Free?