Grim Fandango Deluxe - Original Thread [Locked]

ultraneonoirantihero · Mon Oct 03, 2011 5:10 pm

bgbennyboy wrote:Old as hell? How dare you sir! (Yes they are a bit)

Well, not your apps, they still work fine :) Although I only have Cutscene player and that old launcher (which is now replaced by the newer Quick'n'Easy's Insteller+Launcher). Did you make more apps for GF? Thanks on those, I really enjoyed using them to enhance my GF experience.

No, I was thinking about ScummRevisited apps. I have v3 and v5. The third version works fine on my PC but doesn't handle GF very well (every time I export something it comes out as junk). The fifth version is unusable on my PC due to some graphical glitch (maybe my dual monitor setup messes things up) so I can't see any buttons and main window cant stretch all the way so I can't see all the workspace too. And both versions are the only ones I could find... and they're old as hell, the website's latest entry is from 2002!

But that v5 is a mystery, I found it somewhere and I can't find the source site anymore. It's from 2003, version: 5.0.0.130 pre-release.

bgbennyboy · Post by **bgbennyboy** » Mon Oct 03, 2011 5:20 pm

The only one I did that's related to this is the Resource file creator/dumper which is a little clunky now (used to unpack/repack lab the files). I've already offered to re-write it for Thunderpeel (command line use etc) if he needs it as part of a workflow for this.

There's V2, 3 and 5 of ScummRev with each supporting different things. For the tools you need, look through the rest of this thread. There's a tool to convert 3do models to obj so they can be modified and for the textures you need something that can handle .mat/.cmp files like matmaster (there are a few old tools). Other tools can be found here https://github.com/somaen/residual-tool ... ools/tools

cplhenshaw · Post by **cplhenshaw** » Mon Oct 03, 2011 5:24 pm

There's also a Scummrev 2 which you can get here http://www.freewebs.com/lucasstyle/downloads.htm

I don't know how you're exporting them but if you just dump the file it's probably not what you want. When you select something there is usually some additional button that will appear:

i.e in Scummrev 2 select a .mat file

Then a "View Material" button appears, clicking that displays the material, and you can then click File -> Save in the window to export it as a .gif or .bmp

Between versions 2,3 and 5 one of them should be able to export whatever you need.

ultraneonoirantihero · Mon Oct 03, 2011 11:08 pm

cplhenshaw wrote:I don't know how you're exporting them but if you just dump the file it's probably not what you want. When you select something there is usually some additional button that will appear

I know that :) But except for the backgrounds, textures and music, you can't export cutscenes and 3d models in some standard filetype from ScummRev... which I wanted to do at one point. So I gave up on ScummRev since every version had something fixed but also something broken. I'll leave the reverse engineering, modelling and programming to you guys, and I'll stick with the graphics.

So... no one working on hi-poly models and widescreen support?

An interesting thing: if you make GF widescreen, then it doesn't have to upscale itself too much to fill the screen and it gets less of that interpolation smudge. That way you don't notice (that much) the AntiAliasing problem and the low-res textures. Also, the 16-bit gif-like compression is less noticeable.

The original GF size iz 640x480px, and if you look the video I made in fullscreen you'll see it's 600px in height. That's just 25% increase in size. But when I play GF from ResidualVM, it is upscaled more than 100% on modern widescreen monitors due to it's original 4:3 aspect ratio and gets awful blurry.

I have to ask in what programming languages are those resource tools and ResidualVM written? I visited those links on GitHub and didn't recognize any of the files. But then again, I'm only familiar with Java.

Longcat · Post by **Longcat** » Tue Oct 04, 2011 7:57 pm

I have a tool by johndoe that will export the cutscenes to *.png and *.wav files. wouldn't know how to reencode them in smush format though, but if they were to be redone in 16:9(dunno how you would do that either), making residual play an avi container instead should be a relatively easy task. I am, however, on an airplane right now(yes, that's right, internet in the cloud!) so I won't be able to send you the tool until sometime tomorrow. You can search the lucasforums for it tho, if you are in a hurry:)

giucam · Post by **giucam** » Tue Oct 04, 2011 8:04 pm

ultraneonoirantihero wrote:
cplhenshaw wrote:I have to ask in what programming languages are those resource tools and ResidualVM written? I visited those links on GitHub and didn't recognize any of the files. But then again, I'm only familiar with Java.

It's C++. More complex that Java but still object oriented.

JohnnyWalker2001 · Post by **JohnnyWalker2001** » Sat Oct 08, 2011 8:41 pm

Longcat wrote:I have a tool by johndoe that will export the cutscenes to *.png and *.wav files. wouldn't know how to reencode them in smush format though, but if they were to be redone in 16:9(dunno how you would do that either), making residual play an avi container instead should be a relatively easy task. I am, however, on an airplane right now(yes, that's right, internet in the cloud!) so I won't be able to send you the tool until sometime tomorrow. You can search the lucasforums for it tho, if you are in a hurry:)

Wow, that's pretty nifty. I wonder if we could use it to clean up the compression a little? What other tools are hidden around that might not be known to everyone, I wonder?

JohnnyWalker2001 · Post by **JohnnyWalker2001** » Sat Oct 08, 2011 8:58 pm

Ok, with regards the discussion of the workflow, it's clear to me that a tool that could help people with graphical skills to jump right into the project would be a huge help right now.

I've written a step-by-step guide for people who want to help out, but it's still a pain.

Here's what people need to do in order to help out at the moment:

1. Copy the *.LAB files to their harddrive. (I wonder if this is totally needed, actually.)
2. Extract the contents of the *.LAB files to their harddrive... or specifically, the .CMP and .MAT files.
3. Use MatMaster, or some other program, to extract the BMPs.

Then they can fire up Photoshop and starting helping the project. That sounds simple enough, but it's actually quite troublesome, especially for someone with little or no technical background.

The biggest problem comes from .CMP files. Without these being correctly matched up with the .MAT files, it's impossible to create a new hires texture... and correctly matching the two is very difficult indeed.

Thanks to cplhenshaw we have a complete list of .3DO -> .CMP relationships here:

https://docs.google.com/spreadsheet/pub ... utput=html

But there's no easy way for someone who wants to help out to know what 3DO file is what model, and even if they do know that, which MATs are required for each 3DO.

I was thinking of putting together a simple webpage with a database connecting everything, so that someone can select a 3DO file and then be told which MATs they need to update and what CMP palette they should use.

As for what model is what, I wonder if cplhenshaw's Grim Fandango Model Viewer could be updated to include everything from the game. And as for extracting the required files painlessly, perhaps BGBennyBoy could write a bespoke extractor just for the MAT and CMP files (and maybe 3DO files, if we even manage to get to the 3D models)?

Who does everyone else think? Can anyone else think of a better solution to this problem?

somaen · Post by **somaen** » Sat Oct 08, 2011 9:49 pm

The 3do/mat-mappings are done in the .cos-files iirc, those are purely text in Grim, so gleaming information from them is just a matter of reading. (It is also a nice place to start for finding out which 3dos belong to which costume). Even a few comments are still in those files, so they should be quite readable.

As for extracting only MAT/CMP-files, that should just be a matter of adding a filter to any current tool, a simple matter of minutes, although IIRC there exists no tool to replace single files in the LABs right now, only to create new ones (correct me if I'm wrong).

As for support for video, currently ResidualVM supports Bink and SMUSH, with MPEG2-support started (although probably not working right now, as the only proper testing-platform for that is Escape From Monkey Island for PS2, which doesn't get much further than the "Press Start"-screen BEFORE the intro-video.) I did do some testing back when support was added, by making the EFMI-demo use the MPEG-decoder on a video I picked at random, but that isn't much of a test. (Besides support there might have been broken by the recent rewrites in the movie-systems, which I didn't care to test heavily, as MPEG2 didn't really work that well the first time around, and probably needs a rewrite anyhow, whenever it's actually testable in some engine).

There is one problem with recoding them though, the current system in ResidualVM selects codec purely on Game/Platform, not by the actual format of video (which makes sense, what game suddenly changes it's codecs?

) Switching for testing is a simple matter (Replace CreateSmushPlayer with Create*****Player in grim.cpp). Do tell if you plan to do any such work beforehand though, so I can have a chance to test/fix the MPEG2-wrapper

(Oh, and for that matter, the current implementation is really quite friendly towards most of the ScummVM-codecs (scummvm/video), so adding in any of those (although not in the main-fork of ResidualVM) shouldn't be much work either).

As for any texture-work, all work should be done with 24-bit in mind, as TGA-support will be added down the line (for EFMI), with .SUR-files for animated textures, .SUR-files are simply textfiles listing TGAs. And the TGAs in question are 24-bit uncompressed. This _MIGHT_ eventually remove the need for the .CMP-work in some cases. (But I can't say for sure though before I've tried and tested it).

JohnnyWalker2001 · Post by **JohnnyWalker2001** » Sat Oct 08, 2011 10:34 pm

Yep, the related MAT files are also (human) readable in the 3DO files, even the binary ones.

Not sure what you mean about "adding a filter" to the MAT extraction tool (currently MatMaster), but I like that it sounds like it will help and especially that it only takes a matter of minutes

My current workflow is this:

1. Open MatMaster and look for a texture that resembles something I think is worth updating (i.e. something I recognize from the game). Note: The colours are always completely messed up with I'm scanning, so it's not always obvious what I'm looking at, but you can usually guess from the shapes.

2. Try and find the 3DO model that relates to that texture.

3. Find the other MAT files using the 3DO file.

4. Cross-reference the 3DO file with the CMP file using cplhenshaw's list.

5. Go back to MatMaster and manually extract each of the BMPs from the MAT files I've located using the correct CMP file.

It's quite slow going.

cplhenshaw · Post by **cplhenshaw** » Sun Oct 09, 2011 10:29 am

With my model viewer there's a program "extractModels.exe" that you need to run before using it, and this copies all the .3do, .mat and .cmp files from all the lab files into a folder so they can be used by the viewer.

I suppose what would be better for this project would be if they were organised properly. For example, for each 3do file a folder is made and this contains the .3do file, any .mat files it uses and any .cmp files associated with that .3do (based on the .cos files)

As for what model is what, I wonder if cplhenshaw's Grim Fandango Model Viewer could be updated to include everything from the game.

Only the "interesting" models are hardcoded into the program menu, but the program can view any model if you pass it in as an argument. i.e >gfmv mannysuit.3do manny.cmp

JohnnyWalker2001 · Post by **JohnnyWalker2001** » Sun Oct 09, 2011 11:57 am

cplhenshaw wrote:Only the "interesting" models are hardcoded into the program menu, but the program can view any model if you pass it in as an argument. i.e >gfmv mannysuit.3do manny.cmp

Ah! I did not know this. This is very promising! I wonder how I could put this to use. Hmmm.

Longcat · Post by **Longcat** » Sun Oct 09, 2011 12:50 pm

here is the cutscene tool and source if anyone would find them interesting:

http://quick.mixnmojo.com/temp/grisp/grisc-0.1a.zip
http://quick.mixnmojo.com/temp/grisp/grisp-0.1-src.zip

Hosted by bgbennyboy.

and here is the aforementioned thread:
http://lucasforums.com/showthread.php?t=155743

Documentation on how to use the tool should be in the readme.

ioannis · Post by **ioannis** » Sun Oct 09, 2011 4:28 pm

Hey all, would love to help out if I can...

Let me get this straight though, we're supposed to match palettes with textures?

mats to cmps ?

JohnnyWalker2001 · Post by **JohnnyWalker2001** » Mon Oct 10, 2011 3:36 pm

ioannis wrote:Hey all, would love to help out if I can...

Let me get this straight though, we're supposed to match palettes with textures?

mats to cmps ?

That's great! Thanks!

You'll have to match the palettes in MatMaster, otherwise the images will look totally wrong. You don't need to worry once you get into Photoshop, though. Just make the image 4x the size and don't worry about the number of colours.