PINNED: Understanding GF's Files + Tools to work with them

[JohnnyWalker2001]

This thread will hopefully become a useful resource to anyone wishing to help out the project. In it I aim to provide a complete breakdown of the files used by Grim Fandango, with some detailed reference information to help anyone new, or anyone needing a refresher on how things are structured (like me!).

LAB FILES
LAB files are where all of GF's assets and scripts are stored. They are container files (or "bundles" aka "LucasArts Bundles") that store many other files within (much like a .zip file). Luckily, thanks to GF's popularity, there are some great quality fan-made tools available to open, extract and even create LAB files.

The best is from BGBennyBoy and can be found here:
http://quickandeasysoftware.net/softwar ... and-dumper

The English version of Grim Fandango comes with the following LAB files:

CREDITS.LAB
DATA000.LAB
DATA001.LAB
DATA002.LAB
DATA003.LAB
DATA004.LAB
MOVIE00.LAB
MOVIE01.LAB
MOVIE02.LAB
MOVIE03.LAB
MOVIE04.LAB
VOX0000.LAB
VOX0001.LAB
VOX0002.LAB
VOX0003.LAB
VOX0004.LAB
YEAR0MUS.LAB
YEAR1MUS.LAB
YEAR2MUS.LAB
YEAR3MUS.LAB
YEAR4MUS.LAB

As you might guess: MOVIE* files contain movies (in SMUSH format), VOX* files contains voice and lipsync files, *MUS* files contain music in (IMC format), and DATA* files contain everything else... The files we're most interested in

INSIDE DATA00*.LAB FILES
This is really what we're interested in. Inside these files (roughly broken up by into Intro (000) and Years (001 to 004)) you'll find:

.3do files (3D Objects) These are the 3D meshes for each object and character in the game.
.mat files (Materials) These are the actual textures, devoid of any colour palette information.
.cmp files (Colour Maps) These are matched with the .mat files to create a texture with the right colours.
.cos files (Costumes) These are plain text files that contain information on (among other things) which which .cmps are used by different .3do files.

You'll also find: .key, .lua, .set, .wav (sound effects), .bm (backgrounds) and .zbm files, but they're of less interest to us.

[JohnnyWalker2001]

In order to improve the models and textures in Grim Fandango, these are the files we need to modify:

.3DO FILES (3D Objects)
These files are in an ancient 3D format that is difficult to open in modern 3D modelling programs. Thankfully we have command-line tools written by cplhenshaw that convert .3do files to the commonly used .obj format and back again.

It can be downloaded here: http://www.mediafire.com/download.php?g589e8eehnp1bai

.MAT FILES (Materials)
These are essentially textures, but devoid of colour. Like 3DO files they need to be converted into a more commonly used format in order to be opened. They also need to be opened with a corresponding .CMP file if you wish to see how they look in game.

Thanks to Nitrus we have a dedicated tool to do just that: https://dl.dropboxusercontent.com/u/399 ... Reaper.rar

.CMP FILES (Colour Maps)
Colour map files are simply a list of indexed colours to be used with Material files. They can be converted to .ACT files to be used in Photoshop using Nitrus's tool (above).

.COS FILES (Costumes)
These are simply plain text files, and their structure is something similar to this (but not quite):

+COS
|
|---3DO
|---|
|---|-MAT
|
|---3DO
|---|
|---|-MAT
|
|---CMP

(ASCII diagram by ultraneonoirantihero)

This allows you to see which Colour Maps go with which 3D Object.

[JohnnyWalker2001]

INDEPTH: 3DO FILES

Converting these files into common .obj files has been pretty tricky. cplhenshaw's tool works wonders most of the time, but unfortunately there are some issues with it that were spotted by Nitrus and Blacksad. For the most part you should be good using it to convert 3DO files into OBJ files (and back again).

The tool can be downloaded here: http://www.mediafire.com/download.php?g589e8eehnp1bai

What we're aiming for:


WORKING WITH 3DO FILES
When it comes to actually modelling, cplhenshaw created a graph showing where the polygon "sweet spot" is (between 2K and 3K). More than that and the game engine starts to creak, so be aware:



That said, we're not looking to completely change the look of the game, so there shouldn't really be a need to push the boundaries too high.

KNOWN ISSUES
Unfortunately it's not possible to simply drop in new higher-res textures without some consideration. The coordinates all need to be updated, which affects the entire .3do file. This means that all textures will need to be uniformly 4x their original size, for example.

[JohnnyWalker2001]

INDEPTH: MAT FILES

Previously we had to rely on older programs written to mod games like Jedi Knight (which used the same 3D file formats). MatMaster 2 and Mat16 were of some use, but they've since been eclipsed by Nitrus's tool, so use that.

Download it here: https://dl.dropboxusercontent.com/u/399 ... Deluxe.rar

WHAT WE'RE AIMING FOR
You can read a little more about what we're aiming for here:
http://grimfandangodeluxe.blogspot.co.u ... angos.html

But essentially the textures are designed to look a little "hand painted", and we wish to maintain that style. There's no need to add a ton of detail that wasn't there in the original. Here's a visual reference:

From this:


To this:


WORKING WITH MAT/CMP FILES
Extracting and conversion is super easy thanks to Nitrus's tool, but you will need to find the corresponding CMP file for each MAT file so you can extract the image with the correct colours.

Once you've extracted the image as a BMP, change the colour to RGB and increase the size to FOUR times what it was previously.

Then get to work!

Note: (As previously mentioned) if you begin editing the textures for a 3D object, you must convert ALL the textures used by that 3do file to 4x their original size. This is because (if I'm explaining this correctly) the coordinates are absolute in each 3DO. So either the all the textures on the model are 1x or they're 4x.


HOW DO I FIND THE RIGHT CMP FOR MY MAT?
This is an arduous process, but it goes like this:

1. Find the object you wish to edit the textures for (aka the 3DO file).

e.g. glottis.3do

2. Do a search in the contents of all the .COS files until you find where your 3DO file appears.

e.g. glottis.cos and gl_akimbo_idles.cos

3. Look in one (or possibly all) of the found COS files until you find the reference to the 3DO file.

e.g.

Code: Select all

ID	TagID	Hash	ParentID	Name
0	0		0		-1			glottis.3do

4. Find the CMP reference with a ParentID of the 3DO

e.g.

Code: Select all

11	3		0		0			glottis.cmp

5. Now you know the name of the CMP file that goes with all the textures in the 3DO you chose, so open the 3DO file and look for all the references to MAT files.

e.g.
gl_tummy.mat
gl_butt.mat
gl_j_chest.mat
gl_j_back.mat
gl_j_skin.mat
gl_jaw.mat
gl_eye.mat
ear.mat
gl_shldr.mat
gl_arm.mat
gl_thigh.mat
gl_shin.mat
gl_heel.mat
gl_toe.mat

6. You now have all the MAT files and the correct CMP file to go with them. Use Nitros's tool to extract them all and you're ready to play!


OK! MY NEW TEXTURES LOOK AWESOME. NOW WHAT?
When it comes to saving your work for use in the game, you will need to save them back down to 256 colours using the original Colour Map palette. You can do this by converting the CMP file, to an ACT file. (Nitrus's tool will do that for you, too.)

Then you need to convert your document to an INDEXED colour format in Photoshop using a CUSTOM PALETTE -- the ACT file.

Yes, this will result in an 8-bit texture... the limitation of the GF engine.

Note: Make sure you keep your original RGB layered file, though! There is some talk about using 24-bit textures in Residual and it may still happen.


KNOWN ISSUES
There's no known issues with this yet. It's important to note that sometimes the same 3DO can have several different CMP files associated with it. So the MAT files themselves may be identical, but a different colour is used to change the look of the texture.

[JohnnyWalker2001]

INDEPTH: COS FILES

Finally we come to COS files. Mercifully these are simply plain text documents that can be opened by anybody, and they also don't need to be edited. Phew. They're mentioned here because they're great references for discovering which CMPs go with each MAT file.

As giucam helpfully pointed out, the inside of .cos files can be read thusly:

Example: mcc_thunder.cos (Thunderboy Manny)

Code: Select all

#  ID	TagID  Hash	ParentID	Name
...
	62	5		0		42			mth_head1.3do
	63	5		0		42			mth_head2.3do
	64	5		0		42			mth_head3.3do
...
	78	3		0		64			thunder.cmp
	79	3		1		63			thunder.cmp
	80	3		2		62			thunder.cmp

Here you see that the costume sets the CMP thunder.cmp on the head models, since the ParentID of the CMPs are the IDs of the models.

A costume is a set of Components and Chores. Each Component represents an on-screen object, like an object state, a 3D mesh, a material, or a keyframe animation, or a sound effect. Each Chore (diminutive for choreography) gives a set of instructions for how to move, change state or activate or deactivate each component at certain times.

[sallim]

The Dropbox links with GrimReaperDeluxe.rar doesn't work anonyme.
This post can be deleted or modified, if the links in the main posts are working =)

Thanks to Nitrus we have a dedicated tool to do just that:
GrimReaperDeluxe.rar

Here is an updated link:
https://www55.zippyshare.com/v/3zpR4zJM/file.html

Thanks for this great threads, even in 2017!