SCI Compression Tool
Moderator: ScummVM Team
Don't worry, I know how to use a hex editor. The version of the game is also 1.000. I don't think there was another one for the CD version.
Actually, it seems to be in the middle of a bunch of data for me as well. The only time "SOL" appears in the file is at the beginning though. I'm starting to think that my resource.aud file might have just gotten messed up somewhere, the disc image has been archived and moved between various harddrives for quite a few years. The in-game speech all works fine though, so I'm not sure what exactly the problem with it is.
Actually, it seems to be in the middle of a bunch of data for me as well. The only time "SOL" appears in the file is at the beginning though. I'm starting to think that my resource.aud file might have just gotten messed up somewhere, the disc image has been archived and moved between various harddrives for quite a few years. The in-game speech all works fine though, so I'm not sure what exactly the problem with it is.
Just wanted to post a quick update -- for some reason, after building from the current HEAD, everything now compresses fine with both the CLI and the GUI. Nice work, whatever you changed!
The issue I was having before with the GUI was that it is not capable of creating directories, and I was trying to make a new one for the compressed file. Also, for some reason it seems that the GUI compresses files roughly 2x as much as the CLI.
The issue I was having before with the GUI was that it is not capable of creating directories, and I was trying to make a new one for the compressed file. Also, for some reason it seems that the GUI compresses files roughly 2x as much as the CLI.
Last edited by KuroShiro on Sun Aug 29, 2010 6:19 am, edited 1 time in total.
Yes, but not for Mac that I know of -- you'll have to build them yourself. To do so, in addition to all the libraries listed on the compiling ScummVM page, you will also need to install wxWidgets. Other than that, if you can compile ScummVM, you should be able to build the tools. Someone with more experience building on OSX could correct me.Kyahx wrote:Are there any nightly builds of the tools available anywhere?
I would like to compress my SCI games, but I'm not having any luck building the tools for OSX myself
Digging up this old thread because I have a related question.
The audio file from KQ5 from the KQ collection is in a different format -- it's not RESOURCE.[AUD|SFX], it's AUDIO001.002 -- but it still has a .MAP file associated with it. It's not recognized by the compression utility.
Is this simply unimplemented or is it a format that doesn't lend itself to compression as well as the RESOURCE.AUD format, or is it already compressed in some rudimentary way that renders additional compression less useful?
If it's just unimplemented I might take a crack at it myself.
Thanks.
The audio file from KQ5 from the KQ collection is in a different format -- it's not RESOURCE.[AUD|SFX], it's AUDIO001.002 -- but it still has a .MAP file associated with it. It's not recognized by the compression utility.
Is this simply unimplemented or is it a format that doesn't lend itself to compression as well as the RESOURCE.AUD format, or is it already compressed in some rudimentary way that renders additional compression less useful?
If it's just unimplemented I might take a crack at it myself.
Thanks.
The audio in KQ5CD is in an earlier format, but the compression tool does support it, AFAIK. The digital audio file decoding code can be found in the SCI engine in sci/resource_audio.cpp. Check loadResource(), loadFromAudioVolumeSCI1(), loadFromAudioVolumeSCI11(), readAudioMapSCI1() and readAudioMapSCI11()agf wrote:Digging up this old thread because I have a related question.
The audio file from KQ5 from the KQ collection is in a different format -- it's not RESOURCE.[AUD|SFX], it's AUDIO001.002 -- but it still has a .MAP file associated with it. It's not recognized by the compression utility.
Is this simply unimplemented or is it a format that doesn't lend itself to compression as well as the RESOURCE.AUD format, or is it already compressed in some rudimentary way that renders additional compression less useful?
If it's just unimplemented I might take a crack at it myself.
Thanks.
Thanks for the quick response.md5 wrote:The audio in KQ5CD is in an earlier format, but the compression tool does support it, AFAIK. The digital audio file decoding code can be found in the SCI engine in sci/resource_audio.cpp. Check loadResource(), loadFromAudioVolumeSCI1(), loadFromAudioVolumeSCI11(), readAudioMapSCI1() and readAudioMapSCI11()
Code: Select all
./scummvm-tools-cli.exe --tool compress_sci --vorbis -o AUDIO001.00g
../../../games/sierra/King\'s\ Quest\ Collection/KQ5/AUDIO001.002
Fatal Error : Input file doesn't seem to be a valid sci audio resource file
I tried the GUI (1.2.0), and after selecting the file it suggests either compress_agos or encode_dxa, which are both obviously incorrect. I tried compress_agos anyway and it quickly gives a wrong format error (as expected) once it starts. Renaming the file to RESOURCE.AUD causes the GUI to select compress_kyra, which errors out with an assertion failed once it starts.
EDIT: compress_sci isn't listed in the 1.2.0 tools readme, was it perhaps added after that?
EDIT2: looking at the compress_sci source, this error is thrown if it doesn't find a recognized header at the beginning of the file -- maybe there is no header in the SCI1 audio file.
EDIT3: So, according to resource_audio, there is no header in SCI1 audio resources, and the map file looks like this:
Code: Select all
AUDIOnnn.MAP contains 10-byte entries:
Early format:
w 5 bits resource type and 11 bits resource number
dw 7 bits volume number and 25 bits offset
dw size
Later format:
w nEntry
dw offset+volume (as in resource.map)
dw size
ending with 10 0xFFs
EDIT Final:
From somewhere else in compress_sci.c:
Code: Select all
// First find out how many samples are in this file
// We only support SOL audio and WAVE
// We can't support games that use "raw" audio (would have to walk through audio map for those, which would
// complicate this code - all talkie games that use this (kq5) don't use much space anyway, so not supporting
// those isn't a big issue
I wrote a python script to convert the KQ5 AUDIO001.002 to a format compress_sci will work on. I just use the AUDIO001.MAP file to run each resource through python's wave module then write them out to one file.
compress_sci dropped the file size (using vorbis default settings) from 93,229KB to 35,700KB -- this seems like a worthwhile improvement to me.
Next step is to change scummvm/engines/sci/resource_audio.cpp to recognize this format for KQ5.
EDIT: The error I get right now is "could not translate offset to compressed offset in audio volume!"
EDIT2: Realized what I missed. I didn't actually write the new offsets to a new map file.
compress_sci dropped the file size (using vorbis default settings) from 93,229KB to 35,700KB -- this seems like a worthwhile improvement to me.
Next step is to change scummvm/engines/sci/resource_audio.cpp to recognize this format for KQ5.
EDIT: The error I get right now is "could not translate offset to compressed offset in audio volume!"
EDIT2: Realized what I missed. I didn't actually write the new offsets to a new map file.
OK, I'm using KQ5 now with compressed audio.
The python script below rewrites the AUDIO001.002 from raw sounds to WAVE sounds, so compress_sci understands it.
It also rewrites AUDIO001.MAP to match the new WAVE formatted file, which compress_sci needs to create an offset translation table.
The files created by this program work fine with ScummVM without compression as well, and the AUDIO001.002 file is actually slightly smaller than the original because the raw sounds were padded to 2048 bytes.
The script should work on any normal python2 install (2.5+)
The python script below rewrites the AUDIO001.002 from raw sounds to WAVE sounds, so compress_sci understands it.
It also rewrites AUDIO001.MAP to match the new WAVE formatted file, which compress_sci needs to create an offset translation table.
The files created by this program work fine with ScummVM without compression as well, and the AUDIO001.002 file is actually slightly smaller than the original because the raw sounds were padded to 2048 bytes.
The script should work on any normal python2 install (2.5+)
Code: Select all
#!/usr/bin/env python2
from __future__ import with_statement
from struct import unpack, pack
from StringIO import StringIO
from wave import open as wave_open
# dir with KQ5 installed
kq5_dir = "C:\\Users\\adam\\Games\\sierra\\King's Quest Collection\\KQ5"
# original AUDIO001.002
kq5_aud = "AUDIO001.002.bak"
# original AUDIO001.MAP
kq5_map = "AUDIO001.MAP.bak"
# new AUDIO001.002
aud_new = "AUDIO001.002.new"
# new AUDIO001.MAP
map_new = "AUDIO001.MAP.new"
def parse_map():
with open('\\'.join((kq5_dir, kq5_map)), 'rb') as file: map = file.read()
return tuple((fields[0], fields[1] >> 28, fields[1] & 0x0fffffff, fields[2])
for fields in (unpack('<HII', map[n:n+10])
for n in range(0, len(map) - 10, 10)))
def convert_to_wav():
res_map = parse_map()
with open('\\'.join((kq5_dir, kq5_aud)), 'rb') as aud_in:
with open('\\'.join((kq5dir, aud_new)), 'wb') as aud_out:
with open('\\'.join((kq5dir, map_new)), 'wb') as new_map:
for num, volume, offset, size in res_map:
aud_in.seek(offset)
temp = StringIO('')
wav = wave_open(temp, 'wb')
wav.setparams((1, 1, 11025, 0, 'NONE', 'not compressed'))
wav.writeframes(aud_in.read(size))
new_offset = aud_out.tell()
aud_out.write(temp.getvalue())
new_map.write(pack('<HII', num,
((volume << 28) & 0xffffffff) + new_offset,
aud_out.tell() - new_offset))
new_map.write(pack('<HII', 0xffff, 0xffffffff, 0xffffffff))
if __name__ == '__main__': convert_to_wav()
I've posted a patch to the tracker to add this functionality to compress_sci.
https://sourceforge.net/tracker/?func=d ... tid=418822
EDIT: And I posted a bug report about an issue I spotted while writing the patch.
https://sourceforge.net/tracker/?func=d ... tid=418820
https://sourceforge.net/tracker/?func=d ... tid=418822
EDIT: And I posted a bug report about an issue I spotted while writing the patch.
https://sourceforge.net/tracker/?func=d ... tid=418820
I don't have this game, but you could try changing theNoiZje wrote:The CD version of Jones in the Fast Lane also uses the AUDIO001.MAP AUDIO001.002 file structure.
Code: Select all
if (_input.size() == 0x05C9B000) {
Code: Select all
if (_input.size() == 0x05C9B000 or _input.size() == 0xjones_file_size_in_hex) {
It's dependent on the sample format (8bit mono 11025 with each sample padded to multiples of 2048 bytes) as well as the AUDIO001.MAP format.
Re: SCI Compression Tool
Sorry for bringing this old topic up again, but no matter what I try, I always get the error message:
Unexpected input file 'AUDIO001.002'!
I've been trying with the command
scummvm-tools-cli --tool compress_sci --flac AUDIO001.002
after using the python tool above and I made sure I indeed have WAVE files in the archive.
Am I doing something wrong or is the tool defective?
Unexpected input file 'AUDIO001.002'!
I've been trying with the command
scummvm-tools-cli --tool compress_sci --flac AUDIO001.002
after using the python tool above and I made sure I indeed have WAVE files in the archive.
Am I doing something wrong or is the tool defective?
- Praetorian
- ScummVM Developer
- Posts: 883
- Joined: Tue May 08, 2007 8:54 am
- Location: Greece
- Contact:
Re: SCI Compression Tool
From the source code, it looks like the compress_sci tool expects an input file strictly named in the format of "resource.*".SimSaw wrote: ↑Tue Jun 18, 2024 8:51 pm Sorry for bringing this old topic up again, but no matter what I try, I always get the error message:
Unexpected input file 'AUDIO001.002'!
I've been trying with the command
scummvm-tools-cli --tool compress_sci --flac AUDIO001.002
after using the python tool above and I made sure I indeed have WAVE files in the archive.
Am I doing something wrong or is the tool defective?
https://github.com/scummvm/scummvm-tool ... ci.cpp#L55
Copy the AUDIO001.002 and AUDIO001.MAP files out of the King's Quest game data folder, and rename the first (AUDIO001.002) to "resource.002". Keep the map file with its original name (AUDIO001.MAP).
Run the command line script:
Code: Select all
scummvm-tools-cli.exe --tool compress_sci --flac -o AUDIO001.flac resource.002
I was able to run the tool with either version of AUDIO001.002 resource (original or modified with the python script), without errors, and the result had the same size, but the actual contents were different.
Also keep in mind that the above script as pasted needs editing and a couple of fixes to run correctly.
My "corrected" version looks like this (with the kq5_dir also edited to point to the folder for my KQ5 data files):
Code: Select all
#!/usr/bin/env python2
from __future__ import with_statement
from struct import unpack, pack
from StringIO import StringIO
from wave import open as wave_open
# dir with KQ5 installed
kq5_dir = "C:\\Users\\adam\\Games\\sierra\\King's Quest Collection\\KQ5"
# original AUDIO001.002
kq5_aud = "AUDIO001.002.bak"
# original AUDIO001.MAP
kq5_map = "AUDIO001.MAP.bak"
# new AUDIO001.002
aud_new = "AUDIO001.002.new"
# new AUDIO001.MAP
map_new = "AUDIO001.MAP.new"
def parse_map():
with open('\\'.join((kq5_dir, kq5_map)), 'rb') as file: map = file.read()
return tuple((fields[0], fields[1] >> 28, fields[1] & 0x0fffffff, fields[2])
for fields in (unpack('<HII', map[n:n+10])
for n in range(0, len(map) - 10, 10)))
def convert_to_wav():
res_map = parse_map()
with open('\\'.join((kq5_dir, kq5_aud)), 'rb') as aud_in:
with open('\\'.join((kq5_dir, aud_new)), 'wb') as aud_out:
with open('\\'.join((kq5_dir, map_new)), 'wb') as new_map:
for num, volume, offset, size in res_map:
aud_in.seek(offset)
temp = StringIO('')
wav = wave_open(temp, 'wb')
wav.setparams((1, 1, 11025, 0, 'NONE', 'not compressed'))
wav.writeframes(aud_in.read(size))
new_offset = aud_out.tell()
aud_out.write(temp.getvalue())
new_map.write(pack('<HII', num,
((volume << 28) & 0xffffffff) + new_offset,
aud_out.tell() - new_offset))
new_map.write(pack('<HII', 0xffff, 0xffffffff, 0xffffffff))
if __name__ == '__main__': convert_to_wav()