Core Audio vs. MT32 Emu on a MAC

[fingolfin]

Note: I also have a MacBook Prof (since recently). Before, I still had a G4 PB, on which there was no hope to ever get the MT32 emu going fine, so I never bothered. I might take another look at the MT-32 emu code with a profiler at some point, maybe I can do something about it. But I don't want to raise your spirits on this prematurely, and I am not promising anything.

I am not sure whether this would fit into GSoC, but we'll see that next year, *if* we there is a GSoC 2009, and *if* we make it into it.

[fingolfin]

I just did a quick hack to verify that it's really a latency issue: If i modify OSystem_SDL::setSoundCallback() in backends/platform/sdl/sdl.cpp to set samples to 256 instead of the usual value (i.e. a tiny sound buffer), then the MT-32 emu seems to produce flawless output on my MacBook Pro.

In general, setting a smaller audio buffer ensures that audio output is more responsive, and has various advantages to other stuff in ScummVM besides the MT-32 emu.

So, why not just set it to a small value all the time? Because if your computer is not that fast, and scummvm takes up lots of CPU time, then a small audio buffer will increase stuttering. To quote from the SDL docs:

Good values seem to range between 512 and 8192 inclusive, depending on the application and CPU speed. Smaller values yield faster response time, but can lead to underflow if the application is doing heavy processing and cannot fill the audio buffer in time.

So on my system, I really have to use a buffer of at most 256 for the MT-32 emu to work fine. It already sounds terrible in some spots with 512 bytes. Right now we use 2048 bytes by default. I am not quite comfortable reducing the buffer that drastically over night (a factor of 8)... but it certainly would be a much simpler remedy than adding double buffering to the audio code.

[clem]

So on my system, I really have to use a buffer of at most 256 for the MT-32 emu to work fine. It already sounds terrible in some spots with 512 bytes. Right now we use 2048 bytes by default. I am not quite comfortable reducing the buffer that drastically over night (a factor of ... but it certainly would be a much simpler remedy than adding double buffering to the audio code.

Wouldn't it make sense to make this value definable via the launcher (or just the .ini?)

[fingolfin]

No, not really.

[hippy dave]

interesting, and nice to know there's at least a way around it for the time being
would it be worth asking people to try out the smaller buffer setting on their various machines, to see if it's worth considering as a permanent change?

[rented mule]

No, not really.

Shouldn't the MT32 emulation just be removed from the options then? With such a large default value, no computer in the world will allow a fluid stutterless emulation of the MT32. Not even my Mac Pro 4x3.0 GHz can handle the MT32 emulation.

I also find it amusing that the Mac versions outputted sound quality pretty much on par with the MT32. In fact, many audio aspects of the Mac versions of LucasArts games was utterly superior to the PC/Amiga versions. I wish I didn't have to resort to BasiliskII or Sheepshaver to enjoy the audio of all the LucasArts classics.

Just to make myself clearer, the music was pretty much the same on Mac as on the MT32 (with the exception on the excessive reverb effects on the MT32) and lots of sound effects on the Mac version don't even sound correct with the PC version on any audio setting. For example, when you get to choose the puzzle difficulty of MI2 right at the beginning, the Mac version had a very clear spit sound...the sound on the PC version is definitely not clear and does not sound like spitting. Another example, when the bartender is shining his beer mugs with spit, you hear some sound on the Mac version, not so on the PC version (unless the MT32 emulator in ScummVM is inaccurate since I've never played with a real MT32...I've only listened to samples and tried the MT32 emulation in ScummVM...and judging on the samples and the emulation, the sound is almost identical in quality to the Mac version).

My question is, how was the audio being handled in the Mac versions of Sierra and LucasArts games 18 years ago? And why isn't it possible to emulate it? Or is it possible? Setting the music driver to QuickTime (or CoreAudio) comes close but isn't quite what the Mac version sounded like.

Another oddity (and I noticed this today) is that the Mac data files and PC data files produce completely different music.

I used to play all my LucasArts games with the Mac data files. On the default setting, the sound is acceptable. But I wondered for a long time why the QuickTime and CoreAudio setting outputted such terrible sound. When I tried the PC data file, QuickTime and CoreAudio started sounding *much* better. So obviously the Mac data files had a different way of handling the music.

That said, the Mac version of all LucasArts games seem to sound better than the PC version (if we exclude MT32 sound) in ScummVM on the default setting (makes some sense). This is probably a little known fact and I'm sure most people are playing the DOS version on the default setting. The thought of that makes me shudder. But I guess it's human nature...whatever someone hears and sees first is what he/she cherishes and longs for. I'm sure people that played the EGA version of Monkey Island find it more comforting to replay in EGA resolution today rather than in VGA resolution (even though the VGA version was visually much better).

[fingolfin]

No, not really.

Shouldn't the MT32 emulation just be removed from the options then? With such a large default value, no computer in the world will allow a fluid stutterless emulation of the MT32. Not even my Mac Pro 4x3.0 GHz can handle the MT32 emulation.

That is just plain wrong. Many computers with much weaker hardware can use the MT-32 emulation just fine. It's a "problem" of the OS, not the hardware.

In particular, the sound buffer size is just fine, the problem only is that with CoreAudio, just-in-time sample generation is only an option if you can produce the data in virtually no time. That is, if we have to fill a 100ms sample buffer, and the code takes 10ms to generate that much data, then clearly it is able to produce data in (faster-than) realtime.

But if the OS only waits 1ms before it produces sound artifacts / gives up on the sound producer, then this is still too slow. You can compensate that by only producing 10ms of data, which then takes you only 1ms, and is fine. but on other systems, due to *their* peculiarities, it doesn't.

So, the *real* solution is a trick from the (programmer's) stone age: Double buffering. Generate the required data slightly ahead of time, and store it in buffer A. When the OS asks for audio data, give it buffer A, and start producing data in buffer B. Next time around, the two buffers swap roles.

I also find it amusing that the Mac versions outputted sound quality pretty much on par with the MT32. In fact, many audio aspects of the Mac versions of LucasArts games was utterly superior to the PC/Amiga versions. I wish I didn't have to resort to BasiliskII or Sheepshaver to enjoy the audio of all the LucasArts classics.

While I personally think that the MT-32 audio output is still superior (I have a real MT-32 to compare with), I understand your feelings.

Just to make myself clearer, the music was pretty much the same on Mac as on the MT32 (with the exception on the excessive reverb effects on the MT32) and lots of sound effects on the Mac version don't even sound correct with the PC version on any audio setting. For example, when you get to choose the puzzle difficulty of MI2 right at the beginning, the Mac version had a very clear spit sound...the sound on the PC version is definitely not clear and does not sound like spitting.

Actually, that depends on our audio driver. The PC version has the exact same sound sample in it, and can play it, too

My question is, how was the audio being handled in the Mac versions of Sierra and LucasArts games 18 years ago? And why isn't it possible to emulate it? Or is it possible?

Of course it is possible. It's just hard work to make this possible, and somebody has to do it. The Mac versions implemented music in various ways over the years. AFAIK, none used real MIDI, rather most (all?) used their own built-in set of samples (of rather low quality by modern standards, namely 8 bit data at 8000-22050 Hz). So emulating this boils down to (a) extracting the samples from the mac resource fork in a "portable" way, (b) reverse engineering and reimplementing their softsynthesizer code.

Personally, I never bothered to work on that, since IMO the samples provided by modern MIDI softsynths are of so much better quality. Of course others (like you) will have a different opinion (nostalgic feelings play an important role here -- after all, for many of us, they are an important reasons why we want to replay those games ). But apparently, so far nobody with the coding abilities for it shared your view. Which is, I believe, the main reason why it's not being supported by ScummVM.

Setting the music driver to QuickTime (or CoreAudio) comes close but isn't quite what the Mac version sounded like.

No it doesn't, sorry. Here your memory is playing tricks on you, I think -- at least when I hear them both in parallel, then to me they sound vastly different.
In fact, even the ScummVm QuickTime and CoreAudio drivers differ, if you pay a bit attention.

[bushwakko]

the double buffering principle seems simple enough, however the problem is that it needs threads, which isn't a part of scumm atm?

[md5]

the double buffering principle seems simple enough, however the problem is that it needs threads, which isn't a part of scumm atm?

Er, no, double buffering does not necessarily need more than one threads. You can just process/read more data than the MT-32 actually needs, and have it readily available to it

[bushwakko]

ok, I thought I remember reading that that was the problem.

[rented mule]

No it doesn't, sorry. Here your memory is playing tricks on you, I think -- at least when I hear them both in parallel, then to me they sound vastly different.
In fact, even the ScummVm QuickTime and CoreAudio drivers differ, if you pay a bit attention.

Oh...don't get me wrong I know they are different. The instruments are not at all the same. But it's a heck of a lot closer than listening to the music on the other settings. The range of instruments is greater. The sound is more vibrant.

I know CoreAudio and QuickTime differ. But like I said, you can hear notes that you would not otherwise hear on other settings. CoreAudio definitely has different instrument sounds and adds an reverb much like that of an MT32.

edit: alright...I've taken the liberty to resetup classic Mac OS emulation through BasiliskII (ugh) and record the MI2 theme. The recording isn't as good as it was considering it's coming from an emulator but close enough (ignoring the static and the fact that the emulator is incapable of producing stereo sound, you get a real feel of what the Mac version sounded like).

Mac sample: http://www.mediafire.com/?zoul01sydj7
MT32 sample from page 1 of this thread: http://www.crossfire-designs.de/downloa ... sland2.ogg

I know these kinds of comparisons are completely subjective but here's my opinion: the MT32 could synthesize smoother sounds and had interesting effects like reverb, the Mac synth code, however, seemed to be capable of outputting much more depth and could produce many more interesting instruments.

Going through many MT32 samples, you really can tell that most games made use of the same instruments and extensive use of reverb effects.

The Mac version of MI2 had some very distinct differences. The example that is probably the most evident from the two samples above is the bell at the very start of the MI2 theme. On the Mac you really hear the bell...and it actually sounds like a bell. On the MT32, the bell sounds like a tin music box. The other example is just a few seconds further...the pan flute on the Mac, sounds, well, like a pan flute. On the MT32, it just sounds like same old synth'ed notes put together.

I think the reason why most people prefer the MT32 sound (or other 'modern' softsynths) is that a) they never heard the Mac version (not saying everyone hasn't heard the Mac version, I know you have heard both the Mac and MT32 versions...just that there's an overwhelming majority of people that have heard one or the other but not both) and b) it produces the reverberation effect which smooths out the MIDI instead of cutting it short. But 'reverb' doesn't mean the quality of the sound is better. It makes things sound like you're playing in a huge concert hall or in a catherdral and, that, to most people is why people the sound. The MT32 makes things sound grandiose when deep down it's just plain old MIDI.

When I'm done with school and have some spare time I may one day look into lending the ScummVM team a hand to finally get the authentic Mac sound. I'd hate to see it be ignored just because some people truly believe the MT32 (and modern-day softsynth) sound was better.

[hippy dave]

I just did a quick hack to verify that it's really a latency issue: If i modify OSystem_SDL::setSoundCallback() in backends/platform/sdl/sdl.cpp to set samples to 256 instead of the usual value (i.e. a tiny sound buffer), then the MT-32 emu seems to produce flawless output on my MacBook Pro.

i just got around to trying this out myself, and can confirm it works nicely on my macbook

[Spinning Chrysalis]

I might have an idea. My G4 was stuffed with extra RAM (2 additional modules) and it used to run the Monkey Island 2 music with quality as high as that of the MP3s available on Scumm Bar or International House of Mojo. However, a few months ago one of the RAM modules went bad and I had to remove it from the computer. Now the MI2 music sounds far more primitive. Maybe if you add more RAM the computer can handle the higher-quality MIDI? I don't know enough about the inner workings of computers, but it seems logical.

[fingolfin]

Err, no, that is not logical in this context. It makes absolutely no sense, in fact

[Ceri Cat]

I have to agree with Fin, MIDI isn't a very CPU or RAM intensive application, the only time RAM makes a difference is when you're using higher quality samples for the instruments, such as some of the soundfonts I use for composition which use up a fair amount of cache by default settings. And even then a lot of that impact is still on the hard drive with the SF being in the system cache usually rather than active memory. And I can't see Mac OS X regardless of how advanced it might seem at times doing something automatically which hasn't been implemented elsewhere with automatically switching between substantially higher and lower quality samples of instruments for MIDI and my friends that work with Macs professionally agree with my thoughts that this is unlikely to be the case as it interferes too much with the user experience.