Speech Recognition?

[FlvZ2gWPdj9w71Fd97it]

Some older games had text-based interfaces. Would it be possible to use speech recognition with newer games too? Perhaps as a ScummVM add-on or feature?

Are there any speech recognition or standardized keyboard inputs/shortcuts for ScummVM or the supported games?

[digitall]

To be honest, this has been on my personal task list for a while... With SDL providing sound capture function, we should be able to add a simple voice command backend to emulate keyboard and mouse control... maybe based on CVoiceControl: http://www.kiecza.net/daniel/linux/

Not sure how functional the result would be, but could be an interesting experiment for someone... or maybe as a future GSoC task.

[digitall]

https://arjo129.wordpress.com/experiments/%C2%B5speech/ could also provide a lightweight basis for a limited vocabulary command recogniser...

[icanntspell]

Android seems to have something build in for this.

http://developer.android.com/reference/ ... nizer.html

It sounds like a nice idea, but I don't see myself playing LSL using this At least not in public

[md5]

Even if there is speech recognition, I doubt it'll be THAT useful, as you have to move around with the mouse anyway. It will save some typing, but all the mouse-related controls won't be affected.

[digitall]

I was mainly going to try doing this for general code twiddling fun, but I wondered about the utility for profoundly physically disabled gamers ie. using a http://tetramouse.com/ say...

I was intending to allow "MOUSE-HOLD. UP. DOWN. LEFT. RIGHT. SMALL. CLICK. RIGHT-CLICK. DRAG. RELEASE. MOUSE-RELEASE." as a commandset for controlling mouse movement and control by a limited vocab recogniser and then keyboard control by "KEYBOARD-ON. KEYBOARD-OFF" and a more general purpose recogniser and/or a way of voice-typing.

[FlvZ2gWPdj9w71Fd97it]

It's possible to control the mouse with things like VoiceFinger or Windows Speech Recognition macros. Though, my experience is that doing so is quite slow. "A New Beginning" does have a very helpful keyboard shortcut to show all hotspots (space bar). When used with WSR or VF it's possible to play without very slowly scanning the screen hunting for hotspots.

Given the limited vocabulary necessary for most adventure games it seems possible to leverage Windows Speech Recogition (on Windows) or other speech engines since the verbs, hotspot names, and objects should be known to ScummVM. Perhaps there could be a module or plugin system to work with different speech systems.

Even just adding keyboard input or shortcuts for verbs and objects would be a big help. Those can often be spoken using WSR or SphinxKeys.

AbleGamers is an organization with some great guidelines for tips to improve accessibility.

[DrMcCoy]

since the verbs, hotspot names, and objects should be known to ScummVM

That's highly game/engine specific. Some engines do, some probably don't. And there's nothing in ScummVM to abstract that, to bind the hotspot concepts of different engine together and pass it somewhere.

For example, for the gob games, ScummVM (or rather, the gob engine in ScummVM; the info doesn't get passed to any common code) only knows about general hotspots. When the mouse enters a hotspot, when it leaves a hotspot, or when a mouse button is clicked inside a hotspot, a script is triggered. Those scripts then do things like displaying the hotspot name. Opaquely, without the engine code knowing anything about that from a high-level perspective. The engine code doesn't know what those hotspot are, what they mean to the game.

Potentially, an engine could go even further: the scripts could just periodically query the current mouse position and state. Where the hotspots are could also be handled by the scripts. I don't know if any of our engines does that already, but it's certainly possible.

[FlvZ2gWPdj9w71Fd97it]

Thanks for the quick reply. I didn't realize ScummVM had isolated engine code. Though, I don't think the problem is insurmountable. Creating such an abstraction to work with the most widely used engine would be my first step. Unfortunately my day job leaves my hands too tired to get into the source at this time.

Regardless, I look forward to whatever future accessibility improvements do get incorporated.

[md5]

FlvZ2gWPdj9w71Fd97it: We're happy for the work you're putting in ScummVM would you mind changing your nick to something that's readable by human beings? Thanks!

[Serious Callers Only]

Extracting that information from when it can be extracted into a common parser would be great indeed.

[Serious Callers Only]

Although it would probably require some expansion of older games primitive pathfinding systems maybe, to be used completely hands off. And then there are the puzzles that require positioning...
A invasive change to the wrong engines, although i wonder if the parser based AGI/SCI wouldn't be easier.