Slow OpenGL mode in SCUMM

UrQuan · Post by **UrQuan** » Wed Oct 10, 2018 10:22 am

The forum rules state not to report bugs here, but I have to assume that this is known and is not considered a bug, or that there is no trivial fix in any case.

No steps to reproduce needed. Everyone in the Speedy Adventures discord has this problem, without exception. To my knowledge, all SCUMM-engine games are affected, as are all game versions. I cannot say if any other engines are affected but none that I have tested.

First, the screen transitions are noticeably slower than on SDL, but the slow and laggy mouse cursor is the most problematic. It is unbearable for speedrunning, or even for casual play once you are used to the non-laggy cursor.

If you don't feel a difference; play for a minute on OpenGL, then play the same segment on any other scaler.

I am left scratching my head here since I haven't found anything reported on this. In any case, we cannot play on OpenGL for this reason, though many people like me would like to do so for the better scaling at arbitrary resolutions.

Raziel · Post by **Raziel** » Wed Oct 10, 2018 11:33 am

iirc the OpenGL implementation is rudimentary and only serves as a window/screenmode resolution fits-all.
No optimization done for any of the games, i think the game engines doesn't even know they can use OpenGL for rendering, or so i remember, might be wrong though.

I've done some tests myself and there is absolutely no difference in speed, so i went back to the "2x" scaler

UrQuan · Post by **UrQuan** » Wed Oct 10, 2018 12:05 pm

I see...

Is there a possibility to implement pixel-perfect output for SDL, like OpenGL has?

I dread going back to ScummVM now that I've gotten used to DOSBox ECE. If something like this was implemented for ScummVM I wouldn't need to use DOSBox again: https://www.vogons.org/viewtopic.php?f=32&t=49160

I play in fullscreen that is. When you say you play on 2x I understand you mean windowed, to avoid this whole scaling debacle. I mean, isn't the window just too small, even at 3x... let alone for people with higher resolution monitors?

I've done some tests myself and there is absolutely no difference in speed, so i went back to the "2x" scaler

Well, there is no difference in speed per se, but the screen transitions are noticably slower, along with the mouse issue (let's say: every sceen transition in The Dig, or escaping out of the opening cutscene in The Dig or Fate of Atlantis).

Let me stress again that OpenGL seems to work flawlessly with every other engine that I've tested. Only SCUMM is a problem.

Raziel · Post by **Raziel** » Wed Oct 10, 2018 12:46 pm

I need to leave your question to a dev, as i don't really know the internals of that specific problem.

I play in fullscreen too, but using 1x gives me a just too small list to work with my many games, and 3x is too big ad looks "off".

I settled with 2x

Post by **criezy** » Wed Oct 10, 2018 5:11 pm

I have never noticed speed issues with OpenGL on macOS, but maybe this is specific to Windows (you didn't specify what OS you and the other users in the Speedy Adventures discord are using). Or this might be because I have not played a SCUMM game for a while. I will try to remember to try this tonight...

In any case pixel-perfect rendering was added at the same time for both the OpenGL and SDL graphics modes (back in July). You can see a description in the Github Pull Rest (I have not yet updated the user manual on our wiki). This will be included in ScummVM 2.1.0 when we release it, but for now it is only available in daily builds. If you say that you have the pixel-perfect option for OpenGL though, this probably means you already have such a version (or that you are not really using pixel perfect rendering - but maybe nearest neighbor interpolation, which if you are lucky with your screen resolution might end up giving a pixel-perfect display).

Raziel · Post by **Raziel** » Wed Oct 10, 2018 6:01 pm

I just tried Sam & Max with OpenGL, together with the Pixel Perfect scaler and the mouse pointer is as fast as it ever was.

AmigaOS4 here

UrQuan · Post by **UrQuan** » Wed Oct 10, 2018 6:06 pm

criezy wrote:I have never noticed speed issues with OpenGL on macOS, but maybe this is specific to Windows (you didn't specify what OS you and the other users in the Speedy Adventures discord are using). Or this might be because I have not played a SCUMM game for a while. I will try to remember to try this tonight...

In any case pixel-perfect rendering was added at the same time for both the OpenGL and SDL graphics modes (back in July). You can see a description in the Github Pull Rest (I have not yet updated the user manual on our wiki). This will be included in ScummVM 2.1.0 when we release it, but for now it is only available in daily builds. If you say that you have the pixel-perfect option for OpenGL though, this probably means you already have such a version (or that you are not really using pixel perfect rendering - but maybe nearest neighbor interpolation, which if you are lucky with your screen resolution might end up giving a pixel-perfect display).

Ah, sorry for not mentioning that... Windows, indeed.

While it is very noticeable, the laggy mouse cursor you have to feel more so than see, while the slow screen transitions I could post a clip of if needed.

Oh, you must excuse my ignorance on this. I meant exactly that; nearest-neighbor, but indeed it looks good to me, and doubly so compared to my monitor's or graphics card's bilinear filtering if I just play in fullscreen or maximized window with the pixel-duplication scaler.

In any case, that is what I would like for SDL, and what was a much requested feature for DOSBox as well: "good enough" scaling like that, without filters and at arbitrary resolutions. I'm also wholly ignorant on if this is a tall task to implement or not.

Oh yeah, that's right, the daily build does have that pixel-perfect "stretch mode" option, but unless I'm missing something here, it is a little bit misleading. It just scales the image integrally up to your screen resolution, which you could also do manually (1x, 2x, 3x...) Doesn't have anything do with rendering per se?

UrQuan · Post by **UrQuan** » Wed Oct 10, 2018 6:13 pm

Raziel wrote:I just tried Sam & Max with OpenGL, together with the Pixel Perfect scaler and the mouse pointer is as fast as it ever was.

AmigaOS4 here

Hah, I'm not doubting your judgement on the mouse cursor thing, but were the screen transitions as fast as well compared to SDL? I'm talking about the "fade out / fade in" effect which I'm pretty sure is used in all SCUMM games.

UrQuan · Post by **UrQuan** » Wed Oct 10, 2018 6:53 pm

I have a very quick comparison here with the screen transitions. This doesn't tell you anything about how it just feels slow and weird in general, but it's a quick benchmark for anyone to test if they have this problem.

SDL
https://www.youtube.com/watch?v=2qAq-UUTZ1Y
That first fade-out takes 174 ms

OpenGL
https://www.youtube.com/watch?v=-31JMgDsdzU
That first fade-out takes 254 ms

Post by **criezy** » Wed Oct 10, 2018 7:25 pm

UrQuan wrote:Oh, you must excuse my ignorance on this. I meant exactly that; nearest-neighbor, but indeed it looks pretty good to me, and doubly so compared to my monitor's or graphics card's bilinear filtering if I just play in fullscreen or maximized window with the pixel-duplication scaler.

In any case, that is what I would like for SDL, and what was a much requested feature for DOSBox as well: "good enough" scaling like that, without filters and at arbitrary resolutions. I'm also wholly ignorant on if this is a tall task to implement or not.

Oh yeah, that's right, the daily build does have that pixel-perfect "stretch mode" option, but unless I'm missing something here, it is a little bit misleading. It just scales the image integrally up to your screen resolution, which you could also do manually (1x, 2x, 3x...) Doesn't have anything do with rendering per se?

The option to use either nearest neighbour or bilinear filtering when stretching the game display to your screen resolution is controlled by the "Filter graphics" option. When toggled on ScummVM uses bilinear filtering and otherwise it uses nearest neighbour. This should work for both SDL and OpenGL graphics mode (and it does for me). And this option should already be present in the 2.0.0 release.

The new stretch mode works in combination with the Filter graphics option. If for example the stretch mode is to not scale the display, then the filter option should have no impact. On the other hand if the stretch mode is pixel-perfect scaling, then when using nearest neighbour you will get big pixels that all have the same size, while with Fit to window stretch you will get big pixels that might have slightly different sizes. When using one of the SDL x2 or x3 graphics mode, this is applied first before the stretching kicks in. There is some discussion about it in this other forum thread. The "Fit to window" stretch mode is the same behaviour you have in older ScummVM versions (such as version 2.0.0) that do not have the stretch option.

Note that older ScummVM version that use SDL 1.2 instead of SDL 2 do not behave in the same way, and in particular do not have the "Filter graphics" option for SDL graphics modes.

Post by **criezy** » Wed Oct 10, 2018 9:48 pm

Also by curiosity I just recorded that intro with both OpenGL and SDL 2x graphics modes in fullscreen and checking frame by frame I get exactly 11 frames for the first fade out for both (and the recording is at fixed 60 FPS for both which gives a 183 ms transition).

And if anything the cursor movement actually look smoother with OpenGL for me. But there is not a big difference.

UrQuan · Post by **UrQuan** » Thu Oct 11, 2018 10:55 am

criezy wrote:
UrQuan wrote:Oh, you must excuse my ignorance on this. I meant exactly that; nearest-neighbor, but indeed it looks pretty good to me, and doubly so compared to my monitor's or graphics card's bilinear filtering if I just play in fullscreen or maximized window with the pixel-duplication scaler.

In any case, that is what I would like for SDL, and what was a much requested feature for DOSBox as well: "good enough" scaling like that, without filters and at arbitrary resolutions. I'm also wholly ignorant on if this is a tall task to implement or not.

Oh yeah, that's right, the daily build does have that pixel-perfect "stretch mode" option, but unless I'm missing something here, it is a little bit misleading. It just scales the image integrally up to your screen resolution, which you could also do manually (1x, 2x, 3x...) Doesn't have anything do with rendering per se?
The option to use either nearest neighbour or bilinear filtering when stretching the game display to your screen resolution is controlled by the "Filter graphics" option. When toggled on ScummVM uses bilinear filtering and otherwise it uses nearest neighbour. This should work for both SDL and OpenGL graphics mode (and it does for me). And this option should already be present in the 2.0.0 release.

The new stretch mode works in combination with the Filter graphics option. If for example the stretch mode is to not scale the display, then the filter option should have no impact. On the other hand if the stretch mode is pixel-perfect scaling, then when using nearest neighbour you will get big pixels that all have the same size, while with Fit to window stretch you will get big pixels that might have slightly different sizes. When using one of the SDL x2 or x3 graphics mode, this is applied first before the stretching kicks in. There is some discussion about it in this other forum thread. The "Fit to window" stretch mode is the same behaviour you have in older ScummVM versions (such as version 2.0.0) that do not have the stretch option.

Note that older ScummVM version that use SDL 1.2 instead of SDL 2 do not behave in the same way, and in particular do not have the "Filter graphics" option for SDL graphics modes.

I'm starting to see what's going on here. I had only played DOSBox in windowed recently so I didn't realize that its pixel-perfect output mode does exactly what ScummVM does, only that DOSBox has a native 4x scaler, which just happens to be very close to fullscreen at 1080p.

*But* here is the thing; unlike ScummVM SDL, DOSBox's nearest-neighbor doesn't leave any artifacts, neither on SDL nor on OpenGL (it looks very close to ScummVM OpenGL basically).

Let me show you what I see exactly: (fullscreen 1080p, aspect ratio correction, stretch mode <default> meaning fit to window)

3x Filter graphics OFF: https://i.imgur.com/IWB4bhM.png
3x Filter graphics ON: https://i.imgur.com/kcHh6kO.png

OpenGL Filter graphics OFF: https://i.imgur.com/PXytzIT.png
OpenGL Filter graphics ON: https://i.imgur.com/GUnX46B.png

DOSBox surfacepp: https://i.imgur.com/YMfiAFd.png
DOSBox surfacenb: https://i.imgur.com/CcKSugx.png
DOSBox openglenb: https://i.imgur.com/NVauX7L.png

To further complicate these comparisons, imgur compresses the images a little bit too much. I uploaded them to Google Drive: https://drive.google.com/open?id=1UA9QL ... 6Tx8aPx9QS

ScummVM SDL has some aftifacts, but at least it's sharp, whereas Filter graphics only makes everything fuzzy and terrible for me. If this is the intended beheavior, I am going to be upfront and suggest that option to be removed, lest someone actually use it....

"filter graphics" sounds like something fancy that you actually want to use.

Otherwise, Filter graphics OFF with OpenGL looks great: Not fuzzy, no scaling artifacts.... It is seemingly equivalent to DOSBox ECE nearest-neighbor.

However, I've come to realize an alternative solution here: just adding a 4x scaler. I have to assume that it's not trivial to implement or it would have been done already but.... It would circumvent this whole scaling debacle. From what I've read it's been a requested feature in the past and it used to be a requested feature for DOSBox too. Native 3x is just too small even at 1080p, let alone for people with higher resolution monitors nowadays.

Actually, while staring at all of these screenshots now.... I see that ScummVM SDL's pixel-duplication even at the native resolution is not quite as sharp as DOSBox pixel-perfect or ScummVM OpenGL. Is this intended?

3x Filter graphics OFF pixel-perfect scaling: https://i.imgur.com/dRrMfeh.png

UrQuan · Post by **UrQuan** » Thu Oct 11, 2018 11:13 am

criezy wrote:Also by curiosity I just recorded that intro with both OpenGL and SDL 2x graphics modes in fullscreen and checking frame by frame I get exactly 11 frames for the first fade out for both (and the recording is at fixed 60 FPS for both which gives a 183 ms transition).

And if anything the cursor movement actually look smoother with OpenGL for me. But there is not a big difference.

Interesting. So could it be that those two problems are unrelated? If you indeed have the slow cursor that is.

We all hate the "smooth" cursor so much that I have perhaps overstated how bad it is. It is still relatively subtle, especially if you have not played a particular game for an extended period of time. I can assure you though that it is unbearable for speedrunning, or even for casual play once you are used to the fast cursor.

Post by **criezy** » Thu Oct 11, 2018 12:56 pm

To clarify, in my case the cursor does not seem to be slower in one case than the other. It just seems to be more jerky in the SDL case, as if the display was updated less often.

UrQuan · Post by **UrQuan** » Thu Oct 11, 2018 3:23 pm

Hmm, I think I found something here.

Look at these three images:

DOSBox SDL 3x with aspect ratio correction: https://i.imgur.com/89R3Egb.png
ScummVM SDL 3x without aspect ratio correction: https://i.imgur.com/OlZ6odT.png
ScummVM SDL 3x with aspect ratio correction: https://i.imgur.com/EKV5yoq.png

Google Drive for lossless images: https://drive.google.com/open?id=152ZYl ... OvHPh2V_U-

For whatever reason, ScummVM SDL with aspect ratio correction doesn't render the original image properly (even more noticeable on 2x and 1x). Once you have that "anti-aliased" image, those scaling artifacts appear when you then scale to arbitrary resolutions with nearest neighbor. Whereas if you have a clean or sharp "source" image, nearest neighbor scales it without artifacts exactly like ScummVM OpenGL and DOSBox SDL.

If someone could take a look at this.... that would be great. Not only would it solve the scaling issue, but in my opinion it would obsolete the need for a 4x (or any) scaler, since nearest neighbor would then be good enough on SDL as well. It would even obsolete my original issue with OpenGL and SCUMM, given that people could just use SDL instead.