Scaling question

[clamshellpocket]

Hi all,

I am currently examining the ideal scaling settings for ScummVM for a possible (new) handheld application.
I am a little puzzled, because of the unexpected results (the same screenshot quality for both graphical modes ).
I am hoping sev, or someone of this great community can help me out to understand my findings?
I have read the section "Understanding the graphics settings" via this URL:
https://docs.scummvm.org/en/latest/adva ... phics.html

---

Before that:
I was intrigued by the concept of Integer Scaling, especially compared to Bilinear Scaling.
A famous 'Mario' picture is flooting around the Internet comparing both methods via this URL:
https://www.techpowerup.com/img/kLGzbamqwTe0uFMb.jpg
I can grasp the concepts of Integer scaling, multiple the pixel amount with a whole number (integer) so the result will be a larger, crisp-clear picture.
Only possible downside are 2 extra black bars (top/botom), if the display-height isn't compatible with this integer number.

There are also tons of examples what the power of Integer Scaling is and why you want to make use of this;
If your diplay unit has more-or-less the right resolution.
For example: https://custompc.raspberrypi.com/articl ... -explained

---

Now onwards to my question:

ScummVM has, next to the great Pixel Aspect-ratio Correction (PAC) two main graphical modes:
1. OpenGL
2. SDL Surface

With the OpenGL mode selected ScummVM can scale the picture to a window, lets say 1280x720 px using bilinear interpolation.
With the SDL Surface mode selected ScummVM can scale the picture by a whole number (integer), let say 640x400 px (PAC off) or 640x480 px (PAC on).

First thing first, but please correct me if I am wrong, because I want to learn:
I was expecting SDL Surface to be an equivalent of Integer Scaling AND OpenGL to be an equivalent of Bilinear Scaling,
am I right in this assumption??

1. as such I was expecting the SDL Surface mode (selected to scale 2X) to be pixel-perfect on a 640x480 px display.
2. as such I was expecting the OpenGL mode (selected to vertical fit a 720 px window) to be a little pixel-jagged centered on a 1280x720 px display.

But when doing both methods real live, on ScummVM v2.6.0 today I got these results (real screenshots), see the 4 pictures in the attachment!

Perhaps I have someting wrong with my eyes, but it seems to me that both results are great.

I hope someone is not thinking right now, 'so what's your problem then'

I am in the middle of doing reseach to implement a TN display 5.0 inch 640x480 ... I also have a super nice AMOLED display 5.5 inch at 1280 x 720px.
I am puzzled right now what display to use... I always figured that a 640x480 px 4:3 display is superior to another aspect ratio because I didn't whant to use the scale to fit display height feature...

I hope someone can eloborate on this?

cheers Jeroen Wolf
www.clamshellpocket.com

[clamshellpocket]

When I read back my post I noticed, although very minor, a little difference:

the SDL Surface scaling (640x480 px) had a little better color reproduction (colors are somewhat more vibrant) then it's OpenGL scaling (1280x720 px) counter part.

But still no jagged edges... I hope someone, or lots off people can post their opinion about this

Thanks, Jeroen

[criezy]

I was intrigued by the concept of Integer Scaling, especially compared to Bilinear Scaling.
I can grasp the concepts of Integer scaling, multiple the pixel amount with a whole number (integer) so the result will be a larger, crisp-clear picture.
Only possible downside are 2 extra black bars (top/botom), if the display-height isn't compatible with this integer number.

Rather than Interger Scaling, we usually compare Bilinear Interpolation with Nearest Neighbour Interpolation.

Integer Scaling is not just about using an integer scale factor. There are two separate things to consider when you are scaling graphics:
1. The scale factor, for example 4.5 if you scale a game from 300x200 to 1350x900.
2. The way the new pixels are interpolated from the original ones.

As I understand it, Integer Scaling means that you have both an integer scale factor, and that the interpolation uses nearest neighbour interpolation. But you could have an integer scale factor and use another interpolation method, and that could make the result blurry (depending how pixels are interpolated). And you can have a non-integer scale factor and use nearest neighbour interpolation, which gives crisp graphics, but with "scaled" pixels of different sizes (so it may look a bit uneven).

ScummVM has, next to the great Pixel Aspect-ratio Correction (PAC) two main graphical modes:
1. OpenGL
2. SDL Surface

With the OpenGL mode selected ScummVM can scale the picture to a window, lets say 1280x720 px using bilinear interpolation.
With the SDL Surface mode selected ScummVM can scale the picture by a whole number (integer), let say 640x400 px (PAC off) or 640x480 px (PAC on).

First thing first, but please correct me if I am wrong, because I want to learn:
I was expecting SDL Surface to be an equivalent of Integer Scaling AND OpenGL to be an equivalent of Bilinear Scaling,
am I right in this assumption??

No you are not. Both can do Integer Scaling, but this is actually easier to achive with the OpenGL graphics mode. And both can use bilinear interpolation.

In ScummVM the image goes from:
original -> scaled -> aspect ratio correction -> stretched

For both the aspect ratio correction and the stretching steps, ScummVM has the ability to use either nearest neighbour or bilinear interpolation. This is controled by the Filter graphics option. And that option is available for both the OpenGL and SDL Surface graphics mode. So you can have sharp pixels or blurred pixels with both of those modes.

The scaling step uses a custom interpolation that may neither be nearest neighbour nor bilinear. But I will come back to that later.

For the OpenGL mode, the scaling step is removed, and the aspect ratio correction and stretching are done together in a single step. So you go from original to final image in a single step. That makes that mode easier to understand and control. If you turn off the Filter graphics option (which means it uses nearest neighbour interpolation), and the final size is a multiple of the original size, then you get what is called Integer Scaling.

The selected stretch mode and aspect ratio correction both control the final size. If you use the Even Pixels stretch mode, you are guaranteed to get an integer scaling factor for both the width and the height (although it might be different, with for example a x4 factor for the width and x5 factor for the height). If you use the Pixel Perfect stretch mode you are guaranteed to get an integer scaling factor for the width, but you may get a non-integer scaling factor for the heigh if aspect ratio correction is turned on (for example x4 for the width and x5.067 for the height). The Center stretch mode does not scale the display at all. All the other stretch mode likely result in non-integer scale factors (unless you are lucky).

For the SDL Surface mode, the process is more complicated due to the scaling step that occurs before aspect ratio correction and stretching. That scaling step applies an integer scaling factor (selected by the user in the options), but the interpolation method depends on the selected scaler. Normal mode means nearest neighbour, but all other modes (e.g. AdvMAME or HQ) use different interpolation methods (neither nearest neighbour nor bilinear). Also because the aspect ratio correction is done on its own and not together with the stretch mode, it will usually introduce a non-integer scaling of the display height when turned on.

However for the SDL Surface mode you can use the Center stretch mode to remove any scaling at that step, and use the user defined scale factor only for the scaling step. In that case if PAC is off you will get an integer scale factor, and if you select the Normal mode you get what is called Integer Scaling. But if you want to turn PAC on, then the only way to get an Integer Scaling is to use a scale factor of x5. Which requires a screen with a resolution of at least 1600x1200. On the other end, the OpenGL mode with Even Pixels stretch mode and PAC turned on can achieve that resolution but also for example 1280x1000 (x4 horizontally and x5 vertically) or 1920x1400 (x6 horizontally and x7 vertically) and thus offer more flexibility on the final size (at the cost of having slightly incorrect aspect ratio).

[criezy]

In ScummVM the image goes from:
original -> scaled -> aspect ratio correction -> stretched

For the OpenGL mode, the scaling step is removed.

Actually I forgot that this was changed in version 2.6.0. In that version scalers are also available in OpenGL mode. So what I wrote assumes a 1x Normal scaler. Otherwise this has to be factored in.

[clamshellpocket]

Hi criezy, thanks so much for you extensive answer before I reply with more 'meat-to-the-bone' let me comprehend fully your provided answers...
Ps I do appreciate your help and your provided time a lot
I actually also made a little mistake when writing my orginal question... I more-or-less (foolishly) choosen 1280x720 px as the other display I actually don't want to target, but the two selected games run at 320x200 px nativly... it was actualy an integer value, because I set the PAC to on (320x240 px x 3 = 960x720 px)

Well speak soon if I have more questions, ok?

In the meanwhile to you and the rest of this community:

I am on the hunt for a 4.5inch 640x480 display.
After looking for allmost a month I have found 0 models.
Is there anyone who perhaps know of a model in existence ???

Jeroen

[clamshellpocket]

Hi criezy, I have done some research on nearest-neighbor interpolation, bilinear interpolation and integer scaling...

ALl concepts are super interesting,
I also did a comparisment between SDL Surface and OpenGL, these methods (although different) proof excellent contenders for my project

Ps, I have given up on my quest to find a suitable >3,5 inch or <5.0 inch 640x480 px display.
One famous YT'er (of the handheld scene) stated that a 4:3 aspect ratio display with a 4.5 inch screen size would be his personal sweet spot: https://www.youtube.com/watch?v=IUjJa1pA9tE&t=403s
I totaly agree with him, also regarding pocketability,
but It's just impossible to find this size in combination with this resolution anywhere in the World!

When reviewing my next best option I came to the conclusion that a 5.0 inch 800x480px (5:3), or a 5.5 inch 1280x720px (16:9) isn't that bad
I made a comparisment (actual display sizes) .pdf of my investigated displays:
http://www.clamshellpocket.com/promo/displays.pdf
The 5.36 inch (1920x1152px) was actually not that good for the intended Point and Click games.
The 1280x720px is an AMOLED display, where the 800x480px is a TFT (TN) display....
I have tested the AMOLED (TT550RAN20A) in real-life, it's absolutely gorgeous.
Combined with a perfect (maxed-out) screen height (when using a pixel scaling factor of 3x and PAC on) I'm thinking of going for this setup.
(I take the wasted black bars left/right for granted.)
I hope someone, from this community perhaps can sent me their 2 cents on this topic??

Now on-wards to redisign my controls... a 4:3 housing is a little different than a 16:9 housing
On the up-side of things, I now have space for the extra 20 character MT-32 display bar on the bottom part of the clamshell design
(I want to implement a Roland MT-32 (though FPGA) core for this handheld.)
http://www.clamshellpocket.com/promo/bar1.png
http://www.clamshellpocket.com/promo/bar2.png
See also this excellent YT video from the 8-Bit Guy
https://www.youtube.com/watch?v=BMmFcs-_4x4&t=218s

Cheers and thank for your insights earlier, it really helped to get things in perspective.... Jeroen

http://www.clamshellpocket.com

[sev]

I think, 1280x720px screen is a great option. It will let you run almost every game supported by ScummVM, only a few Wintermute games run on 1280x800 or 1280x768. And the games with smaller resolution will have better upscaling possibilities. And of course, larger screen will help with pixel hunting on some of the adventures.


Eugene

[clamshellpocket]

Super, sounds all good Eugene
thx for your reply...