If not, I will create a blog this weekend and start posting my progress. In a couple of years when all those DLL's are decoded we'll have an offical ScummVM release party!
Blade Runner Reverse Engineering
Moderator: ScummVM Team
Blade Runner Reverse Engineering
Is anyone actively working on reverse engineering Westwood's Blade Runner?
If not, I will create a blog this weekend and start posting my progress. In a couple of years when all those DLL's are decoded we'll have an offical ScummVM release party!
If not, I will create a blog this weekend and start posting my progress. In a couple of years when all those DLL's are decoded we'll have an offical ScummVM release party!
First of all, I would start with formal details, you need to satisfy everyone from the team (whatever that means), even guys which never REed any engine. So get a clear green light BEFORE investing years of your work into engine.
Overwise they will definitely say that OMG IT'S 3D NO NO NO, SCUMMVM IS 2D GO RESIDUAL (not even that rendering of voxels could be done in one pass, like they do in every engine, because no colorkey was ever supported in graphics lol) Then, if you will try to automate something, because there's dozens of similar c++ "scripts" in those dlls, then they will get angry, just because everything in scummvm must be written by hand (wtf).
Irrational? Yes. Motivating? No. Waste of time arguing with picky guys resting on their laurels.
So I would not recommend you to repeat my dreamweb engine mistakes. That's why I stopped REing of blade runnner. I don't want to go through all this <s-word> again. There's plenty of things more intertaining than doing it again lol. Thank you for your effort.
If you still want to do it, Eugene(or I, as he'd say) will set up private repo for you. But get a green light first. Good luck.
Overwise they will definitely say that OMG IT'S 3D NO NO NO, SCUMMVM IS 2D GO RESIDUAL (not even that rendering of voxels could be done in one pass, like they do in every engine, because no colorkey was ever supported in graphics lol) Then, if you will try to automate something, because there's dozens of similar c++ "scripts" in those dlls, then they will get angry, just because everything in scummvm must be written by hand (wtf).
Irrational? Yes. Motivating? No. Waste of time arguing with picky guys resting on their laurels.
So I would not recommend you to repeat my dreamweb engine mistakes. That's why I stopped REing of blade runnner. I don't want to go through all this <s-word> again. There's plenty of things more intertaining than doing it again lol. Thank you for your effort.
If you still want to do it, Eugene(or I, as he'd say) will set up private repo for you. But get a green light first. Good luck.
-
DrMcCoy
- ScummVM Developer
- Posts: 595
- Joined: Sat Dec 17, 2005 1:33 pm
- Location: Braunschweig, Germany
- Contact:
Okay, after deleting the angry rant I had up here for a minute, and composing myself, let me try for a more elaborate answer to whoozle.
It is not. ResidualVM is a lovely project handled by nice people. I may not 100% agree with all their decisions, but that's true everywhere.
The issue of whether a game is a "ScummVM-worthy" 2D game, or "ResidualVM-worthy" 3D game is sometimes blurry and people in the team may disagree on specifics, yes. For me personally, I draw the line at games that use vertex-geometry as still handled by current-gen graphics cards. So Gabriel Knight 3 is definitely a game for ResidualVM. Discworld Noir, despite its extended Tinsel-engine, and despite only using vertices for the main character (IIRC), is also a ResidualVM-game in my mind.
Where does that leave games using voxel-graphics like Blade Runner? Frankly, I have no idea. Can the voxel geometry be easily transformed into vertex geometry? If yes, then maybe going the ResidualVM route would be easier. If not, the voxel geometry would need to be rendered into sprite buffers, and then the rest of the engine is purely 2D? In that case, that seems like a fit for ScummVM for me. YMMV.
In either case, neither of the two options, going ScummVM or going ResidualVM, is better than the other one. I love both projects and they both have their niches. Sure, those may overlap on occasion, but so what?
What the code should not look like is assembly, or even some high-level assembly dialect. Just taking the code, throwing it through a disassembler and pasting the result verbatim into a .cpp file is, in my opinion, not what ScummVM stands for.
When I RE a game, I look at the disassembly from a high-level perspective. I try to understand what it does, and then replicate the behaviour in C++ code. The key there is the understanding of what a piece of code does: this is what gives the whole work meaning for me; it's like solving a puzzle.
This also helps in maintaining the engine later. When something gets accidentally broken because of some change of common code in ScummVM, or maybe a new backend for some new device, it's far easier to pinpoint the problem when you have readable native C++ code. When all you have is some assembly-like mess, this will be far more difficult. Ditto when maybe later on some new game using a similar (or the same) engine is discovered (see for example the newly discovered advertisment game Croustibat for the gob engine).
Now, did I take some shortcuts in my code from time to time? Yes, of course. But I'm not proud of them, and I try to minimize that wherever I can.
You make that sound like moving something over to ResidualVM is some kind of punishment.whoozle wrote:OMG IT'S 3D NO NO NO, SCUMMVM IS 2D GO RESIDUAL
It is not. ResidualVM is a lovely project handled by nice people. I may not 100% agree with all their decisions, but that's true everywhere.
The issue of whether a game is a "ScummVM-worthy" 2D game, or "ResidualVM-worthy" 3D game is sometimes blurry and people in the team may disagree on specifics, yes. For me personally, I draw the line at games that use vertex-geometry as still handled by current-gen graphics cards. So Gabriel Knight 3 is definitely a game for ResidualVM. Discworld Noir, despite its extended Tinsel-engine, and despite only using vertices for the main character (IIRC), is also a ResidualVM-game in my mind.
Where does that leave games using voxel-graphics like Blade Runner? Frankly, I have no idea. Can the voxel geometry be easily transformed into vertex geometry? If yes, then maybe going the ResidualVM route would be easier. If not, the voxel geometry would need to be rendered into sprite buffers, and then the rest of the engine is purely 2D? In that case, that seems like a fit for ScummVM for me. YMMV.
In either case, neither of the two options, going ScummVM or going ResidualVM, is better than the other one. I love both projects and they both have their niches. Sure, those may overlap on occasion, but so what?
Many people in the ScummVM team believe that a RE job on an engine has to be done "properly". The resulting code should look like native C++ code; it should be readable and understandable like real native C++ code.whoozle wrote:Then, if you will try to automate something, because there's dozens of similar c++ "scripts" in those dlls
What the code should not look like is assembly, or even some high-level assembly dialect. Just taking the code, throwing it through a disassembler and pasting the result verbatim into a .cpp file is, in my opinion, not what ScummVM stands for.
When I RE a game, I look at the disassembly from a high-level perspective. I try to understand what it does, and then replicate the behaviour in C++ code. The key there is the understanding of what a piece of code does: this is what gives the whole work meaning for me; it's like solving a puzzle.
This also helps in maintaining the engine later. When something gets accidentally broken because of some change of common code in ScummVM, or maybe a new backend for some new device, it's far easier to pinpoint the problem when you have readable native C++ code. When all you have is some assembly-like mess, this will be far more difficult. Ditto when maybe later on some new game using a similar (or the same) engine is discovered (see for example the newly discovered advertisment game Croustibat for the gob engine).
Now, did I take some shortcuts in my code from time to time? Yes, of course. But I'm not proud of them, and I try to minimize that wherever I can.
Thanks to DrMcCoy for his very nice elaboration on these things.
I want to stress this point by DrMcCoy again: We are not against using tools like HexRays, etc. (or even custom ones) to aid reverse engineering. However, the end result should be ("native") understandable C++ code. Thus, if one wants to use such automation tools as an intermediate step, that's really fine with us.DrMcCoy wrote:Many people in the ScummVM team believe that a RE job on an engine has to be done "properly". The resulting code should look like native C++ code; it should be readable and understandable like real native C++ code.whoozle wrote:Then, if you will try to automate something, because there's dozens of similar c++ "scripts" in those dlls
What the code should not look like is assembly, or even some high-level assembly dialect. Just taking the code, throwing it through a disassembler and pasting the result verbatim into a .cpp file is, in my opinion, not what ScummVM stands for.
Since whoozle mentioned (with loads of drama) Dreamweb, and since I was one of the people who rewrote it, here are my 2c: Whoozle's work on Dreamweb was, IMHO very good. However, The end result was a higher level Assembly-like language built on top of C.
This was just a glorified mini-Assembly language on top of C, and it was very hard to read it, understand it and maintain it from a casual C++ programmer's point of view. Rewriting it to actual C++ made it much more readable and easy to understand and maintain. Therefore, I totally agree with sev, DrMcCoy and LordHoto: ScummVM is not about rewriting a game into higher-level assembly. It's about understanding what the binary does, and try and replicate it using native C++. You, whoozle, did not understand that back then when you flamed about Dreamweb. And you still don't get it: we don't want glorified Assembly. ScummVM is about rewriting code into something understandable. This is why approaches like what whoozle did with Dreamweb and yaz0r did with Igor: Objective Uikokahonia are great, but are not what ScummVM stands for.
So, although whoozle's work with Dreamweb was great as a starting step, it was not something that would be considered finished and useable work. whoozle, please try and understand that, and stop flamebaiting and trolling for no reason. After all, Dreamweb has been supported, based on the initial work you did.
Now, regarding voxel graphics: no modern graphics card can accelerate these, AFAIK, so they're handled using software renderers. If the extra code needed to render voxel graphics isn't much, then it does fit within ScummVM's scope. If we need to rewrite parts of our graphics code just for one game, and if the Voxel graphics can be translated into vertex graphics, then it's more suitable for ResidualVM.
Both ScummVM and ResidualVM are great projects. They both have their own goals, with the main difference being the kind of graphics used (2D vs 3D), which are totally different in nature. There are some in-between games that are pseudo-3D (like Zork: Grand Inquisitor), which are suited for ScummVM. Having said that, voxel graphics aren't terribly complex, so this game could be more suited for ScummVM. But it's better to judge that if we look at the final outcome, and any extra functionality needed at the backend level.
Please stop flaming. Blade Runner is still a work-in-progress, so it's not worth it starting a flame war on something that isn't mature yet.
This was just a glorified mini-Assembly language on top of C, and it was very hard to read it, understand it and maintain it from a casual C++ programmer's point of view. Rewriting it to actual C++ made it much more readable and easy to understand and maintain. Therefore, I totally agree with sev, DrMcCoy and LordHoto: ScummVM is not about rewriting a game into higher-level assembly. It's about understanding what the binary does, and try and replicate it using native C++. You, whoozle, did not understand that back then when you flamed about Dreamweb. And you still don't get it: we don't want glorified Assembly. ScummVM is about rewriting code into something understandable. This is why approaches like what whoozle did with Dreamweb and yaz0r did with Igor: Objective Uikokahonia are great, but are not what ScummVM stands for.
So, although whoozle's work with Dreamweb was great as a starting step, it was not something that would be considered finished and useable work. whoozle, please try and understand that, and stop flamebaiting and trolling for no reason. After all, Dreamweb has been supported, based on the initial work you did.
Now, regarding voxel graphics: no modern graphics card can accelerate these, AFAIK, so they're handled using software renderers. If the extra code needed to render voxel graphics isn't much, then it does fit within ScummVM's scope. If we need to rewrite parts of our graphics code just for one game, and if the Voxel graphics can be translated into vertex graphics, then it's more suitable for ResidualVM.
Both ScummVM and ResidualVM are great projects. They both have their own goals, with the main difference being the kind of graphics used (2D vs 3D), which are totally different in nature. There are some in-between games that are pseudo-3D (like Zork: Grand Inquisitor), which are suited for ScummVM. Having said that, voxel graphics aren't terribly complex, so this game could be more suited for ScummVM. But it's better to judge that if we look at the final outcome, and any extra functionality needed at the backend level.
Please stop flaming. Blade Runner is still a work-in-progress, so it's not worth it starting a flame war on something that isn't mature yet.
Last edited by md5 on Sat Feb 15, 2014 4:30 pm, edited 2 times in total.
That's exactly what I'm talking about. Noone would ever consider your arguments, like "voxels could be rendered in one pass, cost you as much as every colorkey routine". They have spent man-year rewriting and fixing bugs in stable code, just because it was "high level assembly". this is "<f-word> you that's why" kind of the argument, I'd prefer
That's exactly what I'm talking about #2. Think twice, it seems clear to me that BladeRunner will never be merged in scummvm, judging by this thread, so you have no reason to waste your time for it.
That's exactly what I'm talking about #2. Think twice, it seems clear to me that BladeRunner will never be merged in scummvm, judging by this thread, so you have no reason to waste your time for it.
Last edited by whoozle on Sat Feb 15, 2014 6:04 pm, edited 2 times in total.
Ok, could you please clear the things up, what tool was used to recreate dreamweb code exactly? HexRays? IDA? Has anyone ever used raw HexRays code dumps to rewrite games? Did you ever reverse-engineered any game?LordHoto wrote:...
I want to stress this point by DrMcCoy again: We are not against using tools like HexRays, etc. (or even custom ones) to aid reverse engineering. However, the end result should be ("native") understandable C++ code. Thus, if one wants to use such automation tools as an intermediate step, that's really fine with us.
The answer to this questions is obvious.
Last edited by whoozle on Sat Feb 15, 2014 6:05 pm, edited 1 time in total.
Wow. More flamebait. I can't believe it. You either choose to ignore whatever has been mentioned, or you're doing this on purpose, just to piss people off. Re-read the replies, and you'll see that we are saying that Blade Runner can be included in ScummVM, if it doesn't need changes to the graphics pipeline. Now stop trollingwhoozle wrote:That's exactly what I'm talking about. Noone would ever consider your arguments, like "voxels could be rendered in one pass, cost you as much as every colorkey routine". They have spent man-year rewriting and fixing bugs in stable code, just because it was "high level assembly". this is "<f-word> you that's why" kind of the argument, I'd prefer
That's exactly what I'm talking about #2. Think twice, it seems clear to me that BladeRunner will never be merged in scummvm, judging by this thread, so you have no reason to waste your time for it.
Dreamweb's original source code was in Assembly. ScummVM is written in C++. We want actual C++, not Assembly masqueraded as C. Therefore, your argument is invalid.whoozle wrote:Ok, could you please clear the things up, what tool was used to recreate dreamweb code exactly? HexRays? IDA? Has anyone ever used raw HexRays code dumps to rewrite games? Did you ever reverse-engineered any game?LordHoto wrote:...
I want to stress this point by DrMcCoy again: We are not against using tools like HexRays, etc. (or even custom ones) to aid reverse engineering. However, the end result should be ("native") understandable C++ code. Thus, if one wants to use such automation tools as an intermediate step, that's really fine with us.
The answer to this questions is obvious.
As for reverse-engineering... Really? Are you seriously asking such questions HERE?