Do you think ScummVM release testing is cut too short?
Moderator: ScummVM Team
-
- ScummVM Team Member
- Posts: 377
- Joined: Sat Sep 24, 2005 12:25 pm
- Location: Austria
Do you think ScummVM release testing is cut too short?
Dear ScummVM fans, users, testers and programmers!
I have been thorougly testing the upcoming ScummVM 0.8.0 release for some time now.
Although stability and performance has very much improved with every new CVS version, I encountered many new bugs introduced with updates of the CVS.
Now I want to know if you think that the release plans have been set to eager by setting the release testing phase to approximately 3-4 weeks.
What do you think?
Best regards
Joachim Eberhard
I have been thorougly testing the upcoming ScummVM 0.8.0 release for some time now.
Although stability and performance has very much improved with every new CVS version, I encountered many new bugs introduced with updates of the CVS.
Now I want to know if you think that the release plans have been set to eager by setting the release testing phase to approximately 3-4 weeks.
What do you think?
Best regards
Joachim Eberhard
Last edited by joachimeberhard on Fri Oct 28, 2005 1:55 pm, edited 4 times in total.
-
- ScummVM Team Member
- Posts: 377
- Joined: Sat Sep 24, 2005 12:25 pm
- Location: Austria
While I really admire the improvements introduced with 0.8.0, I have encountered many regressions in CVS introduced in the past few days.
Thats why I ask.
I really respect your opinion, and admire the good work you have done in the past few weeks in terms of bugfixing.
And, as I promised I will spend 20$ every 3 months from now for helping you buy some games.
I pay for games I like, so why shouldn't I pay for making it possible to play games I LOVE.
You have made me so very happy with ScummVM and I admire your work very much.
Best regards
Joachim Eberhard
Thats why I ask.
I really respect your opinion, and admire the good work you have done in the past few weeks in terms of bugfixing.
And, as I promised I will spend 20$ every 3 months from now for helping you buy some games.
I pay for games I like, so why shouldn't I pay for making it possible to play games I LOVE.
You have made me so very happy with ScummVM and I admire your work very much.
Best regards
Joachim Eberhard
Last edited by joachimeberhard on Tue Nov 01, 2005 1:40 pm, edited 1 time in total.
To be honest (and I will reply to your e-mail soon :), if you are mainly donating $20/month for the purpose of better FM-TOWNS support... Your better off actually saving up and just buying the games for us :)
It's still quite hard to find most of these titles.
It's still quite hard to find most of these titles.
Last edited by Ender on Sun Oct 30, 2005 9:45 am, edited 2 times in total.
-
- ScummVM Team Member
- Posts: 377
- Joined: Sat Sep 24, 2005 12:25 pm
- Location: Austria
To be honest, one of my biggest problems is "saving up".
I have only been able to buy some of the rare games because I was surprisingly lucky enough to be the only bidder.
Yesterday I bought 4 different Mac-Versions for total of 35$.
Another problem is, I would pay shipment twice, one time to me and one time to you.
And I think you have a better idea which games you need.
But for helping me and some other possible donators, you could state somewhere which games are needed most (ie no-one in the team has a copy of a particular game.)
So, for now I just spend my 20 bucks every now and then just because I think you deserve it. If I ever manage to get another copy of any game cheap enough (Won't pay over 100$), I will donate it.
Best regards
Joachim
I have only been able to buy some of the rare games because I was surprisingly lucky enough to be the only bidder.
Yesterday I bought 4 different Mac-Versions for total of 35$.
Another problem is, I would pay shipment twice, one time to me and one time to you.
And I think you have a better idea which games you need.
But for helping me and some other possible donators, you could state somewhere which games are needed most (ie no-one in the team has a copy of a particular game.)
So, for now I just spend my 20 bucks every now and then just because I think you deserve it. If I ever manage to get another copy of any game cheap enough (Won't pay over 100$), I will donate it.
Best regards
Joachim
- eriktorbjorn
- ScummVM Developer
- Posts: 3550
- Joined: Mon Oct 31, 2005 7:39 am
At the risk of sounding flippant...
Given how rare and/or expensive the FM-Towns games tend to be, it might be both quicker and cheaper to invest the money in a good C++ book (or maybe borrow one at the library, if that's an option) and see if you can figure out any more details about the bugs yourself.
In some cases, it helps to compare the scripts of a version that works to the scripts of the one that misbehaves to see what they're doing differently. (In one case, a particularly amusing bug in the Indy 3 intro was tracked down by comparing the game's scripts to the ones from a demo.)
Given how rare and/or expensive the FM-Towns games tend to be, it might be both quicker and cheaper to invest the money in a good C++ book (or maybe borrow one at the library, if that's an option) and see if you can figure out any more details about the bugs yourself.
In some cases, it helps to compare the scripts of a version that works to the scripts of the one that misbehaves to see what they're doing differently. (In one case, a particularly amusing bug in the Indy 3 intro was tracked down by comparing the game's scripts to the ones from a demo.)
Last edited by eriktorbjorn on Tue Nov 01, 2005 7:39 pm, edited 1 time in total.
Regarding FM-TOWNS games: I own the Indy3 version for FM-TOWNS, and have access to ZakTowns. However, I never even saw the other games (Loom, Monkey, Monkey 2, Indy4 -- are there more?).
Please believe us: It's annoying to see that many bugs on our bug tracker, and it's made worse by the fact that I can't even do anything about it since I don't have those games and hence can't work on fixes.
However, considering how (comparatively) rare those games are, and considering that at least in the case of MI/MI2/Indy4 they don't seem to offer any advantage over the regular (and common) PC versions, the priority for fixing these issues is rather low. It's a bit different for Zak and Loom, since those are "unique".
Please believe us: It's annoying to see that many bugs on our bug tracker, and it's made worse by the fact that I can't even do anything about it since I don't have those games and hence can't work on fixes.
However, considering how (comparatively) rare those games are, and considering that at least in the case of MI/MI2/Indy4 they don't seem to offer any advantage over the regular (and common) PC versions, the priority for fixing these issues is rather low. It's a bit different for Zak and Loom, since those are "unique".
That is all of them. The games were adapted to the FM-Towns Marty by James Leiterman. The two "unique" games (Loom and Zak) were adapted by a two man team - consisting of James and another team member. The other games were adapted to the FM-Towns Marty solely by Mr. Leiterman himself.fingolfin wrote:Regarding FM-TOWNS games: I own the Indy3 version for FM-TOWNS, and have access to ZakTowns. However, I never even saw the other games (Loom, Monkey, Monkey 2, Indy4 -- are there more?).
In a bit of unrelated info - James Leiterman was also responsible for localizing the NES version of Maniac Mansion to French, German, Italian, Spanish and Swedish. He also localized the PC version of Maniac Mansion to Italian.
http://66.15.18.176/leiterman/lfilm.html
Jims a very friendly guy, we've had a few chats about the FM-TOWNS games :)
MetaFox wrote:The games were adapted to the FM-Towns Marty by James Leiterman. The two "unique" games (Loom and Zak) were adapted by a two man team - consisting of James and another team member. The other games were adapted to the FM-Towns Marty solely by Mr. Leiterman himself.
-
- ScummVM Team Member
- Posts: 377
- Joined: Sat Sep 24, 2005 12:25 pm
- Location: Austria
I know that it is hard for you to fix bugs for FM-Towns versions.
But the problem with the Kanji versions was introduced AFTER CVS 10/23/2005, short before release.
This is a very very sad regression of the final.
Considering that you were in feature freeze mode at that time, and I (hopefully) clearly statet in the bug-reports that this was introduced SHORT before release, this is a sad fact.
I hope you didn't ignore my bugreports just because I reported to many already and you didn't take me serious.
I am sorry that I am no programmer, so I just can't fix this bug myself.
And just to complete the list of unique FM-Towns-Games:
INDY3Towns is also much better than the standard PC version because of the CD-Soundtrack.
However, I DO understand your point about the difficulty to fix these bugs.
What can I do?
I already wrote to some good eBay sellers I know, if they could make me a special price if they find any FM-Towns games, for donating it to ScummVM.
But this could take some time.
Well, until then...
Best regards
Joachim
But the problem with the Kanji versions was introduced AFTER CVS 10/23/2005, short before release.
This is a very very sad regression of the final.
Considering that you were in feature freeze mode at that time, and I (hopefully) clearly statet in the bug-reports that this was introduced SHORT before release, this is a sad fact.
I hope you didn't ignore my bugreports just because I reported to many already and you didn't take me serious.
I am sorry that I am no programmer, so I just can't fix this bug myself.
And just to complete the list of unique FM-Towns-Games:
INDY3Towns is also much better than the standard PC version because of the CD-Soundtrack.
However, I DO understand your point about the difficulty to fix these bugs.
What can I do?
I already wrote to some good eBay sellers I know, if they could make me a special price if they find any FM-Towns games, for donating it to ScummVM.
But this could take some time.
Well, until then...
Best regards
Joachim
- eriktorbjorn
- ScummVM Developer
- Posts: 3550
- Joined: Mon Oct 31, 2005 7:39 am
I doubt that. In fact, people are often encouraged to write individual bug reports rather than reporting several different bugs as one. It's frustrating to see so many bug reports, of course, but that's hardly a reason to ignore them. It's just that most ScummVM users and developers neither have the games, nor speak Japanese.joachimeberhard wrote:I hope you didn't ignore my bugreports just because I reported to many already and you didn't take me serious.
I already have the movie soundtrack CD, so I don't think I'm missing that much.joachimeberhard wrote:INDY3Towns is also much better than the standard PC version because of the CD-Soundtrack.
- Lord Savage
- Posts: 27
- Joined: Thu Oct 27, 2005 9:41 pm
- Location: Canada
I voted that the testing for 0.8.0 was too short
First off I'd like to thank everyone involved with this release for their hard work. Please don't consider that I am slagging off on your hard work.
So, why did I vote that I felt that the testing was too short?
Because my primary (Mac OS X) and secondary (PalmOS) platforms did not offer test builds during the testing phase. The only platform that got a thorough testing workout this time around was the Windows one. The last two releases have had frequent official CVS builds of the Mac OS X port for me to test. That was definitely not the case this time around. The most recent official CVS build available on the site was dated June 24th.
To make matters worse, from my point of view, the PalmOS version isn't even ready yet. To me, this looks like this release was rushed out the door.
Any way that is why I voted that I felt that this version was rushed out the door.
So, why did I vote that I felt that the testing was too short?
Because my primary (Mac OS X) and secondary (PalmOS) platforms did not offer test builds during the testing phase. The only platform that got a thorough testing workout this time around was the Windows one. The last two releases have had frequent official CVS builds of the Mac OS X port for me to test. That was definitely not the case this time around. The most recent official CVS build available on the site was dated June 24th.
To make matters worse, from my point of view, the PalmOS version isn't even ready yet. To me, this looks like this release was rushed out the door.
Any way that is why I voted that I felt that this version was rushed out the door.
Re: I voted that the testing for 0.8.0 was too short
Let me explain it as a 0.8.0 release coordinator.Lord Savage wrote: Because my primary (Mac OS X) and secondary (PalmOS) platforms did not offer test builds during the testing phase.
...
To make matters worse, from my point of view, the PalmOS version isn't even ready yet. To me, this looks like this release was rushed out the door.
MacOS X is one of the main development platforms in ScummVM. Several key developers and moreover our co-lead Fingolfin run it every day. So it is very tested. Moreover, there is no difference between MacOS, Windows, Linux, *nix, OS/2 ports. All they use exactly same codebase (except some Altivec code in MacOS and some audio drivers). That means that if there is a bug in MacOS, it will appear in Windows too (technically endianness could play a role here, but every engine was tested on both big- and little-endian platforms, and only sword2 had alignment ptoblems which are already fixed in 0.9.0). All of that makes MacOS test builds redundant.
Speaking of PalmOS, yes, it's a pity. The problem that the porter decided to redo whole backend and make it much more faster, he is just didn't came up to deadline, and he warned about it long beforehand. I'd suggest you to read scummvm-devel archives for details. So if we'd even postpone release because of PalmOS by 2 weeks, that wouldn't guarantee us presence of the port in final 0.8.0. All of us are volunteers, PalmOS porter included, and he just can't jump over his head.
Eugene
- Lord Savage
- Posts: 27
- Joined: Thu Oct 27, 2005 9:41 pm
- Location: Canada
Re: I voted that the testing for 0.8.0 was too short
Thanks for taking the time to reply to my concerns. I do realize that this project is a completely volunteer based. I also find it to be one of the most professional volunteer based projects I have ever seen.sev wrote: Let me explain it as a 0.8.0 release coordinator.
MacOS X is one of the main development platforms in ScummVM. Several key developers and moreover our co-lead Fingolfin run it every day. So it is very tested. Moreover, there is no difference between MacOS, Windows, Linux, *nix, OS/2 ports. All they use exactly same codebase (except some Altivec code in MacOS and some audio drivers). That means that if there is a bug in MacOS, it will appear in Windows too (technically endianness could play a role here, but every engine was tested on both big- and little-endian platforms, and only sword2 had alignment ptoblems which are already fixed in 0.9.0). All of that makes MacOS test builds redundant.
Speaking of PalmOS, yes, it's a pity. The problem that the porter decided to redo whole backend and make it much more faster, he is just didn't came up to deadline, and he warned about it long beforehand. I'd suggest you to read scummvm-devel archives for details. So if we'd even postpone release because of PalmOS by 2 weeks, that wouldn't guarantee us presence of the port in final 0.8.0. All of us are volunteers, PalmOS porter included, and he just can't jump over his head.
Eugene
However, I still believe that 0.8.0 was rushed out the door. Using the Mac OS X build as example you stated that there are three areas where the Mac codebase differs the standard codebase. They are endianness, AltiVec and audio drivers.
You already mentioned that sword2 alignment problem (endianness) so, that is one out of three having an issue. Looking at the bug tracker shows priority 6 bug [ 1342732 ] MAC: hq2x/hq3x filters broken when altivec is enabled in which fingolfin had to disable the AltiVec code in 0.8.0 in order for the hq filters to work. That's two out of three having an issue. Further up the bug tracker shows two possibly related priority 5 issues (correct me if I'm wrong) [ 1344631 ] build fails on Mac OS X 10.2.8 when MT32 emulation is used and [ 1343971 ] MAC: ScummVM 0.8.0 fails to launch which might be related to a conflict between Mac OS X 10.2.8 and MT32 emulation through the audio drivers. No one is assigned to either of these cases so, there is no telling if they are duplicatable or related or if they actually is related to the audio drivers so, this one is just a potential at this point.
So definitively two of the three Mac specific areas have issues in the 0.8.0 release. Depending on the outcome 1344631 and 1343971 there may be an issue in the audio drivers as well making it three for three. Had there been at least one official CVS build for Mac OS X after the Oct 11 call for testers I truly believe that these would have been caught and corrected before 0.8.0 release. So, in my opinion an official CVS build for Mac OS X is not redundant.
Thanks for clarifying the hold up on the PalmOS port; it would have been nice to have been informed in the main page ahead time that the PalmOS port wasn't going to be included in the 0.8.0 release to temper our disappointment. Since my programming claim to fame is 10 PRINT "Hello World", I've never felt that I was worthy of reading to the scumm-devel archives
- eriktorbjorn
- ScummVM Developer
- Posts: 3550
- Joined: Mon Oct 31, 2005 7:39 am
Re: I voted that the testing for 0.8.0 was too short
Just to clarify, endianness and alignment are two completely different beasts. Endianness concerns in which order the bytes of a multi-byte value are stored, while alignment places restrictions on where in memory such values may be stored. E.g. for a 32-bit value the address may have to be dividable by four.Lord Savage wrote: You already mentioned that sword2 alignment problem (endianness) so, that is one out of three having an issue.
Broken Sword 2 should already be endian safe. I haven't heard any Mac-specific bug reports about it, either. Then again, it wasn't until recently that I heard there may be alignment problems in -- I think -- the PalmOS port. The alignment problems have hopefully been straightened out on the CVS trunk, but that was quite a large change so it didn't go into the 0.8 branch. Maybe if it had been reported earlier. Maybe.
Yet a third porting issue is whether or not the code is 64-bit clean. Particularly in Broken Sword 2, which used to do very naughty things with data pointers. The jury is still out on that one, but I hope it is safe.
Last edited by eriktorbjorn on Tue Nov 01, 2005 7:40 pm, edited 1 time in total.