Lack of Basic Testing

(previously 'DEVELOPER') Private forum for registered community members. To register, please visit www.prison-architect.com/register.

Moderator: NBJeff

Mas Tnega
level5
level5
Posts: 7898
Joined: Sat Mar 02, 2002 11:54 pm
Location: Edinburgh
Contact:

Postby Mas Tnega » Thu Jun 06, 2013 1:58 pm

wribit wrote:Problem is, we shouldn't judge the means/ways by which the update was created, when all we can see as consumers is the end result. There's just no way of knowing their reasoning, or the size of their operation.

All we know is they're independent developers with much different business styles as the mainstream, big-wig companies. Plus, they didn't try to pull the wool over our eyes - we were told it was in alpha, and to expect lots of bugs.


Problem is, the way of noticing these particular bugs is to have done the following:

Use the clone tool once. This bug is 100% reproducible by this means.
Use lockdown and then turn it off. This bug is 100% reproducible by this means.

Hence my wording.
Sprooty
level1
level1
Posts: 10
Joined: Wed May 29, 2013 3:20 am

Re: Not sure

Postby Sprooty » Fri Jun 07, 2013 12:38 am

AgentPaper wrote:
TaintedArcher wrote:I just think if they are going to continue to not test because "it's an alpha and later things might be removed is the exact same as building a house without foundation


It's not that they aren't going to test, it's that they aren't going to do exhaustive testing and root out every single bug to make it into a polished,



How is my suggestion of playing their release for 10min - 1 hour "exhaustive"?


Problem is, the way of noticing these particular bugs is to have done the following:

Use the clone tool once. This bug is 100% reproducible by this means.
Use lock-down and then turn it off. This bug is 100% reproducible by this means.


What this whole thread pertains to is basic "Shakedown Testing" - Which is usually done by the Dev's so they don't end up with Pie on their face.

http://barsand.wordpress.com/2013/03/05 ... e-testing/
AgentPaper
level2
level2
Posts: 124
Joined: Fri May 17, 2013 2:04 am

Re: Not sure

Postby AgentPaper » Fri Jun 07, 2013 12:54 am

Sprooty wrote:How is my suggestion of playing their release for 10min - 1 hour "exhaustive"?


It's not, and I've never suggested that it was. I've explicitly stated multiple times that exhaustive testing takes more than an hour.
User avatar
paktsardines
level5
level5
Posts: 1752
Joined: Mon Oct 01, 2012 11:10 am
Location: Australia

Postby paktsardines » Fri Jun 07, 2013 4:16 am

What this whole thread pertains to is basic "Shakedown Testing"


Just wondering if you read the article you linked to?
To give an idea the test may last 30 minutes and cover only the happy-path unit test scenarios.


The fact every other component of the game is working strongly suggests they did more than shake-down testing prior to release. Clearly the alarm and cloning tool weren't on the happy path. They will be next time though.
Ten98
level1
level1
Posts: 16
Joined: Mon Feb 18, 2013 11:14 am
Location: United Kingdom

Postby Ten98 » Fri Jun 07, 2013 8:48 am

Sprooty, get off your high horse.

You can expect ZERO testing between "releases". Not 10 minutes, not an hour, none. This is an Alpha of a crowd-funded game, and game-breaking bugs are to be expected. We are the testers. Found a bug? Good for you, submit it to the bug tracker.

Open the new Alpha, play it for however long you want, report all the bugs on the tracker, then close it again. Don't try and play it for days and build up a working, sprawling prison as if it was a finished game, it isn't.

This is an early access to a work-in-progress game which is coming out sometime in the future. If they break everything to test one new feature, so be it. You have no right to expect anything other than a broken, buggy mess until Beta.
Mas Tnega
level5
level5
Posts: 7898
Joined: Sat Mar 02, 2002 11:54 pm
Location: Edinburgh
Contact:

Postby Mas Tnega » Fri Jun 07, 2013 3:38 pm

Ten98 wrote:You can expect ZERO testing between "releases". Not 10 minutes, not an hour, none. This is an Alpha of a crowd-funded game, and game-breaking bugs are to be expected. We are the testers. Found a bug? Good for you, submit it to the bug tracker.

No. You can expect better than zero testing, always. No testing whatsoever by a developer is utterly lazy no matter what the project is. The notion that a developer shouldn't be doing some degree of testing is a toxic one that only wastes everyone's time. I don't think for a second Chris has done zero testing and I'd be seriously shocked (and appalled) if that were true.
User avatar
111none
level4
level4
Posts: 970
Joined: Fri Oct 19, 2012 3:32 am
Location: Wangjing, Beijing, Peoples Republic of China

Postby 111none » Fri Jun 07, 2013 4:48 pm

Play all of the alphas, then realize how much less bugs there are than most large company betas.
xo
With the sincerest regards,
111none
Mas Tnega
level5
level5
Posts: 7898
Joined: Sat Mar 02, 2002 11:54 pm
Location: Edinburgh
Contact:

Postby Mas Tnega » Fri Jun 07, 2013 5:26 pm

Demand something a little better than before, then maybe receive it.
Causeless
level2
level2
Posts: 166
Joined: Fri Sep 28, 2012 11:49 pm

Postby Causeless » Sat Jun 08, 2013 7:15 pm

Introversion aren't under any obligation to fix bugs. Think about it this way - it's not that they are working towards a release for us. Instead they are working towards the end game product, and every update we get is just a development snapshot every time they get a new feature to a workable state. Bug fixing comes later.

Anyways, I'm not sure why anybody cares. Alpha 1 and 2 were dramatically worse than this. Consistently, prisoners would get stuck and refuse to do anything and you could get money just through zoning an area. That was far far more gamebreaking.
Cral
level0
Posts: 4
Joined: Mon Jun 10, 2013 1:58 pm

Postby Cral » Mon Jun 10, 2013 2:35 pm

Causeless wrote:Introversion aren't under any obligation to fix bugs. Think about it this way - it's not that they are working towards a release for us. Instead they are working towards the end game product, and every update we get is just a development snapshot every time they get a new feature to a workable state. Bug fixing comes later.

Anyways, I'm not sure why anybody cares. Alpha 1 and 2 were dramatically worse than this. Consistently, prisoners would get stuck and refuse to do anything and you could get money just through zoning an area. That was far far more gamebreaking.


I'm not too fussed about the current bugs in the game as it is an alpha, but I highly doubt this is how Introversion operate. Having a group of alpha testers who report bugs for them for free is a huge benefit, but the value they get out of that free testing is dependent on the existing game features being functional to some degree. Taking the clone tool as an example, by breaking it in a release build they're essentially losing three weeks of testing on that feature, so while they're not necessarily obligated to fix major bugs, doing so benefits development and keeps the game enjoyable.

Alpha build or not, I don't think it's fair to attack people who question the level of testing before each release, as getting the correct balance between progressing the game for the long-term and keeping it functional for the short-term is only going to benefit everyone involved.

The "bug fixing comes later" approach doesn't fit the development model being employed by the way.
Causeless
level2
level2
Posts: 166
Joined: Fri Sep 28, 2012 11:49 pm

Postby Causeless » Mon Jun 10, 2013 10:17 pm

They aren't losing 3 weeks of testing on that feature. They are by definition getting useful info by learning it is broken, via testing.
User avatar
paktsardines
level5
level5
Posts: 1752
Joined: Mon Oct 01, 2012 11:10 am
Location: Australia

Postby paktsardines » Tue Jun 11, 2013 1:13 am

They are losing 3 weeks of testing on the feature. Determining it's broken takes no time at all and no further testing can be done with the clone tool until it's fixed.
Causeless
level2
level2
Posts: 166
Joined: Fri Sep 28, 2012 11:49 pm

Postby Causeless » Tue Jun 11, 2013 3:31 am

But it was already known to work, and any bugs will have been reported long long ago. Anyways, that argument about testing could be applied to features not yet implemented too. Surely it makes more sense to get new features pushed out ASAP for testing instead of a relatively small, already extensively tested feature?

We have no idea how difficult the fix may have been, either to find or implement, either - all the posts here make it out to be incredibly simple, but we don't know that.

Really, if anyone here saying the reason they are pissed about the lack of clone tool is testing purposes, that's a bare-faced lie. We all know everyone wants it for gameplay purposes.
User avatar
paktsardines
level5
level5
Posts: 1752
Joined: Mon Oct 01, 2012 11:10 am
Location: Australia

Postby paktsardines » Tue Jun 11, 2013 3:37 am

if anyone here saying the reason they are pissed about the lack of clone tool is testing purposes, that's a bare-faced lie

I couldn't care less about the lack of clone tool, nor the lack of testing of it. I also totally agree that testing can't be applied to features not yet implemented.
Mas Tnega
level5
level5
Posts: 7898
Joined: Sat Mar 02, 2002 11:54 pm
Location: Edinburgh
Contact:

Postby Mas Tnega » Tue Jun 11, 2013 6:19 pm

Causeless wrote:But it was already known to work, and any bugs will have been reported long long ago. Anyways, that argument about testing could be applied to features not yet implemented too. Surely it makes more sense to get new features pushed out ASAP for testing instead of a relatively small, already extensively tested feature?

We have no idea how difficult the fix may have been, either to find or implement, either - all the posts here make it out to be incredibly simple, but we don't know that.

Really, if anyone here saying the reason they are pissed about the lack of clone tool is testing purposes, that's a bare-faced lie. We all know everyone wants it for gameplay purposes.
I believe it was mentioned before: The issue is not so much clone itself being unusable and the siren not turning off, but the circumstances one will conclude to exist to let those be in a released alpha version. That the siren was a new feature makes it the more concerning.

Return to “Community Members”

Who is online

Users browsing this forum: No registered users and 9 guests