Lack of Basic Testing

[Sprooty]

Hi guys,

I've been following PA since about Alpha 6 and have been extremely excited watching this game grow and continue to be enjoyable.

I however feel let down by the fact that even most basic testing before release does not appear to happen.

My job is actually a Software Test Analyst in Australia.

Alpha 10 as many will now know has quite a few major bugs.. which would have been picked up by performing a 10 MINUTE shakedown test.
- Clone not working
- Door Siren constantly going off


I do understand that PA is still in Alpha, however i cannot get over the fact that the DEVs don't spend 10 minutes before release to do at minimum a shakedown test, which would have revealed these issues! My only hope while annoying is that the DEVs knew of these issues but released anyway..

[acegamer08]

...

[AgentPaper]

While this is alpha, I agree with your basic premise that shakedown tests need to be made and the game needs to be in a playable state even for an alpha release.

However, neither of the bugs you posted are anywhere near game-breaking. It took me much longer than 10 minutes to realize that the cloning tool wasn't there anymore, and I still haven't seen the siren bug yet.

These things seem more like the kind of thing that would show up in more exhaustive testing, the kind of thing you polish out before a beta release, when the game is supposed to be feature-complete and stuff like this becomes a priority.

If they really weren't doing such basic testing, we would have far, far worse bugs than we're currently seeing.

[kommissarw]

We are the testers.

I don't know what you thought when you bought it at Alpha 6, but I bought it at alpha 9 on steam & the situation was made very clear to me. I'm happy to help build a fairly different game, although I'm a bit crap at bug reports.

[Sprooty]

12 hours ago the devs tweeted: "I broke the clone tool, sorry" .... so I guess they know (knew) that.
the other issue regarding sound, I don't think they knew...(if they did they didn't admit to breaking it)

Well many do feel that is an important tool. It is not a deal breaker however.

I fully understand that this is alpha, however the idea that the Devs did not play the latest build for an hour or two before release to myself as a professional software tester is bewildering.

It took me much longer than 10 minutes to realize that the cloning tool wasn't there anymore, and I still haven't seen the siren bug yet.

My point exactly.

[AgentPaper]

They aren't professional software testers, they're game designers. And now ten minutes has turned into 1-2 hours?

You still also haven't brought up any actual major bugs. The two you posted are minor for an alpha, and both have very simple workarounds (turn off the sound, build stuff normally).

I can't help but feel like you're trying to provoke a response, so I would suggest coming back with more concrete evidence before you start leveling accusations again.

[Sprooty]

My main concern here is the idea that more sever/major bugs could have occurred and the Developers would apparently not realise until the alpha is out.

As mentioned above, 10 minutes would have likely shown clone is not working, however i would expect 1-2 hours playtime before a release.

My assumption is that Introversion want to appear as a professional company? however when they have to say "sorry" on twitter, it does not give a good look.


I do not want to start a war, I'm simply suggesting that the Devs spend 10 minutes playing their game before release.

[Collic001]

My main concern here is the idea that more sever/major bugs could have occurred and the Developers would apparently not realise until the alpha is out.

As mentioned above, 10 minutes would have likely shown clone is not working, however i would expect 1-2 hours playtime before a release.

My assumption is that Introversion want to appear as a professional company? however when they have to say "sorry" on twitter, it does not give a good look.


I do not want to start a war, I'm simply suggesting that the Devs spend 10 minutes playing their game before release.

They are a professional company. This alpha was advertised as having 'game breaking bugs'. We were warned as buyers that builds would happen that would break the game and or other features. you knew all of this when you bought it, or you would have if you had watched any of their alpha promotional videos or read what it says on the Prison Architect page where you buy the game. What you bought is a pre order for the finished game Prison Architect, and as an early adopter you get alpha access and a say in how the game develops.

[youtube]KDDzSOS0vzc&t=1m20s[/youtube]

Since I don't seem to be able to link a specific part of the video with this embed code, the part you want starts at 1.20

[AgentPaper]

My main concern here is the idea that more sever/major bugs could have occurred and the Developers would apparently not realise until the alpha is out.

As mentioned above, 10 minutes would have likely shown clone is not working, however i would expect 1-2 hours playtime before a release.

My assumption is that Introversion want to appear as a professional company? however when they have to say "sorry" on twitter, it does not give a good look.


I do not want to start a war, I'm simply suggesting that the Devs spend 10 minutes playing their game before release.

So now the issue is that they might have missed major bugs, but didn't?

I'm really having a hard time understanding what you're trying to say here.

You still have yet to prove evidence of this lack of testing: The clone bug is not something that would show up in 10 minutes of testing, more likely than not, and I don't know why you think it would. If you're starting a prison from scratch, it takes almost 10 minutes just to unlock the clone feature, if you rush for it from the very start, and if you're playing an existing prison, you're not using the clone tool because you're not building stuff.

The only way it would show up would be either playing for much longer, or by doing a exhaustive check on all features of the game, neither of which is really that necessary for an alpha release.

It's also very possible that they knew that the clone bug was there before releasing it, and simply decided not to delay the release of the patch (potentially creating new bugs in the process) since, again, this is alpha and a few minor bugs don't really matter. Much better to push out the release and start moving towards the next one.

[Mas Tnega]

The issue is the software appears to the software tester to have been thrown over the wall after coding finished.

[Collic001]

The issue is the software appears to the software tester to have been thrown over the wall after coding finished.

This is the last thing I'm going to say in this thread. If you want to play a bug free game, play a finished product. Not an alpha build of a game you were warned would have bugs and broken features. Things break in alpha all the time. If this was a finished product you would have a reason to question how much testing had taken place before release. It isn't. You and I, everyone on this forum; we are the testers.

That was the deal when you bought this early access preorder. I'm sorry to break it to you, but Introversion made it very clear what to expect.

[Dosedmonkey]

Hi guys,

I've been following PA since about Alpha 6 and have been extremely excited watching this game grow and continue to be enjoyable.

I however feel let down by the fact that even most basic testing before release does not appear to happen.

My job is actually a Software Test Analyst in Australia.

Alpha 10 as many will now know has quite a few major bugs.. which would have been picked up by performing a 10 MINUTE shakedown test.
- Clone not working
- Door Siren constantly going off


I do understand that PA is still in Alpha, however i cannot get over the fact that the DEVs don't spend 10 minutes before release to do at minimum a shakedown test, which would have revealed these issues! My only hope while annoying is that the DEVs knew of these issues but released anyway..

They realise the clone tool was not working before release.... BUT its an Alpha... so just because it worked before, doesn't mean they can get it working currently, because of possibly something else they done.

Its an Alpha. BTW Its an Alpha. Got it?

Also what the hell is a Software Test Analyst? You make the excel spreadsheets of data that the other testers collect, to analyse later patterns? Really this gives you no qualification to criticise actual programmers.

My main concern here is the idea that more sever/major bugs could have occurred and the Developers would apparently not realise until the alpha is out.

As mentioned above, 10 minutes would have likely shown clone is not working, however i would expect 1-2 hours playtime before a release.

My assumption is that Introversion want to appear as a professional company? however when they have to say "sorry" on twitter, it does not give a good look.


I do not want to start a war, I'm simply suggesting that the Devs spend 10 minutes playing their game before release.

Yes you can find the big in 10 minutes, but you want wait 30 more days for they fix the bug? Then in Alpha 11 something else old breaks due to a new idea, so you need wait another 30 days for them to fix it AGAIN. This would double the time between release of Alphas and possibly add months on to the final project. Don't want to play and Alpha, don't buy an Alpha.

[obrhms]

We are the testers.

this! reading the title of this thread I expected self-guilt

[BlueTrin]

We are the testers.

this! reading the title of this thread I expected self-guilt

lol very funny.

The problem is that most of these things which were broken would require some work to non-regression test them.

As a developer, personally, I would invest the time but I am not a game developer, I am developing software mostly used by executives in fairly large companies. Here for a game, the incentive to put such restrictive non-regression test at an early stage of a non critical app which can evolve greatly could be judged too costly ?

[paktsardines]

would have been picked up by performing a 10 MINUTE shakedown test

The underlying problem is that games are fundamentally very difficult things to test because they have proprietary GUIs. Unlike web apps and the like, a lot of the testing simply cannot be automated.

A major difference between traditional life-cycle vs agile software development is that life-cycle development has one rigorous test cycle, but agile development needs rigorous testing at every release. If that testing is a manual process then it's nearly impossible to be comprehensive.

That said, now that these two issues have been identified I'm sure they'll be added to the pre-release checklist in the future.