[Suggestion] Use a real bug tracker.

(previously 'DEVELOPER') Private forum for registered community members. To register, please visit www.prison-architect.com/register.

Moderator: NBJeff

User avatar
jelco
level5
level5
Posts: 6018
Joined: Sat Feb 18, 2006 7:45 am
Location: Cygnus X-1
Contact:

Postby jelco » Sat Sep 29, 2012 12:26 am

Molano wrote:I personally find it a lot easier to find bugs in a good tracker than in a forum. Also the developer can assign a status to a bug so that the users don't have to scan an entire thread for it.

How good you find something you're looking for really depends on how good you look for it. A bug would be a ticket or issue in a tracker, and is a topic here. There's the same deal of search functionality in this piece of forum software that can get you exactly where you want to be. Case in point: the topics I linked to in my previous post were found on the first go because I know how to look for something. Your claim gets me curious: can you give an example of how you find something in a tracker way easier than on a forum?

Assigning statuses to bugs really only matters when people are actively waiting for it to be fixed. People will +1 a bug regardless of its status until a new build is being released. Unless we would have daily builds, this point is moot.

Molano wrote:There are plenty of reasons bug trackers work for a lot of companies and I won't go into them all here. I personally think introversion would benefit from using one immensely.

While meaning no particular offense to anyone, the majority of the current set of testers seems to have zero experience reporting bugs in a structured manner. If you don't know how to report properly, the reporting tool will not make a difference. I'm not saying that's not something you can learn (most testers grow accustomed to the process along the way) but it is one hell of a reason to not let just anyone use a bug tracker. Although I can't speak for IV, I'd say a private bug tracker is far more useful - the developers read the forums, filter the material and enter it in the tracker in a way that they know is helpful for internal use. Remember, they can use tools without you knowing (or needing to know) about it.

Bottom line: your output is only as good as your input, no matter how good the vessel. This goes for reporting as well as searching (and probably some other things).

Jelco
User avatar
KingOfZeal
level1
level1
Posts: 34
Joined: Mon Mar 07, 2005 7:02 pm
Location: Behind you

Postby KingOfZeal » Sat Sep 29, 2012 12:28 am

Mas Tnega wrote:
Lance_Lake wrote:Agreed in full. Uploading my dxdiag every single time I write a bug report.. Bah.
Why would a bug tracker be better for referencing your dxdiag.txt than just writing down where you last uploaded it?


Fogbugz (at least, which is the system we use at work) allows for automated bug reporting, if you integrate into it with their API. Basically, you can write code that when an unhandled exception is thrown resulting in a crash, it will automatically generate and upload the details to the system.
User avatar
jelco
level5
level5
Posts: 6018
Joined: Sat Feb 18, 2006 7:45 am
Location: Cygnus X-1
Contact:

Postby jelco » Sat Sep 29, 2012 12:33 am

That is a different point entirely. Automatic uploading of debugging data doesn't require a tracker per se, nor does setting up a tracker automatically offer this functionality without having to change the code of the game first.

Manually uploading or posting your dxdiag doesn't need to happen more than once. I actually pointed out in my bug reporting howto sticky that after the first post, it's preferred you link to your first post of the dxdiag to save cluttering up the forum. (That might not be the same as having everything done automatically but I'm sure you can spare those five seconds of effort per post.)

Jelco
"The ships hung in the sky much the same way that bricks don't."
- Douglas Adams
User avatar
Molano
level1
level1
Posts: 27
Joined: Wed Sep 26, 2012 4:25 pm

Postby Molano » Sat Sep 29, 2012 1:40 am

jelco wrote:
Molano wrote:I personally find it a lot easier to find bugs in a good tracker than in a forum. Also the developer can assign a status to a bug so that the users don't have to scan an entire thread for it.

How good you find something you're looking for really depends on how good you look for it. A bug would be a ticket or issue in a tracker, and is a topic here. There's the same deal of search functionality in this piece of forum software that can get you exactly where you want to be. Case in point: the topics I linked to in my previous post were found on the first go because I know how to look for something. Your claim gets me curious: can you give an example of how you find something in a tracker way easier than on a forum?

Assigning statuses to bugs really only matters when people are actively waiting for it to be fixed. People will +1 a bug regardless of its status until a new build is being released. Unless we would have daily builds, this point is moot.

Molano wrote:There are plenty of reasons bug trackers work for a lot of companies and I won't go into them all here. I personally think introversion would benefit from using one immensely.

While meaning no particular offense to anyone, the majority of the current set of testers seems to have zero experience reporting bugs in a structured manner. If you don't know how to report properly, the reporting tool will not make a difference. I'm not saying that's not something you can learn (most testers grow accustomed to the process along the way) but it is one hell of a reason to not let just anyone use a bug tracker. Although I can't speak for IV, I'd say a private bug tracker is far more useful - the developers read the forums, filter the material and enter it in the tracker in a way that they know is helpful for internal use. Remember, they can use tools without you knowing (or needing to know) about it.

Bottom line: your output is only as good as your input, no matter how good the vessel. This goes for reporting as well as searching (and probably some other things).

Jelco


Some good points and I won't awnser them all. I have tested using bug trackers on 3 occasions and without on a few more. Using a forum can work out just fine. Having just 2 forums for a game and a lot of different kind of threads does not help though. I remember testing I think asherons call 2 and they got a bug tracker they filled themselves. All testers were able to add reports and saves how they reproduced certain bugs. This resulted in a long list of bugs with very descriptive titles, statuses and in the end possible fixes that could be tested before a new build was released.

This list was easy to read due to the fact that you did not have to go through a lot of pages with other threads and post in them.

I'm not saying a forum is a bad tool for finding/reporting bugs. I am saying that a bug tracker is a very focussed tool for this specific topic and I think it would work well / better.

In any case at least make a dedicated moderated forum with only bug reports in them
User avatar
xyzyxx
level5
level5
Posts: 3790
Joined: Wed Jul 02, 2003 7:50 pm
Location: Iowa, USA
Contact:

Postby xyzyxx » Sat Sep 29, 2012 6:03 am

It's true, it'd be nice if the discussion threads and comedy threads could be moved to another board, making searching for bugs and feature requests easier, since bugs and feature requests would then be the only thing in the board.
Some people talk because they have something to say. Others talk because they have to say something.
SlashLife
level1
level1
Posts: 33
Joined: Fri Sep 28, 2012 8:52 am

Postby SlashLife » Sat Sep 29, 2012 9:12 am

Thing is: I'm pretty sure there's a bug tracker running "inside" already. and reports from the forum will be moved there. After all, there's most likely been quite a few bugs BEFORE the public alpha. The catch is: Do you really want people not associated with the team to have access to the internal bug info?
Do you want people to see that their bug is reported as "bogus" or "could not reproduce"?
Which other bugs are people allowed to access (for matters of finding a duplicate of your new bug)?
What about resolved, but yet unreleased bugs? (For example because they didn't pass QA yet.) How would people react if they don't make the next public build?
And what about the bugs which will be considered unimportant enough to go beta or even release without solving them first?
We also see the discussions about feature requests - I guess there are a lot of ideas floating around inside IV, which might be on a tracker for discussion away - do you allow the public to flood these tickets with useless comments along the lines "I like this idea" or "this sucks" (because we all know that well over 80% of the comments would simply be approval or denial of the idea without giving any reasoning for it)? Or would you instead deny the public the option to comment on the feature ideas - which would result in a shit storm alon "you're having a two class system here and don't even want our opinions!!!!11!". So if not for anything else - at least for feature requests opening the tracker for the public would put IV in a lose-lose situation.

And finally, having worked on the ticket receiving side of a bug tracker for quite a while ... every single bug tracker instance I've been involved with gives a great first impression of "Holy shit! How can so few people make so many mistakes." - especially if someone not involved with this line of work sees a five digit overall number of bugs. Sure - >90% of them may be fixed, but how can something be good if it had that many mistakes? That's definitely not a message you'd want to convey to potential (or in this case even actual) customers.

So coming down on it: Opening the tracker for the public would surely make life easier on the front line - but there are a lot of problems associated with it which are not trivial to resolve. That put aside we already *do* have the forum and we can *discuss* on bugs here - something bug tracker tickets are not meant for. Even only moving the forum over to the (hypothetically, but pretty likely existing) tracker would be a tedious process, binding IV resources which can be put to better use by continuing development of the product. And last but not least it would likely chase away a lot of not-so-technical people from reporting bugs. Because let's face it: Many of the bug reports here are far below standard and us technical folks may (or may not!) be better at making bug *reports*. But then again we also are locked into our analytical thinking and may not even percieve a lot of the bugs because we don't do "strange" things to the product. So in the end we'll need a good mix of people here and a forum makes a fine common denominator for everyone.
Pod
level1
level1
Posts: 28
Joined: Fri Feb 15, 2002 4:59 pm

Postby Pod » Sat Sep 29, 2012 11:56 am

jelco wrote:While meaning no particular offense to anyone, the majority of the current set of testers seems to have zero experience reporting bugs in a structured manner. If you don't know how to report properly, the reporting tool will not make a difference.


They're not testers, they're players who have bought the game and happen to report bugs.

It's important to remember that :)
k4kicks
level0
Posts: 8
Joined: Fri Sep 28, 2012 3:13 am

Postby k4kicks » Sun Sep 30, 2012 12:33 am

Pod wrote:
jelco wrote:While meaning no particular offense to anyone, the majority of the current set of testers seems to have zero experience reporting bugs in a structured manner. If you don't know how to report properly, the reporting tool will not make a difference.


They're not testers, they're players who have bought the game and happen to report bugs.

It's important to remember that :)


I agree, no one means any disrespect by double posting. We all love the game, and want to help.
Anovadea
level0
Posts: 3
Joined: Sun Sep 30, 2012 2:20 am

Postby Anovadea » Sun Sep 30, 2012 2:42 am

Mas Tnega wrote:
Illanair wrote:I'm all for this - Not only would it potentially help to keep duplicates from happning (to an extent),
I find that highly dubious. If you can't be arsed to check the forum, chances are you just can't be arsed to check. Just the fact that this thread exists proves that. It doesn't matter if you look for tracker or tracking, there's a result for both. The OP didn't do this.


The benefit of a proper bug management system isn't to try and find duplicates (although frequently that's a bit easier, so that is a bonus), but that the developer is able to wrangle the extra reports and easily mark them as duplicates. That's the sort of thing that makes bug wrangling that much easier. It also has a few other benefits (like being able to track how much work each developer has).

For what it's worth, my suggestion is JIRA, but that's because I've seen it in bitbucket, and it didn't make me want to tear my hair out.
User avatar
jelco
level5
level5
Posts: 6018
Joined: Sat Feb 18, 2006 7:45 am
Location: Cygnus X-1
Contact:

Postby jelco » Sun Sep 30, 2012 2:59 am

Anovadea wrote:The benefit of a proper bug management system isn't to try and find duplicates (although frequently that's a bit easier, so that is a bonus), but that the developer is able to wrangle the extra reports and easily mark them as duplicates. That's the sort of thing that makes bug wrangling that much easier. It also has a few other benefits (like being able to track how much work each developer has).

Let's assume for a moment that IV use a bug tracker internally and manually transfer info from this forum to that tracker - how would them checking issues and marking them as duplicate differ from them checking topics and not transferring them if they're duplicate? In the end it's the same amount of work (within margins of insignificance anyway).

At the end of the day, these are arguments that would help developers, but not reporters. In fact, most other arguments I can name for using trackers (e.g. at work) are from a developer's point of view. So why are some reporters (yes, that's what we all are in here) concering themselves so much with this decision that really doesn't make a lot of difference to the reporting side of things? I repeat: the way in which reporting is done is far more deciding in the quality of the results than the tool used for it.

Pod wrote:They're not testers, they're players who have bought the game and happen to report bugs.

That's avoiding the point. People might not have signed up for the tester role per se, but what happens in this forum is really very much about testing. It doesn't matter what label you have - whether you call yourself a tester or not, bug reporting should happen in a structured manner to be useful. ;)

k4kicks wrote:I agree, no one means any disrespect by double posting. We all love the game, and want to help.

I never said I interpreted it as disrespect, or felt offended by it - what gave you that impression? I'm just trying my best to keep things organized which is in everyone's interest, which involves explaining some ground rules and guidelines. I'm basically telling you "you might want to do this differently" - that's a long way from "your ways suck", which is very much not how I bring this.

Although I appreciate the explanations you all come up with for this unstructured behavior, it's really not necessary. I know full well that most of the newcomers will not be familiar with bug reporting in any way, let alone this particular method, but I don't blame anyone for that (that would be stupid). I'm just trying to point out some basic methods and tools to keep everything organized which will help both the developers (by providing structured feedback) and the reporters (by e.g. avoiding doing work that others have already done).

Jelco

PS: Just please don't start using these explanations as excuses: if you want to hide behind a statement of "I'm inexperienced" and don't process the feedback that's being given about your style of reporting (because that's what it is - feedback) you're not being very helpful at all and might just as well not post here at all.
"The ships hung in the sky much the same way that bricks don't."

- Douglas Adams
Pod
level1
level1
Posts: 28
Joined: Fri Feb 15, 2002 4:59 pm

Postby Pod » Sun Sep 30, 2012 7:22 pm

jelco wrote:That's avoiding the point. People might not have signed up for the tester role per se, but what happens in this forum is really very much about testing. It doesn't matter what label you have - whether you call yourself a tester or not, bug reporting should happen in a structured manner to be useful. ;)


I agree. But you can't control players on a forum, whereas you can control testers. We can all sit here talking about reposts and proper bug structure til the cows come home but that's not going to stop jonny-just-purchased from splurging all over the place, as it's just a forum.
Anovadea
level0
Posts: 3
Joined: Sun Sep 30, 2012 2:20 am

Postby Anovadea » Mon Oct 01, 2012 3:20 am

jelco wrote:
Anovadea wrote:The benefit of a proper bug management system isn't to try and find duplicates (although frequently that's a bit easier, so that is a bonus), but that the developer is able to wrangle the extra reports and easily mark them as duplicates. That's the sort of thing that makes bug wrangling that much easier. It also has a few other benefits (like being able to track how much work each developer has).

Let's assume for a moment that IV use a bug tracker internally and manually transfer info from this forum to that tracker - how would them checking issues and marking them as duplicate differ from them checking topics and not transferring them if they're duplicate? In the end it's the same amount of work (within margins of insignificance anyway).


But the situation you're describing there (players log to forums, and the devs copy to internal tool) doesn't make it that much easier. It still keeps the forum noisy, and now you have to wrangle TWO areas, the forum and the tracker. And if you're not managing the duplication in the forums, then you still have to wade through a whole load of noisy and duplicated topics in order to gather information from each into the bug report. That would be a costly process (in terms of developer time).

At the end of the day, these are arguments that would help developers, but not reporters. In fact, most other arguments I can name for using trackers (e.g. at work) are from a developer's point of view. So why are some reporters (yes, that's what we all are in here) concering themselves so much with this decision that really doesn't make a lot of difference to the reporting side of things? I repeat: the way in which reporting is done is far more deciding in the quality of the results than the tool used for it.


Yes, I agree fully that the quality of the reports matter more, but process management (and accompanying software) can take SUCH a sting out of this. Basically, the less time that's put into wrangling reports, the better. I think we can agree on this, right?

However, I disagree with an earlier contention of yours that a forum is as useful as a dedicated tracker. And I disagree that having both the forum and the tracker would be a good solution either. I work in software QA, so I'm biased in favour of a tracker :) , because they DO help me as a reporter, partially because they help me avoid logging duplicate bugs. A bug tracker is also helpful to me because, if designed well, it can facilitate the conversation between the reporter and the developer, or even the conversation between developers.

Imagine this, someone logs into the tracker, and they click "Show all customer issues (no duplicates)". Imagine how many subject lines you'll see compared to how many you'll see on this forum? Would that, in your mind, make it easier for a person to say, "Ok, someone else has seen this"? Would it make it easier for people to view that topic and contribute any extra information that they can? In my case, because I'm trained not to log something unless I *KNOW* it's a dupe, I'll probably hold off logging it and end up getting lost in the results thrown up by phpBB's search function.

That's my main reason for wanting a tracker exposed to us and, to be fair, it's not the highest thing on my list. I'd rather enjoy the game and chew the fat on the forums, rather than stick my oar into their dev processes any further than I already have. So I'll shut up now. :)
User avatar
XmAkiNA
level2
level2
Posts: 80
Joined: Tue Aug 06, 2002 8:56 pm
Location: Oxford
Contact:

Postby XmAkiNA » Mon Oct 01, 2012 10:28 am

Mark has heard you and proposing Mantis. Might be worth taking any further conversation over to his thread: http://forums.introversion.co.uk/prison ... .php?t=471

Return to “Community Members”

Who is online

Users browsing this forum: No registered users and 2 guests