It was a Monday afternoon, our testers finally got together. So we decided to start our testing dojo.
and 5 - 10 testers. 1 or 2 testers played the role as performer who tested on the pc which was connected to projector. While others played the role as auditor who watched the process on the projector. After testing finished, auditors could challenge on the testing process, tools selected, testing methodologies applied, etc.
In this session, we decided to do pair testing:
* one tester testing and
* one helper recording issues and contributing ideas.
Jing as tester, and Yan as recorder, made the performer team. The SUT(system under testing) was a http proxy server with white list control.
The performer team spent 10 minutes discussing the functionality, then started testing.
* Jing started Firefox, and configured proxy server to the system.
* She defined several allowed URL in the white list, then tried to access the URL in Firefox.
* She tried to access several out of scope URL.
* Here Yan reminded the risks with special char in URL, Jing followed her suggestion with ? and # in the URL.
* then a series of allow/deny with different types of URL. (ftp, http, https, file://)
Testing part took about 50 minutes. That left us 30 minutes for dicussion. All 3 of our auditors were lost while following Jing's testing process. So we asked a lot of questions to clear our minds.
Q: Are you doing function testing or scenario testing?
A: I'm not sure. I just did what I think is necessary.
Q: If you are not certain about what kind of testing you are doing, how could you focus?
A: I think both function testing and scenario testing would end up into almost the same test cases eventually.
Q: while you are testing special chars, why those 2 - # and ? - but not the others?
A: I think it is the most commonly used special char in URL.
Q: Most commonly used by user? So you are doing scenario testing?
A: Yes, I guess. I might be doing user scenario testing, I think that's what users probably do.
Q: What would you call it - your methodology today?
A: I think it is kind of like exploratory testing, I do not know the software before, and I'm trying to get familiar with it while testing it.
Our session ended in discussing whether a tester should always be aware and follow some kind of testing methodology or tester sometimes could follow one's instinct. Some of us believed that testing a new product was OK to follow one's experience or say nature, just test whatever occured.
I disagreed. Tester should always be aware of testing methodologies. Tester without testing methodology became black box software tester. It was not the one who did black box software testing. It was a type of tester who took requirements or sometimes no requirements, pre-designed test cases or sometimes no test cases, and the SUT, went back into a black box. Several days later, the tester came back with a list of bugs. No one would ever know how he/she did it.
That including
* how he/she analyzed the requirement,
* how he/she intepretted the test cases,
* how he/she interacted with the SUT.
We could only believe in black box software tester's experience or instinct. That didn't sound good. But the truth was everyone was a black box software tester for some time. When you were tired of a product, when your schedule was too tight to fit in, when you were unfamiliar with the SUT, when you had test cases too hard to understand, you became
black box software tester, you knew you were testing, you were finding bugs, but you didn't know what exactly you were doing and why you were doing so.
没有评论:
发表评论