Discussion of "STAR" with Dale Perry
I attended StarEast and StarWest - the international software testing conferences - for several times. Before that, I successfully got ISTQB foundation level certification. I always have a doubt, i.e. why is it called STAR – Software Testing Analysis and Review? At StarWest2011, I came across Dale Perry at the exhibition area of SQE, so the following conversation took place. (Fortunately I took some notes right after that conversation.)
Xiaomei: Hi Dale! I have a question. Can you explain to me what is STAR, why you call it STAR?
Dale: STAR represents Software, Testing, Analysis, and Review, the acronym.
Xiaomei: I know that. But in my opinion (based on ISTQB), review and analysis are part of testing.
Dale: But reviews are different. Reviews are static testing.
Xiaomei: So you mean that testing here is actually dynamic testing?
Dale: Yes, for software testing, analysis and review, because analysis should not only for “testing around analyzing”. You do risk assessment, risk validation…
Xiaomei: But these belong to testing field.
Dale: But analysis is not just testing, you can do design analysis; you can also do integration analysis. So here we are talking about using analysis as a testing function. STARS are based on all three, testing, analysis and review, which are both static and dynamic quality of software.
Xiaomei: Well, you know, ISTQB divides testing into static testing and dynamic testing. And static testing is further divided into static analysis and review. So analysis and review should not be in parallel with testing, but be included into testing instead.
Dale: But reviews are not just for testing. Reviews are also used for making sure that you have all requirements.
Xiaomei: Yes, you can review the requirements documentation; you can review design documentation. You can review a lot of things. That is part of testing job.
Dale: Two risks to review. You review for quality, like the ambiguity things; or you can review for completeness. In other words, this is what I want to do, and this is what I don’t want to do. So reviews can be used in many different ways. It is not just a testing technique. Review has been around since the early 60’s. They were developed by IBM. But their purpose was not related with testing.
Xiaomei: Reviews are about quality.
Dale: Well, reviews are not just about quality. They are also about: do I have everything I need to have. That’s not a quality issue.
Xiaomei: You want to make sure that the function is correct and complete. This is a quality issue.
Dale: No, not only the functions. I want to make sure that I identify all the purposes that the user wants in the software.
Xiaomei: That is one of the purposes of testing.
Dale: That’s not part of testing. That’s part of development process.
Xiaomei: According to ISO9126, functionality includes four parts: accuracy, completeness, interoperability and security.
Dale: You are talking about a single function. I’m talking about the whole system. You can do architecture reviews with nothing to do with quality. They can decide which platform I’m going to use, Microsoft, Linux, etc. That’s not a quality issue. That’s an architecture issue. But you still have to do a design review to decide which type of architecture you are going to build on. Then you can start to build functions. Testing is to make sure those functions can work within the architecture.
Xiaomei: Then, in you opinion, when is the time we start our testing work? Do you mean before we know what kind of architecture we will use, we’re not doing testing?
Dale: If a tester participates an architecture review, he brings the testing viewpoint. But that’s just contributing a testing viewpoint to a non-testing activity. Reviews are used for two things: to ensure quality (part of testing), and to gather information. For example, I want to identify all user stories. I can review it to guarantee the user stories are complete before I get into the development and test.
Xiaomei: Are you familiar with Richard Bender’s Requirements Based Testing Methodologies? The first step of ReqBT is to make sure requirements are complete, accurate, etc.
Dale: Well, you’re talking about once the requirements, once you have them, then you have to check the ambiguity for completeness. First you have to gather them. Reviews are not just for testing. Because ISTQB defines that way, people get false conception. Review can be used for both development purposes and for testing purposes. Reviews are different from testing. Analysis is something we do to decide which approach to take to testing and which staff to test. There is no actual testing involved in it. It is just decision-making. For example, risk analysis model is not testing. That’s analysis.
Xiaomei: When I do analysis, this is part of my test analysis and test design work.
Dale: But analysis for testing is different from analysis for other things. Risk analysis has nothing to do with testing. But we teach risk analysis at STAR. We do that because analysis makes testing more efficient, but it is separate from testing. Testing can be static and dynamic. I can do static testing without reviews.
Xiaomei: You can do static testing without reviews, without analysis? How do you do that?
Dale: This is the problem of ISTQB. It is too narrowly focused. It gives people the false perceptions: this is the answer. I can do static testing without going to a meeting, without anyone involved, using a technique called fault-attack and error guessing. I can use that as a form of static testing.
Xiaomei: isn’t that a kind of review?
Dale: Review requires two people at a minimum. You do something and you give it to somebody else to review. But I can do that type of testing without giving anybody else. But it is static testing. Static testing simply means applying test methods, test techniques without executing the software. If you write something down, and if you try to review it, you know what you know.
Xiaomei: But still I can find problems. Every time I write a blog, after that, I re-read my blog, and I often find some faults in it.
Dale: You’re doing simply syntax checking and grammar checking.
Xiaomei：No. Including the ideas checking as well.
Dale: If you’re doing by yourself, you’re not doing review. You’re doing proof checking.
Xiaomei: OK, maybe our understanding to some certain words are different.
Xiaomei: But people do the same activities with different understanding, different focus.
Dale: If I’m a developer and I’m reading something, I’m not checking quality. I’m checking for how I’m going to build it.
Xiaomei: Whether you’re checking the quality depends on your purpose of reading. If I re-read my blog and my purpose is to find some errors, I think this is a kind of testing. And I did not run it, so I think this is review.
Dale: Well, you can call it that, but reviews typically involve somebody else. Review is to get more opinions. Re-reading your own staff, you still have only one opinion.
Xiaomei: Why we must define a term in such rigidity?
Dale: We don’t have to have that rigidity. But the industry in general likes to have terms defined, so we have a way communicating. Otherwise I will call it an apple, and you will call it a watermelon. We have no common vocabulary. We need to have some standard definitions. The same thing happens with reviews and analysis. After you know the definition, you have flexibility. The key for a definition is not to have rigidity, but for a platform to work from. So now you can say, for me that is review. We can agree and disagree for certain kind of things.
Xiaomei: Even the testing glossary itself has a lot of space for discussion.
Xiaomei: Thank you so much.
Dale: You’re welcome.
You can see how deeply affected I was by ISTQB at that time. After that discussion, gradually I begin to walk out of ISTQB's shadow – considering testing in a much broader range and in more practical way; I begin to learn many other testing theories or approaches like exploratory testing, testing mind, rapid software testing, etc.
Thanks to Dale.