Tree-Type Notes in Exploratory Testing
Tree-Type Notes in Exploratory Testing
March 29th, 2017
What are Tree-Type Notes?
During years of coaching and training of exploratory testing, I gradually developed my own style of notes-taking, and I named it - Tree-Type Notes.
What do Tree-Type Notes look like? Let's look at an example. Within around one hour's exploratory session, I recorded this notes using one piece of A4 paper. It really looks like a tree, isn't it?
How to Draw A Tree-Type Note?
It's so easy to draw a Tree-Type Note.
Just start from the root of the tree, then draw different branches in turn, upwardly, along the trunk.
Exploratory testing notes are not taken afterwards through your memory. In stead, you take necessary notes while you are exploring a product. And you can make whatever marks you like based on your needs.
So, what contents can be included in a Tree-Type Note?
Exploration Routes (branches of a tree)
Each branch can be an area of exploration. For the case above, I explored several areas of a mobile App, such as UI layout, device rotation, capture image, etc., which I marked as circled 1 to circled 5 in the graph.
Testers care about bugs. Whenever finding a bug, mark it clearly in notes. You may draw a separate branch called "Bugs". Here in this example, I just noted down each bug nearby its testing areas, but I draw a special "bug" icon to emphasize. So it's easy to know I found 3 bugs in this session by counting the number of "bug" icons.
Similarly, you may set different branches called "Issues", "Questions" or "Risks", etc. These branches are usually what you need to do some following-up work after your exploratory session.
Measurement Marks (T/B/S/N/O)
Sometimes managers care about measurements related with sessions, such as time used for "T - Testing design and execution", "B - Bug Investigation and Report", "S - Setup", "N - Non-session activities", "O - Opportunities", etc. These activities can be marked clearly in a Tree-Type Notes (Figure 2).
Figure 3 shows another way of marking measurement activities. Blocks colored in green represent T time, yellow for S time, and red for B time. (This note is taken through XMind and is a upside-down Tree-Type Note.)
Your Testing Mind
What? Can testing mind be recorded, too? The answer is "Definitely yes". Of course, a pre-assumption is that you know your testing mind very well.
I like to use simple heuristics to learn my testing mind and I coined a name "KLTDR" for it.
K - Knowing Your Mission. To put it simply, in most cases, KYM is the very first thing you need to do when you start to explore. KYM is to collect information about your customers, system under test, your project and your mission. (I use a whole chapter to describe the skill of KYM in my book Buccaneer Testing Analysis: MFQ&PPDCS, in Chinese.)
L - Learning and collecting more information during exploration
T - Trying to solve problems or trying to explore
D - Deep Exploration. Not just shallow testing, but deep testing around a testing point.
R - Repeating the above steps.
Please be aware of that K-L-T-D are dynamic activities, so there does not exist fixed sequences of them. In fact, K-L-T-D can be in any form of combinations, each of them can appear multiple times during an exploration session, i.e. the number of cycles R can be many times.
Moreover, KLTDR heuristics can be applied in any exploration activities, not just software testing. The following notes were taken during my exploration of Sudoku game. I recorded activities of KYM, different tries, deep exploration and learning. In the upper left comer, you can see that I even draw an exploration depth graph. (In Chapter 5 of my book, the concept of "Testing Depth Graph" is described in detail.)
In figure 2, you can a see a smile face icon. It represents the feeling of the explorer at that moment. Right beside the smile face icon are some words like "defocusing" and "creeping and leaping". These are the testing techniques that the explorer is just using. Usually, a tester won't record techniques he or she is using, but this information is quite valuable for a coach - I often record testing techniques used while I'm pairing with and coaching a tester so that we can have an effective debrief after the session.
Another branch you can add in Tree-Type Notes is "Not Tested", which means things that you thought of but you won't test in the current session - you will probably test them in future sessions or you may plan not to test them at all.
Actually, you can record any things you care about in Tree-Type Notes. Just be careful not to make your notes messy by including too many things. You need to make a balance.
What are the benefits of Tree-Type Notes?
As far as my limited coaching and practicing experience, I would dare say that I benefit a lot from this Tree-Type Notes. Let me just name a few.
Explicit Whole Picture
Tree-Type Notes offer a whole picture of your exploration along a timeline. At any time during or after your exploration, you always have a clear clue about your exploration routes. This is crucial for your self-management and self-improvement.
Once I let the tester I coached to take some notes while doing exploratory testing. Here is her notes in Figure 5.
When we had the debrief after the session, the tester could hardly remember the routes explored or those interesting points, but only a few tests she did.
I believed many valuable ideas were come up during her exploration. Tree-Type Notes can remind us how ideas travelled. we can pick up those interesting ideas to analyze and to learn. With messy notes, we can hardly do this.
Beneficial For Communication
It's no doubt that Tree-Type notes help a lot during debriefs. You can show what you did, what you found, and how well your exploration was. You can discuss quality risks, testing strategy, measurement evidences and your testing progress based on your notes.
What interests me most is that I can do a private debrief for myself with Tree-Type Notes. Right after a session, I will take some time carefully studying what I can learn from this session from skills and knowledge perspective. Only by doing so, can I steadily get improved with my exploration. Sometimes, I draw another learning Tree-Type Note just for the sake of recording what I have learned from a session, like this one. Noticed in Figure 6 I didn't record the full exploration map, but only record the points from which I got much insight.
Showing Connections Between Ideas
Once knowing connections between exploratory ideas, we can utilize them to do many different things. Let me just show one example (Figure 7).
When I solve the Sudoku puzzle No.074, I made a mistake at the 9th trial branch at the time marked 9:26. I found a conflict with the result of 4th trial branch. I have explored for about 45 minutes up to now. How can I know in which step I made a mistake? I felt exhausted at first. But fortunately I have my Tree-Type Notes. So I decided to go back step by step to find the error. Soon, I found the mistake I made at the 8th trial branch. Now I felt much happier. I fixed the error and then solved the puzzle very quickly. Then by studying the notes for minutes in a self-debrief, I learned why I made such a mistake. I also think of some ways that can help me avoiding similar mistakes in the future.
This example has proven Tree-Type Notes to be highly effective means of skill improvement - I improved my skills from the mistakes I made before.
Showing Anchor Points
What is an Anchor Point in exploration? I often use the words to describe such a moment in exploration: sometimes you fall into a confused state and don't know what to explore next. You spend a lot of time thinking and trying, finally you think of an idea and then begin an in-depth exploration from it. So, after a long period of no progress, there was an idea that suddenly advanced your progress. I call this idea an Anchor Point.
Right after an exploratory session, with my Tree-Type Notes at hand, I can easily identify those Anchor Points. I can learn a lot from analyzing these Anchor Points by asking questions like: Why does it take me so much time to think of this idea? What skills or knowledge do I lack of? How do I discover this Anchor Point? Can I come up this idea again in future explorations?
Don't Take everything down in Tree-Type Notes
Sometime managers require testers to note down every detail of their exploration. Or sometimes testers just want to record everything they did during exploration. But what you did is different from what you explored. To explore means trying to find new information or trying to find unknown things. Not everything you did is worth to being recorded, actually. However, there is frequently much value in what you explored. Your notes are not just peer copies of what you did. Allocate the time for taking notes and for exploration wisely.
The example in Figure 8 shows some over-detailed information in notes. For instance, if you go from one page to another page, you don't need to note it down.
Finally, like everything else, you need practices to have high quality Tree-Type Notes. You need a number of skills to have high quality exploration and the ability to draw Tree-Type Notes is just one of them. But one thing for sure, if you have mastered the skill of Tree-Type Notes, you will learn faster in whatever explorations you do.