A detailed overview of Chapter 1 of Alison Gopnik’s book, ‘The Philosophical Baby’ [1]

We don’t just live in the present world.  ‘Counterfactuals’ are the ways we could have lived in the past, and all the ways the world could be in the future (‘the woulda-coulda-shouldas of life’).  Scientists have thought it puzzling that humans would think about those alternate worlds rather than focusing on the present.  Alison Gopnik has answered this and similar questions by studying small children [2].  Rather than opposing concepts of fact/fiction, science/fantasy, ‘the same abilities that let children learn, allow them to imagine alternate worlds.’

‘Children’s brains create causal theories of the world, maps of how the world works.  And these theories allow children to envisage new possibilities, and to imagine and pretend that the world is different.’

Counterfactual thinking is an important part of our lives.  In experiments psychologists have found that barely missing a flight has a greater impact on a person than missing a flight by a large margin.  The evolutionary answer why this matters is that counterfactuals let us change our lives.  Considering alternatives allows us to ‘intervene’ in the future.  ‘Counterfactual thinking lets us make new plans, invent new tools and create new environments.’

Can children think counterfactually?

In an experiment with a stacking rings toy, one ring was taped up.  Fifteen month olds would try stacking the taped up ring repeatedly without success.  Slightly older, eighteen month olds would look at the taped up ring in disbelief.  Without trying, they could imagine the consequences.  ‘Simple trial and error’ can be effective in many situations; ‘anticipating future possibilities lets us plan in this other more insightful way.’

Reconstructing the past [3]

To determine if young children can imagine past possibilities, psychologists conducted experiments when children were asked questions such as, ‘What would have happened to the floor if Ducky had cleaned his feet first?’  Children could successfully ‘imagine and reason about an alternative past.’

Counterfactual thinking in play

Children start pretending at a very early age.  Pretend play is soon followed by treating objects as if they were something else (a ball is an apple).  Children would engage in pretend play even if they didn’t have toys that support this behavior.

Talking gives babies a ‘whole powerful way to imagine.’ Babies, using single words, talk about ‘actualities’ as well as ‘possibilities’ as soon as they begin to talk, such as the universal ‘night-night when putting their doll to bed.’  Gopnik describes how the word ‘no’, more than stubborn refusal, has much more subtle and important variations, such as ‘are you kidding’ or ‘no way’.  ‘No’ and ‘uh-oh’ ‘puts you in the world of the counterfactual and the possible – the road not taken, the possibility that isn’t real.’

Two and three year olds are very comfortable creating pretend creatures and worlds.  ‘They are adept at playing out the consequences of their counterfactual pretend problems.’  ‘When imaginary teddy spills imaginary tea, the imaginary floor will need an imaginary mop up.’  Like scientists in the past, we might be under the misconception that imaginative play is a sign of ‘cognitive limitations’.  Recent studies show that children are ‘extremely good at distinguishing imagination and pretense from reality.’

‘Children may seem confused because they are such expressive and emotional pretenders.  They can have real emotional reactions to entirely imaginary scenarios.’  Although they may deny there is a monster in the box, they step carefully away from the box when no one is looking.  Their reaction is not very different from adults.  Putting a label marked ‘Cyanide’ on an ordinary bottle of water, will invoke a reaction from most adults.  It’s not that children ‘don’t understand the difference between fiction and fact…they are more moved by both.’

How do we produce counterfactuals?

Counterfactual thinking is partly because of our ability to think causally.  Causal knowledge is when one event causes another and not just when one event follows another.  Knowing that one event causes another allows us to think about what might happen in the future and what could have happened in the past.  ‘Once I know that smoking causes cancer I can imagine possible worlds in which my actions cause people to stop smoking, and conclude that in those worlds they will be less likely to become ill.’

Causal knowledge allows us to change the future. We can take preventive actions or change the course of events.  This is different from noticing that things are different without understanding why.  When a fifteen month old can’t stack the taped up ring, he will not repeat that action, without really understanding why.  ‘But having a casual theory of the world makes it possible to consider alternative solutions to a problem, and their consequences, before you actually implement them, and it lets you make a much wider and more effective range of interventions.’

Can children think causally

Recent psychologists believe that children can think causally. Even very young children can demonstrate causal thinking, e.g. ‘the teddy bear’s hand broke because you twisted it too far.’  ‘Causation is what gives fantasy its logic.’  Children’s pretend play and fantasy games work ‘by establishing imaginary premises and then working out the consequences of those premises quite strictly.  ‘You hafta drink your milk ‘cause you’re the baby!’ (Gopnik’s example).

Children have unconscious theories of the world

In order to ‘have causal knowledge about the world and use that knowledge to make predictions about the future to explain the past, and to imagine possible worlds that might or might not exist’, Gopnik asserts that children must have ‘everyday theories of the world – everyday ideas about psychology, biology and physics… these theories are largely unconscious’.

Maps and blueprints – spatial maps [4]

‘Children’s brains construct causal maps of the way the world works.’  These are similar to the maps we use for travel.  Gopnik introduces the term ‘blueprint’, for a map of how we would like the world to be.  You can also think of blueprints as a map with alternate routes.  Imagining different routes is equivalent to creating a blueprint, i.e. changing the map to how you want it to be.  When looking at a landscape a gardener constructs a blueprint, a modified version or vision of how things should be, rather than the natural landscape.  Creating a blueprint allows us to move objects to realise our vision.  Maps also allow us to imagine alternate possibilities without yet committing ourselves.

Causal maps – complex causal relations among events

‘Human beings construct a map of the complex causal relations among events.  The neurologist has a kind of map of migraines – outlining all the causal links between neural activity and pressure and pain.’  A group of computer scientists and another group of philosophers used the idea of causal maps to create software, which could ‘accurately generate new predictions, interventions and counterfactuals.’

‘The core idea of cognitive scientists is that our brains are a kind of computer, though far more powerful than any of the actual computers we know about.’  Gopnik asserts that ‘children’s brains just might be constructing causal maps and using them in the same way as computer programs.’  Children’s theories of every day life are encoded as causal maps.

Do children make causal maps?

Gopnik created a machine [5] that responded when ‘blickets’ [6]  were placed on it.  Children very quickly understood the causal connections, i.e. which object is a blicket and which isn’t.  They were also able to think about possibilities – what would make the machine stop.  In further experiments, after observing a machine responding to spoken comments, children changed their expectations/causal maps and spoke to the machine to manipulate it.  In the same way, for adults, ‘… new causal knowledge allows us to imagine possibilities that would have seemed unimaginable before.’

‘Often, people treat knowledge and imagination as if they were different, even as if they were necessarily opposed to each other, but the new work on causal maps suggests just the opposite.  Understanding the causal structure of the world and generating counterfactuals go hand in hand.  In fact, knowledge is actually what gives imagination its power, what makes creativity possible.

It’s because we know something about how events are connected in the world that we can imagine altering those connections and creating new ones.  It’s because we know about this world that we can create possible worlds.’

What does this tell us about software testing?

As software testers, we want to keep creating causal maps that will help us generate counterfactuals. In this discussion let’s imagine that we are testing a physical object, such as a Lego set or a home furniture construction kit (I explain the significance of this later).

I will use examples from software, but will treat software mostly as a black box.

Imagine that I am testing a web browser. Of course like babies, I have multiple tabs and multiple windows open. The browser seems to get slower and suddenly crashes. When I reopen the browser I don’t know which tabs were open! After checking I realize that there is an option to open the browser with the last saved tabs. What is the default option? If the default option is not to save tabs, doesn’t that mean that the user will lose his work through no fault of his own?

I continue banging on the browser as before. This time when the browser crashes I feel almost happy that when I reopen it, I won’t have lost my work. Alas! The browser does open with tabs, only in the active window! I realize that the browser only saved the tabs in the active window. Unfortunately, my active window was the more recent. The other window had much more tabs. I realize that the browser has the concepts of window and tabs. A window can have multiple tabs and the browser itself can have multiple windows. I wonder if there are other situations when the concept of a window is relevant. What about if the user has multiple monitors? If the browser has multiple windows on different monitors, do other applications or the operating system know that the browser now spans multiple monitors? I also wonder what happens if the browser window is not opened by the user, but opened as a popup by a web page, e.g. a popup opened by a banking application?  If the browser then crashes, the user may lose all his windows and tabs.

What I was doing here is building a causal map of how this part of the browser works.  Once I infer how windows and tabs work I can generate counterfactuals about other behavior, for example when using multiple monitors.

Software is not visible

A very important aspect of software is that most of what happens in software is not visible, in fact is not tangible [7].  In my office parking lot when exiting, I saw the exit LED sign display the message ‘6 days’.  I figured that my parking permit expires in 6 days.

Would it have been better if the sign had said ‘Expires in 6 days’?  Since I have been using the garage for a few years, I knew that earlier it actually used the phrase  ‘Expires in 6 days’.  I’m not sure if someone made a mistake when reprogramming the sign, but I don’t really know what is going on in the software (of the sign).  In the case of software applications there is much more complex processing which is not evident to an external observer.  The cause and effects or causal maps of physical objects may not directly apply to software.  What this means is that you will still build causal maps, but there may be far reaching effects of a user’s actions that you may not be aware of.  You could/should infer these effects by focusing on various external behaviors of the software.  Of course, you could also work with the developer to understand how the software was implemented and then think about cause and effect (you still want to keep building causal maps).

Should you act completely uninhibited when testing?

If you wear mismatched socks to work or throw applesauce at your co-workers during lunch you probably won’t increase your chances of a promotion.  Suppose you were testing a bank ATM, should you bang away at the keys like a two year old?  Although you may not spend the whole day doing that, you could spend some time banging the keys (not literally; although you could do that too).  Generally, when testers spend time banging keys, and they discover a defect, i.e. the software crashes or fails in some way, it’s usually the final outcome.  Of course defects are important.  However, what Gopnik’s book should teach us is that what you are really looking for is events that help you to create a causal map.  When my TV recorder acts up I think it may be due to excessive usage and the disk being fragmented.  If I were testing the TV recorder software I would then think of ways to create excessive usage and to see what happens when the hard disk is fragmented.  Note that is the starting point of my exploration, or as Gopnik might say ‘generating counterfactuals.’

It is also probably not effective to spend the whole day ‘banging keys’.  Gopnik’s analogy to adult fiction writers is well suited to adult software testers:

‘Adult fiction sits between the wildly uninhibited counterfactuals of childhood and the sternly practical ones of adulthood.  One way of thinking about adult fiction writers is that they combine the cognitive freedom of childhood with the discipline of adulthood.

Adult dramatists, unlike most adults and like children, are exploring the possibilities of human experience for their own sake.  But unlike children, and like their fellow adults, they do this on purpose and with the dedication and discipline that all adult endeavors require.’

What can adult testers do differently from babies?

• They can prioritise what they want to focus on.
• They can brainstorm ideas with others.
• They can periodically review the work they have done and choose to change course if necessary or desirable to do so.

No matter what their approach, what they cannot avoid is the creation of causal maps and generation of counterfactuals.  You should use these innate abilities in learning/testing.

Why is software testing difficult?

In an experiment, children replied in the negative when asked if a machine would respond to spoken commands (such as ‘stop’).  However, when the experimenters demonstrated the machine responding to speech, the children were willing to try and even used their own commands on the machine.  ‘Giving the children new causal knowledge changed the way they thought about possibilities, and changed the kinds of actions they would take.  Children could imagine a listening machine that seemed downright impossible before.’

‘In the same way, for adult scientists, new causal knowledge allows us to imagine possibilities that would have seemed unimaginable before.  In science fiction films, where imagination should run riot, what strikes you most is that the directors’ imaginations are so limited by their current knowledge.  In Blade Runner, for example, Harrison Ford runs desperately to a pay phone – with a video screen.  The screenwriters could imagine a pay phone with a TV, but not that pay phones would disappear altogether.  Nothing is more dated than visions of the future, because to imagine the future’s possibilities, we need to have the futures knowledge.’

If the software that you are testing works with devices, such as a camera, until you realize that the camera may not respond, you may not be able to imagine what the user can do when that happens – do they reconnect the camera, do they reinstall the drivers?

If your software has a dependency on an external device driver, what happens if the driver is updated?  Can you make sure your software will continue to work?

As a software tester your imagination of future possibilities is limited by your knowledge of the future.  Discovering causal maps, which in turn lead to more causal maps can help you to imagine future possibilities (generate counterfactuals).

Related work

The phrase ‘exploratory testing’ has been used for more than twenty years.  James Bach’sarticle as well as Cem Kaner’s article on exploratory testing has a direct comparison of ‘exploration’ to testing.  Michael Bolton has written a series of articles related to maps and exploration – Testing Without a Map, A Map by Any Other Name, Cover or Discover.  James Lyndsay’s Blackbox testing machines explore/teach cause and effect.

Note that in the last few years’ mindmaps have become very popular in software testing.  Mindmaps are different from the causal maps described by Gopnik.

About Alison Gopnik

Alison Gopnik has her own website.  You can follow her on twitter.  She has spoken onTED.
The book discussed in this article is ‘The Philosophical Baby’.

References

[1] This overview has been written in Gopnik’s voice – this is a condensed version of her text.
[2] The book ‘focused on children under five’. ’Babies’ refers to children under three.
[3] This is the same section title as used in the book
[4] The book uses the section title ‘Maps and Blueprints’
[5] This is similar to James Lyndsay’s blackbox testing machines.
[6] A ‘blicket detector… is a square box which lights up and plays music when some blocks, but not others are placed on it.’  The experimenters told children ‘Look here’s my blicket machine! Blickets make the machine go.  Can you tell me which things are blickets?’
[7] How to Observe Software Systems, Gerald M. Weinberg, Chapter 3, Visualizing the Product

About the Author

Nilanjan Bhattacharya manages a test team in the R&D lab of IBM Security Systems in Singapore. He has 18 years experience working with software development and testing teams across U.S., India and Singapore, working on both consumer and enterprise products. His experience includes CAD/3D graphics, enterprise security software and text analytics software. He strongly believes in the principles of context-driven testing. He has an MBA from National University of Singapore.

Nilanjan’s detailed profile is available at sg.linkedin.com/in/nilanjanswmanager/. His twitter handle is @nilanjanb and he blogs at swtestmanager.wordpress.com andwww.revelutions.com

Source: http://www.ministryoftesting.com/2014/08/childrens-minds-tell-us-software-testing-part-1-physical-worlds/