keynote: asking the right questions? what journalism can teach testers

KT1 Keynote

5/2/2013 8:30:00 AM

Asking the Right Questions? What

Journalism Can Teach Testers

Presented by:

Thomas McCoy

Australian Department of Families, Housing, Community Services and

Indigenous Affairs

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

Thomas McCoy

Australian software testing professional, teacher, and journalist Thomas McCoy has worked in the IT industry for more than two decades. Much of this time was spent as a software developer and IT manager with Australian government agencies. Wanting to make a greater contribution to software quality, Thomas re-oriented his career into the emerging discipline of software testing and has been an enthusiastic promoter of the profession ever since. At conferences in Australia, New Zealand, Canada, and the US, Thomas is a popular keynote speaker, who has received several best presentation awards. You can reach Thomas at [email protected].

STAR East 2013 2 May 2013

Thomas McCoy 1

What Journalism Can Teach TestersASKING THE RIGHT QUESTIONS

Thomas McCoyBJourn, BSc, DipEd, MInfTech

Australian Department of Families, Housing, Community Services and Indigenous Affairs

DISCLAIMER

STAR EAST 2013 2

The views expressed in this presentation are not necessarily those of the Department of Families, Housing, Community Services and Indigenous Affairs, the Australian Government, or even the presenter. ☺

Australia

STAR EAST 2013 3

I’m from Australia, which is around 9,500 miles away and now has a population of 23 million (just a few million more than the population of Florida).

Canberra

STAR EAST 2013 4

I’m based in Canberra, Australia’s capital city, which is in the southeast of Australia between Sydney and Melbourne. It was built in 1913 and designed by Chicago architect Walter Burley Griffin (who won an international design competition). Its population is around 370,000.


Thomas McCoy 2

FaHCSIA

STAR EAST 2013 5

FaHCSIA, the government department I work for, is responsible for social security in Australia, with payments and support for groups including pensioners, the unemployed, people with disabilities, and our Indigenous Australians. It manages about one fifth of the federal budget.We have testing staff working within both the agile (embedded in development teams for the smaller systems) and waterfall (working as a large testing group for our major application) contexts.

Department of Families, Housing, Community Services and Indigenous Affairs

STAR EAST 2013 6

Background

Started in journalism and media, then moved into IT, first into programming and finally into software testing.Have become increasingly aware of the importance of communication, both in terms of project team communication and computer / end user communication.This session has been inspired by the parallels I’ve seen between journalism and software testing.

WHAT DOES SOFTWARE TESTING HAVE TO DO WITH JOURNALISM?

STAR EAST 2013 7

JOURNALISM SKILLS

STAR EAST 2013 8

• A questioning, curious, and inquisitive mind

• Communication skills• Writing ability• News sense• Listening skills• General knowledge• Empathy, patience and

understanding• Sense of public

responsibility

• Language skills• Interviewing skills• Ability to meet deadlines• Knowledge of computers

and technology• Analysis skills• Resourcefulness• Persistence• Truthfulness• Accuracy• Integrity

Notice big overlap with testing skills.


Thomas McCoy 3

JOURNALISM PRINCIPLES

STAR EAST 2013 9

• Its first obligation is to the truth • Its practitioners must maintain independence from

those they cover• It must serve as an independent monitor of power• Its practitioners must be allowed to exercise their

personal conscience

These are principles in an “ideal world”, but again there is strong overlap with our testing career.

THE FOURTH ESTATE

STAR EAST 2013 10

Journalism has been called “the Fourth Estate”. Comes from British Parliament which has the monarch (1st estate), House of Lords, containing nobility and clergy (2nd estate), and House of Commons containing the other elected members (3rd estate).“In the reporters gallery yonder there sits a fourth estate more important by far than they all,” said 18th century British politician Edmund Burke.Fourth Estate rests on idea that it’s the media's function to act as a guardian of the public interest and as a watchdog on those in power.Are there parallels between this power structure and IT projects?

THE FOURTH ESTATE

STAR EAST 2013 11

Estate Journalism IT1st King Business areas?2nd House of

Lords (nobility)

Project Managers?

3rd House of Commons

Developers?

4th Journalists Testers?

TESTING: JOB OR VOCATION?

STAR EAST 2013 12

The enormous impact technology has on society, and our position in reviewing quality (too often at the end of the process), has led some to ask whether software testing is more of a vocation than a job. Vocations are sometimes differentiated from jobs because of their strong sense of social responsibility (e.g. teachers, nurses, clergy, journalists, etc.).


Thomas McCoy 4

TESTING: JOB OR VOCATION?• Sacrifice• Burnout• Long hours• Commitment• Thanklessness?• Social conscience• Self-development• Fighting “the good fight”

STAR EAST 2013 13

Do you identify with any of these aspects in your role?

STAR EAST 2013 14

Impact of a recent technology issue.Apart from that, we’ve had people killed, fried by radiation machines, not be able to use the brakes on their car, etc.

STAR EAST 2013 15

Then there are the less publicised daily annoyances caused by deficient technology (which can’t be good for people’s health).

SHARING NEWS OF OUR SUCCESS

STAR EAST 2013 16

Because of the nature of our role it’s sometimes challenging to publicise our success without upsetting people.We might want to do something like this …


Thomas McCoy 5

Testing News

STAR EAST 2013 17

… but this could cause offence, so we publicise in more benign ways, as in my newsletter story about our new test lab …

More palatable news

STAR EAST 2013 18

NEWS VALUES

STAR EAST 2013 19

Three useful news values are:• Impact: angle your message for maximum

impact in relation to your target audience• Proximity: if something is close to your

target audience or their aspirations it will have greater effect

• Timeliness: don’t delay messages unnecessarily

• For us, this is a common and difficult communication task

• Nobody likes to hear bad news

• Nobody ever says “thank you” for receiving bad news

DELIVERING BAD NEWS 1

STAR EAST 2013 20


Thomas McCoy 6

DELIVERING BAD NEWS 2 • Avoid it being a surprise (perhaps it

can first be passed through informal communication channels so people can adjust to the idea)

• Pick a time when the recipient is relaxed (sometimes Friday afternoons work well)

• If possible, propose some alternatives when deliveringthe news

STAR EAST 2013 21

The original Larson dog cartoon …

STAR EAST 2013 22

Sometimes “selective listening” is a problem whenreceiving bad news, as illustrated by this re-tooling of a classic Larson cartoon.

… can be adapted to our situation

STAR EAST 2013 23

REHEARSING BAD NEWS DELIVERY WITH AUSTRALIAN NATIVEANIMAL PERSONAS

STAR EAST 2013 24

Sometimes, when having to deliver bad news, it can help to think about the “persona” you’ll adopt when delivering the message.In Australia, we have native animals with unique characteristics that can be used to rehearse the bad news delivery (we even have hand puppets). ☺Sometimes we might have to go in aggressively, while at other times it might be with an attitude of resignation (because we choose not to fight this particular battle), or even with a humorous approach.


Thomas McCoy 7

TASMANIAN DEVIL

Aggressive and energetic

STAR EAST 2013 25

“Well what do you expect when we weren’t given the resources we needed!”

KANGAROO

Cheerful and easy going

STAR EAST 2013 26

“OK, we’ll see what can be done”

KOALA BEAR

Relaxed and nonchalant

STAR EAST 2013 27

“Yeah, right, whatever.”

KOOKABURRA

Always ready for a laugh

STAR EAST 2013 28

“Ha, ha, you’re joking, you know that was always going to be impossible.”


Thomas McCoy 8

A KEY PART OF JOURNALISM:THE 5WS & 1H

STAR EAST 2013 29 STAR EAST 2013 30

“I keep six honest serving-men(They taught me all I knew);Their names are What and Why and When and How and Where and Who”Rudyard Kipling from “The Elephant’s Child”

Think about the 5 Ws & 1H in anything you write.For example, in a defect report, as shown on the next slide …

5WS & 1H

STAR EAST 2013 31

• Who: Fred Nurk• What: Discovered system crashes if

negative quantities entered• When: Last night• Where: Test lab• Why: Inadequate input field

checking?• How: Running automated test scripts

Defect report example. Note that you won’t necessarily need all of the 5Ws & 1H in everything you write so consider which ones to leave out.

WRITING EFFECTIVELY

STAR EAST 2013 32


Thomas McCoy 9

STAR EAST 2013 33

“Always grab the reader by the throat in the first paragraph, sink your thumbs into his wind pipe in the second, and hold him against the wall until the last word.”

This might be a bit ambitious for the average test plan but the reason so few documents are read is because they’re so totally unengaging.

GEORGE ORWELL’S ADVICE

STAR EAST 2013 34

1. Never use a long word whena short one will do.

2. If it is possible to cut out aword, always cut it out.

3. Never use the passive voicewhen you can use the active.

4. Break any of these rules sooner than saying anything outright barbarous.

If you remember nothing else from this session, remember these rules. You will sometimes need to violate #3 and use passive voice to avoid blaming somebody.

Active Voice: Fred Nurk introduced a defect (blames Fred)Passive Voice: A defect was introduced (blames nobody)

THE INVERTED PYRAMID

STAR EAST 2013 35

Importance (signified by width)

Position

Top

Bottom

Content

People are so deluged with textual information that they may not read more than the first few lines.So use the inverted pyramid to present your information.

AN EFFECTIVE LAYOUT

STAR EAST 2013 36

• First paragraph: summarise the situation, covering the most important aspects of who, what, when, where, why and how (this may be the only paragraph the person reads)

• Next: provide more details• Finally: use a “call to action”

to have the issue resolved

When communicating about issues needing attention you may want to have a “call to action” at the end rather (than entirely following the inverted pyramid style of having less and less important information going down the page).


Thomas McCoy 10

EXAMPLE

STAR EAST 2013 37

On the registration date screen February 29 is being accepted for all years, not just leap years.[Summary, covers “where” and “what”]The problem may be due to the new external date routine, which was recently implemented on this screen. (The other screens still work fine.)[More info]This could damage customer relations and create adverse publicity for the organisation and needs urgent attention.[Call to action]

WRITING 1

STAR EAST 2013 38

• Need a strong headline that will push relevant buttons in people. The “headline” might be, for example, the:

summary of your defect report; or thesubject line of your e-mail message

• The headline may be all they read• Think about the “angle” of your

message (what are you trying to achieve)?

The “buttons” that will work best depend on our target audience and this is where research and prior planning are important.

WRITING 2

STAR EAST 2013 39

• Start with most important point, which should answer some of 5Ws & 1H

• Secondary information later• Use Plain English (avoid jargon)• Use mix of long and short sentences• Keep paragraphs to 3 sentences on average

(large slabs of writing intimidate readers)

WRITING 3

STAR EAST 2013 40

• Clichés should be avoided like the plague ☺

• Check spelling and grammar• A reading level of Grade 8 is

recommended (can use Flesh-Kincaid readability checker in Word)

Newspapers are typically written at Grade 8 level.


Thomas McCoy 11

USING THE READABILITY CHECKER

STAR EAST 2013 41

In Word 2010 do: File Tab, Options, Proofing…

… and make sure you tick the Show readability statistics checkbox. Also think about what you want to use for Writing Style (e.g. just Grammar or Grammar & Style) and check further options with the Settings …button.

STAR EAST 2013 42

Settings …These are some of the grammar checks the system can do under Grammar & Style but be aware that there are many it can’t do well (because writing is a human activity) so don’t rely on it too much and don’t follow it blindly (sometimes it gets confused).Passive sentences is worth using, though.

READABILITY STATISTICS

STAR EAST 2013 43

These are only a guide and have some quirky aspects.Flesh-Kincaid is based on average number of syllables per word and average number of words per sentence.This results in the kids’ classic “Green Eggs and Ham” achieving a grade level of minus 1.3, which would suggest it could be understood by kids before they start school and have learnt to read.We tried an interesting experiment …

STAR EAST 2013 44

Nuclear Physics: 15.5 grade level

We ran the Encyclopaedia Britannica article on Nuclear Physics through Flesh-Kincaid and it came back with a Grade level of 15.5(i.e. written at university level) …


Thomas McCoy 12

STAR EAST 2013 45

Nuclear Physics & Green Eggs: 2.4 grade level

… and then we pasted the entire text of Green Eggs and Ham at the end of the article, and when we ran readability again it brought the Grade level down to 2.4!Does this mean a kid halfway through Grade 2 could now understand nuclear physics?

After this somebody helpfully suggested that we should paste Green Eggs and Ham at the end of all our unreadable documents (using a 1-point font and white text for minimum visibility) to bring our readability scores down, but that’s another story …

OTHER WAYS OF PRESENTING INFORMATION

STAR EAST 2013 46

Wordle Word Cloud of Seven Testing Principles

STAR EAST 2013 47

A word cloud is an innovative way of illustrating key concepts in a document. The more often a term occurs the larger it is.This is from the 7 principles of testing in the ISTQB Foundation Syllabus and was generated by Wordle(http://www.wordle.net/).

0500

100015002000

Jan Feb Mar Apr

Overtime Hours

True RepresentationSTAR EAST 2013 48

Graphics can be a useful way to present information (and they have enormous credibility).Here is a graph of a project where the overtime hours are doubling every month, from 200 to 400 to 800 to 1,600.But would reporting this trend cause unnecessary alarm?


Thomas McCoy 13

1

10

100

1000

10000

Jan Feb Mar Apr

Overtime Hours

Logarithmic ScaleSTAR EAST 2013 49

Not if we switch to a logarithmic scale, which ensures that the world is still a happy place (and the slippery slope suddenly looks so much less slippery.)

Jan, 200 Feb, 400

Mar, 800Apr, 1600

Overtime Hours

Irrelevant Pie ChartSTAR EAST 2013 50

Or how about choosing an completely irrelevant chart type (Excel offers so many, and I’ve often seen them misused).

0%50%100%

JanFeb

Mar

Apr

Jan, 200Feb, 400Mar, 800

Apr, 1600

Overtime Hours

Meaningless 100% Stacked Horizontal Cone ChartSTAR EAST 2013 51

Or a completely meaningless chart (but you should probably remove the numbers from the cones).By the way, I’m not suggesting that we should deceive people. I’m just demonstrating how easily information can be manipulated even with so-called “scientific” approaches.

STAR EAST 2013 52

And many years ago (before everybody could do this on their iPhone) we created a rap-song video to expose the organisation to automation testing. It was done with a low-end camera and heaps of staff support.Some of the claims were (admittedly) exaggerated but this clip is still available on Sticky Minds (www.stickyminds.com/) just search for “Automation Rap”.Think about whether using video might be useful in your communication.


Thomas McCoy 14

VOICE 1

STAR EAST 2013 53

1. PitchWe need to raise our pitch to instil a sense of urgency and panic (or lower it to restore calm)

The way we use our voice is critical to getting our message across effectively.And vocal variation can help keep people awake.There are 4 important aspects:

VOICE 2

STAR EAST 2013 54

2. PaceQuick pace can also convey urgency. Or can be used to gloss over boring details. Use slow pace for important information

3. PauseAn appropriate pause (count silently to 5) can help focus attention before a key word

4. ProjectionNeed to project voice to ensure it is heard

VOICE 3

STAR EAST 2013 55

• Invest some time in voice work• Use a recorder to play yourself back (and don’t be

shocked the first time you hear yourself – that happens to all of us)

• Volunteer for presentations (even in non-work environments such as sporting clubs, kids school, etc.)

• Think about joining a group like Toastmasters (http://www.toastmasters.org/) which is dedicated to public speaking

ACQUIRING INFORMATION

STAR EAST 2013 56


Thomas McCoy 15

INVESTIGATIVE JOURNALISM

STAR EAST 2013 57

• Find several sources (don’t burn them)• Speak to as many relevant people as

possible• Verify information through reliable, factual

(preferably documented) evidence• Don’t take everything at face value

(because people may be running their own agendas)

SOURCES• Can increase our

understanding of “what’s really going on”

• Can serve as early warning of risks

• Respect “off the record” comments

STAR EAST 2013 58

You can independently verify off the record comments but you can’t credit them to the source, or even mention the source.

DEVELOPING SOURCES 1

Administrative

Business

Marketing

Technical

Finance

ManagementTentacled

Tester

STAR EAST 2013 59

To develop sources of the greatest value it can be useful to put your tentacles out as widely as possible (rather than remaining cloistered within IT).

DEVELOPING SOURCES 2

• Work social club• Sporting teams• Lunches• Attending presentations

and events• Joining the office choir

Departmental Choir

STAR EAST 2013 60

And there are many ways of reaching out beyond your immediate work environment. I can’t sing but still joined the office choir and was able to hear how the IT group appeared to people on the outside.


Thomas McCoy 16

CRITERIA FOR EFFECTIVE SOURCES

• Past suitability• Reliability• Trustworthiness

• Authoritativeness• Articulateness• Discretion

STAR EAST 2013 61

INVESTIGATIVE TECHNIQUES

STAR EAST 2013 62

• Infiltration• Impersonation• Secret filming and/or recording

Sometimes it may be necessary to “infiltrate” meetings you haven’t been included in.Or to impersonate somebody (e.g. via a phone call) to obtain information. (You should not impersonate an existing person, but you might impersonate a generic role e.g. “I’m ringing from tech support”).Be very careful with secret filming or recording (more on this later).

Sheila

Chaz

Bluey

Kylie

Ernie

Tezz

Molly

Nicole

Sidney

Snowy Talisha

Bazza

Brian

Hugo

Kezza

Magda

MAPPING THE ORGANISATION 1

STAR EAST 2013 63

In understanding the operation of an organisation it can be useful to map the informal relationships onto the formal org chart as shown in the next slide.

Sheila

Chaz

Bluey

Kylie

Ernie

Tezz

Molly

Nicole

Sidney

Snowy Talisha

Bazza

Brian

Hugo

Kezza

Magda

is cousin of

disagrees with

got promoted over

plays rugby with

teaches trombone to son of

MAPPING THE ORGANISATION 2

RELATIONSHIPSSTAR EAST 2013 64Much more informative.


Thomas McCoy 17

OVERT RECORDING: USABILITY TESTING

STAR EAST 2013 65

ALWAYS DO USABILITY TESTING

STAR EAST 2013 66

• It is cheap and can be run on a laptop• Testing even one user is better than no users• Have them perform some tasks and ask them

to “think out loud” as they complete them• Software like TechSmith’s Morae product will

capture the screen, the user and the audio in real time

• The “video evidence” can be compelling when shown to developers

STAR EAST 2013 67

We watched a usability clip where the “save” button had been removed from a new version of the system (which now did automatic saves).This traumatised a user of the old system who could no longer find a “save” button and did now know whether her work would be lost when she exited. She became “stuck” and would have rung the help desk for advice.

Fake SAVE button

STAR EAST 2013 68

My recommended, innovative, low-cost solution, of adding a fake “save” button to the application so users could still click it and reduce their anxiety, was rejected.

SAVE


Thomas McCoy 18

COVERT RECORDING

STAR EAST 2013 69

COVERT RECORDING 1

STAR EAST 2013 70

• With the capability of today’s mobile devicesit is trivial to record audio and/or video covertly(you no longer need to be a secret agent)

• Many technical meetings generate so much information so quickly that recording is the only way of obtaining accurate notes

• Ideally you would ask participants if you can record (however, this makes some people so uncomfortable that sometimes not asking may be more humane) but check with local laws and company policies

COVERT RECORDING 2

STAR EAST 2013 71

• In the ACT (where I come from) we have a funny law where you’re allowed to covertly record any conversation that you were part of as long as at least one person agrees (and that can be you) ☺

• Note that covert recordings should never be used against somebody (once relevant notes have been made they should be destroyed)

• Here is an example where somebody inappropriately used covert recording to request a testing extension while the boss was going about his business (with the aim of later playing it back to the boss as proof of the extension): www.thomas.mccoy.name/DontDoThis.mp3

OUR TESTING RESPONSIBILITY(AND BAD ERROR MESSAGES)

STAR EAST 2013 72


Thomas McCoy 19

Client Registration

Family Name:

Date of Birth:

Given Name:

Date of Death:

Gender:

System Date/Time: 22/4/2013 2:25PM

Nurk

Fred

M

01/01/2015

STAR EAST 2013 73

In testing a client registration application I entered a future Date of Birth to check for the error condition (note that Date of Death was left blank).As expected, an error message was displayed and I could have marked the test as “passed”.

STAR EAST 2013 74

However, I was horrified by the meaningless, technobabble “error message” that was clearly written by tech-heads with zero communication skills. As testers, when we see something like this, I feel we owe it to the end user to report it, even if it isn’t on the official test plan.

[1] Wrong field values or value types detected in Date of Birth.Please re-enter your field values. If you need additional assistance, please refer to the documentation.(SBL-UIF-00299)[2] The value '01/01/2015' for field 'Date of Birth' is required to be '<= Today() AND <= [Date of Death]'.Please enter a value that is '[Birth Date] <= 22/04/2013 AND [Birth Date] <= '.(SBL-DAT-00521)

STAR EAST 2013 75

Looking at this message reveals many common problems:1. Should not use terms like “wrong” and “detected”, they are negative2. Should not use “field values or value types”, this is technobabble3. The cryptic error code is meaningless4. Why are there three versions of the error message?


STAR EAST 2013 76

5. Why is a function call i.e. Today() in an error message?6. Why are mathematical comparison operators and Boolean logic used?7. Why is Date of Birth written in 3 different ways: Date of Birth, ‘Date of

Birth’ and [Birth Date]?8. Why are square brackets used around variable names?9. Why do dates sometimes have single quotes and sometimes not?


Thomas McCoy 20


STAR EAST 2013 77

10. Why does the last comparison end with <= '. (this is because Date of Death was blank, so why are we comparing with blank)?

11. Why is there a second meaningless error code?The entire error message, of more than 50 words and meaningless codes and symbols, could be replaced with:

Date of Birth may not be in the future. Please re-enter.

FIREFOX ERROR MESSAGE

STAR EAST 2013 78

Firefox has moved towards some soft and cuddly error messages where the computer takes the blame and becomes humanised.This does not add value and quickly becomes tedious.Better to just write meaningful and useful messages in plain English.

ERROR MESSAGES 1• Good error messages should answer some

key 5W&1H questions:1. Describe Problem (clearly and succinctly)

WHAT2. Cause (in as much detail as necessary)

WHY3. Solution (steps to fix problem)

HOW

STAR EAST 2013 79

ERROR MESSAGES 2Should be:

understandablerelevantactionableuser-centredbriefclearspecificcourteousrare

STAR EAST 2013 80


Thomas McCoy 21

PROMPTING ACTION

STAR EAST 2013 81

• If you are experiencing difficulty in conveying the importance of having an issue addressed, mocking up adverse media coverage can help

• On the next slide I have created a fictitious story about the error message and can then innocently ask “but what would happen if something like this came out in the paper”?

• Having it looking real, rather than being hypothetical, may be more persuasive

• “Stock prices plummet” might be another example

Cryptic error messagecauses user chaos

Help Desk phones were inundated as users struggled with a cryptic error message that would require a computer science degree to decipher.

The message, which included comparison operators, Boolean logic, function calls, field names and

variables, was described by one user as “complete gobbledegook”.

The error condition prompting the message was triggered when a user typed a date of birth for a client that was in the future.

While this is clearly invalid, the message was of little use in

THE TESTING TIMESwww.testingtimes.com.au May 2013

The cryptic error message

CHECKING ACCURACY (OURS)

STAR EAST 2013 83

MISTAKES

STAR EAST 2013 84

• Since we critique the work of others it is reasonable that we are held to high standards

• In the same was as newspapers sometimes issue corrections, we should promptly admit mistakes

• An excellent way of checking our written material is to hear it read out loud

• The Office 2010 products (Word, Outlook, Excel) have an inbuilt reader that can read any text highlighted by the user


Thomas McCoy 22

THE OFFICE 2010 INBUILT READER

STAR EAST 2013 85

• To use the reader (called “Speak”) you need to add the Speak icon to your Quick Access Toolbar

• You can then select any text and click the icon to have it read out

• You can use it to read back e-mails before you press “Send” or to read out long, boring documents when your eyes are fading

• A video to show you how to do this is available at:www.thomas.mccoy.name/SpeakFeature.htm

QUESTIONING

STAR EAST 2013 86

QUESTIONING

STAR EAST 2013 87

• At the heart of journalism• Requires substantial research and

preparation• Journalist often knows answer in advance• Questions can be used to convey an

implicit message in a less threatening way than statements

EFFECTIVE QUESTIONING 1

STAR EAST 2013 88

• Do research, plan questions and strategy and write them down

• Be clear on the objectiveof every question

• Think about likely responsesand your follow-up question

• Make person comfortable with informal conversation to start with


Thomas McCoy 23

EFFECTIVE QUESTIONING 2

STAR EAST 2013 89

• Ask mixture of open and closed questions

• Do not make it feel like an interrogation

• Adopt a non-threatening and inquisitive manner

• Smile and nod, endure silences,don’t interrupt, be polite,stay neutral

EFFECTIVE QUESTIONING

STAR EAST 2013 90

• Record answers (shows interest)• Repeat answer back for confirmation• Take special note of anything that comes “after the

interview”• Use leading questions, if necessary e.g. “When this

error message frustrates the user, who are they supposed to call”

CHOOSING THE MEDIUM

STAR EAST 2013 91

• Face to face• E-mail• Phone call• Post-it note• SMS • Social media

Your medium will depend on the message. For an informal message a Post-it note may be fine. For something more serious you may want to do face to face with e-mail follow-up.

CHOOSING THE CONTEXT

STAR EAST 2013 92

• One-on-one in private• One-on-one in open plan• Daily stand up• Team meeting• Project meeting• Executive meeting

Choose your context carefully and keep participants to an absolute minimum.


Thomas McCoy 24

WHAT KIND OF QUESTIONS?

STAR EAST 2013 93

• This depends on who you are talking to, what you are hoping to achieve and what motivates them, for example:

Project Manager – not meeting deadlinesBusiness Analyst - incomplete or defective requirementsDeveloper - coding errors

EMBEDDED MESSAGE EXAMPLES

STAR EAST 2013 94

Project Manager• “How much overtime are we

able to accommodate under the current budget?”

• Embedded message: There is no way we can finish testing on time with the existing resources.


STAR EAST 2013 95

Business Analyst• “I’m just wondering what

happens on the registration screen if the user has registered more than once?”

• Embedded message: These requirements have holes.


STAR EAST 2013 96

Developer• “What happens if the

user puts in the year with just the last two digits?”

• Embedded message: There is a bug in the date routine.


Thomas McCoy 25

DEVELOPERS AND TESTERSHOW CLOSE IS TOO CLOSE?

STAR EAST 2013 97

EMBEDDED JOURNALISM …

… OR IN-BEDDED?

STAR EAST 2013 98

STAR EAST 2013 99

“I found myself falling in love with my subject.I fell in love with ‘my’ Marines … the point wasn’t that I wasn’t reporting the truth, the point was that I was reporting the marine grunt truth … which had also become my truth”.US journalist Gordon Dillow quoted in Journalism Studies journal.

AGILE TEAMS: TOO CLOSE?

STAR EAST 2013 100

• Gordon Dillow was one of around 800 journalists embedded with soldiers in the Iraq war

• They depended entirely on the soldiers for food, shelter and safety

• It has been suggested, not just by Gordon, that their reporting lost some of its independence

• Is there a risk that our testing may lose independence (despite all the benefits that Agile allegedly offers)?


Thomas McCoy 26

INDEPENDENT TEST TEAM

There are several ways in which a test team could be kept “independent”:• Physical?• Administrative?• Both?

STAR EAST 2013 101

• With our independent testing team we have found that it:• frees members from

relationship pressures• lessens “informal” fixes• builds a collegiate atmosphere• allows us to unashamedly

celebrate successes (which may be development failures)

Give me a hug,I found a bug!


STAR EAST 2013 102

It has its own space, which allows staff contributions and celebrations to be put on public display, creating positive energy.


STAR EAST 2013 103 STAR EAST 2013 104

And it allows us to shamelessly promote testing principles (and call out the numbers when the inevitable defect misses occur).


Thomas McCoy 27

PERSONAL COURAGE

STAR EAST 2013 105

Like journalism, testing often requires personal courage, as shown in the following slides…

PROJECT MANAGER AFTER READING YOUR E-MAIL ON HIS IPHONE

STAR EAST 2013 106

PROJECT MANAGER DISCUSSINGIT WITH YOU

STAR EAST 2013 107

HOW YOU FEEL AFTER THE “DISCUSSION”

STAR EAST 2013 108


Thomas McCoy 28

PROMOTING OURSELVES

STAR EAST 2013 109

• We created a 2-minute videoclip entitled Life Without Testing

• The clip was effective in focusing attention on the serious consequences of undetected bugs in the Medicare payment system

• You can watch it at:https://vimeo.com/65502104

FINAL THOUGHTS

STAR EAST 2013 110

• As with journalism, we have to keepre-inventing ourselves

• Our successes need to be communicated appropriately

• We have to expand our network of allies• And we must keep the end users at the

centre of what we do and advocate for them whenever possible

KEY POINTS

STAR EAST 2013 111

• Journalism and testing have similar skills and principles, particularly as they relate to an obligation to the truth and a sense of public responsibility.

• Testing is considered by some to be a vocation, rather than a job.

• Publicising our successes is a challenge.• Think about news values such as impact, proximity

and timeliness in your communication.

KEY POINTS

STAR EAST 2013 112

• Take the angle into account, use the 5Ws & 1H where possible, and remember the inverted pyramid.

• Take care with delivering bad news. Sometimes people only hear what they want to hear.

• Use personas to rehearse bad news delivery (if that works for you).

• Work on writing and speaking skills so your communication cuts through the noise.


Thomas McCoy 29

KEY POINTS

STAR EAST 2013 113

• Pay special attention to the “headline” and first paragraph in your writing; people may not read beyond that.

• Use Flesh-Kincaid to check readability (but avoid using Green Eggs and Ham to fake it).

• Use the Speak feature in Word and Outlook to help you catch writing errors before you send them out (it also lets you rest your eyes when reading long documents).

• Remember George Orwell's writing advice.

KEY POINTS

STAR EAST 2013 114

• Consider using innovative communication methods, such as word clouds, video clips, and even mocking up a negative news story to illustrate the potential public impact of a defect.

• Remember that graphs can lie.• Practice recording yourself to improve voice

delivery.• Develop sources throughout the organisation and

consider mapping its relationships onto the formal structure chart.

KEY POINTS

STAR EAST 2013 115

• Capture information in “overload” situations by recording audio and/or video.

• Do a usability test whenever possible.• Review error messages for clarity and usefulness

as they appear during your testing.• Always correct mistakes and review your written

work.• Use questioning to gather information, but also to

convey implicit messages.

KEY POINTS

STAR EAST 2013 116

• When planning questioning, think about the medium, context, drivers (for the person you are questioning) and what you hope to achieve.

• Are embedded testers (e.g. agile) sometimes too close to developers?

• Personal courage is an essential aspect of journalism and testing. Be brave.

• Don’t lose your sense of humour (especially not if you can use it in a video to get your message through).


Thomas McCoy 30

THANK YOU!

STAR EAST 2013 117

[email protected]

keynote: asking the right questions? what journalism can teach testers

Technology