Testability is all about building quality-in. It’s about identifying known issues before they become a problem while coding. Pairing testers into this process can supercharge the testability feedback loop. It can allow you to pick up known and unknown issues.
But pairing devs and testers together needs courage. Courage so that both disciplines can take interpersonal risks and share hard things such as what they don’t know, don’t understand or mistakes they’ve made. This will need both groups to listen, understand and ask questions to help each other through the process. Both groups will need to show curiosity, humility and empathy for one another. You will not only feel uncomfortable during the process but it will take time too. The temptation to go back to inspecting for quality – dev and test handing work off to each other – will be hard to resist.
Pairing for testability is not just pair programming but working together to understand what the behaviour of the code being written should and shouldn’t do.
Devs and testers should work together to leverage the skills that each have, not get hung up about the skills they lack. If your pair is more exploratory focused identify ways that allow you to make the best use of those skills. If they are more technically inclined then focus there.
Remember the key is to build quality-in not inspect for quality. So what can you do now that helps your team move in that direction?
Don’t report the bugs your test automation catches. Report the reduction in uncertainty that the system works.
When you report the bugs you send the signal that test automation is there to catch bugs. But that’s not what it’s for. Test automation is there to tell you if your system is still behaving as you intended it to.
What are automated tests for?
Each automated test should be some isolated aspect of the behaviour of the system. Collectively these tests tell you that when you make a change to the system it still behaves as you want it to. What automated tests do is reduce your uncertainty that the system still behaves as you expect it to.
Framing test automation as reducing uncertainty
Framing test automation as reducing uncertainty help emphasize that there are always things we don’t know. Whereas if you frame it as increased certainty it can give the impression that we know more than we do.
Framing testing as increasing certainty
Framing testing as reducing uncertainty
What happens when a test passes or fails
When an automated test passes it’s sending a signal that this specific behaviour still exists. Therefore reducing some of your uncertainty that whatever changes you made have not affected this specific behaviour.
When a test fails it signals that this expected behaviour didn’t occur, but that’s it. What it doesn’t tell you is if it is a bug or if it was due to the change to the system. Someone still needs to investigate the failure to tell you that.
So what we should report is to what extent our uncertainty has been reduced by these tests. But how do we do that?
How to frame test automation as reducing uncertainty
Well a good place to start is to help people understand what behaviour is covered by the tests. For instance, you could categorise the behaviour of your system into 3 buckets such as primary, secondary and tertiary.
Primary could be things that are core to your product’s existence. For example for a streaming service, this could be video playback, playback controls and sign up etc. Tests in this bucket must pass before a release can be made.
Secondary could be behaviour that supports the primary behaviours but if they didn’t exist would be annoying at most but still allows the core features to function. For example, searching for new content or advanced playback controls (think variable playback speeds). Tests in this bucket can fail but they should not render the application unusable. Issues discovered here can be fixed with a patch release.
Tertiary behaviours could be experiments, new features that haven’t yet been proven out or other less frequently used features that are not considered core. Tests in this bucket can also fail and don’t have to be fixed with patch releases.
But be careful of accessibility behaviours falling into Secondary and Tertiary buckets. They might not be your biggest users but those features are critical for others to be able to use your systems.
Defining these categories is a team exercise with all the main stakeholders as it is key that they have a joint understanding of what the categories mean and what behaviours can fall into them.
Then when you report that your primary and secondary tests are passing you signal that the core and supporting features are behaving as expected. This reduces the team’s uncertainty that the system behaves as we expect. You can then decide what you want to do next.
Exploratory testing is about testing in an unpredictable context and therefore detecting unpredictable failures in our software. Automated testing is about testing in a predictable context and therefore detecting predictable failures. The mistake we make with automation is we try to apply it to the wrong context. You can’t use testing methods developed for predictable context in an unpredictable environment.
While there is nothing physically stopping you neither practice is particularly efficient if used in the wrong context. Exploratory testing in a predictable environment would just confirm what you already knew only slower and less consistent when repeating the testing . While automated testing in an unpredictable environment would lead to false negatives.
It’s also not a one size fits all solution either as we work in both contexts. Predictable when initially developing the software and unpredictable once running in the live environment.
The only way you can replace exploratory testing with automation is to make the test environment predictable. But that would then mean you are trying to detect predictable issues. This then negates the outcome you were looking for which is trying to detect unpredictable or complex failures.
Testing in unpredictable contexts
The best way to detect unpredictable failures is to use methodologies that can operate in an unpredictable environment.
One of the best known methods is exploratory testing (sometime called manual testing) but there are other technique too. Such as monitoring of the live environment. Which is good for issues we can predict in an unpredictable environment. Observability using logs, graphs and other telemetry to see how the system is behaving in the live environment. This is helpful for issues we can’t predict and need to debug in the live environment. Phased rollout of features using techniques such as feature toggles, blue/green deployments, canary releasing etc. Useful for limiting the impact of unintended issues in a unpredictable environment. Basically anything that allows you to slowly enable a feature for subsets of users.
Using monitoring and observability in conjunction with phased rollouts can greatly improve your ability to understand and limit how new code behaves in unpredictable environments.
Testing in predictable contexts
This is not to say automated testing is invaluable as it can help detect smaller predictable issues. Which if left unchecked could develop into larger unknown failures that only occur with the right mix of other smaller issues. Some issues maybe within our control (software we develop) and some outside of our control (other people’s software). For software in our control (a predictable environment) automated testing is almost a prefect match. For software outside of our control (an unpredictable environment) contract testing, exploratory testing, monitoring and observability and phased roll outs of software is preferable.
Control and isolation
Next time you’re looking at testing techniques think about how much control (and therefore isolation) you have over your test environment. The greater the level of control then the more automation you should consider, but the less control you have then the more you should consider exploratory testing coupled with monitoring, observability and phased rollouts.
Testing techniques
The following diagram will help you see how different testing techniques stack up against each other. This is by no means an exhaustive list and is only comparing them on a speed of feedback, value of feedback and testing environment bases. So the next time you get into a discussion about testing you could use these characteristics as a good way to frame that discussion.
Testing techniques plotted on a speed, value and environment axis
Are there testing techniques that should be plotted on the chart?
Do you agree with the axis? Is there another more important characteristics of testing that should be captured?
📻 How Can You Stop Comparing Yourself With Other People? If you manage people then this podcast is worth a listen. Having a better understanding of why we compare ourselves to others (social creatures living in hierarchical structures) and what issues it can cause (de-motivation, decreased self-esteem and confidence) can help you from stop doing it but also help you help your reports from falling into the trap. They also cover some biases that can lead to it such as casual inference and narrative fallacy.
💪 How Resilience Works The post calls out three traits for resilience: 1. A grasp of reality 2. Life has a purpose (for you) and 3. An ability to improvise. I’ve not see resilience called out like this before but there are some good anecdotal stories in there and in broad strokes I agree with it. But like a lot of things with the mind its easier said than done. Especially when you’re in the thick of things going wrong. (Book to add to the reading list: Mans search for meaning)
🍏 How Apple controlsthe App Store and therefore the end users How Ben explains the App Store Integration in stages is really interesting and key to understanding how Apple has so much control over developers and users. This is a long read but worth it to understand Apple’s almost unbelievable control of developers and users. If you want to access Apple’s users then you almost have no choice but to do as they say otherwise they can revoke your certificates and cut you off in a instant. The thing is this integration is so complicated most people are either not going to understand it or take the time to figure it out. This is very different to how Microsoft controlled Windows.
17th August
🗺 Things Jobs said I’m no Steve Jobs fan (in the literal sense of the word) but no one can deny he helped create some incredible products. Every so often I read these quotes from him and depending on what’s going on in my work life they take on a different meaning. But one that always stays with me is this one: “You can’t connect the dots looking forward; you can only connect them looking backwards. So you have to trust that the dots will somehow connect in your future. You have to trust in something–your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life.”
🏌️♀️The Beginner’s Guide to Deliberate Practice What is it? Deliberate practice requires focused attention and is conducted with the specific goal of improving performance, while regular practice might include mindless repetitions. But its not that simple deliberate practice requires that you break down the task into small sub-sections and practice each one till you get better. This is easy if the domain you’re trying to learn is well known, but if it doesn’t have any existing training that you can make use of or you don’t have access to trainers that can help (e.g. mentors or coaches) than you might struggle. I believe this is why it is always a good idea to learn from multiple sources when skilling up in something new that pushes you out of your comfort zone in different ways. If you’re learning something just from one source then keep in mind that it might be one sided…
10th August
🎼 What software teams can learn from music masterclasses Via twitter from @katrinaclokie. Feedback is by far one of the best ways you can learn and Helen makes a great point in that software teams can learn a lot from music masterclasses and studio classes too. Both are great ways to get feedback from more established artists, peers and teachers. But also from peers in different disciplines who can give a viewpoint that your own peer group might not be able to. Another key point Helen makes is that giving and receiving feedback is a skill and as such needs to be practiced to really help people. I don’t think we do this enough in software teams and when we do it’s not always the best. There is a lot we can learn from artistic masterclasses as an industry which I guess reflects the maturity of their professions and the relative youth of ours.
🚽 Code Coverage Best Practices This post from Google testing on the toilet series makes some great points on how code (or test) coverage can be a useful metric for teams to use. The biggest one being about how it highlights code that isn’t covered by tests. This is the perfect opportunity for teams to discuss if it should or shouldn’t. Also the advice on using it to inform on conversations topic for code reviews per commit it also a really good idea. But as the articles points out going straight in with “We should use code coverage!” is probably not going to get you very far. Most engineering teams have been burnt pretty badly by it in the past with developers just trying to hit numbers or it being used to measure the effectiveness of them. Both of which lead to the wrong incentives of number gaming rather than productive conversation starters on what are good and bad tests for your context.
🐦🧵 Everything you needed to know about 2+2=5 Kareem makes a great point that it’s all about context. If you’re thinking just about raw numbers then 2+2 =4 but if the context was say a male cat and a female cat give it some time then could quite easily be 1+1=8. Numbers are an abstraction of the underlaying reality therefore context matters when you’re looking at numbers. One to ponder the next time you’re looking at statistics 🤔
📹 What is white privilege? Via BBC Bitesize from psychologist John Amaechi. This short 3 minute video does a really good job of explain what privilege is and what white privilege in particular means.It’s not that white people have it easy or struggle any less than people of other races. It’s that their struggles are not going to be about their race where as race can be an additional limiting factor from people in the BAME community. In short white privilege means your skin colour will not be used against you.
At the intersection of software, technology and people
Things I’ve been reading this week that I’ve found interesting or intriguing.
At the intersection of software, technology and people
What is this?
Things I’ve been reading this week that I’ve found interesting or intriguing. Sharing because I thought you might like them too. Most of the links will revolve at the intersection between software, technology and people – with the occasional testing slant. I aim to update them weekly, with some commentary on my thoughts and findings. Feedback always welcome 😁
📬 Latest post what do testers do next if the risks mitigated by manual testing can be reduced through other means? Is it about moving more towards creating a quality culture and if so what do you need to know?
📝 My notes on Kind and Wicked learning environments and how they affect your ability to pick up skill.
Some more notes on a really interesting idea from Eugen Wei on Invisible asymptote. See July 10th below for more or head over to my notes on the article that pull out some of the bits I found interesting.
31st July
❓ Four-Level Training Evaluation Model some useful ideas on what to look for when trying to get feedback on your training or other presentations. Another question that comes to mind: Is the training for the learners or for you to accomplish/be recognised for something… 🤔
💭 10 signs you’re an over thinker While thinking is obviously a good thing overthinking isn’t. But how do you know when you’re doing the good type of thinking? Simple rule: overthinking is focusing on the problems (by either ruminating about the past or worrying about the future). Good thinking is problem solving by focusing on the solutions and self-reflective thinking is looking at situations from a different perspective and finding new insights.
👷♀️ 3 things that motivates us to work From Dan Pink’s RSA lecture based on his book Drive. The three things being autonomy, mastery and purpose. Autonomy is about being self directed over what and how you do something. Mastery is having the ability to get better at something that challenges us and making a contribution. Purpose is the reason for being or why are we doing the thing we do. The interesting thing is this is about individual motivation to work. Does it still apply when working in teams as we do in software?
27th July
A model of what could happen if you dropped the ‘In Test’ column…
👷♀️ From ‘In Testing’ to ‘In Progress’ columns on team boards: This has a very narrow focus on just dev and test relationship. This model helps illustrate how improving their relationship and getting them to actively collaborate to improve confidence that the code changes work as intended is going to start having an affect on work in progress (WIP). Which as @johncutlefish shows high WIP can lead to a whole host of other problems. The grey lines are what it was previously with the ‘In Testing’ column broken out into it’s own section.
👯♂️ Don’t Mock Types You Don’t Own This happens more often then you realise and leads to lots of other problems the main one being you now have to maintain a mock of a service you don’t own or fully understand how it works. Therefore you’re testing against your assumptions of that service you’re mocking. This could lead to a false sense of confidence that everything will work when you go to production. Ideally you want to be using a stub with little to no logic e.g. little to no assumptions and any made are obvious to other developers. Contract and consumer driven contract testing particularly can help here. The other issue is people use the word mock to mean a whole host of other types of Test doubles (fakes, stubs, spies etc) which leads to more confusion so check what they mean when they say mock before assuming you’re talking about the same thing.
🎓 Accountability vs Responsibility This has been really useful when thinking about who is accountable within teams for tasks and who is responsible. I found others (and myself included!) mix these up. Accountability can not be shared and means you are answerable for your actions where as responsibility can be shared and you must respond when someone questions your actions. Having these distinctions can be really helpful in making sure people understand what they are accountable and responsible for. The comments are worth a read too…
20th July
😱 Programming is not acraft from Dan North in 2011 and I have to say I agree with his take even from back then this still stands. I think this really sums it up “Non-programmers don’t care about the aesthetics of software in the same way non-plumbers don’t care about the aesthetics of plumbing – they just want their information in the right place or their hot water to work”. By putting programming at the centre (by treating it as a traditional craft) and not the value you are delivering you risk building what you want and not what users want/need/care/value. Thats not to say the that the code can be shoddy far from it, but just like the plumbing it needs to work but does it need to be gold plated with silver fixings?
📻 How to make your own luck (podcast) the frame with which you look at world (people, events, things that happen etc) are going to have a big impact on the opportunities that you’re going to find. So what frame are you using when making decisions? The world is a wicked learning environment (slow feedback hard to tell which variable caused the outcome) while poker can be kind learning environment (fast environment, low number of variables, easier to identify mistakes and learning from them) therefore helps you to understand your decision making easier and then possibly translate over to the real world.
Kind give lots of feedback as you progress which aid deliberate learning
The rule of the system don’t change either so what it is today is the same tomorrow
Golf, chess and poker are such environments
Wicked learning environments
Mixed levels of feedback as you progress
Rules of the system keep changing
I think software engineering maybe a wicked environment
13th July
🤩 Invisible asymptote (AKA The Invisible glass ceiling of testing) Excellent (and long) read from Eugen Wei and a must read for anyone working in product and software development in general. Brilliantly articulates that all products have an invisible glass ceiling and that by recognising your total addressable market it can help you understand when you’re going to hit it and actually do something about it.
Why should testers care?
This is a great way to understand how your product owners might be thinking (or should be if they are not). In terms of product quality this could be one of the lenses from which you should look at your products to understand what is valuable to product owners. It’s also a great way to start understanding what value your product is potentially bringing to your users and what cohort that it is and isn’t addressing. My notes on the article pull out some of the bits I found interesting.
In terms of analysis this hits two of the three domains that testers should have a grasp of: business and users. From that angle we can help the third domain (teams) understand how this affects them.
Remember testing doesn’t always look like testing
🔈 How do you handle criticism Getting feedback is by far the best way to get better but not all feedback is equal. You need to filter out the valuable parts from the things that sting the ego. One way to get better at receiving feedback is to rate yourself on how you respond to it. 5 being excellent and 1 being poor. Did you respond positively and thank them (4 out of 5) or did you try and talk them out of their opinion (2 out of 5). This will help you get better at hearing feedback but also more likely to do something about it.
👩💻 Develop your culture like its software Interesting post from 2017 from the ex-engineering manager of The New York times. They used a google doc to make it collaborative and to start iterating on it. Culture is something that either just happens and evolves in a direction out of your control or you try and be deliberate about it. My preference is towards deliberate because then if it starts heading in a direction you don’t want you’re in a position to do something about it. Otherwise you find out when something hits the headlines. At which point its too late to do something…
🏚 Extreme testingCool video of what IBM do to make sure their mainframes can handle earthquakes. Makes you wonder what type of testing AWS/Asura/GCC do for all their server farms
…when you are doing something in a recurring way to diminish risk or doing it in the same way as you have done it before, it is clear why professionalism is not enough. After all, what is required in our field, more than anything else, is continuous transgression. Professionalism does not allow for that because transgression has to encompass the possibility of failure and if you are professional your instinct is not to fail, it is to repeat success. So professionalism as a lifetime aspiration is a limited goal.
🥾 New employee bootcamp really interesting approach to getting people (product owners in this case) up to speed quickly and productive within their work. I really like the concept of “put your own gas mask on first before helping others” in terms of helping them figure out their own career paths. What would this look like for on boarding new testers in a team?
🧫 What is culture? I was doing some research on this and it turns out (unsurprisingly) that its not that easy of a question to answer but the Centre for Applied Linguistics at the University of Warwick (UK) has some really good resources. In particularly this doc which tries to answer that very question in a way that is approachable and can actually help you understand what it is. They break it down into 12 key characteristics but I think this explanation from Spencer-Oatey (2008) does a pretty good job:
“Culture is a fuzzy set of basic assumptions and values, orientations to life, beliefs, policies, procedures and behavioural conventions that are shared by a group of people, and that influence (but do not determine) each member’s behaviour and his/her interpretations of the ‘meaning’ of other people’s behaviour“
🤑 What is value? Interesting way of thinking about what value means. In this model there are two focusing areas: revenue and costs. How does something sustain revenue, increase revenue, avoid cost and/or reduce cost. By applying a monetary number to these you can then discuss them in a way that everyone understands and can hopefully agree on. The other reason for relating this back to a number is having a discussion on what assumptions people are making about those numbers.
Thanks to Duncan Nisbet for his intriguing blog series on cost of delay Vs cost of poor quality which linked me to the above post. In Duncans post he does a really good job of showing why trying to answer that question is really difficult and is setting up a framework in trying to do just that. I’m looking forward to seeing how this works out!
I’ve been thinking a lot recently about what the future of the testers role could look like. Especially in teams that not only fully embrace CI/CD or DevOps but actually get some way towards implementing the ideas behind the approaches.
I’ve broken up my thinking into two posts with the first about whats could happen and this follow up on what testers could do next…
Raising quality awareness in teams
Raising quality in teams isn’t about banging the drum of “We need to make this a quality product” or showing how the product failed some quality criteria e.g. raising a defect. It’s about helping the team understand what quality is and what that means for their system. To be able to do this testers need to be able to articulate what quality means and then apply this to their teams context.
For each tester this context will be unique to their environment. But in a large part, this will be based on how their team works, what their organisation expect that team to produce and who their end users are. Simply put, testers will sit at this intersection of teams, business and users.
Teams
This view point is all about understanding how the team works through their combination of tools, processes, technology, the people involved and what the resulting output is. How does the team do what it does but also why does it do it the way it does.
Business domain
This viewpoint is about understanding the organisation that the team works in. Why does the business exist? What is it trying to accomplish? Yes you can argue that pretty much all businesses are trying to make a profit but how exactly is it trying to do that? Is it by selling advertising, software as a service, access to some physical world good? What is your companies unique selling point? How does it compete against other companies? What external factors affects its ability to achieve its mission? How does the organisation expect the team to contribute to its mission?
Users
This viewpoint is probably the one most familiar to testers as this has been a view they’ve almost always considered. Who are the systems users and what do they expect from it? But they should take this view and expended it further. Why do the users take their time to use your product over others? What do they find valuable about it? Why do some users stay but others leave? Are these the intended people that your organisation expected to use it? Are the users the actual people who pay for the product or does someone else? Who is that someone else and why do they pay for it? Testers should take their time to build, broaden and deepen their understanding of the users, intended users and future users.
At the intersection of teams, business and users
Due to the subjective nature of quality, tester need to understand the reasons behind others views of what quality means to them. These three core areas give the testers the foundational knowledge to do just that.
Now armed with this knowledge and the why of their stakeholders quality measures they can begin to translate this into something their teams can understand and incorporate into their daily work. It’s within this translation/incorporate that you can begin to create a quality culture that isn’t about demanding quality, but creating the understanding of what it means within your teams context.
I’ve been thinking a lot recently about what the future of the testers role could look like. Especially in teams that not only fully embrace CI/CD or DevOps but actually get some way towards implementing the ideas behind the approaches.
I’ve broken up my thinking into two posts with the first about whats could happen and the follow up on what they could do next. So what could happen to testers?
For a lot of software teams the testers role is to assess the quality of the work being produced by the developers. This is usually done by manually testing the software using techniques such as exploratory testing to find any issues that may impact the end users. Any issues found will be raised with the developer who did the work (as defects or an informal chat) to fix if the team deems necessary.
Some teams found this process to be one of their biggest bottlenecks in releasing software so attempted to automate more of this type of testing using UI tests with varying levels of success. Others meanwhile started to look at the testability of what they where producing and started to build quality in.
In both of those scenarios the testers role looks obsolete. With the first supposedly replacing the work they did through automation and the second by removing the role as issues are mitigated before they have end user impact. In both cases what they alway assume is that a testers role is just that, to test.
If the perceived value of testers is just to test the changes made by the development team then the future of the testers role looks bleak but could it be more than this?
Some testers are starting to repurpose the old QA acronym to mean Quality Awareness. What they are doing is shifting the perceived value of their work from purely a testing activity, which was to assure the quality of the system to that of raising awareness of the quality of the system.
This may, at first, look as if they are still doing the same work under a different name, but on closer inspection it is a vastly different role.
I’ve always found that tester representation at agile conferences to be lacking. It’s a bit like it doesn’t have the word test in the titles so it’s not for me. Personally I’ve always found a treasure trove of information from talks that are directly or indirectly related to software testing. Remember testing doesn’t always look like testing you sometimes need to change the frame with which you look at things to get the best out of them.
Below are the talks that I attended with a brief summary of the talk and what it could mean for testers.
Summary of the talk Good talk covering all the basic with keeping your system update and why. Well delivered and really useful for less experienced team members and a good recap for “they should know better” members.
For Testers For testers understanding what needs to be updated when can help them understand how that change could affect end users. Be proactive what do the release notes say for X, how do we use system Y. Building this knowledge takes time but can be really valuable in the long run. Start small and work your way up. Developers can help you but try and help by having specific questions for them.
Summary of the talk: Forecasts will always beat estimates for non-deterministic projects (think all software projects). As they help you understand what could happen with a confidence rating. You’ve probably already got most of this data but knowing your lead times and throughput can help with this and some spreadsheets. The key thing to remember is you need to talk to your stakeholder and make sure they understand what the numbers mean and how it affects them. Don’t just give them the spreadsheets and expect them to understand.
For Testers: If quality means value to someone then value to people working in delivery roles is greater predictability with delivering our software systems. Understanding what these metrics are, how they are used and what affects them will not only enable you to have more productive conversation with delivery but also understand why they are important to that group. This will help you articulate risk better within your teams as you will be able to tailor the message to that specific audience. This will not only help you help them understand risk better but increase your value from just being the person that finds all the issues. This elevates you from being just a tester towards a test analysts.
Summary of the talk Really interesting talk by @cj_smithy at #agilemanc focusing around the book Crucial Conversations but bringing in ideas from the Chimp paradox, 5 dysfunctions of team, Radical Candor and generative cultures by Ron Westrum. Conversations within agile software teams are incredibly important (remember individuals & interactions over process & tools) so being better skilled at them is a great thing.
For testers We have conversation with team members all the time. Whether that is to find out information or to inform others about whats going on, its a core part of our skill set. Getting better at communicating verbally should be part of our personally development as testers. The resources mentioned in this talk would go a long way to help you build up that skill and help keep it sharp.
Leading an agile organisation through hyper growth
By Patrick Kua
Summary of the talk The company Patric was a CEO for went through some really fast growth over a very short period. The model he used was simple and acknowledged that it wouldn’t work forever so they kept iterating and scaling the organisation. Useful to benchmark your company against to see where you are in the growth of your organisation.
For testers Understanding how a company develops from startup to enterprise is really helpful in in seeing what types of problems you’re likely to face. This can help you help your team understand how quality is likely to be affected when scaling and what they can do to mitigate it.
Improve your agile coaching skillswith a Training from the BACK of the Room
By Sabine Khan
Summary of the talk: Its called back of the room as you’re not using slides decks and you presenting but getting the participants to stand and present instead. Hence coaching from the back of the room. Really interesting approach to learning and using coaching skills in teaching others. The 6 learning principles are really quite easy to apply and can make almost any session interactive. This combined with the 4C’s gives you a framework to turn any learning topics into something more than just sit and listen.
For tester needs updating: Need to help team members understand what exploratory testing is then why not do something interactive instead of just another slide deck. Teach them through doing and the key points might actually stick.
Culture + Code ≠ Delivery
By Vimla Appadoo @thatgirlvim
Summary of the talk: Vim makes a great point in that just delivering through code and a good culture isn’t enough as this misses how that delivery affects users. Especially if the system has biases built in unintentionally. She says we need communications as well. Communication to be able to link together all the parts of the culture of the organisation, systems, processes and people + the code can help us deliver the right things.
For tester: Testers help raise the awareness of quality within systems, but for them to be able to do that affectively they need to take into account the team, the business and users as well. The key skill to be able to do this is communication and helping to link together all the parts to understand the other. This is not to say that testers are the key to culture but on a smaller team scale they can have huge influence over what direction that culture moves in. Do the teams care about how their systems affect users or wait to see what happens?
To build peoples confidence with automation you first need to understand why you’re doing it.
Why do we automate things?
If you look at automation in general the reason to do it is because we have some sort of repetitive manual task that we want to be able to do automatically. By doing so you would remove any inconsistency that could occur from doing it manually. This would also make the output of the process reliable and repeatable as and when you need it. In short automation can make processes consistent, reliable and repeatable.
This usually leads to other benefits too such as the ability to scale up the automated process in terms of frequency and speed all while reducing costs in some scenarios. Essentially you can take advantage of economies of scale.
The benefits of consistency, reliability, repeatability and scalability is that this helps the people associated with that process to have confidence in the output of that automaton. They can either see the process happening again and again or can inspect the output to validate their confidence in the process. You could even take it step further and automate the inspection too.
But what about test automation?
The above works well for say automating a physical process such as making a glass bottle. You can either see how the bottle is made or inspect the end product. But when it comes to test automation you can’t “see” the test occurring (or any software process for that matter) and the only output is likely to be a result: pass or fail.
The only way to gain confidence in the automation is to either inspect the code (process) or your confidence is based more on the person doing the automation. You trust that they wouldn’t fake it or maliciously do anything wrong.
If that confidence is lacking then the only way to feel confident that the system being tested works is to test the system again. Therefore any of the benefits gained from automation have been lost as you are now duplicating the effort. The biggest loss being the economies of scale.
This is by far one of the biggest reasons why tester in teams have very little confidence in the automation. They don’t know what is covers, how it works or even if it’s being done to a high standard. If your job is to understand and raise risks within the team then this almost leaves them with no choice but to test it again.
Building confidence
If you’re a developer or automation specialist then you have two options in improving peoples confidence in the output of the automation. Help them “see” the process or build their trust with you. Both of which will go some way to improving their confidence with the process.
Better yet, help them understand the principles behind the automation which if done with humility and compassion is naturally going to lead to those people trusting you as well. By helping them understand the principles behind the automation you enable them to work out for themselves what is and isn’t being automated but also to what standard it is happening to. This then lets them see where the gaps are in the process which they can raise as risks or work to plug them up.
I’ve written about building a team understanding of unit testing which details how you can document your principles in a way that is accessible. You can use this method to document any team principle not just unit testing.
Whenever you talk about unit testing with teams they never tell you what it means to them. They go straight to of course we do and show you 100s of passing tests. Interesting thing is by calling it unit testing everyone thinks they are talking about the same thing. But when you start digging into how they understand it you begin to see that everyone talks about it and understands it differently.
What do the unit tests test?
A selection of responses to the question what do the unit tests test
A unit means different things to different people but we never stop and ask what does a unit and unit testing mean to you? Why? Well that could be risky as you’re potentially questioning someone’s ability. Which probably says more about psychological safety in your team but thats a topics for another day.
A Unit means different things to different people
So what should you call it then? Well maybe as a stop gap just call it what it is a test that checks code; Code test. Now I know what you’re thinking “thats way too generic!” Which is kind of the point because when you do that the first thing people ask is “What’s a code test?” Now you can start the discussion without anyone feeling that you’re questioning their ability.
What are Code tests?
How do you build a team understanding of what it is?
One of the best and easiest ways is to get the team together and pose them three questions:
Three questions to ask teams about unit testing
What does a unit mean to you in unit testing?
What characteristics make a good unit test?
What characteristics make a bad unit test?
Hand out sticky notes or use whatever online tool your team prefer. (Miro is a pretty good online collaborative white board). Then ask each question one at a time. If you can do it in person then doing this in a big room with lots of wall space is best as it allows for people to talk to one-another during the idea generating stage. Allowing them to talk is advantageous as people will build on top of each others ideas. But this may not be practical for distributed teams.
Building a Team Understanding of Code Tests
Once everyone has had a chance to contribute, group and theme the responses. Then as a team look through them and see if there are any contradictions or if anyone strongly disagrees with the groups. If there is then this is a perfect time to build the teams understanding of what code testing is.
Building a Team Understanding of Code Tests
If you’re looking for some inspiration then watching as a group Ian Cooper: TDD, where did it all go wrong and J.B. Rainsberger: Integrated Tests Are A Scam are a good places to start. Both these talks are quite old now so more up-to-date versions maybe available.
You may find that you need to run the sticky note exercise again to build consensus but essentially you want the groups agreement on what a unit is and what are good and bad characteristic of a test. This will give you a high level understanding of what a code test is.
What do you do once you have group agreement?
So You’ve got a high level understanding but you need to turn that group understanding into something more solid. Something that gives them
Alignment with each others understanding
Autonomy with how they actually implement code level tests
But also Accountability so not only is it their responsibility to do but to do it well
autonomy, alignment and accountability
You could just say “look at the code for examples” but as we’ve seen from before this isn’t always the best way as the intent behind the code may not be clear to everyone that reads it.
Ideally it would be something that is lightweight, but not too light that it’s too open to interpretation e.g. sticky notes. But not too heavy either that no one ever reads e.g. 10,000 word essay hidden in a Wiki.
Lightweight documentation
We need to document it in a way that is quick and easy to read and therefore remember.
The best way to demonstrate this is through an example. Now this example isn’t describing code testing (you need to have that discussion with the team first) however it has all the elements we are looking for.
Example principle
The title is short and to the point which makes it easier to remember but also acts like super short summary of the principle itself.
The first paragraph describes what it is about. The language used is really easy to understand too. It takes no effort to read and comprehend. This allows the reader to spend more time understanding the content rather than trying to decipher the words used.
The second and third paragraphs detail good and bad behaviours respectively. Finally they have a list of links that show where they have demonstrated this behaviour.
The great thing about this structure is that each part builds on top of the previous part. The title gets built on by the description. The good and bad behaviours builds on top of the description and the links give concrete examples of those behaviours so the reader can see them in action or even gives them the opportunity to add their own.
Back to the sticky notes
What To Do With The Sticky Notes
You might have worked this out already by those sticky notes will map onto this simple title/description/good/bad framework quite easily. The what does a unit mean to you would be used to write the description of what unit testing is. The key points from the good and bad characteristics would make up the good and bad behaviours descriptions. Finally all those unit tests you have should be used to demonstrate where those good and bad behaviours have been shown in your code base. You’re on your own for coming up with a snappy title.
Autonomy, Alignment and Accountability
You’ve got your lightweight documentation but how does this relate to creating team autonomy, building alignment between developers and making them accountable for their actions?
Building Alignment through a common language
The description is all about what a unit is and gives a common language for the team to use when talking about code testing. This helps to build aliment between team members.
Creating Autonomy through why not how
The good behaviours say nothing about how to write good code tests just what makes a good test within this team. Hence the focus on characteristics during the sticky notes session. The good behaviours coupled with the bad act as guard rails in what we do want and less of what we don’t. This works to keep the developer autonomy as they still have to workout how to actually do it. If they are unsure they have links to where the team have actually implemented tests that demonstrate this behaviour or they can always speak to the other developers.
Autonomy & Alignment enables Accountability
By documenting the principle using easy to comprehend language to build a common team vocabulary and describing behaviours instead of instructions to create autonomy you increase the responsibility within the development team that they are accountable for enabling the principle. Not only that it makes it that much easier for people to find more information and lowers the barrier to approaching the subject in the first place.
One of the great things about documenting things is that you can point at that thing and say you don’t agree which is a lot easier then pointing at a person and saying the same thing.
How Does This Map Onto Alignment Autonomy Accountability?
In Summary
By following this model you can begin to create a team understanding of what unit testing means to them and create a unified language so that they can talk about. It also lowers the barrier to understanding the approach for others which really helps to improve the overall team confidence in what unit testing does and doesn’t give them.
Documenting your teams understanding of unit testing using this lightweight model means that when people eventually leave that knowledge doesn’t leave with them or slowly erode from the teams memory. Another benefit is as new members join the team they can use this to build up their understanding of how the team approach unit testing.
There is a risk that the information does become outdated but you could use the new joiners as motivation for the team to re-visit old principles and see if they are still valid or need updating. You never know by including the new joiner in this process they may add something that you hadn’t considered before and gives them an opportunity to start positively contributing to the team. At a minimum it kicks starts the conversation again and allows the team to visit old assumptions and behaviours.
You could also use this model to document other principles that the team would like to work by all while maintaining their individual autonomy, alignment with one another and emphasising accountability that it’s up to them to make it happen.
Now you can see if it still makes sense calling them code test, unit tests or something else all together.
