12 things I learnt from Agile Manchester 2023

Reading time: 5 minutes

Below are my key highlight and insights from 3 days of Agile Manchester 2023. You can skim-read it to things that catch your eye or deep dive by following the links to my notes taken during the talks. Please note: expect lots of typos and mistakes in my notes. If you have any questions, then let me know.

Still too long, didn’t read:

Day 1 – Software is not like wine but more like milk. You want to use it quickly before it goes bad. – From Charity Major’s talk, is it time to fulfil the promise of continuous deployment
Day 2 – The bottlenecks in any team are always at the interaction points – From Remote mob programming in a high-stakes environment.
Day 3 – Working with people with different perspectives and backgrounds is essential because your domain context can limit what you can see. Jon Ayre emphasised this point in his talk on Agile at Scale.

Day 1

Mathew Skeltons talk On CD at scale made me realise that agile, scrum, continuous delivery, DevOps, quality engineering etc., are all trying to achieve the same outcomes – smoothing the flow of work while simultaneously improving speed and quality in a sustainable way for engineering teams to deliver regularly, repeatedly and consistently. With each, in some instances building on top of whatever came before. But after some time of trying to work those methodologies and failing, those terms become trigger words, so we have to find another way to talk about it and get people on board.

So it doesn’t matter what you call your approach. As long as you have a common understanding, then you’ll make progress.

Emily Webbers, Why can we just get along? It showed me that multi-disciplinary teams are critical to successful teams, and you can use the capability comb to help people connect. Another insight I had was that how we collaborate in teams varies a lot, so having a better definition of what collaboration means might help people do more of the effective type.

Lianne Mellor and Nikola Goger’s BFFs & rocket ships talk showed me how valuable a strong metaphor can be to hang your idea off. They used rocket ships, allowing people to bring in their ideas of space exploration. This creates a shared understanding and minimises the interpersonal risk people need to take when asking what something means. Documenting the terminology is also valuable so people have a backup when clarifying their thinking. This only lasts for a while but can be enough to get an approach started and people to buy in.

Racheal Shah’s talk, Delivering delivery metrics, showed me how valuable building credibility is before heading into metrics conversations with teams. She did this by first getting to know as many people 1 to 1 as she could, having open forums for people to chat with her and sharing documents of her thinking and background. This way, the people that want to engage know where you’re coming from and the value of what you’re trying to do. They will eventually help spread that message if they buy into your approach and ideas.

Charity Major’s talk, is it time to fulfil the promise of continuous deployment taught me two things. The first is that software is not like a wine that ages better with time but is more like milk that gets worse, so we should ship it as soon as possible before it spoils. The other was having the engineering team truly practising continuous deployment is an excellent tool for retaining and recruiting people. Everyone wants to work on delivering meaningful value, not toiling with technical debt you might not have even caused.

Day 1 highlight

Software is not like wine but more like milk. You want to use it quickly before it goes bad. – From Charity Major’s talk, is it time to fulfil the promise of continuous deployment

Day 2

From Dr Lewis’s talk, psychological safety – how to boost creativity and increase collaboration– backed up many of my thoughts and ideas about PS. But one I had yet to hear before is the first attempt at learning or, shorter, Fail.

I learned from Jaimella Espley Laughter and High-performing Teams that Laughter could help people bond and feel more psychologically safe. I also learned that play is an excellent tool for encouraging Laughter, but also that it shows people that it’s ok to express yourself in a way that is none judgmental. You also don’t have to be funny. Find the little things that make people laugh, amplify them, and iterate. It won’t always work, but eventually, you’ll find something that does.

For me, the underdog talk of the day was Giovanni Asproni’s remote mob programming in a high-stakes environment. This team built the UK’s COVID-19 app for track and trace. I got four takeaways from this. 1. In a high-stakes and pressurised environment, remote mobbing is a great way to ensure you create a high-quality and on-time product, regardless of your team’s skill level. 2. The bottlenecks in any team are always at the interaction points. So limit them as much as possible by stopping handovers and mobbing/pairing on work. 3. Leaders can act as firewalls and gateways to your engineering team, so use them to shield them from distractions. And 4. working in a mob forced you to work through your differences. You can’t put it off or ignore it for very long, so you have no choice but to either find a way through or someone has to leave.

Day 2 highlight

The bottlenecks in any team are always at the interaction points – From Remote mob programming in a high-stakes environment.

Day 3

Annette Joseph’s talk, seven steps to Unlocking the Power of diverse teams, showed me a great way to help people connect to the idea that our in-groups are often less diverse than we might think and are usually products of our environments. So if you want more mixed opinions in your life, change your environment. She also introduced me to the concept of group attribution error which is similar to the fundamental attribution error but applied to how we think of our in-groups as rational and logical people but our views of out-groups as illogical and prone to bias. Also, our brains reward us when out-groups fail and feel pain when our in-group fails. This reinforces why we like people that look, sound and act like us.

Jon Ayre’s talk, Agile at Scale, taught me an important lesson: context limits what you can see within a given situation using a simple game that asked people what the following symbol meant (X). Most said the letter X. But to a Roman, that would have been the number 10. He then asked how you would make it a 9. You would add ‘I’, which makes ‘IX’. He then asked how did you get 6? We all said it would be ‘VI’, but one person said to add ‘S’. Because we had all been thinking of Roman numerals, our brains couldn’t see the seemingly obvious right in front of us. He also showed me how much we are products of our environments using an example of experiments on rats. If given limited food, they would fight and kill each other. But if they were put into enriched environments and given the same amount of food, they were more willing to share and survive together. It’s often not the person (which is our bias of naive realism at play) but the environment that makes people behave the way they do. Culture is a response to the climate, so don’t try and change the culture to change people’s behaviour. Change the environment.

Valerie McLean’s talk, it’s simply not that simple, showed me soo many things. But, if I had to narrow it down, it reminded me about complex adaptive systems and the four categories people can often fall into when coming up with solutions to problems. These are – Hierarchise, who see issues due to lack of rules. Egalitarians see problems stemming from the weaknesses of a community and that we need more solidarity. Individualists see things done to people not playing their parts, and fatalists see everything as doomed. Ceri Newton-Sargunar helped me understand how these could connect to our fight, fawn, flight and flip response to fear. Valerie also reminded me of enabling constraints, which I’d forgotten as a tool and need to use more. She’s also onto a great idea on 7 steps on how to work with complexity, which is worth keeping an eye on.

Day 3 highlight

It’s essential to work with people who have different perspectives and backgrounds from you because your domain context can limit what you’re able to see. Jon Ayre emphasised this point in his talk on Agile at Scale.

How We Built Testability with Psychological Safety [External post]

Ben Linders recently interviewed me for my talk at AgileTD on how we failed at testability. That resulted in this InfoQ post about how to build in testability you need developers and testers to collaborate. But to be able to do that, you need psychological safety

Testability can enable teams to make changes to their code bases without requiring extensive regression testing. To build testability, team members must collaborate and leverage each other’s unique skills. Unfortunately, effective collaboration does not come naturally to people and therefore needs leadership to nurture people’s ability to speak up and share their knowledge.

To continue reading, head over to https://www.infoq.com/articles/testability-psychological-safety/

The courage to supercharge your testability

Testability is all about building quality-in. It’s about identifying known issues before they become a problem while coding. Pairing testers into this process can supercharge the testability feedback loop. It can allow you to pick up known and unknown issues.

But pairing devs and testers together needs courage. Courage so that both disciplines can take interpersonal risks and share hard things such as what they don’t know, don’t understand or mistakes they’ve made. This will need both groups to listen, understand and ask questions to help each other through the process. Both groups will need to show curiosity, humility and empathy for one another. You will not only feel uncomfortable during the process but it will take time too. The temptation to go back to inspecting for quality – dev and test handing work off to each other – will be hard to resist.

Pairing for testability is not just pair programming but working together to understand what the behaviour of the code being written should and shouldn’t do.

Devs and testers should work together to leverage the skills that each have, not get hung up about the skills they lack. If your pair is more exploratory focused identify ways that allow you to make the best use of those skills. If they are more technically inclined then focus there.

Remember the key is to build quality-in not inspect for quality. So what can you do now that helps your team move in that direction?

13 things I’ve learned from running remote workshops

Over the last 12-18 months I’ve been running a lot of remote workshops and during that time I’ve learned a few valuable lessons along the way. Most of my experience is based on translating and running an in-person workshop to a remote online workshop. The in-person workshop I’ve run 4-5 times and the online workshop I’ve run 8-10 times. The nature of the workshop is to help a team collaboratively create a model of how they work and then use that model to improve their ways of working. So it requires a lot of participation from every team member to be successful. What was my measure of a successful workshop? One that results in a visual model that everyone in the team agrees is how they work.

With that in mind, this is what I have learned from running remote workshops that require group participation.

1. Practice your setup if using screen sharing with slides and other apps

Always check that what you plan to screen share works with your video call tool of choice. I’ve found that it never quite works the way you think so it’s always worth checking out beforehand. I’ve noticed that different apps can behave differently when screen sharing in full screen mode so a dry run of exactly what you plan to do can help iron out some of these issues. I’ve also noticed if you’re switching between apps e.g. Miro and Powerpoint then check that things switch back correctly as sometimes they don’t always switch the way you expect it to. I’ve also started to check with the audience that they can see what I want them to as there have been too many times I thought I was sharing and wasn’t.

2. You need to do more up front planning for remote workshops

You have to plan for any workshop but with remote workshops, you need to plan for every eventuality.

With in-person workshops, you can experiment with ideas on the fly but this is trickier when people are remote. The main issue is communication. With in-person workshops, it’s really easy to tell everyone what you want them to do and then get almost instant feedback on whether or not they’ve understood it. But when people are remote you can’t see the feedback as easily so often you have to wait for the output and judge from that which can be harder if attendees are in breakout rooms.

3. Factor in extra time for tools

Online sessions typically require digital tools which if the group is unfamiliar with will have a learning curve which eats into your workshop time. But with in-person workshops you tend to use pens and paper which everyone understands how to use and if there are any difficulties in using the tools you can physically see what’s going on and intervene. Unfortunately I’ve found that with remote sessions it may not become apparent that something isn’t quite right until the results of the task can be seen. I’ve started to add 5 minutes of buffer to any group activity as it normally takes them that long to get going on most occasions.

4. Remote workshops can be more democratic and can give everyone a voice

Remote workshops tend to level the playing field and physical presence becomes less of an issue. It can also be more democratic in giving everyone a voice if tool based signalling is used to indicate you want to say something. In some cases I’ve found using chat features can help some to share their thoughts or approach hard to discuss subjects too. But all these features can take longer so make sure you factor this into the session.

5. Breakout out rooms of 3 are best

I found that limiting breakout rooms to 3 participants works best. The lines of communication are simpler, people contribute more and the overall result tends to be better. With groups of 4 or more, you start to see the dominant members taking over and the quieter ones sitting back. But in groups of 3, this doesn’t happen anywhere near as much.

6. Remote sessions need more breaks

I’ve seen participants during in-person workshops, while not ideal, quite easily contribute to 2-hour workshops, have a 10-minute break and go again. But for remote sessions, most people need a break almost every 50 minutes to be able to participate in half-day or whole-day workshops.

One of the main reasons being reported is fatigue. My guess is that in-person sessions do not cognitively overload the participants as you can use a lot more of your senses to understand what is going on and move about the room. But with remote sessions all you have is sound and a small field of view. So you have to listen more intently to what people are saying and generally stay seated/standing (if you have the option). Therefore giving people regular breaks gives their eyes and ears a rest.

One thing I would also suggest is encouraging the participants to step away from their screens during this time. I’ve found that some take the time to catch up on emails/notifications instead and then feedback that the workshop was too long.

7. Ice breakers are a really good idea

I’ve found on occasions that people have come in from back-to-back meetings so it can take them some time to get into the workshop. Ice breakers can help participants to break out of whatever they were doing before and quickly and easily get into the new workshop/meeting. It’s a bit like warming up before you get into your gym session.

They can also double up as a way to introduce people to a new tool via a simple game, get them thinking creatively or a way for people to connect before getting into the main session. I’ve found that while it does eat into the beginning of your workshop it can result in a better overall outcome for you and your participants. So where you can I highly recommend doing it.

8. Sessions never start on time

It doesn’t seem to matter what time the meeting starts, I’ve played with 5 past, 10 past and even 15 minutes past. Some people still arrive 5 minutes after the allotted start time. It’s never many people but some people just have more meetings going on than others and it can be hard to get a comfort break in sometimes. On the other hand you get some people who show up on time everytime. So it’s worth considering what will people do that show up on time and how will you get the stragglers up-to-speed?

Now you can ask the early arrivals to chat ideally or make themselves a drink but I’ve always felt like you’re punishing them for being on time. Which unintentionally sends the message that it’s ok to be late too. Ice breakers can be a good fit here as it gives the early arrivers something useful to do and if it’s a good ice breaker the late arrivers might just feel they missed out on something so they might try and be on time in the future. It also means they are less likely to disrupt the main session trying to get up to speed.

9. Remote session can take longer

Now, this may vary from workshop to workshop but from my experience, the modelling sessions I’ve been running have taken nearly 2-3 times as long. I think this comes down to a lot of different factors such as the participants’ familiarity with each other, tools being used, the outcome you’re trying to obtain and more breaks. So being able to test out a workshop is a good idea to figure out your timings and if it will need multiple sessions.

10. People are not paying as much attention as you might like

I tend to find that I end up repeating instructions during workshops quite a few times so keeping them simple and available in multiple formats can make this easier.

This goes for in-person workshops too but any instructions, objectives (if critical) and information (if complex or new to the participants) should be verbally and visually represented during workshops. I’ve also found making it easy for participants to find for themselves can be handy too.

11. Don’t expect people to follow your instruction to the letter

Just because you said use a felt tip pen they might use a biro. Just because you said to use plain paper they might use lined. Just because you offered to send them the resources don’t expect them to say yes to them. Always think about how people could interpret your instructions. What might be significant to you may just seem incidental to them. In this way, in-person workshops are easier as you can give them everything they need and tweak instructions as you see things happening. So if some requirement for your workshop is important and an equivalent can not be used, make sure they know this upfront otherwise you might get some unexpected results.

12. You’d be surprised at what a pain it can be to take a photo and upload it

For my workshops I thought that doing some drawings, taking a photo and uploading it to Miro would be quite easy but in general, it causes quite a few issues. Most people’s photos are taken at quite a high resolution and not everyone is familiar with how to lower it. So when they do upload them they can take a little time and being quite large means they almost always need to be resized – taking more time. Getting the images onto your PC can be tricky too as there are quite a few ways to do this and I was surprised how few people knew how to do it. While none of the issues are a major problem when you have enough people experiencing different issues at different times, the easy job of uploading a photo becomes a real time drain. I’m sure this will get easier with time and/or using dedicated apps but be sure to factor this in if you want people to upload images.

13. Always summarise what people have been doing and why

I have always underestimated how often it’s worth repeating the aims of a workshop to the participants. Just because they have accepted the meeting invite and you have personally explained it to them it doesn’t mean they will remember why they are doing what they are doing. One of the things that I’ve regularly gotten in feedback is people not being sure what the outcome of the workshop is.

I think this is because most workshops are completely new to attendees so they are taking on board a lot of new information. So they tend to miss the details and instead focus on the current task. What they tend to remember is the beginning, any significant events during and the end of the session. Also significant events might not be what you want them to remember as it could be someone making a joke or how a tool failed. So make the best of the start and end if you want them to take something away from your sessions.

Which is best, remote or in-person?

From my experience, I’ve found that both have their strengths and weaknesses. Remote sessions can allow people from anywhere to participate and usually involve digital tools which means the outputs are much easier to share and reuse. But the downside is you lose that person-to-person connection and the energy that comes from that. They can also take much longer and may need to be split up over multiple workshops depending on the type you are running.

In-person workshops on the other hand can be much more dynamic. By allowing you to change things on the fly, have a lot more energy which can be motivating and keep people engaged in the session for much longer. I also find they can become a bit of an event especially since we’ve all been working remotely which could work towards helping people engage in the process. However, they can favour the more confident participants and making the outputs sharable and reusable can be tricker.

So which is best? Well, it depends on your context. If getting together is difficult then remote might be best for you but if the team is still new or you’re experimenting with an idea then an in-person session might be better. I still prefer in-person workshops as I find that the social element of getting people together and working on something as a group still produces better ideas, faster and in my opinion builds stronger and longer-lasting bonds between people. Which all contribute toward a more cohesive team.

Three things of 2021

Every week I spend some time reflecting on what I learned or found interesting and this is a summary of my year. After doing this for nearly 3 years one of the biggest ways it’s helped me with is seeing the thread through my work which reminds me of this quote:

You can’t connect the dots looking forward; you can only connect them looking backwards. So you have to trust that the dots will somehow connect in your future…
Steve Jobs’ 2005 Stanford Commencement Address

Where is that thread leading me? On a strategy that could help with improving team collaboration and heading towards a more generative culture.

Remote workshops

Remote workshops are constrained in ways that I hadn’t appreciated before the lock down. Such as by tools, participants work environments and people just getting tried in ways that just doesn’t happen in real life. I’m going to be following up with a blog post on 13 things I’ve learned from running remote workshops so keep an eye out if you want to know more.

Uncertainty

Your ability to identify and work through uncertainty, I believe, will be a big predictor in how successful you will be in the long run but also how satisfied you will be with life. The more I’ve learned about uncertainty and how it affects our behaviour the more I’ve changed the way I look at uncertain situations and approach them. What I’ve found is my attitude towards uncertainty has changed in a way that has made me much more comfortable to be uncomfortable with it.

How? By identifying what about the situation makes me uncomfortable. For example a situation has multiple directions each one with unknown outcomes. Then looking at how it makes me feel uncomfortable. For example a feeling in my stomach, a tremor in my hands, a tightness in my chest, a dry throat etc

This is known as interoception or the ability to sense your internal bodily state and this Guardian article does a good job of explaining it. Only then proceeding to work through the situation and deciding which direction to go in. To be honest this is much easier said than done but with practice can become habit and almost become a default way to approach unknown situations.

My experiences of this has been that by paying attention to how the situation makes me feel internally (interception) I’m able to make much more rational decisions and feel more in-control of myself even if I don’t have control of the situation.

This I believe is what helped me get over my fear of public speaking. It’s not that I got over the fear of getting up on stage but I was able to show my brain that there was nothing to fear in the first place. Over time (and this is important) my brain learns that fear isn’t the right response and tones down my bodies automatic reaction to the situation. Which in turn make me feel much more able to handle the uncertainty of it.

This I think is what can help people move out of their comfort zones and get them more comfortable with being uncomfortable.

Psychological safety

The idea of psychological safety has been on my radar for a few years. Starting with reading the The Phoenix Project in 2016 , The DevOps Handbook book in 2017. Which led me to State of DevOps Report 2018 and hearing about Google’s Project Aristotle the same year which both mentioned psychological safety for me for the first time. But I didn’t look into it until I read Amy Edmundson’s Fearless Organisation in 2019 via reading Kim Scott’s Radical Candor: How to get what you want by saying what you mean which referenced Amy’s work.

Then all through 2020 and 2021 all I could see was how so many people are holding themselves back in their teams by not saying what’s on their minds due to the uncertainty of what would happen. But I still didn’t act on psychological safety as I believed it was confirmation bias leading me to think that it was the key to getting people to speak up.

It wasn’t until late 2021 and I did an internal talk on Psychological safety: What the heck is it and why should you care? that I began to realise that this wasn’t confirmation bias. That we have a problem with speaking up in teams but we never tried to tackle what’s preventing them from speaking in the first place.

It was only after this talk that I felt much more certain that what Google had discovered back in 2012 that psychological safety is foundational to highly effective teams. Why? As this is what enables people to speak up and share what they do and don’t know. Speaking up is key for effective inter-team collaboration and enabling them to work through problems and head towards continuous improvement.

Which teams will need if we ever want them to be able to autonomously use the 4 key metrics to improve their throughput and stability of their products.

Connecting the dots

It is now that I feel I can now look back through all the different things I’ve done and learned over the years. And see how it is all connecting together into a strategy that could be helpful in increasing psychological safety at the team level.

I’ve worked at a product level in teams to see how listening and asking questions is key for being able to work through problems. I’ve immersed myself at the process level trying to understand and apply agile and DevOps principles to improve those products. I’ve collaborated with as many different disciplines to try and understand what their problems are at applying those principles to deliver those products.

But as Steve Job said you can’t see how things will connect in the future. I could never have predicted how all the little things I’ve done over the years would line up in the future.

You have to just trust that they will. This is why living with and working directly through uncertainty is going to be the biggest predictor of your success and happiness.

If you can get comfortable being uncomfortable, work through uncertainty and trust that things will workout you might just get what you want… or at least closer to where you want to be.

Interested to see my other past dots then check out my 3 things of 2020 and 2019.

Speed Vs Quality: Can you have both?

5 minutes reading time

I was recently part of a panel discussion around the topic “What is quality?” and an interesting question came up. Is it always a choice between speed of delivery and the quality of that delivery? The thinking being that if you focus on one you will sacrifice the other. Therefore, you have to choose one or find some way to balance the two which typically translates to being mediocre at both. That’s when it occurred to me that this is actually a version of working harder in teams and what we should be doing in working smarter.

Working Harder for Speed or Quality

Working harder in the context means doing the work in the same way you have always done it but trying to do more of it. This typically leads to people taking one of two approaches. One is by working longer hours and the other is taking shortcuts by skipping steps not providing immediate value, then using the spare time for doing other work.

At first both these strategies work quite well, you get the speed improvement and quality remains about the same. The problem is that all the effort is put into doing the work and nothing into improving the capability to do the work. If sustained for long periods those initial speed improvements will begin to slow and quality will diminish as the system gets harder to work with. Essentially you degrade your capability to do that work which further slows you down and brings quality down with it.

This is probably why most people, including myself, have always thought of speed and quality as trade-offs against each other. Every time we’ve tried to go faster, the quality of the system has gone down. On the other hand when we work more systematically and usually slower, however, things remain more stable.

But is there another way?

Working Smarter for Speed and Quality

In this case working smarter is not focusing on speed or quality but on your capability to do the work. Instead of just trying to go faster or improving the quality of the work, attention is focused on how that work is done. Specifically if any improvement can be made that could lead to improved speed, quality, or both.

This is where things can be a little counterintuitive. At first, while the team experiments with their capability, their speed is likely to drop as they figure out a new way of working. There is an even higher chance of quality being affected as things can be missed, or unintended bugs are introduced. But, as long as the team can collaborate effectively and learn from their failures the speed and the quality is very likely to improve in the long run.

The Trap of Working Harder

The problem with working harder is that it’s addictive once you start. When you work harder you tend to see immediate results so it feels productive. Not only that but others notice it too and are likely to praise you for going the extra mile. That initial speed boost can make it feel like you’re getting more done in less time, especially if you’re taking a shortcut here and there. This can trap you into thinking that working harder is the best way to work. The long-term damage of not maintaining your way of working is that it can end up being the only way to get things done.

The Virtuous Cycle of Working Smarter

Once teams start improving their capability to do the work it tends to free them up as they are working more efficiently. This additional capacity is re-invested into spending more time on improving their capability further which can create a virtuous cycle of improvement.

Let’s call it Continuous Improvement

Personally I’m not a fan of saying you shouldn’t work harder but work smarter instead. Not only does it come across patronising but it leads to more ambiguity and misunderstanding. I do appreciate that we need to have some form of common language and having a defined terminology helps minimise the ambiguity and risk of misunderstandings.

If we must call “working smarter” something then I’d opt for continuous improvement. While there is still a chance of it being misinterpreted, at least it has two of the keywords that working smarter is trying to achieve.

Conclusion

Working harder and working smarter are two ways you can approach and think about how you are working in your teams. What we need to recognise is that working harder only allows you to choose between speed of delivery or the quality of that delivery. While working smarter, over time, can enable teams to deliver a quality product at speed.

A good place to get started is having a shared understanding of what working harder looks like for you and your teams. How does it help and hinder your ability to deliver at speed or a quality release? From that identify areas that you can measure so you can objectively show that things are either getting faster, but quality is dropping or slower but quality is stable/increasing. Some good measures of speed are throughput of delivery such as leads times and release frequency. Quality can be tricker as it depends on what quality means to your team but a good proxy can be stability of releases based on change failure rates and mean time to recovery.

From there you can start experimenting with different ways of working that could incrementally improve your ability to do the work. All the while measuring to see if it is moving you towards delivering a quality product but at a pace that is faster than what came before. Which will enable you to move towards working smarter and begin continuously improving your ways of working

Incremental improvements: Why don’t teams do it more?

(Reading time: 10 minutes) I’ve been working with a lot of different engineering teams over the years and there has been a strong tendency for them to avoid incremental improvements and instead go for bigger changes. There are many reasons they give as to why but three themes stand out: wrong incentives, lack of knowledge and risk aversion. What are the causes of these problems and is there anything we can do about it?

What are incremental improvements?

The incremental improvements are typically process based such as reducing queue times, limiting work in progress, using smaller batches of work, employing automation, using better testing techniques or removing the test column. Which all contribute towards aiding the team to ship value faster. But rather than go after the incremental improvements that they can get started on now they advocate for the new tool instead.

The tools are usually dependent on the problems the teams are facing at that moment but typically can be a new CI system (usually moving away from Jenkins), the latest monitoring tool, new frameworks (e.g. automation tools) or even advocating for a whole new programming language.

Reasons teams often give for avoiding incremental improvements

The reasons for avoiding incremental improvements and advocating for the new tools vary but I’ve heard versions of the following over the years:

It feels easier to do big changes in one go rather then lot’s of smaller ones over time
It’s hard to see how the smaller individual wins over time improve the teams ability to deliver value
Nobody ever gets credit for fixing smaller things or preventing issues that then never occur
People get credit for making big bold moves that can be clearly seen
They tried the smaller ones before and it failed
They don’t see any problems with their ways of working so If it’s not broken, don’t fix it

I tried listing out as many as I could and noticed a few themes starting to emerge specifically around incentives, understanding of agile development practices and risk.

List of reasons people give for avoiding incremental improvements grouped into themes

Why does this happen?

Which got me thinking, why this avoidance of incremental improvements? What is it in software teams that could cause so many of them to give such similar responses? Some of the teams have never met each other so it’s not like they’ve been swapping reasons. Which led me to look at what in the teams environments could cause people to behave this way.

Wrong Incentives

What incentives do we setup in teams and how do we do it? From my experience the recent trend has been all about Objective and Key results (OKRs). Now there is nothing wrong with having goals that teams should aim for, I actually think this is a good idea. But do those goals unintentionally encourage feature delivery over improving the system? In some ways we maybe rewarding teams to go after bigger ideas rather then working to incrementally improve their system. To see what I mean have a look at your teams goals, do any of them encourage the team to improve their ways of working or are the focused on some business objective?

These goals may also encourage leaders to recruit people that are there to simply achieve that goal and incentives them to do so either via praise or other rewards. This has another affect in that the job adverts they put out ask for people with skills that can achieve that goal. Which could lead people to make sure they have those skills on their CV so that they can apply for those roles. Again further reinforcing team members to go after the bigger wins – no one is explicitly asking for them to go after the smaller wins that improve their system. It’s just assumed that they would do this.

Just to reiterate I’m not saying we shouldn’t use OKR or other goal setting techniques but we should stop and look to see what behaviours they collectively do and don’t encourage.

Lack of knowledge

In some teams (especially smaller organisation) they offer very little time if any to actually train on the job. So people are more likely to prioritise personal development that is actually going to get them more of what they want which is usually pay. Which after reading a few job adverts (see incentives above) they are likely to be tangible skills and not how to successfully improve the team’s processes.

In organisations that do have training opportunities and especially ones around agile ways of working than the team member has another issue: how does the training relate to them and their teams? These training options while on paper are a great thing they tend to be quite generic and leaves the actual implementation to the novice who needs all the help they can get. Therefore unless their team actively encourages them to share what they learned (e.g. by setting up experiments to try things out) the individual has little chance of changing the teams ways of working.

The another issue is how does experimenting with their ways of working fit in with the organisational goals? This goes back to incentives again in that even though you have this new insight from the training trying this out doesn’t seem like the right thing to be doing. Especially if it looks disruptive and the outcome uncertain and the current ways of working seem to be doing a good enough job. If it wasn’t then they’d surely have it as a team objective?

The other big issue I’ve seen in teams is no consistent understanding in how their team actually works. The process by which a lot of team adopted tend to happen out of chance or things that were implemented long before most of the team was even around. That coupled with varying levels of experience in the team means no one person fully understands how the teams works.

Risk aversion

The final issue and the one I think people are most familiar with is avoiding risk. You’ve probably heard the reasons such as we’ve tried that before and it didn’t work, or nothing is broken so why try and fix it. In some cases they do nothing as there are so many choices they don’t know which to go for so they stick with what they’re already doing as that kind of works and has a more certain outcome.

This I feel comes from people not wanting to take risks or if they do then they have a high probability of success – so it’s not really a risk. Think about when was the last time you were rewarded for failing? This hardly ever happens but I bet you can think of a time you or another team was praised for doing something successfully. By only ever rewarding success (an email with some praise can be enough) then you unintentionally punish failure. Not only that failing never feels good and knocks our self-esteem so we tend to do all we can to avoid it. If failure does occur we will deny it was us, distort it so it looks better then it was or just ignore it and carry on as if nothing ever happened.

Add to this that even speaking up about failure is hard as no one wants to be judged about being incompetent, looking ignorant, causing disruption or generally being considered negative then no wonder we tend to avoid failure. This all leads us towards going after a sure thing like a big process change that will give us everything we want in one go.

What do we do?

Trying to solving any of these issues is going to be a difficult thing to do and likely to lead to lots of false starts, dead ends and failures too. For most teams and organisations these issues are just too complex and are likely to be ignored all together or left for someone else which typically means no one. So is there anything we can do?

3 Questions

I think a good start would be if we asked ourselves at the team level how are we framing incentives, training and risk taking? This line of thought led me to 3 questions that could help Team Leads start to tackle some of the reasons why teams go for big changes rather then smaller incremental improvements.

Question 1 – Have we unintentionally incentivised big wins over smaller incremental improvements?

Take a look at your teams goals and objectives. Do any of these encourage your teams to make smaller incremental improvements or big wins? Could you reframe the conversation around any of these goals that could encourage more smaller incremental improvements? How can you maintain that framing as the team work towards accomplishing that goal? Are there any examples you can highlight to the team that show others working in this way in a similar context?

Question 2 – Could centralised training unintentionally remove responsibility from the teams needs to the individuals needs?

How do you identify the training needs of your team? Is it on an individual-by-individual bases or a more whole team view? How does the training your team members go on relate back to how the team works? How often do you do whole team training? When an individual does do some training how are they encouraged to use what they learned back in the team?

How does your team communicate their ways of working to others? Do they have a consistent structured approach or is it more ad-hoc and depends on who you ask? When do you take the time to understand how the team is currently working and if things need to change? Do you have any metrics on team performance that tell the team if they are getting better or worse at what they do?

Question 3 – By only ever rewarding successes have we unintentionally punished failure?

How do you recognise successes and failure in your teams? Do they get equal amounts of attention or is it just the successes that are talked about? When was the last time you failed and you shared that with the team? What was the focus of the discussion? Was it positively or negatively framed? Do you have a structure to understand and learn from failure or is it up to the individual to figure it out for themselves? Learning from failure is not easy and needs strong leadership to enable it so what can you do to encourage and enable it to happen more frequently and positively?

It’s always about the discussion

None of these questions actually solve the problem of teams focusing on bigger wins over smaller incremental improvements and perhaps in some cases the bigger win is the right solution. For example cases such as do or die situations where the team isn’t going to exist if it doesn’t make huge changes overnight. But I would hope this isn’t a regular occurrence for teams otherwise you’ve probably got bigger problems than training or misaligned incentives.

What these questions do is get the discussion started by getting people into the right frame of mind to have that conversation and start working towards a solution. The solutions to these problems are likely to be very context dependent so what may work for one team might be the wrong for another. Therefore me or anyone else proscribing a particular solution may not be that helpful. You need to be agile in your context. Starting with the right questions might just get you to the solutions you need.

What do you think?

Have I got it wrong and are big wins the best way most of the time?
Am I missing any common reasons teams give?
Have I made assumptions in my causes to fit the narrative I’ve set out?
Is my analysis just plain wrong?
Let me know in the comments

The risk with direct questions

The risk with the direct question is that the person being asked could assume intent within the question. E.g. asking what risk there in this release could be assumed that you think there is a risk in the release or that you don’t trust the individuals ability.

This could lead to a break down in your relationships and make asking any further question almost impossible. This is more likely if you don’t have a working relationship with the person and is another good reason why taking time to get to know each other is so important. See foundations of great teams start with relationships to learn more.
So what do you do if you need to ask questions that could be interpreted as having intent?

Use indirect questions

The indirect question come across much more tentatively and allows the person being asked to offer more if they want to. If it is taken in the wrong way it also allows you to back out and try and get back to a productive conversation.

Now if they respond in the negative with no additional information as to why then you can tentatively inquire as to what makes the individual so sure. e.g. That’s great, what is it about this release that makes you so certain?

Examples

Direct: What risks are there in this release?
Indirect: Do you think there could be any stakeholder impact in this release?

Direct: What could go wrong with this release?
Indirect: Are there any ways in which you think this release could behave unintentionally?

Direct: What risk mitigation have been carried out for this release?
Indirect: Are there any areas you think we could have impacted with this release?

The indirect questions asks the person for their opinion on the situation which takes away any emphasis on their work. While the direct questions don’t mention anything about their part in the work the risk that they could interpret your body language/tone or some past interaction as the reason behind you asking could derail the conversation. Essentially they may not give you the benefit of the doubt and jump straight to malicious intent even though there is none.

Trade-offs of indirect questions

The downsides of indirect questions is that they take longer to ask and more effort to construct. Which slows down feedback loops and learning from each other. It also makes long term collaboration that much harder and more likely for people to avoid situations all together.

While building effective working relationships seems like a lot of effort I believe the long terms benefits of more effective collaboration is well worth it. Good relationships lets you just talk to each other.

3 traits of Self-leadership

A colleague of mine Qambar Raza recently shared this great Ted talk with me on self-leadership 👏

🎥 🍿🥤 Great leadership starts with self-leadership | Lars Sudmann

In it Lars describes three behaviour traits to improve your self-leadership:

1. Self awareness
2. Self reflection
3. Self regulation

This is a brilliant way to change any personal trait that you would like to be different.

The key to it is Step 1 by recognising what you do want, how that differs to what you do now and then identifying the incremental steps towards it.

The difficulty comes in Step 2 being honest with yourself everyday about how you are progressing towards your goal and what feedback mechanisms you are using.

Finally Step 3 is recognising that situations outside of your control will still occur where you might default back to your existing behaviour. So instead of trying to repress your response reframe how you look at the situation instead that enables you to look at the situation in a different more positive mindset .

Don’t know where to start yourself improvement journey then start by identifying some of the default behaviours you might exhibit.

Reducing Uncertainty in Software Delivery

I recently attended a half day online event that InfoQ held on Reducing Uncertainty in Software Delivery. The thing that made this half day event different was the underlying focus on testing but without a single tester present in the talks or panel discussions. The majority of speakers where developers and there was even a few Engineering managers, Product people and a CEO or two. It also appeared to me that none of them have come from a traditional testing background. However they all made points that a good tester would and then some. The advantage they appear to have over testers is that they were able to incorporate their knowledge of their discipline to give a much broader view than just focusing on the testing itself.

A key theme that I’m seeing from these talks is that they are spending a lot of effort on learning from failure. Either by analysing ones that have happened in production or actively encouraging teams to cause failures. It was only the more advanced organisations that were taking this approach but the others were not far behind. Why? To make their systems even more resilient. Their approach appears to be using Site Reliability Engineers (SRE) to work along side their engineering teams to help them do the work but also enable the teams to extract the learnings from it too. This isn’t simply having chaos testing to cause failures or postmortems for production failure analysis but to also help teams with the people side of working with and handling failure productively.

The talks that caught my interests were Building in reliability (SRE at Gremlin), User Simulation for Rapid Outage Mitigation (SRE at Uber), and a panel discussion on Testing in production (with 2 CEOs, Product person and an Engineering Manager).

Now this is a small sample, the speakers are very experienced and working or have worked at some of the best known web based organisations (Google, Uber etc) and US focused too. But I’m seeing a lot of things that testers could advocate for being pursued and implemented by Site Reliability Engineers (SRE). For example:

testing in production,
building in observability,
pushing testing earlier in the process,
encouraging developers to test their own work

The advantage SREs have is they already have the technical ability and are now starting to build out the socio-technological skills that they were lacking previously. These organisations have another advantage in that they are heavily focused on learning from their failures. So when they do get things wrong they work hard to make sure they extract as much value from that failure as possible. On top of that some of these organisations are actively causing failures within their systems to further limit catastrophic failures that could occur. Some of these organisations have never had tester and from the looks of things never will. If you’re pursuing a true continuous improvement strategy testers could look like a bottleneck in the process slowing down information flow. How can testers enable the flow of information and what can they add that makes this information even more valuable?

I’ve pulled my summaries of the talks I found interesting below

Talk: User Simulation for Rapid Outage Mitigation

Uber uses an alternative approach to end-to-end testing due to their system being so big that no one person can ever fully understand it. Instead they use composable tests that each team will create that allows that team to tests their part of the system but mix in other parts pre and post steps built by their dependent teams. These are then run in a simulation environment that allows them to see how the system will perform when that change is deployed. To incentives team to build the tests they use a mixture of pain (woke-up at 3AM due to production failure) and mitigation support team (hold their hands at 3AM) to encourage them to build the tests. For example if you had these test you wouldn’t be awake at 3AM trying to mitigate the issue. They also don’t try and solve the issues at 3AM but mitigate them so others can also learn about outages that affect their system.

Talk: Building in reliability

Interesting talk focusing in on availability of systems within organisations. The speaker walked through how you could go from 99% availability to 99.99% and how it is a learning journey. Used a simple analogy going from crawling, walking and running to get your availability towards what makes sense for your organisation. Essentially can you do it manually, can you script it and can you automate it? I find this slide as a great way to help others understand what the outcomes are at each stage going from 99% to 99.99%.

Panel: Measuring Value Realisation Through Testing in Production

I usually only see these types of conversation from tester focused panels but none of this panel where testers. Tester focused panels typically focus on testers testing in production but this was very much focused on learning from real users in production. Interesting thing from my prospective was they made all the points that I would expect a reasonably experienced tester to bring. In some cases due to their roles being out of testing they focused on the costs and benefits that were outside of simply testing in production e.g. down side of A/B testing or product management mindset shifts that need to happen to embrace learning from users rather then whatever the road map they have decided says.

In some ways testers testing in production almost act like the middlemen of the learning that happens during testing. Could it be that in some cases testers are getting in the way for teams to learn effectively from testing in production?