As I sat with a professional services team at my client, launching into my tried and tested description of what agility means and how to do it sustainably, the last 11 years of my life flashed in front of me and I saw something that I hadn’t realised before. It both scare and energised me.
My typical approach to this particular exercise is to begin by introducing my audience to the Agile Manifesto – the spark of the movement. I remember to use that word – ‘movement’ because I truly believe this is what those who truly do agile software development are part of.
Then I pretty quickly move on to processes (or process frameworks) – the ‘how’. I do this partly because I assume that is what folks want to hear – “enough of this happy clappy, pie-in-the-sky bollocks. Show us the meetings, the artefacts and stuff we can put on our CVs”.
I also do it because I hadn’t really stopped long enough to consume the manifesto, to meditate deeply on it. And that might be because I wasn’t ready.
Well now I am.
As I wrote the 4 value statements on the whiteboard, time stopped. For context, here is the Manifesto for Agile Software Development:
Manifesto for Agile Software Development
We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
That is, while there is value in the items on
the right, we value the items on the left more.
What actually triggered this epiphany was a simple question:
What would it take to fully understand each of those value statements deeply and live them deliberately – every day and in every facet of your team or organisation?
If you imagine a journey to increased agility was a 12 step program where you couldn’t proceed to the next step without satisfactorily completing the preceding step, then seeking to seriously answer that question would be steps 1 – 11 and mostly like a ‘formal’ process or framework like Scrum or SAFe would not even feature.
Don’t be fooled – this stuff is complex. Take an even modest attempt at going a little deeper into the first value statement – ‘ Individuals and interactions over processes and tools’ – :
Who are the individuals, what are the interactions? How do we know they are the right ones?
What personal and professional skills and competencies does each individual need in order to participate meaningfully in these interactions? How do they acquire and develop them?
What conditions sustain meaningfully interactions, how we create and maintain them?
How do we continuously sense that the interactions are not yielding what they are intended to yield? What do we do when that happens?
This is not a remotely exhaustive list, but they are considerable questions. In my mind, these are not ask-once-and-forget questions, they are to be asked and answered regularly.
There is even an earlier starting point. Do we have the skills to ask the right questions?
Tools like powerful questions, clean language might be useful places to look to gain those skills. I have met far more people who suck at asking the most effective questions than not.
Now I’m not saying don’t do Scrum or use Kanban or XP or whatever the flavour of the month is. But it is very possible that you might do those and not enjoy the agility you seek, worse still you may get lost in process and without a solid understanding of these fundamental first principles you will struggle to regain direction.
I am saying go deeper into these 4 statements. Fight the urge to go into processes and the tickboxes. Do not simply ask questions and pat yourselves on the back that you answered them – actually execute on the answers. Make those answers make a difference to your work and your life.
And oh, if any of your answers is ‘do process X’ , then please consider getting one of these:
If your company/team needs a spark to improve it’s delivery capabilites, perspectives, focus, vision, value system and (no tomatoes please) culture. Then I have a gift for you. I’ll trade you a spark for £500 before 5pm BST, Thursday, May 12th 2016.
If you or your company donate a minimum of £500 to my camino walk for ME/CFS – and thereby help me reach/exceed my funding target of £3000 by 5pm BST, on Thursday, May 12th – then I will come to your company/team on a mutually agreed day in July or August to help you improve how you deliver software or any products or service.
Whilst there is no magic to it – just experience, honesty, empathy , a desire to cut through the bullshit and help your company/team rise to new heights. I’ll bring my experience of working with 1500+ people and 180+ teams over the last 11 years as a coach with some of the worlds most successful companies.
Things I can help you with:
super easy way to plan your releases (or even get rid of releases entirely)
getting pragmatic on just how agile you need to be to get where you want to get to
get *everyone* working together to increase value delivery
focus more on sustainable value delivery versus some whacky velocity
waste a lot less times in meetings
Just think about it – but not for too long – then donate.
Tick tock, thank you.
ps. Open to everyone, everywhere but… I’ll pay my way to Europe based teams/companies. Anywhere else we need to talk about travel costs.
How often have you met people – usually executives and management at conferences or other spaces outside of their company and they proudly proclaim ‘we do Scrum’.
I meet them alot – not only at conferences – on planes and more worryingly in their own companies. It’s like the Sixth Sense – only with Process.
Then with the slightest of prodding – examining what they produce, how they produce it, who they produce it for, what feedback they get – and how often they get it, what they use the feedback for and how quickly they apply it – the delusion begins to become clear. This is what I call the Process Delusion:
The Process Delusion is the pretense – for whatever reason – that you are doing something a certain way for certain benefits but there is little or no evidence that you are doing it or getting the expected benefits.
Let me say straight off – I applaud the willingness of anyone in any organisation who tries out any process to get some improvement.
It takes recognition that something needs to improve. So many just live with the gross imperfections – often the downright insanity – of how they work. I’ve met them. Through whatever path they came to this point, they don’t really give a crap about what they are asked to to do. They’ve arrived at a place where they gave up or never started trying to improve things for themselves or their organization. But I digress.
So, willingness to try is wonderful. But it is – sadly – not enough. It is like taking out gym membership. You get kudos for recognising you need to get fit and ‘well done’ for taking out the gym membership. But the real applause comes when – most importantly – you start to see actual improvements. And that takes persistence and focus.
How does the Process Delusion play out in your team or organisation?
I’ve been in the software development industry for 21 years, using agile software development practices and principles for 14 years to actually deliver software. I’ve been working with Scrum for nigh on 11 years now and over the last 7 years I’ve coached over 120 teams and worked with over 1200 people.
With all this experience one thing has become clear – Scrum is broken.
How I assess the quality of any agile process
In theory, it seems all neat and tidy, but in practice it is abused, misused and generally ineffective. And I wonder why? I have seen many more poor implementations of the Scrum framework than successful ones. Actually – let me be explicit about how I assess the quality of implemented Scrum.
For me the success of *any* organisation’s way of working – formal or otherwise – can be checked by simple questions that explore 5 fundamental concepts:
Value: Do we know the value we seek to deliver and are we consistently delivering the maximum value?
Flow: Do we understand how we reach that value and are we consistently reducing the time and/or increasing the ease by which we reach it?
Quality: Do we understand how good our product and workmanship needs to be and are we consistently and demonstrably achieving it?
Joy: Do we know what we collectively and individually need to be joyful and are we consistently meeting those needs?
Continuous Improvement: Do we know what we need to improve across VFQJ and are we demonstrably pursuing those improvements?
This simple list of questions invites great questions and other avenues of inquiry. It can function as both binary (yes/no) and range (how well are we doing). Works for me – mostly.
The non-recipe recipe
The fundamental reason why Scrum – as it currently exists – is broken is that it is a set of instructions, roles, rituals and artifacts that purports to *not* be a recipe. But almost every one takes it as a recipe and they follow it. But there is not enough there to be a an all encompassing recipe – there can never be.
The trouble is that, as a product , Scrum needs these illusions of completeness in order that it is taken seriously . By and large this has worked. In a world that is seeking silver bullets, Scrum has succeeded beyond it’s practicality commands. It is by far the leading agile process framework in use today. It is in this illusion of completeness that lies the myriad of problems.
I remember in 2008 or 2009, there was a statement about there never being ‘Scrum 2.0’ – that Scrum was complete as it is. A perfectly chiseled gem that needed nothing removed nor added. This was clearly bollocks and persisted until politics and greed spilled over in the Scrum Alliance causing a bitter split between one of the co-creators of Scrum and the rest of the Alliance. All of a sudden a Scrum 2.0 in emerged in all but name.
When people have applied Scrum as a recipe the commentary from the unsympathetic dogmatics has been of the nature of:
“Scrum shines a light on where you need to fix in your organisation”
“Scrum works, you just did it badly”
“You didn’t have buy in”
Overwhelmingly any reasonable objection to some of the design flaws – and design flaws can emerge after the fact , exposed because of a change of operating environment etc – is often met with “those people are resistant, they are against change”.
When Things Work, Its the Process – When They Don’t, It’s The People
I was first attracted to Scrum because I believed it would revolutionise the ‘world of work’.
10 years on and I’m deeply disappointed. It did not revolutionise the world of work. Instead it got gobbled up by the Death Star of Work. If it had any early promise – for me it was that it would promote unprecedented collaboration between the people focused on delivering value and the people consuming it.
It may have had some success in some organisations – but this is far from ‘changing the world of work’. A tiny street maybe, but never a world!
Of course a process doesn’t do anything – people do. But how a process is designed has a huge impact on how readily the people take to it and how easily they can use it for the intended purpose.
More often than not I see a growing perception of infallibility of a process and an overwhelming blaming of the teams and individuals who play the Scrum roles. I feel deeply sad and angry at this.
Multiple single points of failure
Scrum Master, Product Owner and Team all collaborating beautifully – each with its clear role and responsibility – what’s not to love?
Except that in *most* companies roles are so entrenched and siloed that this is the worst thing you could do – add more roles that become siloed. I hear you scream – “yes but the problem is the organisation and not Scrum” – and you are both right and wrong.
You are right because it is the organisation that is siloed and you are wrong because adding more roles when you *know* that most organisations are siloed is a dumb idea. That is like asking the morbidly obese to incorporate 3 more meals – however small – into their already dangerous diet.
Defined roles and a culture of collaboration are difficult bedfellows in any context. Especially in the context of trying to explore value through . Defined roles present an extra hurdle to overcome as we truly try and work together.
I have seen product development be disastrous for many reasons and the most common one is the single point of failure problem. Examples abound:
Overworked, overextended and ineffective Product Owners;
Scrum Masters left to single-handedly be responsible for improvement and impediments and latterly ‘coaching’ the team.
Teams that aren’t to effectively own their tools, their craft or their collaboration.
Rituals become checklists vs achieved outcomes.
If I have to walk into another planning meeting that is schedul for 4 hours because Scrum said so – with the hope that somehow the information we don’t have about the work we are expected to commit to will magically appear – I may have to shoot some unicorns.
Sprint after sprint, people do the meetings, the stand ups, the reviews etc. Scrum Masters – bless ’em – scour the communities for how to make their standups more interesting. Velocity is going up, teams are 7±2 people – things in the product backlog read like ‘As a …, I want…”. The process seems to be working perfectly.
Yet software is still sitting on the shelf. Customers are not getting anything to feedback on. Developers still don’t talk to customers.
Velocity becomes a stick to beat teams up
So velocity is not a defined part of Scrum – but it is a defacto part of Scrum implementations and it is both a arbitrary and crude measure of something.
Velocity doesn’t mean what you think it means. It is about capacity,not speed It is *always* historic and never comparable to any other team and barely comparable to the same team over time. Sure it is a number and numbers can be compared – and that in itself is a problem because the human factors that help that number pop out are deeply flawed in most organisations.
Thankfully, the #NoEstimates movement is bringing some well deserved improvement in thinking in this whole area.
Of course you need a measure to know how you’re doing. But velocity is a poor one. The emergent product and how well it is meeting the needs it was intended to meet is the ultimate measure – sadly many organisations are unable to get this – so they overdose on the measure that came with the process.
So narrow as to be quickly redundant
I was in the market for car seat a few years ago and was faced with buying something that was super specific for my son at his then age – he was 3 years old – or buying something that would support him till he was 8 years old.
Guess which I bought.
Scrum purports to be for ‘new product’ development – and I wholeheartedly agree. Except I don’t know *any* teams that only do ‘new product development’.
Even if a team that started out with only new product development – after their product reached its first users – their work mix would need to change. The fixes, bugs and so on that are different classes of work and different service responses change the mix. You could use Scrum to do these classes of work – but that would be akin to picking your teeth with a vacuum cleaner.
And this is just product related work – but what about the work everyone needs to do to continue to improve? The Human Development work? How about using Scrum for that?
If Scrum was the only thing you did – and many organisations adopt it as such – then it would get misused pretty quickly because getting shit done always trumps following a process. The question is how much of a hatchet job do we have to do to the process to get the shit done.
Scrum Shows You What Needs Fixing, It Doesn’t Fix Them For You
If I had a penny for every time I’ve heard this, I would be rather rich.
Of course Scrum should fix something – everyone I have ever met in a company *know* what is wrong. They don’t need a fancy methodology to tell them what is broken. They need something to help to try and address the problems and Scrum does not not address any of those.
If the problem that Scrum fixes is exposure of problems then it needs to be pitched as ‘a framework for exposing what is preventing you from doing joyful, fulfilling new product development work’.
It ain’t all bad.
I don’t hate Scrum – I’ve just use it long and deep enough to understand which bits I would throw away and which I would keep, what contexts I would use it in and which I would bury it so deep – no one would ever find it.
If you are using Scrum – I beg you to question it all. Ask yourself, your team, organisation and customers whether this is really helping you and what you are really trying to achieve. Don’t even mention Scrum during that conversation – simply talk about ‘how are we delivering what we seek to deliver” .
As with most things, all this can boil down to simple choices.
Do you choose to blindly follow a process or do you choose to adapt your own way of working that is focused on value and joy?
I often get to work with teams trying to ‘do’ Scrum and after about 100+ teams, I can confidently say one fundamental issue comes up again and again as a key stumbling block to how effective these teams are. In this post I’ll tell you what it is and 5 steps to addressing it. The rest is up to you.
So the fundamental issue is this:
The Scrum team does not understand the nature and type of work they are doing and so are not self-organising around it effectively.
This post might help your team if you suffer any or all of the following symptoms.
You don’t have a product backlog
You have a product backlog full of much more than items adding direct value to the product
You have a Product Owner who is struggling to navigating the crap that found its way into their backlog
Your development team is screaming that they often do not have enough work to keep some of their members busy during a sprint
You are delaying sprint starts and/or extending sprints to absorb the impact of hidden work and compromised capability
Your team is having Sprint Zeros like they were 2-for-1 at the supermarket
You are doing analysis sprints, design sprints… and so on
If you have these problems here is what I suggest you do:
#1. Understand the work you are doing:
Specifically, understand the work in terms of time and value.
First and foremost, let’s look at value:
Direct value – work to implement features or fixes that people will pay for or have paid for. I group non-critical bugs and enhancements and such like under ‘fixes’.
Capability value – work to add things that increase your capability to deliver or remove/refactor the things that are reducing your capability to deliver.
That is it. If any work does not fall into either value type, it is waste – do not do it.
This product elaboration picture illustrates this a little better:
Direct value are the green and reddish-orange items (basically that stack is your product backlog on its side) and the capability value items are the things you know you did, that narrow ‘your development team capability’ aperture or the things you know you can do, that will widen it.
Next, consider time:
Past – this is usually work to improve capability value by addressing stuff you put on the never-never. This includes known hacks, fudges, crappy code you added that “TODO” on, shoddy design, smelly tests, no tests , all those outdated comments that everyone spends an hour wading through only to find they aren’t relevant, those manual tasks you thought you would only ever have to do that one time and then you actually do them several times a day and they are long and boring. Basically anything that affects your ability to sprint effectively. This is not an exhaustive list.
Present – the ‘READY’ items that your team comprehensively knows enough about to be confident enough to commit to deliver in a sprint. Deliver as in ‘DONE” – meeting a Definition of Done that your team and your Product Owner agree is a consistently achieved level of goodness. For example it meets the conditions of acceptance, it hasn’t broken any existing value , the code is clean and generally efficient etc.
Near Future – items from the backlog for upcoming sprints that you need to create knowledge for now – basically the next reddish-orange items. If you don’t know why you should be doing this, allow me to elucidate.
See the product elaboration picture above? Basically if you don’t do work on elaboration now, your team will arrive at a future sprint planning with big reddish-orange items that won’t fit. Except you will need to commit because your management will be screaming at you and you will be too chicken shit to say no. So you’ll pretend they are green and ‘ready’ and ‘commit’ to them knowing fully well there is little hope of them getting ‘DONE’. Of course you’ll try your best, you’ll work hard when you should have worked smart. You’ll cut corners and more than likely build crap and increase your tech debt., further reducing the capability of your team.
FYI: Typically most ‘past’ work affects capability and most ‘present’ and ‘near future’ work is around direct value. Also the Product Owner owns the Product Backlog and the development team owns the Capability list.
So now that you know what to look for, you’re ready for the next step.
#2. Make work visible
Now go gather work. People are clearly busy – raid the backlog, declare a ‘hidden work’ amnesty, collate it all and make it big and visible. I prefer using post-its in the first instance. Put it all on a big wall.
Bring stuff from the product backlog, get the programmers to ante up their secret side projects, encourage testers to confess all the work they end up doing that no one gives them any credit for. Ask the rest of the organisation what others are doing that you secretly depend on every sprint but no one acknowledges.
Take the view that any work that isn’t on the list doesn’t get done – simple as that.
Invite ideas for capability work (you might need to first talk about what capabilities you need though – so do that). It might be training or having a consultant in to give an injection of skills, it might be building up specific expert knowledge. Whatever it is, get it , write it down and make it visible.
None of the lists of items is locked down – that would be denying emergence and is a recipe for undoing your improvements. Resist the temptation to get Gestapo on it. Instead, consider just making it absolutely visible and develop a cadence of reviewing it regularly.
One thing though – when you have your capability value items list up – go round and make everyone swear on the life of something precious to them that they won’t knowingly add anything that constitutes any kind of debt without stopping the line and getting everyone to agree to it. That is not to say you are going never have any more, only that you all knowingly accept to do so.
With all the work visible and normalised – you’ve removed all the duplicates and everyone is clear what each post it represents, you can move on to the next step.
#3. Prioritise the work
The Product Owner should already have the backlog items prioritised by business value anyway – if not – they need to do that. Everyone can help with that, but ultimately it is their call. The immediate goal for this is to have a backlog is at least prioritised for the next 2 sprints. At this point, you need to make a clear distinction about what is ‘ready’ and what is ‘not ready’ for a sprint.
Next, prioritise the capability value items based on how much you all generally believe they might enhance your ability to deliver over time – the stuff that will speed it up now at the top and the less impactful stuff further down the list. Crappy code that rarely gets touched – whilst crappy – might hold little value for tackling now. But don’t worry, they are on the list and will get done.
So now, all the work is in an ordered state, you are ready for step 4 – making time to do it.
#4. Make time to do the work every sprint
Now you need to figure out how you’re going to spend the time to do each type of valuable work. I recommend you do some of each type every sprint. The basic idea is, every sprint, to do a combination of now work (ready items off the backlog), elaboration (non ready , near future items , again from the product backlog) and capability improvement work. FYI – only the ‘now’ work – ready items off the product backlog count as your sprint commitment.
Without this you are going to get screwed. One or both of the following things will happen:
You will run out of ready items to work on for later sprints and return to that place where your sprints are crap, nothing gets delivered, you work long hours just to create the knowledge you needed to have to even consider committing to anything.
You keep delaying the capabilities improvements until everyone would rather drink arsenic than touch the code and you are delivering far less that you ever did. Frustration and mistrust is growing and most likely your team is shrinking because people just got fed up and found a different job.
Without getting bogged down in numbers, I suggest you start with 70/20/10 and adapt. This means figure out the time you have for the sprint and then:
Spend 70% of it working on the items you committed to get DONE in the sprint. Clearly you will need to adjust your sprint commitment too – so DO NOT commit to items based on 100% of your time and then work like the clappers to get them done in 70% of the time. That is just nuts.
Spend 20% of it working on elaborating near future ‘not ready’ work (this covers every knowledge creating task that enables your team to be able to commit to doing the near future work in sprints like UX analysis, exploring test cases, reviewing with customers, tech spikes and all that kind of stuff).
Spend 10% of it working on the capability improvements. You might only get to start them – when you are in sprint planning try to find ways to dissect the large capability improvement items into things that you can complete in a sprint to minimise branching issues and gates in the code to separate large refactoring.
The fact of the matter is, with the type of work you have to, you have to invest to do it. You will end up paying one way or another.
The preceding 4 steps are hard to do and sustain, but are essential. At your retrospectives remember to acknowledge that you got this far and celebrate too. Then seek ways to improve. Check your percentages – are they still working for you? Evaluate your capability now, are you getting the benefit you expected to get? How are people feeling about being more open with the work they previously had as covert ops?
Pay attention to what the ways the work is requiring you to self organise. Are you learning and amplifying that learning? Is everyone helping out the best way they can?
The larger the development team capability grows, that more strain on your Product Owner to make enough backlog to sustain the development team. Is it time to slow the elaboration? Or maybe the team needs to help by doing more.
Lots of ideas – but the key is to retrospect effectively. Please do so with care, respect and openness.
Great you got this far – well done! Now go do it for real and let me know how you get on.
I am profoundly grateful to all the teams I have worked with who been the experimental jets for this learning. Also all the wonderful agile practitioners who I have known and whose ideas have mingled with mine. The product elaboration funnel is inspired by the work of Jeff Patton– who rocks in so many ways. Please check out his site and say I said ‘Hi’.