Subordinating the Scrum Team

Agilists intuitively get that it's a bad idea to build an inventory of detailed specs way ahead of when they are going to get turned into software. Why? Because we know that as our product emerges, we are going to want to make changes to those requirements. There is a real risk that a lot of that work will end up being waste. That's a big part of why we document agile requirements as user stories... we want the placeholder now, but we don't need the details until we get closer to building that part of the product. It keeps our options open and our overhead low.

Building an inventory of requirements has a more insidious impact than just opening up the possibility of waste. Sure, we risk specifying stuff that the business might not need, but we also build in assumptions and risk that can't be validated. Everything that we define now, everything that can't be built and tested immediately after specification, represents something that we'll have to validate or mitigate somewhere down the road. Those uncertainties make it really hard to forecast when we are going to get to done. They make it tough to be predictable.

Furthermore, requirements gathering an analysis shouldn't be done in a vacuum. We know that we're going to want some help from our development team to discuss what's possible from a technology perspective. When we pull our team into meetings to discuss these things, we are taking their attention away from the task at hand. Our team is working on the highest priority features, and here we are pulling them away to help forecast lower priority requirements that may or may not ever make it into the product. It's a pretty solid formula for going slower than we have to.

Even though we may have some capacity to do the requirements work... and it would be nice to get ahead of the game a little... it is not something we generally do on agile projects. Managing all those requirements changes, tracking all those assumptions and risks, and absorbing all the productivity impact to the development team is just too great a cost. We KNOW it is going to cause us problems down the road. Any benefits we'd gain from getting ahead of the requirements work would be lost due to the overhead necessary to maintain our requirements inventory. That inventory slows us down and makes us more resistant to change.

We understand... in this kind of a scenario... that the team has to subordinate the requirements gathering process to the overall production cadence of the delivery team.

Now, let's extend this metaphor up a little and talk about our financial services company... and specifically our new hyper-productive Credit Card Processing Scrum team. With regard to the rest of the organization... our Scrum team is able to produce working software much faster than everyone else around them. The question we're dealing with is... SHOULD they produce working software much faster than everyone else around them? In this context, under this set of circumstances, I'm saying no.

I'm suggesting that the Scrum team in our complex product organization becomes very much like the requirements gathering function in our earlier example.

When the Credit Card Processing team builds software features faster than the rest of the organization can consume those features, they risk the possibility of waste and rework. They introduce assumptions and risks that have to be managed and mitigated throughout the life of the project. They will disrupt the other development teams around them and force those teams to multi-task. The other teams will be forced to pay attention to features that they won't be ready to build for another several months. All that code will slow us down and make us resistant to change.

While it is technically possible for the Credit Card Processing team to get a head of the game... and it might make them feel more productive... it might be great for their team morale... ask yourself what impact they are having on the rest of the organization. It might not be as positive as you think. So... for all the same reasons you might ask a PO or a BA to slow down writing requirements to fill the product backlog, you might want to ask your Scrum team to slow down building software until the rest of the organization can catch up.

It's counter-intuitive, but if you start thinking past the single team, it might be exactly what you need to do.

Subscribe to Leading Agile

Managing to the Constraint

There is a part of me that is a little uncomfortable with my conclusion at the end of the 'Velocity in the Enterprise' series. Using a Kanban metaphor at the team, project, and portfolio level makes sense... I also think it is the right way to look at things... but I have a hard time recommending things that I think are really, really hard to actually do... at least without providing some sort of transition state to help you get there. I want to explore another mechanism for accomplishing basically the same effect, but without having to fully re-factor your entire organization.

For the purpose of this post, we are going to make some assumptions about your organization. We'll assume that in some form or fashion you are organized around small cross functional teams. These could be either feature teams or component teams. As long as we have a team that can establish a velocity, that's sufficient for our discussion We'll also assume for the time being that each team is working in a many-to-many fashion across several projects (or some combination of projects and non-project work). In short, we have assumed ourselves into the exact environment where velocity at the enterprise level doesn't work.

What if... rather than introduce a new planning framework... and a new language around Kanban, constraints, and cycle time... we just handled things a little more implicitly?

Managing Projects

Here's what I mean... Let's say that we have a project with a backlog of 'value features'... items that will actually create value for an end-user... something our customer is willing to pay us for. What would our project goal be? Just like a team wants to establish a stable velocity, to be able to use past performance as an indicator of future performance, the project needs to be able to do this too. Our goal is to be able to reliably and consistently deliver value features. As a project team, we pull the next set of value features into the iteration. Next we break down those value features into user stories that get allocated out to the various team level backlogs.

As a project, we know that we're only going to start things we can finish. We know we aren't going to tee up work that isn't going to get done in the next iteration or so. What we'll find is that some teams will have too much work and other teams won't have enough. Those teams with too much work are our constraints... they are our critical path. How do you shorten your critical path? You apply resources or you figure out how to do things in parallel. Shortening the critical path is the only way to actually shorten the project. At the project level, our job will be to groom backlog up to the point we have consumed all the resources on our most constrained team. The availability of capacity at the constraint limits how fast we can deliver value features as a project team.

What we have (in effect) done, is create an abstraction layer between team performance and project performance. At the team level I care about velocity because I can use it as a metric for continuous improvement. I can use my velocity to predict throughout and make commitments. At the project level I am not trying to map team level performance to project performance... I have abstracted them from each other... because at the end of the day, all I REALLY care about is project level velocity... it's the project level velocity that delivers value. I will also have to pay attention to the velocity at the constraint too... but not because it is directly correlated to project performance... but because that is where I need to focus resources and improvement initiatives to get my project to go faster.

Managing Portfolios

Now here is the deal... projects need to be prioritized just like we prioritize user stories. The number one project in the queue needs to get all of the resources at the constrained team. Okay... maybe the first and second interleave a bit... but in general... the constrained team should always be working on the most important stuff. This is going to force some amount of serialization of the project portfolio... we are only going to have one or two of the most important initiatives going at any one time. We'll be able to have that conversation because we know what subset of the organization is constraining our ability to build software. Giving them more stuff to do will only slow them down.

But what about those teams that have extra capacity? We learned earlier that some of the teams are gong to be less busy than the constrained team. True... but here you have a few hard decisions to make. You can either redeploy those people to work at the constraint... or... you can give them features to build that can be delivered independent of the constraint. Here is the hard part... if you a team work to do, work that requires resources at the constraint... and the constrained resource isn't available to do the work... your taking a pretty significant risk that the team is building the wrong product or that teams will get out of sync with each other. You risk that they are creating waste and distracting your organization from it's highest priority objectives.

So what happens at the portfolio level? Just like the project velocity is abstracted from team velocity... the portfolio will establish a velocity that is independent of the team as well. At some point we'll be able to measure how many 'project points' we are able to get done a quarter. We still care about team velocity... we still care about value feature velocity... it's just that we don't count on these metrics to be directly related to portfolio performance. We are in effect applying the same principles of abstraction between individuals and teams to team and projects and ultimately projects and portfolios. Each layer in the management hierarchy has its own velocity but it is not directly related to any of the velocities of the lower order management structures.

Managing your organization by managing to the constraint allows this to work.

Velocity in the Enterprise, redux

You see... we are really applying many of the same TOC (theory of constraints) principles we talked about when we talked about enterprise Kanban, but rather than try force a whole paradigm shift... we are leveraging much of the same language and all of the same principles to actually make velocity work. The difference is that we are leveraging the management team... or the project management team... maybe even the product owner teams... to allocate (or limit) work based on knowledge of the constraint, and to stop assigning work when teams are at capacity. We are dealing with the constraint implicitly using shared understanding of the methods and principles rather than the explicit constraint of a Kanban board.

Subscribe to Leading Agile

Reuse Creates Bottlenecks

Here is something to think about. Is re-use always good? Is it always a good idea to have shared services and rely on common components? I jotted down a great quote at Oredev this week but can't seem to recall who said it. "Reuse Creates Bottlenecks". Isn't that what we have really been talking about over the past few weeks? When you have a team... or a department... or a division... or a business unit that has everything it needs to deliver... it sure does make planning easier. Why? Because you reduce dependencies on the rest of the organization. There is less coordination... there are less constraints.

But is that how we tend to think most of the time? We seem to think that we'll pull out all the stuff that is common and share it across the enterprise. That might make sense... it might be the best use of our resources... but it might create unnecessary dependencies that we're going to have to manage for the life of the product. So here is a word of caution... if you are going to move toward shared services, have you considered the impact that decision will have on your ability to be responsive to your external customers? Have you considered what you are optimizing for?

Organizations that have people in functional silos are, in a sense, reusing people. They believe that having folks with similar abilities working for the same manager is the best way to optimize their limited resources, to get the most out of their investment dollar. Agile teams know that optimizing for throughput is better and tend to rely more on specializing generalists to level variations in the need for talent over time. Reusing system components has a similar effect on the organization as resource silos... and leads to the same kind of behavior. More documentation... more up front planning... and more managing dependencies.

So... if reuse is essential... or strategic... go for it. Just understand your trade-offs. If you are doing it because you think it is going to lower costs... you might want to think about it. If you trade shared systems for increased coordination and management overhead... you might not save as much as you think. Anyway... something to think about.

Subscribe to Leading Agile