Need training, coaching, or help leading an agile transformation?
email: mike@cottmeyer.com or call: 404.312.1471

Saturday, November 21, 2009

Agile... and the Pursuit of Happiness

I want to turn you guys onto a new blogger in the agile community. His name is Victor Hernandez and he blogs at http://agilehappiness.blogspot.com. Victor comes from a QA leadership background... he is here in Atlanta... and he active in the local agile community. I am anxious to hear what Victor has to say, especially as it relates to integrating the QA role into the team. His first post is excellent and you guys should all head over and check him out.


Subscribe to Leading Agile

Managing to the Constraint

There is a part of me that is a little uncomfortable with my conclusion at the end of the 'Velocity in the Enterprise' series. Using a Kanban metaphor at the team, project, and portfolio level makes sense... I also think it is the right way to look at things... but I have a hard time recommending things that I think are really, really hard to actually do... at least without providing some sort of transition state to help you get there. I want to explore another mechanism for accomplishing basically the same effect, but without having to fully re-factor your entire organization.

For the purpose of this post, we are going to make some assumptions about your organization. We'll assume that in some form or fashion you are organized around small cross functional teams. These could be either feature teams or component teams. As long as we have a team that can establish a velocity, that's sufficient for our discussion We'll also assume for the time being that each team is working in a many-to-many fashion across several projects (or some combination of projects and non-project work). In short, we have assumed ourselves into the exact environment where velocity at the enterprise level doesn't work.

What if... rather than introduce a new planning framework... and a new language around Kanban, constraints, and cycle time... we just handled things a little more implicitly?

Managing Projects

Here's what I mean... Let's say that we have a project with a backlog of 'value features'... items that will actually create value for an end-user... something our customer is willing to pay us for. What would our project goal be? Just like a team wants to establish a stable velocity, to be able to use past performance as an indicator of future performance, the project needs to be able to do this too. Our goal is to be able to reliably and consistently deliver value features. As a project team, we pull the next set of value features into the iteration. Next we break down those value features into user stories that get allocated out to the various team level backlogs.

As a project, we know that we're only going to start things we can finish. We know we aren't going to tee up work that isn't going to get done in the next iteration or so. What we'll find is that some teams will have too much work and other teams won't have enough. Those teams with too much work are our constraints... they are our critical path. How do you shorten your critical path? You apply resources or you figure out how to do things in parallel. Shortening the critical path is the only way to actually shorten the project. At the project level, our job will be to groom backlog up to the point we have consumed all the resources on our most constrained team. The availability of capacity at the constraint limits how fast we can deliver value features as a project team.

What we have (in effect) done, is create an abstraction layer between team performance and project performance. At the team level I care about velocity because I can use it as a metric for continuous improvement. I can use my velocity to predict throughout and make commitments. At the project level I am not trying to map team level performance to project performance... I have abstracted them from each other... because at the end of the day, all I REALLY care about is project level velocity... it's the project level velocity that delivers value. I will also have to pay attention to the velocity at the constraint too... but not because it is directly correlated to project performance... but because that is where I need to focus resources and improvement initiatives to get my project to go faster.

Managing Portfolios

Now here is the deal... projects need to be prioritized just like we prioritize user stories. The number one project in the queue needs to get all of the resources at the constrained team. Okay... maybe the first and second interleave a bit... but in general... the constrained team should always be working on the most important stuff. This is going to force some amount of serialization of the project portfolio... we are only going to have one or two of the most important initiatives going at any one time. We'll be able to have that conversation because we know what subset of the organization is constraining our ability to build software. Giving them more stuff to do will only slow them down.

But what about those teams that have extra capacity? We learned earlier that some of the teams are gong to be less busy than the constrained team. True... but here you have a few hard decisions to make. You can either redeploy those people to work at the constraint... or... you can give them features to build that can be delivered independent of the constraint. Here is the hard part... if you a team work to do, work that requires resources at the constraint... and the constrained resource isn't available to do the work... your taking a pretty significant risk that the team is building the wrong product or that teams will get out of sync with each other. You risk that they are creating waste and distracting your organization from it's highest priority objectives.

So what happens at the portfolio level? Just like the project velocity is abstracted from team velocity... the portfolio will establish a velocity that is independent of the team as well. At some point we'll be able to measure how many 'project points' we are able to get done a quarter. We still care about team velocity... we still care about value feature velocity... it's just that we don't count on these metrics to be directly related to portfolio performance. We are in effect applying the same principles of abstraction between individuals and teams to team and projects and ultimately projects and portfolios. Each layer in the management hierarchy has its own velocity but it is not directly related to any of the velocities of the lower order management structures.

Managing your organization by managing to the constraint allows this to work.

Velocity in the Enterprise, redux

You see... we are really applying many of the same TOC (theory of constraints) principles we talked about when we talked about enterprise Kanban, but rather than try force a whole paradigm shift... we are leveraging much of the same language and all of the same principles to actually make velocity work. The difference is that we are leveraging the management team... or the project management team... maybe even the product owner teams... to allocate (or limit) work based on knowledge of the constraint, and to stop assigning work when teams are at capacity. We are dealing with the constraint implicitly using shared understanding of the methods and principles rather than the explicit constraint of a Kanban board.


Subscribe to Leading Agile

Thursday, November 12, 2009

What We Call Stuff Matters...

In my post 'Organizational Myopia' we talked about an increasingly common problem with the language we use. We 'talk' quite often like we are delivering a cross-functional end-to-end feature of the overall system, when in reality... we are really only delivering a subset of the solution. We are delivering a 'feature' that has to be integrated with other 'features' in order to deliver real customer value. Why does this matter? It matters because when we allow ourselves to only care about... only track progress on... only get better at our little slice of the world... we risk investing money and time building 'features' that might never get sold to an actual customer. Focusing on end-user value is critical to overall organizational success.

Part of our problem is that the language we use to describe value is ambiguous... it's open to interpretation. Exactly what is a user story? We can go with the Ron Jefferies definition... a card, a conversation, and a confirmation. What is Ron saying? A user story is a placeholder for a future conversation about the requirement. That conversation is supposed to be with the 'customer' and we've already discussed that a 'product owner' is not necessarily our end 'customer'. What about Bill Wake and the INVEST model? A story should be independent, negotiable, valuable, estimate-able, small, and testable. This definition is more specific as long as we are clear about what adds value. Valuable to the customer? Maybe... but again... who is the customer?

Back in my project management days... we were blending methodologies all over the place and had to get really specific about the language we used. People were learning and we couldn't take anything for granted. The default corporate methodology was RUP so we talked a bunch about use cases and I spent a lot of time explaining how to break down requirements in that language. During that time we had the notion of a use case, a use case scenario, and a system level use case. Use cases were an umbrella description... an aggregate maybe... of some number of use case scenarios. Each use case would have a primary success scenario and some number of alternate scenarios. A single success scenario and some number of alternate scenarios were typically the minimal increment of business value.

Our applications were complex and quite often any given use case scenario would touch several significant system components. We might have a use case scenario that hit the mainframe transaction processing engine, the enterprise data warehouse, a statement processing engine, a set of web-services, a message bus, and a java based customizable front-end. Each component would have a series of system level use cases that expressed the changes that had to be made for each of our components. Whereas the 'actor' for the use case or use case scenario was likely an end-user, the 'actor' for the system level use case was likely another system. It was only by building out a number of system level use cases that I would have a scenario, and only some number of scenarios before I had a 'feature' that was relevant to an end-user.

Given this as our backdrop... where does a user story live? Is a user story a use case? How about a use case scenario? Could it be a system level use case? I think that in practice most folks would agree that a top level use case is too big for a user story. A top level use case is more like an epic or a feature group. I tend to think of a user story more like a use case scenario. There is a primary scenario, and some number of alternate scenarios, that each individually provide some capability to an end-user of the system. We can prioritize these items, decide what is minimally marketable (how much error handling for example) and be potentially shippable when we are done. Sound like user stories to me. Lately though... and this is the nut of the discussion... I've talked with lots of teams that are using the idea of a system level use case as the definition of a user story.

System level use cases ARE an increment of working software. They do have 'value' in the sense that they are meaningful to the overall system. They are potentially shippable. The problem is that your customer doesn't care about them. Another problem is that a team's velocity delivering these system level use cases is not in any way related to your ability to deliver an end-to-end use case scenario. So... here we have the language problem. If my backlog is filled with system level use cases... but I talk about these use cases as if I were delivering a use case scenario... and measure my performance like I am delivering a use case scenario...then I am only pretending that I am delivering value and my team based metrics are pretty much crap. Many, many teams are starving their organizations... not delivering real value to the business... but think they are doing great agile.

That is a problem.

So... a few posts ago... when we were talking about program level Kanban, project level Kanban, and team level Kanban... this is what we were talking about. We might have any number program level use case... small projects moving its way through the system. These use cases are broken down into any number of use case scenarios and then ultimately system level use cases [that are assigned to teams]. The teams are responsible to deliver the increment of software... the enterprise is responsible for prioritizing that work, planning that work, sequencing that work, and making sure that the work comes together into some sort of aggregated deliverable. If the work of the team is not ever aggregated... in a systematic, disciplined way... we run the risk of having great team velocity and really poor organizational velocity. We need both.

The main takeaway here is that language matters. What we call stuff matters. Thinking about the entire enterprise like and end-to-end system matters. When we allow organizational myopia... when we bend the words to make things work... when we play games with language, we are going to have problems. You need to know what these words mean for your organization... how all of this works together... and what it takes to ensure we are driving real value for the business.

Subscribe to Leading Agile

Monday, November 9, 2009

Reuse Creates Bottlenecks

Here is something to think about. Is re-use always good? Is it always a good idea to have shared services and rely on common components? I jotted down a great quote at Oredev this week but can't seem to recall who said it. "Reuse Creates Bottlenecks". Isn't that what we have really been talking about over the past few weeks? When you have a team... or a department... or a division... or a business unit that has everything it needs to deliver... it sure does make planning easier. Why? Because you reduce dependencies on the rest of the organization. There is less coordination... there are less constraints.

But is that how we tend to think most of the time? We seem to think that we'll pull out all the stuff that is common and share it across the enterprise. That might make sense... it might be the best use of our resources... but it might create unnecessary dependencies that we're going to have to manage for the life of the product. So here is a word of caution... if you are going to move toward shared services, have you considered the impact that decision will have on your ability to be responsive to your external customers? Have you considered what you are optimizing for?

Organizations that have people in functional silos are, in a sense, reusing people. They believe that having folks with similar abilities working for the same manager is the best way to optimize their limited resources, to get the most out of their investment dollar. Agile teams know that optimizing for throughput is better and tend to rely more on specializing generalists to level variations in the need for talent over time. Reusing system components has a similar effect on the organization as resource silos... and leads to the same kind of behavior. More documentation... more up front planning... and more managing dependencies.

So... if reuse is essential... or strategic... go for it. Just understand your trade-offs. If you are doing it because you think it is going to lower costs... you might want to think about it. If you trade shared systems for increased coordination and management overhead... you might not save as much as you think. Anyway... something to think about.

Subscribe to Leading Agile

Sunday, November 8, 2009

Interesting Post... 11/1/2009 through 11/8/009

Got a interesting story for you guys...

Arrived in the Copenhagen airport today for my 11:10 AM flight back to Atlanta. I looked up on the board... no flight to Atlanta. I went over to the Information Desk and the lady tells me that there was a flight on the 7th... and another scheduled for the 9th... but no flight on the 8th. Hmmmmmmmm. Since Delta does not have local gate agents, we head over to the servicing organization that handles Delta ticketing. Apparently, Delta has decided that they aren't going to fly direct from Copenhagen to Atlanta on Sunday anymore. Kimi and I are booked on the Monday flight.

Well... we had a VERY helpful agent in Copenhagen that called and talked to a VERY UNHELPFUL agent in Atlanta. I am starting to see a CLEAR pattern with Delta phone agents. Apparently, there was only one other flight back to Atlanta today , and unfortunately the gate for that flight had just closed. Delta was willing to move us to a SkyTeam partner, but not to another airline to get us home as planned. Come to find out, the agent in Atlanta was wrong and there WAS another flight back to Atlanta, through Paris, that was going to leave in a few hours. Our agent in Copenhagen got us on that flight. This guy was my hero today... he was seriously dedicated to providing outstanding customer service.

Here is the lesson learned. About six weeks ago, I did get an email from Delta telling me that my fight had been changed. I get these messages all the time, but up until this point they had only communicated small schedule changes or seat assignments. I didn't catch that they had canceled my flight and moved me to an airplane on an entirely different day. I am not sure if I should be mad at Delta or mad at myself for not paying more attention. I guess I'll take responsibility and be mad at myself. Kimi and I are going to be a few hours late back to Atlanta tonight, but I am very happy to be going home.

With all that drama as a backdrop... this weeks installment of 'Interesting Post' is coming to you from the Startbuck's in CPH Terminal 3. Hope you guys enjoy!

Dean Leffingwell's User Story Primer http://bit.ly/2SoOT6

Oredev 2009 - LIVE (now recorded) Closing Panel Video http://bit.ly/pJ3mB

Fun with Release Planning http://bit.ly/3TOHhC

Technical Debt http://bit.ly/2pzmoB

The "decline" of Agile is our own fault... http://bit.ly/2lQU27

The mistake most developers and development organizations make http://bit.ly/3uvTQH

Why Agile? Why Now? http://bit.ly/opmxS

Five Myths of Agile Development: Myth #4 - Agile Development Does Not Scale http://bit.ly/2m1LJR

Balancing Anticipation and Adaptation http://bit.ly/3c9hSN

Agile Release Planning Musings http://bit.ly/175Ydu

Agile isn't Just About Change http://bit.ly/482noY

Making Agile Requirements Work-No 4 http://bit.ly/35lGAz

Seth Godin Misunderstands Dunbar's Number http://bit.ly/32knNw

Agile Projects and Project Management http://bit.ly/4vEuCK

Subscribe to Leading Agile

Thursday, November 5, 2009

Organizational Myopia?

Okay... I want to write a quick little post here to point out something that is becoming increasingly obvious to me. I've talked quite a bit over the past year about the ongoing feature team/component team debate. We talk about how agile teams are supposed to deliver an end-to-end cross-functional slice of the application architecture. When applications get big, I've talked about how sometimes we have to organize around large scale system components or services. So here is the deal... I think that in practice... most people actually get this. It seems that the more I talk to people doing agile in large organizations... the component team model might be THE primary organizational model in the enterprise.

Great right... no debate? Well... actually no. The problem is in the language we are using. If we are only building component features... software that is theoretically 'valuable' to the business... we are not building software that is sellable in and of itself. It is only the integration of multiple services or components that our external customers really care about. To some degree we are deluding ourselves about what we are really creating for the business. We get better at building our feature subset... our component features... but as an enterprise we are not getting any better at getting value out of the overall system. We are not getting better at delivering the end-to-end apps that our customers care about.

I think that to some degree we are guilty of a sort of institutional myopia. Because we can't control anything outside of our team... because we don't own, or have influence over, the PMO... over management... over the architecture group... over the integrated feature set... we control what we can control... we pretend the rest doesn't exist. We focus on our team. We desperately want to be good at what we do and we want to deliver value. We want to do valuable work. So what do we do? We change the rules. We redefine value and call it a backlog item. We redefine our customer and call him a product owner. We say that we are delivering working software that our 'customer' wants... but at the end of the day... we are just delivering a piece of the overall puzzle.

Subscribe to Leading Agile

Monday, November 2, 2009

Velocity in the Enterprise, Part 7

Okay... we've been going on a while now with this whole Velocity in the Enterprise series. Let's see if we can get things wrapped up with one final post. Last time around we talked about the idea of extending the Kanban metaphor to the enterprise. Just like agile user stories and teams are extended into agile projects and agile project portfolios... Kanban can be extended past the team to drive the flow of value across multiple teams working in concert to deliver multiple projects. Let's explore this a little further... starting with the team... then considering the project... and then looking at the project portfolio. We'll wrap by looking at cycle time and the role of management in enabling the organization to deliver.

The Kanban Team

We've already talked a bit about the Kanban team explicitly defining the states associated with getting a user story from backlog to done. These states are represented on a board... work in process limits are set for each state... and the stories move from state to state until they are accepted by the customer. When work gets backed up at any one process step... either due to an impediment or a constrained resource... the team has to deal with the underlying issue in order to get back to work delivering value. The work in process limits ensure that intermediate work-products get finished before new work comes into the system.

The Kanban Project

Now imagine a project that involves several agile teams all working toward a common set of requirements. Remember... our scenario here is that these teams are in some way dependent on each other to deliver an end to end feature. When teams must work together, it is important that one team not get too far ahead of the others before value gets delivered from the entire system. Imagine that the overall project has a Kanban board... one where the steps required to deliver the value feature include not only the normal design, develop, and test activities... but also include columns for the work delivered by each of the individual project teams.

The teams themselves limit the number of user stories they can have in play before and end-to-end feature is delivered on behalf of the overall project. Likewise, the project limits the number of value stories that can be in play at any one time. Let's be explicit... this can create a scenario where some teams may have to stop work while other teams on the project catch up. This forces the system to deal with the team that is going too slow before they can move forward. We need to focus our resources on speeding up the constraint rather than building inventory of partially competed value features.

The Kanban Portfolio

So... we have suggested that teams limit the number of user stories that can be in play at any one time. We have also suggested that projects limit the number of value features that can be in play at any one time. In a Kanban portfolio... we introduce the idea that the organization limits the number of projects that can be at play at any one time. The Kanban board for projects might have columns like initiation, iteration 0, release planning, execution, and deploy... each with their own work in process limits. These limits inherently result in fewer projects active across the organization... forcing the PMO to get projects finished before new projects can be started. Projects can't be started unless there are sufficient resources available to actually finish.

Cycle Time instead of Velocity

If we look at each level of the organization having only a limited number of work items (user stories, value stories, or projects) that can be in play at any one time... understanding that we are going to intentionally slow down some teams to speed the performance of others... velocity isn't going to be our ultimate measure of throughput. What we are going to start measuring... the metric we are really going to care about... is the amount of time it takes to get any single work item through the system. The team will have a cycle time... the project will have a cycle time... and the portfolio will have a cycle time. The question isn't how we can increase velocity... the question is how we can decrease the amount of time it takes us to deliver value back to the business.

Managing to the Constraint

Now we have a system that is laser focused on the delivery of value... not just at the team level... but across the entire project portfolio. No one team can get too far building user stories before they have to aggregate into a value story. No single project can go on too long with insufficient resources before it gets stopped in its tracks. The portfolio cannot keep approving projects that the organization does not have the capability to deliver. As managers, our role becomes understanding the flow of value across our organizations... understanding where that value is constrained... and redeploying people or teams to do the work that is most likely to actually speed up our ability to produce working software. Only by applying people and money to those areas that are slowing us down can we really get better at delivering faster.

In closing...

Okay... I think we have hit all the major points I wanted to address with this series. We've talked about where velocity works and in what kinds of organizations it is an effective measure. We've also explored where velocity doesn't work and where it can't be used as a measure of overall enterprise performance. With some of the Kanban, cycle time, work-in-process limit discussions, we've proposed an alternative to velocity that will help us measure our overall ability to deliver value.

I am concerned that we still have a problem in that this whole Lean, value stream analysis... this whole manage to the constraint idea is a pretty drastic shift in thinking about how to manage teams and projects. It really asks us to take a whole new approach to planning and delivery in our organizations. The more I think about it... I want to do one more post... an epilogue if you will... to maybe bridge some of this thinking with conventional wisdom to see if we can make a compromise that is easier to understand and implement. Either way... this is not the end of the conversation around this. I sincerely believe that Lean is the secret to scaling agile in the enterprise... but you know... we have to make this stuff simple and easy to understand.

Thanks for hanging in with me as I explored these ideas. As always... open to your feedback!

Subscribe to Leading Agile