What Do I Mean by a Complex Product?

I've been rambling on for the past few years about agile in larger, more complex enterprises. Quite often in that discussion, I'll get asked to show a real life example of what I mean when I refer to a larger, more complex product. I think there are quite a few folks in our industry that have not worked on products of this scale, so I think it is a fair question. This post is going to explore this a bit.

A large, complex product is one where many products are integrated into some larger product that has some distinct value proposition. The core products are products in and of themselves, with teams and product owners and revenue projections and customers. The integrated products also have teams, and product owners, revenue projections, and customers.

The resulting matrix looks something like this:

In this case, we have several integration products... online banking, phone banking, payment processing, and remittance processing. These products leverage payment services, risk services, business intelligence, and corporate financials.

I want to reiterate... both the products identified in the rows AND the products identified in the columns have teams, product owners, revenue projections, and customers. As a company, I sell both dimensions to different customer bases and different markets.

Let's make this look a little more physical than just a matrix in an Excel spreadsheet...

... and here lies the nut of the issue with the feature team vs. component team debate. Agile tells us that we have to build a cross functional team that is made up of individuals from each of the blue boxes and maybe some folks from the orange boxes. They become a team and deliver cross-functional features... but from who's perspective? Which dimension are we accounting for? We need to account for both.

Just to give you some insight as to the environment... the technologies involved here included Java and .NET and Cobol; Web Services and Enterprise Data Warehousing and ERP systems... not to mention that there is specialized business domain knowledge held by the developers and analysts in each part of the product organization. At the very least it is a bit naive to think that we are going to mash all this up and get hyperproductivity in just a few sprints.

I am all for disruptive change... but this is the kind of disruption that results in nothing getting delivered.

So what I am saying when I talk about organizing around capabilities... or services... or components... is to build agile teams around the sub-products in the organization. You might also form agile teams around the integration the various sub-products. The challenge then becomes how you manage the enterprise portfolio so that all of the teams are in sync and constantly converging to deliver value along both dimensions of the matrix.

The reason that I think most teams don't get the value from Scrum that they expect is that they have narrowed their definition of value to something that they can control.

If I'm part of a Scrum team organized around the Credit Card Payment Processing engine... and I build a lot of great features into my product... but the business really cares about the Phone Banking System... which you are a an integral part, but not the entire thing.... what happens? Your value is not recognized by the business... and THAT is the problem. Many of us say we are systems thinkers... but what system are we thinking about.

We have to care about the flow of value... not features from our part of the product.

Note: These images were created by a good friend of mine, Brian Sondergaard, who blogs (sometimes) over at http://blog.softwarearchitecture.com. He created these pictures for a talk we did together in DC at Agile 2007 around how to use RUP to scale Scrum. The 7 or so people that came really seemed to like it ;-)

Subscribe to Leading Agile

Bye, bye PJ's... Hello Brown Bag Deli

A few months ago, my favorite coffee shop closed. This was the primary place I'd go to write this blog and where I was always the most productive writing the book. The vibe was cool and I had a great view of a nearby park. Needless to say that I was profoundly disappointed when they shut their doors.

For a while the place was empty, but a couple of weeks ago, the new owners tipped their hat and put up a sign. We were getting a place called the Brown Bag Deli. I happened by today and noticed the place was open. I'm now sitting in my favorite corner but in totally different surroundings.

It's kinda surreal, and it's sounds odd, but it feels okay. Kinda like I have my favorite writing place back. Sorry for the diversion, but given this location's role in the life of this blog... I didn't want the moment to go unnoticed.

Subscribe to Leading Agile

What if I'm Not the Constraint?

What if you are a manager that wants to do Scrum? You ask yourself if it's possible to encapsulate the entire value stream into a single Scrum team? What if you learn that the answer is no? What if you think this through even further, and discover that your team is not the constraint? Does it makes sense to even give Scrum a try?

What if give Scrum a try and are wildly successful? Your team becomes hyper-productive and potentially disrupts the overall balance of the system. Is that a good thing or a bad thing? It might be great for your team and their morale... but what about everyone else? Will everyone else benefit from your hyperproductivity... or will it actually slow them down. What did Goldratt teach us about what happens when one part of the system overproduces?

Consider this for a minute... if you are a senior leader looking to transform your organization to Scrum... where should you start? Start by figuring out how you create value, and what teams are constraining value... and pilot Scrum there. That way Scrum will be tied to something that actually helps the overall system get better at creating actual value. It's not overnight transformation, but it is a way to help deliver real value.

If you are a team that wants to do Scrum, understand your upstream processes and downstream processes. Don't produce software any faster than you can receive quality inputs from the upstream groups, or faster than your downstream customers can consume quality outputs. Building software faster than you have requirements, or faster than your software can be consumed is waste. It's might not be hyperproductive, but it is respectful of the overall system.

Ideally you want a blend of both... you want to see bottom up adoption with top down intent. What does this mean? Understand your system... understand how the system creates value... identify the constraints in your value stream... build Scrum teams around the constraints... build on your success at the single-team level to systematically spread Scrum to new teams that are focused on the newfound, value oriented constraint.

With this approach, hyperproductivity can be encouraged while we simultaneously focus on actual end-to-end value being delivered back to the business. I'm all for teams owning the entire value stream... I am all for overnight top-down, bottom-up transformations. I'm also for doing things that make sense and managing risk along the way. I'm not so big on pretending, or hoping for things to be different, without a well designed strategy for getting there.

Subscribe to Leading Agile

Vanilla Scrum and Multi-Team Value Streams

Okay... so what if my value stream isn't encapsulated within a single Scrum team? What do I do then? To some degree, I think it depends on how much of the value stream is outside the team. If external dependencies are the exception rather than the norm... or they can be easily managed and tracked... and they don't really impact the teams ability to establish a stable velocity... I might go with Vanilla Scrum and just work really hard to make sure that the dependencies were very well managed.

If external dependencies are the norm... if they can't be easily managed or tracked... if they tempt you to want to build Gantt charts or dependency spreadsheets... and they seem to impact your team's ability to establish a stable velocity or deliver value... this is where you need some serious transformation work. You really need to redesign your entire organization in such a way that the external dependencies become the exception to the rule.

I think this is where most Scrum evangelists find themselves. When they talk about cross-functional feature teams that deliver value to their customers every sprint... they are really saying that we want an environment where the entire value stream is owned by the team. Product owner has unilateral decision making authority on what... the teams owns the how... and the delighted customer gets working software every sprint.

Like Glen Alleman says over on Herding Cats... that is great work if you can get it.

Most organizations aren't there... and unless you have the ability to retool the entire organization in a way where the teams can own the entire value stream... I think unmodified Scrum is going to fail. Even if the team is successful, they become that sub-obtimatization I talked about in my last post. The good news is, we have options at our disposal that go beyond just doing Vanilla Scrum and fail trying.

I find myself, more often than not, advocating for Scrum or Kanban at the team level... and almost always Kanban at the product or enterprise value stream level. Now we can start talking about managing the organization end-to-end using a flow of value metaphor... we can identify our constraints... we can choose not to start work we can't finish... and we can apply our limited resources where they are most likely to help get value out the door faster.

Like I said... I think there is a time and a place for Vanilla Scrum. I just want us to think through the problem and really look at why so many aren't getting the value they expect from Scrum. If we need to augment what we know works in Scrum with what we are learning works in Kanban... and maybe even a little about what we already know from AUP and DSDM... why not? Maybe if we can find reliable transition patterns, maybe someday we can get to our single Scrum team utopia.

For now... I think we need to be somewhat pragmatic... meet teams and organizations where they are... and help them get to where they need to be. Most of our teams need tools beyond what Vanilla Scrum is prepared to offer.

Subscribe to Leading Agile

Will Vanilla Scrum Work for You?

In the midst of all the methodology wrangling... I've always felt that there is a time and a place for Vanilla Scrum. The problem is that most of the time, folks are giving vanilla Scrum a try when Vanilla Scrum just isn't a very good fit for their organization. So that begs the question... when is it safe to apply Vanilla Scrum in your environment?

Here is my take, borrowing a little language from the Lean/Kanban community... ask yourself, can your entire value stream be encapsulated within a single Scrum team? If there are steps in your process that happen either before your team starts, after your team starts, or you have dependencies on other teams during the development lifecycle, Vanilla Scrum probably isn't going to work.

Giving Vanilla Scrum a try without understanding your entire value stream, only results in the dev team being a local optimization in the larger enterprise. When I'm talking to clients that want to do Scrum, this is the first question I ask. If more than one team is at play... chances are I need more than Scrum to be successful.

Subscribe to Leading Agile

How Agile is Agile?

Agile ain't just any damn thing - Ron Jeffries

Here is what I want to know... when does agile stop being agile? Is agile only agile when we have 6 or 8 people sitting in one room doing pair programming? Is agile only agile when those 6 or 8 people are delivering cross functional features every week or two? Do I have to pass the Nokia test to be agile? Can agile be agile when I have to blend it with some PMI style practices? Can agile be agile when I have to do documentation to meet our corporate governance standards? Is agile agile when I blend it with RUP or Lean or Kanban? How about when I have 1000 people working across three continents, can that be agile?

Maybe the question really is... how agile is agile?

Maybe it's time to recognize that Ron is right. Agile ain't just any damn thing. Maybe we need a new term for what might be agile-like, but adapted to meet the needs of larger more complex enterprises. I am generally of the mind that agile is about quickly delivering value to our customers, getting fast feedback, being able to quickly respond to change, creating people centric organizations... one where individuals can make difference. One where we plan, but are not beholden to the plan. One where everyone is aligned toward creating the best possible outcomes. For me, it's always been more about the value I am creating than the rules I follow to create it.

So... maybe we are just learning that agile isn't as agile as we thought.

Thoughts?

Subscribe to Leading Agile

How to Build a Large Agile Organization

Okay... consider this scenario. We have a 300 person IT shop responsible for developing control systems that automate large buildings. These systems require front end developers, middleware developers, firmware developers, and hardware engineers. A feature in this system requires us to write code that touches every layer of the overall systems architecture. As it stands right now, the teams are organized around the major subcomponents within the overall solutions framework.

How does a company like this adopt agile? First inclination might be to break up the component teams and create cross-functional feature teams. Each team would have some number of front end developers, middleware developers, firmware developers, and hardware guys... right? The general idea is that any one of these cross functional feature teams would be able to write software for any part of the overall system. Everyone on the team becomes a specializing generalist.

While it's not impossible to cross train someone who specializes in UI development with someone passionate about firmware (or hardware)... but my guess is that most folks aren't up for it. My bet is that most organizations don't have the test coverage to make this kind of experiment safe. I've talked with several organizations that have given this a shot, and the change was so disruptive, they couldn't get anything done. Seriously... nothing done.

I'd like to suggest for a moment, that our goal here isn't really creating cross- functional feature teams; the goal is to build teams that have everything they need to build an increment of working software. I think that there is a difference between the two. To be agile, we need to to build teams that can stay together over time, become high performing over time, and build trust with their customers over time. They need to build working product.

We tend to assume that in order for this to happen, we need to organize teams around end-to-end features. I agree with this approach when we are talking about small teams and small products. This advice breaks down when we talk about larger teams, and larger... more complicated... more technologically diverse product lines. The agile community needs a more stable, more robust scaling metaphor for discussing these kinds of organizations.

Here is how I have started thinking about this over the past few weeks, and I am starting to think that the language works... let's start talking about building teams around those things in our organization that are least likely to change.

If we are a small product company, with a single product... the thing that doesn't change might be a product. If we are a larger product company, with a more complex product offering, I might organize teams around a somewhat static grouping of features. If we are in a really large organization, one with multiple interdependent products, where investments are not level over time... the thing we organize around might be a component or some set of services.

The thing is... it's not always the flow of features into a product that is often the least transient. Sometimes we can't build a team with enough specialists... or specializing generalists... to get the job done. When that happens... and at some scale it will happen... we want to start looking for the stuff that doesn't change. When we find the stuff that doesn't change, that is where we start building agile teams. That gives us our best shot at keeping teams together.

When you take this approach, you have to keep in mind, that there is no longer any one team that is responsible for end-to-end value delivery. Each team delivers 'done-done' work every iteration, but as an overall enterprise, you'll have to develop the capability to manage the flow of value effectively across each of your various agile capability teams. This is where agile stops giving good guidance and we need something else.

That something else is Lean/Kanban.

So think about this... build agile teams around the persistent objects in your enterprise, whatever they happen to be. Use Scrum or XP or Kanban or whatever other agile method floats your boat at the team level. The teams can become hyper-productive and have as much agile goodness as you can get out of them. To scale all this agile goodness, use a Lean/Kanban approach to manage the flow of value across teams.

So, getting back to my initial question... what's the most effective agile strategy for our large control systems company?

Organize agile teams around the front end, the middleware, the firmware, and the hardware. Create an enterprise backlog of value features that get broken down into user stories and distributed across teams. Subordinate each team's backlog to the enterprise backlog, and create a pull system, where new features only get started once the teams have capacity to work on them. Set work in process limits for each team to reduce the amount of work we can get started before we roll back up into a value feature. Invest in constrained capabilities to improve the overall flow of value across the system.

Make sense? Any questions?

Subscribe to Leading Agile

It's about time...

Subscribe to Leading Agile

Getting Started with Agile... Two of 'Who Knows'

Okay... way too big a gap between meaningful blog posts. Since it has been so long, you might want to go back and take a look at the first post in this series called "Getting Started with Agile... One of Five". There is a bit of irony here around how this post came to pass.

I did almost all of this post two weeks ago. The post got really long. There were a few parts I was still hammering out, a few parts I still am as a matter of fact. The whole post sat on the shelf waiting for the unfinished parts. This morning I decided to break the post in smaller parts and start releasing it in chunks... pretty smart huh? ;-) Amazing how sometimes it is hard to eat your own dog food!

Here goes...

At last year's Scrum Gathering in Orlando, I heard Ken Schwaber make the comment that in his opinion, over 75% of all organizations using Scrum won't get the benefit they hope from it. That is a pretty depressing perspective, especially coming from one of the guys that invented Scrum. I don't know about you, but I am pretty convinced that this agile stuff works. I've seen it work, and I've seen it work well. If we know agile works, why do we have so little confidence in our ability to make it stick in most organizations?

Personally, I am not prepared to believe that Scrum's failure rate is because people don't implement Scrum properly. That's the whole Scrum-but argument. Scrum is such a light framework, if you give it a serious go, it's pretty hard to really do it wrong. That said, I do believe that many organizations aren't prepared to fix the organizational disfunction that Scrum is going to show them. Scrum's primary benefit is that it shows you the stuff you need to fix. If you aren't willing to fix the problems, you aren't going to get the benefit.

How many companies using Scrum are really trying to transform the entire enterprise? I'd wager that most Scrum teams live within a larger, more traditional enterprise. In these organizations, structure and culture are going to trump anything that a single Scrum team does with people, process, or tools. You could have the best team members, run a perfect Scrum, have big visible charts everywhere, and I bet you still stand a chance of falling into Schwaber's 75% statistic.

For everything we do to build a high-performing agile team, the organization is going to work to pull that team a part. The pull might be intentional, led by people that are threatened by any change to the status-qou. More than likely, the organization just doesn't get it. The value delivered by the Scrum team just get's lost within the larger underperforming organization. Even though the team was effective, it just never really translated to any kind of measurable value for the larger organization.

So therein lies the problem. What does it mean to have a successful Scrum team in an organization that isn't performing very well? What do you do when Scrum helps you identify an impediment that can't be solved by the team, but no one outside the team seems to care? Scrum ends up being a local optimization within the larger enterprise value stream. Unfortunately, it doesn't matter how hyper-productive your team becomes... getting higher team velocity isn't your biggest problem.

The trick to adopting agile is to form a team around a business problem the organization actually cares about solving... one that will increase bottom line performance. And this is where the conversation get's interesting. Many of us aren't responsible for improving the whole system. We are only empowered to do our part. We can only operate within our circle of concern. Our challenge is to figure out a way to make our local improvements, but in a way that is respectful of the whole.

There are a few things I want to suggest that will help you understand how your company looks at value delivery. For the rest of this post, I want to talk about aligning your agile pilot team to something your organization actually cares about. The important thing to realize is that not every company has the same kind of goals. If we are going hook ourselves up to big business problems, we need to understand how our companies look at value... how they look at the unit of production. For most, a unit of production is not a single user story delivered by a single team.

Next post we'll talk about a few 'units of production' that I see larger organizations really care about.

Subscribe to Leading Agile

Agile Coach Camp, North Carolina, March 19-21

Are you registered for the Agile Coaches Camp this month in Durham, NC? If not, you should. I'll admit... I was a bit skeptical about the first Agile Coach Camp in Ann Arbor. The guys at VersionOne asked me if I wanted to go. I was like... fly to Michigan... on a weekend... with no agenda... no speakers... we all just get together and figure it out? Sounded like a train wreck to me, but for some reason, I went anyway.

Needless to say, I had never been to an Open Space conference...

I learned that there is something very powerful about being a room full of passionate people that are trying to make the world of software development a better place. It was a pretty amazing experience... some of the closest relationships I have in the agile community were formed at the last Agile Coach Camp. Nothing like a couple of late nights, talking about agile stuff while smoking cigars and drinking martini's, to bond a group of strangers together!

Head over the the AgileCoachCamp Wiki for more information about the camp and links to register. I have no doubt this camp will be as powerful as the last one. Seriously... you gotta be there... I'll be there... go... register now!

Subscribe to Leading Agile