Archive for the ‘Design Excellence’ Category

• Musings on (Agile) Metrics

  Date: 2010.10.18 | Category: Agile, Conferences, Design Excellence, Design for Six Sigma, Lean, Measures, Scorecards | Response: 0

  Last week I attended a really interesting session at Agile Open NoCal about what are good metrics for Agile teams.  Since I spent a lot of time thinking about this when working with Cardinal Health on their Design Excellence program, here are some thoughts I have.

  Metrics are a powerful tool to influence behavior.  They should be applied judiciously and cautiously to any problem since they ALWAYS create unintended and unexpected behavior.  The larger the organization, the more important it is for measurement since that is the way big organizations receive feedback and can adjust their behavior.  However, it is unreasonable to insist all metrics are destructive and irresponsible to expect a large business to fly blind without some sort of data.

  I tend to think, most people do not understand how to design good metrics or they create metrics that are easy to gather, i.e. using time sheets or something else similarly foolish, or they pick metrics which are valuable to the data collector, not anyone else.  I don’t claim to be an expert, but I have some ideas which I think are consistent with Lean Thinking and Agile.  There are four guiding principles I can provide:

  1. The only metrics that matter are the one tied to customer value or satisfaction.  All the rest are intermediate measurements.
  2. The people being measured should have a say in creating the measurement system on which they are judged.
  3. Apply the Lean concept of Measure Up when creating your metrics.
  4. The best metrics are temporary.  Once the metric has produced the desired outcome, discard it.

  Pascal Dennis in his book, Getting the Right Things Done (not my favorite book – long on story, short on specifics), gives some great guidance on the first principle when flushing out the concept of True North. True North is an emotional description of the strategic and philosophical goals of the organization tied to real-life, customer-focused metrics.  Not the kinds of metrics middle managers care about or software teams, but the types of metrics that show up in reports to shareholders and are on the minds of the executives and senior leaders.  These are the metrics EVERYONE in the organization needs to be focused on and thinking about every day.

  In my experience, the people doing the work know what metrics are meaningful to them and which have no impact on their day-to-day activities.  To create a powerful measurement system that gives accurate, real-time data, engage the people doing the work in creating this system.  In addition, the metrics they come up with, are the metrics on which they are evaluated.  The common objections one runs across when developing this idea with middle manager are “what if the workers don’t come up with anything?” or “what if they are not sophisticated enough?”  I feel if you explain to the workers what is the True North, the main business metrics, they will come up with meaningful metrics that tells them if their work is contributing to the objectives of the organization.  The people who work in the company want the business to succeed (and if you feel otherwise, you have a different, more serious problem).

• New Offering – Innovation Games®

  Date: 2010.09.02 | Category: Agile, Collaboration, Design Excellence, Games, Innovation Games, Product Owner, Scrum, Tools, Voice of the Customer | Response: 1

  On May 6^th and 7^th, I attended an Innovations Games^® consultant’s class hosted by Luke Hohmann.  Innovations Games^® are collaborative games designed to help business people develop and prioritize new product ideas.  In the context of Scrum, these games are tools the Product Owner and product designers can use to engage the customers and different business stakeholders in defining the requirements for a product and thinking about product roadmap and multigenerational release plan.  Not a lot is written about the “fuzzy front-end” for Scrum teams and Innovations Games^® fill that significant gap in way that is consistent with the Scrum values and principles.

  It was quite instructive to hear about the games and how they work from Luke.  From the different case studies discussed, we really illuminated the dynamics involved with selecting the right game for problem.  In addition, a few of my misunderstandings about the purpose of the games and how they are played from reading the book were cleared up as well.  What I liked most about the class was in addition to talking about the games, we played a lot of them in the course of two days.

  1. Remember the Future (played)
  2. Prune the Product Tree (played)
  3. Speed Boat (played)
  4. Product Box (played)
  5. Buy a Feature (played)
  6. 20-20 Vision (played)
  7. Show and Tell (played)
  8. The Apprentice
  9. Start Your Day
  10. Spider Web
  11. Me and My Shadow
  12. Give Them a Hot Tub

  Below are pictures of the Product Box I created for Look Forward Consulting announcing the new service available.  I look forward to using these games more and helping Scrum teams with improving prioritization and collaboration with their customers.

  

  

• Speaking at Orlando Scrum Gathering – Mar 9th 2010

  Date: 2010.02.23 | Category: Conferences, Design Excellence, Lean, Presentations, Product Owner, Pugh Concept Selection, Scrum | Response: 0

  Finally had a few moments to write about my upcoming speaking engagement at the Orlando Scrum Gathering from March 8th to March 10th.  I am VERY excited about the chance to speak at a Scrum conference and I was lucky enough to be selected to provide two presentations in Orlando.

  1. Prioritization with Pugh – this workshop is designed to provide Product Owners a new tool to help evaluate conflicting priorities and focus the discussion on factors that matter to the business by using a Pugh matrix.  Pugh matrices come from the Lean world and are an excellent collaboration tool to resolve conflicts from conflicting stakeholders.
  2. Insights Into Scrum Illuminated by Lean – this Pecha Kucha will focus on how we can learn more about the essential elements of Scrum by looking at the Lean principles embedded natively in Scrum.

• Prioritization with Pugh

  Date: 2009.03.03 | Category: Collaboration, Design Excellence, Design for Six Sigma, Lean, Product Owner, Pugh Concept Selection | Response: 0

  I came across a recent blog entry where Brandon Carlson was discussing one technique he used to speed up the prioritization of Product Backlog items with Product Owners.  He describes the scene like this:

  “It was late July and I was sitting through yet another agonizing prioritization meeting. The sounds of debating product stakeholders filled the air, each weighing in on his or her pet feature’s relative merit, desperately trying to inch it toward the beginning of the development queue.”

  Reading further along in the article, you learn they solved this problem by scoring each feature (on a scale of 0 to 3) how well each feature met twenty-two key attributes from the perspective of both the business and the system.  The features that scored the highest were moved to the top of the prioritization queue and the ones which scored the lowest moved to the bottom.  What broke the logjam for them was having the ability to talk about the features using data and moving away from advocacy.

  What Brandon described is essentially Pugh Concept Selection – a Design for Six Sigma tool.  Bernie Thompson gives an excellent description on his blog of using a more “pure” application of Pugh to select between different cell phone plans.  Pugh was originally used to rank various design alternatives during Analyze, but again the Lean world has yet another mature tool that the Agile community can apply in our lightweight environments.

• It’s Just Lean, No Hyphen Please

  Date: 2009.02.12 | Category: Agile, Design Excellence, Design for Six Sigma, Extreme Programming, Lean | Response: 4

  Alan Shalloway has posted an interesting blog about how Scrum is evolving in its understanding of itself much the same way Extreme Programming (XP) did in the early part of 2000.  As someone who was very much into XP at that time, I agree that XP was first defined as the original twelve practices that you always had to follow and then morphed into XP is what you do after doing all the practices – not a very useful definition for people not familiar with the process steps.

  However, the more I think about Agile and Lean, the more I become convinced that its ALL Lean and we do not need to rebrand what we do as “Lean-Agile”.  As I alluded to earlier, I believe that Agile is an instance of parallel evolution of Lean in the software industry.  Either by accident or deliberate thought, the Agile thoughtleaders of the late 1990′s came up with a manifesto for software development that is essentially Lean.  Only later when the Poppendiecks showed up (not sure when) is when many more people began to make the connections to Lean.

  So what is Agile?  How is it related to Lean?  Lean is a set of values and principles that are portable across industries.  Unfortunately, many of the Lean practices were developed for manufacturing and if we have learned anything from Scrum it is this: software development is NOT a manufacturing activity, but new product development.  As a result, many of the original Lean practices are not applicable.  Normally, we would be out of luck as practitioners went through trial-and-error to learn what Lean tools worked in software, but we got lucky.  We had an independent instance of Lean appear in our industry about 10-15 years ago and we have gone through a lot of trial-and-error to understand what works and what does not.

  IMO, Agile is the toolkit you use when you want to do Lean software development.  Many of the tools are mature, but there are also many gaps.  One that comes to mind is the weak state of product design in the Agile community.  Agile’s default position on product design is you iterate on your code until you get something the market wants – I think that might work if you have small products and\or only serving a single market segment.  For big organizations, multiyear projects that approach could be exceeding naive.  There are Lean tools out there that have gone through the trial-and-error process, so why not use them?

• Agile 2009 Submissions

  Date: 2009.01.22 | Category: Agile, Conferences, Design Excellence, Design for Six Sigma, Product Owner, Transitions | Response: 0

  Agile 2009 is accepting submissions.  This year I really feel I have something unique to bring to the larger Agile community, so I submitted two proposals - Product Design Tools for Agile Product Owners (on the Agile Product Management stage) and Charting the Agile Transition (on the Agile Adoption stage). Both build off of my recent work looking for the best tools from the Design for Six Sigma world to apply on Agile teams.  Please feel free to comment on the Agile 2009 website if you have any feedback.

• QFD & Agile Process Roll Out

  Date: 2009.01.06 | Category: Agile, Agile SD, Collaboration, Design for Six Sigma, Presentations, Scorecards, Transitions, Voice of the Customer | Response: 0

  I have been wanting to blog on my last presentation at XPSD for a while, but have been so busy with the Holiday Season – so here are my thoughts on how to use the Design for Six Sigma (DfSS) quality function deployment (QFD) tool to gather the Voice of the Customer data around creating a customized Agile process for your organization based on the issues your organization faces and deriving meaningful measures which will allow your group to monitor and track the roll out.

  Step 1 - The first thing you need to do is collect the Voice of the Customer (VOC) about what you want your new process to do.  So, to look forward, we need to look back for a little bit.  Gather all the people who will be participating in the process change and ask them some variation on these two questions:

  1. What is your biggest pain point with the current process?
  2. What is the number one thing you would change about the current process?

  To gather the VOC data, I used the Wall of Wonder - allow people to work in small groups quietly for a few minutes, capture each comment on a Post-it note and hang them up on the Wall.

  Step 2 -The next step is to group the VOC data into logical categories using an affinity diagram.  I find the best way to do this is ask the participants to get out of their seats and rearrange the items on the Wall of Wonder until everyone is satisfied with the groupings.  In DfSS parlance, each of these categories is called a user need.  A user need is simply trait the new development process must satisfy in order to be considered successful.

  After we have labeled all the groupings, I ask the participants to force rank the user needs with the most important traits of the new process receiving the highest ranking.  In my example, addressing issues surrounding requirements and adequate resources were the highest priority for the new process.  You can also rank the user needs from 1 to 5 and allow some user needs to have equal wights.

  Step 3 - Once we have ranked user needs, we enter them into the rows of the QFD chart (Room 1 in DfSS language).  Either as a big group or smaller work teams, I then ask the participants to think about what sorts of requirements the new process must have in order to meet our user needs.  This is the point where someone (usually an engineer) gives a solution to the problem and I need to remind the participants that requirements are solution agnostic.  The requirements are then listed as columns along the top of the QFD chart (Room 2).  Requirements are allowed to fulfill more than one user need, but more on that in Step 4.

  Step 4 - OK, we have our user needs which define the characteristics that our new process must provide our organization and we have requirements that fulfill those user needs.  How do we make sense of the requirements and find out which requirements are critical to the success of our new development process?  Or put in the language of DfSS, what requirements are critical-to-quality (CTQ)?

  To find the CTQ, we first need to complete the interrelation matrix between the user needs and the requirements (Room 3).  For each requirement we need to identify the degree they are related to each user need using the scale of 9 (strongly related), 3 (moderately related), 1 (weakly related) or 0 (not related).  It is best not to spend too much time discussing each of the relationships since they are a lot of boxes to fill in, so a tool like Planning Poker helps move things along.

  Step 5 - Next we rank the requirements by multiplying the interrelatedness score of each requirement with the user need ranking in each row and sum each requirement-user need product into a total at the foot of the column.  This example shows how we arrived at the total of 162 for the “on-time delivery” requirement:

    Total = [3 x 8] + [9 x 7] + [3 x 6] + [9 x 5] + [0 x 4] + [3 x 3] + [0 x 2] + [1 x 3]
          = 24 + 63 + 18 + 45 + 0 + 9 + 0 + 3
          = 162

  Repeat for each column to rank the requirements from high to low.  The requirements with the highest score are the CTQ and are the MOSTimportant requirements our process must fulfill in order for the roll out to be a success.  Or put another way, these requirements provide us the most leverage on making a successful product for our customers.  If we cannot meet these requirements, it does not matter what process we use – we have a very low probability of making a product that will satisfy our customers.

  Step 6 - The final step is to derive our measures that allow us to monitor our CTQ with a scorecard.  In order for measures to be meaningful, they need to tie to something that the organization or customer (this even better) cares about.  It is time well spent to create the measures as a group and reach consensus that the measures are indeed meaningful and what the organization chooses to evaluate itself against, otherwise people will game the scorecards and there will be no accountability.  I find it useful to have about 50% of the measures tied to the customer so they process is grounded in the market.  Too much process transformation is done for the sake of the organization and the improving the customer or the bottom line is often an afterthought.

  So where’s the beef?  Really – the word Agile has not even been mentioned up until this point.  Actually, that is the point.  A lot of Agile transitions start out backwards – we want to do Scrum\XP\whatever, so let’s find our problem.  QFD says let us examine and analyze our problem, find out what is most important for US and monitor those items with a scorecard to know if we are on track or not.  This approach is superior because once we have the QFD and the scorecard, we can then begin to think about what Agile process and\or practices that allow us to meet our CTQ and enhance our measures.

  Maybe for your organization, you have a CTQ around requirements, then perhaps you can try our user stories and see if they move your metrics?  Perhaps you have another CTQ around quality, then maybe having cross-functional team moves that metric?  The beauty of the QFD is it allows you to mix-and-match to meet your objectives, not what it says in some book.  At least now you have thoughtful information to help you tailor your Agile approach and an objective way to know if the change you are making are having the intended affect.

• Agile is Lean for Software Development

  Date: 2008.12.10 | Category: Agile, Design Excellence, Lean | Response: 0

  I know – not the most profound insight or the most original.  I think I am finally beginning to understand the relationship between the two after studying all this Lean Six Sigma and Design Six Sigma stuff.  Agile is simply a co-evalent instance of the Lean concepts and principles applied to software development.   I don’t think there really is a “Lean software development” – it is just “Lean applied to software development”.

  While I respect the Poppendiecks, and they have been thinking about this WAY longer than I have, I think Lean software development misses the mark.  IMO, they have tried too hard to build the bridge from Lean manufactoring to the software world.  It seems to me that some of the subtleties of Lean are lost with the Poppendiecks’s books, Lean Software Development and Implementing Lean Software Development.

  OK, that is a serious charge to make against some of the most respected thoughtleaders on Lean software development by a guy who has, admittedly, not that much experience with Lean.   Now, it is very likely I am completely misundertanding their writings or what they are saying has evolved from their writings.   What is missing from their writings is much of the innovation the Lean Six Sigma and Design for Six Sigma world has brought to the table.  Alan Shalloway seems to be pushing the Lean envelope further than the Poppendiecks.  I think that he gets there is really no such this as Lean software development, but that it is all just Lean.

• Agile Control Charts

  Date: 2008.12.08 | Category: Agile, Design Excellence, Design for Six Sigma, Measures | Response: 0

  I stumbled across Agile Shrugged today and read a post on how the author experienced a lot of variation in their team velocity.   The author did not seem too concerned with all the “sawtooth” behavior of velocity chart because they felt the velocity graph was a “fairly accurate measure of our teams velocity”.  Let’s see if this qualitative assessment matches up with statistical process control, one of the Six Sigma tools.

  In Six Sigma, a common tool Green Belts and Black Belts use to monitor their processes are control charts.  The control chart is an objective measurement tool to determine if the process under inspection can reliably produce what is expected of it in the future.  By examining the patterns of the data on the control chart we can understand if the process is “in control” or “out of control”.  If the process is “out of control”, then we need to inspect the process, discover the cause of the variation, “mod” the process and monitor the changes with…a control chart.

  The diagram below is the velocity data from the team.  You can see the “sawtooth”, or oscillation, in velocity over the course of ten iterations.  The team does seem to have improved their velocity over time, but we still have an order of magnitude difference in the completed output of the team from week-to-week.  Is this significant?  Should we care about the oscillation?

  

  To make a control chart, we need to run some basic statistics on the data: compute the mean, standard deviation (σ) and determine the upper and lower spec limits (mean ± 3σ).  Here are the results for this data set:

   mean: 6.6 ideal days    upper spec limit (USL): 24 ideal days
   σ:    5.8 ideal days    lower spec limit (LSL): -10.8 ideal days

  Next, we will plot the data on a control chart showing the mean (green line), the USL (red line), 1σ and 2σ (purple lines).  I have eliminated the LSL because a negative velocity is meaningless.

  

  Now here is the interesting part: looking at the patterns.  Using the Nelson Rules, we can examine the patterns in the control chart to see if the process is “in control” or “out of control”.  Here are the rules that I think are relevant for this chart (please review all the rules if you are interested in learning more):

  1. One point is more than 3 standard deviations from the mean. Nope
  2. Fourteen (or more) points in a row alternate in direction, increasing then decreasing. Only 8 points so far; maybe something to watch out for.
  3. Four (or five) out of five points in a row are more than 1 standard deviation from the mean in the same direction.  Only 2 out of 5 data points.

  What do we find by creating and examining the control chart?  Statistical confirmation that our qualitative opinion that our planning process is indeed fine.  Great!  Keep up the good work!

• Function Points as an Agile Metric

  Date: 2008.12.02 | Category: Agile, Measures | Response: 0

  Dave Nicolette posted an interesting blog post about using function points (FP) as a means to compare productivity between IT projects using different processes.  I understand the concept – find a metric that speaks to the audience we are trying to communicate with that allows an apples-to-apples comparison.  However, why do I find this still an unsatisfactory answer to the productivity issue?

  I tend to think this metric would only be useful for academic types or a research paper.  Sure, it is create more evidence that “Agile is great and waterfall is bad”, and how long do we have to wait for this evidence?  One business cycle?  Since I am on a roll smashing your good idea, I also think the barrier for entry is too high for people wishing to make the case Agile would be good for their organization.  Not only do they need to understand Agile practices & techniques, they now need to become FP counters.  Now we need to start counting FP on all our IT projects?  Is this something our industry has the skill to do?  The evidence I have seen shows we do not.

  Yes, FP are “good” because ISO has made them the standard, there is clear way to count them (for the people who know) and academics (and attornies) love them.   There are real reasons why FP are not used in US business today.  I work with the ideal organization (large Fortune 500 company) for FP analysis, they are always hungry for metrics and they do not use FP.  I wonder why that is the case?  It is not due to a lack of education.  There are well-placed, knowledgable individuals aware of this metric, so there must be something else preventing FP.  We need to understand why people who know about this metric, can afford the consultants to do the analysis and still do not use the metric.  When we overcome those issues, then let’s talk about pushing this metric on Agile teams.  In the meantime, let’s stick with “basics” like throughput, takt time or use EVM (as much as I hate it – it is comprehensible to business folks).