Archive for the ‘Design for Six Sigma’ Category

• Musings on (Agile) Metrics

  Date: 2010.10.18 | Category: Agile, Conferences, Design Excellence, Design for Six Sigma, Lean, Measures, Scorecards | Response: 0

  Last week I attended a really interesting session at Agile Open NoCal about what are good metrics for Agile teams.  Since I spent a lot of time thinking about this when working with Cardinal Health on their Design Excellence program, here are some thoughts I have.

  Metrics are a powerful tool to influence behavior.  They should be applied judiciously and cautiously to any problem since they ALWAYS create unintended and unexpected behavior.  The larger the organization, the more important it is for measurement since that is the way big organizations receive feedback and can adjust their behavior.  However, it is unreasonable to insist all metrics are destructive and irresponsible to expect a large business to fly blind without some sort of data.

  I tend to think, most people do not understand how to design good metrics or they create metrics that are easy to gather, i.e. using time sheets or something else similarly foolish, or they pick metrics which are valuable to the data collector, not anyone else.  I don’t claim to be an expert, but I have some ideas which I think are consistent with Lean Thinking and Agile.  There are four guiding principles I can provide:

  1. The only metrics that matter are the one tied to customer value or satisfaction.  All the rest are intermediate measurements.
  2. The people being measured should have a say in creating the measurement system on which they are judged.
  3. Apply the Lean concept of Measure Up when creating your metrics.
  4. The best metrics are temporary.  Once the metric has produced the desired outcome, discard it.

  Pascal Dennis in his book, Getting the Right Things Done (not my favorite book – long on story, short on specifics), gives some great guidance on the first principle when flushing out the concept of True North. True North is an emotional description of the strategic and philosophical goals of the organization tied to real-life, customer-focused metrics.  Not the kinds of metrics middle managers care about or software teams, but the types of metrics that show up in reports to shareholders and are on the minds of the executives and senior leaders.  These are the metrics EVERYONE in the organization needs to be focused on and thinking about every day.

  In my experience, the people doing the work know what metrics are meaningful to them and which have no impact on their day-to-day activities.  To create a powerful measurement system that gives accurate, real-time data, engage the people doing the work in creating this system.  In addition, the metrics they come up with, are the metrics on which they are evaluated.  The common objections one runs across when developing this idea with middle manager are “what if the workers don’t come up with anything?” or “what if they are not sophisticated enough?”  I feel if you explain to the workers what is the True North, the main business metrics, they will come up with meaningful metrics that tells them if their work is contributing to the objectives of the organization.  The people who work in the company want the business to succeed (and if you feel otherwise, you have a different, more serious problem).

• Prioritization with Pugh

  Date: 2009.03.03 | Category: Collaboration, Design Excellence, Design for Six Sigma, Lean, Product Owner, Pugh Concept Selection | Response: 0

  I came across a recent blog entry where Brandon Carlson was discussing one technique he used to speed up the prioritization of Product Backlog items with Product Owners.  He describes the scene like this:

  “It was late July and I was sitting through yet another agonizing prioritization meeting. The sounds of debating product stakeholders filled the air, each weighing in on his or her pet feature’s relative merit, desperately trying to inch it toward the beginning of the development queue.”

  Reading further along in the article, you learn they solved this problem by scoring each feature (on a scale of 0 to 3) how well each feature met twenty-two key attributes from the perspective of both the business and the system.  The features that scored the highest were moved to the top of the prioritization queue and the ones which scored the lowest moved to the bottom.  What broke the logjam for them was having the ability to talk about the features using data and moving away from advocacy.

  What Brandon described is essentially Pugh Concept Selection – a Design for Six Sigma tool.  Bernie Thompson gives an excellent description on his blog of using a more “pure” application of Pugh to select between different cell phone plans.  Pugh was originally used to rank various design alternatives during Analyze, but again the Lean world has yet another mature tool that the Agile community can apply in our lightweight environments.

• It’s Just Lean, No Hyphen Please

  Date: 2009.02.12 | Category: Agile, Design Excellence, Design for Six Sigma, Extreme Programming, Lean | Response: 4

  Alan Shalloway has posted an interesting blog about how Scrum is evolving in its understanding of itself much the same way Extreme Programming (XP) did in the early part of 2000.  As someone who was very much into XP at that time, I agree that XP was first defined as the original twelve practices that you always had to follow and then morphed into XP is what you do after doing all the practices – not a very useful definition for people not familiar with the process steps.

  However, the more I think about Agile and Lean, the more I become convinced that its ALL Lean and we do not need to rebrand what we do as “Lean-Agile”.  As I alluded to earlier, I believe that Agile is an instance of parallel evolution of Lean in the software industry.  Either by accident or deliberate thought, the Agile thoughtleaders of the late 1990′s came up with a manifesto for software development that is essentially Lean.  Only later when the Poppendiecks showed up (not sure when) is when many more people began to make the connections to Lean.

  So what is Agile?  How is it related to Lean?  Lean is a set of values and principles that are portable across industries.  Unfortunately, many of the Lean practices were developed for manufacturing and if we have learned anything from Scrum it is this: software development is NOT a manufacturing activity, but new product development.  As a result, many of the original Lean practices are not applicable.  Normally, we would be out of luck as practitioners went through trial-and-error to learn what Lean tools worked in software, but we got lucky.  We had an independent instance of Lean appear in our industry about 10-15 years ago and we have gone through a lot of trial-and-error to understand what works and what does not.

  IMO, Agile is the toolkit you use when you want to do Lean software development.  Many of the tools are mature, but there are also many gaps.  One that comes to mind is the weak state of product design in the Agile community.  Agile’s default position on product design is you iterate on your code until you get something the market wants – I think that might work if you have small products and\or only serving a single market segment.  For big organizations, multiyear projects that approach could be exceeding naive.  There are Lean tools out there that have gone through the trial-and-error process, so why not use them?

• Agile 2009 Submissions

  Date: 2009.01.22 | Category: Agile, Conferences, Design Excellence, Design for Six Sigma, Product Owner, Transitions | Response: 0

  Agile 2009 is accepting submissions.  This year I really feel I have something unique to bring to the larger Agile community, so I submitted two proposals - Product Design Tools for Agile Product Owners (on the Agile Product Management stage) and Charting the Agile Transition (on the Agile Adoption stage). Both build off of my recent work looking for the best tools from the Design for Six Sigma world to apply on Agile teams.  Please feel free to comment on the Agile 2009 website if you have any feedback.

• QFD & Agile Process Roll Out

  Date: 2009.01.06 | Category: Agile, Agile SD, Collaboration, Design for Six Sigma, Presentations, Scorecards, Transitions, Voice of the Customer | Response: 0

  I have been wanting to blog on my last presentation at XPSD for a while, but have been so busy with the Holiday Season – so here are my thoughts on how to use the Design for Six Sigma (DfSS) quality function deployment (QFD) tool to gather the Voice of the Customer data around creating a customized Agile process for your organization based on the issues your organization faces and deriving meaningful measures which will allow your group to monitor and track the roll out.

  Step 1 - The first thing you need to do is collect the Voice of the Customer (VOC) about what you want your new process to do.  So, to look forward, we need to look back for a little bit.  Gather all the people who will be participating in the process change and ask them some variation on these two questions:

  1. What is your biggest pain point with the current process?
  2. What is the number one thing you would change about the current process?

  To gather the VOC data, I used the Wall of Wonder - allow people to work in small groups quietly for a few minutes, capture each comment on a Post-it note and hang them up on the Wall.

  Step 2 -The next step is to group the VOC data into logical categories using an affinity diagram.  I find the best way to do this is ask the participants to get out of their seats and rearrange the items on the Wall of Wonder until everyone is satisfied with the groupings.  In DfSS parlance, each of these categories is called a user need.  A user need is simply trait the new development process must satisfy in order to be considered successful.

  After we have labeled all the groupings, I ask the participants to force rank the user needs with the most important traits of the new process receiving the highest ranking.  In my example, addressing issues surrounding requirements and adequate resources were the highest priority for the new process.  You can also rank the user needs from 1 to 5 and allow some user needs to have equal wights.

  Step 3 - Once we have ranked user needs, we enter them into the rows of the QFD chart (Room 1 in DfSS language).  Either as a big group or smaller work teams, I then ask the participants to think about what sorts of requirements the new process must have in order to meet our user needs.  This is the point where someone (usually an engineer) gives a solution to the problem and I need to remind the participants that requirements are solution agnostic.  The requirements are then listed as columns along the top of the QFD chart (Room 2).  Requirements are allowed to fulfill more than one user need, but more on that in Step 4.

  Step 4 - OK, we have our user needs which define the characteristics that our new process must provide our organization and we have requirements that fulfill those user needs.  How do we make sense of the requirements and find out which requirements are critical to the success of our new development process?  Or put in the language of DfSS, what requirements are critical-to-quality (CTQ)?

  To find the CTQ, we first need to complete the interrelation matrix between the user needs and the requirements (Room 3).  For each requirement we need to identify the degree they are related to each user need using the scale of 9 (strongly related), 3 (moderately related), 1 (weakly related) or 0 (not related).  It is best not to spend too much time discussing each of the relationships since they are a lot of boxes to fill in, so a tool like Planning Poker helps move things along.

  Step 5 - Next we rank the requirements by multiplying the interrelatedness score of each requirement with the user need ranking in each row and sum each requirement-user need product into a total at the foot of the column.  This example shows how we arrived at the total of 162 for the “on-time delivery” requirement:

    Total = [3 x 8] + [9 x 7] + [3 x 6] + [9 x 5] + [0 x 4] + [3 x 3] + [0 x 2] + [1 x 3]
          = 24 + 63 + 18 + 45 + 0 + 9 + 0 + 3
          = 162

  Repeat for each column to rank the requirements from high to low.  The requirements with the highest score are the CTQ and are the MOSTimportant requirements our process must fulfill in order for the roll out to be a success.  Or put another way, these requirements provide us the most leverage on making a successful product for our customers.  If we cannot meet these requirements, it does not matter what process we use – we have a very low probability of making a product that will satisfy our customers.

  Step 6 - The final step is to derive our measures that allow us to monitor our CTQ with a scorecard.  In order for measures to be meaningful, they need to tie to something that the organization or customer (this even better) cares about.  It is time well spent to create the measures as a group and reach consensus that the measures are indeed meaningful and what the organization chooses to evaluate itself against, otherwise people will game the scorecards and there will be no accountability.  I find it useful to have about 50% of the measures tied to the customer so they process is grounded in the market.  Too much process transformation is done for the sake of the organization and the improving the customer or the bottom line is often an afterthought.

  So where’s the beef?  Really – the word Agile has not even been mentioned up until this point.  Actually, that is the point.  A lot of Agile transitions start out backwards – we want to do Scrum\XP\whatever, so let’s find our problem.  QFD says let us examine and analyze our problem, find out what is most important for US and monitor those items with a scorecard to know if we are on track or not.  This approach is superior because once we have the QFD and the scorecard, we can then begin to think about what Agile process and\or practices that allow us to meet our CTQ and enhance our measures.

  Maybe for your organization, you have a CTQ around requirements, then perhaps you can try our user stories and see if they move your metrics?  Perhaps you have another CTQ around quality, then maybe having cross-functional team moves that metric?  The beauty of the QFD is it allows you to mix-and-match to meet your objectives, not what it says in some book.  At least now you have thoughtful information to help you tailor your Agile approach and an objective way to know if the change you are making are having the intended affect.

• Agile Control Charts

  Date: 2008.12.08 | Category: Agile, Design Excellence, Design for Six Sigma, Measures | Response: 0

  I stumbled across Agile Shrugged today and read a post on how the author experienced a lot of variation in their team velocity.   The author did not seem too concerned with all the “sawtooth” behavior of velocity chart because they felt the velocity graph was a “fairly accurate measure of our teams velocity”.  Let’s see if this qualitative assessment matches up with statistical process control, one of the Six Sigma tools.

  In Six Sigma, a common tool Green Belts and Black Belts use to monitor their processes are control charts.  The control chart is an objective measurement tool to determine if the process under inspection can reliably produce what is expected of it in the future.  By examining the patterns of the data on the control chart we can understand if the process is “in control” or “out of control”.  If the process is “out of control”, then we need to inspect the process, discover the cause of the variation, “mod” the process and monitor the changes with…a control chart.

  The diagram below is the velocity data from the team.  You can see the “sawtooth”, or oscillation, in velocity over the course of ten iterations.  The team does seem to have improved their velocity over time, but we still have an order of magnitude difference in the completed output of the team from week-to-week.  Is this significant?  Should we care about the oscillation?

  

  To make a control chart, we need to run some basic statistics on the data: compute the mean, standard deviation (σ) and determine the upper and lower spec limits (mean ± 3σ).  Here are the results for this data set:

   mean: 6.6 ideal days    upper spec limit (USL): 24 ideal days
   σ:    5.8 ideal days    lower spec limit (LSL): -10.8 ideal days

  Next, we will plot the data on a control chart showing the mean (green line), the USL (red line), 1σ and 2σ (purple lines).  I have eliminated the LSL because a negative velocity is meaningless.

  

  Now here is the interesting part: looking at the patterns.  Using the Nelson Rules, we can examine the patterns in the control chart to see if the process is “in control” or “out of control”.  Here are the rules that I think are relevant for this chart (please review all the rules if you are interested in learning more):

  1. One point is more than 3 standard deviations from the mean. Nope
  2. Fourteen (or more) points in a row alternate in direction, increasing then decreasing. Only 8 points so far; maybe something to watch out for.
  3. Four (or five) out of five points in a row are more than 1 standard deviation from the mean in the same direction.  Only 2 out of 5 data points.

  What do we find by creating and examining the control chart?  Statistical confirmation that our qualitative opinion that our planning process is indeed fine.  Great!  Keep up the good work!

• Agile Transitions Workshop

  Date: 2008.11.13 | Category: Agile, Agile SD, Design for Six Sigma, Presentations, Transitions | Response: 0

  I am quite busy with the presentations lately.  Next up is an appearance at XPSD on December 4th where I will run a workshop showing how to use some simple tools to chart and monitor an Agile transition for your organization.  This will be a hands-on session.   The tools I will be demonstrating come from my Design for Six Sigma work.   Look forward to seeing you in December!

• Pay Attention to the Obvious

  Date: 2008.07.25 | Category: Design for Six Sigma, Practices, Voice of the Customer | Response: 0

  The other day we were talking about DfSS and how much training material we were going to produce and what sort of timelines we could commit to. I was having a hard time wrapping my head around the scope of the problem because it seemed very large and undefined. Specifically, I was thinking I had a LOT of original training I needed to create around some Agile code practices. Somewhere along the way I had missed aan important concept about my role on the DfSS rollout team. For some reason, I was under the impression that DfSS was about making better engineers.

  “Noooo. DfSS (DESIGN for Six Sigma) is about taking good engineers (people with good skills and proven engineering aptitude) and training them to become good designers.”

  While had been consistently told that DfSS is the program is about “designing quality into the system”, what I heard was “building quality into the system”. During the implementation (ostensibly the work that happens after DfSS), quality engineering practices are monitored and controlled via the scorecard. It is through the scorecard that good engineering practices like continuous integration, automated unit tests,refactoring, etc., etc. are implied to the Team. During implementation, you have to do “something” which will satisfy the scorecard, that “something” is up to you.

  Call me a bit skeptical about using the scorecard as a control mechanism for good engineering practices. I feel comfortable that the design coming from DfSS will be excellent, reflect the Voice of the Customer and be marketable, but leaving implementation as a black box to be monitored via a scorecard at phase gates ,or through periodic in-process reviews, leaves me less than satisfied. It seems to me that DfSS relies heavily on the ability of the DfSS Black Belt to bring both an excellent design AND quality engineering practices to the implementation team. In many of the environments I have encountered, there is big gap between the skills of the designers and the implementation team. It might be a bridge too far, I don’t know.

• New Directions & Design for Six Sigma (DfSS)

  Date: 2008.07.17 | Category: Design for Six Sigma | Response: 0

  At my current location we develop products that are regulated by the FDA, so we must have a quality system based on CFR Title 21, Part 820. Around here, that quality system is called the Global Development Process or GDP. However, those regulations do not tell you how to build a quality product, just that you need to have a documented process and you need to follow it. Because the FDA allows companies to define their own quality process within the guidelines of Title 21, Part 820, when auditors come for an inspection, what they are looking to see if we are adhering to our own process.

  For us, the GDP is simply the minimum requirements we must follow in order to be in compliance with the FDA. While our GDP explains the deliverables we must create and when, it does not provide any guidance on how to produce these deliverables. In addition, it is my understanding that while the GDP is focused on compliance for the FDA, it does not show how to build excellent products within the GDP. I am now part of the core team which will create a set of Lean tools that will help us create excellent products that excite our customers and better help their patients. This concept is calledDesign for Six Sigma (DfSS) and has been described to me as a way to build quality into our new products. I have been selected to be on this team to provide an Agile viewpoint and create tools that will aid software engineers make more robust products and apply widely-used “best practices”.

  At first glance, DfSS sounds a lot like waterfall and I am a very Agile guy at heart. Initially, I was very skeptical about this initiative, but I have changed my thinking. This could be a very interesting avenue of study since most of my experience has been working on small teams creating small products using lightweight tools. However, big companies with complex hardware and software requirements, especially ones regulated by the FDA, need a different set of tools, tools that are a more heavyweight than I am used to.

  I am not certain that the concepts of DfSS and Agile can be combined, but what intrigues me is the emphasis on the Voice of the Customer (VOC). If we remain committed to learning the VOC and focused on what the customer’s user needs, I can see how Agile and DfSS can play nice together. How that plays out in real life remains to be seen, but I definitely have an open mind that something “bigger” than Agile is needed for big companies and DfSS might be it. Perhaps DfSS is the way big companies make products in a recognizably Agile way?