Archive for the ‘Practices’ Category

• Top 25 Most Dangerous Programming Errors

  Date: 2009.01.13 | Category: Practices | Response: 0

  Jeff Atwood has posted some great stuff on some of the most dangerous mistakes you can make when programming.  Check it out.

• The problem with most TDD trainings is…

  Date: 2009.01.12 | Category: Coaching, Legacy Code, Test-Driven Development, Testing | Response: 0

  … they are too short.

  Mark Levison has posted an interesting article, Making TDD Stick, on InfoQ that covers some of my observations about how to sustain test-driven development (TDD) in an organization.  I have found the best way to teach TDD is to mentor\pair with an individual one-on-one until they “get it” (about 4 to 6 weeks).

  Unfortunately, one-on-one mentoring is not feasible or economic as bigger organizations try to improve their quality with TDD.  There are just not that many self-proclaimed TDD experts out there.  To solve this scarcity issue, often times big companies bring in a TDD expert to provide classroom training for the 40 or so developers deficient in this skill.  Once the trainer has been scheduled, the participants are rounded up, sit in a one or two day class, the consultant gets their check and then nothing happens!

  I have not seen the model of flying in a training for a one or two day training ever work.  Why?  People do not know what to do after the trainer leaves.  TDD is hard and makes highly skilled (and highly paid) professionals feel like morons.  After a day of flailing about in their IDE, most developers jettison the whole thing and walk away with the idea that TDD only works on “toy projects”.  Not surprising since that is all they saw in their training – an easy-to-understand and easy-to-teach example that fits within the confines of a day.  Little guidance or no guidance is provided on how to apply the TDD concepts back at your desk.

  What is my alternative?  Make the class longer – about two to three weeks.

  1. Week One – Basic TDD training (up to 1 day) followed by 4 days of practical application.
  2. Week Two – Advanced TDD concepts (up to 1 day) followed by 4 days of practical application.
  3. Week Three (optional) – Five days of practical application and mentoring.

  At the end of each day, I gather all the team members together for an afternoon show-and-tell of all the tests they wrote.  Everyone is required to show at least one test.  The purpose of the discussion is to learn where people are having trouble, who needs help from the trainer, what is working for the developers and allow people to express their frustrations.  The daily retrospective amplifies the learning and allows the teams members (or the trainer) to make mentoring connections among themselves.  At least you now have a fighting chance to make TDD succeed instead of people suffering alone at their desk hating life.

• Is Code Foreclosure In Your Future?

  Date: 2008.12.12 | Category: Agile Design, Quality, Refactoring | Response: 0

  I want to use this post to talk about the concept of design debt (or technical debt) to describe the variety of “shortcuts” software developers take to compromise design quality in order to gain a short term boost in productivity.  While there may be many good reasons to take these “shortcuts” at the time, from that point forward any future design changes to that section of the code are more difficult, i.e. expensive to make.  In almost all cases, most well-meaning software developers really do intend to return to the code and fix these shortcomings, but due to schedule pressures they rarely do.  The cumulative effects of these “shortcuts” are called design debt and are very toxic to your software.

  Much like credit card debt, a small amount of design debt is acceptable.  Unfortunately, due to the nature of software, if real attention is not spent “paying back” the design debt, eventually the system in question collapses under the weight of its highly coupled design.  I created the list below to help you spot the four stages of design debt in your system, so to avoid a “repo” of your software.

  1. Impulse Buy - Not doing the right thing today or having the attitude of “I’ll fix that tomorrow”.
  2. Late Payments - Adding new features to new code begins to take longer and longer.
  3. Notice of Default - The project stops adding new features to enter a “refactoring phase” or “just a week for clean-up”.
  4. Foreclosure - It simply becomes easier to re-write the system rather than modify it.

  As you get into later half of Stage Three, more and more of your resources go into paying back debt and you cannot extend the functionality of your system to keep up with market demands.  The competitive advantage your software provided the business falls to the wayside as all your time is spent in maintaining a brittle system your customers don’t even like anymore.  Like most ailments in life, the longer you wait to apply a remediation, the more “painful” the cure.  It is far more economical to apply an ounce of prevention, by taking your vitamins in the form of daily refactoring, than wait for the “foreclosure” process to overcome your business assests.

• XP c.2008

  Date: 2008.10.29 | Category: Conferences, Documentation, Extreme Programming, Pair Programming | Response: 0

  In Boston, I attended a session lead by Neal Ford from Thoughtworks.  He shared his company’s experience of applying XP in a variety of domains.  Thoughtworks almost exclusively uses XP on all their consulting engagements.  Here are some thoughts on three pieces of misinformation about the XP community.

  Pair Programming

  Pair programming continues to remain controversial, but removing it from XP cripples the process.  The main reason is because eliminating pair programming takes away one of the key feedback loops XP is built upon – review of code by a second pair of eyes as it is written.  Having developers pair program increases the concentration, focus and productivity of the team.  I know it is counter-intuitive, but pair programming is one of the main reasons why XP teams make such great improvements in productivity.  Everyone is focused either with the writing code or thinking about how to improve the design.  No one is off surfing the net, killing time on the phone or responding to email – they’re working and working hard!

  Kent Beck has clarified his position on pair programming in Extreme Programming Explained (2nd Edition) - not even Kent pair programs all the time! – and it remains very important to the process.  I feel pair programming is so tightly coupled to the identity of XP, that I would find it hard to believe a team is “really” doing XP (as if that really matters to anyone but us process nerds) if they are not pair programming.   IME, if a team says they are not doing pair programming and you dig into the details why, you normally find out they are not working in timeboxed iterations, not refactoring, don’t refactor and on-and-on.  The just took their ad hoc, undisciplined process and labeled it XP.

  Documentation

  Contrary to what people think they know about XP, good XP teams document their work.  They just tend to use light-weight documents for communicating to themselves, only preserve documents that are useful and avoid the “write-only” (think about it) documents often mandated by process police.  XP teams tend to use a wiki as a living document repository and their unit tests as a specification document built from code examples.   The wiki (probably one of the more revolutionary pieces of software ever created by XP’s “grandfather” and “midwife”, Ward Cunningham) is the place to hold user stories and the details surrounding the stories, preserve design discussions, design briefs and useful design diagrams and a place to maintain strategic and tactical decisions made.  I have seen Confluence be used very well when I worked at a large government contractor, but like anything else in XP, you need to be disciplined about maintaining it.

  Code Comments

  Code comments are a special form of documentation and are commonly thought of as a “best practice”.  Yes, we want to write comments about why we do something or capture the domain rules, but we do NOT want to write a comment telling me what the code does.  IME, if you need to write a comment in your code, then you have failed to express yourself clearly in your code.  Class, method and variable names all should be easy-to-understand and expressive.  When you read code, it should read like a blog entry, i.e. be coherent and comprehensible.  Document and comment your API, but writing JavaDoc style comments for code that will never been seen by the outside world only gets in the way of refactoring the code later.

• Extract Class & the Single Responsibility Principle

  Date: 2008.08.29 | Category: Agile Design, Extreme Programming, Legacy Code, Refactoring, Refactoring 101 | Response: 0

  Last week we talked about Extract Method and why it is so important to know and love.  This week, we are going to discuss another very important principle in object-oriented design and how Extract Class can help us build and maintain better code.

  If you are familiar with Robert Martin’s book Agile Software Development: Principles, Patterns and Practices, you will recall he recast a very powerful concept (the Single-Responsiblity Principle or SRP) know from the “old days”, aka 1979,: a class should have one, and only one, reason to change. Or as I like to add my own spin to it: a class should only do one thing. The reasoning behind the SRP is if your class has two responsibilities, invariably they become coupled (coupled code == bad) and changes in one will impair or inhibit the ability of the class to meet the other responsibilities – and this is only in the case where you have just two responsibilities.  Imagine the confusion when we have three or four responsibilities in a single class!!!

  If SRP is so important, then why are talking about Extract Class refactoring?  This is the refactoring you use when you want to apply the SRP and applying the SRP is key to writing good code, i.e. code that is testable, decoupled, maintainable, etc., etc.

  How Do I Find It?: You have one class doing the work that should be done by two.

  What Do I Do Once I Find It?: Create a new class and move the relevant fields and methods from the old class into the new class.

  Any guidelines you care to share?  I am glad that you asked because I do have a few tell tale signs that a new class is waiting to bust out of your old one:

  1. Find your big classes -> any class with more than a 1000 lines is probably doing more than one thing.
  2. Classes that have “too many” methods or fields\properties -> if class has more than 10 to 15 methods or fields\properties, you are probably missing an abstraction.
  3. Methods that don’t belong -> every class has a reason for existence, or a theme, that unifies the method and data.  Methods or data that don’t “fit in” with the theme are likely candidates for a new class.

  In an application I am working on, we have a real simple Bus class that shepherds messages between simulated hardware and simulated devices (in our case we are talking about drawers, buttons, LED, etc.).  When you examine the class you see it is about 250 lines, has three fields, a number of events around connecting and disconnecting devices and some private methods.  Looking further, you can find two methods which stick out and don’t match the theme of the class:

  45  private IBusMessage BytesToBusMessage(byte[] bytes)
  46  {
  47      // convert to string
  48      string msg = Encoding.ASCII.GetString(bytes, 0, bytes.Length);
  49      // split the msg into address and actual command
  50      string[] splits = msg.Split(new char[] { '|' });
  51      BusMessage message = new BusMessage();
  52      message.Address = long.Parse(splits[0]);
  53      message.Message = splits[1];
  54      if (splits.Length > 2)
  55      {
  56          message.SequenceNumber = int.Parse(splits[2]);
  57      }
  58      return message;
  59  }
  60
  61  public static byte[] BusMessageToBytes(IBusMessage message)
  62  {
  63      // form the message
  64      string msg = message.Address + "|" + message.Message + "|" + message.SequenceNumber;
  65      return Encoding.ASCII.GetBytes(msg);
  66  }
  67

  So we have a class which is responsible for event registration AND parsing messages.  My question is this: why in the world does a class which mostly deals with event registration have anything to do with parsing?  Why would this class even care how we parse messages back-and-forth?  It sounds like a class doing more than one thing – that is a violation of the SRP – and we will use Extract Class to apply the SRP to Bus.

  Next week, we will talk about the Move Method refactoring, which is how you extract your class from the old one.

• Extract Method – The Most Important Refactoring EVER

  Date: 2008.08.23 | Category: Extreme Programming, Legacy Code, Refactoring, Refactoring 101, Testing | Response: 0

  Coming off my last rant, I thought it might be valuable to the community if I were to blog on the various refactorings I felt software developers should be using if they consider themselves professionals. To that end, I am going to borrow the refactoring patterns from Martin Fowler’s book, but provide up-to-date examples in C# code. To start the series off, I will start with the one refactoring ever developer should know and love – Extract Method.

  How Do I Find It?: You have a code fragment that can be grouped together.

  What Do I Do Once I Found It?: Turn the fragment into a method whose name explains the purpose of the method.

  Fear not if you have never heard of this refactoring. It is so fundamental that many developers execute this pattern as a matter-of-course, but did not know the name for it; well, now you do – it is called Extract Method. IMO, mastering this refactoring is the key to understanding the later refactorings, like Extract Class, which we will discuss next week.

  So, how do you located code that is a candidate for this refactoring? Here are some guidelines I use:

  1. Long methods -> methods that exceed 20 lines are a good candidate for this refactoring since they are normally doing more than one thing.
  2. Code comments -> comments are normally a visual marker that the next lines are conceptual similar and this refactoring is about grouping conceptual similar things in methods.

  I randomly searched through our source code repository at work and in a mere 24.4 seconds I found this sample code from a utility class (Util.cs). It is a nice method that does a useful thing (removes extra decimal points from a string – who knew you needed to do that, but why not?), and it needs some help based on my guidelines above:

  44  static public string RemoveExtraDecimalPoint(string text)
  45  {
  46      string returnString = "";
  47      int countDecimal = 0;
  48
  49      foreach (char character in text)
  50      {
  51          if (character == '.')
  52          {
  53              countDecimal++;
  54          }
  55      }
  56
  57      if (countDecimal > 1)
  58      {
  59          bool first = true;
  60          foreach (char character in text)
  61          {
  62              returnString += character.ToString();
  63              int length = returnString.Length;
  64              if (character == '.')
  65              {
  66                  if (first == true)
  67                  {
  68                      first = false;
  69                  }
  70                  else
  71                  {
  72                      returnString = returnString.Remove(returnString.Length - 1, 1);
  73                  }
  74              }
  75          }
  76      }
  77      else
  78      {
  79          returnString = text;
  80      }
  81
  82      return returnString;
  83  }

  What makes this a good candidate for Extract Method is this method us 39 lines long – WAY too long for a method if our target is to create method from 5 to 10 lines long (which is the target of this refactoring). Remember, one of the main goals of Extract Method is to create small methods with intention revealing method names that read like comments. Our main focus will be to turn this long method in to a series of method calls which tell the reader what happens inside RemoveExtraDecimalPoint.

  The first step after identifying a candidate method for Extract Method is to check the unit tests for this method and make sure they run.  Surprise, surprise – this method has no tests!!  Just a polite reminder – never, ever refactor code without having either an automated unit test or integration test confirming you did not break anything.  That is called hacking.  Yes, there will be times you have to “hack” something in, but while learning how to refactor is not it.  Writing a unit test after-the-fact is called a characterization test.  Here is one of my unit tests (I actually wrote five tests) that will serve as my safety net during the refactoring:

  22  [TestMethod]
  23  public void RemoveExtraDecimalPointTest()
  24  {
  25      // setup
  26
  27      // execute
  28      string test = Utils.RemoveExtraDecimalPoint("23..45.0.");
  29
  30      // verify
  31      Assert.AreEqual("23.450", test, "Extra decimal points not removed.");
  32
  33      // teardown
  34  }

  My very first Extract Method refactoring will focus on line 47 in the original sample code.  The code below shows the new method call I made for this refactoring:

  44  static public string RemoveExtraDecimalPoint(string text)
  45  {
  46      string returnString = "";
  47      int countDecimal = Utils.CountDecimalsInString(text);
  48
  49      if (countDecimal > 1)
  50      {

  The Extract Method refactoring product a method 10 lines long and has an intention revealing name.

  77  private static int CountDecimalsInString(string text)
  78  {
  79      int countDecimal = 0;
  80      foreach (char character in text)
  81      {
  82          if (character == '.')
  83          {
  84              countDecimal++;
  85          }
  86      }
  86      return countDecimal;
  87  }

  We run all our tests, everything passes, we are fine! Time to move on to find another chunk of code to extract.  Next we take a look at line 59 from the original code and use Extract Method to make the following new method.

  89  private static string ParseExtraDecimalPoints(string text)
  90  {
  91      string returnString = "";
  92      bool first = true;
  93
  94      foreach (char character in text)
  95      {
  96          returnString += character.ToString();
  97          if (character == '.')
  98          {
  99              if (first == true)
  100             {
  101                 first = false;
  101             }
  102             else
  103             {
  104                 returnString = returnString.Remove(returnString.Length - 1, 1);
  105             }
  106         }
  107     }
  108     return returnString;
  109 }

  Again: run all the test, everything passes, we are fine!  Notice that our extracted method is a little long (20 lines), but conceptually all the code in ParseExtraDecimalPoints is related, so we are OK.  Remember, we are more interested in finding related code than following a rule of “10 lines or less”.  Even if all you can pull out is a 200 line method, then that is cool – it is a first start once you pull one chunk out, you can begin to see the smaller chunks embedded in there.

  But what does the refactored method look like?  I highlighted the addition of the two static methods – CountDecimalsInString and ParseExtraDecimalPoints – to make it easier to see our changes.

  44  static public string RemoveExtraDecimalPoint(string text)
  45  {
  46      string returnString = "";
  47      int countDecimal = Utils.CountDecimalsInString(text);
  48    
  49      if (countDecimal > 1)
  50      {          
  51          returnString = Utils.ParseExtraDecimalPoints(text);
  52      }
  53      else
  54      {
  55          returnString = text;
  56      }
  57
  58      return returnString;
  59  }

  Upon examination, we see the refactored version of RemoveExtraDecimalPoint is only 15 lines long and does just two things: counts decimal points and parses code to remove extra decimal points.  Oh, that is real easy to understand as opposed to trying to read the original 39 lines of code (take a look if you don’t believe me) with all the loops, conditionals and white space and trying to figure things out.

  And that is the point of the Extract Method refactoring.

• A Rewrite is NOT Refactoring!

  Date: 2008.08.14 | Category: Communication, Legacy Code, Refactoring | Response: 0

  Just the other day I was listening to, i.e. eavesdropping, a conversation between peers where they were trying to decide on what version of Visual Studio.NET their team should migrate to. This team was still using Visual Studio 6 (!!) and wanted to know if they should go to 2005 or 2008. During the conversation the motivation for the upgrade was discussed:

  “We want to refactor some of the old [C++] code in C#.”

  [OK, now imagine the sound of metal scraping on your fillings. It is that bad.]

  Refactoring has been in print since 1999 and has a really clear definition. I also tend to think it is one of the fundamental skills every software developer should be proficient in. Frankly, I feel if you are not refactoring your code, you are not doing your job. So, why do people in this day and age of our profession still consider a rewrite of old code in a new language refactoring? Just another example of sloppy language and, unfortunately, sloppy thinking. So, to be precise, here are the definitions of refactoring.

  refactoring (n.): a change to made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behavior.

  refactoring (v.): to restructure software by applying a series of refactorings without changing its observable behavior.

  For those of you who have stumbled upon my rant, here is a short list of things I do NOT consider refactoring:

  • adding new functionality to the code.
  • changing the design in the absence of automated tests – either unit, functional or acceptance.
  • a separate “stage” in the project you do either after, or before, the “real” work.
  • a replacement, or substitution, for a genuine software design.
  • “fixing things up while you are in there.”

• Fake Objects

  Date: 2008.08.11 | Category: Legacy Code, Testing | Response: 0

  I was writing some tests today around some legacy code (its about 2 weeks old and not written with TDD) and wanted to make sure that some method was called when the parent method was invoked.  Unfortunately, I was having some problems with getting RhinoMocks working.  Since I was getting frustrated with some “stupid” tool and not making any progress, I decided to go back to first principles to look for a simpler way to write my test.

  This was the code I was working with and I wanted to write a test that proved line 28 was called:

  22  public virtual void AcceptMessage(IBusMessage message)
  23  {
  24      if (!Power)
  25      {
  26          return;
  27      }
  28      ProcessMessage(message);
  29  }

  If you look at the rest of the code, this method comes from an abstract base class and the method in question (line 28) is abstract, protected AND void, so just calling it and looking at the return value is not a strategy I can use.  To write the other tests I had written earlier in the day I had used the Fake Object concept just so I can get the object in the testing harness.  Keep in mind, I am not trying to fake any collaborator, just test the base class since it does real things I want to make sure happen (or get updated) as the design evolves (you might be thinking this is a dumb place to start with a test, but you have to start somewhere).

  Since I have complete control over my fake object and the method I want to make sure is abstract, I can insert a sensing variable in my fake object and set its state when the method (ProcessMethod) is called, like so:

  10  public bool MessageProcessed = false;
  11
  12  protected override void ProcessMessage(IBusMessage message)
  13  {
  14      MessageProcessed = true;
  15  }

  Here is my test:

  63  [TestMethod]
  64  public void AcceptMessageWithPowerTest()
  65  {
  66      // setup
  67      FakeDevice test = new FakeDevice();
  68      test.Power = true;
  69
  70      IBusMessage message = new BusMessage();
  71                    
  72      // exercise
  73      test.AcceptMessage(message);
  74
  75      // verify
  76      Assert.IsTrue(test.MessageProcessed, "Process message not called.");
  77
  78      // clean-up
  79  }

  OK, kinda like using a sledgehammer to hit a carpet nail, but it got me unstuck and back to a green bar.  Now, maybe I can spend sometime with RhinoMocks and figure out the easier way to do this.

• Pay Attention to the Obvious

  Date: 2008.07.25 | Category: Design for Six Sigma, Practices, Voice of the Customer | Response: 0

  The other day we were talking about DfSS and how much training material we were going to produce and what sort of timelines we could commit to. I was having a hard time wrapping my head around the scope of the problem because it seemed very large and undefined. Specifically, I was thinking I had a LOT of original training I needed to create around some Agile code practices. Somewhere along the way I had missed aan important concept about my role on the DfSS rollout team. For some reason, I was under the impression that DfSS was about making better engineers.

  “Noooo. DfSS (DESIGN for Six Sigma) is about taking good engineers (people with good skills and proven engineering aptitude) and training them to become good designers.”

  While had been consistently told that DfSS is the program is about “designing quality into the system”, what I heard was “building quality into the system”. During the implementation (ostensibly the work that happens after DfSS), quality engineering practices are monitored and controlled via the scorecard. It is through the scorecard that good engineering practices like continuous integration, automated unit tests,refactoring, etc., etc. are implied to the Team. During implementation, you have to do “something” which will satisfy the scorecard, that “something” is up to you.

  Call me a bit skeptical about using the scorecard as a control mechanism for good engineering practices. I feel comfortable that the design coming from DfSS will be excellent, reflect the Voice of the Customer and be marketable, but leaving implementation as a black box to be monitored via a scorecard at phase gates ,or through periodic in-process reviews, leaves me less than satisfied. It seems to me that DfSS relies heavily on the ability of the DfSS Black Belt to bring both an excellent design AND quality engineering practices to the implementation team. In many of the environments I have encountered, there is big gap between the skills of the designers and the implementation team. It might be a bridge too far, I don’t know.

• “Unit testing never took off”

  Date: 2008.06.27 | Category: Practices, Testing | Response: 0

  … said Agitar CEO Jerry Rudisin from SD Times.

  “The practice works, but it hasn’t taken off as a mainstream practice.”
  

  In 2008, I find this quote surprising and saddening that we still do not have software developers recognizing the need for automated unit testing. Sure, a lot of people now talk a good game about unit testing (how can they not since it is talked about so much), but they don’t really buy it. I see it with the teams I work with today and teams I worked with in the past. When I was interviewing at SAIC, I specifically asked the team if they did unit testing (it was my minimal qualification for the job) and I got nods from everyone at the table. Little did I know that we were talking about completely different things: I was talking about Michael Feathers’s definition of unit testing and they were talking about a more classic definition of unit testing. It was an immediate mismatch of expectations and goes back to an earlier post on being precise in language.

  So how do we bridge the gap and help people understand the value of automated unit testing? I think the first step is to help people understand that testing is really a design activity. Writing automated unit tests is defining your API so that it may be consumed by its very first client, the test harness. I have always believed (and found through practice), that if your application is hard to test, the design is likely overly complex. Tests that are hard to set-up, or exercise, normally mean the classes are too tightly coupled. Or put another way, the classes know too much about the internal workings of other classes and your design concepts are not properly encapsulated, i.e. you have leaky abstractions.

  I understand it is hard to change the way we code, especially if we have habits that have helped us be successful for many years. However, what really has helped me crystallize the importance of writing automated unit tests and practicing TDD is a quote I heard from Alan Shalloway, and I am paraphrasing here:

  “I am putting the bugs into the system. There are no people, or gremlins, coming in at night sprinkling bugs while I am gone. It is me, so I need to do practices that stop me from putting more bugs into the system and TDD is what works for me.” [emphasis added]

  So, I guess the first thing that needs to happen is not learning testing is a design activity, but accepting responsibility we are responsible for the low quality and wanting to make things better. Without that, unit testing will never take off.