I’ve been having a conversation with some folks on Twitter about their evidence that mob programming makes Brooks’ Law irrelevant. Someone offered evidence based upon his own experience that mob programming breaks this rule.  I hope I’m not mischaracterizing his argument. That is,  I don’t think he’s saying that mob programming negates Brooks’ Law–I think he’s saying that multiple programmers can work faster via mob programming.

I had pointed out that while his experience is certainly evidence it’s anecdotal evidence.  This is not to belittle the importance of anecdotal evidence. It is to make a distinction between experimental evidence and anecdotal.  I’ll relate an example that I hope will help to clarify my thinking on this point.

I’ve raised domesticated pigeons for a good portion of my life.  I know lots of folks who race their pigeons (I do not).  I was told that one of the old sachems of pigeon breeding, a gentleman named Joe Quinn, was trying to convince breeders to vaccinate their birds.  He was having a hard time convincing the racers to vaccinate their birds and as a result, flocks were getting hit with easily preventable diseases.  So he hit upon an idea.  He convinced one of the better breeders to let him (Joe) vaccinate his flock at no charge.  The breeder didn’t mind since it was free.  At the end of the season the other fanciers asked the master breeder what was his secret of winning races.  The master breeder said that he did pretty much what everyone else did (that is, training, selective breeding etc.) but he did get his flock vaccinated.  The next season all of the fanciers wanted to get their flock vaccinated.

This is what I mean by anecdotal evidence. Did the vaccinations make a difference?  Of course they did–they protected the birds against preventable diseases. But did the vaccinations make a difference in the races?  That’s more debatable.

Experimental evidence would be vaccinating half of the flock and leaving the other half unvaccinated and then comparing how the birds raced at the end of the season.  Of course everything else would need to be held equal–equal diet, equal training, equal housing etc.  The result would be experimental evidence that the vaccination either made the birds race faster or not.

As it stands however, saying vaccinating the flock made the birds race faster is anecdotal evidence.  Heck we might even be able to prove it had no effect by looking at the race times pre-vaccination and post-vaccination–although there are likely to be other complicating factors present.

The thing is it’s really hard to do a double-blind study this way when it comes to software development because there are just too many other factors to try to hold equal.  If one team uses mob programming and they’re more productive–well was it the mob programming that made them more productive or was it that they had an easier task to accomplish or was it the fact that the team had long experience together or so forth and so on.

So we’re mostly stuck with anecdotal evidence when it comes to software development because producing experimental evidence is so difficult.  As I said on Twitter I’m not doubting the anecdotal evidence–I’m just doubting that mob programming is the sole difference.  The fact, all by itself,  that a team is willing to try mob programming probably indicates a team more receptive to trying new ideas and hence possibly more productive for that reason alone.  I’m not doubting that mob programming is beneficial; while I don’t have experimental evidence to prove it, I also believe it’s a good idea.