Left Side vs. Right Side Agile

We were discussing agile today and I came up with something that I thought might be worth sharing with others.

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

And I was saying to some of the folks on my team that it seems as if we’re getting right side agile and not left side.  While lots of firms these days are calling themselves agile they’re still very much on the right side of that list. As someone put it to me, lots of places are doing waterfall with daily stand ups and two week sprints.

So maybe we need to start calling out our managers and our VP’s on right side agile and tell them that we favor the left side and we want them to favor it too.

Premature Generalization

I’m sure most developers reading this post will be familiar with the term “premature optimization”. For those who may not be familiar, this refers to the tendency that some developers have to worrying about the performance of their code, usually but not always in terms of execution time, before their code is proven to work correctly in terms of customer requirements. It’s like saying, “Hmm, this function call may take 3 clock cycles to execute but if I inline the code it won’t take any extra clock cycles” before the developer has correctly defined the function to be called. It’s generally considered a poor way to write software. I believe it was probably Joe Armstrong, one of the co-creators of Erlang, who said “Make it right, make it fast, then make it pretty.” Getting the code right always comes before any sort of optimization should be attempted.

Risking the perils that may befall anyone who coins a neologism, I propose another premature condition which developers should try to avoid: premature generalization. I would define premature generalization as being the tendency to assume that what you see or what you want is what every other developer sees and/or wants. It’s the tendency of the novice developer to post messages like “Hey function x isn’t working. What’s wrong?” assuming that the rest of us also have the problem they have and we therefore don’t need any more detail to know what he or she is talking about. It’s also the tendency to assume that because they need to address some particular use case that most developers working in the language will need to address the same use case: “Hey I need library function x to return an error status when I pass it a bad argument but throw an exception when I pass it a good but out of range argument. Can we modify the standard library to do this?” In both cases, I believe they’re prematurely assuming their question is generic and therefore they don’t need to provide details to others.

Although I feel I’m flogging a dead equine I hasten to point out that as software developers we should usually proceed from the assumption that the error we’re seeing is due to something specific to us. We should also proceed from the assumption that any use case we have is likely to be a use case specific to us until we see evidence otherwise. Yes, I am aware of the irony of making generic statements here—but given what I’ve seen over 25+ years of software development I don’t believe that I’m indulging myself in premature generalization.

Athena Code

One of the perverse side effects of code that’s “the simplest thing that could possibly work” is that it can appear much easier to create than it actually was.  I refer to this as Athena Code because it can appear as if the code sprung full-grown and ready to go from the head of the developer that created it.  In my experience as a developer this almost never happens and it can give developers the false impression that they should be able to spit out simple solutions on their first try.

A Specific Example

Recently I had a need to generate a list of all dates between a given start date and an end date in Elixir. This is the code that I actually ended up with:

defp generate_all_valid_dates_in_range(start_date, end_date) when start_date <= end_date do (:calendar.date_to_gregorian_days(start_date) .. :calendar.date_to_gregorian_days(end_date)) |> Enum.map (&:calendar.gregorian_days_to_date/1)

#And here's how it would work.
#iex(4)> generate_all_valid_dates_in_range({2012,1,1},{2012,2,1})
#[{2012, 1, 1}, {2012, 1, 2}, {2012, 1, 3}, {2012, 1, 4}, {2012, 1, 5},
# {2012, 1, 6}, {2012, 1, 7}, {2012, 1, 8}, {2012, 1, 9}, {2012, 1, 10},
# {2012, 1, 11}, {2012, 1, 12}, {2012, 1, 13}, {2012, 1, 14}, {2012, 1, 15},
# {2012, 1, 16}, {2012, 1, 17}, {2012, 1, 18}, {2012, 1, 19}, {2012, 1, 20},
# {2012, 1, 21}, {2012, 1, 22}, {2012, 1, 23}, {2012, 1, 24}, {2012, 1, 25},
# {2012, 1, 26}, {2012, 1, 27}, {2012, 1, 28}, {2012, 1, 29}, {2012, 1, 30},
# {2012, 1, 31}, {2012, 2, 1}]

I have deliberately omitted comments so the simplicity of the code is, hopefully, a bit more apparent. For those that don’t read Elixir, basically I take the start and end date and I convert them to Gregorian dates. I then create a range from the two Gregorian dates and pass the range as the first argument to the Enum.map function which follows.  As those of you familiar with functional programming will guess, the map takes each element in the list and performs the specified function (in this case the Erlang stdlib calendar module gregorian days to date function) on it, creating a new list with the result.  {yyyy,mm,dd} is how one specifies a date literal in both Erlang and Elixir.

By the way, many thanks to the folks on the Elixir Lang Talk mailing list for helping me to simplify my first pass at this code.

Now some may argue that that code above while simple is not particularly robust.  What happens if someone passes a bad date?  Well, that’s the Erlang way–fail fast.  If someone passes a bad date, the code will crash as soon as it’s executed. And it wouldn’t be hard to add a couple of lines of code to validate good inputs and it wouldn’t detract from the basic simplicity of the code.

Try Number 4

The point I’m trying to make is while that code seems obvious (I hope) it’s far from the first code I came up with to solve my problem.  It was actually try number 4.  I won’t share the code from tries 1 through 3 mainly because like any developer I want people to think I’m brilliant so I don’t want to keep failed experiments around.  But in general terms this is what I tried:

First try:

Add one day to the first date

Is the resulting date equal to the end date?

Yes -> Stop

No -> Add the result to the output list and loop to the top.

There was nothing particularly wrong with this approach–the code was just a lot more complicated than that sounds.

Second try:

Use a comprehension to try to generate the list

Got nowhere with this approach at all.  Totally failed idea.

Third try:

Modify the comprehension from try 2.

Again, total failure.

Fourth try:

Actually looked like this:

def generate_all_valid_dates_in_range(start_date, end_date) when start_date <= end_date do 
  (:calendar.date_to_gregorian_days(start_date) .. :calendar.date_to_gregorian_days(end_date)) 
  |> Enum.to_list
  |> Enum.map (&(:calendar.gregorian_days_to_date(&1)))

As I say, my fellow developers on the Elixir Talk mailing list helped me to make my code even simpler yet by pointing out that the Enum.to_list was redundant and that I could directly invoke the :calendar.gregorian_days_to_date function without having to wrap it in a lambda.

My point is this: it’s almost never easy to create “the simplest thing that could possibly work”.  Simplicity is extremely hard to create.  Almost never will your first pass be the best answer.  Don’t beat yourself up if you can’t manage “Athena Code”; almost no one can.  And don’t look at someone else’s code which is a monument to simplicity and think that that’s the way it looked when they first wrote it.  Our job as software engineers can be hard enough without putting unrealistic expectations on ourselves.

I Am Tired Of Swimming Against The Current

I have been a vocal advocate of F# for a long time. I’ve shared some of my little interesting discoveries in the language from time to time. I’ve been trying to help other .Net developers to see what a great, cool language F# is. I’ve tried to add intelligent conversation to the world although I have a bad habit of speaking before I think; I’ve been holding off on saying this for a while exactly because I do have a bad habit of speaking before I think. But this has been coming for a while.

I am just tired of seeing such a great technology being bungled by Microsoft.  I don’t know what the internal politics are and I don’t much care.  This has been coming for a while for me and today I finally reached the breaking point.

It started when I saw that the C# team rebuilt async functionality in C# 5.  I had always been told that Microsoft’s position was to encourage people to use C# where it made sense and to steer people to F# where F# made better sense.  The addition of async functionality to C# 5 shows that is simply a lie.  No matter what they do to C#, async will be easier to code in F#.  Yet rather than encouraging people to build their async code in F# and then link the libraries, the decision was made to build async into C#.  That disturbed me but at the time I thought to myself–oh well.  Now I am beginning to wonder if F# is being used as a way of prototyping ideas that Microsoft wants to pull into C#.

Then there were the conferences.  I went to LambdaJam in July.  There were maybe 30 or so of us who like and use F#.  There were probably easily 100 who like Clojure and probably a just slightly smaller number for Scala.  There were even folks interested in Erlang and Elixir–and I’m pretty sure there were more than 30 of them.

When I went to Strange Loop a few years ago–several sessions on Clojure, a few sessions on Scala, and one session on F#.  Just one poorly attended session given by one of the guys from the MS F# team.  Nothing speaks to Microsoft’s apathy regarding F# any louder or any clearer.

I tried leading a BOF for F# at CodeMash in 2011 as well.  Two people showed up.  Well two if you count me.  Don’t get me wrong; no doubt F# is starting to gain some traction.  F# is finally starting to gain some popularity.  But Clojure, Scala and Erlang are also gaining in popularity as well.

I don’t feel the need to follow the herd but I am also aware that if everyone is going in one direction there’s a good possibility that they’re all right and I’m the one that’s wrong.  That sounds like a statement of the obvious now that I reread it but I’ll leave it in anyway.

I’ve been waiting to see if  maybe the open source community might embrace F#.  F# is really a good language and I want to see it succeed.  But the open source community for F# is basically the same small set of people that have always worked with F#.  And somehow I don’t believe they will embrace F# any more than they’ve ever embraced Mono because they’ve grown up with “Microsoft is evil!” drummed into their heads.

So today someone for whom I have a great deal of respect basically told me to not criticize Microsoft for their apathetic approach to F# and that we as a community need to do more to push F#.  I guess those two conferences I organized here in Detroit and that F# user group I ran for a while just demonstrate what a slacker I am.  I guess the F# talks I’ve given basically show that I need to do much more for F#.  David Giard, Microsoft C# MVP once referred to me as a “F# Fanboy” (he was kidding but only a little) but apparently that’s simply not enough.

So to sum up; I’ve had it.  I’m going to learn Scala.  I know a few decent folks in the Scala community and I have no doubt that the fundamental ideas of functional will transfer just fine to Scala.  But I’m tired of fighting for F#.  Tired of fighting the tide of apathy coming from Microsoft.  I guess those folks who are fighting for F# within Microsoft  just don’t know how to fight the politics of Redmond.

I am only bothering to share this with the world because I have been a vocal advocate of F# in a lot of forums.  I hate the taste of crow but I hate being a hypocrite even more. Either way I’m not wasting any more of my time on the betamax of .Net languages.  I sincerely wish good luck to those sticking with F#; they’re a great, smart bunch tilting at windmills as they are.  As for me, I’m just tired of fighting for F# only to be told I’ve not done enough.  I really hope they succeed but they really have a tough course to chart because MS doesn’t seem to care at all about F# and I think the OSS community only sees evil Microsoft and they’re not willing to look beyond that.


A Comment On RSVP’s On Meetings

While I’m thinking about it, another quick comment.  If you’re asked to RSVP to a meetup, user group, etc. please do not say “I’ll attend” unless you’re sure you’ll attend.  As dumb as it sounds we have a few people around the Metro Detroit area (not naming any names) that seem to consistently sign-up for meetups and indicate they are planning to attend and then simply never show up.

Perhaps if I explain why this is a problem my point will become clearer.  The organizers of these user groups, meetups and so forth often want to feed attendees.  In order to get the right amount of food because if we get too little some will go hungry and if we get too much food may go to waste we need an idea of how many people to expect.  Hence if you RSVP that you’re going to attend and then just don’t bother to show up you’re simply making it harder for those who run these meetings.  By the way, the people I have in my mind have done this multiple times.

If you join a meetup group because you’re interested in the topic you are not obliged to RSVP yes to attendance.  You won’t be kicked off of the meetup group’s mailing lists. But if you do RSVP yes and don’t bother to show up you may eventually find yourself removed.

Why I Am Not A Fan of Screen Sharing For User Group Meetings

So I want to capture these thoughts while I can for whatever value they may have to myself or others in the future.

Last night we did a coding dojo on building Android apps with F#.  Reluctantly, after a little cajoling from a co-worker I decided to add a Google Hangout to the meeting.  NB: While I specifically mention Google Hangout don’t misunderstand; the problems I’m addressing have been problems with pretty much any teleconferencing solution I’ve had experience with.  AnyMeeting, WebEx etc. etc. etc. all seem to have the same issues.

So I don’t think it went particularly well (others may feel differently).  And I think I’ve finally figured out my issues with the screen share/videoconferencing approach.

The technology still has some rough edges. Repeatedly last night I was sharing a screen to show people on-line some point that I was discussing.  And repeatedly the screen share simply stopped by itself. The video conferencing portion just stopped working by itself at some points too.  This issue will go away as the technology improves, of course, but we’ll also ask more from the tech so this issue will never really resolve completely.

The speaker’s focus is split. For me, the bigger issue was trying to help the developers in the room (coding dojo) and trying to assist the developers online.  There was no way I could simply talk to the people online and also address the people in the room.

The speaker ends up having to run the video conference. This is a relatively tough issue to address because the speaker has the slide deck and the speaker knows which examples he (or she) is going to show so they end up running the video conference from their machine.  It’s tough enough to give an intelligent talk without having to worry about the side issues of when you should share your screen and when you should be showing the video feed and such.  I think if I were to do this at a user group meeting again there would be a designated video conf assistant and I’d practice things with that video conf assistant at least three or four times before the actual presentation.

If others have thoughts or suggestions on these issues, please feel free to comment.  Perhaps I’m simply not seeing what may be simple answers to these issues.

Why The Defaults Matter

I often think of why I feel that it’s important to have a functional language—not just write code that’s pseudo-functional in an imperative language.  As I think this through, I wanted to record some of my ideas on the subject.

Even the best developers I’ve known are almost always inclined to take the defaults.  I mean if the default string type is ASCII then you’re very unlikely to see Unicode strings in the app unless someone specifies that this is needed.  This is not laziness; it’s simply that these developers have other issues to concentrate upon.

Why is this important?  Simply this: I’ve come to realize that default mutability is an accidental complexity–in the sense that Fred Brooks used that term in his “No Silver Bullet” essay.  Don’t misunderstand what I’m saying–mutability is necessary and appropriate.  But having all values in software mutable by default is an invitation to lots of unexpected side effects.  And this is why I really believe that functional languages will continue to gain in use.  Functional languages are immutable by default.  Impure functional languages (OCaml, F#, Clojure, and Scala among others) do allow a programmer to specify mutable quantities where it’s appropriate due to performance considerations but they default to quantities being immutable–and that’s why they’re important.  If things default to being immutable, developers will learn to work with immutable quantities and become accustomed to them in the same way they’re now accustomed to thinking in Objects.  People seem to forget that 30 years ago, OO was as much an academic discipline as Functional is now.

Every developer knows they should mark quantities as constant (or final or readonly or whichever keyword expresses immutability) but most do this only as an afterthought and they do it incompletely.  Default immutability could eliminate a whole class of side-effect errors and therefore reduce errors in our code by a large fraction of all errors present. The issue is not the question of does “const correctness” (for lack of a better way of phrasing it) make software less error-prone; the issue is getting developers who are always under impossible deadlines to use constants by default.  Languages that default to constant, immutable quantities are a large step in that direction.  This is why functional programming is important and this is why you’re going to inevitably see more and more developers gravitating toward these functional languages.