Detroit Day Of Functional

After a bit of discussion and debate a few of us have decided to go ahead with the Detroit Day of Functional on March 25, 2017. Detroit Labs is kind enough to let us borrow their space for the day so that’s where we’ll have it. We’re charging $30 a ticket but that’s mainly to cover lunch and maybe a t-shirt for the attendees if there’s enough left over for that.

We’ve also decided to do something a bit novel (well novel for me anyway). We’re organizing the event as an unconference. That means no pre-selected speakers or sessions. We’re running the whole thing on the principle of open spaces.

I know some people will wonder why we’ve decided on this approach. I can’t speak for my colleagues but for me there were a few good reasons:

  1. I wanted to keep the event relatively simple and I wanted to focus on content more than anything else. A call for speakers is a lot of work and frankly as much as I’m usually happy with the folks we end up with I also feel bad for those who submit great ideas that we can’t accommodate. This way if anyone’s got something really neat they want to discuss, they have the opportunity.
  2. I’ve observed that no matter what I pick for a conference there are people who feel that I’ve got some sort of blinders in regards to content. I’ve been told that my conferences are too “Microsoft-centric”. I’ve also had people express concerns that underserved populations aren’t represented enough by speakers at my conferences. Well here’s your big chance to show me what I’ve been missing. Literally anyone who cares to lead a session can do so. If, for instance, you don’t feel like there’s enough content on stack-based languages at conferences, here’s your chance to remedy that. Want to have a conversation on why functional programming stinks? Feel free. That’s the point of holding this as an unconference–you decide what we discuss, not the organizers (well, not only the organizers anyway).

I wanted to insure that people know exactly what to expect from the start. I wish I were a bit more glib because I think that sometimes people feel that I swing some pretty blunt language but I can’t think of a clever way to say “Hey, we’re not going to have speakers for this event. We will have lunch and we may have t-shirts but no set agenda in terms of speakers.” I’m tempted to set up a pool to see how long it takes someone to ask me “Hey why aren’t you guys going to have speakers at your event?” This is my attempt to forestall the question before it’s even asked.

Railway Oriented Programming In Elixir

A while back Scott Wlaschin posted a great blog post on what he called “Railway Oriented Programming.”  It’s a great treatment of a common problem in adopting functional programming; how to handle a chain of functions when one of the functions in the chain may return an error.  Of course stopping the chain when the first error is returned isn’t a difficult issue; the difficulty arises in structuring the code such that the code can continue running all the functions until and unless an error is encountered.  It’s a great read and a powerful idea.

So today someone asked a question on StackOverflow about how to accomplish basically that same idea in Elixir.  A little tricky to translate Scott’s ideas to Elixir because in his article his code is specified in F# and he created a special type to capture the success or failure of a function. F# is statically typed; elixir is dynamically typed which is, of course, a substantial difference.

At any rate, I saw the question and didn’t see that Saša Jurić had already posted a great answer.  So I posted my answer and then noticed his.  I mention this because I want to be clear; I wasn’t trying to steal anyone’s thunder or trying to enhance my rep or anything like that.  I just got excited; “hey, I know the answer to that question!” so I just rushed off and hacked together some demonstration code without bothering to look any further.  I am pleased to see that the answer I came up with was pretty much the same as Saša’s; I consider him an extremely smart developer and coming up with an answer pretty similar to his  independently makes me feel a bit more confident of my own skills.

The TL;DR version is that you need to construct the functions to be applied in such a way that you can carry forward an indication of either success or failure along with the value to be checked.

defmodule Password do

  defp password_long_enough?({:ok = _atom, p}) do
    if(String.length(p) > 6) do
      {:ok, p}
    else
      {:error,p}
    end
  end

  defp starts_with_letter?({:ok = _atom, p}) do
   if(String.printable?(String.first(p))) do
     {:ok, p}
   else
     {:error,p}
   end      
  end


  def password_valid?(p) do
    {:ok, _} = password_long_enough?({:ok,p}) 
    |> starts_with_letter?
  end

end

And one would use it like so:

iex(7)> Password.password_valid?("ties")
** (FunctionClauseError) no function clause matching in Password.starts_with_letter?/1
    so_test.exs:11: Password.starts_with_letter?({:error, "ties"})
    so_test.exs:21: Password.password_valid?/1
iex(7)> Password.password_valid?("tiesandsixletters")
{:ok, "tiesandsixletters"}
iex(8)> Password.password_valid?("\x{0000}tiesandsixletters")
** (MatchError) no match of right hand side value: {:error, <<0, 116, 105, 101, 115, 97, 110, 100, 115, 105, 120, 108, 101, 116, 116, 101, 114, 115>>}
 so_test.exs:21: Password.password_valid?/1
iex(8)>

A couple of comments on the code.  Someone might ask why password_long_enough?  gets a parameter of a tuple of atom and string. Since it’s the first function in the chain it’s not really needed; all it needs is the string.  I constructed in this way for a few reasons:

  1. It’s more parallel to the other function
  2. The functions are really intended to be rules which should be checked so there should not be an order dependency. That is, they should be interchangeable.

Basically by adding any predicate function that we wish given that it takes a tuple of an atom and a string and the atom is matched against :ok, we can add any arbitrary set of tests we wish to add.

As with most things on this blog, I write this down as much for my own reference as for the edification of others who might read it.  I’m pretty sure at some point in the future, I will need to validate a value through a set of predicate functions and I’ll look at this blog post to see how I can do it.


EDIT: I was reminded of a great blog post written up by Zohaib Rauf on precisely this same idea.  Well worth reading.

Athena Code

One of the perverse side effects of code that’s “the simplest thing that could possibly work” is that it can appear much easier to create than it actually was.  I refer to this as Athena Code because it can appear as if the code sprung full-grown and ready to go from the head of the developer that created it.  In my experience as a developer this almost never happens and it can give developers the false impression that they should be able to spit out simple solutions on their first try.

A Specific Example

Recently I had a need to generate a list of all dates between a given start date and an end date in Elixir. This is the code that I actually ended up with:

defp generate_all_valid_dates_in_range(start_date, end_date) when start_date <= end_date do (:calendar.date_to_gregorian_days(start_date) .. :calendar.date_to_gregorian_days(end_date)) |> Enum.map (&:calendar.gregorian_days_to_date/1)
end

#And here's how it would work.
#iex(4)> generate_all_valid_dates_in_range({2012,1,1},{2012,2,1})
#[{2012, 1, 1}, {2012, 1, 2}, {2012, 1, 3}, {2012, 1, 4}, {2012, 1, 5},
# {2012, 1, 6}, {2012, 1, 7}, {2012, 1, 8}, {2012, 1, 9}, {2012, 1, 10},
# {2012, 1, 11}, {2012, 1, 12}, {2012, 1, 13}, {2012, 1, 14}, {2012, 1, 15},
# {2012, 1, 16}, {2012, 1, 17}, {2012, 1, 18}, {2012, 1, 19}, {2012, 1, 20},
# {2012, 1, 21}, {2012, 1, 22}, {2012, 1, 23}, {2012, 1, 24}, {2012, 1, 25},
# {2012, 1, 26}, {2012, 1, 27}, {2012, 1, 28}, {2012, 1, 29}, {2012, 1, 30},
# {2012, 1, 31}, {2012, 2, 1}]

I have deliberately omitted comments so the simplicity of the code is, hopefully, a bit more apparent. For those that don’t read Elixir, basically I take the start and end date and I convert them to Gregorian dates. I then create a range from the two Gregorian dates and pass the range as the first argument to the Enum.map function which follows.  As those of you familiar with functional programming will guess, the map takes each element in the list and performs the specified function (in this case the Erlang stdlib calendar module gregorian days to date function) on it, creating a new list with the result.  {yyyy,mm,dd} is how one specifies a date literal in both Erlang and Elixir.

By the way, many thanks to the folks on the Elixir Lang Talk mailing list for helping me to simplify my first pass at this code.

Now some may argue that that code above while simple is not particularly robust.  What happens if someone passes a bad date?  Well, that’s the Erlang way–fail fast.  If someone passes a bad date, the code will crash as soon as it’s executed. And it wouldn’t be hard to add a couple of lines of code to validate good inputs and it wouldn’t detract from the basic simplicity of the code.

Try Number 4

The point I’m trying to make is while that code seems obvious (I hope) it’s far from the first code I came up with to solve my problem.  It was actually try number 4.  I won’t share the code from tries 1 through 3 mainly because like any developer I want people to think I’m brilliant so I don’t want to keep failed experiments around.  But in general terms this is what I tried:

First try:

Add one day to the first date

Is the resulting date equal to the end date?

Yes -> Stop

No -> Add the result to the output list and loop to the top.

There was nothing particularly wrong with this approach–the code was just a lot more complicated than that sounds.

Second try:

Use a comprehension to try to generate the list

Got nowhere with this approach at all.  Totally failed idea.

Third try:

Modify the comprehension from try 2.

Again, total failure.

Fourth try:

Actually looked like this:

def generate_all_valid_dates_in_range(start_date, end_date) when start_date <= end_date do 
  (:calendar.date_to_gregorian_days(start_date) .. :calendar.date_to_gregorian_days(end_date)) 
  |> Enum.to_list
  |> Enum.map (&(:calendar.gregorian_days_to_date(&1)))
end

As I say, my fellow developers on the Elixir Talk mailing list helped me to make my code even simpler yet by pointing out that the Enum.to_list was redundant and that I could directly invoke the :calendar.gregorian_days_to_date function without having to wrap it in a lambda.

My point is this: it’s almost never easy to create “the simplest thing that could possibly work”.  Simplicity is extremely hard to create.  Almost never will your first pass be the best answer.  Don’t beat yourself up if you can’t manage “Athena Code”; almost no one can.  And don’t look at someone else’s code which is a monument to simplicity and think that that’s the way it looked when they first wrote it.  Our job as software engineers can be hard enough without putting unrealistic expectations on ourselves.

A Naive Stack Implementation In Elixir

So one of the things I’ve done recently to help me to learn a new language is to tackle creating certain data structures and common operations on those structures in a given language.  And lately I’ve been digging Elixir quite a bit.  So I decided to tackle one of the easiest things I could tackle–a simple, naive stack implementation in Elixir.  First here’s the actual Elixir code:


defmodule Stack do

defstruct elements: []

def new, do: %Stack{}

def push(stack, element) do
 %Stack{stack | elements: [element | stack.elements]}
 end

def pop(%Stack{elements: []}), do: raise("Stack is empty!")
 def pop(%Stack{elements: [top | rest]}) do
 {top, %Stack{elements: rest}}
 end

def depth(%Stack{elements: elements}), do: length(elements)
 end

# Stack.new |> Stack.push(1) |> Stack.push(2) |> Stack.pop

First off, credit where credit’s due: this is more Saša Jurić’s code than it is mine.  I posted my initial code to Code Review and a few folks gave me some great suggestions on how to improve it. But his code seemed the best implementation so this is pretty much his code.

Looking at the code a bit closer, one thing long time OO folks will notice, and something that gave me a lot of trouble initially, is the fact that there’s no member variable to stash the stack in.  The stack has to be passed into each call and the new stack is returned.  This is a big leap in logic and even now after having played with functional for a few years it can still form a large stumbling block for me when I’m trying to solve a problem.

Another thing to notice is this line:


defstruct elements: []

This is within the scope of the module but what is it doing if there’s no member data associated with the module?  It’s easy to assume that modules are analogous to classes in OO and therefore it’s also easy to assume that elements is member data–but that would be a bad assumption.  Elements is actually an abstract data type closely associated with the Stack module.  It’s a way of specifying more information about the data that the various functions in Stack will work with.  You cannot access elements from outside of Stack and in fact it’s not private data within Stack.

One comment Saša made resonates for me. To paraphrase his point,  there is already a stack implementation in Elixir; it’s called a list.  So why bother to build another one?  It’s a fair question; my answer is that if I needed a stack, I’d rather see code that deals with “Stack” directly than see code that deals with lists and has comments littered all over the place that it’s using a list as a stack.  Of course, it’s syntactic sugar but at some level anything other than raw 1’s and 0’s is syntactic sugar.  It’s a question of making it easier for other developers to understand the intent of the code.

I share this not because it’s great or interesting code (although it was much improved by feedback from Johnny Winn, Saša and José Valim) but because someone may want to study this relatively simple code to learn more about Elixir.

 

 

 

What Is Immutability? And Why Should I Care?

One of the more hotly debated topics among software developers these days is adopting functional programming. Those people who ponder these questions have presented good arguments regarding adopting (or avoiding) functional programming.  But the term “functional programming” is a bit slippery and often when developers say functional programming they’re actually referring to a few different ideas.  One of the more important ideas is default immutability in data in the language. But as sometimes happens, people can load a term with different meanings unrelated to the original. This makes it harder to discuss technical merits and drawbacks because people don’t agree on exactly what they mean. Sometimes it’s helpful in debate and discussion to spend some time to clarify exactly what we mean by certain terms.

Mutability and the converse immutability are two terms that are debated in a less-than-rigorous fashion.  In this post, I will endeavor to explain these terms and what they mean in practical terms.  I also hope to give those making decisions about technology a bit of insight into why immutability is an important concept to understand.

Mutability

It’s difficult to define the absence of something so I will first define mutability; it will then be easier to discuss what we mean by its opposite.

When all the other abstractions are stripped away, a computer doesn’t know anything except 1 and 0.  Either a particular bit is on or it’s off.  Creating software is an extremely taxing mental activity and anything that can be done to lighten the load for the software developer is a good thing.  One way to lighten the load is to allow the developer to think of things in abstract terms.  If a developer can think of a value stored in memory somewhere as opposed to thinking of a series of 1s and 0s at some particular circuit, it’s far easier for the developer to focus on more important concerns.  Hence very early in the history of software development the abstraction of a variable was created.  A variable was simply a way of naming a memory location to make it easier for a developer to reason about it.  It’s the difference between saying “memory location 3000 has the value 25 stored in it” and saying “age equals 25” where age is defined as memory location 3000.  To the computer, they’re effectively the same thing but to the software developer the latter is far easier to understand.

Now at the hardware level unless something in the hardware specifically prevents it, any memory location can be altered at any time.  That is, unless the hardware has some mechanism to prevent it, my software can alter “age” to be 26 pretty much anywhere it cares to do so.

To give a more specific definition, mutability is simply the ability of software to overwrite any memory location at any time with a new value.

Now, a developer has to have the ability to store a new value over an existing value at least once.  That is, if he or she could never write a new value to a memory location then everything would be 0 all over memory and we couldn’t get anything accomplished.  So to dig a bit deeper, mutability is the ability to overwrite a memory location which has been modified from the default with a new value.

That is to say if I store 25 at memory location 3000, with mutability I can later overwrite memory location 3000 with 26 (or anything else I care to).

Immutability

So then if mutability is the ability to store a new value in a memory location that has already been set to a non-default value then what is immutability?  Immutability is simply the property that a memory location can be set one time and then never altered again.

That is to say that if mutability is being able to set age to 26 after it’s been initially set to 25 then immutability prevents the age from ever being changed from 25 after that.  The memory location can be given an initial value but once it’s given an initial value the software is not permitted to alter it.

So Why Should I Care?

So maybe now you’re saying to yourself—ok, so I get it.  With mutability I can keep on changing a value anytime I want; with immutability, I can only set a value one time.  Why is this important?

There are a few reasons this is worth discussing.  I’ll cover some of those of which I’m aware but this is hardly an exhaustive list.

It’s easier to reason about immutable code

Every separate detail that a developer has to keep track of in his or her head is a form of a mental tax.  Psychologists have determined that, on average, the typical human being can keep 5 to 7 distinct items in their short-term memory at any given time.  As a developer has to remember more and more details, it’s more and more likely that he or she will forget some detail.  Not having to worry about a value being changed after it’s been set is just one less bit of mental taxation.

It’s easier to prevent accidental changes if the data is immutable

Almost every time you hear of a new exploit, it’s likely that it involves software writing to memory that it was not intended to write.  Buffer overflow bugs in software are all errors involving writing memory that was not meant to be written.  While no system devised by human developers will ever be perfect, it’s less likely that a system that cannot accidentally write to memory once the memory is set will suffer from these sorts of security exploits.

Beyond even these security exploits there’s the simple matter of the number of bugs arising in software due to inadvertent modification of memory.  Anyone who’s ever coded in C will have stories of a bad pointer overwriting memory.  They will have stories about this because these kinds of bugs are usually hard to reproduce and hard to isolate.  Imagine you have people entering their name and the name is stored in the wrong location in memory.  Each time a different name is entered, there’s potential for a different behavior and therefore a different bug.  Default immutability lessens this possibility.

It’s easier to execute code in parallel when the data is immutable

This is one of the major reasons for the debate/discussion around functional programming these days.  Executing code in parallel is likely the way forward as far as running our software faster.  However, when code is executed in parallel, the number of possible writes to the same data is multiplied by the number of executing processes accessing the data.  Therefore developers have to modify their code to prevent multiple processes from accidentally writing the same data.  Making data immutable nicely side-steps this issue.  If you can’t write existing memory, you can’t accidentally modify existing memory.

By the way, many thanks to Ms. Sarah Trenz for her invaluable feedback on this article.

Partial Function Application In Elixir

So I’m trying to fill out my understanding of basic functional programming concepts in terms of Elixir and Erlang.  One concept that I wanted to know a bit more about is how Partial Function Application and Currying work in Elixir.  Especially given that the function signature of a function in Elixir includes its arity, it seems somewhat unlikely that Elixir would support currying and/or partial function application.  But a little bit of hacking turned up a nice surprise.  While Elixir may not support currying (I really cannot tell if it is supported or not) it does support partial function application which seems to be the main reason to concern one’s self with currying anyway.

Partial Function Application With Lambdas


multiply = &(&1 * &2)

timestwo = &(multiply.(&1,2))

Interestingly, using this technique you can set any number of parameters to a fixed value.  Consider this:


multiply4 = &(&1 * &2 * &3 * &4)

times20 = &(multiply4.(&1,&2,5,4))

Lest I be misunderstood, I wouldn’t advocate writing code like that; I’m just saying that it’s possible to do.

Partial Function Application With Modules


defmodule PaTest do
  def f1(n, m) do
    n * m
  end

  def f2(m) do
    f1(3,m)
  end
end

I hope this may save someone a bit of work.

Configuring Elixir For Development

One of the more exciting new functional languages of the last couple of years is a marriage of Erlang and Ruby called Elixir.  Besides adding an arguably better syntax with which to program the Erlang VM (also known as the BEAM VM), Elixir adds meta-programming, addresses issues Erlang had with dealing with Unicode strings, and adds few other nice touches.  It’s nice icing on the Erlang cake.  Joe Armstrong, the creator of Erlang, has gone on record as endorsing Elixir. I leave it to those who are interested in reading Mr. Armstrong’s comments to read for themselves; tl; dr he likes Elixir just like he likes Erlang.

So I’ve been wanting to play with Elixir for a while and when I want to play with a new language, I often configure a quick command prompt set up with the proper development environment.  It’s halfway between creating a new VM and not doing anything special. So if others care to build an elixir command prompt, here are the steps.  Note that you can stop before installing Elixir and have a great Erlang environment to play around with.

1.) Install Erlang. You can download a nice Windows installer package for Erlang from Erlang Solutions.

2.) Once erlang is installed, you’ll want to grab the prebuilt zip of Elixir from the Elixir GitHub repository.

3.) Create a powershell script to configure the environment.  Here’s mine:

Set-Variable -Name elixirBase -option Readonly -value "C:/dev/elixir_0.12.4"
Set-Variable -Name erlangBase -option Readonly -value "C:/Program Files/erl5.10.2"
Set-Variable -Name gitPath -option Readonly -value "c:/program files (x86)/git"

$env:Path = "$erlangBase/bin;$elixirBase/bin;$gitPath/bin;"

Set-Variable is the Powershell analog of setting an environment variable.  $env:Path is the actual Path environment variable in this case.  Since the environment will only apply for the duration of the Powershell session, I simply replace the existing Path with one set up specifically for the Elixir/Erlang combo. I named the file ElixirSetup.ps1; it’s convention for Powershell scripts to have a ps1 extension.  NB: These paths are on my machine and they’re for a slightly older version of Erlang (as of this writing current version installs to elr5.10.4).

Basically if I’m going to do a command prompt on Windows, Powershell is the way to go.  It remedies countless failings of the old cmd shell.

4.) Create a new shortcut with the following command line:

%SystemRoot%\system32\WindowsPowerShell\v1.0\powershell.exe -NoExit -File C:\Users\Onorio_Development\Documents\WindowsPowerShell\ElixirSetup.ps1

Obviously you’ll want to replace these values with the correct values for your particular machine.

Once you get the shortcut set up and going, there are a few things to test.  You can run the Erlang REPL by typing erl at the command prompt.  You can test the Elixir REPL by typing iex.bat at the command prompt.  You have to type the entire name (with the .bat extension) due to there being an iex alias present in Powershell.

Elixir includes a nice little project management tool called mix which was inspired by Clojure’s Leiningen.  Mix basically lets you create a small skeleton of an Elixir project complete with a .gitignore file and a README.md as well as a skeleton unit test.

I hope others will consider this an invitation to start playing with Elixir and learning more about both Elixir and Erlang.

EDIT:

One annoyance that I discovered after I wrote this post is that the native Powershell terminal doesn’t support ANSI escape sequences.  IEx uses ANSI escape sequences and this is a real annoyance.  So I did some digging and found that ConEmu has nice support for ANSI and it remedies the issue with IEx very nicely.  There are other ways of getting support for the ANSI escape sequences as well but this is a nice robust remedy for the problem.