Rebar3 Beta1 on Chocolatey NuGet

I had a need for the Erlang build tool rebar for a project I was trying to start on; a bit of pen testing via Elixir.  I was going to pull down the rebar3 script from the site and fix it up for myself and it occurred to me that others might want a way to get the script easily too.  So rather than just building something to work for myself, I took a little extra time and built a Chocolatey NuGet package for Rebar3-Beta1.

The package takes care of insuring you’ve got Erlang first although honestly I can’t imagine anyone interested in Rebar that wouldn’t already have Erlang on their machine.  It also takes care of setting up a shortcut for Rebar and putting it in the path.  And, of course, like everything else in CNG, it also handles uninstalling the tool if you decide you no longer want or need it.

For now the package is still awaiting moderation on CNG but please feel free to pull it down and give it a try.  Feedback is a gift and I hope some of you will share that gift with me.

Rebar3 on ChocolateyNuGet

New Versions Of Chocolatey NuGet Packages For Elixir and Erlang

Quite some time ago, I took over maintenance of the Chocolatey NuGet (CNG) packages for both Elixir and Erlang.  I did release new versions of both packages as new versions of Erlang and Elixir became available but I didn’t do a lot of work on either of them.  That changed a little bit today.

One thing that had always kind of annoyed me was the fact that I could take my pick; I could modify people’s paths on their machines without asking or I could tell people that after they installed Erlang and Elixir they’d need to fix up their paths to get Elixir to work “out of the box”.  Neither option seemed particularly good but I decided to go with the latter because I don’t believe in messing around with other people’s machines without explicit permission and I didn’t see anything in the CNG framework to prompt them.

Recently the question of why Elixir doesn’t work “out of the box” came up yet again.  I gave my standard answer and then one of the CNG mods mentioned an option that I’d not heard of before.  CNG now has the ability to create a small shim file to run another command. I say “now has” but this feature may have been in CNG all along; maybe I just never heard of it.

Regardless this is a great approach.  I can create these shims to point to the right bat files and the right exe’s, put the shims in the Chocolatey bin directory which is added to the path by Chocolatey and I get behavior much closer to the behavior that apt-get and homebrew users have been getting for a long time.

So today, I’m pushing up slightly modified CNG packages for both Elixir and Erlang.  Besides adding the shims I also removed some dead code that I had simply been too lazy to remove and I also modified the Erlang uninstall process to fix a long-standing issue.

One thing some people may take some issue with: I have called the shim to iex.bat ielixir. I did this because iex is a command in Powershell and if you’re on Windows, you should be using Powershell for your terminal.  Note that since this is simply a shim, iex.bat is still present and can still be called if anyone cares to.  I realize that on Mac and Linux it’s iex but I think this is important enough to allow this small divergence.  Maybe we could get Mac and Linux to add an ielixir alias to their packages.  After all iex is short for “interactive elixir” so ielixir isn’t totally unreasonable.

I would appreciate it if anyone who uses CNG to install Elixir would take a look at this new package and beat it up a bit.  If you find issues, please let me know via the Github repo. 1.0.5 and 18.0 will continue to be on CNG if you don’t care to make this change.

The packages are Elixir 1.0.5.20150709 and Erlang 18.0.20150709

Always the way; upload packages, let people know and then realize you forgot to test something.  The corrected packages are Elixir 1.0.5.20150710 and Erlang 18.0.20150709.2

Thanks to Gary Ewan Park and Rob Reynolds for their help in figuring out the problems I was having with creating the shim files I needed.

Some Thoughts On Windows 10

Recently I saw a great post on Extreme Tech about the upcoming Windows 10 release and, to put it in my own words, the heart of the problem with both Windows 8.x and Windows 10.  Well, one of the problems with the great “Desktop/Tablet Convergence” that Microsoft has been trying to foist on all of us since Windows 8.0.  I think that the author of that article, David Cardinal, got at the essential problem with the whole convergence strategy that Microsoft was (and still is) pursuing: the fact that people fundamentally don’t want the convergence that MS is offering.

Now I’ve seen the posts about Windows 10 supporting Android and iOS apps natively.  While I’d like to be charitable and assume that MS has decided to actually care about its users rather than its bottom line, I don’t believe that’s happened.  I see this instead as a rather cynical ploy to further fragment Android and maybe to start to try to fragment iOS.  Given the level of control that Apple exercises over iOS, I doubt Microsoft will get far in fragmenting it.  But on the Android side, I can believe they’d have high hopes of fragmenting the platform further.  After all they’re backing Cyanogen which is “going to put a bullet through Google’s head“.

The thing is, I’m still not sure who they think is going to build (or even modify) Android or iOS apps for Windows 10.  I mean they’ve offered money for developers to build apps for Win Phone and yet the platform still lacks many of the most popular mobile apps. And telling developers that there’s yet another platform for them to test their Android or iOS app on for what’s likely to be a miniscule market share–well I’m sure developers will be lining up to invest in that (not really).  Now some of you will read this and say “Miniscule market share?  What about all those Windows 7 and 8 boxes which people will upgrade to Windows 10?”  And, yes, that’s true.  But in terms of the Android and iOS market, Windows 10 will still be a drop in the bucket.  I will be shocked if that changes.

I guess, in spite of the fact that I’ve seen it time and again, I’m still surprised that people have such short memories. I can recall the days of DOS when people complained that the interfaces of apps weren’t the same from one app to another.  “It should be like a car!  Every app should work the same!” they wailed. But what was true then is still true today; when two things serve two fundamentally different objectives, they need different interfaces.  By the way, lest anyone think I don’t know my history: when Windows came along some of the superficial UI differences were removed by Microsoft. But they were superficial differences. Tablets serve a fundamentally different purpose than laptops and desktops.  Trying to force convergence isn’t going to change the fundamental issue.  Cars and forklifts are different if similar machines with different purposes and therefore different affordances.

I just wish someone at Microsoft would realize that trying to force a single OS on us for tablets, gaming consoles and laptop/desktop devices is simply not going to work.

The Detail-Oriented Nature Of Software Development

One thing I have noticed about myself over the years; I’m not as good as I would care to be at focusing on details. In fact, sometimes I’m awful at paying attention to the details. As a software developer, of course, details are extremely important.

I’ve come up with (and read about) a few tricks/heuristics to keep me from forgetting important details:

  1. Checklists
  2. Scripts/code to automate repetitive chores (i. e. the more I can put into a script the less I need to pay attention to).
  3. Rubber duck debugging (i. e. sometimes just the act of talking through a problem helps me to discover an overlooked detail).

I would like to know what other techniques and tactics other developers do to cope with the amount of detail that we deal with in the course of our job.  Please share any thoughts/ideas you have on this subject in the comments.

Athena Code

One of the perverse side effects of code that’s “the simplest thing that could possibly work” is that it can appear much easier to create than it actually was.  I refer to this as Athena Code because it can appear as if the code sprung full-grown and ready to go from the head of the developer that created it.  In my experience as a developer this almost never happens and it can give developers the false impression that they should be able to spit out simple solutions on their first try.

A Specific Example

Recently I had a need to generate a list of all dates between a given start date and an end date in Elixir. This is the code that I actually ended up with:

defp generate_all_valid_dates_in_range(start_date, end_date) when start_date <= end_date do (:calendar.date_to_gregorian_days(start_date) .. :calendar.date_to_gregorian_days(end_date)) |> Enum.map (&:calendar.gregorian_days_to_date/1)
end

#And here's how it would work.
#iex(4)> generate_all_valid_dates_in_range({2012,1,1},{2012,2,1})
#[{2012, 1, 1}, {2012, 1, 2}, {2012, 1, 3}, {2012, 1, 4}, {2012, 1, 5},
# {2012, 1, 6}, {2012, 1, 7}, {2012, 1, 8}, {2012, 1, 9}, {2012, 1, 10},
# {2012, 1, 11}, {2012, 1, 12}, {2012, 1, 13}, {2012, 1, 14}, {2012, 1, 15},
# {2012, 1, 16}, {2012, 1, 17}, {2012, 1, 18}, {2012, 1, 19}, {2012, 1, 20},
# {2012, 1, 21}, {2012, 1, 22}, {2012, 1, 23}, {2012, 1, 24}, {2012, 1, 25},
# {2012, 1, 26}, {2012, 1, 27}, {2012, 1, 28}, {2012, 1, 29}, {2012, 1, 30},
# {2012, 1, 31}, {2012, 2, 1}]

I have deliberately omitted comments so the simplicity of the code is, hopefully, a bit more apparent. For those that don’t read Elixir, basically I take the start and end date and I convert them to Gregorian dates. I then create a range from the two Gregorian dates and pass the range as the first argument to the Enum.map function which follows.  As those of you familiar with functional programming will guess, the map takes each element in the list and performs the specified function (in this case the Erlang stdlib calendar module gregorian days to date function) on it, creating a new list with the result.  {yyyy,mm,dd} is how one specifies a date literal in both Erlang and Elixir.

By the way, many thanks to the folks on the Elixir Lang Talk mailing list for helping me to simplify my first pass at this code.

Now some may argue that that code above while simple is not particularly robust.  What happens if someone passes a bad date?  Well, that’s the Erlang way–fail fast.  If someone passes a bad date, the code will crash as soon as it’s executed. And it wouldn’t be hard to add a couple of lines of code to validate good inputs and it wouldn’t detract from the basic simplicity of the code.

Try Number 4

The point I’m trying to make is while that code seems obvious (I hope) it’s far from the first code I came up with to solve my problem.  It was actually try number 4.  I won’t share the code from tries 1 through 3 mainly because like any developer I want people to think I’m brilliant so I don’t want to keep failed experiments around.  But in general terms this is what I tried:

First try:

Add one day to the first date

Is the resulting date equal to the end date?

Yes -> Stop

No -> Add the result to the output list and loop to the top.

There was nothing particularly wrong with this approach–the code was just a lot more complicated than that sounds.

Second try:

Use a comprehension to try to generate the list

Got nowhere with this approach at all.  Totally failed idea.

Third try:

Modify the comprehension from try 2.

Again, total failure.

Fourth try:

Actually looked like this:

def generate_all_valid_dates_in_range(start_date, end_date) when start_date <= end_date do 
  (:calendar.date_to_gregorian_days(start_date) .. :calendar.date_to_gregorian_days(end_date)) 
  |> Enum.to_list
  |> Enum.map (&(:calendar.gregorian_days_to_date(&1)))
end

As I say, my fellow developers on the Elixir Talk mailing list helped me to make my code even simpler yet by pointing out that the Enum.to_list was redundant and that I could directly invoke the :calendar.gregorian_days_to_date function without having to wrap it in a lambda.

My point is this: it’s almost never easy to create “the simplest thing that could possibly work”.  Simplicity is extremely hard to create.  Almost never will your first pass be the best answer.  Don’t beat yourself up if you can’t manage “Athena Code”; almost no one can.  And don’t look at someone else’s code which is a monument to simplicity and think that that’s the way it looked when they first wrote it.  Our job as software engineers can be hard enough without putting unrealistic expectations on ourselves.

Stupid Doc Tricks In Elixir

So last night we had our first Detroit Erlang/Elixir meetup and as part of it I wanted to do a little coding exercise to help people to get their feet wet with Erlang and/or Elixir, as they chose.  We simply worked through FizzBuzz and we came up with two different Elixir solutions and two different Erlang solutions. I never stop being surprised by developer creativity.

That said, I keep striving to improve the code I write and striving to use the tools that the language provides me.  Elixir has module attributes and it’s also got module documentation as a first class citizen in the language.  So I thought to myself why not extend my simple fizzbuzz code a bit to use these two features–to better learn how to use them.  Here’s the code such as it is:


defmodule FizzBuzz do
@fizz 3
@fizzmsg "Fizz"
@buzz 5
@buzzmsg "Buzz"

@moduledoc "
Ye olde FizzBuzz test in all of its glory.
If the number is a multiple of #{@fizz} then print #{@fizzmsg}.
If the number is a multiple of #{@buzz} then print #{@buzzmsg}.
If it's a multiple of both then print #{@fizzmsg<>@buzzmsg}.
Else print the number itself.

Example usage:
iex> for i <- 1..100, do: FizzBuzz.getFB(i)
1
2
#{@fizzmsg}
4
#{@buzzmsg}
"

  def getFB(n) do
    cond do
      rem(n,@fizz) == 0 and rem(n,@buzz) == 0 ->
        IO.puts @fizzmsg <> @buzzmsg
      rem(n,@fizz) == 0 ->
        IO.puts @fizzmsg
      rem(n,@buzz) == 0 ->
        IO.puts @buzzmsg
      true ->
        IO.puts "#{n}"
    end
  end
end

Now the code itself isn’t anything very special.  But the module attributes and the fact that I can avoid specifying magic numbers is just awesome.  What if I wanted to change the code to print “Argh!” on 3 instead of “Fizz”?  Of course were I to change the number for “Fizz” then I would need to make more extensive changes but this does cover one common type of change. With the code set up in this way, I only need to change things in one place and my docs will be updated as well as my code.

I know this is sort of a “Stupid Doc Trick” but it still seemed worth sharing with others.  The idea of only needing to change a constant in one place and having everything else update is a very appealing one.

A Naive Stack Implementation In Elixir

So one of the things I’ve done recently to help me to learn a new language is to tackle creating certain data structures and common operations on those structures in a given language.  And lately I’ve been digging Elixir quite a bit.  So I decided to tackle one of the easiest things I could tackle–a simple, naive stack implementation in Elixir.  First here’s the actual Elixir code:


defmodule Stack do

defstruct elements: []

def new, do: %Stack{}

def push(stack, element) do
 %Stack{stack | elements: [element | stack.elements]}
 end

def pop(%Stack{elements: []}), do: raise("Stack is empty!")
 def pop(%Stack{elements: [top | rest]}) do
 {top, %Stack{elements: rest}}
 end

def depth(%Stack{elements: elements}), do: length(elements)
 end

# Stack.new |> Stack.push(1) |> Stack.push(2) |> Stack.pop

First off, credit where credit’s due: this is more Saša Jurić’s code than it is mine.  I posted my initial code to Code Review and a few folks gave me some great suggestions on how to improve it. But his code seemed the best implementation so this is pretty much his code.

Looking at the code a bit closer, one thing long time OO folks will notice, and something that gave me a lot of trouble initially, is the fact that there’s no member variable to stash the stack in.  The stack has to be passed into each call and the new stack is returned.  This is a big leap in logic and even now after having played with functional for a few years it can still form a large stumbling block for me when I’m trying to solve a problem.

Another thing to notice is this line:


defstruct elements: []

This is within the scope of the module but what is it doing if there’s no member data associated with the module?  It’s easy to assume that modules are analogous to classes in OO and therefore it’s also easy to assume that elements is member data–but that would be a bad assumption.  Elements is actually an abstract data type closely associated with the Stack module.  It’s a way of specifying more information about the data that the various functions in Stack will work with.  You cannot access elements from outside of Stack and in fact it’s not private data within Stack.

One comment Saša made resonates for me. To paraphrase his point,  there is already a stack implementation in Elixir; it’s called a list.  So why bother to build another one?  It’s a fair question; my answer is that if I needed a stack, I’d rather see code that deals with “Stack” directly than see code that deals with lists and has comments littered all over the place that it’s using a list as a stack.  Of course, it’s syntactic sugar but at some level anything other than raw 1’s and 0’s is syntactic sugar.  It’s a question of making it easier for other developers to understand the intent of the code.

I share this not because it’s great or interesting code (although it was much improved by feedback from Johnny Winn, Saša and José Valim) but because someone may want to study this relatively simple code to learn more about Elixir.

 

 

 

What Is Immutability? And Why Should I Care?

One of the more hotly debated topics among software developers these days is adopting functional programming. Those people who ponder these questions have presented good arguments regarding adopting (or avoiding) functional programming.  But the term “functional programming” is a bit slippery and often when developers say functional programming they’re actually referring to a few different ideas.  One of the more important ideas is default immutability in data in the language. But as sometimes happens, people can load a term with different meanings unrelated to the original. This makes it harder to discuss technical merits and drawbacks because people don’t agree on exactly what they mean. Sometimes it’s helpful in debate and discussion to spend some time to clarify exactly what we mean by certain terms.

Mutability and the converse immutability are two terms that are debated in a less-than-rigorous fashion.  In this post, I will endeavor to explain these terms and what they mean in practical terms.  I also hope to give those making decisions about technology a bit of insight into why immutability is an important concept to understand.

Mutability

It’s difficult to define the absence of something so I will first define mutability; it will then be easier to discuss what we mean by its opposite.

When all the other abstractions are stripped away, a computer doesn’t know anything except 1 and 0.  Either a particular bit is on or it’s off.  Creating software is an extremely taxing mental activity and anything that can be done to lighten the load for the software developer is a good thing.  One way to lighten the load is to allow the developer to think of things in abstract terms.  If a developer can think of a value stored in memory somewhere as opposed to thinking of a series of 1s and 0s at some particular circuit, it’s far easier for the developer to focus on more important concerns.  Hence very early in the history of software development the abstraction of a variable was created.  A variable was simply a way of naming a memory location to make it easier for a developer to reason about it.  It’s the difference between saying “memory location 3000 has the value 25 stored in it” and saying “age equals 25” where age is defined as memory location 3000.  To the computer, they’re effectively the same thing but to the software developer the latter is far easier to understand.

Now at the hardware level unless something in the hardware specifically prevents it, any memory location can be altered at any time.  That is, unless the hardware has some mechanism to prevent it, my software can alter “age” to be 26 pretty much anywhere it cares to do so.

To give a more specific definition, mutability is simply the ability of software to overwrite any memory location at any time with a new value.

Now, a developer has to have the ability to store a new value over an existing value at least once.  That is, if he or she could never write a new value to a memory location then everything would be 0 all over memory and we couldn’t get anything accomplished.  So to dig a bit deeper, mutability is the ability to overwrite a memory location which has been modified from the default with a new value.

That is to say if I store 25 at memory location 3000, with mutability I can later overwrite memory location 3000 with 26 (or anything else I care to).

Immutability

So then if mutability is the ability to store a new value in a memory location that has already been set to a non-default value then what is immutability?  Immutability is simply the property that a memory location can be set one time and then never altered again.

That is to say that if mutability is being able to set age to 26 after it’s been initially set to 25 then immutability prevents the age from ever being changed from 25 after that.  The memory location can be given an initial value but once it’s given an initial value the software is not permitted to alter it.

So Why Should I Care?

So maybe now you’re saying to yourself—ok, so I get it.  With mutability I can keep on changing a value anytime I want; with immutability, I can only set a value one time.  Why is this important?

There are a few reasons this is worth discussing.  I’ll cover some of those of which I’m aware but this is hardly an exhaustive list.

It’s easier to reason about immutable code

Every separate detail that a developer has to keep track of in his or her head is a form of a mental tax.  Psychologists have determined that, on average, the typical human being can keep 5 to 7 distinct items in their short-term memory at any given time.  As a developer has to remember more and more details, it’s more and more likely that he or she will forget some detail.  Not having to worry about a value being changed after it’s been set is just one less bit of mental taxation.

It’s easier to prevent accidental changes if the data is immutable

Almost every time you hear of a new exploit, it’s likely that it involves software writing to memory that it was not intended to write.  Buffer overflow bugs in software are all errors involving writing memory that was not meant to be written.  While no system devised by human developers will ever be perfect, it’s less likely that a system that cannot accidentally write to memory once the memory is set will suffer from these sorts of security exploits.

Beyond even these security exploits there’s the simple matter of the number of bugs arising in software due to inadvertent modification of memory.  Anyone who’s ever coded in C will have stories of a bad pointer overwriting memory.  They will have stories about this because these kinds of bugs are usually hard to reproduce and hard to isolate.  Imagine you have people entering their name and the name is stored in the wrong location in memory.  Each time a different name is entered, there’s potential for a different behavior and therefore a different bug.  Default immutability lessens this possibility.

It’s easier to execute code in parallel when the data is immutable

This is one of the major reasons for the debate/discussion around functional programming these days.  Executing code in parallel is likely the way forward as far as running our software faster.  However, when code is executed in parallel, the number of possible writes to the same data is multiplied by the number of executing processes accessing the data.  Therefore developers have to modify their code to prevent multiple processes from accidentally writing the same data.  Making data immutable nicely side-steps this issue.  If you can’t write existing memory, you can’t accidentally modify existing memory.

By the way, many thanks to Ms. Sarah Trenz for her invaluable feedback on this article.

New Version of Elixir Chocolatey Goodness

Paul Schoenfelder and I are sort of trying to get more Windows developers to take a look at Elixir.  We both think Elixir’s got a great story to tell and Windows developers will be very excited when they find out more about it.

In that spirit, we’ve updated the Chocolatey NuGet package for Elixir to the latest development release (v0.12.5).  This is just the first step in getting better support for Elixir on Windows–we’ve got more planned for the future.

Please feel free to grab that Chocolatey goodness and let us know if you run into any problems or if you’ve got suggestions.  We’ve set up a github organization and repository for our work; feel free to open an issue there or post something on the Elixir mailing list or post a comment here.

EDIT:

The Chocolatey NuGet Package is now up to v0.13.2 (as of May 12, 2014)

Partial Function Application In Elixir

So I’m trying to fill out my understanding of basic functional programming concepts in terms of Elixir and Erlang.  One concept that I wanted to know a bit more about is how Partial Function Application and Currying work in Elixir.  Especially given that the function signature of a function in Elixir includes its arity, it seems somewhat unlikely that Elixir would support currying and/or partial function application.  But a little bit of hacking turned up a nice surprise.  While Elixir may not support currying (I really cannot tell if it is supported or not) it does support partial function application which seems to be the main reason to concern one’s self with currying anyway.

Partial Function Application With Lambdas


multiply = &(&1 * &2)

timestwo = &(multiply.(&1,2))

Interestingly, using this technique you can set any number of parameters to a fixed value.  Consider this:


multiply4 = &(&1 * &2 * &3 * &4)

times20 = &(multiply4.(&1,&2,5,4))

Lest I be misunderstood, I wouldn’t advocate writing code like that; I’m just saying that it’s possible to do.

Partial Function Application With Modules


defmodule PaTest do
  def f1(n, m) do
    n * m
  end

  def f2(m) do
    f1(3,m)
  end
end

I hope this may save someone a bit of work.