Making Exrm Work On Windows

So as many of you will know I’m very interested in Elixir.  I also want to see it work better on Windows.  There are a few of us that have been working on it.

I’ve also been very interested in how one can deploy an Elixir app on a machine which doesn’t have Erlang and Elixir already installed.  While most of us don’t mind having to have Erlang and Elixir around because we are doing development work, it would be much more secure to only have to deploy the binaries that need to be deployed for an app.

Since I started on Elixir one of the nicest folks in the community (and that’s saying something) has been Paul Schoenfelder. When I was first struggling to learn Elixir, Paul very patiently helped me to get started.  So I became aware of one of the utilities that Paul built: EXRM.  Exrm is a tool to automate the production of a package of the needed Elixir and Erlang binaries to deploy an app. It allows a developer to deploy his or her Elixir app with only the needed runtime files so that the target machine doesn’t have to have Elixir or Erlang installed.

So I asked Paul what I could do to sort of return the favor for the help he gave me when I started and he mentioned that he wasn’t as pleased as he might be with the support for Windows in EXRM.  So I started digging into EXRM to see what I might do to get it working on Windows.

A few instructive things I spotted right away. I noticed that the way the Windows batch files were constructed they will not work correctly if they’re not run from the correct directory. I mention this for others who may try to use EXRM on Windows.

Spaces In Directory Names

I noted also that there were several places where directories containing spaces and too many characters were also a problem. The original developers of the batch files wrapped double quotes around all the directory names. If you decide to build Windows batch files, please note that wrapping double quotes around directory names is a bit error prone and tough to get right.  Hence I opted to modify all the various places where they had double quoted directory names to using 8.3 versions of the names.  That fixed up all the references to directories so that everything was found correctly.

Getting Services Working

Next there was the question of getting Windows Services working for Erlang. While digging into the batch files I discovered that basically the issue hinged on the installation of the service.  Once the service was installed correctly, it looked as if the rest of the operations on the services (starting, stopping, restarting etc.) would work correctly.

I had to hunt around for a while to figure out what I needed to do to get the service to install correctly.  I finally figured out that what was happening was the wrong parameter was getting passed as the binary to be started when the service gets started.  And so I repaired it.

A Few More Details

You’ll also need to unblock the erl.exe and epmd.exe files for the Windows Firewall.  You can wait for Windows to prompt you or you can proactively unblock them via this escript.

You may also need to deploy msvcr100.dll to your deployment machine as well.  If it’s a Windows 10 machine you may need to do this.

So we’re almost ready to roll the changes into EXRM so we can get people to be able to deploy Elixir apps on Windows.

I will continue to work with Paul on improving support for EXRM on Windows.  Keep watching this space; we’ve got some interesting ideas we’ll be trying to roll out in the next few months or so.

 

 

 

Three Rules Of All Testing

There are three rules applicable to all testing environments.  I’ve articulated one of the rules before but now I’ll add two more.

Rule 1 – Dead Simple

Tests must be dead simple to run.  It doesn’t matter what you’re testing.  It doesn’t matter if you’ve got 100% code coverage. It doesn’t matter if you’ve tested everything from the smallest unit to the largest UI.  If the tests aren’t easy to run the people who most need to run them that is the developers and the analysts will figure out ways to game the system and avoid running the tests.  The rule is that running tests must be easier than any workaround the developers and analysts can come up with to avoid running the tests.

Rule 2 – Some Tests Are Better Than No Tests

I can’t count the number of times we’ve been discussing testing and someone will say “well, we can’t test X” with the implication being that since we can’t test everything there’s nothing to be gained from any testing.  This is analogous to saying the only way we can engineer a rocket is to build the whole thing and then fire it off.  Never mind building prototypes or doing any of that silly math to check our assumptions.

Now, granted there can be some cases where certain properties are so essential to a system that if they can’t be tested it really isn’t worth testing at all.  But those are extremely rare cases.  Even if you can only run tests on 50% of your code, that’s still 50% that’s tested that would not get tested otherwise.

Rule 3 – Don’t Test the Language or the Libraries

I’ve seen unit tests where someone will set some property of a class.  Then they will immediately read the property to check that is hasn’t changed.  While it’s a valid test to check an invariant on a class, it’s not valid to test it immediately after its set.  If you set a value and then read it and it’s different, you’ve got problems more substantial than a unit test will ever solve.

These are just my big three rules for testing.  I’d be interested to hear the thoughts of others on this subject.

Which Is Better A or B?

It often surprises me that people who spend their whole day focusing on details, software developers, so often ask question in broad and generic terms.  Here’s some that I’ve heard over the years:

  • Why should I learn Object Oriented Programming?  (I have been writing software for a while.  Long enough to remember the days before we discovered that OO was the only right way to write software.)
  • Won’t having a VM with Garbage Collection be really slow?  (This question came up a lot when Java was just being rolled out.)
  • Isn’t C# just Java with different keywords? (When C# was first rolling out.)
  • Why should I learn functional programming?  (One of the more recent vague and generic question I’ve gotten tons of.)

And today:

  • Will I be more productive if I write a website in Elixir/Phoenix than if I write it in (Web Framework X)?

Now consider some of the implied assumption in those questions.  All other things being equal will B (new technology) be better than A (which I’m already using)?

Remember we are discussing human beings performing a human activity–writing software.  Of anything humanity has ever tried to measure, quantify and generally treat in an empirical way, software development seems to be one of the least amenable to analysis.  Just some of the implied parameters of some of those questions above are:

a.) Is B more productive than A?

How do you measure “productive”?  Economists and experts on the subject of productivity don’t agree about measuring productivity in terms of hard goods (cars, appliances, etc.) so how should we measure productivity in terms of something we can’t even really measure?

b.) Will B allow me to create better software than A?

Define “better”.  Better in what sense?  Less errors?  Runs faster?  Less source code?  Easier for maintenance developers to understand?  More secure against hacking?

The point is this: while we wish we could answer the question of whether a new technology is better than an existing technology at all there’s so much ambiguity in so many facets of what we’re discussing there’s no way to even frame these questions in a way they can be discussed intelligently.  So for goodness sake–look for yourself and make your own decision.

Railway Oriented Programming In Elixir

A while back Scott Wlaschin posted a great blog post on what he called “Railway Oriented Programming.”  It’s a great treatment of a common problem in adopting functional programming; how to handle a chain of functions when one of the functions in the chain may return an error.  Of course stopping the chain when the first error is returned isn’t a difficult issue; the difficulty arises in structuring the code such that the code can continue running all the functions until and unless an error is encountered.  It’s a great read and a powerful idea.

So today someone asked a question on StackOverflow about how to accomplish basically that same idea in Elixir.  A little tricky to translate Scott’s ideas to Elixir because in his article his code is specified in F# and he created a special type to capture the success or failure of a function. F# is statically typed; elixir is dynamically typed which is, of course, a substantial difference.

At any rate, I saw the question and didn’t see that Saša Jurić had already posted a great answer.  So I posted my answer and then noticed his.  I mention this because I want to be clear; I wasn’t trying to steal anyone’s thunder or trying to enhance my rep or anything like that.  I just got excited; “hey, I know the answer to that question!” so I just rushed off and hacked together some demonstration code without bothering to look any further.  I am pleased to see that the answer I came up with was pretty much the same as Saša’s; I consider him an extremely smart developer and coming up with an answer pretty similar to his  independently makes me feel a bit more confident of my own skills.

The TL;DR version is that you need to construct the functions to be applied in such a way that you can carry forward an indication of either success or failure along with the value to be checked.

defmodule Password do

  defp password_long_enough?({:ok = _atom, p}) do
    if(String.length(p) > 6) do
      {:ok, p}
    else
      {:error,p}
    end
  end

  defp starts_with_letter?({:ok = _atom, p}) do
   if(String.printable?(String.first(p))) do
     {:ok, p}
   else
     {:error,p}
   end      
  end


  def password_valid?(p) do
    {:ok, _} = password_long_enough?({:ok,p}) 
    |> starts_with_letter?
  end

end

And one would use it like so:

iex(7)> Password.password_valid?("ties")
** (FunctionClauseError) no function clause matching in Password.starts_with_letter?/1
    so_test.exs:11: Password.starts_with_letter?({:error, "ties"})
    so_test.exs:21: Password.password_valid?/1
iex(7)> Password.password_valid?("tiesandsixletters")
{:ok, "tiesandsixletters"}
iex(8)> Password.password_valid?("\x{0000}tiesandsixletters")
** (MatchError) no match of right hand side value: {:error, <<0, 116, 105, 101, 115, 97, 110, 100, 115, 105, 120, 108, 101, 116, 116, 101, 114, 115>>}
 so_test.exs:21: Password.password_valid?/1
iex(8)>

A couple of comments on the code.  Someone might ask why password_long_enough?  gets a parameter of a tuple of atom and string. Since it’s the first function in the chain it’s not really needed; all it needs is the string.  I constructed in this way for a few reasons:

  1. It’s more parallel to the other function
  2. The functions are really intended to be rules which should be checked so there should not be an order dependency. That is, they should be interchangeable.

Basically by adding any predicate function that we wish given that it takes a tuple of an atom and a string and the atom is matched against :ok, we can add any arbitrary set of tests we wish to add.

As with most things on this blog, I write this down as much for my own reference as for the edification of others who might read it.  I’m pretty sure at some point in the future, I will need to validate a value through a set of predicate functions and I’ll look at this blog post to see how I can do it.


EDIT: I was reminded of a great blog post written up by Zohaib Rauf on precisely this same idea.  Well worth reading.

Configuring A WordPress Site On EC2

There’s an excellent and practical write-up on configuring an EC2 container to host a WordPress site here.  But there are a few minor amplifications I’d add.  So I thought I’d write them up here.

1.) As some folks noted in the comments, you should assign the Elastic IP address before you install WordPress.  Everything with WP just seems to work much better when you do it in that order.

2.) You’ll want to configure your instance to autostart both Apache and Mysql.  Credit where it’s due: I got this information from here.

  • sudo /sbin/chkconfig –add mysqld
  • sudo /sbin/chkconfig –list mysqld
  • sudo /sbin/chkconfig mysqld on
  • sudo /sbin/chkconfig –add httpd
  • sudo /sbin/chkconfig –list httpd
  • sudo /sbin/chkconfig httpd on

You really don’t need the –list step in either case.  It’s just a way to insure you’ve got chkconfig command right. Note that those commands are specific to the CentOS image that Amazon provides.  If you’re hosting a different flavor of Linux, you’ll need to figure out the commands for your flavor.

Other than those two minor corrections, his guide is sound and well-written.

Interesting And Unexpected Benefit Of Pattern Matching

So recently I’ve been working on automating some interactions with a website.  Elixir’s Hound library is a great help, of course.  But I also ran across an interesting and unexpected benefit to pattern matching that greatly simplified some code.

I was trying to dynamically build a URL to pass parameters. The URL would be in the form:

http://www.exampledomain.com?param1=p&param2=q&param3=r

Whenever I see code like this, it’s always a little tricky because you want to add the & at the end of each parameter except the last.  This always used to entail writing a special case in the loop to test if the index of the current element was the maximum index.  But with pattern matching and recursion there’s a much simpler and cleaner solution.  Like this:

defp add_params_to_url(url,[%{:name => name, :value => value}]) when is_binary(url) do     #1
 "#{url}#{name}=#{value}"
end
defp add_params_to_url(url,[%{:name => name, :value => value} | t]) when is_binary(url) do   #2
 url = "#{url}#{name}=#{value}&"
 add_params_to_url(url,t)
end
defp add_params_to_url(url,[]) when is_binary(url), do: url   #3

This is Elixir for those unfamiliar with it.  Basically it’s three function clauses.  The first clause (#1) is hit if only one element is in the list passed into the function. The second clause (#2) is hit if multiple elements are present in the list and the third clause (#3) is hit if the list is empty.

So this is how this works.  When I call add_params_to_url, Elixir will automatically try to match the correct function call dependent on which list I pass.  It will also stop at the first function clause which matches so that other function clauses will not be evaluated.

If I pass a list of URL params which has only one element, clause 1 is matched and the parameter and value are appended and I’m done.  The URL already has the ? on the end, of course.  If I pass a list of URL params with mutliple elements, clause 1 doesn’t match so Elixir jumps down to clause 2 and that matches.  Clause 2 takes the first element from the list tacks it on to the list of parameters with a & at the end and then recursively calls itself with the tail of the list.  If the tail of the list contains more than one parameter, clause  1 will once again fail to match and clause 2 will match again.  If it contains only one parameter, clause 1 is matched.  Because clause 1 doesn’t call itself recursively, clause 3 should never be matched; it’s actually more of a guard case in case someone calls the function with an empty parameter list by mistake.

This, to me, is a small bit of genius.  Much simpler to get exactly the effect I want, that is not having an extraneous character tacked to the end of the string, and it’s nice and simple.

By the way, it occurs to me that I might be able to use a reduce or a fold function to achieve this same effect.  The issue there is that it’s less apparent what it is that I’m trying to do and I also run into the same problem–that is, how do I deal with that last element in the list so I don’t get my separator appended?