QCon London – Good Fun

I had great fun at QCon. This was one of my first “commercial” conferences – I’ve mainly frequented research conferences like ICFP and POPL before, partly because Erlang has enjoyed little commercial interest in the past.

I enjoyed Steve Vinosky’s account of the conference, as well as Ola Bini’s. I also enjoyed Steve’s talk on RPC, and Paul Downey’s on standardization in the Historically Bad Ideas track. And of course, listening to Sir Charles Anthony Hoare was a great treat. I’m a long-time fan of Hoare’s and warmly recommend his Turing Award Speech, “The Emperor’s Old Clothes”. There are some great lessons in there.

On Thursday, I stuck with the “Functional and Concurrent Programming Languages Applied” track, hosted by my friend and colleague Francesco Cesarini. When speaking in a track, I want to have heard the talks before mine, and also want to stay to hear the remaining talks.

This time, I was preceded by Rich Hickey’s talk on Persistent Data Structures and Managed References in Clojure. I was greatly intrigued by this talk and made a note to take a closer look at Clojure. Interestingly, Rich stressed that Clojure Agents are not actors a la Scala or Erlang. Part of my talk was to illustrate how Erlang processes are not “fibers”, and trying to use them for data-parallel algorithms can be problematic. Nice setup.

Friday ended with a few quick beers with Rich, Steve, Paul and Joe Armstrong, and then dinner at Francesco and Alison’s with Joe and Steve. Francesco is a great cook!

(Didn’t notice that Paul was taking pictures though. Judging by the angles, he was shooting from the hip…)

I'm speaking at the Erlang Factory

I’ll give the same talk at SF Bay Area Erlang Factory, April 27-29. I’m really looking forward to this conference. It should be the greatest Erlang conference yet, and I hope to see you there.

Sourdough on the rise

A major Swedish newspaper had a feature about sourdough today. Apparently, it’s becoming quite popular, with more and more small bakeries selling exclusive sourdough bread and giving courses on how to make your own sourdough.

Just having returned home from QCon in London, where there were some great talks about, and significant interest in, functional programming, I’m reminded of my own keynote from the ACM SIGPLAN Erlang Workshop in Tallinn 2005. It was called “Sourdough and Erlang”, and as much as I would like to flatter myself by calling it “prophetic”, I’m sure it would just tempt others who were there to propose some other, more fitting descriptions. I feel this is as good a time as any to finally release the slides. Perhaps it’s at least good for a few laughs. I still think that “stalking the wild yeast” is an outstanding metaphore for creative software development, though…

Micro-fiddling with erlhive

Every once in a while, my thoughts touch upon erlhive. While it may seem that the project isn’t going anywhere, I’m still pretty comfortable with it. It’s not as if I started it for the money, anyway, and the problems it addresses are real and interesting.

Someone mentioned to me that erlhive had been dismissed in a discussion with the argument the benefits are not worth a throughput of 50 pages per second.

I’m not sure where the 50 pages per second comes from, and in honesty, I’ve cared so little about performance that I haven’t benchmarked it myself. So, just to satisfy my curiosity, I ran a little speed test…

I chose to benchmark only the blog back-end, partly because erlhive (oddly enough for a “web framework”) doesn’t really have any web front-end yet. However, the back-end does have a functional API, and the front-end part can be made arbitrarily fast or slow, depending on context anyway.

Here is my little benchmark program:

fetch(N) when N > 0 ->
  erlhive:with_user(
    <<"user1">>,
    fun(M) ->
       Id = M:apply(erlhive.blog.flexiblog,
                    string_to_id,
                    ["ba/user1/1/1"]),
       [_|_] = M:apply(erlhive.blog.flexiblog,
                      read_article, [Id])
    end),
  fetch(N-1);
fetch(_) ->
  done.

Clocking this with timer:tc() in coLinux/Vista on my 2.17GHz dual-core Pentium laptop (not using SMP erlang, though), I achieve 1450 article fetches/second. That’s a lot better than I had expected. Erlhive, apparently, is not so slow after all.

To explain what the above does, I assume that the surrounding web framework has authenticated the user (<<“user1”>> in this case), and that there exists an article 1 in blog 1 of the user. All objects in erlhive can be given user-defined properties, so coming up with more creative names is perfectly possible. However, the blog example uses the following naming convention:

  • [blogs,User,N] to identify a blog
  • [ba,User,BlogN,ArticleN] to identify an article
  • [bc,User,Blog,ArticleN,CommentN] to identify a comment

The call, erlhive:with_user(User, Fun) starts a database transaction, verifies that User is a known erlhive user, and creates an “access record” in the form of a parameterized module M. The user-provided fun is called with Fun(M), and through the module M, the whole erlhive API is available. Any function called through M (e.g. using M:apply(Mod,Func,Args)) is subject to erlhive’s access control. Of course, anything else in Fun/1 is still ordinary Erlang, and can be as “dirty” as you like. M is the entry point to erlhive, and erlhive:with_user/2 creates the transaction context.

The first call, to erlhive.blog.flexiblog:string_to_id(“ba/user1/1/1”), simply translates a web-friendly identifier to the internal form, [ba,<<“user1”>>,1,1]. The second call, to erlhive.blog.flexiblog:read_article(Id), performs the following steps:

  • Checks that the blog, [blogs,<<“user1”>>,1] exists and that the current user (<<“user1”>> in this case) has read access to it.
  • Reads the properties of the article, and verifies that the current user is authorized to perform the requested operation.
  • Returns the property list associated with the article. The actual article content is also a property, tagged ‘content’.

The blog example is reasonably clever for a blog module. It allows users to be categorized as ‘admin’, ‘author’, ‘reviewer’ and ‘reader’. A reviewer can read and comment on an article before it has been published. You can list blogs and articles by property – all properties are automatically indexed, and you can also clone an article, as a form of revision handling. It is also possible to view previous/next in a revision succession.

All this in less than 1000 lines of code. I do feel pretty comfortable with that.

There is also a short text file that illustrates the blog interface through the interactive erlhive shell. Not exactly stellar documentation, but if I get some indication that there are potential users out there, I promise that it will improve.

BTW, in the course of this experiment, I noticed that the interactive shell was broken, and that the read_article/1 function actually didn’t exist. Bug fixes have been committed to the repository. I never did promise that it was production-quality, did I?