Understanding Contramap

Recently I've been trying to get my head around what contravarient functions and contramap are used for in practice. In particular I've been searching for some practical examples that turn the types of the function into something concrete. This all started because after implementing a lot Json parsing / writing using Play's Json libraries. In doing so I'd periodically have to use contramap to implement a Write and I wanted to understand what it actually did.

This led me to here: http://blog.tmorris.net/posts/functors-and-things-using-scala/index.html. and here: http://igstan.ro/posts/2013-10-31-contravariant-functors-an-intuition.html.

And now my own attempt to explain.

The problem:

The thing that confused me straight up was this:


Being relatively new to functional programming my mind jumps straight to lists when I think about functors. How can you have a List[Int], a function that converts a String to Int and then somehow generate a List[String]? The function is backwards, this can't be possible? The problem is lists are applicable for the covariant map function, but not contramap.

An example:

Below I take what I've learnt from the above blogs and apply to a very basic serialisation. For me this finally made the idea of contramap stick. But before we talk about types and functions I wanted to try and talk about the concrete example.

Serialsation is pretty common concept. If we look at Json, the idea of taking a string of characters in a generic structure and converting it into an object in memory well understood. When we talk about reading Json we are implementing an abstraction that is covariant and which implements map. For example, if you a have a container (let's call it a Reads[T]) that can read a string into a JsObject (so this would be Reads[JsObject]), and you have a function that can convert a JsObject into the type you are interested (let's call it Company), then it should be possible to create Reads[Company]. The function would look something like:

The implementation of this would probably use the underlying Reads[JsObject] to convert our serlizaed format to a JsObject and then our function to convert the JsObject to Company. So this matches the definition of a covariant map:

But what about contramap?

For this example, let's forget about JsObject and Json and take a really simple example of serializing an object to a string. Let's define a very basic serialisation function.

Along with this we create a simple class and it's serialisation implementation.
Very basic. Just print out a key and a value. Now let's implement contramap. This will allow us to serialise any object using only the traits defined above.


Which matches the contrmap functor defined by Tony. In this case A is Writeable and B is Test and F is writes. If we switch these out we have:

And finally we can define ourselves a Writes[Test] using contramap.

Scala: Functions can be Keys of Maps

This came up in a discussion we had recently about what could be a key for a map in various languages. I jokingly suggested that we should take it to the extreme and have a function as a key for the map. On second thought, there was no reason we shouldn't be able to do this in Scala.

And it turns out it's possible with some limitations. I'm not sure why you'd do this. My only use case was to do with dependency injection. If you have a function what accepts a function, there might be a case where you'd like to know what implementation you were given. In most case you would not (and probably should not) care: that's the whole reason for injecting the implementation. Having said that, you may want it for logging purposes so you can tell what implementation was used.

Anyway, the use case doesn't really matter: this was all about can you, not would you.

So here's the REPL output of my little experiment. The big limitation here is that the function needs to be assigned to a val for this to be useful, and then that val must be used whenever the function is invoked. This seems intuitive to me: without having the actual function implementation, we can't tell at runtime whether an anonymous function or partially applied function is identical to a function assigned to a val.

scala> def add(a: Int, b:Int) = a + b
add: (a: Int, b: Int)Int

scala> val addFunc = add _
addFunc: (Int, Int) => Int = 

scala> def sub(a: Int, b: Int) = a - b
sub: (a: Int, b: Int)Int

scala> val subFunc = sub _
subFunc: (Int, Int) => Int = 

scala> val funcMap = Map(addFunc -> "add", subFunc -> "sub")
funcMap: scala.collection.immutable.Map[(Int, Int) => Int,String] = Map( -> add,  -> sub)

scala> def operation(a: Int, b: Int, op: (Int, Int) => Int, opMap: Map[(Int,Int) => Int, String]) = {
     |   val opString = opMap.getOrElse(op, "Unknown Function")
     |   println("Function: " + opString + " a: " + a + " b: " + b)
     |   op(a, b)
     | }
operation: (a: Int, b: Int, op: (Int, Int) => Int, opMap: Map[(Int, Int) => Int,String])Int

scala> operation(1, 2, addFunc, funcMap)
Function: add a: 1 b: 2
res0: Int = 3

scala> operation(1, 2, subFunc, funcMap)
Function: sub a: 1 b: 2
res1: Int = -1

scala> operation(1, 2, (a: Int, b: Int) => { a * b }, funcMap)
Function: Unknown Function a: 1 b: 2
res2: Int = 2

scala> operation(1, 2, add _, funcMap)
Function: Unknown Function a: 1 b: 2
res3: Int = 3

Ruby services in Docker using Passenger

This weekend I was working a little project I wanted to deploy to Digital Ocean. The app was a simple server written Ruby using Sinatra to server some JS. I already had an instance running in Digitual Ocean with Docker installed, so I figured I'd just copy one of my existing docker files and deploy it that way.

Having a look at the Dockerfiles I've used for my existing projects, I realised they all used the standard Ubuntu base and then used a dreaded curl | sudo bash install. I'd recently read this hacker news article about the state of sysadmin in a world with containers, and whilst I disagree with many of the points, I thought it was time to look for a more robust way of dealing with containers.

Enter Phusion and their base-image and passenger dockerfiles. Phusion aims to provide a set of stable, secure and better setup docker images then what most people would be used to. In my case this was definitely true.

The base-image is a standard ubuntu build with a number of tweaks to make it more docker friendly. The main thing I picked up on is they use a version of runit as a lightweight process supervisor to manage the processes running in your container.

The passenger images are aimed at application deployment. There are number of builds for different version of ruby, node etc, and it also comes with Nginx and a few other services bundled together.

Getting down to it, I decided I'd use the ruby21 passenger with everything disabled (ie., Nginx) and just use runit to start my app. This seemed to be the best way to get things up and running as I already had scripts setup to run my app using unicorn. I didn't really want to port the app over to use Nginx and Passenger just yet.

So my Dockerfile ended up looking like this. It's very simple which I like.

My run script was as follows:

And my start script simply called unicorn:

A few things to note:
* Don't put an Entry point in your file.
* Instead use the runit and my_init command that is built in. Basically, you use my_init to start and monitor your daemons rather than using an entry point.

The doco explains how to do this (add a script to /etc/service/my_service_name/run) and it pretty much works out of the box

... except it didn't. On running the service and tailing the logs I got the following:

*** Booting runit daemon...
*** Runit started as PID 9
Rack env: production
Running using: 8080 and rack env: production
Apr 26 05:58:08 67f2f5a43939 syslog-ng[21]: syslog-ng starting up; version='3.5.3'
Rack env: production
Running using: 8080 and rack env: production
master failed to start, check stderr log for details
Rack env: production
Running using: 8080 and rack env: production
master failed to start, check stderr log for details
Rack env: production
Running using: 8080 and rack env: production
master failed to start, check stderr log for details

So the runit process wasn't picking up that service was running, and was constantly trying to restart it. This is obviously not the desired behaviour.

I spent a fair bit of time googling around and reading the doco for the base-image. I also looked at runit in more detail to understand what was going on. It wasn't until I re-read the passenger page that I picked up on this:

Note that the shell script must run the daemon without letting it daemonize/fork it. Usually, daemons provide a command line flag or a config file option for that.

Oh. Right. runit wants to manage the process itself. It doesn't want to be looking for a pid file associated with a daemon that is already running as a daemon. It wants to treat it as a process. The solution is to drop the -D out of the call to unicorn as follows:

And there you go. Now I have a small Dockerfile from a well maintained repository that I can easily use to run any kind of ruby service, be it a worker or a web app.

TL;DR: If you have a ruby process you want to run using passenger--don't daemonize it!

Scala Functional Toolkit: Handling exceptions in for comprehensions

As we develop more and more code in Scala, we are trying to stick to a few key principles. One of these is to avoid throwing Exceptions at all costs. In pretty much every case, this can be avoided by using Options or Eithers, however, sometimes it gets a little messy.

One such example we came across recently is when you are dealing with Futures, in particular doing something like a web request.

For example: Let's say you want to request some data from an API using Play's WebService library (or any HTTP library). Once you've got the result back you want to check whether the response was  a 200 or not, if so, you want to parse the JSON body. Finally, this all should be happening in side a Future so that it doesn't block your code.

In this case, the initial Future may succeed, but contain a 400, in which case you still want to fail.

Here is a naive implementation:

This is the kind of thing we don't want to do. Regardless of how it is handled by the caller, we'd prefer not to throw Exceptions anywhere in our code.

So we can get around this using Future.failed, however, as we are failing the future once the response has been received (that is the initial request future has now completed) we need to use flatMap instead of map. Otherwise our failure case would be Future[Future[JsValue]] and our success case would be Future[JsValue].

Ok. This is better. We are no longer throwing exceptions. We are now using the Future.failed method instead. However, it is a bit clunky. We can make this a slightly nicer using a for comprehension, however to do this we need to create a predicate method:

This method takes a boolean condition and then depending on the result either evaluates the success block, or returns a failed future with the exception specified. Using this we can now re-write the function as follows:

You can play around with this a little to create more versatile or specific variants. Whilst in the end this is all syntactic sugar, it has been a useful in making our code more concise.