Text

Open Sourcing PEPS: A modern webmail server

There was some discussion yesterday on Hacker News when I wrote in a comment that we are considering releasing a new open source project.

Screenshot of PEPS hn account

PEPS is secure webmail server with a rich UI, with the goal of being an open source GMail alternative. No doubt there is a huge interest in having such project as an open source product. We can (and will) speak about the product in other posts.

We began its development two years ago for a major European customer, but the deal was structured so that we own the IP, meaning in particular that we are free to open source it.

We love open source, and already have a strong open source commitment as we researched, built and open sourced the Opa technology. Choosing the right license for Opa wasn’t an easy task and we changed the licensing along the way. As a company, we need revenue to live and grow. As open source lovers, we want our users to be able to access the source code of all the software they use.

I also read the Snowden disclosures as an important reminder that the source code of the systems and applications we use should be publicly accessible. But at the same time, how can we get revenue from open source?

I know that the subject has been debated many times. I will spare you the study of thousands of links to such previous studies to focus on what I consider three solutions to open source business.

MIT + Proprietary

MIT is a great open source license. And at first sight, a difficult way to make a business.

Most of Opa is MIT-licensed. To monetize MIT software, the easiest way is to develop “Value-Added” paid extensions that you can sell as a bundle with the product to customers, together with support. For instance, Opa has support for ActiveSync, but this part of the standard library is not open source.

Applied to PEPS: We can provide a MIT license of a modern webmail, hopefully with a vibrant community and contributions. And, as a company, we will sell “PEPS Pro” ready-to-use server images together with extensions such as the aforementioned ActiveSync support.

Pros:

  • MIT license is very popular
  • helps building a developer community

Cons:

  • Parts of the source code is not publicly accessible
  • Two versions of the product to maintain

AGPL + Support

The second solution is to release everything as open source under the Affero GPL (AGPL) license. The license itself is not very business-friendly as the AGPL imposes to release the source code from the product itself. To escape the license restrictions, companies can buy a proprietary license of the same product, together with support.

The AGPL solution seems great: Only one product to maintain, the whole code is publicly accessible and many companies will have a strong enticement to buy.

We originally chose the AGPL for Opa but this choice brought us an unexpectedly high number of haters. Back then, Opa was not yet a polished technology, so the combination of a rough product and an unpopular license might have been the trigger of their discontent. PEPS is not Opa: It’s a product and not a technology. Should we try AGPL again?

Pros:

  • one product to maintain
  • source code fully publicly accessible

Cons:

  • AGPL license is not very popular
  • there might be less contributors to the codebase

Source Access

The third and last solution is not even open source according to the OSI definition: Source access.

Provide every user (or even non-user) with the source code, but don’t necessarily provide the right to use the software in production for free. For instance, we can make the software free to demo for anyone and free to use for individuals but paying for companies (again with support).

Users may even have the right to distribute their modified versions of the product, given that the end user acquires a license from us. Even better, we could pay back some of the developers of modified versions and make them partners.

Pros:

  • one product to maintain
  • source code fully publicly accessible
  • value easy to understand: Software is not free

Cons:

  • who wants to create a new license?

Discussion

The choices are infinite but according to our experience, it boils down to one of these choices. You are welcome to discuss your thoughts in the comments. We haven’t made a choice yet, your feedback could make the decision!

  • The demo system is a new temporary link that enables account creation. Don’t use for real, data will be wiped.

EDIT: Awaiting the release, PEPS now has a home: https://github.com/MLstate/PEPS

Text

Power Rows, part 2

Last week, I explained how the Power Rows bring together the easiness of records (named objects in JavaScript) and the cleanliness of strongly statically typed records. And we told there was more to it. Some expert readers indeed knew that the features we mentioned in part 1 are something along the lines of old research papers. What is unique about Power Rows, hence the name, is their ability to also handle variants.

Variants?

Also called tagged unions, variants are a way to express that a given value is either of this type, or another. For instance, we can express that a value of type material is either

wood or metal or plastic

In OCaml, the above type is defined as:

type material = Wood | Metal | Platic

and used like this:

let value = Wood

let burns = function
| Wood | Plastic -> true
| Metal -> false 

Combined with type variables, they become a very powerful way to construct datastructure. For instance, the type option, defined as

type 'a option = None | Some of 'a

allows to build options of something (something being named ‘a here)

let winner = Some "foo"
...
let congrats = function
| Some winner -> printf "Congratulations %s" winner
| None -> printf "No winner. Try again!"

This is much nicer and proper code than something like:

var winner = "";
...
function congrats(winner) {
    if (winner == "") { return "No winner. Try again!" }
    else return "Congratulations " + winner
}

because we might have an empty name given by the user or any other reason which would result in displaying the “No winner” message in error.

Variants in Opa

Opa supports variants straightforwardly, without even dealing with types

function burns(m) {
    match (m) {
        case {wood}
        case {plastic}: true;
        case {metal}: false;
    }
}

Variants in Opa are polymorphic, meaning that you can reuse labels such as { wood } in other types. The beauty of variants is that if you somewhere write:

burns({ leather })

the compiler will complain before you can run the program, at compile time, that

The argument of function burns should be of type
    { metal } / { plastic } / { wood }
instead of 
    { leather } / 'c.a

When adding a new variant to a type, it’s extremely convenient that the Opa compilers will pinpoint every place in your application code that needs to be updated to handle the new case. To do so, you can introduce explicitely the type

type material = { wood } or { plastic } or { metal }

and enforce additional checks by stating that function burns argument is of type material

function burns(material m) {

Then, if you update type material,

type material = { wood } or { plastic } or { metal } or { leather }

but not the code of function burns, the Opa compiler will complain that

File "material.opa", line 5, characters 28-107, (5:28-11:1 | 136-215)
Incomplete record pattern matching: case {leather} is missing

You too can play with the gist.

Power Rows = Extensible Records + Variants

You may have noticed that the syntax of Opa variants is the same as the syntax of records. They are indeed the same construction!

And you can use them in rows +and+ columns, hence the name of Power Rows (and later, the now abandoned nickname of Power Rangers). Let’s have a look at a real use of their features combined. Starting a new server with Opa, like in the Hello World application is written

Server.start(Server.http, 
    { title: "Hello, world", 
      page: function() { <>Hello, world</> } })

In other applications, the server can be called in other ways such as

Server.start(
    { Server.http with name : "liveroom" },
    { custom: dispatcher })

The magic behind it that in the Server.start function, we see that its second and main argument is a Server.handler defined as

type Server.handler =
   { { string text } }
or { { string title, (→ xhtml) page } }
or { { Parser.general_parser(resource) custom } }
or { { (Uri.relative → resource) dispatch } }
or { { Server.filter filter, (Uri.relative → resource) dispatch } }
or { { stringmap(resource) resources } }
or { { Server.registrable_resource register } }
or { list(Server.handler) }

Omitting the types of each element, a Server.handler can be represented in a table like this

first element second element
text
title page
custom
dispatch
filter dispatch
resources
register
nil
hd tl

Regular records are represented as rows, variants as columns. And the might of Power Rows is that you can combine both at will, while benefiting from unobtrusive strong, static typing.

Some of you might ask: But what are hd and tl in the last row and nil in the row above? Good question! They are the construct of a list in Opa, where hd is the first element of the list (or head), and tl the rest of the list (or tail). And nil matches the case of the empty list. From a user point of view, Server.handler can also be a list of handlers. With syntactic sugar:

[ handler1, handler2, handler3 ]

That’s all Folks!

Text

Power Rows (the datatype)

When I conceived Opa, one of the key features was the support of Power Rows. Power Rows are a powerful, statically-typed, extension of records and it’s one of the features that makes static programming as fun as dynamic programming.

This is the first part of a two parts article. Part 2 is here.

(An early version of this post mentioned the internal name of Power Rangers which were given to them as a joke, as it sounded close to Power Rows. We’re back to Rows, the original name of the construct in Semantics of PL.)

The problem

Dynamic programming is cool

For instance, in JavaScript, we can write:

> a = { name: "John", surname: "Doe" }
[object Object]

to define a record (indeed an object) with two fields. We can access a field easily:

> a.name
John

and add new fields:

> a.age = 33
33
> a.age
33

Even better, we can define a function that accesses a given field for any value, no matter what the other fields are:

> b = {name: "Mary"}
[object Object]
> f = function(x) { return x.name }
function (x) { return x.name; }
> f(a)
John
> f(b)
Mary

This is the flexibility that we all love about dynamic programming. And although I have been coding mostly in OCaml since 2001, I also did quite big amounts of Python, JavaScript, PHP and loved this part of dynamic programming.

Static programming is boring

Compared to JavaScript, records in OCaml suck. Let’s have a look:

# let a = { name = "John"; surname = "Doe" };;
Error: Unbound record field label name

That’s a good start! Not mentioning the terrible syntax, we have to define types manually.

# type person = { name : string; surname: string };;
type person = { name : string; surname : string; }
# let a = { name = "John"; surname = "Doe" };;
val a : person = {name = "John"; surname = "Doe"}

What if we want to add a field? Well, we can’t. The best we can do is to create a new type that binds together a person and its age.

# type person_with_age = { person : person; age : int };;   
type person_with_age = { person : person; age : int; } 
# let aa = { person = a; age = 33 };;
val aa : person_with_age = {person = {name = "John"; surname = "Doe"}; age = 33}

But then, it becomes cumbersome to use. What if we want to write a function that takes a person or a person_with_age? Well, we can’t. We can write a sum-type with a constructor:

# type any_person = P of person | PA of person_with_age;;
type any_person = P of person | PA of person_with_age

and use pattern-matching to deconstruct an any_person:

# let get_name = function
  | P p -> p.name
  | PA pa -> pa.person.name;;
val get_name : any_person -> string = <fun>

But then, our program has to use any_person all over. Of course, OCaml has objects which you can use instead, but they are much harder to construct and handle.

So, all the fun of programming, the flexibility of JavaScript is gone. If you read through here, you might wonder why I was writing OCaml. I am not a masochist and chose the best tool for the work I had to do.

But strong static types rock

When writing complex programs, strong static typing (with inference, like OCaml does) is extremely handy to write high-quality code, and spend a lot less time debugging. Especially when the size and complexity of programs grow.

Back to JavaScript, we get undefined when accessing a field which is not present.

> f({})
undefined

That’s great. But computation continues… So I could have:

> "Hi " + f({})
Hi undefined

And when, again, my program gets bigger, it becomes harder and harder to find the bugs timely (ideally, just when you write them). And it becomes potentially a nightmare to iron out some bugs. In OCaml,

# type another = { another: string };;
type another = { another : string; }
# let v = { another = "foo" };;
val v : another = {another = "foo"}
# get_name(v);;
Error: This expression has type another but an expression was expected of type any_person

Not only we get the error during compilation (i.e. before we run, perfect timing), but the error allows us to locate the bug easily.

When we learn we can trust the compiler, we have the satisfaction to write high-quality code when it compiles.

The solution

So, for a few years, I dreamed about what we now have built in Opa: Power Rows.

Power Rows are the best of both world. The flexibility of records from dynamic programming languages, and the type checker that goes with it.

This is how it works when you write code (there is no REPL in Opa yet, you have to print return values when you run your app).

a = { name: "John", surname: "Doe" }

a.name

Same JavaScript syntax. No type declaration beforehand ala OCaml.

It’s slightly different to add a field:

a = { a extend age: 33 }

a.age

but it’s much better to enforce the extend semantics so we can check for instance that a.nam = "Jack" is not a typo! But again, no hard declaration. It just works.

b = {name: "Mary"}

// there is no return in Opa, the last statement is the return
f = function(x) { x.name }

f(a)
f(b)

Gives us the expected result. It’s as simple as JavaScript and it just works. But unlike JavaScript, we know we write good code, when the compiler straightforwardly rejects, at compile time, the following mistake:

c = { foo: "bar" }
f(c)

with the error message (again, at compile time, before runtime):

Type Conflict
  (8:17-8:26)         { name: 'a; 'r.a }
  (12:5-12:18)        { foo: string } / 'c.a

The argument of function f should be of type
{ name: 'a; 'r.a }
instead of 
{ foo: string } / 'c.a

See, the best of both worlds! Gist is here.

There’s more to it

Power Rows might doesn’t stop here. More is written in a subsequent blog post. Meanwhile, you can play with the Opa Power Rows if you feel like it.

I also didn’t plan to explain how our type checker works in this post, but let me tell you it was a very complex task. If you are doing language research, there is no paper yet but the OCaml implementation is hidden in here.

Read on to Part 2

Photo
skeuit:

I KNEW I’D LIVE TO SEE THE DAY THAT PEA COAT INNOVATION MADE ITS WAY TO APP UI.  THANKS @dribbble!
via @ddefenba

I&#8217;d love to see the iOS UI where you struggle to put it on or off. Like a kid protection where you need to ask your kid.

skeuit:

I KNEW I’D LIVE TO SEE THE DAY THAT PEA COAT INNOVATION MADE ITS WAY TO APP UI.  THANKS @dribbble!

via @ddefenba

I’d love to see the iOS UI where you struggle to put it on or off. Like a kid protection where you need to ask your kid.