In the previous post I described the continuous end-to-end system
that we’ve set up for some of the MirageOS projects — automatically going from
git push all the way to live deployment, with everything under
Everything I described previously already exists and you can set up the workflow for yourself, the same way many others have done with the Travis CI scripts for testing/build. However, there are a range of exciting possibilities to consider if we’re willing to extrapolate just a little from the tools we have right now. The rest of this post explores these ideas and considers how we might extend our system.
Previously, we had finished the backbone of the workflow and I discussed a few ideas about how we should flesh it out — namely more testing and some form of logging/reporting. There’s substantially more we could do when we consider how lean and nimble unikernels are, especially if we speculate about the systems we could create as our toolstack matures. A couple of things immediately come to mind.
The first is the ability to boot a unikernel only when it is required, which opens up the possibility of highly-elastic infrastructure. The second is the ease with which we can push, pull or otherwise distribute unikernels throughout a system, allowing new forms of deployment to both cloud and embedded systems. We’ll consider these in turn and see where they take us, comparing with the current ‘mirage-decks’ deployment I described in Part 1.
The way cloud services are currently provisioned means that you may have services operating and consuming resources (CPU, memory, etc), even when there is no demand for them. It would be significantly more efficient if we could just activate a service when required and then shut it down again when the demand has passed. In our case, this would mean that when a unikernel is ‘deployed to production’, it doesn’t actually have to be live — it merely needs to be ready to boot when demand arises. With tools like Jitsu (Just-In-Time Summoning of Unikernels), we can work towards this kind of architecture.
Jitsu allows us to have unikernels sitting in storage then ‘summon’ them into existence. This can occur in response to an incoming request and with no discernible latency for the requester. While unikernels are inactive, they consume only the actual physical storage required and thus do not take up any CPU cycles, nor RAM, etc. This means that more can be achieved with fewer resources and it would significantly improve things like utilization rates of hardware and power efficiency.
In the case of the decks.openmirage.org unikernel that I discussed last time, it would mean that the site would only come online if someone had requested it and would shut down again afterwards.
In fact, we’ve already been working on this kind of system and Jitsu will be presented at NSDI in Oakland, California this May. In the spirit of looking ahead, there’s more we could do.
At the moment, Jitsu lets you set up a system where unikernels will boot in response to incoming requests. This is already pretty cool but we could take this a step further. If we can boot unikernels on demand, then we could use that to build a system which can automate the scale-out of those services to match demand. We could even have that system work across multiple machines, not just one host. So how would all this look in practice for ‘mirage-decks’?
Our previous toolchain automatically boots the new unikernel as soon as it is pulled from the git repo. Using Jitsu, our deployment machine would pull the unikernel but leave it in the repo — it would only be activated when someone requests access to it. Most of the time, it may receive no traffic and therefore would remain ‘turned off’ (let’s ignore webcrawlers for now). When someone requests to see a slide deck, the unikernel would be booted and respond to the request. In time it can be turned off again, thus freeing resources. So far, so good.
Now let’s say that a certain slide deck becomes really popular (e.g. posted to HackerNews or Reddit). Suddenly, there are many incoming requests and we want to be able to serve them all. We can use the one unikernel, on one machine, until it is unable to handle the load efficiently. At this point, the system can create new copies of that unikernel and automatically balance across them. These unikernels don’t need to be on the same host and we should be able to spin them up on different machines.
To stretch this further, we can imagine coordinating the creation of those new unikernels nearer the source of that demand, for example starting off on a European cloud, then spinning up on the East coast US and finally over to the West coast of the US. All this could happen seamlessly and the process can continue until the demand passes or we reach a predefined limit — after all, given that we pay for the machines, we don’t really want to turn a Denial of Service into a Denial of Credit.
After the peak, the system can automatically scale back down to being largely dormant — ready to react when the next wave of interest occurs.
If you think this is somewhat fanciful, that’s perfectly understandable — as I mentioned previously, this post is very much about extrapolating from where the tools are right now. However, unikernels actually make it very easy to run quick experiments which indicate that we could iterate towards what I’ve described.
A recent and somewhat extreme experiment ran a unikernel VM for each URL. Every URL on a small static site was served from its own, self-contained unikernel, complete with it’s own web server (even the ‘rss.png’ icon was served separately). You can read the post to see how this was done and it also led to an interesting discussion on the mailing list (e.g. if you’re only serving a single item, why use a web server at all?). Of course, this was just an experiment but it demonstrates what is possible now and how we can iterate, uncover new problems, and move forward. One such question is how to automatically handle networking during a scale-out, and this is an area were tools like Signpost can be of use.
Overall, the model I’ve described is quite different to the way we currently use the cloud, where the overhead of a classic OS is constantly consuming resources. Although it’s tempting to stick with the same frame of reference we have today we should recognise that the current model is inextricably intertwined with the traditional software stacks themselves. Unikernels allow completely new ways of creating, distributing and managing software and it takes some thought in order to fully exploit their benefits.
For example, having a demand-driven system means we can deliver more services from just the one set of physical hardware — because not all those services would be consuming resources at the same time. There would also be a dramatic impact on the economics, as billing cycles are currently measured in hours, whereas unikernels may only be active for seconds at a time. In addition to these benefits, there are interesting possibilities in how such scale-outs can be coordinated across different devices.
As we move to a world with more connected devices, the software and services we create will have to operate across both the cloud and embedded systems. There have been many names for this kind of distributed system, ranging from ubiquitous computing to dust clouds and the ‘Internet of Things’ but they all share the same idea of running software at the edges of the network (rather than just cloud deployments).
When we consider the toolchain we already have, it’s not much of a stretch to imagine that we could also build and store a unikernel for ARM-based deployments. Those unikernels can be deployed onto embedded devices and currently we target the Cubieboard2.
We could make such a system smarter. Instead of having the edge devices constantly polling for updates, our deployment process could directly push the new unikernels out to them. Since these devices are likely to be behind NATs and firewalls, tools like Signpost could deal with the issue of secure connectivity. In this way, the centralized deployment process remains as a coordination point, whereas most of the workload is dealt with by the devices the unikernels are running on. If a central machine happens to be unavailable for any reason, the edge-devices would continue to function as normal. This kind of arrangement would be ideal for Internet-of-Things style deployments, where it could reduce the burden on centralised infrastructure while still enabling continuous deployment.
In this scenario, we could serve the traffic for ‘mirage-decks’ from a unikernel on a Cubieboard2, which could further minimise the cost of running such infrastructure. It could be configured such that if demand begins to peak, then an automated scale-out can occur from the Cubieboard2 directly out onto the public cloud and/or other Cubieboards. Thus, we can still make use of third-party resources but only when needed and of the kind we desire. Of course, running a highly distributed system leads to other needs.
When running services at scale it becomes important to track the activity and understand what is taking place in the system. In practice, this means logging the activity of the unikernels, such as when and where they were created and how they perform. This becomes even more complex for a distributed system.
If we also consider the logging needs of a highly-elastic system, then another problem emerges. Although scaling up a system is straightforward to conceptualise, scaling it back down again presents new challenges. Consider all the additional logs and data that have been created during a scale-out — all of that history needs to be merged back together as the system contracts. To do that properly, we need tools designed to manage distributed data structures, with a consistent notion of merges.
Irmin addresses these kinds of needs and it enables a style of programming very similar to the Git workflow, where distributed nodes fork, fetch, merge and push data between each other. Building an end-to-end logging system with Irmin would enable data to be managed and merged across different nodes and keep track of activity, especially in the case of a scale down. The ability to capture such information also means the opportunity to provide analytics to the creators of those unikernels around performance and usage characteristics.
The use of Irmin wouldn’t be limited to logging as the unikernels themselves could use it for managing data in lieu of other file systems. I’ll refrain from extrapolating too far about this particular tool as it’s still under rapid development and we’ll write more as it matures.
You may have noticed that one of the benefits of the unikernel approach arises because the artefacts themselves are not altered once they’re created. This is in line with the recent resurgence of ideas around ‘immutable infrastructure’. Although there isn’t a precise definition of this, the approach is that machines are treated as replaceable and can be regularly re provisioned with a known state. Various tools help the existing systems to achieve this but in the case of unikernels, everything is already under version control, which makes managing a deployment much easier.
As our approach is already compatible with such ideas, we can take it a step further. Immutable infrastructure essentially means the artefact produced doesn’t matter. It’s disposable because we have the means to easily recreate it. In our current example, we still ship the unikernel around. In order to make this ‘fully immutable’, we’d have to know the state of all the packages and code used when building the unikernel. That would give us a complete manifest of which package versions were pulled in and from which sources. Complete information like this would allow us to recreate any given unikernel in a highly systematic way. If we can achieve this, then it’s the manifest which generates everything else that follows.
In this world-view, the unikernel itself becomes something akin to caching. You use it because you don’t want to rebuild it from source — even though unikernels are quicker to build than a whole OS/App stack. For more security critical applications, you may want to be assured of the code that is pulled in, so you examine the manifest file before rebuilding for yourself. This also allows you to pin to specific versions of libraries so that you can explicitly adjust the dependencies as you wish. So how do we encode the manifest? This is another area where Irmin can help as it can keep track of the state of package history and can recreate the environment that existed for any given build run. That build run can then be recreated elsewhere without having to manually specify package versions.
There’s a lot more to consider here as this kind of approach opens up new avenues to explore. For the time being, we can recognise that the unikernel approach lends itself to the achieving immutable infrastructure.
As I mentioned at the beginning of this post, most of what I’ve described is speculative. I’ve deliberately extrapolated from where the tools are now so as to provoke more thoughts and discussion about how this new model can be used in the wild. Some of the things we’re already working towards but there are many other uses that may surprise us — we won’t know until we get there and experimenting is half the fun.
We’ll keep marching on with more libraries, better tooling and improving quality. What happens with unikernels in the rest of 2015 is largely up to the wider ecosystem.
That means you.
Thanks to Thomas Gazagnaire and Richard Mortier for comments on an earlier draft.Share / Comment
In my Jekyll to Unikernel post, I described an automated workflow that would take your static website, turn it into a MirageOS unikernel, and then store that unikernel in a git repo for later deployment. Although it was written from the perspective of a static website, the process was applicable to any MirageOS project. This post covers how things have progressed since then and the kind of automated, end-to-end deployments that we can achieve with unikernels.
If you’re already familiar with the above-linked post then it should be clear that this will involve writing a few more scripts and ensuring they’re in the right place. The rest of this post will go through a real world example of such an automated system, which we’ve set up for building and deploying the unikernel that serves our slide decks — mirage-decks. Once you’ve gone though this post, you should be able to recreate such a workflow for your own needs. In Part 2 of this series I’ll build on this post and consider what the possibilities could be if we extended the system using some of our other tools — thus arriving at something very much like our own Heroku for Unikernels.
Almost all of our OCaml projects now use Travis CI for build and testing (and deployment). In fact, there are so many libraries now that we recently put together an OCaml Travis Skeleton, which means we don’t have to manually keep the scripts in sync across all our repos — and fewer copy/paste/edits means fewer mistakes.
If you’re familiar with the build scripts from last time, then
you can browse the new scripts and you’ll see that they’re broadly similar.
In many cases you may well be able to depend on one or other of the scripts
directly and for a handful of scenarios, you can fork and patch them to
suit you (i.e. for MirageOS unikernels). We can do this because we’ve made it
quick to set up an OCaml environment using an Ubuntu PPA. The rest
of the work is done by the
mirage tool itself so once that’s installed, the
build process becomes fairly straightforward. The complexity around secure
keys was also covered last time, which allowed us to commit the
final unikernel to a deployment repo. That means the remaining step is
to automate the deployment itself.
Committing the unikernel to a deployment repo is where the previous post ended
and a number of people forged ahead and wrote about their
experiences deploying onto AWS and Linode. Many of these deployments
(understandably) involve a number of quite manual steps. It would be
particularly useful to construct a set of scripts that can be fully automated,
such that a
git push to a repo will automatically run through the cycle of
building, testing, storing and activating a new unikernel. We’ve done
exactly this with some of our repos and this post will talk through those
MirageOS unikernels can currently be built for Xen and Unix backends. This is a straightforward step and typically the build matrix is already set up to test that both of them build as expected. For this post, I’ve only considered the Xen backend as that’s our chosen deployment method but it would be equally feasible to deploy the unix-based unikernels onto a *nix machine in much the same way. In this sense, you get to choose whether you want to deploy the unikernels onto a Hypervisor (for isolation and security) or whether running them as unix-processes better suits your needs. The unikernel approach means that both options are open to you, with little more than a command-line flag between them.
In terms of the deployment machines there are several options to consider. The most obvious is to set up a dedicated host, where you have full access to the machine and can install Xen. Another is to have a machine running on EC2 and create scripts to deal with unikernels. You could also build and deploy onto Xen on the Cubieboard2. If you’d rather test out the complete system first, you could set up an appropriate machine in Virtualbox to work with.
For our workflow, we use Xen unikernels which we deploy to a dedicated host. For the sake of brevity, I won’t go into the details of how to set up the machine but you can follow the instructions linked above.
Decks is the source repo that holds many of our slides, which we’ve presented at conferences and events over the years (I admit that I have yet to add mine). The repo compiles to a unikernel that can then serve those slides, as you see at decks.openmirage.org. For maximum fun-factor, we usually run that unikernel from a Cubieboard2 when giving talks.
The toolchain for this unikernel includes build, store and deploy. We’ll recap the first two steps before going through the final one.
Build — In the root of the decks source repo, you’ll notice the
.travis.yml file, which fetches the standard build script mentioned earlier.
Building the unikernel proceeds according to the options in the build matrix.
language: c install: wget https://raw.githubusercontent.com/ocaml/ocaml-travisci-skeleton/master/.travis-mirage.sh script: bash -ex .travis-mirage.sh env: matrix: - OCAML_VERSION=4.02 MIRAGE_BACKEND=unix MIRAGE_NET=socket - OCAML_VERSION=4.02 MIRAGE_BACKEND=unix MIRAGE_NET=direct - OCAML_VERSION=4.02 MIRAGE_BACKEND=xen MIRAGE_ADDR="22.214.171.124" MIRAGE_MASK="255.255.255.128" MIRAGE_GWS="126.96.36.199" DEPLOY=1 global: - secure: ".... encrypted data ...." - secure: ".... encrypted data ...." - secure: ".... encrypted data ...." ...
In this case, two builds occur for Unix and one for Xen with different parameters being used for each. If you look at the actual travis file, you’ll notice there are 26 lines of encrypted data. This is how we pass the deployment key to Travis CI, so that it has push access to the separate mirage-decks-deployment repo. You can read the section in the previous post to see how we send Travis a private key.
Store — One of the combinations in the build matrix (configured for Xen), is intended for deployment. When that unikernel is completed, an additional part of the script is triggered that pushes it into the deployment repo.
After the ‘build’ and ‘store’ steps above, we have a
deployment repository with a collection of Xen unikernels. For
this stage, we have a new set of scripts that live in this repo alongside those
unikernels. Specifically, you’ll notice a folder called
contains four files.
. ├── Makefile ├── README.md ├── scripts │ ├── crontab │ ├── deploy.sh │ ├── install-hooks.sh │ └── post-merge.hook ...
A quick summary of the setup is that we clone the repo onto our deployment
machine and install some hooks there. Then a simple cronjob will perform
git pull at regular intervals. If a merge event occurs, then it means the
repo has been updated and another script is triggered. That script removes the
currently running unikernel and boots the latest version from the repo. It’s
fairly straightforward and I’ll explain what each of the files does below.
Makefile - After cloning the repo, run
make install. This will trigger
install-hooks.sh to set things up appropriately. It’s worth remembering that
from this point on, the git repo on the deployment machine will not be
identical to the deployment repo on GitHub.
install-hooks.sh — The first two lines ensure that the commands
will be run from the root of the git repo. The third line symlinks the
post-merge.hook file into the appropriate place within the
This is the folder where customized git hooks need to be placed in
order to work. The final line adds the file
scripts/crontab to the
deployment machine’s list of cron jobs.
ROOT=$(git rev-parse --show-toplevel) # obtain path to root of repo cd $ROOT # symlink the post-merge.sh file into the .git/hooks folder ln -sf $ROOT/scripts/post-merge.hook $ROOT/.git/hooks/post-merge crontab scripts/crontab # add to list of cron jobs
crontab — This file is a cronjob that sets up the deployment machine to
git pull on the deployment repo at regular intervals. Changing the
file in the repo will ultimately cause it to be updated on the deployment
deploy.sh). At the moment, it’s set to run every 11 minutes.
*/11 * * * * cd $HOME/mirage-decks-deployment && git pull
post-merge.hook — Since we’ve already run the Makefile, this file is
symlinked from the appropriate place on the deployment machine’s copy of the
repo. When a
git pull results in new commits being downloaded and merged,
then this script is triggered immediately afterwards. In this case, it just
ROOT=$(git rev-parse --show-toplevel) # obtain path to root of repo exec $ROOT/scripts/deploy.sh # execute the deploy script
deploy.sh — This is where the work actually happens and you’ll notice that there really isn’t much to do! I’ve commented in the code below to explain what’s going on.
VM=mir-decks XM=xm ROOT=$(git rev-parse --show-toplevel) cd $ROOT crontab scripts/crontab # Update cron scripts # Identify the latest build in the repo and then use # the generic Xen config script to construct a # specific file for this unikernel. Essentially, # 'sed' just does a find/replace on two elements and # the result is written to a new file. # KERNEL=$ROOT/xen/`cat xen/latest` sed -e "s,@VM@,$VM,g; s,@KERNEL@,$KERNEL/$VM.xen,g" \ < $XM.conf.in \ >| $KERNEL/$XM.conf # Move into the folder with the latest unikernel. # Remove any uncompressed Xen images found there # (since we may be starting a rebuilt unikernel). # Unzip the compressed unikernel. # cd $KERNEL rm -f $VM.xen bunzip2 -k $VM.xen.bz2 # Instruct Xen to remove the currently running # unikernel and then start up the new one we # just unzipped. # sudo $XM destroy $VM || true sudo $XM create $XM.conf
At this point, we now have a complete system! Of course, this arrangement isn’t perfect and there are number of things we could improve. For example, it depends on a cron job, which means it may take a while before a new unikernel is live. Replacing this with something triggered on a webhook could be an improvement, but it does mean exposing an end-point to the internet. The scripts will also redeploy the current unikernel, even if the only change is to the crontab schedule. Some extra work in the deploy script, using some git tools, might work around this.
Despite these minor issues, we do have a completely end-to-end workflow that takes us all the way from pushing some new changes to deploying a new unikernel! An additional feature is that everything is checked into version control. Right from the scripts to completed artefacts (including a method of transmitting secure keys/data, over public systems).
There is minimal work done outside the code you’ve already seen, though there is obviously some effort involved in setting up the deployment machine. However, as mentioned earlier, you could either use the unix-based unikernels or experiment with Virtualbox VM with Xen just to test out this entire toolchain.
Overall, we’ve only added around 20 lines of code to the initial 50 or so that
we use for the Travis CI build. So for less than 100 lines of code, we have
a complete end-to-end system that can take a MirageOS project from a
git push, all the way through to a live deployment.
In our current system, if the unikernel builds appropriately then we just assume it’s ok to deploy to production. Fire and forget! What could possibly go wrong! Of course, this is a somewhat naive approach and for any critical system it would be better to hook in some additional things.
One obvious improvement would be to introduce a more thorough testing regimen, which would include running unit tests as part of the build. Indeed, various libraries in the MirageOS project are already moving towards this model (e.g see the notes for links).
It’s even possible to go beyond unit tests and introduce more functional/systems/stress testing on the complete unikernel before permitting deployment. This would help to surface any wider issues as services interact and we could even simulate network conditions — achieving something like ‘staging on steroids’.
The scenario we have above also assumes that things work smoothly and nobody needs to know anything. It would be useful to hook in some form of logging and reporting, such that when a new unikernel is deployed a notification can be sent/stored somewhere. In the short term, there are likely existing tools and ways of doing this so it would be a matter of putting them together.
Overall, with the above model, we can easily set up a system where we go from writing code, to testing it via CI, to deploying it to a staging server for functional tests, and finally pushing it out into live deployment. All of this can be done with a few additional scripts and minimal interaction from the developer. We can achieve this because we don’t have to concern ourselves with large blobs of code, multiple different systems and keeping environments in sync. Once we’ve built the unikernel, the rest almost becomes trivial.
This is close enough for me to declare it as a ‘Heroku for unikernels’ but obviously, there’s much more we can (and should) do with such a system. If we extrapolate just a little from where we are now, there are a range of exciting possibilities to consider in terms of automation, scalability and distributed systems. Especially if we incorporate other aspects of the toolstack we’re working towards.
Part 2 of this series is where I’ll consider these possibilities, which will be more speculative and less constrained. It will cover the kinds of systems we can create once the tools are more mature and will touch on ideas around hyper-elastic clouds, embedded systems and what this means for the concept of immutable infrastructure.
Since we already have the ‘backbone’ of the toolchain in place, it’s easier to see where it can be extended and how.
Edit: The second part of this series is now up - “Self Scaling Systems”
Thanks to Anil Madhavapeddy and Thomas Leonard for comments on an earlier draft and Richard Mortier for his work on the deployment toolchain.Share / Comment
Last summer we announced the beta release of a clean-slate implementation of TLS in pure OCaml, alongside a series of blog posts that described the libraries and the thinking behind them. It took two hackers six months — starting on the beach — to get the stack to that point and their demo server is still going strong. Since then, the team has continued working and recently presented at the 31st Chaos Communication Congress.
The latest example goes quite a bit further than a server that just displays the handshake. This time, the team have constructed a Xen unikernel that’s holding a private key to a bitcoin address and are asking people to try and break in. Hence, they’ve called it the Bitcoin Piñata!*
The Piñata unikernel will transmit its private bitcoin key if you can successfully set up a TLS connection but it’s rigged so that it will only create that connection if you can present the certificate it’s expecting to see — which has been signed appropriately. Of course, you’re not being given the secret key with which to do that signing and that means there should be no way for anyone to form a TLS connection with the Piñata. In order to get the private key to the bitcoin address, you’ll have to smash your way in.
Helpfully (perhaps), things are set up so that you can make the Piñata talk to itself, allowing you to eavesdrop on a successful connection and see the encrypted traffic. In addition, all the code and libraries are open-source so you can look through any of the codebase. There isn’t anything that anyone will have to reverse engineer, which should make this a little more enjoyable.
This contest is set to run until mid-March or whenever the coins are taken. If someone does manage to get in, please do let us know how!
Of course there are many other ways to get at the private key and as many people like to comment, the human element is sometimes the weakest link — after all, a safe is only as secure as the person with the combination.
In this case, there is obviously a secret key or certificate somewhere that could be presented so it may be tempting to go hunting for that. Perhaps phishing attempts on the authors may yield a way forward, or maybe just straight-forward Rubber-hose cryptanalysis! Sure, these options might provide a result† but this is meant to be fun. The authors haven’t specified any rules but please be nice and focus on the technical things around the Piñata‡. Don’t be this guy.
Even though the Bitcoin Piñata is clearly a contest, nobody is deluding themselves into thinking that if the coins are still there in March, that somehow the stack can be declared ‘undefeated’ — while pleasing, that result wouldn’t necessarily prove anything. Contests have their place but as Bruce Schneier already pointed out, they are not useful mechanisms to judge security.
However, it does give us the chance to engage in some shameless self-promotion and try to draw vast amounts of attention to the work. That, and the chance to stress-test the stack in the wild. Ultimately, we want to use this code in production but must take a lot of care to get there and want to be sure that it can bear up. This is just one more way of learning what happens when putting something ‘real’ out there.
If the Bitcoins do end up being taken, then there’s definitely something valuable that the team can learn from that. Regardless of the Piñata, if we have more people exploring the TLS codebase or trying it out for themselves, it will undoubtedly be A Good Thing.
For clarity and to avoid any doubt, please be aware that the TLS codebase is missing external code audits and is not yet intended for use in any security critical applications. All development is done in the open, including the tracking of security-related issues, so please do consider auditing the code, testing it in your services and reporting issues.
* If you've never come across a piñata before, hopefully the gif in the post gives you an idea. If not, the wiki page will surely help, where I learned that the origin may be Chinese rather than Spanish!
† Of course, I'm not suggesting that anyone would actually go this far. I'm simply acknowledging that there is a human factor and asking that we put it aside.
‡ Edit to add: After seeing Andrea's tweet I should point out that any part of MirageOS, including the networking stack, OCaml runtime etc is a legitimate vector. It's why there's a manifest of the libraries that have been used to build the Piñata!Share / Comment
Last weekend was spent at one of the world’s biggest open source conferences, FOSDEM. You can check out last year’s review to get an idea of the scale of the event. Since there’s no registration process, it’s difficult to estimate how many people attend but given how many rooms there are, and how full they are, it’s easily several thousand. I was impressed last year at how smoothly things went and the same was true this year.
The main reason to attend this time was to run a demo of MirageOS from an ARM board — one of the main advances since the previous conference. I looked over all the things we’d achieved since last year and put together a demo that showcases some of the capabilities as well as being fun.
The demo was to serve the 2048 game from a Unikernel running on a Cubieboard2 with its own access point. When people join the wifi network, they get served a static page and can begin playing the game immediately.
The components I needed for the demo were:
Code for making a static website — Since the game is completely self-contained (one html file and one js file). I only need to convert a static website into a unikernel. That’s trivial and many people have done it before.
A Cubieboard with a wifi access point — There are pre-built images on the MirageOS website, which make part of this easy. However, getting the wifi access point up involves a few more steps.
The first two pieces should be straightforward and indeed, I had a working unikernel serving the 2048 game within minutes (unix version on my laptop). The additional factors around the ARM deployment is where things were a little more involved. Although this was technically straightforward to set up, it still took a while to get all the pieces together. A more detailed description of the steps is in my fosdemo repository and in essence, it revolves around configuring the wifi access point and setting up a bridge (thanks to Mindy, Magnus and David for getting this working).
Once this was all up and running, it was a simple matter to configure the board to boot the unikernel on startup, so that no manual intervention would be required to set things up at the booth.
I gave the demo at the Xen booth and it went very well. There was a small crowd throughout my time at the booth I’m convinced that the draw of a board with glowing LEDs should not be underestimated. Many people we’re happy to connect to the access point and download the game to their browser but there were two main things I learnt.
Firstly, demos involving games will work if people actually know the game. This is obvious, but I’d assumed that most people had already played 2048 — especially the crowd I’d expect to meet at FOSDEM. It turned out that around a third of people had no idea what to do when the game loaded onto their browser. They stared blankly at it and then blankly at me. Of course, it was trivial to get them started and they were soon absorbed by it — but it still felt like some of the ‘cool-factor’ had been lost.
The second thing was that I tried to explain too much to people in much too short a time. This particular demo involved Xen unikernels, js_of_ocaml and a Cubieboard2 with a wifi access point. There’s a surprisingly large amount of technology there, which is difficult explain to a complete newcomer within one or two minutes. When it was obvious someone hadn’t heard of unikernels, I focused on the approach of library operating systems and the benefits that Mirage brings. If a visitor was already familiar with the concept of unikernels, I could describe the rest of the demo in more detail.
Everything else did go well and next time I’d like to have a demo like this running with Jitsu. That way, I could configure it so that a unikernel would spin up, serve the static page and then spin down again. If we can figure out the timing, then providing stats in the page about the lifetime of that unikernel would also be great, but that’s for another time.
One of the things we’re keen to work towards is the idea of personal clouds. It’s not a stretch to imagine that a Cubieboard2, running the appropriate software, could act as one particular node in a network of your own devices. In this instance it’s just hosting a fun and simple game but more complex applications are also possible.
Of course, there was lots more going on than just my demo and I had a great time attending the talks. Some in particular that stood out to me were those in the open source design room, which was a new addition this year. It was great to learn that there are design people out there who would like to contribute to open source (get in touch, if that’s you!). I also had a chance to meet (and thank!) Mike McQuaid in his Homebrew talk. FOSDEM is one of those great events where you can meet in person all those folks you’ve only interacted with online.
Overall, it was a great trip and I thoroughly recommend it if you’ve never been before!Share / Comment
The mission of Nymote is to enable the creation of resilient decentralised systems that incorporate privacy from the ground up, so that users retain control of their networks and data. To achieve this, we reconsider all the old assumptions about how software is created in light of the problems of the modern, networked environment. Problems that will become even more pronounced as more devices and sensors find their way into our lives.
We want to make it simple for anyone to be able to run a piece of the cloud for their own purposes and the first three applications Nymote targets are Mail, Contacts and Calendars, but to get there, we first have to create solid foundations.
In order to create applications that work for the user, we first have to create a robust and reliable software stack that takes care of fundamental problems for us. In other words, to be able to assemble the applications we desire, we must first construct the correct building blocks.
We’ve taken a clean-slate approach so that we can build long-lasting solutions with all the benefits of hindsight but none of the baggage. As mentioned in earlier posts, there are three main components of the stack, which are: Mirage (OS for the Cloud/IoT), Irmin (distributed datastore) and Signpost (identity and connectivity) - all built using the OCaml programming language.
As you’ve already noticed, there’s a useful acronym for the above tools — MISO. Each of the projects mentioned is a serious undertaking in its own right and each is likely to be impactful as a stand-alone concept. However, when used together we have the opportunity to create applications and services with high levels of security, scalability and stability, which are not easy to achieve using other means.
In other words, MISO is the toolstack that we’re using to build Nymote — Nymote is the decentralised system that works for its users.
Each of the projects is at a different phase but they have all have made great strides over the last year.
Mirage — a library operating system that constructs unikernels — is the most mature part of the stack. I previously wrote about the Mirage 1.0 release and only six months later we had an impressive 2.0 release, with continuing advances throughout the year. We achieved major milestones such as the ability to deploy unikernels to ARM-based devices, as well as a clean-slate implementation of the transport layer security (TLS) protocol.
In addition to the development efforts, there have also been many presentations to audiences, ranging from small groups of startups all the way to prestigious keynotes with 1000+ attendees. Ever since we’ve had ARM support, the talks themselves have been delivered from unikernels running on Cubieboards and you can see the growing collection of slides at decks.openmirage.org.
All of these activities have led to a tremendous increase in public awareness of unikernels and the value they can bring to developing robust, modern software as well as the promise of immutable infrastructure. As more people look to get involved and contribute to the codebase, we’ve also begun curating a set of Pioneer Projects, which are suitable for a range of skill-levels.
You can find much more information on all the activities of 2014 in the comprehensive Mirage review post. As it’s the most mature component of the MISO stack, anyone interested in the development of code towards Nymote should join the Mirage mailing list.
Irmin — a library to persist and synchronize distributed data structures — made significant progress last year. It’s based on the principles of Git, the distributed version control system, and allows developers to choose the appropriate combination of consistency, availability and partition tolerance for their needs.
Early last year Irmin was released as an alpha with the ability to speak ‘fluent Git’ and by the summer, it was supporting user-defined merge operations and fast in-memory views. A couple of summer projects improved the merge strategies and synchronisation strategies, while an external project — Xenstore — used Irmin to add fault-tolerance.
More recent work has involved a big clean-up in the user-facing API (with nice developer documentation) and a cleaner high-level REST API. Upcoming work includes proper documentation of the REST API, which means Irmin can more easily be used in non-OCaml projects, and full integration with Mirage projects.
Signpost will be a collection of libraries that aims to provide identity and connectivity between devices. Forming efficient connections between end-points is becoming ever more important as the number of devices we own increases. These devices need to be able to recognise and reach each-other, regardless of their location on the network or the obstacles in between.
This is very much a nascent project and it involves a lot of work on underlying libraries to ensure that security aspects are properly considered. As such, we must take great care in how we implement things and be clear about any trade-offs we make. Our thoughts are beginning to converge on a design we think will work and that we would entrust with our own data, but we’re treating this as a case of ‘Here Be Dragons’. This is a critical piece of the stack and we’ll share what we learn as we chart our way towards it.
Even though we’re at the design stage of Signpost, we did substantial work last year to create the libraries we might use for implementation. A particularly exciting one is Jitsu — which stands for Just In Time Summoning of Unikernels. This is a DNS server that spawns unikernels in response to DNS requests and boots them in real-time with no perceptible lag to the end user. In other words, it makes much more efficient use of resources and significantly reduces latency of services for end-users — services are only run when they need to be, in the places they need to be.
There’s also been lots of efforts on other libraries that will help us iterate towards a complete solution. Initially, we will use pre-existing implementations but in time we can take what we’ve learned and create more robust alternatives. Some of the libraries are listed below (but note the friendly disclaimers!).
OCaml is a mature, powerful and highly pragmatic language. It’s proven ideal for creating robust systems applications and many others also recognise this. We’re using it to create all the tools you’ve read about so far and we’re also helping to improve the ecosystem around it.
One of the major things we’ve been involved with is the coordination of the OCaml Platform, which combines the OCaml compiler with a coherent set of tools and workflows to be more productive in the language and speed up development time. We presented the first major release of these efforts at OCaml 2014 and you can read the abstract or watch the video.
There’s more to come, as we continue to improve the tooling and also support the community in other ways.
Building blocks are important but we also need to push towards working applications. There are different approaches we’ve taken to this, which include building prototypes, wireframing use-cases and implementing features with other toolstacks. Some of this work is also part of a larger EU funded project* and below are brief summaries of the things we’ve done so far. We’ll expand on them as we do more over time.
Mail - As mentioned above, a prototype IMAP server exists (IMAPlet) which uses Irmin to store data. This is already able to connect to a client to serve mail. The important feature is that it’s an IMAP server which is version controlled in the backend and can expose a REST API from the mailstore quite easily.
Contacts - We first made wireframe mockups of the features we might like in a contacts app (to follow in later post) and then built a draft implementation. To get here, code was first written in OCaml and then put through the js_of_ocaml compiler. This is valuable as it takes us closer to a point where we can build networks using our address books and have the system take care of sharing details in a privacy-conscious manner and with minimal maintenance. The summary post has more detail.
Calendar - This use-case was approached in a completely different way as part of a hackathon last year. A rough but functional prototype was built over one weekend, with a team formed at the event. It was centralised but it tested the idea that a service which integrates intimately with your life (to the point of being very invasive) can provide disproportionate benefits. The experience report describes the weekend and our app — Clarity — won first place. This was great validation that the features are desirable so we need to work towards a decentralised, privacy-conscious version.
The coming year represents the best time to be working on the MISO stack and using it to make Nymote a reality. All source code is publicly available and the projects are varied enough that there is something for everyone. Browse through issues, consider the projects or simply write online and share with us the things you’d like to see. This promises to be an exciting year!
Sign up to the Nymote mailing list to keep up to date!
* The research leading to these results has received funding from the European Union's Seventh Framework Programme FP7/2007-2013 under the UCN project, grant agreement no 611001.Share / Comment