Philip Guo (Phil Guo, Philip J. Guo, Philip Jia Guo, pgbovine)

Automatically create Docker images by running the program you want to package

(somebody please make this happen!)

Five years ago, I started working on CDE since there was no simple way to create a portable, self-contained software package on Linux. Existing solutions involved wrangling library dependencies and wrestling with package managers. In contrast, CDE is super-simple to use: to package up your software, simply run it once. No coding, no configs, no hassle – just run it!

Over 10,000 people have downloaded and used CDE to quickly share prototype software, test software on different Linux distros, reproduce research experiments, and deploy jobs to the cloud.

Here's a 4-minute introductory video:

(To learn more about CDE, read my LISA 2011 paper, watch my Google Tech Talk, or read Year 5 of The Grind.)

Five years later: CDE + Docker

Fast-forward to 2015. It's been almost four years since I've worked on CDE; my research has shifted in a completely different direction. But one related project that I'm really excited about is Docker, which now seems to be everywhere!

Docker runs portable software packages (called images) at near-native speeds. However, it can still be tedious to create Docker images ... just like with traditional package managers, you need to write scripts or mess with config files (Dockerfiles). If you don't remember all of your software's weird dependencies, then you're out of luck. So ...

What if you could create Docker images by simply running the program you want to package?

CDE already provides the machinery to do this, and its code is very robust (I spent a whole year polishing it to handle crazy corner cases). I'm not a Docker expert, but my hunch is that someone can write a script to convert CDE-made packages into Docker images. I don't have the time or energy to work on this now. But if you'd like to make this happen, let me know!

The main limitation of CDE is that packages run up to 20% slower due to its use of ptrace system call interposition. Docker doesn't use ptrace, so it incurs a much smaller (nearly zero) slowdown. By combining CDE and Docker, you get the best of both worlds: create Linux packages super-quickly with CDE, and run packages at near-native speeds with Docker!

Created: 2015-11-23
Last modified: 2015-11-23
Related pages tagged as software:
Related pages tagged as research: