title: Configuration Management for the People
date: 2014-03-29 14:45
author: Christine Lemmer-Webber
tags: configuration, deployment
slug: configuration-management-for-the-people
---
One of the things I've talked about in the talks I've been giving about
MediaGoblin lately is the issue of deployment. Basically it boils down
to the following points:

-   MediaGoblin is harder than we'd like to deploy, but it's probably
    easier than most other Python web applications. All modern web
    applications are currently hard to maintain and deploy.
-   We've worked to try to make it so you don't have to know how to be a
    \$LANGUAGE developer (in this case, Python) to deploy MediaGoblin,
    but once things break you do have to be a \$LANGUAGE developer, or
    know one. (We spend a lot of time answering these things in
    #mediagoblin. This can be improved if we get proper system packaging
    though (which is currently a work in progress, at least for Debian.
    I hope to see it soon...))
-   Even if you get system packaging, that's not enough. You need to do
    a mess of other things to set up web applications: configure the web
    server (Apache, Nginx), configure sending mail, configure a bunch of
    things. And let's not even talk about getting SSL set up right. Oy.
-   The reason people don't see that modern web applications are hard to
    deploy is because even though they are, there's a team of devops
    behind the scenes handling it for them.

We can do as good as we can to try to make MediaGoblin's docs easy to
understand, but I'm convinced the solution needs to be a layer higher
than MediaGoblin. That's probably one of two things:

-   A Platform as a Service solution like OpenShift. (Note, there are
    other proprietary ones out there, but if it's not free software,
    what's the point?) These solutions are kind of heavy, but they seem
    like a step in the right direction.
-   A configuration and deployment abstraction system, like
    Puppet/Chef/SaltStack/Ansible, etc.

The latter seems like the right solution to me, but it's not enough
as-is. The configuration and deployment systems we presently have are
too devops-focused. We can't free society by expecting everyone to join
the world of devops... for one thing, these tools are way too hard to
learn at present, and for second, a world where only the technically
elite are free is a pretty nonfree world, really.

(I don't think things like Docker or virtual machine images are the
answer. Aside from being pretty heavy, they don't solve the
configuration issue.)

But these systems are *pretty close*. Close enough even! I think we can
pull this off. Let's see what we need and what's missing:

-   Needs to have share'able recipes. (Yes yes, people currently do sad,
    sad things like dumping some recipes to code hosting platforms.
    That's not the same thing. I want something like an apt repository,
    but for recipes. (Note, [juju](https://juju.ubuntu.com/) might have
    this... I don't know much about it))
-   Recipes should be pretty simple... based on variables set by the
    user, or variables interpolated by user-specific settings (all these
    config management systems handle this, I think)... mostly I like
    what I've seen of how Salt handles this.
-   There should be an expectation that you should be able to mix and
    match recipes somewhat. This means some agreements higher up the
    chain on how we expect a mail server configuration is going to be
    described, etc. I'm not sure how the standards/governance around
    this could be best handled.
-   I think a layer on top of all this that's really needed is some kind
    of web UI for application install and configuration. If I install
    the MediaGoblin recipe, it may be that a lot of defaults can be set
    and guessed. But what if I want to turn on the video plugin, and I
    want to change my authentication system to using LDAP because that's
    what my company already uses, etc? I think this can be pretty
    minimal, we can have a specification that both describes config
    options as well as how to represent them in the web UI.
-   It shouldn't be tied to any one specific platform. Not to a wall
    wart, not to a VPS, not to a Raspberry Pi. It should be generic
    enough to work on all of these. Again, I see no reason this can't be
    pretty minimal.

I've been fielding this by people for a while, trying to quiz all the
smarter-than-me-people I know about what they think, but I keep coming
back to this. At a dinner at LibrePlanet, all the usual suspects were
raised as possible solutions, and none of them seemed to fit. I was
happy to hear [Joey Hess](https://joeyh.name/) say, "I think that Chris
is right, we need the 'layer above apt' configuration solution." (If
that wasn't some reassurance that I'm on the right track, I don't know
what would be... if anyone would know [it would be
Joey](http://raphaelhertzog.com/2010/11/11/people-behind-debian-joey-hess-of-debhelper-fame/)...)

(Note: GNUToo suggested at FOSDEM that I should look at
[LuCI](http://luci.subsignal.org/trac)... LuCI always felt a bit
clumsier than I'd like when I used it but maybe it does do these things.
I don't know if it handling recipes like I said, but maybe? Worth
looking into.)

Here's a rough plan on how I'd go about doing this, if I had time to do
this (which I don't think I do, though I can help someone kick off the
project, and I can make some contributions):

-   Start by investigating Salt (or Ansible?). It's a little bit heavy
    but not too heavy (the current python package appears to be
    2.8mb...). Plan on using the master-less setup. Salt provides a lot
    of abstractions around installihng packages and a lot of these other
    things, so that may be a good start.
-   It might be that things need to be paired down to something even
    simpler, or that Salt is just too hard to build on top of. If so,
    start simple on recipes. I'd save a dependency graph system as
    something optional as a "milestone 2" type thing, personally.
-   You need some way to bundle a description of the variables to be
    provided but also how to represent them configuration-wise to the
    user. I think a [Hy DSL](http://hy.readthedocs.org/en/latest/) might
    be the fastest way to start writing up variable descriptions
    including web represenation, but also embed code to retreive results
    in case a default is not provided.
-   At this point you should have a command-line tool that you can run,
    reads your currently installed recipes and your current settings,
    and executes them.
-   Build a web UI.
-   Figure out how to publish up a public repository of recipes. (May
    need to be distro specific.)

That's it I think. I think if we had something like this, it would
simplify deployment greatly.

Are you interested in taking this on? [We should talk!](/contact/) ;)

**PS:** We'd like to do further research and work into making
MediaGoblin easier to deploy over the next year. Our capacity to do that
largely depends on how much we're able to raise... consider donating to
[our campaign](http://mediagoblin.org/pages/campaign.html)!

**Edit:** [Mike Linksvayer makes strong points about previous language
around PHP](https://identi.ca/cwebber/note/7ZKVvXCeQ1m4T4o87hPr6Q);
[removed](https://gitorious.org/dustycloud/dustycloud/commit/5959e4f232ddad9cb7bb3dcb806aea91c009b17a).
So, reverting language that makes PHP sounds like a problem, but I'll
still argue that it's not actually a full solution (there are
configuration issues not resolved by language choice)

**Edit 2:** I guess I didn't say this, so it's worth saying... a lot of
difficulties in modern deployment are because people aren't using system
packaging (this includes MediaGoblin's docs, which suggest the
breaks-all-the-time-even-though-I-understand-why-but-our-users-don't
world of Python packaging... we're waiting on Debian packaging. Real
Soon Now, I hope!). Using system packaging certainly solves a lot of
these headaches, but it doesn't solve nearly all of them. There's still
the issues of configuring things, and there's really a lot... too
much!... to configure: mail transfer agents, hooking your application up
to the web server, SSL, so on and so on. That's something that hasn't
been solved, especially not for non-devops people.

I've started cobbling together something that might solve things for
non-devops people, and even devops people both! Maybe it'll see the
light of day sometime. In the meanwhile, I'm really interested in other
peoples' solutions to the problems described above.
