From Drush make to Composer, one module at a time

Drush make is great code, and provides a really innovative way to build your codebases. Its many advantages and wide adoption are among the reasons why I use it to power Drush instance. However, the file format and way of working isn't particularly standard outside the Drupal community, making it something of a "Drupalism". Such Drupal-specific standards can tend to alienate non-Drupal developers, who land in the Drupal project(s) and think to themselves: "what the hell is this? and this?.html"

If possible, we should always root out Drupalisms: though they bind our community together with our shared vocabulary, they can exclude as much as they include, if not more. This spirit is behind the plan to replace core D8 functionality with Symfony where appropriate: Symfony is itself a project built on as many public standards or conventions as possible. Drupalisms made pragmatic sense, when no other PHP-wide or development-wide standard existed, but now we have so many more public resources to choose from, that reinventing things ourselves seems perverse or naive.

With that in mind, we need to be thinking sooner rather than later about the feasibility of replacing Drush make with Composer.

What Drush make gives us

One of the big advantages of Drush make - permit me some nostalgia for a moment - is that, being a Drupalism, it's very Drupal-savvy. That means that it understands what's meant by a Drupal project, and can contact drupal.org to find out more information about that project. So much of what makes your site can become implicit, and you can specify as little as you want. For example, say you want to build a codebase with just Views and Ctools - but you don't care what versions you use - then you can write a mysite.make makefile as follows:

api = 2
core = 7.x
project[] = views
project[] = ctools

Extending this pattern to include other projects would as you can see be very simple. Only if your requirements are more complex - you want to pin a module to a specific version, or use a git repository as a source - does the makefile itself get more complex.

How might Composer give us the same

The equivalent in Composer is remarkably less terse. The first point is that we need to tell Composer about the difference between modules, themes etc. so they get installed in the right subfolder. We therefore immediately need the excellent composer/installers project as one of our composer.json requirements, and each package entry needs a "type" field: "drupal-module", "drupal-theme" etc.

The second is that, while Composer has some autodiscovery too, and will check in the first instance for a Packagist repository: drupal.org is not Packagist, and does not currently provide an official packages.json endpoint, although there are a couple of unofficial alternatives that are works in progress.

This situation is understandable, because so few people use Composer for Drupal yet; but it yields a bootstrapping problem meaning that, for now, our equivalent composer.json has to be so much more verbose, if it's going to be robust and not rely on beta services:

{
  "repositories": {
    "views": {
      "type": "package",
      "package": {
        "name": "drupal/views",
        "version": "7.3.8",
        "type": "drupal-module",
        "dist": {
          "url": "http://ftp.drupal.org/files/projects/views-7.x-3.8.zip",
          "type": "zip"
        }
      }
    },
    "ctools": {
      "type": "package",
      "package": {
        "name": "drupal/ctools",
        "version": "7.1.4",
        "type": "drupal-module",
        "dist": {
          "url": "http://ftp.drupal.org/files/projects/views-7.x-1.4.zip",
          "type": "zip"
        }
      }
    }
  },
  "require": {
    "composer/installers": "~1.0",
    "drupal/views": "7.3.8",
    "drupal/ctools": "7.1.4"
  }
}

But! if we were able to reliably use one of the unofficial services, then here's what the above reduces to:

{
  "repositories": {
    "drupal": {
      "type": "composer",
      "url": "http://static.drupal-packagist.org/v0.1.0/"
    }
  },
  "require": {
    "composer/installers": "~1.0",
    "drupal/views": "7.3.8",
    "drupal/ctools": "7.1.4"
  }
}

We still need installers, to ensure the right subdirectories: but the result is much more terse, and as you can see extending the "require" JSON for new projects should be trivial.

Sounds good? Well, yes, but at the time of writing, at least one third-party service has a few issues: the major one is that it can be quite slow to resolve, even before download noticeably begins. This is possibly because these services are also handling (non-essential) dependencies: I was given a download of advanced_help and panels along with the above. The second issue is that if I include the mothership theme, it installs it in the vendor/drush/ subfolder, not the vendor/themes/ folder. And these bugs can be filed on the github repository (indeed I've recently done so!) but obviously this is less favourable than a community-supported service, where maintenance and downtime could be given more priority (and carried out by a wider team.)

Where do we go from here?

Composer is ultimately the future of automated dependency building within PHP environments, and it makes sense, if possible, for Drupal to move to using it. Right now, makefile technology is at the heart of any automated Drupal site building, for reasons which are compelling enough: these reasons would need to be trivially satisfied by Composer, for any (Drupal) developer coming to it with very limited knowledge of how to wrangle complex Composer configuration.

Drupal relies on makefiles and there's a lot of "technical debt" arising from them: converting every d.o project's makefile to use Composer is no trivial task. There needs to be automated conversion and a simple upgrade path, made available to every module maintainer and anyone who's committed themselves to using makefiles in their own environment.

Drupal.org needs its own packagist service, maintained by the community, or to automatically add d.o packages to packagist.org somehow. It needs at any rate to be a trivial process to add new Drupal projects to a JSON file: as simple as adding new lines to a makefile.

By adopting Composer - at least as a parallel technology to Drush make in the first instance - Drupal.org will give the right message to the wider Drupal community: it's OK; it's safe; you can use Composer; the more active community members will watch your back if you do. And then, with that encouragement - probably - they will.