How I Updated This Site To Drupal 8.8.0 With Composer

Submitted by Mile23 on Wed, 12/11/2019 - 13:04

Hey, Drupal 8.8.0 has a defined Composer behavior!

That's right, Drupal 8.8.0 is Composer-first. That means you can build it from an official, signed-off-by-maintainers type Composer project template. You can do this:

$ composer create-project drupal/recommended-project:^8.8.0 my_web_directory

This is a big deal because previously you had to use a third-party Composer project template to be able to build a codebase that didn't completely misunderstand you when you subsequently typed:

$ composer require drupal/address

Let's Use A Conversion Tool

There are also a few different tools out there which can convert your non-Composer-managed site to one which is Composer-managed.

I had used Composerize-Drupal previously to get mile23.com into Composer-readiness. It had a few limitations, such as not discovering modules outside of modules/contrib/. But I was able to let it do some of its magic and then apply my own as well.

But the reason I'm mentioning this is because under Drupal 8.8, all the Composer plugins are in place, but they need to come together in super-awesome harmonic convergence and stuff. There is not yet a command-line tool to perform the conversion the right way.

That tool is still under development, and didn't make it into the 8.8.0 release. I'm mostly working on it alone, but it's also true that development has been slow since the 8.8.0-alpha days. That's when new features aren't allowed any more, so we couldn't add a new tool to the Composer plugin suite.

The issue is here: https://www.drupal.org/project/drupal/issues/3047468

And you can follow along with the development here on this feature branch if you'd like: https://github.com/paul-m/drupal/tree/3047468

But the main way to encounter this tool is in it's currently in a very-prerelease state through Composer:

$ composer global require mile23/core-composer-converter @dev

This is the same code as a not-official-Drupal Composer plugin. It's a pre-fork. :-)

So I used it this way:

$ cd mile23
$ composer drupal:legacy-convert -n --prefer-projects

Walking Through The Changes

Let's do a mini overview of what the changes are and what they mean.

     "require": {
-        "php": "^5.5.9|>=7.0.8",
-        "composer/installers": "^1.2.0",
-        "drupal-composer/drupal-scaffold": "^2.5.4",
[...]
+        "drupal/core-composer-scaffold": "^8.7",
     },
     "extra": {
+        "drupal-scaffold": {
+            "locations": {
+                "web-root": "./"
+            }
         },

You'll note that it swapped out drupal-composer/drupal-scaffold in favor of drupal/core-composer-scaffold, and explicitly configured drupal/core-composer-scaffold to use the root directory of the Drupal filesystem as the web-root.

     "require": {
-        "drupal/core": "^8.6.12",
+        "drupal/core-composer-scaffold": "^8.7",
+        "drupal/core-project-message": "^8.7",
+        "drupal/core-recommended": "^8.7",
+        "drupal/core-vendor-hardening": "^8.7",

Here we see that we trade drupal/core away for drupa/core-recommended. The core-recommended package is a metapackage. It does two things:

  1. It requires the correct version of drupal/core for you, and
  2. Requires specific versions of dependencies derived from the known-good-and-tested composer.lock file present in Drupal core. Meaning, you shouldn't require drupal/core, and instead you should require drupal/core-recommended, and then you'll get all the known-good versions of dependencies.

If you like to live on the edge, just require drupal/core, issue a composer update --no-dev, and be sure and use your battery of deployment tests. :-)

Speaking of --no-dev...

+    "require-dev": {
+        "drupal/core-dev": "^8.7"
     },

This one's a bit tricky. It's a metapackage, similar to drupal/core-recommended above, except it's all the known-good versions of Drupal core's development requirements.

Because this requirement exists in my composer.json file, I'll have to say this at some time before I deploy:

$ composer install --no-dev

In my workflow this is not hard, but it can be difficult to remember to do this. There is a core issue about changing this in Drupal 9, so there are no development dependencies installed at all unless the user adds them: https://www.drupal.org/project/drupal/issues/3086489

And while we're on the topic of development tools... This line includes a plugin that does a best-faith effort to harden your vendor/ directory against security exploits:

     "require": {
+        "drupal/core-vendor-hardening": "^8.7",

If you are using the drupal/legacy-project template, this plugin is included. But if your Drupal root directory is the same as the web-root directory, you should require this plugin. Why? Because security!

OK, You Talked About The Plugins, What Else?

If you have mile23/core-composer-converter installed, and you type composer install, you might see something like this at the very end:

This project has extensions on the file system which might require manual updating. Read docs here: http://write.these.docs/

Now, of course the docs aren't written. But what it should say right there is that you can do this:

$ composer drupal:reconcile-extensions --help

This will tell you that you have extensions in your Drupal codebase that are not accounted-for by the composer.json file.

So let's try it out.

$ drush dl examples
Project examples (8.x-1.0) downloaded to                             [success]
/Users/paul/pantheon/mile23//modules/examples.
Project examples contains 31 modules: stream_wrapper_example, field_example, render_example, tour_example, batch_example, session_example, menu_example, 
[.. etc ..]
$ composer drupal:reconcile-extensions -n --dry-run --prefer-projects
 - Scanning the file system for extensions not in the composer.json file...
Using version ^1.0 for drupal/examples
 - (Dry run) Add these packages: drupal/examples ^1.0
 - (Dry run) Remove these extensions from the file system: [...] node_type_example, tablesort_example, plugin_type_example, field_permission_example, cache_example,
[.. etc ..]
 - Discovered extensions which are not in the original composer.json, and which do not have drupal.org projects. These extensions will need to be added manually if you wish to manage them through Composer:

 * Mile23

What did we do here?

We used Drush to download the Drupal Examples for Developers project so we'd have some unreconciled extensions to find.

We then asked Composer to reconcile those extensions. Note that we can tell it to --prefer-projects, so that instead of requiring each examples module separately, it figures out that they all belong to the examples project.

After Composer figures out it can do this, it then adds the drupal/examples package to the composer.json file. Since this is a --dry-run it only tells you that it would do this.

And then, after we've accounted for the extensions in composer.json, we remove them from the file system. This is so that when we eventually run composer update, we'll have them in the place where Composer installers wants them to be.

That last bit in the log above, where it finds an extension without a drupal.org project, is what happens for all the extensions you might not be able to manage with Composer.

In this case, it's a theme called Mile23. There are a bunch of complicated ways to account for that extension, but the simplest for now is to just ignore the warning.

And That's How I Updated This Web Site To Drupal 8.8.0 Using Composer

Yup, that's what happened. I ran composer drupal:legacy-convert and then looked at the changes to make sure they made sense. Granted, this is not a very complicated Drupal installation.