(Con)figuring out Drupal 8: The Sequel
In Drupal 7, site deployments could be rather difficult on ambitious sites. Some database level elements were worth programming out in hook_updates (turning on modules, reverting views, etc) and some usually weren't (block placement, contrib module configuration). I remember days where a deployment involved following a three page long Google doc of clicks that had to be carefully replicated. Ugh.
A New Hope
So if you've taken the dive into Drupal 8, you'll quickly discover one of it's most prominent features - Configuration Management. Drupal 8's ability to manage configuration with yml files is absolutely amazing! It's nearly akin to watching Star Wars and thinking "Hey, I can do anything with a lightsaber! Fight bad guys, cut holes in doors, remove my hand cuffs. Sweet!"
The Empire Strikes Back
Here's the rub. Managing Drupal 8 configuration in complex real world apps is akin to building a real world laser sword after watching Star Wars only to promptly burn your face off and lose two limbs as soon as you try to fight with it. "Ambitious digital experiences" essentially equates to "arduous development concerns" and even config management can't save the day simply by existing. You must use it for good. You must unlearn what you have learned. I blogged a bit on this shortly after Drupal 8 released, but oh how much I learned since then!
We've been doing Drupal 8 pretty heavy for about a year and a half here at Ashday and had both the fortune and misfortune of needing to manage a more complex set up which quickly revealed our deficiencies in understanding how to properly manage config.
Here's the scenario: A client needs a site that will become the model for many sites, but they don't want them to be a single site with multiple domains and they also don't want it to be costly or complicated to keep them mostly similar from a functional perspective. Given that our preferred hosting solution is Pantheon, this quickly turned into an obvious Upstream project. And that means figuring out a new way to manage D8 config other than just import/export of the whole site.
If you aren't familiar, a Pantheon Upstream works nearly identical to their core updates - you have a remote repository that, upon code getting pushed to it, notifies you through the dashboard of your updates where you can apply them in the same way you do Pantheon core updates. It's pretty slick because it provides an easy way to have a big shared chunk of code and apply updates to many sites with a few clicks (well, except when nearly every update is major and requires hands-on management - but I'm not bitter).
The Phantom Menace
Our first try at this was to give the Features module a go, but at the time the interface was just too buggy to give us enough confidence to rely on it, it auto-selected what we didn't want and didn't select what we did, and it didn't support some key things we needed like permissions. As a result we decided to home brew our own solution. We knew these sites were going to have a lot of config in common, and a lot of config unique, and we needed to deploy to many of them all in different states without tragedy striking. So to accomplish this, we concocted the following procedure that we would run at deployment time, all from a single drush command.
- Export the current site's live config (using drush) to the config sync folder
- Copy all config files (with uuids removed) in our cross-site custom module over top of the config sync directory
- Copy all config files (with uuids removed) in our site-specific custom module over top of the config sync directory
- Import all config.
What this allowed us was the ability to allow each upstream site to stray a bit as they needed to, but we could be assured the config we cared about was prioritized in the proper stacking order. The approach ultimately wasn't that different than the goal of Features, but we were in control of the process, it was all live and it was relatively quick. And you know what? It worked! For a while...
And then it didn't. You see, the method we used caused Drupal to see every config file we were tracking (upwards of 300) as changed simply because of the missing uuid. So if only 8 config files changed in a deployment, Drupal was attempting to import hundreds of config files every time. This meant that it started to slow significantly over time as the site grew in complexity and eventually, we started having timeout issues and long deployments. We also started to run into issues when there was a significant core update (ie: 8.3) because so much config was being imported unnecessarily that wasn't compatible in that moment with the new code because db updates hadn't run yet. Not good. It was time for something else.
Return of the Jedi
The Jedi in question here is again the Features module. Or maybe it's Mike Potter. At least it's not me anyways. At DrupalCon Baltimore, I was set on speaking with Mike about how we were handling config because I simply knew there was a better way. If you don't know, Mike is one of the founders of the Features module and ran a great BOF on config management in Baltimore.
So I found this delightful man and laid out what we were doing and he reacted exactly as I had hoped. He didn't say that what we were doing was terribly wrong, but it made him visibly uneasy. After a chat, I discovered that Features had come a long way since we initially tried to use it and we should really give it another shot. He also explained some of the configuration of Features to help me better understand how to use it.
So we returned to Features now and are much happier for it. The thing is though that I don't think I would have really known how to manage it if we hadn't taken the deep dive into config and figured out how it needed to work. It all helped us a lot to decide how to incorporate Features properly for this particular situation so that I actually feel good about relying on it again. And that's how most good Drupal development goes. You really should know how something works before simply relying on a contrib module or someone else's code to take care of everything because otherwise you won't really know how to deal with problems - heck, you might not even know you have a problem! I personally don't prefer spending weeks writing code and then depending at a critical moment on a mysterious piece to make it all successfully roll out to production.
So as it all played out, we now understand what Drupal puts in config, what we care about and don't, what belongs in the upstream vs our site-specific modules vs no where, etc. Here is our current process after this 6 month long journey.
- Revert the global base feature
- If needed, revert the site specific feature
- Run our previous script outlined above, but now on only the 5 or 6 role config files so we handle the permissions in the same fashion
So there you have it! For how long-winded this turned out to be, I'm glossing over a lot of details that are pretty critical to understanding Drupal 8 configuration (ex: blocks are a mix of config and content), but I recommend you do the same thing we did and really get your hands dirty and understand what's going on so that you don't get bit at rollout. After all of this, we feel even moreso that Configuration Management is an astoundingly useful component of Drupal 8 and now we find ourselves a bit sad when we update our Drupal 7 sites (a version we absolutely loved!) where we don't have this amazing tool.
So good luck and don't hesitate to drop us a note if you have any questions or thoughts on this stuff. I'll probably change my mind on all of it anyways tomorrow. That's why this job is awesome.
P.S. I apologize that I didn't find room to incorporate Attack of the Clones, Revenge of the Sith, The Force Awakens or Rogue One, but the reality is that I just didn't have time to modify our whole approach to configuration in order to make this blog post more cohesive.