One of the longest standing issues with the plugin system in WordPress is how to solve the issue of dependencies. Plugins and themes want to bring in libraries, other plugins, or parent themes, but right now, the solutions are somewhat terrible. I thought it was time to get my thoughts down on (virtual) paper.

What’s the problem?

Software is invariably never built in isolation (“no man is an island”), so they are naturally drawn to using external libraries. Extending an existing system is also extremely useful; we can see that from the plugin ecosystem in WordPress itself.

However, right now, there’s no good way to do these in a way that interoperates with other plugins and sites. There are various third-party solutions, but often these require code duplication or offer a substandard user experience.

The Jetpack Problem

This lack of proper dependencies is one of the key reasons behind the system of ever-growing codebases, and is exactly why Jetpack is a gigantic plugin rather than being split out. In an ecosystem with a proper dependency system, Jetpack would simply be the “core” of other plugins, being depended on for core functionality, and offering UI to tie it all together.

One of my personal key problems with Jetpack is that it duplicates the plugin functionality in WordPress (poorly, at times), and hence doesn’t work with standard tooling. Real dependencies would help to solve this. A future Jetpack with a plugin dependency system shouldn’t look any different to the current UI, but would use real plugins internally. This would ensure that the Jetpack core stays lightweight while still offering all the functionality.

Changing this to use a real dependency system would have benefits both for developers and users. The install process of Jetpack could be improved by allowing the core of the plugin to be downloaded first, letting the user set up and configure Jetpack while the rest of the plugin downloads in the background. Users and developers concerned about the size of the plugin could install only the parts they need, reducing file size and potential attack surface across the plugin.

User Experience

In the wider ecosystem, we can see other plugins running into the same issue. The largest plugins, including WooCommerce, EDD and Yoast SEO, have some form of an extension list to attempt to solve this, but invariably end up offering a poorer user experience, sending users off to other sites.

Without creating a full library to handle this for a plugin, invariably we end up with terrible UX. I’ve seen plugins do everything from pop up a message on install saying “search for X, and install it”, to straight up installing plugins and breaking a site completely. This run-time verification also breaks workflow for version-controlled sites, as plugin installation and upgrading is typically done independently of the site itself.

Products vs Services

On a more selfish note, plugins like the REST API would see increased adoption from plugin and theme developers if they could use a unified, simple system to require it. For developers who actually care about user experience, giving terrible messages to users or including a complex library just for dependencies isn’t something they want to handle.

This has partially stymied adoption of the API, as “product” developers (theme and plugin developers) don’t want to offer a substandard experience, Worse, it has skewed our development pattern towards “service” developers (agencies doing work for clients, and teams running SaaS platforms), who have the ability to run anything they like without running into these issues. This means that very real issues that we need to tackle in order to scale to the long-tail may be deprioritised in favour of those affecting services.

How do we solve it?

This is one of those ideas that I’ve had floating around in my head for a while, basically fully-formed, but with no time to execute. I’m writing this as a guide to how I see the problem being solved, with the hopes that someone has the ability to execute this the way it should be done. Imagine this as a blueprint for a successful project, albeit not the final design.

(Note that whenever I say a plugin, I actually mean plugins or themes, as behaviour should be the same for both.)

Internal Workings, ft. Composer

Any PHP developer who has worked outside of WordPress recently will know Composer. Composer, for those who aren’t aware, is a command-line tool for managing dependencies in PHP. Composer is also not a good solution to the dependency problem for WordPress plugins: it requires CLI access and knowledge, it has a somewhat clunky interface and user experience (edit a JSON file, then generate a lock file and a vendor directory, then maybe commit one or more of those), and it also requires PHP 5.3+ (a non-starter for core integration, currently).

However, one of the key parts of Composer is the dependency solver, which is a port of the libzypp solver. This is a “SAT solver”: it takes note of what’s available and of what something requires, then it works out whether it can install the software (it solves the satisfiability problem). This solver is the key to working out the dependency chain for openSUSE packages (where libzypp is originally from), and the same system is used by Composer. This system would be a fantastic base for a plugin dependency system.

Developer User Experience (DUX)

The experience for developers needs to be a familiar one. Plugin headers are a great place to start, but they quickly become untenable in their current state, as they’re not built for complexity (check any theme with more than a few tags to see what I mean). It’s possibly that with some tweaking they could be used, but this may be hard to achieve.

Ideally, we’d want the dependencies to be declarative, since this would help out a bunch of automated tooling. However, we can’t solve every problem at once. For bootstrapping this project off the ground, procedural code will work just fine.

I have a semi-working proof of concept that looks something like this:

The top three lines of code are all that’s required to check if your dependencies exist. We can automatically detect which plugin called the function, and parsing it out is relatively simple; we just then need to pass it to WP.org to see if we can get it working.

I’ve also written up some more complex usage patterns for the system for developers doing more advanced usage. (Note that the documents linked here relate to an early prototype I was working on, so not everything there matches this document; notably, allowing Composer dependencies isn’t something I’d suggest for right now.)

End-User Experience (EUX)

The end-user experience is key to gaining adoption. You need to offer an experience that users are familiar with, and that doesn’t require a bunch of manual steps. We are working on computers, after all, which are meant to automate the dumb tasks for us.

The EUX starts before the user even installs a plugin or theme. The information screen needs to show them what the plugin needs (the full dependency tree, not just direct ones), as well as any potential conflicts with existing plugins. Installing that plugin should then also ensure that the dependencies are also installed, failing if any of the dependencies fails to install correctly. All of this needs to occur before the plugin is actually run, ensuring that the plugin doesn’t have to worry about double-checking everything before it can actually do any code. (This tends to overcomplicate a codebase with no gain.)

Once a plugin and its dependencies are installed, they then need to be maintained. Plugins should receive regular updates as usual, but the end user needs to at least be warned if an update will break compatibility with another. To accommodate urgent, breaking changes, users must be allowed to update plugins even if it would cause incompatibility, and the dependency system should ensure that the other plugins are disabled as needed. (If autoupdates for plugins are added to core, this would still be a manual process.) Trust the user to do the right thing, but ensure they cannot break their own system.

On the other end, uninstalling a plugin should correspondingly offer to remove anything it depends on if not being used by anything else. This again should always be the user’s choice, as depended-on plugins may have use apart from just being a dependency.

Distribution

Getting these plugin dependencies available is the hardest part of the equation. Developers need to be able to depend on (ha ha) the system being available to them, otherwise it’s not going to get adoption regardless of how great it is. This is true for any core feature (like a REST API), but especially so for something that needs to essentially be hidden from the user.

The end goal here is core integration. If the solution doesn’t end up in core at the end, the project has failed, as it’s not ubiquitous. If this happens, throw out what you need and try again, but it must be in core to be a viable solution for many users.

The best alternative, and best way to bootstrap in the meantime, is to aim for integration into Jetpack. Jetpack is one of the most widely used plugins, giving you a huge userbase straight out of the gate. This solution would also be incredibly valuable to Jetpack in making it more modular, and allowing it to shed some of the weight it currently has. Obviously, no one except the Jetpack team has a say over this, but it’s a good way to get your foot in the door. (Plus, it gets the Jetpack team potential extra lock-in benefits, as everyone would need to require Jetpack, albeit temporary.)

There’s precedent in WordPress’ past for this too. Sidebar widgets were originally developed as a plugin by Automattic, then eventually integrated into WordPress core. Widgets used WordPress.com to bootstrap their development process, and in a modern WordPress, would likely piggy-back on Jetpack as well.

Potential Issues

One key potential issue I see is dependency versions. By allowing plugins to require certain versions, it’s possible to end up in situations where unrelated plugins cannot both be installed due to a mutual incompatibility with a library. This could be caused by a plugin requiring too specific a version (“only version 1.2.5, please!”) or an actual incompatibility between major branches. In order to balance these concerns, it may be wise to only allow requiring major versions, with the responsibility on plugin developers to stick to this system.

We also need to be careful to avoid situations like DLL Hell, where mutual incompatibilities between plugins cause installs and upgrades to be impossible without breaking something else. Encouraging plugins to maintain full compatibility is a top priority, which removing the ability to depend on specific versions may help with.

Distribution will be the biggest issue. It may be tempting to bundle with another large plugin (Yoast SEO, WooCommerce, etc), but you risk fragmentation by allowing bundling with more than one plugin, and no one’s going to want to be left without it if it’s that good. We can already see this problem with some of the libraries out there now, where mutually incompatible versions are used by different plugins.

Finally

I’m desperately hoping this post serves as inspiration for someone to create a proper solution to this. I don’t care if it gets solved the way I’ve thought of, there are plenty of other ways to skin this particular cat, and none of them is the “right” way.

(I started on a solution, but truly don’t have the time to dedicate to this. However, I’m willing to offer every piece of code I wrote for the prototype right now to kickstart this.)

What we need is something better than the current solutions. And not just better, but radically better.

Will you be the one to create it?