Run your plugin's checks on a real WordPress site with wp-self-test

Unit tests tell you that a function returns the right value. They don't tell you whether your REST route is registered on this particular site, whether the WP-CLI command still parses its flags after a deploy, or whether your Ability still answers.

wp-self-test (opens in a new tab) runs checks like these inside a real WordPress install. A plugin ships a self-test/manifest.json and a few case classes. The same cases then run from wp-admin, WP-CLI, REST, the Abilities API or a git pre-push hook.

Install

composer require mammothwp/wp-self-test

It is a runtime dependency, because the dashboard, the command and the abilities run inside WordPress. It requires PHP 7.4 or newer.

When the environment is production, the package boots nothing: no commands, abilities, dashboard or manifests, because a run creates data on the site. The environment comes from WP_ENV if it is defined, otherwise from wp_get_environment_type(). The mmwp_self_test_enabled filter overrides this per request. Every surface requires manage_options, which you can change with the mmwp_self_test_capability filter.

Register a manifest

In your plugin's main file:

add_action('mmwp_self_test_register', function (\MammothWP\SelfTest\Registry $registry) {
    require __DIR__ . '/self-test/Things_Read_Case.php';

    $registry->register('my-plugin', __DIR__ . '/self-test/manifest.json', [
        'label' => 'My Plugin',
    ]);
});

The hook runs at init, after the package's classes are loaded, so require the case files inside it. A require at the top of the main file would run too early.

self-test/manifest.json:

{
    "version": 1,
    "cases": [
        {
            "id": "my-plugin-things-read",
            "label": "Things: list over REST and the Ability",
            "category": "content",
            "layers": ["rest", "ability"],
            "class": "Things_Read_Case",
            "requires": [],
            "args": {},
            "covers": {
                "rest": ["my-plugin/v1/things"],
                "ability": ["my-plugin-things/list-things"]
            }
        }
    ]
}

Case ids must be unique across every registered plugin. covers is documentation the package carries along. Your own tests can use it to check that every route, command and ability you register is claimed by a case.

Write a case

Every case has four phases, and each can be called on its own:

preview() describes what the case will do, with no side effects.
run() performs the calls and returns a JSON-serialisable context. It may create data.
validate($context) turns what was observed into check() or skip() rows.
cleanup($context) removes what run() created. The runner stores the context, so cleanup can happen in a later request.

This case only reads, so it keeps the default cleanup(), which removes nothing:

use MammothWP\SelfTest\Case_Base;

class Things_Read_Case extends Case_Base
{
    public function preview()
    {
        return [
            'List things over REST (expects 200)',
            'List things through the list-things ability',
        ];
    }

    public function run()
    {
        $o = [];
        $o['rest'] = $this->rest('GET', '/my-plugin/v1/things', ['limit' => 5]);
        $o['list'] = $this->ability('my-plugin-things/list-things', ['limit' => 5]);

        return ['observed' => $o];
    }

    public function validate(array $context)
    {
        $o = $context['observed'];

        return [
            $this->check('rest', 'GET /things returns 200', $this->status($o, 'rest') === 200, $this->detail($o, 'rest')),
            $this->ability_check($o, 'list', 'list-things returns an items list', function ($data) {
                return isset($data['items']) && is_array($data['items']);
            }),
        ];
    }
}

rest() dispatches the request in the same process through rest_get_server() and returns the status and data. ability() calls wp_get_ability($name)->execute($input) and returns found, ok, data and error. When the Abilities API is not available on the site, ability_check() reports a skip instead of a failure.

The route and ability in this example are the ones from the wp-kit guide, but the case works the same for any plugin.

Run it

wp mmwp self-test my-plugin --format=table

That runs one plugin's cases. Without a slug, it runs every registered case. The actions are list, preview, run [--cleanup], cleanup and report. The command exits with code 1 when a case fails, so a script can stop on it.

The same cases are available from:

wp-admin: Tools > Self-test, on non-production sites only.
REST: mmwp/v1/self-test/cases, preview, run, cleanup and report, using cookie authentication with X-WP-Nonce.
Abilities: mmwp/self-test-list-cases, -preview, -run, -cleanup and -get-report. Run and cleanup need confirm: true.

Before every push

Add this to your plugin's composer.json:

"scripts": {
    "test": "bash tests/run-tests.sh",
    "post-install-cmd": ["@php vendor/mammothwp/wp-self-test/bin/install-hooks"]
}

install-hooks writes .githooks/pre-push and sets core.hooksPath. On git push, the hook runs php -l on every tracked PHP file and then composer run-script test.

When to use it

Your plugin registers REST routes, WP-CLI commands or Abilities, and you want to confirm they work on a real install.
You want one set of checks that can run from the admin screen, the command line or a hook.
You need test runs that clean up the data they create.

When not to

You only need unit tests for pure PHP logic.
The site you want to check is production and you are not prepared to create test data there. The package stays off on production unless you turn it on.