Runtime Verification and WP-API

Automating PHPUnit for your WordPress project for more productive development.

The second in a series of posts that investigates using strongly-typed first-class functions with WordPress WP-API to create a composable, testable, verifiable, and productive method of REST API development.

Previously: Strongly Typed WP-API.


Context switching is a productivity killer. What exactly constitutes a context switch though?

Moving to a ping in Slack away from a Vim window? Definitely a context switch.

Switching via cmd-tab between a source code editor and browser window? Also a context switch. Yes, even when duck-duck-going the error from the console.

Everything that reduces context switching during development is a productivity win.

Debugging is a Productivity Killer

Time spent searching logs and reconstructing failure cases from production bugs is time not spent shipping.

It is also time that was not accounted for in the 100% accurate development estimate given to the project manager to complete the task.

Passing a string value to a function that expects an int: bug. Typing the incorrect string name of a function in WordPress’s add_filter: another bug. Calling a method on a WP_Error instance because it was assumed to be a WP_User: bug.

All of these things are caught by static type analysis.

They may all seem like small bugs but they can quickly add up to a non-trivial amount of time debugging. Perhaps these bugs will be discovered quickly at runtime, but that requires the correct codepaths are executed in a runtime. Is every code path in a project going to be executed between each source code change? No.

Static analysis will increase productivity by uncovering these bugs. But even with a 100% typed, fully analyzed codebase validating running code output is still necessary.

Automating runtime validation is another tool to increase productivity.

Runtime Verification

Psalm enforces correct types and API usage. Checking the correctness of the runtime code still requires some manual steps, like booting up an entire WordPress stack. Previously, wp-env was used to verify that the endpoint actually worked.

wp-env start
curl http://localhost:8889/?rest_route=/totes/not-buggy
{"result": "not buggy"}

This isn’t going to scale well when the number of endpoints and the number of ways to call them increases. Jumping from an editor to a browser and back isn’t the best recipe for productive coding sessions either.

Time for automated tests.

In the world of PHP, that means PHPUnit.

The bare minimum code to test totes_not_buggy() is a single implementation of PHUnit\Framework\TestCase with a single test method. It will live in tests/Totes/TotesTest.php:

namespace Totes;

use WP_REST_Request;
use WP_REST_Server;

class TotesTest extends \PHPUnit\Framework\TestCase {

     * @return void
    function testTotesNotBuggy() {
        $request = new WP_REST_Request( 'GET', '/totes/not-buggy' );
        $response = totes_not_buggy( $request );
        $this->assertEquals( [ 'status' => 'not buggy' ], $response->get_data( ) );

To run PHPUnit, the dependency needs to be installed.

composer --dev require phpunit/phpunit

Now run the test:

./vendor/bin/phpunit tests

// yadda yadda

Tests: 1, Assertions: 0, Errors: 1.

The error shows that we don’t have WordPress APIs available to our run runtime:

1) Totes\TotesTest::testTotesNotBuggy
Error: Class 'WP_REST_Request' not found

WordPress is a dependency of this project. It won’t work without it. Time to install it:

composer require --dev johnpbloch/wordpress

The johnpbloch/wordpress package by default will install the WordPress source code in ./wordpress. Setting up a whole WordPress stack to work on some source code: productivity killer. “No install” is faster than any five minute install no matter how famous it is.

If WordPress were a PSR-4 compliant project there wouldn’t be anything left to do. But it isn’t. To illustrate, run the test again and observe the result is the same.

Since Composer doesn’t know how to autoload WordPress source code, PHPUnit needs to be taught how to find WordPress APIs during test execution. A perfect place for this is via PHPUnit’s "bootstrap" system.

Generate a config and tell PHPUnit to use a custom"bootstrap":

./vendor/bin/phpunit --generate-config
PHPUnit 9.0.1 by Sebastian Bergmann and contributors.

Generating phpunit.xml in /Users/beau/code/wp-api-fun

Bootstrap script (relative to path shown above; default: vendor/autoload.php): tests/bootstrap.php
Tests directory (relative to path shown above; default: tests): 
Source directory (relative to path shown above; default: src): 

Generated phpunit.xml in /Users/beau/code/wp-api-fun

This generates ./phpunit.xml and tells phpunit to run test/bootstrap.php before executing tests.

Time to hunt down all of the WordPress dependencies for this test.

One way to find which PHP files need to be included is to keep running the tests and including the files that define the missing classes and functions.

For example, the current error is that WP_REST_Request is not defined.

ack 'class WP_REST_Request' wordpress
29:class WP_REST_Request implements ArrayAccess {

Now add wordpress/wp-includes/rest-api/class-wp-rest-request.php.

Keep going until it passes. This is the end result for now. Note that this is – at this time in our development – 100% of our plugin’s runtime dependencies.


define( 'ABSPATH', __DIR__ . '/../wordpress' );
define( 'WPINC', '/wp-includes' );

require_once __DIR__ . '/../wordpress/wp-includes/functions.php';
require_once __DIR__ . '/../wordpress/wp-includes/plugin.php';

require_once __DIR__ . '/../wordpress/wp-includes/class-wp-error.php';
require_once __DIR__ . '/../wordpress/wp-includes/pomo/translations.php';
require_once __DIR__ . '/../wordpress/wp-includes/l10n.php';
require_once __DIR__ . '/../wordpress/wp-includes/class-wp-http-response.php';
require_once __DIR__ . '/../wordpress/wp-includes/rest-api/class-wp-rest-request.php';
require_once __DIR__ . '/../wordpress/wp-includes/rest-api/class-wp-rest-response.php';
require_once __DIR__ . '/../wordpress/wp-includes/rest-api/class-wp-rest-server.php';
require_once __DIR__ . '/../wordpress/wp-includes/rest-api.php';
require_once __DIR__ . '/../wordpress/wp-includes/load.php';

add_action( 'rest_api_init', 'totes_register_endpoints' );

/** @psalm-suppress InvalidGlobal */
global $wp_rest_server;

$wp_rest_server = new WP_REST_Server();

do_action( 'rest_api_init' );

Now that Composer can install WordPress and PHPUnit, the CI can run these tests too. Add it to the GitHub action:

+    - name: Unit Tests
+      run: vendor/bin/phpunit

Runtime verification of any new route can now be captured in a unit test. Once in a unit test it can be ran in all sorts of ways.

Bonus, with XDebug configured PHPUnit will also report coverage analysis when proper @covers annotations are added:

vendor/bin/phpunit test --coverage-html coverage-report
PHPUnit 9.0.1 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 68 ms, Memory: 8.00 MB

OK (1 test, 1 assertion)

Generating code coverage report in HTML format ... done [12 ms]

68 millisecond execution time with 100% coverage of a one-line function assigned a CRAP score of 1. Gotta love that new project smell.

Screen capture of PHPUnit coverage report
Screen capture of a PHPUnit coverage report.

Safety Nets Engaged

Between Psalm and PHPUnit we now have static analysis and automated runtime tests.

Next up we’ll dive into Higher-Order Kinds with Psalm and start using them with WP-API to create a declarative, composable API.