Blueprints v2 Specification

[adamziel]

The previous, TypeScript-powered, Blueprint schema only makes sense in the browser, not in a portable PHP library. Let's design one that does.

The existing Blueprints should keep working as much as possible. The library could detect an outdated syntax and rewrite it to the new one behind the scenes.

Converging with wp-env

wp-env has its own config format.

Let's converge wp-env and Blueprints. This could mean either:

• Transitioning both to the v2 schema format
• Replacing wp-env with Blueprints using Docker as a backend as both solve the same problem

cc @youknowriad @noahtallen

Runtime setup options

The following Blueprint options are for setting up the PHP.wasm runtime. They make sense in Playground and Docker where we need to set up the runtime, but not in native PHP where the runtime is already provided:

{
	"preferredVersions": {
		"php": "7.4"
	},
	"phpExtensionBundles": [
		"kitchen-sink"
	],
	"features": {
		"networking": true
	}
}

An alternative syntax would make that apparent and allow for environment-specific overrides. Here's one idea:

{
	"php": {
		"version": "7.4",
		"extensions": [ "libcurl", "libxml" ],
		"playground": {
			"networking": "fetch()"
		},
		"docker": {
			"networking": false
		}
	}
}

The Native PHP driver could log a message saying that PHP setup instructions are unsupported and were ignored. Optionally, it could verify if the requested PHP version and extensions match.

Open questions

How should PHP.ini setup be defined and how should it work? Should we do an mu-plugin with ini_set()?

Client setup

The following Blueprint options only make sense in the browser where we control the client:

{
	"landingPage": "/wp-admin/",
	"steps": [
		{
			"step": "login",
			"username": "admin",
			"password": "password"
		}
	]
}

What should they do when building a site using the native PHP CLI. Perhaps they could start a server, open a web browser, and open /wp-admin/ in a logged in state? Controlling the browser isn't trivial, so presumably there would be a special, one-time, magic login link with a ?redirect= query arg or so. If so, we could express it as follows:

{
	"onBoot": {
		"openURL": "/wp-admin/",
		"loginAs": "admin"
	}
}

What if we cannot start the server or open the browser, though? For example, we're building in a headless mode. Should the runtime ignore those options? Should it issue a warning? Should these options be runtime-specific?

WordPress setup

The following Blueprint options are responsible for setting up WordPress:

{
	"preferredVersions": {
		"wp": "5.9"
	},
	"steps": [
		{
			"step": "installPlugin",
			"pluginZipFile": {
				"resource": "wordpress.org/plugins",
				"slug": "friends"
			}
		},
		// ... other steps ...
	]
}

They could be bundled under a single top-level key as follows:

{
	"wpVersion": "5.9",
	"steps": [
		{
			"step": "installPlugin",
			"pluginZipFile": {
				"resource": "wordpress.org/plugins",
				"slug": "friends"
			}
		},
		// ... other steps ...
	]
}

Open questions

• In which directory should WordPress be installed? Current working directory? A directory provided by the runtime, e.g. blueprint --target=./build? How should that directory be referenced in steps like writeFile or runPHP? Using special syntax like %DOCROOT%?

Shorthand properties

The step-by-step syntax is overly verbose for most developers, let's make it shorter.

The verbose syntax must remain available as it's the only way to enable reproducible, reliable builds. Some developers may need to correct site options or import content after plugin A was activated but before plugin B was installed and Blueprints should empower them. However, most the developers will never run into nuances like that.

Let's design a schema that's convenient for most use-cases, yet powerful enough to handle nuanced setups.

Here's what it could look like:

{
	"wpVersion": "5.9",
	"plugins": [
		"woocommerce",
		"gutenberg"
	],
	// Optional: all plugins are activated if this isn't specified:
	"activatePlugins": [
		"woocommerce"
	],

	"themes": [ "pendant", "twentytwentythree" ],
	"siteOptions": {
		"blogname": "My new site"
	},
	"wpConfig": {
		"WP_DEBUG": true
	},
	"content": [
		"./theme-unit-test-data.xml",
		"./accessibility-testing-content.xml"
	],
	"files": {
		"wp-content/mu-plugins/0-my-mu-plugin.php": [
			"<?php",
			"add_filter('wp_init', function() {"
			"echo 'Hello, world';",
			"}"
		]
	},
	"steps": [
		// All the above options are translated into steps and prepended to the `steps` array
		// in an opinionated order. Whenever the developer needs to customize the execution
		// order, they could re-express the relevant part of the Blueprint using explicitly-ordered
		// operations.
	]
}

Metadata

WordPress.org could have a Blueprints community space on WordPress.org for folks to share and collaborate on WordPress setups. Similarly to plugins metadata, Blueprints would need to carry some meta information as well.

Here's what a top-level meta key could look like:

{
    "meta": {
        "title": "WooCommerce Developer Environment",
        "description": "A local development environment for WooCommerce that includes WP-CLI.",
        "version": "0.0.8",
        "author": "zieladam",
        "contributors": ["zieladam", "dmsnell"],
        "authorUrl": "https://example.com",
        "donateLink": "https://example.com",
        "tags": ["woocommerce", "developer environment"],
        "license": "GPLv2 or later",
    }
}

Which other fields would be required? Do we need to separate contributors and authors? Should each contributor be able to provide their own website URL?

It could be validated using the JSON Schema below. Note the main Blueprints library could simply ignore the meta key. The validation would happen exclusively during the submission process to the community space.

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "properties": {
        "meta": {
            "type": "object",
            "properties": {
                "title": {
                    "type": "string"
                },
                "description": {
                    "type": "string",
                    "maxLength": 150
                },
                "version": {
                    "type": "string"
                },
                "author": {
                    "type": "string"
                },
                "authorUrl": {
                    "type": "string",
                    "format": "uri"
                },
                "contributors": {
                    "type": "array",
                    "items": {
                        "type": "string"
                    },
                    "minItems": 1
                },
                "tags": {
                    "type": "array",
                    "items": {
                        "type": "string",
                        "enum": ["woocommerce", "developer environment"]
                    }
                },
                "donateLink": {
                    "type": "string",
                    "format": "uri"
                },
                "license": {
                    "type": "string"
                }
            },
            "required": ["author", "version", "title", "contributors", "tags", "license"]
        }
    },
    "required": ["meta"]
}

What new features would be useful?

• Test: Create a PR to see if things fail blueprints#32
• Blueprint: Support resources defined as URLs, not just objects wordpress-playground#1020
• Blueprint idea: Slide deck setup blueprints#46
• Creating a post and being able to reference it later, e.g. redirect to the editor in the openURL section

cc @bgrgicak @dmsnell @mtias @tellyworth @danielbachhuber

[youknowriad]

Why do you think we need a top level "wp"?

[dmsnell]

given that we already have a JSON format, and we're talking about creating these from PHP, have we given more thought to a function interface?

(new Blueprints())
	-> withPHP( '8.2' )
	-> withWP( 'latest' )
	-> wpConfig( array(
		'WP_DEBUG' => true
	) )
	-> withPlugins( 'gutenberg', 'mathml-block' )
	-> addContent(
		'./theme-unit-test-data.xml',
		'./accessbilitiy-testing-content.xml'
	)
	-> runPHP( <<<PHP
		<?php
		add_filter( 'wp_init', function() {
			echo 'Hello, world';
		}
PHP)

this deals with the multi-line input problem, and provides an interface that gives auto-complete and inline documentation if we build the blueprints class properly.

[bgrgicak]

How should PHP.ini setup be defined and how should it work? Should we do an mu-plugin with ini_set()?

This is what I'm doing for the debugger now. It seems cleaner than messing with php.ini files.

What if we cannot start the server or open the browser, though? For example, we're building in a headless mode. Should the runtime ignore those options? Should it issue a warning? Should these options be runtime-specific?

I wouldn't bother with this now. It's ok for some steps to only work in some environments. It's ok if the code skips these steps. We just need to be clear in docs about which environments are supported.

In which directory should WordPress be installed? Current working directory?

The root dir should be a runtime option. All other paths could be relative so there would be no need for a reference.

"plugins": [
			"woocommerce",
			"gutenberg"
		],
		// Optional: all plugins are activated if this isn't specified:
		"activatePlugins": [
			"woocommerce"
		],

Would it be better if we kept the context of a step inside a single place when possible? This example could be written like this.

"plugins": [
			"woocommerce",
			{
			  "slug": "gutenberg",
			  "activate": false
			}
		],

[adamziel]

Why do you think we need a top level "wp"?

Just my first instinct to distinguish between the different section. I think you're right, though. It is a WordPress tool, everything inside wp should be top-level. I updated the issue to reflect that.

[youknowriad]

I noticed that for wp-env, we have core key instead wpVersion. Basically we want to allow using a specific local install of WordPress, or a specific URL or a specific version. Do you think it's worth supporting. Maybe the tool can be smart enough to understand if it's a path or number or things like that.

[adamziel]

Basically we want to allow using a specific local install of WordPress, or a specific URL or a specific version.

It's the same for Blueprints, you can either specify 6.4 or https://w.org/latest.zip – I'd love to keep supporting that. As for the local paths, it needs more though about portability – would we end up with local-only Blueprints, or perhaps there's a way to make "core": "./wordpress" portable somehow? I'm not sure yet.

[noahtallen]

Another thing that wp-env has is the concept of a separate testing environment, which can accept a lot of the config options. The main purpose of that is to have a separate database and server you can run local e2e tests against, but it does add complexity to the config.

[youknowriad]

@noahtallen I've always wondered whether that multiple environment concept was necessary. It adds complexity to the config but also adds some burden on the user as you never know which environment you're using. One simple solution to that use-case would have been two config files and you provide the desired config file to the CLI. Would that solve all of these use-cases you think?

[noahtallen]

I always thought it was relatively clear which environment you use -- after all, the CLI lets you know about the separate ports, and the main 8888 port is almost always used. But wp-env's roots were zero config, and at the time, I think it still had the second environment. So it was mostly a behind-the-scenes feature that helped support e2e tests in Gutenberg. As more config options were added over the years, the more complicated it got. (I think -- part of wp-env was developed before I started working on it)

But I think the second config file would practically work ok if we were starting from scratch. You'd probably still be running e2e tests via the package.json scripts anyways, so most devs wouldn't have to worry about what args to pass in the context of big repos like Gutenberg. My assumption was also that most people would use about the same config for both environments. So in practice, the flexible config can just let you override settings here or there as needed to get e2e testing working.

[lgersman]

I like the idea of Blueprints and would love to see it integrated within wp-env.

But from my opinion it needs just a bit of documentation.

Why not just using wp-env Lifecycle Scripts to execute a Blueprints configuration as-is within a Blueprints docker image (or even by installing the additional Blueprints NPM/PHAR package beside wp-env in the local development setup) ?

This effort needs more or less no new code (only documentation) and is portable across operating systems.

I see Blueprints as an extension to the current wp-env core functionality. It should not bloat wp-env.

On the other side : If Blueprints should also target wp-cli - which would make sense from my opinion - then it sounds logical to have a Blueprints PHP version.

But I would not add the Blueprints to wp-env. If Blueprints would be available to wp-cli (as an wp-cli package), it can easily be used in wp-env Lifecycle Scripts to customize an wp-env installation. And additionally in any WordPress installation by just using just it as a wp-cli command (installed as additional wp-cli package).

[artpi]

WordPress setup
The following Blueprint options are responsible for setting up WordPress:

I love the separation between those different actions because, it gets at something I've been mentally struggling for a while.
One thing that I wasn't able to figure out in the current implementation of blueprints is:

How do I change a blueprint without losing the current state? In different terms: How can I change the blueprint from inside the site itself?

Some actions are non-destructive - like installing plugins, adding a PHP file, etc.
It would be super sweet if these actions were accessible inside the WP instance, ideally as REST API, but in general, if I could perform them in a non-destructive manner to the site itself.

I would imagine it would split the blueprint handling into 2 parts:

1. Bootstrapper that deals with blueprint stuff related to hosting and wp version and installs the plugin below:
2. A plugin that exposes all the actions that could be performed when you already have a site.

That way, that second plugin can function independently as a "WordPress driver" for all those sites that do have a host, but could use some automation.

I could also imagine that the set of actions assuming you already have a a site is more interesting for folks than the set of hosting-related actions.
Then this repository could really focus on that plugin

[adamziel]

that second plugin can function independently as a "WordPress driver" for all those sites that do have a host, but could use some automation.

That's a super interesting idea @artpi. I like it! Each step could correspond to a local REST API call or a PHP function call, and that would mark a clear path to how Blueprints could eventually be merged into WordPress core.

I'll still try to think of some steps where that model wouldn't work. So far, though, I couldn't find any.

This would split the Blueprint library into the two parts that you've outlined:

1. Environment bootstrapper that could work with Playground, wp-now, wp-env, etc.
2. Site configurator that would configure the site from the context of that site

A few questions to explore:

• How much code would overlap would there be, if any?
• Would this model be less secure in any way? Would keeping the logic in the bootstrapper be more resilient in any way?
• Would the environment bootstrapper also act as a Blueprint preprocessor to, e.g., pre-fetch all the network resources before passing the steps to the WordPress plugin?

How do I change a blueprint without losing the current state? In different terms: How can I change the blueprint from inside the site itself?

I don't follow @artpi, what do you mean by changing the Blueprint? Would you want WordPress to go back into your original blueprint.json file and rewrite it?

---

Edit:

Hmm, I'm having second thoughts. Intertwining site setup manipulation with a code relying on that setup generates a new class of possible errors. Say we're running a Blueprint via a REST API request. That Blueprint changes some files, site options, and invalidates caches. Then, in the original REST API request, WordPress calls get_option
or requires a changed files and we get an undefined result. What should happen if a request to a regular site's REST API turns it into a multisite? There's value in running Blueprints in a non-WordPress context even if the runner is shipped as a WordPress plugin.

[adamziel]

I like the idea of Blueprints and would love to see it integrated within wp-env.

+1 @lgersman, I would love that too. I wouldn't tightly couple the two, as Blueprints have a ton of value outside of wp-env, but it would be lovely to integrate the two. This aspect of Blueprints is explored in WordPress/blueprints#1

[flexseth]

@adamziel

In which directory should WordPress be installed? Current working directory?

My vote is: "Right here"... (relative)

---

One of the biggest struggles I had when working with wp-env and why I ultimately switched to Local was just not being able to easily access the files.

How this works in the various modes of wp-now

• Plugin
• Theme
• and Playground Mode

It's an important distinction to make. We can work on that, a good first step would be easily knowing where the filesystem is and how you should intend to work with files in this Playground instance.

Appreciate the diligence on the spec!!

[artpi]

I currently love the JSON syntax because it has no dependencies. Even if using it inside PHP, I don't have to worry about pulling external libraries.

Additionally, its easy to work on in https://playground.wordpress.net/builder/builder.html

I would also love a tool that would let me convert existing JSON to the PHP syntax if I am ready to make the switch