Split architecture doc

Co-authored-by: Grant Gainey <ggainey@users.noreply.github.com>

[noissue]

.github/workflows/docs.yml

@@ -0,0 +1,31 @@
+name: "Docs"
+
+on:
+  workflow_call:
+
+jobs:
+  test:
+    runs-on: "ubuntu-20.04"
+    steps:
+      - uses: "actions/checkout@v4"
+      - uses: "actions/cache@v4"
+        with:
+          path: "~/.cache/pip"
+          key: "${{ runner.os }}-pip-${{ hashFiles('**/*requirements.txt', '**/*constraints.lock', '**/setup.py', '**/pyproject.toml') }}"
+          restore-keys: |
+            ${{ runner.os }}-pip-
+
+      - name: "Download wheels"
+        uses: "actions/download-artifact@v4"
+        with:
+          name: "pulp_cli_packages"
+      - name: "Set up Python"
+        uses: "actions/setup-python@v5"
+        with:
+          python-version: "3.11"
+      - name: "Install Test Dependencies"
+        run: |
+          # pip install dist/pulp_cli-*.whl pulp-glue/dist/pulp_glue-*.whl -r doc_requirements.txt
+          pip install -r doc_requirements.txt
+      - name: Build docs
+        run: make docs

.github/workflows/pr.yml

@@ -18,6 +18,10 @@ jobs:
     needs:
       - "lint"
     uses: "./.github/workflows/test.yml"
+  docs:
+    needs:
+      - "lint"
+    uses: "./.github/workflows/docs.yml"
   codeql:
     needs:
       - "lint"
@@ -56,6 +60,7 @@ jobs:
       - "check-commits"
       - "lint"
       - "test"
+      - "docs"
       - "codeql"
     if: "always()"
     steps:

CHANGES/pulp-glue/+architecture.doc

@@ -0,0 +1 @@
+Improve the docs split for the pulp-glue architecture documentation.

Makefile

@@ -36,9 +36,13 @@ tests/cli.toml:
 test: | tests/cli.toml
 	pytest -v tests
 
+docs:
+	pulp-docs build
+
 servedocs:
 	pulp-docs serve -w pulp-glue/pulp_glue -w pulpcore/cli/common/generic.py
 
+# This is for the old docs. We may need it once more to push redirect pages.
 site:
 	mkdocs build
 
@@ -66,5 +70,5 @@ $(foreach LANGUAGE,$(LANGUAGES),pulpcore/cli/%/locale/$(LANGUAGE)/LC_MESSAGES/me
 	msgfmt -o $@ $<
 
 compile_messages: $(foreach LANGUAGE,$(LANGUAGES),$(foreach GLUE_PLUGIN,$(GLUE_PLUGINS),pulp-glue/pulp_glue/$(GLUE_PLUGIN)/locale/$(LANGUAGE)/LC_MESSAGES/messages.mo)) $(foreach LANGUAGE,$(LANGUAGES),$(foreach CLI_PLUGIN,$(CLI_PLUGINS),pulpcore/cli/$(CLI_PLUGIN)/locale/$(LANGUAGE)/LC_MESSAGES/messages.mo))
-.PHONY: build info black lint test servedocs site
+.PHONY: build info black lint test docs servedocs site
 .PRECIOUS: $(foreach LANGUAGE,$(LANGUAGES),$(foreach GLUE_PLUGIN,$(GLUE_PLUGINS),pulp-glue/pulp_glue/$(GLUE_PLUGIN)/locale/$(LANGUAGE)/LC_MESSAGES/messages.po)) $(foreach LANGUAGE,$(LANGUAGES),$(foreach CLI_PLUGIN,$(CLI_PLUGINS),pulpcore/cli/$(CLI_PLUGIN)/locale/$(LANGUAGE)/LC_MESSAGES/messages.po))

doc_requirements.txt

@@ -1,3 +1 @@
 pulp-docs @ git+https://github.com/pulp/pulp-docs@main
-.
-./pulp-glue

pulp-glue/staging_docs/dev/learn/architecture.md

@@ -0,0 +1,64 @@
+# Pulp Glue Architecture
+
+The `pulp-glue` library is an abstraction layer that lets you perform high-level operations in pulp.
+Its goal is to abstract interacting with the REST api by parsing the api docs, and waiting on tasks and task groups.
+It is shipped as a separate python package to allow broad use across multiple projects, such as `pulp-squeezer` and `pulpcore`.
+Pulp GLUE is developed in the pulp-cli repository mainly for the consolidated tests effort.
+To this end, `pulp-glue` is the go-to place for all known version-dependent Pulp API subtleties and their corresponding fixes (see Version-dependent codepaths below).
+
+## OpenAPI
+
+This is the part in `pulp_glue` that uses [`requests`](https://requests.readthedocs.io/) to perform low level communication with an `OpenAPI 3` compatible server.
+It is not anticipated that users of Pulp Glue need to interact with this abstraction layer.
+
+## Contexts
+
+Pulp Glue provides the [`PulpContext`][pulp_glue.common.context.PulpContext] encapsulating the [`OpenAPI`][pulp_glue.common.openapi.OpenAPI] object.
+You can use its `call` method to interact with any operation designated by its operation id.
+In addition, to perform specific operations on entities, glue ships a bunch of [`PulpEntityContext`][pulp_glue.common.context.PulpEntityContext] subclasses.
+
+### Deferred Api and Entity lookup
+
+There are some facilities that perform deferred loading to prevent premature http requests.
+Those include:
+
+  - `PulpContext.api`: When accessed, the `api.json` file for the addressed server will be read or downloaded and processed.
+    Scheduled version checks will be evaluated at that point.
+  - `PulpContext.needs_version`: This function can be used at any time to declare that an operation needs a plugin in a version range.
+    The actual check will be performed immediately when `api` already was accessed, or scheduled for later.
+  - `PulpEntityContext.entity`: This property can be used to collect lookup attributes for entities by assigning dicts to it.
+    On read access, the entity lookup will be performed through the `api` property.
+  - `PulpEntityContext.pulp_href`: This property can be used to specify an entity by its URI.
+    It will be fetched from the server only at read access.
+
+### Type Registries
+
+For some operations, it is important to know all specialized subclasses of a Pulp Entity type.
+Therefore certain general Entity types (usually denoted Master/Detail in Pulp) come with a `TYPE_REGISTRY`.
+
+### Plugin Requirements / Version Dependant Code Paths
+
+`PluginRequirement` is the abstracted concept of the existence or non-existence of a plugin in the target Pulp server.
+A `PluginRequirement` is therefore instantiated with the `app_label` of a plugin and optionally a PEP-440 compatible version specifier set.
+Additionally an `inverted` flag can be passed to assert on the absence instead of the presence.
+Lastly, you can add the name of a `feature` to help build useful failure messages.
+
+Typically, plugin requirements are checked by passing into `PulpContext.has_plugins` or `PulpContext.needs_plugin`, but can also be used indirectly by assigning to a capability.
+
+### Capabilities
+
+Some Entities may provide support for different Pulp concepts based on their plugins version.
+e.g. `Pulp Import Export` for a specific repository type may be added in a certain Pulp Plugin version.
+You can add a `capability` to the `PulpEntityContext` subclass with an attached `PluginRequirement`.
+Whenever glue attempts to perform the corresonding action, the capabilities are first checked against the server's versions.
+
+### API quirks
+
+Sometimes, the api documentation simply does not reflect the actual api behaviour.
+In these cases one can add version dependent api quirks to Pulp Glue that will be executed as a hook after fetching the api documentation.
+See the `pulp_glue.common.api_quirk` decorator.
+
+## Plugin System
+
+Pulp Glue comes with a plugin interface to be easily extendible.
+Multiple plugins can be provided by the same Python package and some plugins are shipped with the pulp-glue core package.

staging_docs/dev/learn/architecture.md

@@ -1,55 +1,39 @@
-# Architecture
-
-The Pulp CLI architecture is described in this section.
+# Pulp CLI Architecture
 
 ## Pulp Glue
 
-Pulp CLI provides the `pulp-glue` library as an abstraction layer that lets you perform high-level operations in pulp.
-Its goal is to abstract interacting with the REST api by parsing the api docs, and waiting on tasks and task groups.
-It is shipped as a separate python package to allow broad use across multiple projects, such as `pulp-squeezer` and `pulpcore`.
-To this end, `pulp-glue` is the go-to place for all known version-dependent Pulp API subtleties and their corresponding fixes (see Version-dependent codepaths below).
-
-### OpenAPI
-
-This is the part in `pulp_glue` that uses `requests` to perform low level communication with an `openapi 3` compatible server.
+Pulp CLI uses the [`pulp-glue`](site:pulp-glue/docs/learn/architecture) library as an abstraction layer to perform high-level operations in pulp.
 
-### Contexts
-
-Pulp-glue provides the [`PulpContext`][pulp_glue.common.context.PulpContext] encapsulating the [`OpenAPI`][pulp_glue.common.openapi.OpenAPI] object.
-You can use its `call` method to interact with any operation designated by its operation id.
-In addition, to perform specific operations on entities, glue ships a bunch of [`PulpEntityContext`][pulp_glue.common.context.PulpEntityContext] subclasses.
-
-#### Deferred Api and Entity lookup
+## Deferred Api and Entity lookup
 
 In order to be able to access every (sub-)command's help page,
 it is necessary that no code outside of the final performing command callback accesses the `api` property of the `PulpContext`.
-There are some facilities that perform deferred loading to help with that requirement.
-Those include:
+See `pulp-glue` section about [deferred lookup](site:pulp-glue/docs/learn/architecture#deferred_api_and_entity_lookup).
 
-  - `PulpContext.api`: When accessed, the `api.json` file for the addressed server will be read or downloaded and processed.
-    Scheduled version checks will be reevaluated.
-  - `PulpContext.needs_version`: This function can be used at any time to declare that an operation needs a plugin in a version range.
-    The actual check will be performed when `api` was accessed for the first time, or immediately afterwards.
-  - `PulpEntityContext.entity`: This property can be used to collect lookup attributes for entities by assigning dicts to it.
-    On read access, the entity lookup will be performed through the `api` property.
-  - `PulpEntityContext.pulp_href`: This property can be used to specify an entity by its URI.
-    It will be fetched from the server only at read access.
+## Plugin System
 
-## Pulp CLI
+The Pulp CLI is designed with a plugin structure. Plugins can either live in the pulp-cli package or be shipped independently.
+By convention, all CLI plugins are modules in the open namespace `pulpcore.cli`.
+A plugin must register itself with the main app by specifying its main module as a `pulp_cli.plugins` entrypoint.
 
-### Plugin System
+=== "pyproject.toml"
 
-The Pulp CLI is designed with a plugin structure. Plugins can either live in the pulp-cli package or be shipped independently.
-By convention, all parts of the CLI are packages in the open namespace `pulpcore.cli`.
-A plugin can register itself with the main app by specifying its main module as a `pulp_cli.plugins` entrypoint in `setup.py`.
+    ```toml
+    [project.entry-points."pulp_cli.plugins"]
+    myplugin = "pulpcore.cli.myplugin"
+    ```
 
-```python
-entry_points={
-    "pulp_cli.plugins": [
-        "myplugin=pulpcore.cli.myplugin",
-    ],
-}
-```
+=== "setup.py"
+
+    ```python
+    entry_points={
+        "pulp_cli.plugins": [
+            "myplugin=pulpcore.cli.myplugin",
+        ],
+    }
+    ```
+
+---
 
 The plugin should then attach subcommands to the `pulpcore.cli.common.main` command by providing a `mount` method in the main module.
 
@@ -65,7 +49,7 @@ def mount(main: click.Group, **kwargs: Any) -> None:
     main.add_command(my_command)
 ```
 
-### Contexts
+## Contexts
 
 In `click`, every subcommand is accompanied by a `click.Context`, and objects can be attached to them.
 In this CLI we attach a [`PulpCLIContext`][pulpcore.cli.common.generic.PulpCLIContext] to the main command, which inherits from `pulp-glue`'s [`PulpContext`][pulp_glue.common.context.PulpContext].
@@ -88,11 +72,11 @@ def my_command(ctx, pulp_ctx):
 @my_command.command()
 @pass_entity_context
 def my_sub_command(entity_ctx):
-    ... href = ...
-    entity_ctx.destroy(href)
+    entity_ctx.entity = {"name": "myentity")
+    entity_ctx.destroy()
 ```
 
-### Generics
+## Generics
 
 For certain often repeated patterns like listing all entities of a particular kind,
 we provide generic commands that use the underlying context objects.