Skip to content

Commit

Permalink
more docs cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
theoephraim committed Jan 15, 2025
1 parent 0c88a56 commit 5fff14b
Show file tree
Hide file tree
Showing 12 changed files with 61 additions and 51 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,8 @@ import { Code } from "@astrojs/starlight/components";
import code2 from "./config-example.ts?raw";
---

<Code code={code2} lang="ts" title=".dmno/config.mts - sample config file" />
<Code
code={code2}
lang="ts"
title=".dmno/config.mts - example dmno config schema"
/>
46 changes: 24 additions & 22 deletions packages/docs-site/src/components/homepage/config-example.ts
Original file line number Diff line number Diff line change
@@ -1,39 +1,41 @@
import {
DmnoBaseTypes, defineDmnoService, switchBy, configPath, pick,
} from 'dmno';
import { EncryptedVaultDmnoPlugin, EncryptedVaultTypes } from '@dmno/encrypted-vault-plugin';
import { DmnoBaseTypes, defineDmnoService, switchBy, configPath, pick } from 'dmno';
import { OnePasswordDmnoPlugin, OnePasswordTypes } from '@dmno/1password-plugin';

// use a plugin to fetch secrets from a secure backend like 1Password
// use plugins to fetch sensitive config from secure locations like 1Password
const onePassSecrets = new OnePasswordDmnoPlugin('1pass', { token: configPath('OP_TOKEN') });
// or store them encrypted within your repo
const vaultFileSecrets = new EncryptedVaultDmnoPlugin('vault', { key: configPath('DMNO_VAULT_KEY') });

export default defineDmnoService({
// re-use items defined in other services

// more config specific to this service
schema: {
API_KEY: pick(),
DB_URL: pick(),
DMNO_ENV: pick(),
DMNO_VAULT_KEY: {
// re-use existing types with validation and docs info built-in
extends: EncryptedVaultTypes.encryptionKey,
},
OP_TOKEN: {
extends: OnePasswordTypes.serviceAccountToken,
APP_ENV: {
// docs are built in, and flow into generated TypeScript types / IntelliSense
description: 'our custom environment flag',
extends: DmnoBaseTypes.enum(['development', 'preview', 'staging', 'production', 'test']),
value: 'development', // default value
},
SAAS_API_KEY: {
// built-in types have validation helpers for many common needs
extends: DmnoBaseTypes.string({ startsWith: 'pk_', isLength: 35 }),
// sensitive items get special handling to prevent leaks
sensitive: true,
// load different values based on any other value
value: switchBy('DMNO_ENV', {
value: switchBy('APP_ENV', {
_default: 'my-dev-key',
production: onePassSecrets.item(),
}),
},
SAAS_PROJECT_TAG: {
LOGS_TAG: {
// use a function to set a value - reference any other config
value: () => `myapp_${DMNO_CONFIG.DMNO_ENV}`,
value: () => `myapp-frontend_${DMNO_CONFIG.APP_ENV}`,
},
PORT: {
extends: DmnoBaseTypes.port, // pre-made types built-in for many common cases
value: 4444,
},
// re-use config from other services (if in a monorepo)
API_URL: pick('api', 'PUBLIC_URL'),
OP_TOKEN: {
// re-use existing types with validation and docs info already built-in
extends: OnePasswordTypes.serviceAccountToken,
},
},
});
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ This automatic scaffolding of your config `schema` is meant to be a good startin
`dmno init` will also prompt you to delete the `.env` files that were previously checked into source control because the values in those files have now been incorporated into your new config schema.
:::

## Setting overrides via `.env` files
## Setting overrides via `.env` files ||overrides||

Whenever we are resolving your configuration values, dmno will load any `.env` files within a `.dmno` folder and apply those values as _overrides_. This means that they will take higher precendence than values set directly in your `schema`, and lower precedence than overrides set via actual environment variables passed in from your shell. The overall precedence order from highest to lowest is:
- Environment variables from your shell (e.g., `ENV_VAR=xyz npm run dev`)
Expand Down
20 changes: 11 additions & 9 deletions packages/docs-site/src/content/docs/docs/guides/monorepos.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ While DMNO works great in a traditional single-purpose repo, it was designed fro

## Monorepo structure

In a monorepo, rather than having only a single `.dmno` folder at the root of your project, each child project/service will have its own `.dmno` folder too. Here is an example:
Rather than having only a single `.dmno` folder at the root of your project, in a monorepo each child project/service will have its own `.dmno` folder too.

<FileTree>
- / (root of your project)
Expand All @@ -28,15 +28,17 @@ If you're starting fresh with DMNO, then the [`dmno init`](/docs/reference/cli/i

## Detecting child services

By default, DMNO will rely on your existing tooling to detect where potential child projects are found. This is usually an array of paths or glob patterns. We look in the following locations:
By default, DMNO will rely on your existing tooling to detect where potential child projects are found. This is usually an array of paths or glob patterns.

We look in the following locations:

| Workspace tool | Globs location |
|---|---|
| [npm](https://docs.npmjs.com/cli/v8/using-npm/workspaces), [yarn](https://yarnpkg.com/features/workspaces#how-are-workspaces-declared), [bun](https://bun.sh/docs/install/workspaces) | `package.json`<br/>└ `workspaces` |
| [pnpm](https://pnpm.io/workspaces) | `pnpm-workspace.yaml`<br/>└ `packages` |
| [moonrepo](https://moonrepo.dev/docs/config/workspace#projects) | `.moon/workspace.yml`<br/>└ `projects` |

In some situations, like a large polyglot repo, or a large repo that has multiple smaller monorepos within it, you may need an alternate way of defining where to look for DMNO services. In this case, you can create a `workspace.yaml` in your workspace root's `.dmno` folder. If this file is found, it will override everything else.
In some situations, like a polyglot repo, or a large repo that has multiple smaller monorepos within it, you may need an alternate way of defining where to look for DMNO services. In this case, you can create a `workspace.yaml` in your workspace root's `.dmno` folder. If this file is found, it will override everything else.

```yaml title=".dmno/workpace.yaml"
projects:
Expand All @@ -48,13 +50,13 @@ projects:
These glob patterns define where to look for _potential_ DMNO services, however only those services which have a `.dmno` folder will be considered DMNO services within your workspace. Shared libraries that have no need for config may not need a `.dmno` folder at all and will be ignored by DMNO.
:::

## Concurrent DMNO tasks
## Running concurrent DMNO tasks

When running tasks within a monorepo, you often want to orchestrate many tasks on many child projects at once, using something like [Turborepo](https://turbo.build/repo/docs). Since many of those tasks may rely on DMNO to load and resolve config, this could slow things down due to unnecessary repeated work.

To solve this, [`dmno run`](/docs/reference/cli/run/) boots up a server that other DMNO instances within child processes are able to communicate with, meaning we can load and resolve your config just once. To take advantage of this optimization, run your command via `dmno run` and you should be good to go - for example `dmno run -- turbo build`.

:::tip[Env var pass-through]
:::tip[Env var pass-through (Turborepo)]
We use an injected `DMNO_PARENT_SERVER` env var to detect the parent server, so it must be passed through to child processes.

In [Turborepo "strict mode"](https://turbo.build/repo/docs/crafting-your-repository/using-environment-variables#strict-mode), env vars are not all passed through by default. We must explicitly tell turbo about it using the [`globalPassThroughEnv`](https://turbo.build/repo/docs/reference/configuration#globalpassthroughenv) setting. For example:
Expand All @@ -67,12 +69,12 @@ In [Turborepo "strict mode"](https://turbo.build/repo/docs/crafting-your-reposit
```
:::

## Reusing config
## Sharing config across services

As outlined in our [schema guide](/docs/guides/schema/), services in monorepos are allowed to `pick` items from other services. While there are other mechanisms for reusing values from other services, `pick` allows you to reuse both the schema and value.
As outlined in our [schema guide](/docs/guides/schema/#pick), services in monorepos are allowed to `pick` items from other services. While there are other mechanisms for reusing values from other services, `pick` allows you to reuse the entire item - including all other properties (e.g., description, validation logic).

If you find your services have duplicate config items, or certain config could be derived from another with a function, consider moving that shared config up to a common parent, and then pick them in children. It's ok if not every child service uses that shared config. You can also pick _exposed_ items from other services, which lets you keep config owned by the most relevant service, while still reusing it elsewhere.
If you find your services have duplicate config items, or an item could be derived from one in another service, consider using `pick()` to keep things [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself).

:::caution[Pick cycles are not allowed]
Reusing config items from non-ancestors creates edges in the overall service dependency graph. Currently you are not allowed to create cycles in this graph, and if you do, DMNO will throw an error. To resolve this issue, consider moving this config up to a common ancestor rather than picking from each other.
DMNO will detect any cycles in the overall config graph, and throw errors when necessary.
:::
15 changes: 10 additions & 5 deletions packages/docs-site/src/content/docs/docs/guides/multi-env.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,17 @@ To use `switchBy`, we select another item by name to use as the switch condition
Note that each branch can be a static value, a function, or a resolver - just like setting the value itself. This leads to some powerful composition capabilities.

```typescript
import { NodeEnvType, switchBy } from 'dmno';
import { switchBy } from 'dmno';

export default defineDmnoService({
schema: {
NODE_ENV: NodeEnvType,
APP_ENV: {
description: 'our custom app environment flag',
extends: DmnoBaseTypes.enum(['development', 'staging', 'production', 'test']),
value: 'development',
},
TOGGLED_ITEM: {
value: switchBy('NODE_ENV', {
value: switchBy('APP_ENV', {
_default: 'default/development value',
staging: 'staging value',
test: 'test value',
Expand All @@ -33,7 +37,7 @@ export default defineDmnoService({
},
SOME_API_KEY: {
sensitive: true,
value: switchBy('NODE_ENV', {
value: switchBy('APP_ENV', {
_default: devSecretsVault.item(),
production: prodSecretsVault.item(),
},
Expand All @@ -56,6 +60,7 @@ export default defineDmnoService({
schema: {
...pickFromSchemaObject(VercelEnvSchema, 'VERCEL_ENV', 'VERCEL_GIT_COMMIT_REF'),
APP_ENV: {
extends: DmnoBaseTypes.enum(['development', 'branch-preview', 'pr-preview', 'staging', 'production', 'test']),
value: () => {
if (DMNO_CONFIG.VERCEL_ENV === 'production') return 'production';
if (DMNO_CONFIG.VERCEL_ENV === 'preview') {
Expand All @@ -79,6 +84,6 @@ APP_ENV=production pnpm exec dmno run -- your-build-command
## Overrides from `process.env`
Keep in mind that DMNO loads in process environment variables in as _overrides_, so there are many cases where your schema does not have a value defined for a certain environment flag, because you are expecting to receive it as an override. Sometimes this could be contextual information injected by the hosting platform, like `VERCEL_GIT_PULL_REQUEST_ID`, or it could be env vars that have been set via the platform's env var management UI. This will be especially true if you are migrating an existing project to DMNO.
Keep in mind that DMNO loads in process environment variables as _overrides_, so there are many cases where your schema does not have a value defined for a certain environment flag, because you are expecting to receive it as an override. Sometimes this could be contextual information injected by the hosting platform, like `VERCEL_GIT_PULL_REQUEST_ID`, or it could be env vars that have been set via the platform's env var management UI. This will be especially true if you are migrating an existing project to DMNO.
Over time, you may find it helpful to migrate more of your config into the schema and to a centralized secret store (like 1Password). This reduces secret sprawl, keeping things more secure and easier to reason about. In this case, you may still inject your _secret-zero_ as an override, but everything else will be defined via the schema.
7 changes: 2 additions & 5 deletions packages/docs-site/src/content/docs/docs/guides/schema.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Check out the [monorepo guide](/docs/guides/monorepos/) for more details on usin
Your `.dmno/config.mts` file will define settings for your project, including a full schema of all the config items used within your project. Your config file must include `export default defineDmnoService({ //...`. Here is an example:

```typescript title=".dmno/config.mts" "export default defineDmnoService"
import { DmnoBaseTypes, defineDmnoService, switchBy, NodeEnvType } from 'dmno';
import { DmnoBaseTypes, defineDmnoService, switchBy } from 'dmno';

export default defineDmnoService({
// project settings that affect DMNO itself
Expand Down Expand Up @@ -95,7 +95,7 @@ In a monorepo, aside from the root config file, each child service will have its
- we can now share config items across services (see [sharing config](/docs/guides/schema/#pick) for more info)

```typescript title="apps/docs-site/.dmno/config.mts" /(name):/ /(parent):/ /pick\\([^)]*\\)/
import { DmnoBaseTypes, defineDmnoService, switchBy, NodeEnvType } from 'dmno';
import { defineDmnoService, pick } from 'dmno';

export default defineDmnoService({
name: 'docs-site', // service name (optional - will default to `name` from `package.json`)
Expand Down Expand Up @@ -223,9 +223,6 @@ SOME_ITEM: { extends: pick(), description: 'can update/add more type settings' }
```
:::
If you just want to copy the _value_ once it resolved, but not the rest of the settings, there are 2 other mechanisms for doing so - the `configPath()` and `inject()` resolvers. More details coming soon.
### Validations & required config ||validation||
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -99,5 +99,5 @@ JavaScript is by default not strict - so while you will get nice autocompletion
To keep things simple, by default we always inject the global types for both `DMNO_CONFIG` and `DMNO_PUBLIC_CONFIG`. However, there are cases where you may want to only inject one or the other:

- In a front-end only (i.e., non-SSR) context, you could skip injecting `DMNO_CONFIG` because you'll only want to use non-sensitive config items. That said, injecting the _types_ doesn't actually inject your sensitive secrets, and we inject a placeholder proxy throws a helpful error if you try to use `DMNO_CONFIG`.
- In a back-end only context, you could skip injecting `DMNO_PUBLIC_CONFIG` and exclusively use `DMNO_CONFIG` - just like we do in your `.dmno/config.mts` file. However, there isn't really any harm in using the public version, and it can serve as an extra check that something is definitely not sensitive - which is useful if you were, for example, returning a config item in an API response.
- In a back-end only context, you could skip injecting `DMNO_PUBLIC_CONFIG` and exclusively use `DMNO_CONFIG` - just like we do in your `.dmno/config.mts` file. However, there isn't really any harm in using the public version, and it can serve as an extra reassurance that something is definitely not sensitive - which is useful if you were, for example, returning a config item in an API response.

Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ If you already use 1password and your secrets live in a vault that holds other i

</Steps>

This service account token will now serve as your _secret-zero_ - which grants access to the rest of your sensitive config stored in 1Password. It must be set locally (unless relying on cli-based auth) and in any deployed environments. It is sensitive so we must pass in the value as an _override_ rather than storing it within the config. See [Setting overrides](/docs/guides/env-files/#setting-overrides-via-env-files) for more details.
This service account token will now serve as your _secret-zero_ - which grants access to the rest of your sensitive config stored in 1Password. It must be set locally (unless relying on cli-based auth) and in any deployed environments. It is sensitive so we must pass in the value as an _override_ rather than storing it within the config. See [Setting overrides](/docs/guides/env-files/#overrides) for more details.

:::tip[Vault organization best practices]
Consider how you want to organize your vaults and service accounts, keeping in mind [best practices](https://support.1password.com/business-security-practices/#access-management-and-the-principle-of-least-privilege). At a minimum, DMNO recommends having a vault for highly sensitive production secrets and another for everything else.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ You should use multiple projects to segment your secrets following the [Principl
:::


Machine account access tokens now serve as your _secret-zero_ - which grants access to the rest of your sensitive config stored in Bitwarden. It must be set locally and in deployed environments, but it is sensitive so we must pass in the value as an _override_ rather than storing it within the config. Locally, this usually means storing it in your [`.env.local` file](/docs/guides/env-files/) and on a deployed environment you'll usually set it wherever you would normally pass in environment variables. DMNO will handle the rest. See [Setting overrides](/docs/guides/env-files/#setting-overrides-via-env-files) for more details.
Machine account access tokens now serve as your _secret-zero_ - which grants access to the rest of your sensitive config stored in Bitwarden. It must be set locally and in deployed environments, but it is sensitive so we must pass in the value as an _override_ rather than storing it within the config. Locally, this usually means storing it in your [`.env.local` file](/docs/guides/env-files/) and on a deployed environment you'll usually set it wherever you would normally pass in environment variables. DMNO will handle the rest. See [Setting overrides](/docs/guides/env-files/#overrides) for more details.


------
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ Next, you'll need to create a [Machine Identity](https://infisical.com/docs/docu

How you want to segment your identities and secrets is up to you. You could create a separate identity and secrets for each environment, or each service, or each project. At minimum, we recommend segmenting your production and non-production secrets. See [Secret Segmentation](/docs/guides/secret-segmentation/) for more details.

Also note that the **Client Secret** is highly sensitive and should be treated as your _secret zero_. It will need to be set locally and passed in as an override. Locally, it can be set in your `.env.local` file, and in any deployed environments it can be set however you normally set environment variables for that platform. DMNO will handle the rest. See [Setting overrides](/docs/guides/env-files/#setting-overrides-via-env-files) for more details.
Also note that the **Client Secret** is highly sensitive and should be treated as your _secret zero_. It will need to be set locally and passed in as an override. Locally, it can be set in your `.env.local` file, and in any deployed environments it can be set however you normally set environment variables for that platform. DMNO will handle the rest. See [Setting overrides](/docs/guides/env-files/#overrides) for more details.

------------

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ In general, plugins are initialized with a set of inputs that are used to config
Because plugins are typically used to access sensitive items themselves, this introduces a slight chicken-and-egg problem. In this case, DMNO allows you to define the plugin inputs in the `schema` but not provide a value. The sensitive value is then set via an _override_.

:::tip[More on overrides]
For more on how overrides work and how to use them, see the [Setting overrides via `.env` files](https://dmno.dev/docs/guides/env-files/#setting-overrides-via-env-files).
For more on how overrides work and how to use them, see the [Setting overrides via `.env` files](/docs/guides/env-files/#overrides).
:::

### Wiring up the inputs
Expand Down
Loading

0 comments on commit 5fff14b

Please sign in to comment.