Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: start in-memory secondary tenant dynamically [CRDB-14537 followup] #84604

Closed
Tracked by #91282
knz opened this issue Jul 18, 2022 · 0 comments · Fixed by #84608
Closed
Tracked by #91282

server: start in-memory secondary tenant dynamically [CRDB-14537 followup] #84604

knz opened this issue Jul 18, 2022 · 0 comments · Fixed by #84608
Assignees
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@knz
Copy link
Contributor

knz commented Jul 18, 2022

For in-memory servers we want a behavior that looks like the following:

  • by default, no SQL server is started at all
  • upon first use of a tenant ID, the SQL server is instantiated
  • when idle, the SQL server is shut down

This policy should be applied to all tenants equally including the system/admin tenant.

Additionally, the code licensing would restrict running more than 1 non-admin tenant.

Epic: CRDB-14537

Jira issue: CRDB-17760

@knz knz added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Jul 18, 2022
knz added a commit to knz/cockroach that referenced this issue Jul 19, 2022
Epic: CRDB-14537

This lays the groundwork for the ability to run tenant servers
in-memory.

The secondary in-memory server is started by default. This can
be manually disabled using `--disable-in-memory-tenant`.

For now, the secondary tenant uses separate network listeners for
SQL/HTTP/RPC. The port number is assigned randomly.  To derive a port
number "close" to the base TCP listeners, the operator can pass
e.g. `--secondary-tenant-port-offset=1` (for 1 past the base port
number). NB: This mechanism will likely be superseded when
cockroachdb#84604 is addressed.

Release note: None
knz added a commit to knz/cockroach that referenced this issue Nov 1, 2022
Epic: CRDB-14537

This lays the groundwork for the ability to run tenant servers
in-memory.

The secondary in-memory server is started by default. This can
be manually disabled using `--disable-in-memory-tenant`.

For now, the secondary tenant uses separate network listeners for
SQL/HTTP/RPC. The port number is assigned randomly.  To derive a port
number "close" to the base TCP listeners, the operator can pass
e.g. `--secondary-tenant-port-offset=1` (for 1 past the base port
number). NB: This mechanism will be superseded when
cockroachdb#84604 is addressed.

Release note: None
knz added a commit to knz/cockroach that referenced this issue Nov 4, 2022
Epic: CRDB-14537

This commit lays the groundwork for the ability to run tenant servers
in-memory. It introduces a new `serverController` with two roles:

- it maps tenant names to server instances.
- it only starts servers the first time the tenant name is referenced,
  and only if that tenant is marked active.

As of this commit, no subsystem inside CockroachDB refers to this
new server controller; this means there is no signal hooked up
to start these servers automatically yet.

For testing, a debug HTTP endpoint has been added:
`/debug/tickle?name=<tenantname>`

Example use:

1. start a server. At this point no secondary tenant server is created yet.
2. create a test tenant, e.g. via `select crdb_internal.create_tenant(123, 'hello');`.
   At this point, the secondary tenant server is still not running.
3. Perform a HTTP request to the debug endpoint, e.g. to `/debug/tickle?name=hello`
4. Observe (e.g. in logs): the secondary server has been started.

One can also observe that the controller also serves the name `system`
to refer to the system tenant.

For now, the secondary servers created this way use separate network
listeners for SQL/HTTP/RPC. NB: This mechanism will be superseded when
cockroachdb#84604 is addressed.

The port number is assigned randomly. To derive a predictable port
number for testing (until issue cockroachdb#84604 is addressed), the operator can
pass e.g. `--secondary-tenant-port-offset=100` to the start
command (for 100 past the base port number).

Release note: None
knz added a commit to knz/cockroach that referenced this issue Nov 7, 2022
Epic: CRDB-14537

This commit lays the groundwork for the ability to run tenant servers
in-memory. It introduces a new `serverController` with two roles:

- it maps tenant names to server instances.
- it only starts servers the first time the tenant name is referenced,
  and only if that tenant is marked active.

As of this commit, no subsystem inside CockroachDB refers to this
new server controller; this means there is no signal hooked up
to start these servers automatically yet.

For testing, a debug HTTP endpoint has been added:
`/debug/tickle?name=<tenantname>`

Example use:

1. start a server. At this point no secondary tenant server is created yet.
2. create a test tenant, e.g. via `select crdb_internal.create_tenant(123, 'hello');`.
   At this point, the secondary tenant server is still not running.
3. Perform a HTTP request to the debug endpoint, e.g. to `/debug/tickle?name=hello`
4. Observe (e.g. in logs): the secondary server has been started.

One can also observe that the controller also serves the name `system`
to refer to the system tenant.

For now, the secondary servers created this way use separate network
listeners for SQL/HTTP/RPC. NB: This mechanism will be superseded when
cockroachdb#84604 is addressed.

The port number is assigned randomly. To derive a predictable port
number for testing (until issue cockroachdb#84604 is addressed), the operator can
pass e.g. `--secondary-tenant-port-offset=100` to the start
command (for 100 past the base port number).

Release note: None
knz added a commit to knz/cockroach that referenced this issue Nov 14, 2022
Epic: CRDB-14537

This commit lays the groundwork for the ability to run tenant servers
in-memory. It introduces a new `serverController` with two roles:

- it maps tenant names to server instances.
- it only starts servers the first time the tenant name is referenced,
  and only if that tenant is marked active.

As of this commit, no subsystem inside CockroachDB refers to this
new server controller; this means there is no signal hooked up
to start these servers automatically yet.

For testing, a debug HTTP endpoint has been added:
`/debug/tickle?name=<tenantname>`

Example use:

1. start a server. At this point no secondary tenant server is created yet.
2. create a test tenant, e.g. via `select crdb_internal.create_tenant(123, 'hello');`.
   At this point, the secondary tenant server is still not running.
3. Perform a HTTP request to the debug endpoint, e.g. to `/debug/tickle?name=hello`
4. Observe (e.g. in logs): the secondary server has been started.

One can also observe that the controller also serves the name `system`
to refer to the system tenant.

For now, the secondary servers created this way use separate network
listeners for SQL/HTTP/RPC. NB: This mechanism will be superseded when
cockroachdb#84604 is addressed.

The port number is assigned randomly. To derive a predictable port
number for testing (until issue cockroachdb#84604 is addressed), the operator can
pass e.g. `--secondary-tenant-port-offset=100` to the start
command (for 100 past the base port number).

Release note: None
knz added a commit to knz/cockroach that referenced this issue Nov 15, 2022
Epic: CRDB-14537

This commit lays the groundwork for the ability to run tenant servers
in-memory. It introduces a new `serverController` with two roles:

- it maps tenant names to server instances.
- it only starts servers the first time the tenant name is referenced,
  and only if that tenant is marked active.

As of this commit, no subsystem inside CockroachDB refers to this
new server controller; this means there is no signal hooked up
to start these servers automatically yet.

For testing, a debug HTTP endpoint has been added:
`/debug/tickle?name=<tenantname>`

Example use:

1. start a server. At this point no secondary tenant server is created yet.
2. create a test tenant, e.g. via `select crdb_internal.create_tenant(123, 'hello');`.
   At this point, the secondary tenant server is still not running.
3. Perform a HTTP request to the debug endpoint, e.g. to `/debug/tickle?name=hello`
4. Observe (e.g. in logs): the secondary server has been started.

One can also observe that the controller also serves the name `system`
to refer to the system tenant.

For now, the secondary servers created this way use separate network
listeners for SQL/HTTP/RPC. NB: This mechanism will be superseded when
cockroachdb#84604 is addressed.

The port number is assigned randomly. To derive a predictable port
number for testing (until issue cockroachdb#84604 is addressed), the operator can
pass e.g. `--secondary-tenant-port-offset=100` to the start
command (for 100 past the base port number).

Release note: None
knz added a commit to knz/cockroach that referenced this issue Nov 15, 2022
Epic: CRDB-14537

This commit lays the groundwork for the ability to run tenant servers
in-memory. It introduces a new `serverController` with two roles:

- it maps tenant names to server instances.
- it only starts servers the first time the tenant name is referenced,
  and only if that tenant is marked active.

As of this commit, no subsystem inside CockroachDB refers to this
new server controller; this means there is no signal hooked up
to start these servers automatically yet.

For testing, a debug HTTP endpoint has been added:
`/debug/tickle?name=<tenantname>`

Example use:

1. start a server. At this point no secondary tenant server is created yet.
2. create a test tenant, e.g. via `select crdb_internal.create_tenant(123, 'hello');`.
   At this point, the secondary tenant server is still not running.
3. Perform a HTTP request to the debug endpoint, e.g. to `/debug/tickle?name=hello`
4. Observe (e.g. in logs): the secondary server has been started.

One can also observe that the controller also serves the name `system`
to refer to the system tenant.

For now, the secondary servers created this way use separate network
listeners for SQL/HTTP/RPC. NB: This mechanism will be superseded when
cockroachdb#84604 is addressed.

The port number is assigned randomly. To derive a predictable port
number for testing (until issue cockroachdb#84604 is addressed), the operator can
pass e.g. `--secondary-tenant-port-offset=100` to the start
command (for 100 past the base port number).

Release note: None
knz added a commit to knz/cockroach that referenced this issue Nov 15, 2022
Epic: CRDB-14537

This commit lays the groundwork for the ability to run tenant servers
in-memory. It introduces a new `serverController` with two roles:

- it maps tenant names to server instances.
- it only starts servers the first time the tenant name is referenced,
  and only if that tenant is marked active.

As of this commit, no subsystem inside CockroachDB refers to this
new server controller; this means there is no signal hooked up
to start these servers automatically yet.

For testing, a debug HTTP endpoint has been added:
`/debug/tickle?name=<tenantname>`

Example use:

1. start a server. At this point no secondary tenant server is created yet.
2. create a test tenant, e.g. via `select crdb_internal.create_tenant(123, 'hello');`.
   At this point, the secondary tenant server is still not running.
3. Perform a HTTP request to the debug endpoint, e.g. to `/debug/tickle?name=hello`
4. Observe (e.g. in logs): the secondary server has been started.

One can also observe that the controller also serves the name `system`
to refer to the system tenant.

For now, the secondary servers created this way use separate network
listeners for SQL/HTTP/RPC. NB: This mechanism will be superseded when
cockroachdb#84604 is addressed.

The port number is assigned randomly. To derive a predictable port
number for testing (until issue cockroachdb#84604 is addressed), the operator can
pass e.g. `--secondary-tenant-port-offset=100` to the start
command (for 100 past the base port number).

Release note: None
knz added a commit to knz/cockroach that referenced this issue Nov 22, 2022
Epic: CRDB-14537

This commit lays the groundwork for the ability to run tenant servers
in-memory. It introduces a new `serverController` with two roles:

- it maps tenant names to server instances.
- it only starts servers the first time the tenant name is referenced,
  and only if that tenant is marked active.

As of this commit, no subsystem inside CockroachDB refers to this
new server controller; this means there is no signal hooked up
to start these servers automatically yet.

For testing, a debug HTTP endpoint has been added:
`/debug/tickle?name=<tenantname>`

Example use:

1. start a server. At this point no secondary tenant server is created yet.
2. create a test tenant, e.g. via `select crdb_internal.create_tenant(123, 'hello');`.
   At this point, the secondary tenant server is still not running.
3. Perform a HTTP request to the debug endpoint, e.g. to `/debug/tickle?name=hello`
4. Observe (e.g. in logs): the secondary server has been started.

One can also observe that the controller also serves the name `system`
to refer to the system tenant.

For now, the secondary servers created this way use separate network
listeners for SQL/HTTP/RPC. NB: This mechanism will be superseded when
cockroachdb#84604 is addressed.

The port number is assigned randomly. To derive a predictable port
number for testing (until issue cockroachdb#84604 is addressed), the operator can
pass e.g. `--secondary-tenant-port-offset=100` to the start
command (for 100 past the base port number).

Release note: None
knz added a commit to knz/cockroach that referenced this issue Nov 23, 2022
Epic: CRDB-14537

This commit lays the groundwork for the ability to run tenant servers
in-memory. It introduces a new `serverController` with two roles:

- it maps tenant names to server instances.
- it only starts servers the first time the tenant name is referenced,
  and only if that tenant is marked active.

As of this commit, no subsystem inside CockroachDB refers to this
new server controller; this means there is no signal hooked up
to start these servers automatically yet.

For testing, a debug HTTP endpoint has been added:
`/debug/tickle?name=<tenantname>`

Example use:

1. start a server. At this point no secondary tenant server is created yet.
2. create a test tenant, e.g. via `select crdb_internal.create_tenant(123, 'hello');`.
   At this point, the secondary tenant server is still not running.
3. Perform a HTTP request to the debug endpoint, e.g. to `/debug/tickle?name=hello`
4. Observe (e.g. in logs): the secondary server has been started.

One can also observe that the controller also serves the name `system`
to refer to the system tenant.

For now, the secondary servers created this way use separate network
listeners for SQL/HTTP/RPC. NB: This mechanism will be superseded when
cockroachdb#84604 is addressed.

The port number is assigned randomly. To derive a predictable port
number for testing (until issue cockroachdb#84604 is addressed), the operator can
pass e.g. `--secondary-tenant-port-offset=100` to the start
command (for 100 past the base port number).

Release note: None
knz added a commit to knz/cockroach that referenced this issue Nov 28, 2022
Epic: CRDB-14537

This commit lays the groundwork for the ability to run tenant servers
in-memory. It introduces a new `serverController` with two roles:

- it maps tenant names to server instances.
- it only starts servers the first time the tenant name is referenced,
  and only if that tenant is marked active.

As of this commit, no subsystem inside CockroachDB refers to this
new server controller; this means there is no signal hooked up
to start these servers automatically yet.

For testing, a debug HTTP endpoint has been added:
`/debug/tickle?name=<tenantname>`

Example use:

1. start a server. At this point no secondary tenant server is created yet.
2. create a test tenant, e.g. via `select crdb_internal.create_tenant(123, 'hello');`.
   At this point, the secondary tenant server is still not running.
3. Perform a HTTP request to the debug endpoint, e.g. to `/debug/tickle?name=hello`
4. Observe (e.g. in logs): the secondary server has been started.

One can also observe that the controller also serves the name `system`
to refer to the system tenant.

For now, the secondary servers created this way use separate network
listeners for SQL/HTTP/RPC. NB: This mechanism will be superseded when
cockroachdb#84604 is addressed.

The port number is assigned randomly. To derive a predictable port
number for testing (until issue cockroachdb#84604 is addressed), the operator can
pass e.g. `--secondary-tenant-port-offset=100` to the start
command (for 100 past the base port number).

Release note: None
@craig craig bot closed this as completed in #84608 Nov 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant