Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

server: use a shared HTTP listener for all in-memory tenant servers #91744

Merged
merged 4 commits into from
Nov 28, 2022
Merged

server: use a shared HTTP listener for all in-memory tenant servers #91744

merged 4 commits into from
Nov 28, 2022

Commits on Nov 28, 2022

  1. server: enabling instantiating secondary tenant servers in-memory 

    Epic: CRDB-14537

This commit lays the groundwork for the ability to run tenant servers
in-memory. It introduces a new `serverController` with two roles:

- it maps tenant names to server instances.
- it only starts servers the first time the tenant name is referenced,
  and only if that tenant is marked active.

As of this commit, no subsystem inside CockroachDB refers to this
new server controller; this means there is no signal hooked up
to start these servers automatically yet.

For testing, a debug HTTP endpoint has been added:
`/debug/tickle?name=<tenantname>`

Example use:

1. start a server. At this point no secondary tenant server is created yet.
2. create a test tenant, e.g. via `select crdb_internal.create_tenant(123, 'hello');`.
   At this point, the secondary tenant server is still not running.
3. Perform a HTTP request to the debug endpoint, e.g. to `/debug/tickle?name=hello`
4. Observe (e.g. in logs): the secondary server has been started.

One can also observe that the controller also serves the name `system`
to refer to the system tenant.

For now, the secondary servers created this way use separate network
listeners for SQL/HTTP/RPC. NB: This mechanism will be superseded when
cockroachdb#84604 is addressed.

The port number is assigned randomly. To derive a predictable port
number for testing (until issue cockroachdb#84604 is addressed), the operator can
pass e.g. `--secondary-tenant-port-offset=100` to the start
command (for 100 past the base port number).

Release note: None
    @knz
    knz committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    b8dad07 View commit details
    Browse the repository at this point in the history

  2. cli,server: make PreStart responsible for starting diagnostics 

    Release note: None
    @knz
    knz committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    65c7eb0 View commit details
    Browse the repository at this point in the history

  3. server,netutil: clean up the connManager logic 

    Prior to this patch, we were using the same netutil.Server object to
manage both `net.Conn` created to serve HTTP connections, and
`net.Conn` created to serve SQL connections.

This was confusing, because a lot of the complexity specific for HTTP
connections (integration with the HTTP2 query handling, etc) is not
required for raw TCP connections as used by pgwire.

This commit clarifies this by separating the two roles.

This also cleans up the server initialization.

Release note: None
    @knz
    knz committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    cf1690d View commit details
    Browse the repository at this point in the history

  4. server: use a shared HTTP listener for all in-memory tenant servers 

    This commit introduces a HTTP (de)multiplexer for all in-memory tenant
servers.

By default, HTTP requests are routed to the system tenant server.
This can be overridden:
- manually in the query URL parameters with `tenant_name=...`
- explicitly, with the header `X-Cockroach-Tenant` (preferred for CSRF).
- with a `tenant` cookie.

Release note: None
    @knz
    knz committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    4dd3f06 View commit details
    Browse the repository at this point in the history