Backport of Multiple instances of a periodic job are run simultaneously, when prohibit_overlap is true into release/1.3.x #16659

* Change api Fields for expose and paths * Add changelog entry * changelog: add deprecation notes about connect fields * api: minor style tweaks --------- Co-authored-by: Seth Hoenig <shoenig@duck.com>

* Ensure infra_image gets proper label used for reconciliation Currently infra containers are not cleaned up as part of the dangling container cleanup routine. The reason is that Nomad checks if a container is a Nomad owned container by verifying the existence of the: `com.hashicorp.nomad.alloc_id` label. Ensure we set this label on the infra container as well. * fix unit test * changelog: add entry --------- Co-authored-by: Seth Hoenig <shoenig@duck.com>

While you can use any string value for a variable Item's key name using characters that are outside of the set [unicode.Letter, unicode.Number,`_`] will require the `index` function for direct access.

… multiple tags (#15962) * docker: set force=true on remove image to handle images referenced by multiple tags This PR changes our call of docker client RemoveImage() to RemoveImageExtended with the Force=true option set. This fixes a bug where an image referenced by more than one tag could never be garbage collected by Nomad. The Force option only applies to stopped containers; it does not affect running workloads. * docker: add note about image_delay and multiple tags

Prior to 2409f72 the code compared the modification index of a job to itself. Afterwards, the code compared the creation index of the job to itself. In either case there should never be a case of re-parenting of allocs causing the evaluation to trivially always result in false, which leads to unreclaimable memory. Prior to this change allocations and evaluations for batch jobs were never garbage collected until the batch job was explicitly stopped. The new `batch_eval_gc_threshold` server configuration controls how often they are collected. The default threshold is `24h`.

…a documentation (#15963)

* refact: add conditional error handling * test: test conditional logic

This pile was deprecated when we starting using HCP Consul for e2e instead of standing up our own cluster and managing Consuls at test runtime.

The ACL token decoding was not correctly handling time duration syntax such as "1h" which forced people to use the nanosecond representation via the HTTP API. The change adds an unmarshal function which allows this syntax to be used, along with other styles correctly.

* consul: reset consul token on job during registration of a reversion * e2e: add test for reverting a job with a consul service * cl: fixup cl entry

…16018)

Also allows for default value of `datacenters = ["*"]`

* Add `bridge_network_hairpin_mode` client config setting * Add node attribute: `nomad.bridge.hairpin_mode` * Changed format string to use `%q` to escape user provided data * Add test to validate template JSON for developer safety Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>

* Extend variables under the nomad path prefix to allow for job-templates (#15570) * Extend variables under the nomad path prefix to allow for job-templates * Add job-templates to error message hinting * RadioCard component for Job Templates (#15582) * chore: add * test: component API * ui: component template * refact: remove bc naming collission * styles: remove SASS var causing conflicts * Disallow specific variable at nomad/job-templates (#15681) * Disallows variables at exactly nomad/job-templates * idiomatic refactor * Expanding nomad job init to accept a template flag (#15571) * Adding a string flag for templates on job init * data-down actions-up version of a custom template editor within variable * Dont force grid on job template editor * list-templates flag started * Correctly slice from end of path name * Pre-review cleanup * Variable form acceptance test for job template editing * Some review cleanup * List Job templates test * Example from template test * Using must.assertions instead of require etc * ui: add choose template button (#15596) * ui: add new routes * chore: update file directory * ui: add choose template button * test: button and page navigation * refact: update var name * ui: use `Button` component from `HDS` (#15607) * ui: integrate buttons * refact: remove helper * ui: remove icons on non-tertiary buttons * refact: update normalize method for key/value pairs (#15612) * `revert`: `onCancel` for `JobDefinition` The `onCancel` method isn't included in the component API for `JobEditor` and the primary cancel behavior exists outside of the component. With the exception of the `JobDefinition` page where we include this button in the top right of the component instead of next to the `Plan` button. * style: increase button size * style: keep lime green * ui: select template (#15613) * ui: deprecate unused component * ui: deprecate tests * ui: jobs.run.templates.index * ui: update logic to handle templates * refact: revert key/value changes * style: padding for cards + buttons * temp: fixtures for mirage testing * Revert "refact: revert key/value changes" This reverts commit 124e95d. * ui: guard template for unsaved job * ui: handle reading template variable * Revert "refact: update normalize method for key/value pairs (#15612)" This reverts commit 6f5ffc9. * revert: remove test fixtures * revert: prettier problems * refact: test doesnt need filter expression * styling: button sizes and responsive cards * refact: remove route guarding * ui: update variable adapter * refact: remove model editing behavior * refact: model should query variables to populate editor * ui: clear qp on exit * refact: cleanup deprecated API * refact: query all namespaces * refact: deprecate action * ui: rely on collection * refact: patch deprecate transition API * refact: patch test to expect namespace qp * styling: padding, conditionals * ui: flashMessage on 404 * test: update for o(n+1) query * ui: create new job template (#15744) * refact: remove unused code * refact: add type safety * test: select template flow * test: add data-test attrs * chore: remove dead code * test: create new job flow * ui: add create button * ui: create job template * refact: no need for wildcard * refact: record instead of delete * styling: spacing * ui: add error handling and form validation to job create template (#15767) * ui: handle server side errors * ui: show error to prevent duplicate * refact: conditional namespace * ui: save as template flow (#15787) * bug: patches failing tests associated with `pretender` (#15812) * refact: update assertion * refact: test set-up * ui: job templates manager view (#15815) * ui: manager list view * test: edit flow * refact: deprecate column-helper * ui: template edit and delete flow (#15823) * ui: manager list view * refact: update title * refact: update permissions * ui: template edit page * bug: typo * refact: update toast messages * bug: clear selections on exit (#15827) * bug: clear controllers on exit * test: mirage config changes (#15828) * refact: deprecate column-helper * style: update z-index for HDS * Revert "style: update z-index for HDS" This reverts commit d3d87ce. * refact: update delete button * refact: edit redirect * refact: patch reactivity issues * styling: fixed width * refact: override defaults * styling: edit text causing overflow * styling: add inline text Co-authored-by: Phil Renaud <phil.renaud@hashicorp.com> * bug: edit `text` to `template` Co-authored-by: Phil Renaud <phil.renaud@hashicorp.com> Co-authored-by: Phil Renaud <phil.renaud@hashicorp.com> * test: delete flow job templates (#15896) * refact: edit names * bug: set correct ref to store * chore: trim whitespace: * test: delete flow * bug: reactively update view (#15904) * Initialized default jobs (#15856) * Initialized default jobs * More jobs scaffolded * Better commenting on a couple example job specs * Adapter doing the work * fall back to epic config * Label format helper and custom serialization logic * Test updates to account for a never-empty state * Test suite uses settled and maintain RecordArray in adapter return * Updates to hello-world and variables example jobspecs * Parameterized job gets optional payload output * Formatting changes for param and service discovery job templates * Multi-group service discovery job * Basic test for default templates (#15965) * Basic test for default templates * Percy snapshot for manage page * Some late-breaking design changes * Some copy edits to the header paragraphs for job templates (#15967) * Added some init options for job templates (#15994) * Async method for populating default job templates from the variable adapter --------- Co-authored-by: Jai <41024828+ChaiWithJai@users.noreply.github.com>

This PR updates golangci-lint to work better with go1.20 - the previous version would cause in oom on 'make check'.

If a deployment fails, the `deployment status` command can get a nil deployment when it checks for a rollback deployment if there isn't one (or at least not one at the time of the query). Fix the panic.

#15997) recommend .nomad.hcl for job files instead of .nomad (without .hcl) * nomad job init -> example.nomad.hcl * update docs

Add `identity` jobspec block to expose workload identity tokens to tasks. --------- Co-authored-by: Anders <mail@anars.dk> Co-authored-by: Tim Gross <tgross@hashicorp.com> Co-authored-by: Michael Schurter <mschurter@hashicorp.com>

* Removed line from previous implementation * remove import Co-authored-by: Luiz Aoqui <luiz@hashicorp.com>

* largely a doc-ification of this commit message: d476780 this doesn't spell out all the possible failure modes, but should be a good starting point for folks. * connect: add doc link to envoy bootstrap error * add Unwrap() to RecoverableError mainly for easier testing

Service jobs should have unique allocation Names, derived from the Job.ID. System jobs do not have unique allocation Names because the index is intended to indicated the instance out of a desired count size. Because system jobs do not have an explicit count but the results are based on the targeted nodes, the index is less informative and this was intentionally omitted from the original design. Update docs to make it clear that NOMAD_ALLOC_INDEX is always zero for system/sysbatch jobs Validate that `volume.per_alloc` is incompatible with system/sysbatch jobs. System and sysbatch jobs always have a `NOMAD_ALLOC_INDEX` of 0. So interpolation via `per_alloc` will not work as soon as there's more than one allocation placed. Validate against this on job submission.

* Demoable state * Demo mirage color * Label as a block with foreground and background colours * Test mock updates * Go test updated * Documentation update for label support

* build: update to go1.20 * build: use stringy go1.20 in circle yaml * tests: handle new x509 certificate error structure in go1.20 * cl: add cl entry

…6051) Many of the functions in the `utils.go` file are specific to a particular scheduler, and very few of them have guards (or even names) that help avoid misuse with features specific to a given scheduler type. Move these functions (and their tests) into files specific to their scheduler type without any functionality changes to make it clear which bits go with what.

Update golangci-lint again, and also hc-install which now has a real version.

This change introduces the Task API: a portable way for tasks to access Nomad's HTTP API. This particular implementation uses a Unix Domain Socket and, unlike the agent's HTTP API, always requires authentication even if ACLs are disabled. This PR contains the core feature and tests but followup work is required for the following TODO items: - Docs - might do in a followup since dynamic node metadata / task api / workload id all need to interlink - Unit tests for auth middleware - Caching for auth middleware - Rate limiting on negative lookups for auth middleware --------- Co-authored-by: Seth Hoenig <shoenig@duck.com>

…16063) * docker: disable driver when running as non-root on cgroups v2 hosts This PR modifies the docker driver to behave like exec when being run as a non-root user on a host machine with cgroups v2 enabled. Because of how cpu resources are managed by the Nomad client, the nomad agent must be run as root to manage docker-created cgroups. * cl: update cl

Signed-off-by: dttung2905 <ttdao.2015@accountancy.smu.edu.sg>

Nothing more motivating than lots of deprecation warnings to get some code refactored.

These command were marked as deprecated in 1.4.0 with intent to remove in 1.5.0. Remove them and clean up the docs.

* main: remove deprecated uses of rand.Seed go1.20 deprecates rand.Seed, and seeds the rand package automatically. Remove cases where we seed the random package, and cleanup the one case where we intentionally create a known random source. * cl: update cl * mod: update go mod

…nspect` command (#15984) Signed-off-by: dttung2905 <ttdao.2015@accountancy.smu.edu.sg>

Adds the sprig functions to the template funcmap prepended with `sprig_` to match the behavior in consul-template

In #13374 we updated the commented-out `license_path` in the packaged example configuration file to match the existing documentation. Although this config value was commented-out, it was reported that changing the value was confusing. Update the commented-out line to the previous value and update the documented examples to match that. This matches most of the examples for Consul/Vault licensing as well. I've double-checked the tutorials and it looks like it'd been left on the previous value there, so no additional work to be done.

Fixes #14617 Dynamic Node Metadata allows Nomad users, and their jobs, to update Node metadata through an API. Currently Node metadata is only reloaded when a Client agent is restarted. Includes new UI for editing metadata as well. --------- Co-authored-by: Phil Renaud <phil.renaud@hashicorp.com>

Release/1.5.0 beta.1

This PR deletes the user.LookupGroupId function as it was only being used in a single test case, and its value was not important to the test.

) When an auth method was not supplied and the OIDC type was given in lower case, the CLI was not matching the default method due to casing and responded with a confusing user message. This change fixes the above problem, along with making use of the santized type easier.

* users: create cache for user lookups This PR introduces a global cache for OS user lookups. This should relieve pressure on the OS domain/directory lookups, which would be queried more now that Task API exists. Hits are cached for 1 hour, and misses are cached for 1 minute. These values are fairly arbitrary - we can tweak them if there is any reason to. Closes #16010 * users: delete expired negative entry from cache

In #15901 we introduced pre-forwarding authentication for RPCs so that we can grab the identity for rate metrics. The `ACL.Bootstrap` RPC is an unauthenticated endpoint, so any error message from authentication is not particularly useful. This would be harmless, but if you try to bootstrap with your `NOMAD_TOKEN` already set (perhaps because you were talking to another cluster previously from the same shell session), you'll get an authentication error instead of just having the token be ignored. This is a regression from the existing behavior, so have this endpoint ignore auth errors the same way we do for every other unauthenticated endpoint (ex `Status.Peers`)

Sometimes GHA gets wedged, and there is no need to waste 30 minutes on a job that usually takes < 3. e.g. stuck loading module cache, which is out of our control https://github.com/hashicorp/nomad/actions/runs/4135098558/jobs/7147128419

…ged file (#16114)

* docs: add dynamic node metadata api docs Also update all paths in the client API docs to explicitly state the `/v1/` prefix. We're inconsistent about that, but I think it's better to display the full path than to only show the fragment. If we ever do a `/v2/` whether or not we explicitly state `/v1/` in our docs won't be our greatest concern. * docs: add task-api docs

Some of the core scheduler tests need the maximum batch size for writes to be smaller than the usual `structs.MaxUUIDsPerWriteRequest`. But they do so by unsafely modifying the global struct, which creates test flakes in other tests. Modify the functions under test to take a batch size parameter. Production code will pass the global while the tests can inject smaller values. Turn the `structs.MaxUUIDsPerWriteRequest` into a constant, and add a semgrep rule for avoiding this kind of thing in the future.

The eval broker's `Cancelable` method used by the cancelable eval reaper mutates the slice of cancelable evals by removing a batch at a time from the slice. But this method unsafely uses a read lock despite this mutation. Under normal workloads this is likely to be safe but when the eval broker is under the heavy load this feature is intended to fix, we're likely to have a race condition. Switch this to a write lock, like the other locks that mutate the eval broker state. This changeset also adjusts the timeout to allow poorly-sized Actions runners more time to schedule the appropriate goroutines. The test has also been updated to use `shoenig/test/wait` so we can have sensible reporting of the results rather than just a timeout error when things go wrong.

* Convert assets from bindatafs to go embeds * Add command/asset to "uninteresting" list for missing test check * Remove generate-examples target * Update paths in tests

The plugin loader loads task and device driver plugins which are not used on server nodes.

* Add information about templating using `env` function to refer to environment variables.

New IDE from jetbrains gets its own config directory.

Made a breaking change in go-set (String() signature), need to update both these dependencies together and also fix a thing in structs.go

Bumps [json5](https://github.com/json5/json5) from 1.0.1 to 1.0.2. - [Release notes](https://github.com/json5/json5/releases) - [Changelog](https://github.com/json5/json5/blob/main/CHANGELOG.md) - [Commits](json5/json5@v1.0.1...v1.0.2) --- updated-dependencies: - dependency-name: json5 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

#15470) Bumps [decode-uri-component](https://github.com/SamVerschueren/decode-uri-component) from 0.2.0 to 0.2.2. - [Release notes](https://github.com/SamVerschueren/decode-uri-component/releases) - [Commits](SamVerschueren/decode-uri-component@v0.2.0...v0.2.2) --- updated-dependencies: - dependency-name: decode-uri-component dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [github.com/docker/cli](https://github.com/docker/cli) from 20.10.23+incompatible to 23.0.1+incompatible. - [Release notes](https://github.com/docker/cli/releases) - [Commits](docker/cli@v20.10.23...v23.0.1) --- updated-dependencies: - dependency-name: github.com/docker/cli dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.52.0 to 1.53.0. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.52.0...v1.53.0) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [github.com/containernetworking/plugins](https://github.com/containernetworking/plugins) from 1.1.1 to 1.2.0. - [Release notes](https://github.com/containernetworking/plugins/releases) - [Commits](containernetworking/plugins@v1.1.1...v1.2.0) --- updated-dependencies: - dependency-name: github.com/containernetworking/plugins dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

#16059) Bumps [github.com/shirou/gopsutil/v3](https://github.com/shirou/gopsutil) from 3.22.12 to 3.23.1. - [Release notes](https://github.com/shirou/gopsutil/releases) - [Commits](shirou/gopsutil@v3.22.12...v3.23.1) --- updated-dependencies: - dependency-name: github.com/shirou/gopsutil/v3 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…func `nodeCSIVolumeNames` (#16138) * Fix unbold header and remove unused var in func Signed-off-by: dttung2905 <ttdao.2015@accountancy.smu.edu.sg> * Add CHANGELOG file Signed-off-by: dttung2905 <ttdao.2015@accountancy.smu.edu.sg> * Apply suggestions from review <Charlie Voiselle> --------- Signed-off-by: dttung2905 <ttdao.2015@accountancy.smu.edu.sg> Co-authored-by: Charlie Voiselle <464492+angrycub@users.noreply.github.com> Co-authored-by: Tim Gross <tgross@hashicorp.com>

This PR fixes the CNI plugin fingerprinter to take into account the fact that the cni_path config can be a multi-path (e.g. `/foo:/bar:/baz`). Accumulate plugins from each of the possible path elements. If scanning any of the named directory fails, the fingerprinter fails. Fixes #16083 No CL/BP - has not shipped yet.

* Warn when Items key isn't directly accessible Go template requires that map keys are alphanumeric for direct access using the dotted reference syntax. This warns users when they create keys that run afoul of this requirement. - cli: use regex to detect invalid indentifiers in var keys - test: fix slash in escape test case - api: share warning formatting function between API and CLI - ui: warn if var key has characters other than _, letter, or number --------- Co-authored-by: Charlie Voiselle <464492+angrycub@users.noreply.github.com> Co-authored-by: Luiz Aoqui <luiz@hashicorp.com>

…16151) * artifact: protect against unbounded artifact decompression Starting with 1.5.0, set defaut values for artifact decompression limits. artifact.decompression_size_limit (default "100GB") - the maximum amount of data that will be decompressed before triggering an error and cancelling the operation artifact.decompression_file_count_limit (default 4096) - the maximum number of files that will be decompressed before triggering an error and cancelling the operation. * artifact: assert limits cannot be nil in validation

The panic bug for upgrades with older servers that shipped in 1.4.0 was fixed in 1.4.1, which makes the versions described in the warning in the upgrade guide misleading. Clarify the upgrade guide.

Fixes #15493 Co-authored-by: Tim Gross <tgross@hashicorp.com>

* docs: remove cores/memory beta label, update driver cpu docs * docs: fixup cr stuff

This PR wraps the cgroups.IsCgroup2UnifiedMode() helper method from runc in a defer/recover block because it might panic in some cases. Upstream fix in: opencontainers/runc#3745 Closes #16179

The `nomad fmt -check` command incorrectly writes to file because we didn't return before writing the file on a diff. Fix this bug and update the command internals to differentiate between the write-to-file and write-to-stdout code paths, which are activated by different combinations of options and flags. The docstring for the `-list` and `-write` flags is also unclear and can be easily misread to be the opposite of the actual behavior. Clarify this and fix up the docs to match. This changeset also refactors the tests quite a bit so as to make the test outputs clear when something is incorrect.

….18 (#16198) Bumps [github.com/containerd/containerd](https://github.com/containerd/containerd) from 1.6.12 to 1.6.18. - [Release notes](https://github.com/containerd/containerd/releases) - [Changelog](https://github.com/containerd/containerd/blob/main/RELEASES.md) - [Commits](containerd/containerd@v1.6.12...v1.6.18) --- updated-dependencies: - dependency-name: github.com/containerd/containerd dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

The RPC TLS enforcement test was frequently failing with broken connections. The most likely cause was that the tests started to run before the server had started its RPC server. Wait until it self-elects to ensure that the RPC server is up. This seems to have corrected the error; I ran this 3 times without a failure (even accounting for `gotestsum` retries). Also, fix a minor test bug that didn't impact the test but showed an incorrect usage for `Status.Ping.`

* api: return error on parse failure * docs: clarify anonymous policy with task api

…16181)

* acl: add validation to binding rule selector on upsert. * docs: add more information on binding rule selector escaping.

Also remove old version indicators. --------- Co-authored-by: Tim Gross <tgross@hashicorp.com>

Add an Elastic Network Interface (ENI) to each Linux host, on a secondary subnet we have provisioned in each AZ. Revise security groups as follows: * Split out client security groups from servers so that we can't have clients accidentally accessing serf addresses or other unexpected cross-talk. * Add new security groups for the secondary subnet that only allows communication within the security group so we can exercise behaviors with multiple IPs. This changeset doesn't include any Nomad configuration changes needed to take advantage of the extra network interface. I'll include those with testing for PR #16217.

* taskapi: return Forbidden on bad credentials Prior to this change a "Server error" would be returned when ACLs are enabled which did not match when ACLs are disabled. * e2e: love love love datacenter wildcard default * e2e: skip windows nodes on linux only test The Logfs are a bit weird because they're most useful when converted to Printfs to make debugging the test much faster, but that makes CI noisy. In a perfect world Go would expose how many tests are being run and we could stream output live if there's only 1. For now I left these helpful lines in as basically glorified comments.

This PR - fixes a panic in GetItems when looking up a variable that does not exist. - deprecates GetItems in favor of GetVariableItems which avoids returning a pointer to a map - deprecates ErrVariableNotFound in favor of ErrVariablePathNotFound which is an actual error type - does some minor code cleanup to make linters happier

This PR fixes a bug where the task group information was not being set on the serviceHook.AllocInfo struct, which is needed later on for calculating the CheckID of a nomad service check. The CheckID is calculated independently from multiple callsites, and the information being passed in must be consistent, including the group name. The workload.AllocInfo.Group was not set at this callsite, due to the bug fixed in this PR. https://github.com/hashicorp/nomad/blob/main/client/serviceregistration/nsd/nsd.go#L114

Signed-off-by: dttung2905 <ttdao.2015@accountancy.smu.edu.sg>

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to 0.7.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](golang/net@v0.5.0...v0.7.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…services (#16248)

* agent: only reload HTTP servers that use TLS * shutdown task api before client and improve names Fixes #16239

The `CSIVolume` struct has references to allocations that are "denormalized"; we don't store them on the `CSIVolume` struct but hydrate them on read. Tests detecting potential state store corruptions found two locations where we're not copying the volume before denormalizing: * When garbage collecting CSI volume claims. * When checking if it's safe to force-deregister the volume. There are no known user-visible problems associated with these bugs but both have the potential of mutating volume claims outside of a FSM transaction. This changeset also cleans up state mutations in some CSI tests so as to avoid having working tests cover up potential future bugs.

The `TaskUpdateRequest` struct we send to task runner update hooks was not populating the Nomad token that we get from the task runner (which we do for the Vault token). This results in task runner hooks like the template hook overwriting the Nomad token with the zero value for the token. This causes in-place updates of a task to break templates (but not other uses that rely on identity but don't currently bother to update it, like the identity hook).

* build: add BuildDate to version info will be used in enterprise to compare to license expiration time * cli: multi-line version output, add BuildDate before: $ nomad version Nomad v1.4.3 (coolfakecommithashomgoshsuchacoolonewoww) after: $ nomad version Nomad v1.5.0-dev BuildDate 2023-02-17T19:29:26Z Revision coolfakecommithashomgoshsuchacoolonewoww compare consul: $ consul version Consul v1.14.4 Revision dae670fe Build Date 2023-01-26T15:47:10Z Protocol 2 spoken by default, blah blah blah... and vault: $ vault version Vault v1.12.3 (209b3dd99fe8ca320340d08c70cff5f620261f9b), built 2023-02-02T09:07:27Z * docs: update version command output

This change resolves policies for workload identities when calling Client RPCs. Previously only ACL tokens could be used for Client RPCs. Since the same cache is used for both bearer tokens (ACL and Workload ID), the token cache size was doubled. --------- Co-authored-by: James Rasell <jrasell@users.noreply.github.com>

admin: post 1.5.0 rc.1 release

…16217) Nomad servers can advertise independent IP addresses for `serf` and `rpc`. Somewhat unexpectedly, the `serf` address is also used for both Serf and server-to-server RPC communication (including Raft RPC). The address advertised for `rpc` is only used for client-to-server RPC. This split was introduced intentionally in Nomad 0.8. When clients are using Consul discovery for connecting to servers, they get an initial discovery set from Consul and use the correct `rpc` tag in Consul to get a list of adddresses for servers. The client then makes a `Status.Peers` RPC to get the list of those servers that are raft peers. But this endpoint is shared between servers and clients, and provides the address used for Raft. Most of the time this is harmless because servers will bind on 0.0.0.0 anyways., But in topologies where servers are on a private network and clients are on separate subnets (or even public subnets), clients will make initial contact with the server to get the list of peers but then populate their local server set with unreachable addresses. Cluster administrators can work around this problem by using `server_join` with specific IP addresses (or DNS names), because the `Node.UpdateStatus` endpoint returns the correct set of RPC addresses when updating the node. So once a client has registered, it will get the correct set of RPC addresses. This changeset updates the client logic to query `Status.Members` instead of `Status.Peers`, and then extract the correctly advertised address and port from the response body.

… not with root user (#16222)

Fixes #16288. An earlier version of `go-plugin` introduced a warning log if `SecureConfig` is unset. For Nomad and other applications that have "internal" `go-plugin` consumers where the application runs itself as a plugin, this causes spurious warn-level logs. For Nomad in particular this means every task driver and logmon invocation emits the log, which is our primary operation. The change was reverted upstream, so this changeset picks up the reverted version.

The signature of the `raftApply` function requires that the caller unwrap the first returned value (the response from `FSM.Apply`) to see if it's an error. This puts the burden on the caller to remember to check two different places for errors, and we've done so inconsistently. Update `raftApply` to do the unwrapping for us and return any `FSM.Apply` error as the error value. Similar work was done in Consul in hashicorp/consul#9991. This eliminates some boilerplate and surfaces a few minor bugs in the process: * job deregistrations of already-GC'd jobs were still emitting evals * reconcile job summaries does not return scheduler errors * node updates did not report errors associated with inconsistent service discovery or CSI plugin states Note that although _most_ of the `FSM.Apply` functions return only errors (which makes it tempting to remove the first return value entirely), there are few that return `bool` for some reason and Variables relies on the response value for proper CAS checking.

@type

* Template and styles * @type to @color on flash messages * Notifications service as wrapper * Test cases updated for new notifs

admin: Post 1.5.0 release

) Signed-off-by: dttung2905 <ttdao.2015@accountancy.smu.edu.sg>

Signed-off-by: dttung2905 <ttdao.2015@accountancy.smu.edu.sg>

When native service discovery was added, we used the node secret as the auth token. Once Workload Identity was added in Nomad 1.4.x we needed to use the claim token for `template` blocks, and so we allowed valid claims to bypass the ACL policy check to preserve the existing behavior. (Invalid claims are still rejected, so this didn't widen any security boundary.) In reworking authentication for 1.5.0, we unintentionally removed this bypass. For WIs without a policy attached to their job, everything works as expected because the resulting `acl.ACL` is nil. But once a policy is attached to the job the `acl.ACL` is no longer nil and this causes permissions errors. Fix the regression by adding back the bypass for valid claims. In future work, we should strongly consider getting turning the implicit policies into real `ACLPolicy` objects (even if not stored in state) so that we don't have these kind of brittle exceptions to the auth code.

Several `nomad job` subcommands had duplicate or slightly similar logic for resolving a job ID from a CLI argument prefix, while others did not have this functionality at all. This commit pulls the shared logic to the command Meta and updates all `nomad job` subcommands to use it.

In Nomad 0.12.1 we introduced atomic job registration/deregistration, where the new eval was written in the same raft entry. Backwards-compatibility checks were supposed to have been removed in Nomad 1.1.0, but we missed that. This is long safe to remove.

Some of the methods in `Allocations()` incorrectly use the `putQuery` in API calls where `put` is more appropriate since they are not reading information back. These methods are also not returning request metadata such as `LastIndex` back to callers, which can be useful to have in some scenarios. They also provide poor developer experience as they take an `*api.Allocation` struct when only the allocation ID is necessary. This can lead consumers to make unnecessary API calls to fetch the full allocation. Fixing these problems require updating the methods' signatures so they take `*WriteOptions` instead of `*QueryOptions` and return `*WriteMeta`, but this is a breaking change that requires advanced notice to consumers. This commit adds a future breaking change notice and also fixes the `Stop` method so it properly returns request metadata in a backwards compatible way.

…ient links/chart (#16274) * Fix for wildcard DC sys/sysbatch jobs * A few extra modules for wildcard DC in systemish jobs * doesMatchPattern moved to its own util as match-glob * DC glob lookup using matchGlob * PR feedback

…16362) Wildcard datacenters introduced a bug where a job with any wildcard datacenters will always be treated as a destructive update when we check whether a datacenter has been removed from the jobspec. Includes updating the helper so that callers don't have to loop over the job's datacenters.

…6358)

…econciled (#16352) * docker: fix bug where network pause containers would be erroneously gc'd * docker: cl: thread context from driver into pause container restoration

This PR updates the provisioning step where we create /opt/nomad/data, such that it is with 0700 permissions in line with our security guidance.

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.5.0 to 0.7.0. - [Release notes](https://github.com/golang/crypto/releases) - [Commits](golang/crypto@v0.5.0...v0.7.0) --- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

#16329) No user facing changes so I assume no change log is required

…lper nomad (#16332) No user facing changes so I assume no change log is required

…d Plugins package (#16330) No user facing changes so I assume no change log is required

…drivers package (#16331) * Update ioutil library references to os and io respectively for drivers package No user facing changes so I assume no change log is required * Fix failing tests

…tively in the client package (#16318) * Update ioutil deprecated library references to os and io respectively * Deal with the errors produced. Add error handling to filEntry info Add error handling to info

* Removes an errant console.log and corrects a default sticky=true on toast notifications * Default so no need to refault

This PR configures - server nodes with a systemd unit running the agent as the nomad service user - client nodes with a root owned nomad data directory

* client: disable running artifact downloader as nobody This PR reverts a change from Nomad 1.5 where artifact downloads were executed as the nobody user on Linux systems. This was done as an attempt to improve the security model of artifact downloading where third party tools such as git or mercurial would be run as the root user with all the security implications thereof. However, doing so conflicts with Nomad's own advice for securing the Client data directory - which when setup with the recommended directory permissions structure prevents artifact downloads from working as intended. Artifact downloads are at least still now executed as a child process of the Nomad agent, and on modern Linux systems make use of the kernel Landlock feature for limiting filesystem access of the child process. * docs: update upgrade guide for 1.5.1 sandboxing * docs: add cl * docs: add title to upgrade guide fix

…16401) When the scheduler tries to find a placement for a new allocation, it iterates over a subset of nodes. For each node, we populate a `NetworkIndex` bitmap with the ports of all existing allocations and any other allocations already proposed as part of this same evaluation via its `SetAllocs` method. Then we make an "ask" of the `NetworkIndex` in `AssignPorts` for any ports we need and receive an "offer" in return. The offer will include both static ports and any dynamic port assignments. The `AssignPorts` method was written to support group networks, and it shares code that selects dynamic ports with the original `AssignTaskNetwork` code. `AssignTaskNetwork` can request multiple ports from the bitmap at a time. But `AssignPorts` requests them one at a time and does not account for possible collisions, and doesn't return an error in that case. What happens next varies: 1. If the scheduler doesn't place the allocation on that node, the port conflict is thrown away and there's no problem. 2. If the node is picked and this is the only allocation (or last allocation), the plan applier will reject the plan when it calls `SetAllocs`, as we'd expect. 3. If the node is picked and there are additional allocations in the same eval that iterate over the same node, their call to `SetAllocs` will detect the impossible state and the node will be rejected. This can have the puzzling behavior where a second task group for the job without any networking at all can hit a port collision error! It looks like this bug has existed since we implemented group networks, but there are several factors that add up to making the issue rare for many users yet frustratingly frequent for others: * You're more likely to hit this bug the more tightly packed your range for dynamic ports is. With 12000 ports in the range by default, many clusters can avoid this for a long time. * You're more likely to hit case (3) for jobs with lots of allocations or if a scheduler has to iterate over a large number of nodes, such as with system jobs, jobs with `spread` blocks, or (sometimes) jobs using `unique` constraints. For unlucky combinations of these factors, it's possible that case (3) happens repeatedly, preventing scheduling of a given job until a client state change (ex. restarting the agent so all its allocations are rescheduled elsewhere) re-opens the range of dynamic ports available. This changeset: * Fixes the bug by accounting for collisions in dynamic port selection in `AssignPorts`. * Adds test coverage for `AssignPorts`, expands coverage of this case for the deprecated `AssignTaskNetwork`, and tightens the dynamic port range in a scheduler test for spread scheduling to more easily detect this kind of problem in the future. * Adds a `String()` method to `Bitmap` so that any future "screaming" log lines have a human-readable list of used ports.

Most job subcommands allow for job ID prefix match as a convenience functionality so users don't have to type the full job ID. But this introduces a hard ACL requirement that the token used to run these commands have the `list-jobs` permission, even if the token has enough permission to execute the basic command action and the user passed an exact job ID. This change softens this requirement by not failing the prefix match in case the request results in a permission denied error and instead using the information passed by the user directly.

Update AWS EC2 CPU tables using `make ec2info`

* cli: add -json flag to alloc checks for completion * CLI: Expand test to include testing the json flag for allocation checks * Documentation: Add the checks command * Documentation: Add example for alloc check command * Update website/content/docs/commands/alloc/checks.mdx Co-authored-by: James Rasell <jrasell@users.noreply.github.com> * CLI: Add template flag to alloc checks command * Update website/content/docs/commands/alloc/checks.mdx Co-authored-by: James Rasell <jrasell@users.noreply.github.com> * CLI: Extend test to include -t flag for alloc checks * func: add changelog for added flags to alloc checks * cli[doc]: Make usage section on alloc checks clearer * Update website/content/docs/commands/alloc/checks.mdx Co-authored-by: James Rasell <jrasell@users.noreply.github.com> * Delete modd.conf * cli[doc]: add -t flag to command description for alloc checks --------- Co-authored-by: James Rasell <jrasell@users.noreply.github.com> Co-authored-by: Juanita De La Cuesta Morales <juanita.delacuestamorales@juanita.delacuestamorales-LHQ7X0QG9X>

Services must be interpolated to replace runtime variables before they can be compared against the values returned by Consul.

* e2e: job expects alloc to run until stopped * e2e: fix case changed by #16306 * e2e: couldn't find a bug but improved test+jobspecs

ACL policies can be associated with a job so that the job's Workload Identity can have expanded access to other policy objects, including other variables. Policies set on the variables the job automatically has access to were ignored, but this includes policies with `deny` capabilities. Additionally, when resolving claims for a workload identity without any attached policies, the `ResolveClaims` method returned a `nil` ACL object, which is treated similarly to a management token. While this was safe in Nomad 1.4.x, when the workload identity token was exposed to the task via the `identity` block, this allows a user with `submit-job` capabilities to escalate their privileges. We originally implemented automatic workload access to Variables as a separate code path in the Variables RPC endpoint so that we don't have to generate on-the-fly policies that blow up the ACL policy cache. This is fairly brittle but also the behavior around wildcard paths in policies different from the rest of our ACL polices, which is hard to reason about. Add an `ACLClaim` parameter to the `AllowVariableOperation` method so that we can push all this logic into the `acl` package and the behavior can be consistent. This will allow a `deny` policy to override automatic access (and probably speed up checks of non-automatic variable access).

Post 1.5.1 release

Which we no longer need since we no longer have consul as a dependency

In preperation for some refactoring to tasksUpdated, add a benchmark to the old code so it's easy to compare with the changes, making sure nothing goes off the rails for performance.

The tests for the system allocs reconciling code path (`diffSystemAllocs`) include many impossible test environments, such as passing allocs for the wrong node into the function. This makes the test assertions nonsensible for use in walking yourself through the correct behavior. I've pulled this changeset out of PR #16097 so that we can merge these improvements and revisit the right approach to fix the problem in #16097 with less urgency now that the PFNR bug fix has been merged. This changeset breaks up a couple of tests, expands test coverage, and makes test assertions more clear. It also corrects one bit of production code that behaves fine in production because of canonicalization, but forces us to remember to set values in tests to compensate.

Bumps [go.uber.org/goleak](https://github.com/uber-go/goleak) from 1.2.0 to 1.2.1. - [Release notes](https://github.com/uber-go/goleak/releases) - [Changelog](https://github.com/uber-go/goleak/blob/master/CHANGELOG.md) - [Commits](uber-go/goleak@v1.2.0...v1.2.1) --- updated-dependencies: - dependency-name: go.uber.org/goleak dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build: update from go1.20.1 to go1.20.2 Note that the CVE fixed in go1.20.2 does *not* impact Nomad. golang/go#58647

…c only (#16435) Signed-off-by: dttung2905 <ttdao.2015@accountancy.smu.edu.sg>

The job evaluate endpoint creates a new evaluation for the job which is a write operation. This change modifies the necessary capability from `read-job` to `submit-job` to better reflect this.

…sts (#16467) * cgv1: do not disable cpuset manager if reserved interface already exists This PR fixes a bug where restarting a Nomad Client on a machine using cgroups v1 (e.g. Ubuntu 20.04) would cause the cpuset cgroups manager to disable itself. This is being caused by incorrectly interpreting a "file exists" error as problematic when ensuring the reserved cpuset exists. If we get a "file exists" error, that just means the Client was likely restarted. Note that a machine reboot would fix the issue - the groups interfaces are ephemoral. * cl: add cl

Our auth token parsing code trims space around the `Authorization` header but not around `X-Nomad-Token`. When using the UI, it's easy to accidentally introduce a leading or trailing space, which results in spurious authentication errors. Trim the space at the HTTP server.

* cli: Add and flag to namespace status command * Update command/namespace_status.go Co-authored-by: James Rasell <jrasell@users.noreply.github.com> * cli: update tests for namespace status command to use must --------- Co-authored-by: James Rasell <jrasell@users.noreply.github.com>

Update for enterprise trial wording and link

…6421) * scheduler: annotate tasksUpdated with reason and purge DeepEquals * cr: move opaque into helper * cr: swap affinity/spread hashing for slice equal * contributing: update checklist-jobspec with notes about struct methods * cr: add more cases to wait config equal method * cr: use reflect when comparing envoy config blocks * cl: add cl

so enterprise tests don't fail due to the default zero time

Fix the e2e case where we download the go-getter bomb.zip test file, which is being removed on main. We can still get it from the version tag - yay git!

This PR fixes the non-root macOS use case where artifact downloads stopped working. It seems setting a Credential on a SysProcAttr used by the exec package will always cause fork/exec to fail - even if the credential contains our own UID/GID or nil UID/GID. Technically we do not need to set this as the child process will inherit the parent UID/GID anyway... and not setting it makes things work again ... /shrug

* docs: dispatch_payload docs had some weirdness Docs said "Examples" when there was only 1 example. Not sure what the floating "to" in the description was for. * docs: missing a heading level on jobs api docs

* landlock: git needs more files for private repositories This PR fixes artifact downloading so that git may work when cloning from private repositories. It needs - file read on /etc/passwd - dir read on /root/.ssh - file write on /root/.ssh/known_hosts Add these rules to the landlock rules for the artifact sandbox. * cr: use nonexistent instead of devnull Co-authored-by: Michael Schurter <mschurter@hashicorp.com> * cr: use go-homdir for looking up home directory * pr: pull go-homedir into explicit require * cr: fixup homedir tests in homeless root cases * cl: fix root test for real --------- Co-authored-by: Michael Schurter <mschurter@hashicorp.com>

In #16217 we switched clients using Consul discovery to the `Status.Members` endpoint for getting the list of servers so that we're using the correct address. This endpoint has an authorization gate, so this fails if the anonymous policy doesn't have `node:read`. We also can't check the `AuthToken` for the request for the client secret, because the client hasn't yet registered so the server doesn't have anything to compare against. Instead of hitting the `Status.Peers` or `Status.Members` RPC endpoint, use the Consul response directly. Update the `registerNode` method to handle the list of servers we get back in the response; if we get a "no servers" or "no path to region" response we'll kick off discovery again and retry immediately rather than waiting 15s.

* e2e: uniformly enable acls across all agents * docs: clarify that acls should be set everywhere

* services: always set deregister flag after deregistration of group This PR fixes a bug where the group service hook's deregister flag was not set in some cases, causing the hook to attempt deregistrations twice during job updates (alloc replacement). In the tests ... we used to assert on the wrong behvior (remove twice) which has now been corrected to assert we remove only once. This bug was "silent" in the Consul provider world because the error logs for double deregistration only show up in Consul logs; with the Nomad provider the error logs are in the Nomad agent logs. * services: cleanup group service hook tests

…espect default auth method (#16504) nomad login command does not need to know ACL Auth Method's type, since all method names are unique. Co-authored-by: James Rasell <jrasell@users.noreply.github.com>

* tests: add WaitForLeaders helpers using must/wait timings * tests: start servers for mtls tests together Fixes #16253 (hopefully)

updated propogating to propagating

* cli: Add and flags to server members * Update website/content/docs/commands/server/members.mdx Co-authored-by: James Rasell <jrasell@users.noreply.github.com> * Update website/content/docs/commands/server/members.mdx Co-authored-by: James Rasell <jrasell@users.noreply.github.com> * cli: update the server memebers tests to use must * cli: add flags addition to changelog --------- Co-authored-by: James Rasell <jrasell@users.noreply.github.com>

* cli: add json and t flags to quota status command * cli: add entry to changelog * Update command/quota_status.go Co-authored-by: James Rasell <jrasell@users.noreply.github.com> --------- Co-authored-by: James Rasell <jrasell@users.noreply.github.com>

* Added and flag to command * cli[style]: small refactor to avoid confussion with tmpl variable * Update inspect.mdx * cli: add changelog entry * Update .changelog/16478.txt Co-authored-by: James Rasell <jrasell@users.noreply.github.com> * Update command/quota_inspect.go Co-authored-by: James Rasell <jrasell@users.noreply.github.com> --------- Co-authored-by: James Rasell <jrasell@users.noreply.github.com>

* Throw your mouse into traffic * Add node metadata with a shortcut * Re-labelled * Adds a toast notification to job start/stop on keyboard shortcut * Typo fix

Fixes #16517 Given a 3 Server cluster with at least 1 Client connected to Follower 1: If a NodeMeta.{Apply,Read} for the Client request is received by Follower 1 with `AllowStale = false` the Follower will forward the request to the Leader. The Leader, not being connected to the target Client, will forward the RPC to Follower 1. Follower 1, seeing AllowStale=false, will forward the request to the Leader. The Leader, not being connected to... well hoppefully you get the picture: an infinite loop occurs.

The security fix in Go 1.20.2 does not apply to Nomad.

The drainer component is fairly complex. As part of upcoming work to fix some of the drainer's rough edges, document the drainer's architecture from a Nomad developer perspective.

Co-authored-by: James Rasell <jrasell@users.noreply.github.com>

* e2e: skip task api test when windows too old * e2e: don't run proxy on windows

In #12916 we updated the events test as part of a larger set of changes around mapstructure serialization fixes. But the changes to the jobs we're deploying in the tests had invalid task configs so they never result in good deployments and the test will always fail. Make the before/after jobs identical (except for the version bump) and make them valid. Also wait for allocations for the 2nd job run to appear before checking the deployment list, so that we don't race with the scheduler.

:(

Matches the "normal" HTTP error detection logic in the same file.

* fix: fix broken test * fix: fix broken test for quota status

* Copyable server and client attribute values * Changelog

* Generate files for 1.5.2 release * Prepare for next release * add 1.4.7 and 1.3.12 to the changelog --------- Co-authored-by: hc-github-team-nomad-core <github-team-nomad-core@hashicorp.com>

#16612) This changeset refactors the tests of the draining node watcher so that we don't mock the node watcher's `Remove` and `Update` methods for its own tests. Instead we'll mock the node watcher's dependencies (the job watcher and deadline notifier) and now unit tests can cover the real code. This allows us to remove a bunch of TODOs in `watch_nodes.go` around testing and clarify some important behaviors: * Nodes that are down or disconnected will still be watched until the scheduler decides what to do with their allocations. This will drive the job watcher but not the node watcher, and that lets the node watcher gracefully handle cases where a heartbeat fails but the node heartbeats again before its allocs can be evicted. * Stop watching nodes that have been deleted. The blocking query for nodes set the maximum index to the highest index of a node it found, rather than the index of the nodes table. This misses updates to the index from deleting nodes. This was done as an performance optimization to avoid excessive unblocking, but because the query is over all nodes anyways there's no optimization to be had here. Remove the optimization so we can detect deleted nodes without having to wait for an update to an unrelated node.

Implement the new `nomad job restart` command that allows operators to restart allocations tasks or reschedule then entire allocation. Restarts can be batched to target multiple allocations in parallel. Between each batch the command can stop and hold for a predefined time or until the user confirms that the process should proceed. This implements the "Stateless Restarts" alternative from the original RFC (https://gist.github.com/schmichael/e0b8b2ec1eb146301175fd87ddd46180). The original concept is still worth implementing, as it allows this functionality to be exposed over an API that can be consumed by the Nomad UI and other clients. But the implementation turned out to be more complex than we initially expected so we thought it would be better to release a stateless CLI-based implementation first to gather feedback and validate the restart behaviour. Co-authored-by: Shishir Mahajan <smahajan@roblox.com>

When a disconnect client reconnects the `allocReconciler` must find the allocations that were created to replace the original disconnected allocations. This process was being done in only a subset of non-terminal untainted allocations, meaning that, if the replacement allocations were not in this state the reconciler didn't stop them, leaving the job in an inconsistent state. This inconsistency is only solved in a future job evaluation, but at that point the allocation is considered reconnected and so the specific reconnection logic was not applied, leading to unexpected outcomes. This commit fixes the problem by running reconnecting allocation reconciliation logic earlier into the process, leaving the rest of the reconciler oblivious of reconnecting allocations. It also uses the full set of allocations to search for replacements, stopping them even if they are not in the `untainted` set. The system `SystemScheduler` is not affected by this bug because disconnected clients don't trigger replacements: every eligible client is already running an allocation.

…hibit_overlap is true Fixes #11052 When restoring periodic dispatcher, all periodic jobs are forced without checking for previous childre.

…hibit_overlap is true Fixes #11052 When restoring periodic dispatcher, all periodic jobs are forced without checking for previous children.

Co-authored-by: James Rasell <jrasell@users.noreply.github.com>

…ng-tortoise

Backport of Multiple instances of a periodic job are run simultaneously, when prohibit_overlap is true into release/1.3.x #16659

Backport of Multiple instances of a periodic job are run simultaneously, when prohibit_overlap is true into release/1.3.x #16659

Commits on Jan 30, 2023

Commits on Jan 31, 2023

Commits on Feb 1, 2023

Commits on Feb 2, 2023

Commits on Feb 3, 2023

Commits on Feb 6, 2023

Commits on Feb 7, 2023

Commits on Feb 8, 2023

Commits on Feb 9, 2023

Commits on Feb 10, 2023

Commits on Feb 13, 2023

Commits on Feb 14, 2023

Commits on Feb 15, 2023

Commits on Feb 16, 2023

Commits on Feb 17, 2023

Commits on Feb 20, 2023

Commits on Feb 21, 2023

Commits on Feb 22, 2023

Commits on Feb 23, 2023

Commits on Feb 27, 2023

Commits on Mar 1, 2023

Commits on Mar 2, 2023

Commits on Mar 3, 2023

Commits on Mar 6, 2023

Commits on Mar 7, 2023

Commits on Mar 8, 2023

Commits on Mar 9, 2023

Commits on Mar 10, 2023

Commits on Mar 13, 2023

Commits on Mar 14, 2023

Commits on Mar 15, 2023

Commits on Mar 16, 2023

Commits on Mar 17, 2023

Commits on Mar 20, 2023

Commits on Mar 21, 2023

Commits on Mar 22, 2023

Commits on Mar 23, 2023

Commits on Mar 24, 2023

Commits on Mar 27, 2023