Register metrics on PropertyModifierScheduler #954
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zbirenbaum
force-pushed
the
property-modifier-metrics-fix
branch
from
25614b2
to
54e5fc9
Compare
zbirenbaum
force-pushed
the
property-modifier-metrics-fix
branch
from
54e5fc9
to
525bd5b
Compare
allada
approved these changes
May 31, 2024
There was a problem hiding this comment.
Reviewed 1 of 1 files at r1, all commit messages.
Reviewable status: 1 of 1 LGTMs obtained, and pending CI: Bazel Dev / ubuntu-22.04, Cargo Dev / macos-13, Cargo Dev / ubuntu-22.04, Installation / macos-13, Installation / macos-14, Installation / ubuntu-22.04, Local / ubuntu-22.04, Publish image, Publish nativelink-worker-lre-cc, Remote / large-ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), macos-13, pre-commit-checks, windows-2022 / stable, and 1 discussions need to be resolved
nativelink-scheduler/src/property_modifier_scheduler.rs line 128 at r1 (raw file):
} // Register metrics for the underlying ActionScheduler
nit: period.
adam-singer
approved these changes
May 31, 2024
There was a problem hiding this comment.
Reviewed 1 of 1 files at r1, all commit messages.
Reviewable status: 1 of 1 LGTMs obtained, and pending CI: pre-commit-checks, and 1 discussions need to be resolved
Fixes metrics being lost due to the underlying ActionScheduler not having it's metrics passed through the PropertyModifierScheduler. This should fix any scaling issues related to queued actions metrics not being available.
zbirenbaum
force-pushed
the
property-modifier-metrics-fix
branch
from
525bd5b
to
b3269c9
Compare
nfarah86
pushed a commit
to nfarah86/nativelink
that referenced
this pull request
Jun 1, 2024
Fixes metrics being lost due to the underlying ActionScheduler not having it's metrics passed through the PropertyModifierScheduler. This should fix any scaling issues related to queued actions metrics not being available.
MarcusSorealheis
pushed a commit
to nfarah86/nativelink
that referenced
this pull request
Jun 1, 2024
author Nathan (Blaise) Bruer <github.blaise@allada.com> 1716060089 -0700 committer Marcus <marcuseagan@gmail.com> 1717216579 -0700 gpgsig -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEpvESN9ZnVKKULFG/x/6q6dPSZF0FAmZapUMACgkQx/6q6dPS ZF2kPQ//ZYXhy26xNq5g2P6U7kf+dpcXok2rzntw9620kdNCLodCm7vceDvwa1mn JwVnXdGO7QcrLeSagHk6LewptncYq+hOc3ZA7wUtM3bSUTwPlwJue7zNVwpMSZHD 9pdu9CDgVWh6h7oR+JoRcyHjJ7NTkSU41aLEtwaHbDWatmtX2Cy4iX0BNk55nWaK BtzzuLLgiWrpOXNLL4MWlRdxLbN2YjbxW+2S3Q5eg7SnlERKU+COLoaHYUrk8v9J qzK35H05wDiiXHQpnWky895G3DDo/NB1znVpyOf/3sPw+OwUvACoZjORIbeh7+0W J5NqfX1ngWTtu0dVT1tehguAplVrBCWGJgJcVs03egVlOVSL/WthdkjW/ZkMfEk5 pUL9Gje8SyuyEMw3tUc8EV6owwLqqyvVsaO+baVZR5hAbxp/yDaT/fewic6OXa4X I58NE5a7w9b5Zsem5kW+yFD5GSa11sybrqffd+ofzPmmDvlVYmJ/CaQvTZ/lQpaA IIZLyPYcicC3v2I1+GA//QFPJsUlK0McW+MR7L6zlY395yvS6TTA3EoCBY5KrsYD inSfeY2IuMKjV81ihQWd56XiK289GKkvspUmjy84ytke0luqM7Xfpu3cii+gCwoq 8B0dUCJOqEKg1AJUlM4VOS7vAwc1tJHHrEm05/opIycQxkwNnhs= =6j1C -----END PGP SIGNATURE----- parent 0a33c83 author Nathan (Blaise) Bruer <github.blaise@allada.com> 1716060089 -0700 committer Marcus <marcuseagan@gmail.com> 1717216536 -0700 gpgsig -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEpvESN9ZnVKKULFG/x/6q6dPSZF0FAmZapRgACgkQx/6q6dPS ZF1U1Q/9G9ZN2PkCUxBc+oTN1BjBbNB/52+xmi+Uao39vrGh2vPIOGYbjXPP36Vv NUQ5cckmlsoqC1VWWgJMts27Y6oSlP1JydRlgrXmhnNV/SjcAz6r2woP+3At58hk iZN0lqft7lZ1Viycq8bDDEDrT56BiGGbDqO0E3UMZncwp/4s5Cw8M+0DTapIz2PU UcxJk2goTlKwPIItlFBUF2dTPL+nxtI95rpIdaGRrP/pn7XaRuY72k5pGMVZHsS+ OVQBmXqYrSh3nUim+YWagvbDrYJU6lksI4ZDN6cnVgdEsF0xEWguCy3g6tR0WVUQ Z5Y9BkedyWt7v1tK+B5cBQgoP2m34pY8TzLFbQTLkv7mrxLIwyNqkDLVJOCPcamk dJ+yf5lp54vUkeRnVaTcp9IMEjF0Vjteg3WHTKNK8ZsBs9DnNzb+7HyjDa6bYcau e0Y5HVPuC5kcTMK95yTkAabdj1qtBr8QpJ7CErqTRBgbxpcgldd3DTNndHv5/Yjz QH/zctjfJcISk/Y+KDTBhZRsNF1VGAzpwsT3KnFUdcTg48xMFC6IBI/YkYywg4Ra ZIv6O9h0V7FivbXF5J0KqawpW4lQa6mLDzNwm7ZNODVNb5TEGYu8jsYMzkxaKVPR ZGEtFuBOPGgLVYcIbuAFeANBERzhw5KUMe7MBSP2DG7r+50pqQI= =zuq5 -----END PGP SIGNATURE----- parent 0a33c83 author Nathan (Blaise) Bruer <github.blaise@allada.com> 1716060089 -0700 committer Marcus <marcuseagan@gmail.com> 1717216503 -0700 gpgsig -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEpvESN9ZnVKKULFG/x/6q6dPSZF0FAmZapPcACgkQx/6q6dPS ZF0xpQ/+ILuc9NSpJpS6BNw5IUyE4cQsquFZhKBTsX1l8/ZMkDZR6pKdO+gsUdKB DyA8OX8VspRAWNoxyiEfLRknwNtclr6XG1znX+WcyVjnaujcezau54dhoEAsImdn pgwfN/RpLh3ii1Z/MbpPlDzdDcYkOtDiqF2l4Cr2wQSs72o6qONUmCrycNdipyHa lED+VesZOZiZTEBOX56hMGQ/NVSPQ9hisIVjFq1J472v+opU5Ht0cu8KSCRA2lud 2DCVwpnyjIcG9Bf/XlCMzw9lODbOTqehaGsmES6CSLngjzeojhiHqABc5qYQT6Ip umWgSV2/AQsNcMG69WB+uPPT1aBwT//XnBfUoSazhGvFQLn9A7oLPLarhZGUP8M1 IND49L3OBpzbbdY3ipN9e22f4NcYg7irhy/dzLIHjtBd9CPlUTQTybmRz6KebJom 6Ha/9CXKPcaW23NqsMNlCoWqSL+cIAEvYgEWcZrceOvkeEHppI6pSLMLyWHw9epF Odd3AX8EP18axCNe+GJlCOZlRFq9cNXAYuboa0HpHhBbMP7FiVuQhapR/iWTA1Pv 2X+hjkWURzpwFz+R/ObTBMiI1YqrTHkD7fukb1ibELenCs8UeIznZ7ScV6YX72Ef rOoxoHMrDRbXsWpc3JG3TlEbYpdpiWN09sQQpktCwZtU/uh5JsQ= =DmRi -----END PGP SIGNATURE----- [Breaking] Digest function now auto-detected from request (TraceMachina#899) The digest function the client requested is now properly implemented in the OriginContext. This allows us to inspect what digest-function the client requested from anywhere in the code. If you use `verify` store and use `hash_verification_function`, simply change it to `"verify_hash": true`. We will now auto-detect the hash function based on the request and if it is not set, we use the `default_digest_hash_function` in the `global` config. Update images for docs (TraceMachina#930) Update Rust crate proc-macro2 to v1.0.84 (TraceMachina#916) Refactor EvictingMap so it does not use DigestInfo (TraceMachina#932) In prep to move to an abstract key instead of DigestInfo, this change cleans up EvictingMap and removes dependencies on DigestInfo. towards: TraceMachina#931 Update Rust crate mimalloc to v0.1.42 (TraceMachina#933) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Update Rust crate parking_lot to v0.12.3 (TraceMachina#936) Include UUID in ActionState (TraceMachina#927) Changes the field used for identifying which action an ActionState corresponds to include a uuid as well as the ActionInfoHashKey. All existing functionality is kept the same by making use of the nested ActionInfoHashKey contained within the Id. These changes will provide the basis for all usage of Id in followup changes to the scheduler. This breaks compatibility for forwarding an operation from one remote execution system to another that does not use our operation name format (ie: very unlikely, but possible). Co-authored-by: Zach Birenbaum <zach@tracemachina.com> Add Operation State Manager API (TraceMachina#937) Introduces API to be implemented by new scheduler state backends. WorkerStateManager is responsible for managing action updates sent to and from workers. ClientStateManager is responsible any state related to the clients client(s) requesting the action execution and can add or filter operations based on a given criteria. MatchingEnginerStateManager can filter, update, or remove actions. Co-authored-by: Zach Birenbaum <zach@tracemachina.com> Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Update dependency protobuf to v26.0.bcr.1 (TraceMachina#887) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Update Rust crate quote to v1.0.36 (TraceMachina#938) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Update Rust crate redis to v0.25.4 (TraceMachina#944) Update Rust crate syn to v2.0.66 (TraceMachina#946) Bump flake and Bazel modules (TraceMachina#947) Fixes TraceMachina#943 Bump trivially bumpable deps (TraceMachina#950) Notably, this changes the S3 schema for multipart upload starts/stops. Add kind-loadbalancer (TraceMachina#929) Provide an envoy loadbalancer container to access the local kind cluster. The loadbalancer adds the ability to reach the local kind cluster trough localhost, so that it is possible to work with a mac on the nativelink cluster. Docker on Mac uses a aarch64 linux vm unter the hood, so the builds in the cluster can now be tested on that architecture. Users can now query the loadbalancer instead of the gateway. The loadbalancer has the following endpoints: localhost:8080 -> hubble-ui localhost:8081 -> tekton-dashboard localhost:8082/eventlistener -> nativelink-eventlistener Fix pulumi ratelimiting build error (TraceMachina#953) On MacOS the pulumi testsuite hit the ratelimiting on github. Disable the pulumi tests with a patch on nixpkgs. Commit 0eed759 didn't fix this issue. Add drake toolchain configs (TraceMachina#942) Register metrics on PropertyModifierScheduler (TraceMachina#954) Fixes metrics being lost due to the underlying ActionScheduler not having it's metrics passed through the PropertyModifierScheduler. This should fix any scaling issues related to queued actions metrics not being available. Use single quotes for char (TraceMachina#955) Nightly clippy reports a style issue when using double quotes around a single char. Increase pre-commit timeout in CI (TraceMachina#956) The original setting of 5 minutes was a bit too flaky. Increase it to 10 minutes so that cache misses don't break the job. Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Update images for docs (TraceMachina#930) Refactor EvictingMap so it does not use DigestInfo (TraceMachina#932) In prep to move to an abstract key instead of DigestInfo, this change cleans up EvictingMap and removes dependencies on DigestInfo. towards: TraceMachina#931 Update Rust crate mimalloc to v0.1.42 (TraceMachina#933) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Update Rust crate parking_lot to v0.12.3 (TraceMachina#936) Include UUID in ActionState (TraceMachina#927) Changes the field used for identifying which action an ActionState corresponds to include a uuid as well as the ActionInfoHashKey. All existing functionality is kept the same by making use of the nested ActionInfoHashKey contained within the Id. These changes will provide the basis for all usage of Id in followup changes to the scheduler. This breaks compatibility for forwarding an operation from one remote execution system to another that does not use our operation name format (ie: very unlikely, but possible). Co-authored-by: Zach Birenbaum <zach@tracemachina.com> Add Operation State Manager API (TraceMachina#937) Introduces API to be implemented by new scheduler state backends. WorkerStateManager is responsible for managing action updates sent to and from workers. ClientStateManager is responsible any state related to the clients client(s) requesting the action execution and can add or filter operations based on a given criteria. MatchingEnginerStateManager can filter, update, or remove actions. Co-authored-by: Zach Birenbaum <zach@tracemachina.com> Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Update Rust crate redis to v0.25.4 (TraceMachina#944) Update Rust crate syn to v2.0.66 (TraceMachina#946) Bump trivially bumpable deps (TraceMachina#950) Notably, this changes the S3 schema for multipart upload starts/stops. Add kind-loadbalancer (TraceMachina#929) Provide an envoy loadbalancer container to access the local kind cluster. The loadbalancer adds the ability to reach the local kind cluster trough localhost, so that it is possible to work with a mac on the nativelink cluster. Docker on Mac uses a aarch64 linux vm unter the hood, so the builds in the cluster can now be tested on that architecture. Users can now query the loadbalancer instead of the gateway. The loadbalancer has the following endpoints: localhost:8080 -> hubble-ui localhost:8081 -> tekton-dashboard localhost:8082/eventlistener -> nativelink-eventlistener Fix pulumi ratelimiting build error (TraceMachina#953) On MacOS the pulumi testsuite hit the ratelimiting on github. Disable the pulumi tests with a patch on nixpkgs. Commit 0eed759 didn't fix this issue. Add drake toolchain configs (TraceMachina#942) Register metrics on PropertyModifierScheduler (TraceMachina#954) Fixes metrics being lost due to the underlying ActionScheduler not having it's metrics passed through the PropertyModifierScheduler. This should fix any scaling issues related to queued actions metrics not being available. Use single quotes for char (TraceMachina#955) Nightly clippy reports a style issue when using double quotes around a single char. Increase pre-commit timeout in CI (TraceMachina#956) The original setting of 5 minutes was a bit too flaky. Increase it to 10 minutes so that cache misses don't break the job.
MarcusSorealheis
pushed a commit
to nfarah86/nativelink
that referenced
this pull request
Jun 1, 2024
author Nathan (Blaise) Bruer <github.blaise@allada.com> 1716060089 -0700 committer Marcus <marcuseagan@gmail.com> 1717216579 -0700 gpgsig -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEpvESN9ZnVKKULFG/x/6q6dPSZF0FAmZapUMACgkQx/6q6dPS ZF2kPQ//ZYXhy26xNq5g2P6U7kf+dpcXok2rzntw9620kdNCLodCm7vceDvwa1mn JwVnXdGO7QcrLeSagHk6LewptncYq+hOc3ZA7wUtM3bSUTwPlwJue7zNVwpMSZHD 9pdu9CDgVWh6h7oR+JoRcyHjJ7NTkSU41aLEtwaHbDWatmtX2Cy4iX0BNk55nWaK BtzzuLLgiWrpOXNLL4MWlRdxLbN2YjbxW+2S3Q5eg7SnlERKU+COLoaHYUrk8v9J qzK35H05wDiiXHQpnWky895G3DDo/NB1znVpyOf/3sPw+OwUvACoZjORIbeh7+0W J5NqfX1ngWTtu0dVT1tehguAplVrBCWGJgJcVs03egVlOVSL/WthdkjW/ZkMfEk5 pUL9Gje8SyuyEMw3tUc8EV6owwLqqyvVsaO+baVZR5hAbxp/yDaT/fewic6OXa4X I58NE5a7w9b5Zsem5kW+yFD5GSa11sybrqffd+ofzPmmDvlVYmJ/CaQvTZ/lQpaA IIZLyPYcicC3v2I1+GA//QFPJsUlK0McW+MR7L6zlY395yvS6TTA3EoCBY5KrsYD inSfeY2IuMKjV81ihQWd56XiK289GKkvspUmjy84ytke0luqM7Xfpu3cii+gCwoq 8B0dUCJOqEKg1AJUlM4VOS7vAwc1tJHHrEm05/opIycQxkwNnhs= =6j1C -----END PGP SIGNATURE----- parent 0a33c83 author Nathan (Blaise) Bruer <github.blaise@allada.com> 1716060089 -0700 committer Marcus <marcuseagan@gmail.com> 1717216536 -0700 gpgsig -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEpvESN9ZnVKKULFG/x/6q6dPSZF0FAmZapRgACgkQx/6q6dPS ZF1U1Q/9G9ZN2PkCUxBc+oTN1BjBbNB/52+xmi+Uao39vrGh2vPIOGYbjXPP36Vv NUQ5cckmlsoqC1VWWgJMts27Y6oSlP1JydRlgrXmhnNV/SjcAz6r2woP+3At58hk iZN0lqft7lZ1Viycq8bDDEDrT56BiGGbDqO0E3UMZncwp/4s5Cw8M+0DTapIz2PU UcxJk2goTlKwPIItlFBUF2dTPL+nxtI95rpIdaGRrP/pn7XaRuY72k5pGMVZHsS+ OVQBmXqYrSh3nUim+YWagvbDrYJU6lksI4ZDN6cnVgdEsF0xEWguCy3g6tR0WVUQ Z5Y9BkedyWt7v1tK+B5cBQgoP2m34pY8TzLFbQTLkv7mrxLIwyNqkDLVJOCPcamk dJ+yf5lp54vUkeRnVaTcp9IMEjF0Vjteg3WHTKNK8ZsBs9DnNzb+7HyjDa6bYcau e0Y5HVPuC5kcTMK95yTkAabdj1qtBr8QpJ7CErqTRBgbxpcgldd3DTNndHv5/Yjz QH/zctjfJcISk/Y+KDTBhZRsNF1VGAzpwsT3KnFUdcTg48xMFC6IBI/YkYywg4Ra ZIv6O9h0V7FivbXF5J0KqawpW4lQa6mLDzNwm7ZNODVNb5TEGYu8jsYMzkxaKVPR ZGEtFuBOPGgLVYcIbuAFeANBERzhw5KUMe7MBSP2DG7r+50pqQI= =zuq5 -----END PGP SIGNATURE----- parent 0a33c83 author Nathan (Blaise) Bruer <github.blaise@allada.com> 1716060089 -0700 committer Marcus <marcuseagan@gmail.com> 1717216503 -0700 gpgsig -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEpvESN9ZnVKKULFG/x/6q6dPSZF0FAmZapPcACgkQx/6q6dPS ZF0xpQ/+ILuc9NSpJpS6BNw5IUyE4cQsquFZhKBTsX1l8/ZMkDZR6pKdO+gsUdKB DyA8OX8VspRAWNoxyiEfLRknwNtclr6XG1znX+WcyVjnaujcezau54dhoEAsImdn pgwfN/RpLh3ii1Z/MbpPlDzdDcYkOtDiqF2l4Cr2wQSs72o6qONUmCrycNdipyHa lED+VesZOZiZTEBOX56hMGQ/NVSPQ9hisIVjFq1J472v+opU5Ht0cu8KSCRA2lud 2DCVwpnyjIcG9Bf/XlCMzw9lODbOTqehaGsmES6CSLngjzeojhiHqABc5qYQT6Ip umWgSV2/AQsNcMG69WB+uPPT1aBwT//XnBfUoSazhGvFQLn9A7oLPLarhZGUP8M1 IND49L3OBpzbbdY3ipN9e22f4NcYg7irhy/dzLIHjtBd9CPlUTQTybmRz6KebJom 6Ha/9CXKPcaW23NqsMNlCoWqSL+cIAEvYgEWcZrceOvkeEHppI6pSLMLyWHw9epF Odd3AX8EP18axCNe+GJlCOZlRFq9cNXAYuboa0HpHhBbMP7FiVuQhapR/iWTA1Pv 2X+hjkWURzpwFz+R/ObTBMiI1YqrTHkD7fukb1ibELenCs8UeIznZ7ScV6YX72Ef rOoxoHMrDRbXsWpc3JG3TlEbYpdpiWN09sQQpktCwZtU/uh5JsQ= =DmRi -----END PGP SIGNATURE----- [Breaking] Digest function now auto-detected from request (TraceMachina#899) The digest function the client requested is now properly implemented in the OriginContext. This allows us to inspect what digest-function the client requested from anywhere in the code. If you use `verify` store and use `hash_verification_function`, simply change it to `"verify_hash": true`. We will now auto-detect the hash function based on the request and if it is not set, we use the `default_digest_hash_function` in the `global` config. Update images for docs (TraceMachina#930) Update Rust crate proc-macro2 to v1.0.84 (TraceMachina#916) Refactor EvictingMap so it does not use DigestInfo (TraceMachina#932) In prep to move to an abstract key instead of DigestInfo, this change cleans up EvictingMap and removes dependencies on DigestInfo. towards: TraceMachina#931 Update Rust crate mimalloc to v0.1.42 (TraceMachina#933) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Update Rust crate parking_lot to v0.12.3 (TraceMachina#936) Include UUID in ActionState (TraceMachina#927) Changes the field used for identifying which action an ActionState corresponds to include a uuid as well as the ActionInfoHashKey. All existing functionality is kept the same by making use of the nested ActionInfoHashKey contained within the Id. These changes will provide the basis for all usage of Id in followup changes to the scheduler. This breaks compatibility for forwarding an operation from one remote execution system to another that does not use our operation name format (ie: very unlikely, but possible). Co-authored-by: Zach Birenbaum <zach@tracemachina.com> Add Operation State Manager API (TraceMachina#937) Introduces API to be implemented by new scheduler state backends. WorkerStateManager is responsible for managing action updates sent to and from workers. ClientStateManager is responsible any state related to the clients client(s) requesting the action execution and can add or filter operations based on a given criteria. MatchingEnginerStateManager can filter, update, or remove actions. Co-authored-by: Zach Birenbaum <zach@tracemachina.com> Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Update dependency protobuf to v26.0.bcr.1 (TraceMachina#887) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Update Rust crate quote to v1.0.36 (TraceMachina#938) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Update Rust crate redis to v0.25.4 (TraceMachina#944) Update Rust crate syn to v2.0.66 (TraceMachina#946) Bump flake and Bazel modules (TraceMachina#947) Fixes TraceMachina#943 Bump trivially bumpable deps (TraceMachina#950) Notably, this changes the S3 schema for multipart upload starts/stops. Add kind-loadbalancer (TraceMachina#929) Provide an envoy loadbalancer container to access the local kind cluster. The loadbalancer adds the ability to reach the local kind cluster trough localhost, so that it is possible to work with a mac on the nativelink cluster. Docker on Mac uses a aarch64 linux vm unter the hood, so the builds in the cluster can now be tested on that architecture. Users can now query the loadbalancer instead of the gateway. The loadbalancer has the following endpoints: localhost:8080 -> hubble-ui localhost:8081 -> tekton-dashboard localhost:8082/eventlistener -> nativelink-eventlistener Fix pulumi ratelimiting build error (TraceMachina#953) On MacOS the pulumi testsuite hit the ratelimiting on github. Disable the pulumi tests with a patch on nixpkgs. Commit 0eed759 didn't fix this issue. Add drake toolchain configs (TraceMachina#942) Register metrics on PropertyModifierScheduler (TraceMachina#954) Fixes metrics being lost due to the underlying ActionScheduler not having it's metrics passed through the PropertyModifierScheduler. This should fix any scaling issues related to queued actions metrics not being available. Use single quotes for char (TraceMachina#955) Nightly clippy reports a style issue when using double quotes around a single char. Increase pre-commit timeout in CI (TraceMachina#956) The original setting of 5 minutes was a bit too flaky. Increase it to 10 minutes so that cache misses don't break the job. Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Update images for docs (TraceMachina#930) Refactor EvictingMap so it does not use DigestInfo (TraceMachina#932) In prep to move to an abstract key instead of DigestInfo, this change cleans up EvictingMap and removes dependencies on DigestInfo. towards: TraceMachina#931 Update Rust crate mimalloc to v0.1.42 (TraceMachina#933) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Update Rust crate parking_lot to v0.12.3 (TraceMachina#936) Include UUID in ActionState (TraceMachina#927) Changes the field used for identifying which action an ActionState corresponds to include a uuid as well as the ActionInfoHashKey. All existing functionality is kept the same by making use of the nested ActionInfoHashKey contained within the Id. These changes will provide the basis for all usage of Id in followup changes to the scheduler. This breaks compatibility for forwarding an operation from one remote execution system to another that does not use our operation name format (ie: very unlikely, but possible). Co-authored-by: Zach Birenbaum <zach@tracemachina.com> Add Operation State Manager API (TraceMachina#937) Introduces API to be implemented by new scheduler state backends. WorkerStateManager is responsible for managing action updates sent to and from workers. ClientStateManager is responsible any state related to the clients client(s) requesting the action execution and can add or filter operations based on a given criteria. MatchingEnginerStateManager can filter, update, or remove actions. Co-authored-by: Zach Birenbaum <zach@tracemachina.com> Unbreak docker-compose workflow (TraceMachina#940) Fixes TraceMachina#939 Update Rust crate redis to v0.25.4 (TraceMachina#944) Update Rust crate syn to v2.0.66 (TraceMachina#946) Bump trivially bumpable deps (TraceMachina#950) Notably, this changes the S3 schema for multipart upload starts/stops. Add kind-loadbalancer (TraceMachina#929) Provide an envoy loadbalancer container to access the local kind cluster. The loadbalancer adds the ability to reach the local kind cluster trough localhost, so that it is possible to work with a mac on the nativelink cluster. Docker on Mac uses a aarch64 linux vm unter the hood, so the builds in the cluster can now be tested on that architecture. Users can now query the loadbalancer instead of the gateway. The loadbalancer has the following endpoints: localhost:8080 -> hubble-ui localhost:8081 -> tekton-dashboard localhost:8082/eventlistener -> nativelink-eventlistener Fix pulumi ratelimiting build error (TraceMachina#953) On MacOS the pulumi testsuite hit the ratelimiting on github. Disable the pulumi tests with a patch on nixpkgs. Commit 0eed759 didn't fix this issue. Add drake toolchain configs (TraceMachina#942) Register metrics on PropertyModifierScheduler (TraceMachina#954) Fixes metrics being lost due to the underlying ActionScheduler not having it's metrics passed through the PropertyModifierScheduler. This should fix any scaling issues related to queued actions metrics not being available. Use single quotes for char (TraceMachina#955) Nightly clippy reports a style issue when using double quotes around a single char. Increase pre-commit timeout in CI (TraceMachina#956) The original setting of 5 minutes was a bit too flaky. Increase it to 10 minutes so that cache misses don't break the job.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes metrics being lost due to the underlying ActionScheduler not having it's metrics passed through the PropertyModifierScheduler. This should fix any scaling issues related to queued actions metrics not being available.
Type of change
Please delete options that aren't relevant.
not work as expected)
Checklist
bazel test //...passes locallygit amendsee some docsThis change is