Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WorkerTuner & Resource based autotuning #1546

Merged
merged 17 commits into from
Aug 6, 2024
Merged

WorkerTuner & Resource based autotuning #1546

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
aa4e5d8
Initial refactoring
Sushisource Jul 2, 2024
fee3b86
Resource based tuner
Sushisource Jul 24, 2024
b6aa27e
Merge nexus work
Sushisource Jul 24, 2024
cdd2daf
Various fixes while testing
Sushisource Jul 25, 2024
18b5da4
Docstrings / visibility / package organization / construction tests
Sushisource Jul 25, 2024
2a643d1
Clean up runPoller error handling
Sushisource Jul 26, 2024
99355a0
Address a number of review comments
Sushisource Jul 29, 2024
af062a4
Interface upgrades
Sushisource Jul 29, 2024
1ad9127
Fix remaining todos
Sushisource Jul 29, 2024
30078da
A few more review fixes
Sushisource Jul 30, 2024
472ea56
Remove test-only nil check
Sushisource Jul 30, 2024
a0be0b4
More review fixes
Sushisource Jul 30, 2024
e7eff0a
Resourcetuner fix / more tests from omes discoveries
Sushisource Jul 31, 2024
d64a605
One more resource tuner improvement / interface polishing
Sushisource Jul 31, 2024
3a1a296
Move logger/handler into newBaseWorker & limit session creation
Sushisource Jul 31, 2024
7313808
Add back comment about session creation worker size
Sushisource Aug 6, 2024
6242cae
Avoid possible rare shutdown lock
Sushisource Aug 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,12 @@ Run the unit tests:
cd ./internal/cmd/build
go run . unit-test
```

## Updating go mod files

Sometimes all go.mod files need to be tidied. For an easy way to do this on linux or (probably) mac,
run:

```bash
find . -name go.mod -execdir go mod tidy \;
```
9 changes: 5 additions & 4 deletions client/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -480,8 +480,9 @@ type (
// Client is the client for starting and getting information about a workflow executions as well as
// completing activities asynchronously.
Client interface {
// ExecuteWorkflow starts a workflow execution and return a WorkflowRun instance and error
// The user can use this to start using a function or workflow type name.
// ExecuteWorkflow starts a workflow execution and returns a WorkflowRun instance or error
//
// This can be used to start a workflow using a function reference or workflow type name.
// Either by
// ExecuteWorkflow(ctx, options, "workflowTypeName", arg1, arg2, arg3)
// or
Expand All @@ -498,8 +499,8 @@ type (
// - Get(ctx context.Context, valuePtr interface{}) error: which will fill the workflow
// execution result to valuePtr, if workflow execution is a success, or return corresponding
// error. This is a blocking API.
// NOTE: if the started workflow return ContinueAsNewError during the workflow execution, the
// return result of GetRunID() will be the started workflow run ID, not the new run ID caused by ContinueAsNewError,
// NOTE: if the started workflow returns ContinueAsNewError during the workflow execution, the
// returned result of GetRunID() will be the started workflow run ID, not the new run ID caused by ContinueAsNewError,
// however, Get(ctx context.Context, valuePtr interface{}) will return result from the run which did not return ContinueAsNewError.
// Say ExecuteWorkflow started a workflow, in its first run, has run ID "run ID 1", and returned ContinueAsNewError,
// the second run has run ID "run ID 2" and return some result other than ContinueAsNewError:
Expand Down
1 change: 1 addition & 0 deletions contrib/opentelemetry/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ require (
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/nexus-rpc/sdk-go v0.0.9 // indirect
golang.org/x/sync v0.7.0 // indirect
)

require (
Expand Down
2 changes: 2 additions & 0 deletions contrib/opentelemetry/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,8 @@ golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
Expand Down
1 change: 1 addition & 0 deletions contrib/opentracing/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ require (
go.temporal.io/api v1.36.0 // indirect
golang.org/x/exp v0.0.0-20231127185646-65229373498e // indirect
golang.org/x/net v0.27.0 // indirect
golang.org/x/sync v0.7.0 // indirect
golang.org/x/sys v0.22.0 // indirect
golang.org/x/text v0.16.0 // indirect
golang.org/x/time v0.3.0 // indirect
Expand Down
2 changes: 2 additions & 0 deletions contrib/opentracing/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
Expand Down
48 changes: 48 additions & 0 deletions contrib/resourcetuner/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
module go.temporal.io/sdk/contrib/resourcetuner

go 1.21

toolchain go1.22.5

require (
github.com/shirou/gopsutil/v4 v4.24.6
github.com/stretchr/testify v1.9.0
go.einride.tech/pid v0.1.3
go.temporal.io/sdk v1.28.1
)

require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/facebookgo/clock v0.0.0-20150410010913-600d898af40a // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/mock v1.6.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.20.0 // indirect
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
github.com/nexus-rpc/sdk-go v0.0.9 // indirect
github.com/pborman/uuid v1.2.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
github.com/robfig/cron v1.2.0 // indirect
github.com/shoenig/go-m1cpu v0.1.6 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/tklauser/go-sysconf v0.3.12 // indirect
github.com/tklauser/numcpus v0.6.1 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.temporal.io/api v1.36.0 // indirect
golang.org/x/exp v0.0.0-20231127185646-65229373498e // indirect
golang.org/x/net v0.27.0 // indirect
golang.org/x/sync v0.7.0 // indirect
golang.org/x/sys v0.22.0 // indirect
golang.org/x/text v0.16.0 // indirect
golang.org/x/time v0.3.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240711142825-46eb208f015d // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240711142825-46eb208f015d // indirect
google.golang.org/grpc v1.65.0 // indirect
google.golang.org/protobuf v1.34.2 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

replace go.temporal.io/sdk => ../../
Loading
Loading