Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add farmerbot #509

Merged
merged 58 commits into from
Dec 26, 2023
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
26bb994
add farm-node-power manager
rawdaGastan Nov 22, 2023
a5a0d76
Merge branch 'development-rmb-peer-router' of github.com:threefoldtec…
rawdaGastan Nov 23, 2023
3fade12
merge
rawdaGastan Nov 23, 2023
0269aef
add data manager and remove db
rawdaGastan Nov 23, 2023
d20bfcc
add some tests
rawdaGastan Nov 23, 2023
6c944a3
fix parsers
rawdaGastan Nov 27, 2023
032dc36
fix parsers
rawdaGastan Nov 27, 2023
9dd1afd
test node manager
rawdaGastan Nov 27, 2023
1cf1551
power manager test
rawdaGastan Nov 27, 2023
670b273
get nodes and farm from substrate
rawdaGastan Nov 28, 2023
78e30c5
add examples
rawdaGastan Nov 28, 2023
f17a162
merge
rawdaGastan Nov 29, 2023
c3c5a4d
merge substrate update
rawdaGastan Nov 29, 2023
60f802b
fix power conflicts
rawdaGastan Nov 29, 2023
a1941a1
use slices
rawdaGastan Nov 29, 2023
ba6af54
update node if its found
rawdaGastan Nov 29, 2023
4a7060b
sync update configs - testing - examples
rawdaGastan Nov 30, 2023
8375aa5
empty configs
rawdaGastan Nov 30, 2023
9c7107c
middleware didn't work
rawdaGastan Nov 30, 2023
211b90c
get networks as contants, remove log file
rawdaGastan Dec 3, 2023
b48867a
substrate connection, logger fatal, return error without logging, tak…
rawdaGastan Dec 3, 2023
a4b89f1
fix sub connection, map nodes
rawdaGastan Dec 3, 2023
2556623
support seed
rawdaGastan Dec 4, 2023
c654fb4
Merge branch 'development' of github.com:threefoldtech/tfgrid-sdk-go …
rawdaGastan Dec 4, 2023
39042fe
update config funcs, remove update go routine, remove graceful shutdown
rawdaGastan Dec 4, 2023
f67a97e
Merge branch 'development' of github.com:threefoldtech/tfgrid-sdk-go …
rawdaGastan Dec 5, 2023
87d07a6
remove models
rawdaGastan Dec 5, 2023
60b6ba8
move state, types with the bot
rawdaGastan Dec 6, 2023
97d72c1
add start command
rawdaGastan Dec 6, 2023
7ef9e66
exclude nodes if powered on and off, update nodes if power changed ou…
rawdaGastan Dec 6, 2023
a6538a7
get power state while loading nodes
rawdaGastan Dec 6, 2023
dbe4701
lock nodes using state
rawdaGastan Dec 6, 2023
07fb6f8
remove data manager
rawdaGastan Dec 6, 2023
9fe3e08
add include node example
rawdaGastan Dec 6, 2023
b29f35f
check not found rent contract
rawdaGastan Dec 6, 2023
9852c2f
don't include and included or excluded farm
rawdaGastan Dec 6, 2023
fb65431
edit logs
rawdaGastan Dec 7, 2023
c6bd1ee
add readme
rawdaGastan Dec 7, 2023
be8a8c4
add gpu and counters from zos
rawdaGastan Dec 7, 2023
89f54fe
remove overprovisioncpu from nodes and get from power configs
rawdaGastan Dec 7, 2023
df9ab6c
merge
rawdaGastan Dec 10, 2023
2328ac9
remove power manager
rawdaGastan Dec 10, 2023
0d89242
include periodic wake up in the main loop
rawdaGastan Dec 10, 2023
4ebeaad
fix tests
rawdaGastan Dec 10, 2023
b2c14ea
add test for find node and fix state tests
rawdaGastan Dec 11, 2023
3fcb135
fix lint
rawdaGastan Dec 11, 2023
5b2d79e
merge
rawdaGastan Dec 14, 2023
3366732
merge
rawdaGastan Dec 20, 2023
2b20bb2
fix rmb changes
rawdaGastan Dec 20, 2023
11409d3
Merge branch 'development' of github.com:threefoldtech/tfgrid-sdk-go …
rawdaGastan Dec 25, 2023
308bac0
add env
rawdaGastan Dec 25, 2023
468b3c4
improvements, goreleaser
rawdaGastan Dec 25, 2023
5e504fb
add dockerfile
rawdaGastan Dec 25, 2023
6f936f0
fix examples
rawdaGastan Dec 25, 2023
72ffd87
fix tests
rawdaGastan Dec 25, 2023
a90ae75
power on node if it's set to never shutdown and was off
rawdaGastan Dec 26, 2023
43cb7d9
add more tests
rawdaGastan Dec 26, 2023
25e132f
merge
rawdaGastan Dec 26, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ jobs:
matrix:
dir:
- activation-service
- farmerbot
- grid-cli
- grid-client
- grid-proxy
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ jobs:
matrix:
dir:
- activation-service
- farmerbot
- grid-cli
- grid-client
- gridify
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ This repo contains the go clients for Threefold grid.
- [monitoring bot](./monitoring-bot/README.md)
- [user contracts mon](./user-contracts-mon/README.md)
- [activation service](./activation-service/README.md)
- [farmerbot](./farmerbot/README.md)

## Release

Expand Down
23 changes: 23 additions & 0 deletions farmerbot/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib

# Test binary, built with `go test -c`
*.test

# Output of the go coverage tool, specifically when used with LiteIDE
*.out

# Dependency directories (remove the comment below to include it)
vendor/


coverage/
*.json
*.yml
*.yaml
*.toml
*.log
29 changes: 29 additions & 0 deletions farmerbot/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
GOPATH := $(shell go env GOPATH)

all: lint test

test:
@echo "Running Tests"
go test -v ./...

build:
@echo "Running $@"
@go build -ldflags=\
"-X 'github.com/threefoldtech/tfgrid-sdk-go/farmerbot/version.Version=$(shell git tag --sort=-version:refname | head -n 1)'"\
-o bin/farmerbot main.go


lint:
@echo "Running $@"
golangci-lint run -c ../.golangci.yml --timeout 10m

coverage: clean
mkdir coverage
go test -v -vet=off ./... -coverprofile=coverage/coverage.out
go tool cover -html=coverage/coverage.out -o coverage/coverage.html
@${GOPATH}/bin/gopherbadger -png=false -md="README.md"
rm coverage.out

clean:
rm ./coverage -rf
rm ./bin -rf
207 changes: 207 additions & 0 deletions farmerbot/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,207 @@
# Farmerbot

<a href='https://github.com/jpoles1/gopherbadger' target='_blank'>![gopherbadger-tag-do-not-edit](https://img.shields.io/badge/Go%20Coverage-73%25-brightgreen.svg?longCache=true&style=flat)</a>

Farmerbot is a service that a farmer can run allowing him to automatically manage power of the nodes of his farm.
rawdaGastan marked this conversation as resolved.
Show resolved Hide resolved

## How to use

- Create a new yml/yaml file `config.yml` and add your configurations:

```yml
farm_id: "<your farm ID, required>"
included_nodes:
- "<your node ID to be included, required at least 2>"
rawdaGastan marked this conversation as resolved.
Show resolved Hide resolved
excluded_nodes:
- "<your node ID to be excluded, optional>"
never_shutdown_nodes:
- "<your node ID to be never shutdown, optional>"
power:
periodic_wake_up_start: "<daily time to wake up nodes for your farm, default is the time your run the command, format is 00:00AM or 00:00PM, optional>"
wake_up_threshold: "<the threshold number for resources usage that will need another node to be on, default is 80, optional>"
periodic_wake_up_limit: "<the number (limit) of nodes to be waken up everyday, default is 1, optional>"
overprovision_cpu: "<how much node allows over provisioning the CPU , default is 1, range: [1;4], optional>"
rawdaGastan marked this conversation as resolved.
Show resolved Hide resolved
```

- [Download](#download) farmerbot binaries.

- Run the bot

```bash
farmerbot run -c config.json -m <mnemonic> -n dev -d
rawdaGastan marked this conversation as resolved.
Show resolved Hide resolved
```

Where:

```bash
Flags:
-c, --config string your config file that includes your farm, node and power configs. Available format is yml/yaml

Global Flags:
-d, --debug by setting this flag the farmerbot will print debug logs too
-m, --mnemonic string the mnemonic of the account of the farmer
-n, --network string the grid network to use (default "main")
-s, --seed string the hex seed of the account of the farmer
```

> Note: you should only provide **`mnemonic`** or **`seed`**

## Download

- Download the binaries from [releases](https://github.com/threefoldtech/tfgrid-sdk-go/releases)
- Extract the downloaded files
- Move the binary to any of `$PATH` directories, for example:

```bash
mv farmerbot /usr/local/bin
```

## Build

Run the following command inside the directory:

```bash
make build
```

## Supported commands
rawdaGastan marked this conversation as resolved.
Show resolved Hide resolved

- `start`: to start (power on) a node

```bash
farmerbot start --node <node ID> -m <mnemonic> -n dev -d
```

Where:

```bash
Flags:
--node uint32 the node ID you want to use

Global Flags:
-d, --debug by setting this flag the farmerbot will print debug logs too
-m, --mnemonic string the mnemonic of the account of the farmer
-n, --network string the grid network to use (default "main")
-s, --seed string the hex seed of the account of the farmer
```

- `start all`: to start (power on) all nodes in a farm

```bash
farmerbot start all --farm <farm ID> -m <mnemonic> -n dev -d
```

Where:

```bash
Flags:
--farm uint32 the farm ID you want to start your nodes ins

Global Flags:
-d, --debug by setting this flag the farmerbot will print debug logs too
-m, --mnemonic string the mnemonic of the account of the farmer
-n, --network string the grid network to use (default "main")
-s, --seed string the hex seed of the account of the farmer
```

- `version`: to get the current version of farmerbot

```bash
farmerbot version
```

## Calls

Calls can be send to the farmerbot via RMB. This section describes the arguments that they accept.

### farmerbot.nodemanager.findnode

This call allows you to look for a node with specific requirements (minimum amount of resources, public config, etc). You will get the node id as a result. The farmerbot will power on the node if the node is off. It will also claim the required resources for 30 minutes. After that, if the user has not deployed anything on the node the resources will be freed and the node might go down again if it was put on by that call.

Arguments (all arguments are optional):

- _has_gpus_ => if you require one or more gpus you can filter on that with this parameter (should be a positive value)
- _gpu_vendors_ => a list of strings that will be used to filter the nodes on gpu vendor (for example AMD)
- _gpu_devices_ => a list of strings that will be used to filter the nodes on gpu device (for example GTX 1080)
- _certified_ => whether or not you want a certified node (not adding this argument means you don't care whether you get a certified or non certified node)
- _public_config_ => whether or not you want a node with a public config (not adding this argument means you don't care whether or not the node has a public config)
- _public_ips_ => how much public ips you need
- _dedicated_ => whether you want a dedicated node (rent the full node)
- _node_exclude_ => the list of node ids you want to exclude in your search
- _hru_ => the amount of hru required in kilobytes
- _sru_ => the amount of sru required in kilobytes
- _mru_ => the amount of mru required in kilobytes
- _cru_ => the amount of cru required

Result:

- `node_id` => the node id that meets your requirements

Example:

- [findnode](./examples/findnode/main.go)

### farmerbot.powermanager.poweron

This call is only allowed to be executed if it comes from the farmer (the twin ID should equal the farmer's twin ID). It will power on the node specified in the arguments. After powering on a node it will be excluded from farmerbot management

Arguments:

- _node_id_ => the node id of the node that needs to powered on

Example:

- [power on](./examples/poweron/main.go)

### farmerbot.powermanager.poweroff

This call is only allowed to be executed if it comes from the farmer (the twin ID should equal the farmer's twin ID). It will power off the node specified in the arguments. After powering off a node it will be excluded from farmerbot management

Arguments:

- _node_id_ => the node id of the node that needs to powered off

Example:

- [power off](./examples/poweroff/main.go)

### farmerbot.farmmanager.version

This call returns the current version of the farmerbot

Example:

- [version](./examples/version/main.go)

### farmerbot.powermanager.includenode

This call is only allowed to be executed if it comes from the farmer (the twin ID should equal the farmer's twin ID). It will include an excluded node from power on and off calls (it should be included in the farmerbot configurations)

Arguments:

- _node_id_ => the node id of the node that needs to be included

Example:

- [include node](./examples/includenode/main.go)

## Examples

Check the [examples](/examples)

To run examples:

- Don't forget to write your mnemonic in the example

```bash
cd <example>
go run main.go
```

## Test

Run the following command inside the directory:

```bash
make test
```
Loading
Loading