Skip to content

Commit

Permalink
Merge pull request #419 from valory-xyz/develop
Browse files Browse the repository at this point in the history
Release `v0.22.5`
  • Loading branch information
annasambrook authored Jan 13, 2025
2 parents 154ebe5 + 2f01c60 commit c951263
Show file tree
Hide file tree
Showing 6 changed files with 226 additions and 34 deletions.
18 changes: 11 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Ensure your machine satisfies the requirements:
## Resource Requirements

- You need xDAI on Gnosis Chain in one of your wallets.
- You need an RPC for your agent instance. We recommend [Nodies RPC](https://www.nodies.app/).
- You need an RPC for your agent instance. We recommend [Quicknode RPC](https://www.quicknode.com/).
- (From release v0.16.0 onwards) You will need a Subgraph API key that can be obtained at [The Graph](https://thegraph.com/studio/apikeys/).

## Run the Service
Expand Down Expand Up @@ -105,7 +105,7 @@ Services can become staked by invoking the `stake()` contract method, where serv
Once the command has completed, i.e. the service is running, you can see the live logs with:

```bash
docker logs trader_abci_0 --follow
docker logs $(docker ps --filter "name=trader" --format "{{.Names}}" | grep "_abci" | head -n 1) --follow
```

To stop your agent, use:
Expand Down Expand Up @@ -153,16 +153,16 @@ Note: In this case, if the service is staked, then it will not update the on-cha
cd trader; poetry run python ../report.py; cd ..
```

3. Use this command to investigate your agent's logs:
3. Use the `analyse_logs.py` script to investigate your agent's logs:
```bash
cd trader; poetry run autonomy analyse logs --from-dir trader_service/abci_build/persistent_data/logs/ --agent aea_0 --reset-db; cd ..
cd trader; poetry run python ../analyse_logs.py --agent aea_0 --reset-db; cd ..
```
For example, inspect the state transitions using this command:
For example, inspect the state transitions using the following command:
```bash
cd trader; poetry run autonomy analyse logs --from-dir trader_service/abci_build/persistent_data/logs/ --agent aea_0 --fsm --reset-db; cd ..
cd trader; poetry run python ../analyse_logs.py --agent aea_0 --fsm --reset-db; cd ..
```
This will output the different state transitions of your agent per period, for example:
Expand Down Expand Up @@ -411,7 +411,11 @@ Error: Service terminatation failed with following error; ChainInteractionError(
## Build deployments without executing the service
The script builds both a Docker Compose deployment (on `./trader/trader_service/abci_build`) and a Kubernetes deployment (on `./trader/trader_service/abci_build_k8s`). Then, by default, the script will launch the local Docker Compose deployment. If you just want to build the deployment without executing the service (for example, if you are deploying to a custom Kubernetes cluster), then execute the script as
The script builds both a Docker Compose deployment (on `./trader/trader_service/abci_build_????`)
and a Kubernetes deployment (on `./trader/trader_service/abci_build_k8s`).
Then, by default, the script will launch the local Docker Compose deployment.
If you just want to build the deployment without executing the service
(for example, if you are deploying to a custom Kubernetes cluster), then execute the script as:
```bash
./run_service.sh --build-only
Expand Down
153 changes: 153 additions & 0 deletions analyse_logs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
import os
import subprocess
import sys
import argparse


def _parse_args():
"""Parse the script arguments."""
parser = argparse.ArgumentParser(description="Analyse agent logs.")

parser.add_argument(
"--service-dir",
default="trader_service",
help="The service directory containing build directories (default: 'trader_service')."
)
parser.add_argument(
"--from-dir",
help="Path to the logs directory. If not provided, it is auto-detected."
)
parser.add_argument(
"--agent",
default="aea_0",
help="The agent name to analyze (default: 'aea_0')."
)
parser.add_argument(
"--reset-db",
action="store_true",
help="Use this flag to disable resetting the log database."
)
parser.add_argument(
"--start-time",
help="Start time in `YYYY-MM-DD H:M:S,MS` format."
)
parser.add_argument(
"--end-time",
help="End time in `YYYY-MM-DD H:M:S,MS` format."
)
parser.add_argument(
"--log-level",
choices=["INFO", "DEBUG", "WARNING", "ERROR", "CRITICAL"],
help="Logging level."
)
parser.add_argument(
"--period",
type=int,
help="Period ID."
)
parser.add_argument(
"--round",
help="Round name."
)
parser.add_argument(
"--behaviour",
help="Behaviour name filter."
)
parser.add_argument(
"--fsm",
action="store_true",
help="Print only the FSM execution path."
)
parser.add_argument(
"--include-regex",
help="Regex pattern to include in the result."
)
parser.add_argument(
"--exclude-regex",
help="Regex pattern to exclude from the result."
)

return parser.parse_args()


def find_build_directory(service_dir):
"""Find the appropriate build directory within the service directory."""
try:
# create a list of all build directories
build_dirs = [
d for d in os.listdir(service_dir)
if d.startswith("abci_build_") and os.path.isdir(os.path.join(service_dir, d))
]
# iterate through the build directories to find the one that contains logs
for build_dir in build_dirs:
build_dir = os.path.join(service_dir, build_dir)
logs_dir = os.path.join(build_dir, "persistent_data", "logs")
# Check if the logs folder exists and contains files
if os.path.exists(logs_dir) and os.listdir(logs_dir):
return build_dir
return os.path.join(service_dir, "abci_build")
except FileNotFoundError:
print(f"Service directory '{service_dir}' not found")
sys.exit(1)


def run_analysis(logs_dir, **kwargs):
"""Run the log analysis command."""
command = [
"poetry", "run", "autonomy", "analyse", "logs",
"--from-dir", logs_dir,
]
if kwargs.get("agent"):
command.extend(["--agent", kwargs.get("agent")])
if kwargs.get("reset_db"):
command.extend(["--reset-db"])
if kwargs.get("start_time"):
command.extend(["--start-time", kwargs.get("start_time")])
if kwargs.get("end_time"):
command.extend(["--end-time", kwargs.get("end_time")])
if kwargs.get("log_level"):
command.extend(["--log-level", kwargs.get("log_level")])
if kwargs.get("period"):
command.extend(["--period", kwargs.get("period")])
if kwargs.get("round"):
command.extend(["--round", kwargs.get("round")])
if kwargs.get("behaviour"):
command.extend(["--behaviour", kwargs.get("behaviour")])
if kwargs.get("fsm"):
command.extend(["--fsm"])
if kwargs.get("include_regex"):
command.extend(["--include-regex", kwargs.get("include_regex")])
if kwargs.get("exclude_regex"):
command.extend(["--exclude-regex", kwargs.get("exclude_regex")])

try:
subprocess.run(command, check=True)
print("Analysis completed successfully.")
except subprocess.CalledProcessError as e:
print(f"Command failed with exit code {e.returncode}")
sys.exit(e.returncode)
except FileNotFoundError:
print("Poetry or autonomy not found. Ensure they are installed and accessible.")
sys.exit(1)


if __name__ == "__main__":
# Parse user arguments
args = _parse_args()

# Determine the logs directory
if args.from_dir:
logs_dir = args.from_dir
if not os.path.exists(logs_dir):
print(f"Specified logs directory '{logs_dir}' not found.")
sys.exit(1)
else:
# Auto-detect the logs directory
build_dir = find_build_directory(args.service_dir)
logs_dir = os.path.join(build_dir, "persistent_data", "logs")
if not os.path.exists(logs_dir):
print(f"Logs directory '{logs_dir}' not found.")
sys.exit(1)

# Run the analysis
run_analysis(logs_dir, **vars(args))
29 changes: 16 additions & 13 deletions report.py
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,9 @@
SECONDS_PER_DAY = 60 * 60 * 24

OUTPUT_WIDTH = 80
TRADER_CONTAINER_PREFIX = "trader"
AGENT_CONTAINER_IDENTIFIER = "abci"
NODE_CONTAINER_IDENTIFIER = "tm"


class ColorCode:
Expand Down Expand Up @@ -238,19 +241,19 @@ def _warning_message(current_value: int, threshold: int = 0, message: str = "")

def _get_agent_status() -> str:
client = docker.from_env()
trader_abci_container = (
client.containers.get("trader_abci_0")
if "trader_abci_0" in [c.name for c in client.containers.list()]
else None
)
trader_tm_container = (
client.containers.get("trader_tm_0")
if "trader_tm_0" in [c.name for c in client.containers.list()]
else None
)

is_running = trader_abci_container and trader_tm_container
return _color_bool(is_running, "Running", "Stopped")
agent_running = node_running = service_running = False
for container in client.containers.list():
container_name = container.name
if TRADER_CONTAINER_PREFIX in container_name:
if AGENT_CONTAINER_IDENTIFIER in container_name:
agent_running = True
if NODE_CONTAINER_IDENTIFIER in container_name:
node_running = True
if agent_running and node_running:
service_running = True
break

return _color_bool(service_running, "Running", "Stopped")


def _parse_args() -> Any:
Expand Down
29 changes: 21 additions & 8 deletions run_service.sh
Original file line number Diff line number Diff line change
Expand Up @@ -737,7 +737,7 @@ directory="trader"
service_repo=https://github.com/$org_name/$directory.git
# This is a tested version that works well.
# Feel free to replace this with a different version of the repo, but be careful as there might be breaking changes
service_version="v0.21.4"
service_version="v0.22.0"

# Define constants for on-chain interaction
gnosis_chain_id=100
Expand Down Expand Up @@ -832,10 +832,15 @@ command -v docker >/dev/null 2>&1 ||
exit 1
}

docker rm -f abci0 node0 trader_abci_0 trader_tm_0 &> /dev/null ||
containers=$(docker ps --filter name=trader_* -aq) &> /dev/null ||
{ echo >&2 "Docker is not running!";
exit 1
}
if [[ -n "$containers" ]]; then
docker rm -f $containers
fi

docker network prune --force

try_read_storage

Expand Down Expand Up @@ -1275,8 +1280,7 @@ export DISABLE_TRADING=false
export STOP_TRADING_IF_STAKING_KPI_MET=true
export RESET_PAUSE_DURATION=45
export MECH_WRAPPED_NATIVE_TOKEN_ADDRESS=$WXDAI_ADDRESS
export MECH_CHAIN_ID=ethereum
export TOOLS_ACCURACY_HASH=QmebjcPizAdVFSUAfMBgAGFJhLPVBMvV68LxhSq4LPvv9d
export TOOLS_ACCURACY_HASH=QmZSkE49cnp3KeR9r6bp3hP4M2LPAmG4beHq4isz55ghv5

if [ -n "$SUBGRAPH_API_KEY" ]; then
export CONDITIONAL_TOKENS_SUBGRAPH_URL="https://gateway-arbitrum.network.thegraph.com/api/$SUBGRAPH_API_KEY/subgraphs/id/7s9rGBffUTL8kDZuxvvpuc46v44iuDarbrADBFw5uVp2"
Expand All @@ -1287,9 +1291,9 @@ if [ -n "$SUBGRAPH_API_KEY" ]; then
fi

service_dir="trader_service"
build_dir="abci_build"
directory=$(ls -d "$service_dir"/abci_build_???? 2>/dev/null || echo "$service_dir/abci_build")
build_dir=$(basename "$directory")
build_dir_k8s="abci_build_k8s"
directory="$service_dir/$build_dir"

if [ -d $directory ]
then
Expand Down Expand Up @@ -1329,19 +1333,28 @@ if [[ -d "$build_dir_k8s" ]]; then
echo "Directory removed: $build_dir"
fi
export OPEN_AUTONOMY_PRIVATE_KEY_PASSWORD="$password" && poetry run autonomy deploy build --kubernetes "../../$keys_json_path" --n $n_agents -ltm
build_dir=$(ls -d abci_build_???? 2>/dev/null || echo "abci_build")
mv $build_dir $build_dir_k8s
echo "Kubernetes deployment built on ./trader/$service_dir/$build_dir_k8s"

export OPEN_AUTONOMY_PRIVATE_KEY_PASSWORD="$password" && poetry run autonomy deploy build "../../$keys_json_path" --n $n_agents -ltm
build_dir=$(ls -d abci_build_???? 2>/dev/null || echo "abci_build")
echo "Docker Compose deployment built on ./trader/$service_dir/$build_dir"

cd ..

# warm start is disabled as no global weights are provided to calibrate the tools' weights
# warm_start

add_volume_to_service_docker_compose "$PWD/trader_service/abci_build/docker-compose.yaml" "trader_abci_0" "/data" "$path_to_store"
add_volume_to_service_k8s "$PWD/trader_service/abci_build_k8s/build.yaml"
directory="$service_dir/$build_dir"
if [ "$build_dir" = "abci_build" ]; then
suffix="abci_build"
else
suffix=${build_dir##*_}
fi
abci_0="trader${suffix}_abci_0"
add_volume_to_service_docker_compose "$PWD/$directory/docker-compose.yaml" "$abci_0" "/data" "$path_to_store"
add_volume_to_service_k8s "$PWD/$service_dir/$build_dir_k8s/build.yaml"
sudo chown -R $(whoami) "$path_to_store"

if [[ "$build_only" == true ]]; then
Expand Down
Loading

0 comments on commit c951263

Please sign in to comment.