Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do every API call async #574

Merged
merged 46 commits into from
Nov 19, 2024
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
e6a6157
improvemnts
jeandemeusy Sep 24, 2024
44968be
reduced calls to channels endpoint
jeandemeusy Sep 25, 2024
36692fa
Merge branch 'main' into jean/various-code-improvements
jeandemeusy Sep 25, 2024
bdfee82
fix
jeandemeusy Sep 25, 2024
c9d17ad
Merge branch 'jean/various-code-improvements' of https://github.com/h…
jeandemeusy Sep 25, 2024
1ac53be
typos
jeandemeusy Sep 25, 2024
c7ca8f1
typo
jeandemeusy Sep 25, 2024
73dc2b8
typos
jeandemeusy Oct 4, 2024
a0a84b0
stop storing messages
jeandemeusy Oct 4, 2024
6c9c51e
imports refactoring
jeandemeusy Oct 4, 2024
7947c0e
enum for subgraphs
jeandemeusy Oct 4, 2024
daf0967
remove extra code
jeandemeusy Oct 4, 2024
0e2614a
Refactor subgraph mode variable names and comments
jeandemeusy Oct 4, 2024
f3f7ccb
Update ct-app/core/model/subgraph/entries/balance.py
jeandemeusy Oct 4, 2024
80cbaff
avoid calling enum vars
jeandemeusy Oct 4, 2024
1ae6a88
reduce channels sg calls
jeandemeusy Oct 7, 2024
a35b316
imports handling
jeandemeusy Oct 7, 2024
93ddebc
typos
jeandemeusy Oct 10, 2024
5ab3005
async api calls
jeandemeusy Oct 28, 2024
d4e4970
Merge branch 'main' into jean/make-api-calls-really-async
jeandemeusy Oct 29, 2024
70a0938
fmt
jeandemeusy Oct 29, 2024
f1921c3
reset staging params
jeandemeusy Oct 29, 2024
d98a825
extract messages only if call is successfull
jeandemeusy Oct 29, 2024
b82d8d2
typo in pop messages
jeandemeusy Oct 29, 2024
c98a75a
Merge branch 'main' into jean/make-api-calls-really-async
jeandemeusy Nov 12, 2024
7e02e30
missing api result conversions
jeandemeusy Nov 12, 2024
d20968c
fix parsing popped messages
jeandemeusy Nov 12, 2024
64776a0
Merge branch 'main' into jean/make-api-calls-really-async
jeandemeusy Nov 12, 2024
de3040c
Merge branch 'main' into jean/make-api-calls-really-async
jeandemeusy Nov 12, 2024
b0ce815
increase prod batch to db size
jeandemeusy Nov 12, 2024
e941a52
change storage values on staging
jeandemeusy Nov 12, 2024
5cf24d5
applied rabbit's recommendations
jeandemeusy Nov 12, 2024
9eb51fa
simplified model's reward calculation
jeandemeusy Nov 12, 2024
13b883e
endurance tests
jeandemeusy Nov 13, 2024
225ccb6
Update ct-app/tests_endurance/test_send_messages.py
jeandemeusy Nov 13, 2024
061aa07
dynamic message in load testing
jeandemeusy Nov 13, 2024
ae7876b
Merge branch 'jean/make-api-calls-really-async' of https://github.com…
jeandemeusy Nov 13, 2024
ce069bb
Merge branch 'main' into jean/make-api-calls-really-async
jeandemeusy Nov 14, 2024
58c3b40
typos
jeandemeusy Nov 14, 2024
dc19d26
store logs in file
jeandemeusy Nov 14, 2024
dd99d35
Merge branch 'main' into jean/make-api-calls-really-async
jeandemeusy Nov 15, 2024
0304f6f
Merge branch 'main' into jean/make-api-calls-really-async
jeandemeusy Nov 18, 2024
0e28dea
node_info safe return
jeandemeusy Nov 19, 2024
dc4627f
Merge branch 'main' into jean/make-api-calls-really-async
ausias-armesto Nov 19, 2024
98a044d
use open channel unified peerAddress
jeandemeusy Nov 19, 2024
c1125b5
remove repeating logs
jeandemeusy Nov 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions ct-app/.configs/core_prod_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ flags:
retrieveChannels: 600
retrieveBalances: 900

openChannels: 300
fundChannels: 300
closeOldChannels: 300
openChannels: 900
fundChannels: 1200
closeOldChannels: 900
jeandemeusy marked this conversation as resolved.
Show resolved Hide resolved
closePendingChannels: 1800
closeIncomingChannels: Off

Expand Down Expand Up @@ -91,8 +91,8 @@ economicModel:
#
# =============================================================================
storage:
count: 25
timeout: 3600 # seconds
count: 100
timeout: 900 # seconds

# =============================================================================
#
Expand Down
20 changes: 10 additions & 10 deletions ct-app/.configs/core_staging_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ flags:

connectedPeers: 120
registeredNodes: 600
topology: 600
topology: 60
allocations: 600
EOABalances: 600
NFTHolders: 600
Expand All @@ -22,23 +22,23 @@ flags:
node:
healthcheck: 60
retrievePeers: 120
retrieveChannels: 600
retrieveChannels: 60
retrieveBalances: 120

openChannels: 120
fundChannels: 120
closeOldChannels: 120
closePendingChannels: 120
openChannels: 900
fundChannels: 900
closeOldChannels: 900
closePendingChannels: 900
jeandemeusy marked this conversation as resolved.
Show resolved Hide resolved
closeIncomingChannels: Off

getTotalChannelFunds: 300

watchMessageQueue: On # should only be On / Off
relayedMessagesToDB: Off
relayedMessagesToDB: 120

peer:
messageRelayRequest: On # should only be On / Off
sentMessagesToDB: Off
sentMessagesToDB: 120

# =============================================================================
#
Expand Down Expand Up @@ -91,8 +91,8 @@ economicModel:
#
# =============================================================================
storage:
count: 100000
timeout: 43200 # seconds
count: 1000
timeout: 3600 # seconds

# =============================================================================
#
Expand Down
3 changes: 3 additions & 0 deletions ct-app/core/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from prometheus_client import start_http_server

from .components import AsyncLoop, Base, Parameters, Utils
from .components.messages import MessageQueue
from .core import Core
from .model.database import DatabaseConnection
from .node import Node
Expand Down Expand Up @@ -34,6 +35,8 @@ def main(configfile: str):

AsyncLoop.run(Core(nodes, params).start)

MessageQueue.clear()
jeandemeusy marked this conversation as resolved.
Show resolved Hide resolved

DatabaseConnection.close()


Expand Down
100 changes: 100 additions & 0 deletions ct-app/core/components/api_types.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
from typing import Any

from .channelstatus import ChannelStatus


def _convert(value: Any):
if value is None:
return None

try:
value = float(value)
except ValueError:
pass

try:
integer = int(value)
if integer == value:
value = integer

except ValueError:
pass

return value


class ApiObject:
def __init__(self, data: dict):
for key, value in self.keys.items():
setattr(self, key, _convert(data.get(value, None)))

self.post_init()

def post_init(self):
pass

def __str__(self):
return str(self.__dict__)

def __repr__(self):
return str(self)


class Addresses(ApiObject):
keys = {"hopr": "hopr", "native": "native"}


class Balances(ApiObject):
keys = {
"hopr": "hopr",
"native": "native",
"safe_native": "safeNative",
"safe_hopr": "safeHopr",
}


class Infos(ApiObject):
keys = {"hopr_node_safe": "hoprNodeSafe"}


class ConnectedPeer(ApiObject):
keys = {"address": "peerAddress", "peer_id": "peerId", "version": "reportedVersion"}


class Channel(ApiObject):
keys = {
"balance": "balance",
"id": "channelId",
"destination_address": "destinationAddress",
"destination_peer_id": "destinationPeerId",
"source_address": "sourceAddress",
"source_peer_id": "sourcePeerId",
"status": "status",
}

def post_init(self):
self.status = ChannelStatus.fromString(self.status)


class TicketPrice(ApiObject):
keys = {"value": "price"}

def post_init(self):
self.value = float(self.value) / 1e18


class OpenedChannel(ApiObject):
keys = {"channel_id": "channelId", "receipt": "transactionReceipt"}


class Channels:
def __init__(self, data: dict):
self.all = [Channel(channel) for channel in data.get("all", [])]
self.incoming = []
self.outgoing = []
jeandemeusy marked this conversation as resolved.
Show resolved Hide resolved

def __str__(self):
return str(self.__dict__)

def __repr__(self):
return str(self)
4 changes: 2 additions & 2 deletions ct-app/core/components/decorators.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,9 @@ async def wrapper(self, *args, **kwargs):
delay = None

if delay == 0:
self.info(f"Running `{params_raw[index]}` continuously")
self.info(f"Running `{func.__name__}` continuously")
elif delay is not None:
self.info(f"Running `{params_raw[index]}` every {delay} seconds")
self.info(f"Running `{func.__name__}` every {delay} seconds")

while self.running:
await func(self, *args, **kwargs)
Expand Down
Loading
Loading