Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically discard task result objects once received by the client #43

Open
rafa-be opened this issue Nov 15, 2024 · 1 comment
Open
Labels
enhancement New feature or request

Comments

@rafa-be
Copy link
Collaborator

rafa-be commented Nov 15, 2024

Currently, results data is kept by the scheduler until the client it's associated disconnects.

Ideally, we could discard the task result once received by the client:

In [1]: import math

In [2]: from scaler import Client

In [3]: client = Client("tcp://127.0.0.1:60338")

# ScalerTop shows 1 scheduler object (the serializer)

In [4]: future = client.submit(math.cos, 0)

# ScalerTop shows 4 scheduler objects (the serializer, the function, the function argument and the return value).

In [5]: future.result()
Out[5]: 1.0

# ScalerTop still shows 4 objects, while the scheduler could clear the task's result data as it reached the client's future. 

In [6]: client.disconnect()

# ScalerTop shows 0 scheduler object
@rafa-be rafa-be added the enhancement New feature or request label Nov 15, 2024
@rafa-be
Copy link
Collaborator Author

rafa-be commented Nov 20, 2024

Note that the client cannot always send a delete request when receiving the result object, as this result object might be required by another dependent graph task.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant