You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Ran into a drastic performance problem. Sherlock is using the Guzzle class incorrectly, creating a new Client() for each request. I assumed that Guzzle shares curl handles across Clients(), but this is not actually true.
As a result, bulk inserts tend to deplete the system of sockets very quickly, causing the entire process to fail.
Need to split the Client initialization into its own class, similar to the Cluster class, so that it may be shared.
Ran into a drastic performance problem. Sherlock is using the Guzzle class incorrectly, creating a new Client() for each request. I assumed that Guzzle shares curl handles across Clients(), but this is not actually true.
As a result, bulk inserts tend to deplete the system of sockets very quickly, causing the entire process to fail.
Need to split the Client initialization into its own class, similar to the Cluster class, so that it may be shared.
Related, I'm going to try some experiments with Rolling-Curl (https://github.com/chuyskywalker/rolling-curl), which I think has a better batch/streaming mode of executing multi-handles
The text was updated successfully, but these errors were encountered: