RateLimitException hit for all write/upload operations

[andy-esch]

cf version: 1.0b1
python version: 3.7.3

I'm not able to write (with cartocontext.write) or upload (with dataset.upload) to my carto account with v1.0b1. I keep getting RateLimitException

Looking into it, I'm not sure why this would be happening 🤷‍♂

@alrocar any idea?

from cartoframes.auth import set_default_context, Context
from cartoframes.data import Dataset


cc = Context(
    base_url='https://eschbacher.carto.com/',
    api_key='abc123'
)

set_default_context(
    context=cc
)

df = pd.DataFrame({
        'lat': range(10),
        'lng': range(10, 0, -1),
        'attr': list('abcdefghij')
    })

## This will raise a RateLimitException
## Behavior is the same for new table and overwriting existing tables
cc.write(
    df,
    'test4',
    overwrite=True
)

## This will raise a RateLimitException too
Dataset.from_dataframe(df).upload(table_name='isthisthingon')

[andy-esch]

Well now it works extremely well as if nothing is wrong. Maybe related to #780 ?

[alrocar]

If I'm not wrong RateLimitException is thrown mainly when a timeout occurs in the database. And CartoRateLimitException is only thrown when there's a legit case of a rate limit hit.

If it was a case of RateLimitException it could mean there was some lock in the database. Let's summon @CartoDB/rt-managers to prioritize now that @oleurud is helping there and could investigate what happened. IMO this is more related to our APIs than to CARTOframes itself.

In any case, from the CARTOframes side, we'll have to be more verbose for this kind of exceptions, otherwise will get eventually a bunch of recurrent support cases.

[simon-contreras-deel]

After testing it with both accounts (yours and mine), I have not been able to reproduce it.

About timeout: now we are using Batch or Copy, and they have a timeout of 12 hours and 5 hours respectively. So, I suppose that is not the case.

About rate limits: it could be possible because we have a limit of 1 copy every ~10 seconds, but the error should be CartoRateLimitException, could it be? It could be for example, if you are doing requests and someone is using your credentials (from an example) at the same time.

Have you suffered it again after that?

[simon-contreras-deel]

Checking it in Kibana, I see 7 responses with a 429 error from your account and all of them are query timeouts. It is rare because the timeout is raising after 25 seconds (the default db timeout) instead of 5 hours.

Further work needed to know the problem importance

[simon-contreras-deel]

In the last 7 days, we have only your 7 requests from a total of 3.370.

For now, I would forget this one, but I will keep an eye on the dashboard the following days.

I am going to close it, but feel free to reopen if you see something wrong or it is happening again

[andy-esch]

Thanks for looking into this @oleurud. Yeah, another one of those strange transient ones. I'll ping back if I see anything again.