Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: rate limit error chaching #2448

Merged
merged 3 commits into from
Sep 17, 2023
Merged

feat: rate limit error chaching #2448

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
aa2e8ff
feat: rate limit error chaching
rickstaa Jan 21, 2023
ba2d9e3
Merge branch 'master' into change_error_cache
rickstaa Sep 17, 2023
ccb6b92
refactor: improve code comments
rickstaa Sep 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion api/gist.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,12 @@ export default async (req, res) => {
}),
);
} catch (err) {
res.setHeader("Cache-Control", `no-cache, no-store, must-revalidate`); // Don't cache error responses.
res.setHeader(
"Cache-Control",
`max-age=${CONSTANTS.ERROR_CACHE_SECONDS / 2}, s-maxage=${
CONSTANTS.ERROR_CACHE_SECONDS
}, stale-while-revalidate=${CONSTANTS.ONE_DAY}`,
); // Use lower cache period for errors.
return res.send(renderError(err.message, err.secondaryMessage));
}
};
9 changes: 7 additions & 2 deletions api/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ export default async (req, res) => {
);

let cacheSeconds = clampValue(
parseInt(cache_seconds || CONSTANTS.FOUR_HOURS, 10),
parseInt(cache_seconds || CONSTANTS.CARD_CACHE_SECONDS, 10),
CONSTANTS.FOUR_HOURS,
CONSTANTS.ONE_DAY,
);
Expand Down Expand Up @@ -100,7 +100,12 @@ export default async (req, res) => {
}),
);
} catch (err) {
res.setHeader("Cache-Control", `no-cache, no-store, must-revalidate`); // Don't cache error responses.
res.setHeader(
"Cache-Control",
`max-age=${CONSTANTS.ERROR_CACHE_SECONDS / 2}, s-maxage=${
CONSTANTS.ERROR_CACHE_SECONDS
}, stale-while-revalidate=${CONSTANTS.ONE_DAY}`,
Comment on lines +105 to +107
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whats happening here?

CONSTANTS.ERROR_CACHE_SECONDS / 2 in max-age
CONSTANTS.ERROR_CACHE_SECONDS in s-maxage
CONSTANTS.ONE_DAY in revalidate?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait this code is on master lol. Have no idea why is it like this 😆

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay this was the change #2124

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, this was when I would sometimes merge PRs without your review since they had been open for a long time. Sorry for the confusion 😅. Feel free to change it to any caching behavior you like. The reasoning for the current caching code can be found at #2124 (comment).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anuraghazra, we can also remove these lines and keep the same behavior as for the case when no error is thrown. Both are fine with me.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep the same behaviour as old one. Not really confident with how this header will behave on prod.

); // Use lower cache period for errors.
return res.send(renderError(err.message, err.secondaryMessage));
}
};
9 changes: 7 additions & 2 deletions api/pin.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ export default async (req, res) => {
const repoData = await fetchRepo(username, repo);

let cacheSeconds = clampValue(
parseInt(cache_seconds || CONSTANTS.FOUR_HOURS, 10),
parseInt(cache_seconds || CONSTANTS.CARD_CACHE_SECONDS, 10),
CONSTANTS.FOUR_HOURS,
CONSTANTS.ONE_DAY,
);
Expand Down Expand Up @@ -83,7 +83,12 @@ export default async (req, res) => {
}),
);
} catch (err) {
res.setHeader("Cache-Control", `no-cache, no-store, must-revalidate`); // Don't cache error responses.
res.setHeader(
"Cache-Control",
`max-age=${CONSTANTS.ERROR_CACHE_SECONDS / 2}, s-maxage=${
CONSTANTS.ERROR_CACHE_SECONDS
}, stale-while-revalidate=${CONSTANTS.ONE_DAY}`,
); // Use lower cache period for errors.
return res.send(renderError(err.message, err.secondaryMessage));
}
};
9 changes: 7 additions & 2 deletions api/top-langs.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ export default async (req, res) => {
);

let cacheSeconds = clampValue(
parseInt(cache_seconds || CONSTANTS.FOUR_HOURS, 10),
parseInt(cache_seconds || CONSTANTS.CARD_CACHE_SECONDS, 10),
CONSTANTS.FOUR_HOURS,
CONSTANTS.ONE_DAY,
);
Expand Down Expand Up @@ -99,7 +99,12 @@ export default async (req, res) => {
}),
);
} catch (err) {
res.setHeader("Cache-Control", `no-cache, no-store, must-revalidate`); // Don't cache error responses.
res.setHeader(
"Cache-Control",
`max-age=${CONSTANTS.ERROR_CACHE_SECONDS / 2}, s-maxage=${
CONSTANTS.ERROR_CACHE_SECONDS
}, stale-while-revalidate=${CONSTANTS.ONE_DAY}`,
); // Use lower cache period for errors.
return res.send(renderError(err.message, err.secondaryMessage));
}
};
13 changes: 7 additions & 6 deletions api/wakatime.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,18 +42,14 @@ export default async (req, res) => {
const stats = await fetchWakatimeStats({ username, api_domain });

let cacheSeconds = clampValue(
parseInt(cache_seconds || CONSTANTS.FOUR_HOURS, 10),
parseInt(cache_seconds || CONSTANTS.CARD_CACHE_SECONDS, 10),
CONSTANTS.FOUR_HOURS,
CONSTANTS.ONE_DAY,
);
cacheSeconds = process.env.CACHE_SECONDS
? parseInt(process.env.CACHE_SECONDS, 10) || cacheSeconds
: cacheSeconds;

if (!cache_seconds) {
cacheSeconds = CONSTANTS.FOUR_HOURS;
}

res.setHeader(
"Cache-Control",
`max-age=${
Expand Down Expand Up @@ -82,7 +78,12 @@ export default async (req, res) => {
}),
);
} catch (err) {
res.setHeader("Cache-Control", `no-cache, no-store, must-revalidate`); // Don't cache error responses.
res.setHeader(
"Cache-Control",
`max-age=${CONSTANTS.ERROR_CACHE_SECONDS / 2}, s-maxage=${
CONSTANTS.ERROR_CACHE_SECONDS
}, stale-while-revalidate=${CONSTANTS.ONE_DAY}`,
); // Use lower cache period for errors.
return res.send(renderError(err.message, err.secondaryMessage));
}
};
2 changes: 1 addition & 1 deletion src/common/retryer.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,5 +72,5 @@ const retryer = async (fetcher, variables, retries = 0) => {
}
};

export { retryer };
export { retryer, RETRIES };
export default retryer;
10 changes: 10 additions & 0 deletions src/common/utils.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -373,11 +373,21 @@ const noop = () => {};
const logger =
process.env.NODE_ENV !== "test" ? console : { log: noop, error: noop };

// Cache settings.
const CARD_CACHE_SECONDS = 14400;
const ERROR_CACHE_SECONDS = 600;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's update this to 30min? Currently it's 10. And if PATs are exhausted that means Vercel will serve fresh responses every 10 minutes, but PATs will only restore after 30min or 1hour.

Changing it to 30min will reduce the load on vercel.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anuraghazra GitHub does restore the individual PATs after 1 hour. However, the frequency at which PATs are refreshed depends on when the PAT was depleted.

image

As a result, the 10 minutes could catch a refreshed token. However, if you think the load of Vercel should be decreased, I am happy to increase it to 1800.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay i think we can keep it as is, load on Vercel is not a huge deal right now.


const CONSTANTS = {
ONE_MINUTE: 60,
FIVE_MINUTES: 300,
TEN_MINUTES: 600,
FIFTEEN_MINUTES: 900,
THIRTY_MINUTES: 1800,
TWO_HOURS: 7200,
FOUR_HOURS: 14400,
ONE_DAY: 86400,
CARD_CACHE_SECONDS,
ERROR_CACHE_SECONDS,
};

const SECONDARY_ERROR_MESSAGES = {
Expand Down
9 changes: 7 additions & 2 deletions tests/api.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,13 +184,18 @@ describe("Test /api/", () => {
]);
});

it("should not store cache when error", async () => {
it("should set shorter cache when error", async () => {
const { req, res } = faker({}, error);
await api(req, res);

expect(res.setHeader.mock.calls).toEqual([
["Content-Type", "image/svg+xml"],
["Cache-Control", `no-cache, no-store, must-revalidate`],
[
"Cache-Control",
`max-age=${CONSTANTS.ERROR_CACHE_SECONDS / 2}, s-maxage=${
CONSTANTS.ERROR_CACHE_SECONDS
}, stale-while-revalidate=${CONSTANTS.ONE_DAY}`,
],
]);
});

Expand Down
4 changes: 2 additions & 2 deletions tests/retryer.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { jest } from "@jest/globals";
import "@testing-library/jest-dom";
import { retryer } from "../src/common/retryer.js";
import { retryer, RETRIES } from "../src/common/retryer.js";
import { logger } from "../src/common/utils.js";
import { expect, it, describe } from "@jest/globals";

Expand Down Expand Up @@ -44,7 +44,7 @@ describe("Test Retryer", () => {
try {
await retryer(fetcherFail, {});
} catch (err) {
expect(fetcherFail).toBeCalledTimes(8);
expect(fetcherFail).toBeCalledTimes(RETRIES + 1);
expect(err.message).toBe("Downtime due to GitHub API rate limiting");
}
});
Expand Down