Merge pull request #307 from jordilin/autothrottle

Implement auto throttling

.gitignore

@@ -12,3 +12,4 @@ result
 contracts/__pycache__
 lcov.info
 /doc/book
+.description

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "git-ar"
-version = "1.0.4"
+version = "1.0.5"
 edition = "2021"
 
 license = "MIT"

doc/src/SUMMARY.md

@@ -5,6 +5,7 @@
   - [Installation](./installation.md)
   - [Configuration](./configuration.md)
   - [Caching](./caching.md)
+  - [Listing data](./listing.md)
 - [Gitar commands](./cmds/index.md)
   - [Merge requests](./cmds/merge_request.md)
   - [Pipelines](./cmds/pipeline.md)

doc/src/getting_started.md

@@ -5,3 +5,5 @@ a configuration file. Let's get started:
 
 - [Installation](./installation.md)
 - [Configuration](./configuration.md)
+- [Caching](./caching.md)
+- [Listing data](./listing.md)

doc/src/listing.md

@@ -0,0 +1,21 @@
+# The list subcommand
+
+<!-- toc -->
+
+The list subcommand is used to pull data from specific resources such as
+pipelines and merge requests. Gitar implements best practices to avoid being
+rate limited, caches responses and uses pagination to pull the required data
+using the `--from-page` and `--to-page` flags.
+
+## Auto throttling
+
+Gitar will automatically throttle the requests after three consecutive HTTP
+calls have been made. The throttling is based on the rate limit headers plus a
+jitter interval between 1 and 5 seconds. The user can also specify a fixed
+throttle interval with `--throttle` or a random one with `--throttle-range`.
+
+## Max pages to fetch
+
+If no configuration is provided, the default is a max of 10 pages. This can be
+overriden with `--to-page` where it will fetch up to the specified page or a
+range of pages with `--from-page` and `--to-page`.

src/api_defaults.rs

@@ -18,3 +18,10 @@ pub const DEFAULT_NUMBER_REQUESTS_MINUTE: u32 = 80;
 pub const DEFAULT_PER_PAGE: u32 = 30;
 
 pub const EXPIRE_IMMEDIATELY: &str = "0s";
+
+// Default jitter values for autorate throttling.
+pub const DEFAULT_JITTER_MAX_MILLISECONDS: u64 = 5000;
+pub const DEFAULT_JITTER_MIN_MILLISECONDS: u64 = 1000;
+
+// Trigger autorate throttling after 3 API calls.
+pub const ENGAGE_AUTORATE_THROTTLING_THRESHOLD: u32 = 3;