Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for setting a larger request body for some endpoints #618

Open
wants to merge 3 commits into
base: sunshowers/spr/main.add-support-for-setting-a-larger-request-body-for-some-endpoints
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions dropshot/src/api_description.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,15 @@ use std::collections::HashSet;
/// provided explicitly--as well as parameters and a description which can be
/// inferred from function parameter types and doc comments (respectively).
#[derive(Debug)]
#[non_exhaustive]
pub struct ApiEndpoint<Context: ServerContext> {
pub operation_id: String,
pub handler: Box<dyn RouteHandler<Context>>,
pub method: Method,
pub path: String,
pub parameters: Vec<ApiEndpointParameter>,
pub body_content_type: ApiEndpointBodyContentType,
pub request_body_max_bytes: Option<usize>,
pub response: ApiEndpointResponse,
pub summary: Option<String>,
pub description: Option<String>,
Expand Down Expand Up @@ -72,6 +74,7 @@ impl<'a, Context: ServerContext> ApiEndpoint<Context> {
path: path.to_string(),
parameters: func_parameters.parameters,
body_content_type,
request_body_max_bytes: None,
response,
summary: None,
description: None,
Expand All @@ -92,6 +95,11 @@ impl<'a, Context: ServerContext> ApiEndpoint<Context> {
self
}

pub fn request_body_max_bytes(mut self, max_bytes: usize) -> Self {
self.request_body_max_bytes = Some(max_bytes);
self
}

pub fn tag<T: ToString>(mut self, tag: T) -> Self {
self.tags.push(tag.to_string());
self
Expand Down
15 changes: 5 additions & 10 deletions dropshot/src/extractor/body.rs
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,8 @@ async fn http_request_load_body<Context: ServerContext, BodyType>(
where
BodyType: JsonSchema + DeserializeOwned + Send + Sync,
{
let server = &rqctx.server;
let (parts, body) = request.into_parts();
let body = StreamingBody::new(body, server.config.request_body_max_bytes)
let body = StreamingBody::new(body, rqctx.request_body_max_bytes)
.into_bytes_mut()
.await?;

Expand Down Expand Up @@ -191,12 +190,10 @@ impl ExclusiveExtractor for UntypedBody {
rqctx: &RequestContext<Context>,
request: hyper::Request<hyper::Body>,
) -> Result<UntypedBody, HttpError> {
let server = &rqctx.server;
let body = request.into_body();
let body_bytes =
StreamingBody::new(body, server.config.request_body_max_bytes)
.into_bytes_mut()
.await?;
let body_bytes = StreamingBody::new(body, rqctx.request_body_max_bytes)
.into_bytes_mut()
.await?;
Ok(UntypedBody { content: body_bytes.freeze() })
}

Expand Down Expand Up @@ -396,11 +393,9 @@ impl ExclusiveExtractor for StreamingBody {
rqctx: &RequestContext<Context>,
request: hyper::Request<hyper::Body>,
) -> Result<Self, HttpError> {
let server = &rqctx.server;

Ok(Self {
body: request.into_body(),
cap: server.config.request_body_max_bytes,
cap: rqctx.request_body_max_bytes,
})
}

Expand Down
5 changes: 5 additions & 0 deletions dropshot/src/handler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,18 @@ pub type HttpHandlerResult = Result<Response<Body>, HttpError>;

/// Handle for various interfaces useful during request processing.
#[derive(Debug)]
#[non_exhaustive]
pub struct RequestContext<Context: ServerContext> {
/// shared server state
pub server: Arc<DropshotState<Context>>,
/// HTTP request routing variables
pub path_variables: VariableSet,
/// expected request body mime type
pub body_content_type: ApiEndpointBodyContentType,
/// Maximum request body size: typically the same as
/// [`server.config.request_body_max_bytes`], but can be overridden for an
/// individual request
pub request_body_max_bytes: usize,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about making this a method instead? This feels pretty similar to RequestContext::page_limit(). In both cases we want to provide the "appropriate" value for some tunable based on some non-trivial policy related to multiple places it might be set.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I think we still need to store the value in the RequestContext somewhere -- what about storing it in RequestInfo? Then this could be a method rqctx.request.body_max_bytes(), or rqctx.request.request_body_max_bytes().

Or do you have a way to avoid storing the value on the request? I couldn't find an easy answer to this.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I should have been clearer -- I'm fine with storing it there. I'm suggesting it not be pub and instead provide a method to access it (that takes into account the various caps).

/// unique id assigned to this request
pub request_id: String,
/// logger for this specific request
Expand Down
3 changes: 3 additions & 0 deletions dropshot/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,9 @@
//!
//! // Optional fields
//! tags = [ "all", "your", "OpenAPI", "tags" ],
//! // An optional limit for the request body size that overrides the
//! // default server config.
//! request_body_max_bytes = 1048576,
//! }]
//! ```
//!
Expand Down
4 changes: 4 additions & 0 deletions dropshot/src/router.rs
Original file line number Diff line number Diff line change
Expand Up @@ -207,10 +207,12 @@ impl MapValue for VariableValue {
/// corresponding values in the actual path, and the expected body
/// content type.
#[derive(Debug)]
#[non_exhaustive]
pub struct RouterLookupResult<'a, Context: ServerContext> {
pub handler: &'a dyn RouteHandler<Context>,
pub variables: VariableSet,
pub body_content_type: ApiEndpointBodyContentType,
pub request_body_max_bytes: Option<usize>,
}

impl<Context: ServerContext> HttpRouterNode<Context> {
Expand Down Expand Up @@ -483,6 +485,7 @@ impl<Context: ServerContext> HttpRouter<Context> {
handler: &*handler.handler,
variables,
body_content_type: handler.body_content_type.clone(),
request_body_max_bytes: handler.request_body_max_bytes,
})
.ok_or_else(|| {
HttpError::for_status(None, StatusCode::METHOD_NOT_ALLOWED)
Expand Down Expand Up @@ -766,6 +769,7 @@ mod test {
parameters: vec![],
body_content_type: ApiEndpointBodyContentType::default(),
response: ApiEndpointResponse::default(),
request_body_max_bytes: None,
summary: None,
description: None,
tags: vec![],
Expand Down
3 changes: 3 additions & 0 deletions dropshot/src/server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -789,6 +789,9 @@ async fn http_request_handle<C: ServerContext>(
request: RequestInfo::from(&request),
path_variables: lookup_result.variables,
body_content_type: lookup_result.body_content_type,
request_body_max_bytes: lookup_result
.request_body_max_bytes
.unwrap_or(server.config.request_body_max_bytes),
request_id: request_id.to_string(),
log: request_log,
};
Expand Down
1 change: 1 addition & 0 deletions dropshot/src/websocket.rs
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,7 @@ mod tests {
request: RequestInfo::from(&request),
path_variables: Default::default(),
body_content_type: Default::default(),
request_body_max_bytes: 0,
request_id: "".to_string(),
log: log.clone(),
};
Expand Down
22 changes: 22 additions & 0 deletions dropshot/tests/fail/bad_endpoint20.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
// Copyright 2023 Oxide Computer Company

#![allow(unused_imports)]

use dropshot::endpoint;
use dropshot::HttpError;
use dropshot::HttpResponseOk;
use dropshot::UntypedBody;

#[endpoint {
method = GET,
path = "/test",
request_body_max_bytes = true,
}]
async fn bad_request_body_max_bytes(
_rqctx: RequestContext<()>,
param: UntypedBody,
) -> Result<HttpResponseOk<()>, HttpError> {
Ok(HttpResponseOk(()))
}

fn main() {}
5 changes: 5 additions & 0 deletions dropshot/tests/fail/bad_endpoint20.stderr
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
error: expected u64, but found `true`
--> tests/fail/bad_endpoint20.rs:13:30
|
13 | request_body_max_bytes = true,
| ^^^^
26 changes: 26 additions & 0 deletions dropshot/tests/fail/bad_endpoint21.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
// Copyright 2023 Oxide Computer Company

// This does not currently work, but we may want to support it in the future.

#![allow(unused_imports)]

use dropshot::endpoint;
use dropshot::HttpError;
use dropshot::HttpResponseOk;
use dropshot::UntypedBody;

const MAX_REQUEST_BYTES: u64 = 400_000;

#[endpoint {
method = GET,
path = "/test",
request_body_max_bytes = MAX_REQUEST_BYTES,
}]
async fn bad_request_body_max_bytes(
_rqctx: RequestContext<()>,
param: UntypedBody,
) -> Result<HttpResponseOk<()>, HttpError> {
Ok(HttpResponseOk(()))
}

fn main() {}
5 changes: 5 additions & 0 deletions dropshot/tests/fail/bad_endpoint21.stderr
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
error: expected u64, but found `MAX_REQUEST_BYTES`
--> tests/fail/bad_endpoint21.rs:17:30
|
17 | request_body_max_bytes = MAX_REQUEST_BYTES,
| ^^^^^^^^^^^^^^^^^
Loading