Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trim nested arrays and drop metadata to save on payload size #295

Merged
merged 3 commits into from
May 11, 2016

Conversation

kattrali
Copy link
Contributor

@kattrali kattrali commented May 11, 2016

When a payload is too large, the request is rejected by the Bugsnag API. This changeset adds a few more last resort adjustments to error reports to ensure at least something is sent.

If a payload is too large to be sent, the following steps are taken, and then only if the previous step did not put the payload size under the threshold:

  1. Trim strings. All strings over 3kb are truncated. This still leaves enough room for the opening paragraphs of War and Peace.
  2. Truncate arrays. All arrays are truncated to having 40 items or less. If an array of 40 items has maximum length strings, ~5kb remains for the remainder of the payload
  3. Drop metadata. Custom metadata attached to the report is dropped. This should ensure that a report is at least generated, even if there is less available to assist with debugging.

If a payload contains an array of max length strings (4kb), trimming to
40 lines leaves 5kb for the rest of the payload including a long message
and API key, et cetera. The only concern is losing debugging information
to a smarter heuristic than truncating the lines outside the allowed
length.
else
value
end
end

def self.remove_metadata_from_value(object)
case object
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be clearer and faster if it were more explicit.

payload.events.map do |event|
  event[:metaData] = {}
end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense.

In the (extreme) case that trimming strings and array truncation does
not result in a payload of appropriate size, remove the `metaData` key
from an array of events. The error will at least be reported, even if 
custom metadata is unavailable to assist with debugging.
@kattrali kattrali force-pushed the kattrali/trim-nested-arrays branch from 969be8e to f338ff1 Compare May 11, 2016 21:26
@kattrali kattrali merged commit bd68f5e into master May 11, 2016
@kattrali kattrali deleted the kattrali/trim-nested-arrays branch May 11, 2016 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants