Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Xray not capturing stack traces in latest version of Flask #269

Open
ianmetcalf opened this issue Jan 20, 2021 · 2 comments
Open

Xray not capturing stack traces in latest version of Flask #269

ianmetcalf opened this issue Jan 20, 2021 · 2 comments

Comments

@ianmetcalf
Copy link

The after_request handler is closing the segment before the teardown_request handler is called so the stack trace isn't captured.

It looks like this behavior changed as of Flask v1.1.0 (specifically pallets/flask#3266)

Currently using:

  • aws-xray-sdk v2.6.0
  • flask v1.1.2

Here is our current work around:

app = Flask(__name__)

def wrap_exception_handler(handler):
    def wrapped_function(exp):
        if not isinstance(exp, HTTPException) or exp.code >= 500:
            try:
                stack = stacktrace.get_stacktrace(
                    limit=xray_recorder.max_trace_back,
                )

                xray_recorder.current_segment().add_exception(exp, stack)
            except Exception:
                app.logger.error('Failed to add stack trace to xray segment')

        return handler(exp)
    return wrapped_function

app.handle_user_exception = wrap_exception_handler(app.handle_user_exception)
@srprash
Copy link
Contributor

srprash commented Jan 29, 2021

Hi @ianmetcalf
Thanks for identifying the issue and the root cause. You're right that the _handle_exception method which is the teardown_request handler is called after the segment is closed in _after_request and is failing silently while trying to add the exception.
I believe there could be two options here.

  1. Do not close the segment in _after_request if the response.status_code is 500. Then the _handle_exception method will add the exception and close the segment.
  2. Add a handle_user_exception handler method to the app similar to your workaround and add the exception there.

I'll test both the options and put out a fix soon.

@srprash
Copy link
Contributor

srprash commented Feb 1, 2021

Did some digging into why the unit tests didn't catch this behavior change. It turns out that if we enable the testing mode, the after_request is not called in case of an unhandled exception which results in segment being still available in the handle_exception method and the test passes.
When I disabled the test mode by setting app.config['TESTING'] = False, the test began to fail as expected. I think for our use case, we can keep disable the TESTING config as disabled since it is required only to propagate the exception to the test_client which we don't really need.

I think the simple fix can the option 1 mentioned above rather than setting the handle_user_exception becuase that could override the customer's application exception handling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants