Adds honeybadger support. #10
Conversation
wilkie
force-pushed
the
support-honeybadger
branch
from
7e1a2a9 to
4de4d0a
Compare
The Dockerfile is required to be in a non-slim image because honeybadger introduces a dependency that is natively built. We will likely run into these things, anyway. Adds the `honeybadger` Python library to `requirements.txt` New configuration option via env to provide the honeybadger project key within the `HONEYBADGER_API_KEY` variable. Sets it up to log all exceptions that are unhandled within the app. Sets it up to log all CRITICAL logs that happen in the app. Sets it up to log all ERROR logs (that aren't exceptions) in the app.
wilkie
force-pushed
the
support-honeybadger
branch
from
4de4d0a to
27940fa
Compare
| * In production: `production/aiproxy.code.org/honeybadger_api_key` | ||
| * In test: `test/aiproxy-test.code.org/honeybadger_api_key` | ||
| * In an "adhoc" development environment: `development/aiproxy-dev-mybranch.code.org/honeybadger_api_key` | ||
| * A prod-like environment, not based on `main`: `production/aiproxy-otherbranch.code.org/honeybadger_api_key` |
There was a problem hiding this comment.
I'm making a decision on naming convention here, open to feedback. We are somewhat limited by our IAM permissions strategy.
|
|
||
| To create a new secret, define it in "[cicd/3-app/aiproxy/template.yml](cicd/3-app/aiproxy/template.yml) in the "Secrets" section. This will create an empty secret once deployed. | ||
|
|
||
| Once created, you can set the value of this secret (via the AWS Console, as an Admin). Finally, deploy your code that uses the new secret, loading it via the `GetSecretValue` function from the AWS SDK. |
There was a problem hiding this comment.
This admittedly isn't the best Dev experience, we could implement a similar script to bin/update_secrets over in the code-dot-org repo if we want to make this easier. Keeping it simple for MVP.
| # Log exceptions to Honeybadger | ||
| app.config['HONEYBADGER_API_KEY'] = honeybadger_api_key | ||
| app.config['HONEYBADGER_PARAMS_FILTERS'] = 'password, secret, openai_api_key, api_key' | ||
| app.config['HONEYBADGER_ENVIRONMENT'] = os.getenv('FLASK_ENV') |
There was a problem hiding this comment.
I don't think this is what we want here. I think we want either os.getenv('ENVIRONMENT') or os.getenv('ENVIRONMENT_TYPE'), but I'm not certain how the HONEYBADGER_ENVIRONMENT is used, or whether we should instead be trying to set FLASK_ENV = ENVIRONMENT_TYPE
For reference:
- ENVIRONMENT is something like aiproxy.code.org or aiproxy-test.code.org
- ENVIRONMENT_TYPE is one of
development,test, orproduction
Neither are set in local environments, however.
There was a problem hiding this comment.
I propose, then, os.getenv('ENVIRONMENT', os.getenv('FLASK_ENV', 'production'))... it is just for the purposes of Honeybadger knowing the environment for its own reporting, I believe. Optional, but maybe useful. I think we would manage grouping errors by environment with separate api keys.
There was a problem hiding this comment.
Or maybe it is ENVIRONMENT_TYPE not ENVIRONMENT in my proposal. either way.
|
The changes to add the AWS secret integration look good. Really simple! Seems to work locally using Pending concerns:
|
| @@ -5,6 +5,9 @@ Description: Provision an instance of the AI Proxy service. | |||
| # Dependencies: This template has dependencies, look for !ImportValue in the Resources section. | |||
|
|
|||
| Parameters: | |||
| EnvironmentType: | |||
| Type: String | |||
| Description: Environment type (e.g. 'development', 'staging', 'test', 'prod'). | |||
There was a problem hiding this comment.
is this indicative of the environment types we plan to support for aiproxy?
There was a problem hiding this comment.
Moreso just examples to provide context. Right now we have a production environment and a non-production environment (named 'test'). We can definitely add another environment ('staging') if we would like to maintain a horizontal slice between code-dot-org and aiproxy, or we could simply point all non-prod environments of code-dot-org at the 'aiproxy-test.code.org' environment.
The Dockerfile is required to be in a non-slim image because honeybadger introduces a dependency that is natively built. We will likely run into these things, anyway.
Adds the
honeybadgerPython library torequirements.txtNew configuration option via env to provide the honeybadger project key within the
HONEYBADGER_API_KEYvariable.If the local
HONEYBADGER_API_KEYvariable is empty, it will check AWS Secrets ManagerSets it up to log all exceptions that are unhandled within the app.
Sets it up to log all CRITICAL logs that happen in the app.
Sets it up to log all ERROR logs (that aren't exceptions) in the app.
Here is a view of the Honeybadger reports:
The first is any time
logging.criticalhappens.The second is any time
logging.errorhappens.The third is for a general
Exceptionbeing raised and unhandled (which would mean a500error)Digging in, here is the backtrace it provides for an Exception: