Skip to content

log drain from heroku to s3. grep by line prefix and buffer upload to s3 as timestamped files

Notifications You must be signed in to change notification settings

nice-registry/heroku-log-s3

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Heroku Log S3

Deploy

Configure

Setup the following ENV (aka heroku config:set)

  • FILTER_PREFIX this is the prefix string to look out for. every other log lines are ignored
  • S3_KEY, S3_SECRET, S3_BUCKET necessary ACL to your s3 bucket
  • DURATION (default 60) seconds to buffer until we close the IO to AWS::S3::S3Object#write
  • STRFTIME (default %Y%m/%d/%H/%M%S.:thread_id.log) format of your s3 object_id
    • :thread_id will be replaced by a unique number to prevent overwriting of the same file between reboots, in case the timestamp overlaps
  • HTTP_USER, HTTP_PASSWORD (default no password protection) credentials for HTTP Basic Authentication
  • WRITER_LIB (default ./writer/s3.rb) defines the ruby script to load Writer class

Using

In your heroku app, add this drain (changing HTTP_USER, HTTP_PASSWORD and DRAIN_APP_NAME to appropriate values)

heroku drains:add https://HTTP_USER:HTTP_PASSWORD@DRAIN_APP_NAME.herokuapp.com/

or if you have no password protection

heroku drains:add https://DRAIN_APP_NAME.herokuapp.com/

Credits

About

log drain from heroku to s3. grep by line prefix and buffer upload to s3 as timestamped files

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Ruby 100.0%