Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Balancing #107

Merged
merged 28 commits into from
Jan 9, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 7 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
Akubra is a simple solution to keep an independent S3 storages in sync - almost
realtime, eventually consistent.

Keeping synchronized storage clusters, which handles great volume of new objects
(about 300k obj/h), is the most efficient by feeding them with all incoming data
Keeping synchronized storage clusters, which handles great volume of new objects,
is the most efficient by feeding them with all incoming data
at once. That's what Akubra does, with a minimum memory and cpu footprint.

Synchronizing S3 storages offline is almost impossible with a high volume traffic.
Expand All @@ -37,6 +37,11 @@ It also backtracks to older cluster when requested for not existing object on
target cluster. This kind of events are logged, so it's possible to rebalance
clusters in background.

### Multi cloud cost optimization
While all objects has to be written in every storage from shard, not all storages
has to be read. With load balancing and storage prioritization akubra will peak
Copy link
Contributor

@wookie41 wookie41 Nov 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pick the

cheapest one.

## Build

### Prerequisites
Expand Down Expand Up @@ -153,32 +158,6 @@ Service:
IdleConnTimeout: 2s
ResponseHeaderTimeout: 2s

# MaxIdleConns see: https://golang.org/pkg/net/http/#Transport
# Default 0 (no limit)
MaxIdleConns: 0
# MaxIdleConnsPerHost see: https://golang.org/pkg/net/http/#Transport
# Default 100
MaxIdleConnsPerHost: 100
# IdleConnTimeout see: https://golang.org/pkg/net/http/#Transport
# Default 0 (no limit)
IdleConnTimeout: 0s
# ResponseHeaderTimeout see: https://golang.org/pkg/net/http/#Transport
# Default 5s
ResponseHeaderTimeout: 5s
# DisableKeepAlives see: https://golang.org/pkg/net/http/#Transport
# Default false

DisableKeepAlives: false

# Maximum accepted body size
BodyMaxSize: "100M"
# Maximum number of incoming requests to process at once
MaxConcurrentRequests: 200
# Backend in maintenance mode. Akubra will skip this endpoint

# MaintainedBackends:
# - "http://s3.dc2.internal"

# List request methods to be logged in synclog in case of backend failure
SyncLogMethods:
- PUT
Expand Down
Loading