Skip to content

Commit

Permalink
Balancing (#107)
Browse files Browse the repository at this point in the history
* Balancing integration
* Signals handling SIGHUP triggers configuration reload, SIGINT gracefully shutsdown server
  • Loading branch information
mjarco authored Jan 9, 2019
1 parent 6297750 commit 2e6aaa9
Show file tree
Hide file tree
Showing 76 changed files with 2,784 additions and 2,177 deletions.
35 changes: 7 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
Akubra is a simple solution to keep an independent S3 storages in sync - almost

This comment has been minimized.

Copy link
@wookie41

wookie41 Jan 9, 2019

Contributor

-> Akubra is a simple solution to keep independent S3 storages in sync
an is invalid with plural

realtime, eventually consistent.

Keeping synchronized storage clusters, which handles great volume of new objects
(about 300k obj/h), is the most efficient by feeding them with all incoming data
Keeping synchronized storage clusters, which handles great volume of new objects,

This comment has been minimized.

Copy link
@wookie41

wookie41 Jan 9, 2019

Contributor

Keeping storage clusters which handle great volume of new object synchronized can be most efficiently achieved by feeding them all at once with the incoming data

is the most efficient by feeding them with all incoming data
at once. That's what Akubra does, with a minimum memory and cpu footprint.

Synchronizing S3 storages offline is almost impossible with a high volume traffic.
Expand All @@ -37,6 +37,11 @@ It also backtracks to older cluster when requested for not existing object on
target cluster. This kind of events are logged, so it's possible to rebalance
clusters in background.

### Multi cloud cost optimization

This comment has been minimized.

Copy link
@wookie41

wookie41 Jan 9, 2019

Contributor

While all objects have to be written to every storage in a shard, not all storages have ... will pick the cheapest ...

While all objects has to be written in every storage from shard, not all storages
has to be read. With load balancing and storage prioritization akubra will peak
cheapest one.

## Build

### Prerequisites
Expand Down Expand Up @@ -153,32 +158,6 @@ Service:
IdleConnTimeout: 2s
ResponseHeaderTimeout: 2s

# MaxIdleConns see: https://golang.org/pkg/net/http/#Transport
# Default 0 (no limit)
MaxIdleConns: 0
# MaxIdleConnsPerHost see: https://golang.org/pkg/net/http/#Transport
# Default 100
MaxIdleConnsPerHost: 100
# IdleConnTimeout see: https://golang.org/pkg/net/http/#Transport
# Default 0 (no limit)
IdleConnTimeout: 0s
# ResponseHeaderTimeout see: https://golang.org/pkg/net/http/#Transport
# Default 5s
ResponseHeaderTimeout: 5s
# DisableKeepAlives see: https://golang.org/pkg/net/http/#Transport
# Default false

DisableKeepAlives: false

# Maximum accepted body size
BodyMaxSize: "100M"
# Maximum number of incoming requests to process at once
MaxConcurrentRequests: 200
# Backend in maintenance mode. Akubra will skip this endpoint

# MaintainedBackends:
# - "http://s3.dc2.internal"

# List request methods to be logged in synclog in case of backend failure
SyncLogMethods:
- PUT
Expand Down
Loading

0 comments on commit 2e6aaa9

Please sign in to comment.