Nonexistent backend error #66
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…to rewrite-config Conflicts: main.go sharding/sharding_test.go
Conflicts: main.go
…o listing-merge
…tent-backend-error
ksitak
reviewed
Dec 5, 2017
httphandler/response_merger.go
Outdated
| if ok { | ||
| return berr.Backend() | ||
| } | ||
| log.Printf("Requested backend is not retrivable from tuple %#v", r) |
ksitak
reviewed
Dec 5, 2017
| res := &http.Response{Request: req, StatusCode: respStatus, ContentLength: int64(len(body))} | ||
| return transport.ResErrTuple{Req: req, Res: res, Err: err, Failed: failed}, nil | ||
| } | ||
| func mkSuccessfulResTuples(method, urlStr string, count int) ([]transport.ResErrTuple, error) { |
ksitak
reviewed
Dec 5, 2017
httphandler/httphandler_test.go
Outdated
| return tuples, nil | ||
| } | ||
|
|
||
| func mkFailedResultTuples(method, url string, count int) ([]transport.ResErrTuple, error) { |
ksitak
reviewed
Dec 5, 2017
httphandler/response_merger.go
Outdated
| return req.Context().Value(log.ContextreqIDKey).(string) | ||
| } | ||
|
|
||
| func backend(r transport.ResErrTuple) string { |
There was a problem hiding this comment.
this function name does not tell much about what this function does
mjarco
added a commit
to mjarco/akubra
that referenced
this pull request
Aug 13, 2018
* Configuration parse fix (#1) Fixed configuration parsing and MakeFile clean up * Removed unused script and heightened linters timeout * README and glide versions cleaning * Fixed typos in README, added badges for docs and releases * Application version compiled in binary * Access log outputs data in JSON format Access log outputs in json format. Added OptionsHandler decorator to chain * fixing glide requirement in Makefile * Fixing log.Fatalf when port already in use (allegro#17) * log.Fatalf(err) if port already in use * Sharding and more (allegro#40) * Fixed configuration parsing * Simple sharding (allegro#9) Introduced simple sharding strategy: Each request is mapped to `cluster` via consistent hash ring. Cluster configuration points backends (host:port) where files should be replicated. All loging and replication strategies for single cluster works like global backends in previous stable version. Behavioral changes to request handling on cluster picking layer - Regression calls: if object is not present on computed cluster check cluster with lower index on client cluster list - PUTs are not passed to regression cluster - DELETE is passed to all backends no matter which cluster should contain object For error handling and data migration purposes logs object identifier if regression cluster was called Introduced examples folder with akubra.config.dist example * Multiple maintained backends in configuration (allegro#15) * Configurable logs implements issue allegro#16 (allegro#19) * Fixed: does not close connection when backends refuses connection * Fixed: clusters weights sum equal 0 causes panic * Removed global backends configuration, updated (allegro#21) example examples/akubra.config.dist * Introduced level option in logger configuration (allegro#22) * Fixed example configuration in README (allegro#23) * Remove custom dialer (allegro#25) * Wait for all backends if performing bucket operation * Replaced custom dialer with build in one * Maxbodysize (allegro#26) limit max body size * Content length to synclog (allegro#27) Added Content-Length to synclog * Fixed: connections leak if cluster recursion occured (allegro#28) * Different ring engine (allegro#29) * changed shard ring engine * Runtime metrics and monitoring resolve allegro#30 (allegro#31) * Fix concurrent map access and limit TIME_WAIT connections (allegro#33) * Fixed: keep-alive connections reuse, replaced content-length header read with http.Request.ContentLength resolves allegro#32 * Replaced hostname with fqdn in default metrics path * Metrics include percentiles (allegro#35) * Add percentiles for timer metrics * Additional metrics * Akubra config validation (allegro#36) Added parameter for YAML configuration validation * Validate config technical HTTP endpoint (allegro#38) * Added TechnicalEndpointListen (required parameter) to example config * SQL logger hook (allegro#41) * Expose ShardsRing API (allegro#42) * Use netcgo (allegro#43) * Use netcgo * FIXED: AdditionalHeaders yaml unmarshaller (allegro#44) * Fixed: not empty body while content-length == 0 (allegro#45) * timeouts to prevent infinite requests * goimports * incoming concurrent request count limit (allegro#47) * Remove defer in error check. (allegro#48) `defer` is redundant if it's last call in a function. * Regions support (allegro#50) * Fixed: client sends nil body and no Content-Length * Regions validation bugfix (allegro#52) * weights must be float * Health check endpoint (allegro#55) * Mark region as default (allegro#56) * Mark region as default * Changed logrus import, fixed base type context key * MultiTransport accepts roundtrippers instead urls * New storages definitions * Maybe I should go slower on this * crdstore remote credentials package prototype * HTTPHandler config * crdstore remote credentials package prototype v2 * [broken] rewiring application start up * wip * wip2 * wip3 * add background refresh * fix concurrency * S3 signing works * S3 signing works * Test fixes * sharding_test - updating part1 * Add storage tests * akubra/config tests build * akubra/config tests build * akubra/config tests build * Config package validators fixes * upgrading unit tests for sharding * upgrading testify vendor library * upgrading akuba's unit tests (prototyping) * upgrading travis for CI with golang 1.9 * CI fixes * CI fixes * CI fixes * adding unit tests for types - CI fixing * adding unit tests for types - CI fixing * Add S3authservice backend type * Fix storages.JoinCluster + add storages tests * updating sharding_test + CI fixing * Fixes after merging with rewrite-config branch * Updating glide.lock * add logs + fixing handler url printing in logs * GitHub tag badge in Readme (allegro#61) This project does not have GitHub releases, so GitHub release badge does not work (first badge in Readme). We can use tag badge instead. * fix: preallocate maps * S3authservice to v1.0 (allegro#60) * Update S3AuthService * update glide * Change CacheTTL to metrics.Interval * Fixed synclog * Validators cleanup * Changed validation calls * Code style improvements (allegro#63) * Listing merge (allegro#64) * Objects list merging * Added: not implemented response on bucket listing v2 * maintenance-mode-for-backends (allegro#67) maintenance mode fixed * Nonexistent backend error (allegro#66) * Fixed: no panic on connection refused * Fixed: debug logs more meaningful * Slow lint (allegro#68) * Fixed: non-fast linting passes * fixing HumanSizeUnits (for 1MB == 1048576 bytes) * fixing doc. for technical endpoint path * Log fatal instead of panic if CredentialsStore is not defined * Added brimapi log hook (allegro#69) * Added brimapi loghook * Empty brim api response body - reuse connections stop leaking descriptors * Fixed: CLOSE_WAIT connections when listing non existing bucket * Server timeout settings moved to configuration * Fix: close successful listing responses body * Fixed: empty body and non-zero content-length header in response to non-existent bucket listing * add AWSv4 signature * Cluster ring search (allegro#71) * Ring regression * Fix bucket acl request handling (allegro#72) * FIXED: bucket acl responses are passed correctly * Fix late responses (allegro#73) * Fixed: waiting for replica t/o instead of earliest success respond * transports behaviours configuration proposal * transports behaviours configuration regexp parser + unit tests * fixing HumanSizeUnits (for 1MB == 1048576 bytes) * fixing doc. for technical endpoint path * Added brimapi log hook (allegro#69) * Added brimapi loghook * Empty brim api response body - reuse connections stop leaking descriptors * Fixed: CLOSE_WAIT connections when listing non existing bucket * Server timeout settings moved to configuration * Fix: close successful listing responses body * Fixed: empty body and non-zero content-length header in response to non-existent bucket listing * some spaces in Authorization header are optional * Multipart upload support (allegro#74) Added multipart_handler with unit tests * OBRAZKI-574 | QueryParam field isn't required * updating GetMatchedTransport for Transports configuration + unit tests * Select RoundTripper (initialized with Transports config) by request properties for backends * removing unused RoudTrip transport arg. in NewRegions func. * logical validator for Transports required last item without Triggers definition * logical validator for Transports required last item without Triggers definition * updating TransportsEntryLogicalValidator validator + unit tests * fixing goimports * fixing CI * fixing CI * removing MergingStrategy from struct, config, README example * changing Transports structure from map to list for fixing CI * changing Transports structure from map to list for fixing CI * fixing for PR comments * fixing unit tests for backend * fixing unit tests for multipart round tripper * refactoring for PR suggestions * refactoring for PR suggestions * refactoring for PR suggestions * removing default transport (conceptual change) + adding YAML validator on akubra setuping * OBRAZKI-574 | changing Triggers -> Matchers section name (in Transports YAML config section) * using empty Matchers as default transport for any request in Transports matching process * refactoring transports container with RoundTripper * refactoring transports container with RoundTripper * refactoring struct - transports container to matcher * fixing CI - renaming struct * refactoring from Matcher to http.RoundTripper interface * Revert "refactoring from Matcher to http.RoundTripper interface" This reverts commit a55bb64. * refactoring Matcher * refactoring from Matcher to http.RoundTripper interface (+fixing unit tests) * CR fix - Readability counts * small refactoring + unit tests for transport matcher * CR fixes * CR fixes * Details in Transport config is required (adding validator + unit tests) * bug fixing in selecting transport after manual tests + updating unit tests * bug fixing in selecting transport after manual tests + updating unit tests * CR fixes - refactoring Transport: Details -> Properties * CR fixes - refactoring Transport: Details -> Properties * CR fixes - refactoring Transport: Matches -> ApplyRule * CR fixes - refactoring Transport: Matches -> ApplyRules * CR fixes - removing encapsulated name in struct * refactoring Transport: ApplyRules -> Rules * CR fix - refactoring compileRules method's condition * bug fix - removing IdleConnTimeout property from transports logical validator (zero value is possibe) * CR fix - remove default const. * CR fix - changing transport name in unit test * Fix listings (allegro#77) * Excluded several types of bucket requests from result merging * Added linux target for crosscompiling * CR fix - changing transport definition name in unit test * CR fixes * CR fixes - in README.md and updating validator + unit tests * CR fixes - typos * CR fixes - alternative message in logs for backend error when transport definition not maching * CI fixed - return with else WARN * CR fix - update exit code * CR fix - mute log message about transport not matcheing in backend log context * CR fix - renaming struct * CR fix - lang. in README * fixing problem with Makefile and upgrading version * fixing problem with Makefile and upgrading version * fixing problem with Makefile and upgrading version * fixing problem with Makefile and upgrading version * fixing problem with Makefile and upgrading version * fixing problem with Makefile commit (version comment with new line text) * More bucket requests handling (allegro#79) * Added s3 datatypes for /?versions and /?uploads requests * Moved s3 responses merging code to own package * ListV2 and Versions merge implemented * Fix: Additional headers configuration is reachable for HeadersSuplier (allegro#83) * Added: metrics for CLOSE_WAIT duration, Added: filter synclog read events when maintenance (allegro#86) * Fixed continuation tokens * sync tasks log by service discovered from consul * update README and glide.lock * update README and glide.lock * update README and glide.lock * reverting README and freezed vendors * reverting README and freezed vendors * updating code with locking * updating structs with syncmap.Map and locking methods * updating method name and removing go-cache from glide * adding timeout for HTTP consul client + updating code with goimports * adding integration tests for brimapi (sync tasks logger) + CR fixes * fixing Go version to 1.10.x in travis * CR fixes and refactoring * CR fixes and refactoring * CR fixes and refactoring + unit testing instead of integration testing * CI fixes * CR fixes * CR fix * Minimalize close waits (allegro#91) * New request-response handling internals. Fixed: excessive number of close_waits, half object copying operations * Leaking goroutines fix * Fixed context cancelation * Debug messages on response body close * More consistent logging and close wait fix * Fix: broken multipart * Fix synclog backend reporting * Added forced auth backends * Bucket location simplistic implementation * More verbose debug log. Added dial timeout * Exported multipart specific errors. Not fully drained response body will attempt to close * Add log on failed request to BrimApi * Revert log * Handle both V2 and V4 when extracting access key * Log success/error when adding sync task to main log instead of stdout * Mem leak2 (allegro#92) * Leaking goroutines fix * Fixed context cancelation * Debug messages on response body close * More consistent logging and close wait fix * Fix: broken multipart * Fix synclog backend reporting * Added forced auth backends * Bucket location simplistic implementation * More verbose debug log. Added dial timeout * Exported multipart specific errors. Not fully drained response body will attempt to close * Add log on failed request to BrimApi * Revert log * Fix linter * ACL hot fix (allegro#94) * Sign only requests with enumerated http methods in S3FixedKey backend type (allegro#96) * Instead of http status 203 append header (allegro#98) * Instead of http status 203 append header
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.