User proper write-once semantics for GCS repository #30438
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ywelsch
added
>enhancement
:Distributed Coordination/Snapshot/Restore
|
Pinging @elastic/es-distributed |
tlrx
approved these changes
May 15, 2018
| if ((e.getStatusCode() == HTTP_PRECON_FAILED) || ((error != null) && (error.getCode() == HTTP_PRECON_FAILED))) { | ||
| throw new FileAlreadyExistsException(blobName, null, e.getMessage()); | ||
| } | ||
| } |
ywelsch
added a commit
that referenced
this pull request
May 17, 2018
There's no need for an extra blobExists() call when writing a blob to the GCS service. GCS provides an option (with stronger consistency guarantees) on the insert method that guarantees that the blob that's uploaded does not already exist. Relates to #19749
|
Thanks @tlrx |
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
May 17, 2018
…ngs-to-true * elastic/master: (25 commits) [DOCS] Replace X-Pack terms with attributes [ML] Clean left behind model state docs (elastic#30659) Correct typos filters agg docs duplicated 'bucket' word removal (elastic#30677) top_hits doc example description update (elastic#30676) [Docs] Replace InetSocketTransportAddress with TransportAdress (elastic#30673) [TEST] Account for increase in ML C++ memory usage (elastic#30675) User proper write-once semantics for GCS repository (elastic#30438) Remove bogus file accidentally added Add detailed assert message to IndexAuditUpgradeIT (elastic#30669) Adjust fast forward for token expiration test (elastic#30668) Improve explanation in rescore (elastic#30629) Deprecate `nGram` and `edgeNGram` names for ngram filters (elastic#30209) Watcher: Fix watch history template for dynamic slack attachments (elastic#30172) Fix _cluster/state to always return cluster_uuid (elastic#30656) [Tests] Add debug information to CorruptedFileIT Preserve REST client auth despite 401 response (elastic#30558) [test] packaging: add windows boxes (elastic#30402) Make xpack modules instead of a meta plugin (elastic#30589) Mute ShrinkIndexIT ...
dnhatn
added a commit
that referenced
this pull request
May 19, 2018
* 6.x: Mute testCorruptFileThenSnapshotAndRestore Plugins: Remove meta plugins (#30670) Upgrade to Lucene-7.4.0-snapshot-59f2b7aec2 (#30726) Docs: Add uptasticsearch to list of clients (#30738) [TEST] Reduce forecast overflow to disk test memory limit (#30727) [DOCS] Removes redundant index.asciidoc files (#30707) [DOCS] Moves X-Pack configurationg pages in table of contents (#30702) [ML][TEST] Fix bucket count assertion in ModelPlotsIT (#30717) [ML][TEST] Make AutodetectMemoryLimitIT less fragile (#30716) [Build] Add test admin when starting gradle run with trial license and [ML] provide tmp storage for forecasting and possibly any ml native jobs #30399 Tests: Fail if test watches could not be triggered (#30392) Watcher: Prevent duplicate watch triggering during upgrade (#30643) [ML] add version information in case of crash of native ML process (#30674) Add detailed assert message to IndexAuditUpgradeIT (#30669) Preserve REST client auth despite 401 response (#30558) Make TransportClusterStateAction abide to our style (#30697) [DOCS] Fixes edit URLs for stack overview (#30583) [DOCS] Add missing callout in IndicesClientDocumentationIT Backport get settings API changes to 6.x (#30494) Silence sleep based watcher test [DOCS] Replace X-Pack terms with attributes Improve explanation in rescore (#30629) [test] packaging: add windows boxes (#30402) [ML] Clean left behind model state docs (#30659) filters agg docs duplicated 'bucket' word removal (#30677) top_hits doc example description update (#30676) MovingFunction Pipeline agg backport to 6.x (#30658) [Docs] Replace InetSocketTransportAddress with TransportAdress (#30673) [TEST] Account for increase in ML C++ memory usage (#30675) User proper write-once semantics for GCS repository (#30438) Deprecate `nGram` and `edgeNGram` names for ngram filters (#30209) Watcher: Fix watch history template for dynamic slack attachments (#30172) Fix _cluster/state to always return cluster_uuid (#30656)
dnhatn
added a commit
that referenced
this pull request
May 19, 2018
* master: Scripting: Remove getDate methods from ScriptDocValues (#30690) Upgrade to Lucene-7.4.0-snapshot-59f2b7aec2 (#30726) [Docs] Fix single page :docs:check invocation (#30725) Docs: Add uptasticsearch to list of clients (#30738) [DOCS] Removes out-dated x-pack/docs/en/index.asciidoc [DOCS] Removes redundant index.asciidoc files (#30707) [TEST] Reduce forecast overflow to disk test memory limit (#30727) Plugins: Remove meta plugins (#30670) [DOCS] Moves X-Pack configurationg pages in table of contents (#30702) TEST: Add engine log to testCorruptFileThenSnapshotAndRestore [ML][TEST] Fix bucket count assertion in ModelPlotsIT (#30717) [ML][TEST] Make AutodetectMemoryLimitIT less fragile (#30716) Default copy settings to true and deprecate on the REST layer (#30598) [Build] Add test admin when starting gradle run with trial license and This implementation lazily (on 1st forecast request) checks for available diskspace and creates a subfolder for storing data outside of Lucene indexes, but as part of the ES data paths. Tests: Fail if test watches could not be triggered (#30392) [ML] add version information in case of crash of native ML process (#30674) Make TransportClusterStateAction abide to our style (#30697) Change required version for Get Settings transport API changes to 6.4.0 (#30706) [DOCS] Fixes edit URLs for stack overview (#30583) Silence sleep based watcher test [TEST] Adjust version skips for movavg/movfn tests [DOCS] Replace X-Pack terms with attributes [ML] Clean left behind model state docs (#30659) Correct typos filters agg docs duplicated 'bucket' word removal (#30677) top_hits doc example description update (#30676) [Docs] Replace InetSocketTransportAddress with TransportAdress (#30673) [TEST] Account for increase in ML C++ memory usage (#30675) User proper write-once semantics for GCS repository (#30438) Remove bogus file accidentally added Add detailed assert message to IndexAuditUpgradeIT (#30669) Adjust fast forward for token expiration test (#30668) Improve explanation in rescore (#30629) Deprecate `nGram` and `edgeNGram` names for ngram filters (#30209) Watcher: Fix watch history template for dynamic slack attachments (#30172) Fix _cluster/state to always return cluster_uuid (#30656) [Tests] Add debug information to CorruptedFileIT # Conflicts: # test/framework/src/main/java/org/elasticsearch/indices/analysis/AnalysisFactoryTestCase.java
ywelsch
added a commit
to ywelsch/elasticsearch
that referenced
this pull request
May 23, 2018
There's no need for an extra blobExists() call when writing a blob to the GCS service. GCS provides an option (with stronger consistency guarantees) on the insert method that guarantees that the blob that's uploaded does not already exist. Relates to elastic#19749
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There's no need for an extra blobExists() call when writing a blob to the GCS service. GCS provides an option (with stronger consistency guarantees) on the insert method that guarantees that the blob that's uploaded does not already exist.
I have not tested this against the real GCS service, only used the API docs to develop this. @tlrx can you help me check that this works against the actual GCS service? Thanks.
Relates to #19749