Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SnapshotDeleteRequest use []string instead of string #237

Merged
merged 6 commits into from
Mar 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .ci/opensearch/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ services:
environment:
- discovery.type=single-node
- bootstrap.memory_lock=true
- path.repo=/usr/share/opensearch/mnt
ports:
- "9200:9200"
user: opensearch
2 changes: 1 addition & 1 deletion .github/workflows/test-integration-unreleased.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ jobs:
working-directory: opensearch/distribution/archives/linux-tar/build/distributions
run: |
tar xf opensearch-min-*
./opensearch-*/bin/opensearch &
./opensearch-*/bin/opensearch -Epath.repo=/usr/share/opensearch/mnt &
for attempt in {1..20}; do sleep 5; if curl -s localhost:9200; then echo '=====> ready'; break; fi; echo '=====> waiting...'; done

- name: Checkout Go Client
Expand Down
41 changes: 15 additions & 26 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,47 +6,36 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)

### Dependencies

- Bumps `github.com/aws/aws-sdk-go` from 1.44.45 to 1.44.225
- Bumps `github.com/aws/aws-sdk-go-v2` from 1.17.1 to 1.17.6
- Bumps `github.com/aws/aws-sdk-go-v2/config` from 1.18.10 to 1.18.18
- Bumps `github.com/aws/aws-sdk-go-v2/config` from 1.18.9 to 1.18.12
- Bumps `github.com/aws/aws-sdk-go-v2/config` from 1.18.8 to 1.18.10
- Bumps `github.com/aws/aws-sdk-go-v2/config` from 1.17.10 to 1.18.9
- Bumps `github.com/aws/aws-sdk-go` from 1.44.195 to 1.44.225
- Bumps `github.com/aws/aws-sdk-go` from 1.44.185 to 1.44.205
- Bumps `github.com/aws/aws-sdk-go` from 1.44.180 to 1.44.195
- Bumps `github.com/aws/aws-sdk-go` from 1.44.176 to 1.44.185
- Bumps `github.com/aws/aws-sdk-go` from 1.44.132 to 1.44.180
- Bumps `github.com/stretchr/testify` from 1.8.1 to 1.8.2
- Bumps `github.com/aws/aws-sdk-go-v2/config` from 1.18.8 to 1.18.18
- Bumps `github.com/stretchr/testify` from 1.8.0 to 1.8.2

### Added

- Github workflow for changelog verification ([#172](https://github.com/opensearch-project/opensearch-go/pull/172))
- Add Go Documentation link for the client ([#182](https://github.com/opensearch-project/opensearch-go/pull/182))
- Add implementation of Data Streams API ([#257](https://github.com/opensearch-project/opensearch-go/pull/257)
- Support for Amazon OpenSearch Serverless ([#216](https://github.com/opensearch-project/opensearch-go/pull/216), [#259](https://github.com/opensearch-project/opensearch-go/pull/259))
- Add Err() function to Response for detailed errors ([#246](https://github.com/opensearch-project/opensearch-go/pull/246))

### Dependencies

- Bumps `github.com/stretchr/testify` from 1.8.0 to 1.8.1
- Bumps `github.com/aws/aws-sdk-go` from 1.44.45 to 1.44.132
- Adds support for Amazon OpenSearch Serverless ([#216](https://github.com/opensearch-project/opensearch-go/pull/216), [#259](https://github.com/opensearch-project/opensearch-go/pull/259))
- Adds Github workflow for changelog verification ([#172](https://github.com/opensearch-project/opensearch-go/pull/172))
- Adds Go Documentation link for the client ([#182](https://github.com/opensearch-project/opensearch-go/pull/182))
- Adds implementation of Data Streams API ([#257](https://github.com/opensearch-project/opensearch-go/pull/257)
- Adds `Err()` function to Response for detailed errors ([#246](https://github.com/opensearch-project/opensearch-go/pull/246))

### Changed

- Workflow improvements ([#242](https://github.com/opensearch-project/opensearch-go/pull/242))
- Opensearchapi check the response for errors ([#246](https://github.com/opensearch-project/opensearch-go/pull/246))
- Uses `[]string` instead of `string` in `SnapshotDeleteRequest` ([#237](https://github.com/opensearch-project/opensearch-go/pull/237))
- Removes the need for double error checking ([#246](https://github.com/opensearch-project/opensearch-go/pull/246))
- Updates workflows to reduce CI time, consolidate OpenSearch versions, update compatibility matrix ([#242](https://github.com/opensearch-project/opensearch-go/pull/242))

### Deprecated

### Removed

- Remove info call before performing every request ([#219](https://github.com/opensearch-project/opensearch-go/pull/219))
- Removes info call before performing every request ([#219](https://github.com/opensearch-project/opensearch-go/pull/219))

### Fixed

- Renamed the sequence number struct tag to if_seq_no to fix optimistic concurrency control ([#166](https://github.com/opensearch-project/opensearch-go/pull/166))
- Fix `RetryOnConflict` on bulk indexer ([#215](https://github.com/opensearch-project/opensearch-go/pull/215))
- Correct curl logging to emit the correct URL destination ([#101](https://github.com/opensearch-project/opensearch-go/pull/101))
- Renames the sequence number struct tag to `if_seq_no` to fix optimistic concurrency control ([#166](https://github.com/opensearch-project/opensearch-go/pull/166))
- Fixes `RetryOnConflict` on bulk indexer ([#215](https://github.com/opensearch-project/opensearch-go/pull/215))
- Corrects curl logging to emit the correct URL destination ([#101](https://github.com/opensearch-project/opensearch-go/pull/101))

### Security

Expand Down
66 changes: 56 additions & 10 deletions UPGRADING.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,71 @@
- [Upgrading Opensearch GO Client](#upgrading-opensearch-go-client)
- [Upgraading to >= 3.0.0](#upgrading-to->=-3.0.0)
- [opensearchapi](#opensearchapi-snapshot-delete)
- [opensearchapi](#opensearchapi-error-handling)

# Upgrading Opensearch GO Client

## Upgrading to >= 3.0.0

### opensearchapi snapshot delete

`SnapshotDeleteRequest` and `SnapshotDelete` changed the argument `Snapshot` type from `string` to `[]string`.

Before:

```go
// If you have a string containing your snapshot
stringSnapshotsToDelete := "snapshot-1,snapshot-2"
reqSnapshots := &opensearchapi.SnapshotDeleteRequest{
Repository: repo,
Snapshot: stringSnapshotsToDelete,
}

// If you have a slice of strings containing your snapshot
sliceSnapshotToDelete := []string{"snapshot-1","snapshot-2"}
reqSnapshots := &opensearchapi.SnapshotDeleteRequest{
Repository: repo,
Snapshot: strings.Join(sliceSnapshotsToDelete, ","),
}
```

After:

```go
// If you have a string containing your snapshots
stringSnapshotsToDelete := strings.Split("snapshot-1,snapshot-2", ",")
reqSnapshots := &opensearchapi.SnapshotDeleteRequest{
Repository: repo,
Snapshot: stringSnapshotsToDelete,
}

// If you have a slice of strings containing your snapshots
sliceSnapshotToDelete := []string{"snapshot-1", "snapshot-2"}
reqSnapshots := &opensearchapi.SnapshotDeleteRequest{
Repository: repo,
Snapshot: sliceSnapshotsToDelete,
```

### opensearchapi error handling
With opensearch-go >= 3.0.0 opensearchapi responses are now checked for errors.
Prior versions only returned an error if the request failed to execute. For example if the client can't reach the server or the TLS handshake failed.

With opensearch-go >= 3.0.0 opensearchapi responses are now checked for errors. Checking for errors twice is no longer needed.

Prior versions only returned an error if the request failed to execute. For example if the client can't reach the server or the TLS handshake failed.
With opensearch-go >= 3.0.0 each opensearchapi requests will return an error if the response http status code is > 299.
The error can be parsed into the new opensearchapi.Error type by using `errors.As` to match for exceptions and get a more detailed view. See the example below.
The error can be parsed into the new `opensearchapi.Error` type by using `errors.As` to match for exceptions and get a more detailed view.

Example:

```go
createIndex := opensearchapi.IndicesCreateRequest{
Index: IndexName,
Body: mapping,
}
ctx := context.Background()
var opensearchError *opensearchapi.Error
createIndexResponse, err := createIndex.Do(ctx, client)
createIndex := opensearchapi.IndicesCreateRequest{
Index: IndexName,
Body: mapping,
}

ctx := context.Background()
var opensearchError *opensearchapi.Error

createIndexResponse, err := createIndex.Do(ctx, client)
// Load err into opensearchapi.Error to access the fields and tolerate if the index already exists
if err != nil {
if errors.As(err, &opensearchError) {
Expand Down
10 changes: 5 additions & 5 deletions opensearchapi/api.snapshot.delete.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ import (
)

func newSnapshotDeleteFunc(t Transport) SnapshotDelete {
return func(repository string, snapshot string, o ...func(*SnapshotDeleteRequest)) (*Response, error) {
return func(repository string, snapshot []string, o ...func(*SnapshotDeleteRequest)) (*Response, error) {
var r = SnapshotDeleteRequest{Repository: repository, Snapshot: snapshot}
for _, f := range o {
f(&r)
Expand All @@ -48,13 +48,13 @@ func newSnapshotDeleteFunc(t Transport) SnapshotDelete {
// SnapshotDelete deletes a snapshot.
//
//
type SnapshotDelete func(repository string, snapshot string, o ...func(*SnapshotDeleteRequest)) (*Response, error)
type SnapshotDelete func(repository string, snapshot []string, o ...func(*SnapshotDeleteRequest)) (*Response, error)

// SnapshotDeleteRequest configures the Snapshot Delete API request.
//
type SnapshotDeleteRequest struct {
Repository string
Snapshot string
Snapshot []string

MasterTimeout time.Duration
ClusterManagerTimeout time.Duration
Expand All @@ -80,13 +80,13 @@ func (r SnapshotDeleteRequest) Do(ctx context.Context, transport Transport) (*Re

method = "DELETE"

path.Grow(1 + len("_snapshot") + 1 + len(r.Repository) + 1 + len(r.Snapshot))
path.Grow(1 + len("_snapshot") + 1 + len(r.Repository) + 1 + len(strings.Join(r.Snapshot, ",")))
path.WriteString("/")
path.WriteString("_snapshot")
path.WriteString("/")
path.WriteString(r.Repository)
path.WriteString("/")
path.WriteString(r.Snapshot)
path.WriteString(strings.Join(r.Snapshot, ","))

params = make(map[string]string)

Expand Down
121 changes: 121 additions & 0 deletions opensearchapi/opensearchapi_integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,9 @@ package opensearchapi_test

import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"testing"
Expand Down Expand Up @@ -192,4 +194,123 @@ func TestAPI(t *testing.T) {
}
}
})
t.Run("Snapshot", func(t *testing.T) {
// Functio to perform requests
//
opensearchDo := func(ctx context.Context, client *opensearch.Client, req opensearchapi.Request, msg string, t *testing.T) {
_, err := req.Do(ctx, client)
if err != nil {
var opensearchError *opensearchapi.Error
if errors.As(err, &opensearchError) {
if opensearchError.Err.Type == "snapshot_missing_exception" {
return
}
t.Fatalf("Failed to %s: %s", msg, err)
}
}
}

// Create Client
//
client, err := opensearch.NewDefaultClient()
if err != nil {
t.Fatalf("Error creating the client: %s\n", err)
}

// Pre Cleanup indices
//
iDeleteReq := &opensearchapi.IndicesDeleteRequest{
Index: []string{"test", "test_restored"},
IgnoreUnavailable: opensearchapi.BoolPtr(true),
}
ctx := context.Background()
opensearchDo(ctx, client, iDeleteReq, "index data", t)

// Index data
//
var buf bytes.Buffer
for j := 1; j <= 1000; j++ {
meta := []byte(fmt.Sprintf(`{ "index" : { "_id" : "%d" } }%s`, j, "\n"))
data := []byte(`{"content":"` + strings.Repeat("ABC", 100) + `"}`)
data = append(data, "\n"...)

buf.Grow(len(meta) + len(data))
buf.Write(meta)
buf.Write(data)
}

bulkReq := &opensearchapi.BulkRequest{
Body: bytes.NewReader(buf.Bytes()),
Index: "test",
Refresh: "true",
}
opensearchDo(ctx, client, bulkReq, "index data", t)

// Test Snapshot functions
//
sRepoCreateReq := &opensearchapi.SnapshotCreateRepositoryRequest{
Body: bytes.NewBufferString(`{"type":"fs","settings":{"location":"/usr/share/opensearch/mnt"}}`),
Repository: "snapshot-test",
}
opensearchDo(ctx, client, sRepoCreateReq, "create Snapshot Repository", t)

sRepoVerifyReq := &opensearchapi.SnapshotVerifyRepositoryRequest{
Repository: "snapshot-test",
}
opensearchDo(ctx, client, sRepoVerifyReq, "verify Snapshot Repository", t)

sDeleteReq := &opensearchapi.SnapshotDeleteRequest{
Snapshot: []string{"test", "clone-test"},
Repository: "snapshot-test",
}
opensearchDo(ctx, client, sDeleteReq, "delete Snapshots", t)

sCreateReq := &opensearchapi.SnapshotCreateRequest{
Body: bytes.NewBufferString(`{"indices":"test","ignore_unavailable":true,"include_global_state":false,"partial":false}`),
Snapshot: "test",
Repository: "snapshot-test",
WaitForCompletion: opensearchapi.BoolPtr(true),
}
opensearchDo(ctx, client, sCreateReq, "create Snapshot", t)

sCloneReq := &opensearchapi.SnapshotCloneRequest{
Body: bytes.NewBufferString(`{"indices":"*"}`),
Snapshot: "test",
TargetSnapshot: "clone-test",
Repository: "snapshot-test",
}
opensearchDo(ctx, client, sCloneReq, "clone Snapshot", t)

sGetReq := &opensearchapi.SnapshotGetRequest{
Snapshot: []string{"test", "clone-test"},
Repository: "snapshot-test",
}
opensearchDo(ctx, client, sGetReq, "get Snapshots", t)

sStatusReq := &opensearchapi.SnapshotGetRequest{
Snapshot: []string{"test", "clone-test"},
Repository: "snapshot-test",
}
opensearchDo(ctx, client, sStatusReq, "get Snapshot status", t)

sRestoreReq := &opensearchapi.SnapshotRestoreRequest{
Body: bytes.NewBufferString(
`{
"indices":"test",
"ignore_unavailable":true,
"include_global_state":false,
"partial":false,
"rename_pattern": "(.+)",
"rename_replacement":"$1_restored"
}`,
),
Snapshot: "clone-test",
Repository: "snapshot-test",
WaitForCompletion: opensearchapi.BoolPtr(true),
}
opensearchDo(ctx, client, sRestoreReq, "restore Snapshot", t)

opensearchDo(ctx, client, sDeleteReq, "delete Snapshots", t)
opensearchDo(ctx, client, iDeleteReq, "index data", t)
})
}