Latest ydb-go-sdk develops in https://github.com/ydb-platform/ydb-go-sdk
YDB API client written in Go.
- Overview
- About semantic versioning
- Prerequisites
- Installation
- Usage
- Credentials
- Generating code
- Examples
Currently package ydb provides scheme
and table
API client implementations for YDB
.
We follow the SemVer 2.0.0. In particular, we provide backward compatibility in the MAJOR
releases. New features without loss of backward compatibility appear on the MINOR
release. In the minor version, the patch number starts from 0
. Bug fixes and internal changes are released with the third digit (PATCH
) in the version.
There are, however, some changes with the loss of backward compatibility that we consider to be MINOR
:
- extension or modification of internal
ydb-go-sdk
interfaces. We understand that this will break the compatibility of custom implementations of theydb-go-sdk
internal interfaces. But we believe that the internal interfaces ofydb-go-sdk
are implemented well enough that they do not require custom implementation. We are working to ensure that all internal interfaces have limited access only insideydb-go-sdk
. - major changes to (including removal of) the public interfaces and types that have been previously exported by
ydb-go-sdk
. We understand that these changes will break the backward compatibility of early adopters of these interfaces. However, these changes are generally coordinated with early adopters and have the concise interfacing withydb-go-sdk
as a goal.
Internal interfaces outside from internal
directory are marked with comment such as
// Warning: only for internal usage inside ydb-go-sdk
We publish the planned breaking MAJOR
changes:
- via the comment
Deprecated
in the code indicating what should be used instead - through the file
NEXT_MAJOR_RELEASE.md
Requires Go 1.13 or later.
go get -u github.com/yandex-cloud/ydb-go-sdk/v2
The straightforward example of querying data may looks similar to this:
// Determine timeout for connect or do nothing
connectCtx, cancel := context.WithTimeout(ctx, time.Second)
defer cancel()
// connect package helps to connect to database, returns connection object which
// provide necessary clients such as table.Client, scheme.Client, etc.
// All manipulations with the connection could be done without the connect package
db, err := connect.New(
connectCtx,
connect.MustConnectionString(
"grpcs://ydb-ru.yandex.net:2135/?database=/ru/home/username/db",
),
)
if err != nil {
return fmt.Errorf("connect error: %w", err)
}
defer db.Close()
// Create session for execute queries
session, err := db.Table().CreateSession(ctx)
if err != nil {
// handle error
}
defer session.Close(ctx)
// Prepare transaction control for upcoming query execution.
// NOTE: result of TxControl() may be reused.
txc := table.TxControl(
table.BeginTx(table.WithSerializableReadWrite()),
table.CommitTx(),
)
// Execute text query without preparation and with given "autocommit"
// transaction control. That is, transaction will be commited without
// additional calls. Notice the "_" unused variable – it stands for created
// transaction during execution, but as said above, transaction is commited
// for us and we do not want to do anything with it.
_, res, err := session.Execute(ctx, txc,
`--!syntax_v1
DECLARE $mystr AS Utf8?;
SELECT 42 as id, $mystr as mystr
`,
table.NewQueryParameters(
table.ValueParam("$mystr", ydb.OptionalValue(ydb.UTF8Value("test"))),
),
)
if err != nil {
return err // handle error
}
// Scan for received values within the result set(s).
// res.Err() reports the reason of last unsuccessful one.
var (
id int32
myStr *string //optional value
)
for res.NextResultSet("id", "mystr") {
for res.NextRow() {
// Suppose our "users" table has two rows: id and age.
// Thus, current row will contain two appropriate items with
// exactly the same order.
err := res.Scan(&id, &myStr)
// Error handling.
if err != nil {
return err
}
// do something with data
fmt.Printf("got id %v, got mystr: %v\n", id, *myStr)
}
}
if res.Err() != nil {
return res.Err() // handle error
}
This example can be tested as ydb/example/from_readme
YDB sessions may become staled and appropriate error will be returned. To
reduce boilerplate overhead for such cases ydb-go-sdk
provides generic retry logic:
var res *table.Result
// Retry() will call given OperationFunc with the following invariants:
// - previous operation failed with retriable error;
// - number of retries is under the limit (default to 10, see table.Retryer docs);
//
// Note that in case of prepared statements call to Prepare() must be made
// inside the Operation body.
err = table.Retry(ctx, db.Table().Pool(),
table.OperationFunc(func(ctx context.Context, s *table.Session) (err error) {
res, err = s.Execute(...)
return
}),
)
That is, instead of manual creation of table.Session
, we give a
SessionPool
such responsibility. It holds instances of active sessions and
"pings" them periodically to keep them alive.
See table.Retryer
docs for more information about retry options.
There is a database/sql
driver for the sql-based applications or those which
by some reasons need to use additional layer of absctraction between user code
and storage backend.
For more information please see the docs of ydbsql
package which provides
database/sql
driver implementation.
There are different variants to get ydb.Credentials
object to get authorized.
Usage examples can be found here at func credentials(...) ydb.Credentials
.
There is lot of boilerplate code for scanning values from query result and for
passing values to prepare query. There is an experimental tool named
ydbgen
aimed to solve this.
go get -u github.com/yandex-cloud/ydb-go-sdk/v2/cmd/ydbgen
ydbgen
helps to generate such things:
- scanning values from result into a struct or slice of structs;
- building query parameters from struct
- building ydb's struct value from struct
- building ydb's list value from slice of structs
The very short example could be like this:
package somepkg
//go:generate ydbgen
//ydb:gen scan
type User struct {
Name string
Age int32
}
After running go generate path/to/somepkg/dir
file with suffix _ydbgen.go
will be generated. It will contain method Scan()
for User
type, as
requested in the generate comment.
Generation may be configured at three levels starting from top:
- ydbgen binary flags (package level)
- comment markers right before generation object in form of
//ydb:set [key1:value1 [... keyN:valueN]]
(type level) - struct tags (field level)
Each downstream level overrides options for its context.
For example, this code will generate all possible code for User
struct with
field Age
type mapped to non-optional type, because the lowermost
configuration level (which is struct tag) defines non-optional uint32
type:
//go:generate ydbgen -wrap optional
//ydb:gen
//ydb:set wrap:none
type User struct {
Age int32 `ydb:"type:uint32,column:user_age"`
}
Flag | Value | Default | Meaning |
---|---|---|---|
wrap |
optional |
+ | Wraps all mapped field types with optional type if no explicit tag is specified. |
wrap |
none |
No wrapping performed. | |
seek |
column |
+ | Uses res.SeekItem() call to find out next field to scan. |
seek |
position |
Uses res.NextItem() call to find out next field to scan. |
Options for comment markers are similar to flags, except the form of serialization.
Tag | Value | Default | Meaning |
---|---|---|---|
type |
T |
Specifies which ydb primitive type must be used for this field. | |
type |
T? |
The same as above, but wraps T with optional type. | |
conv |
safe |
+ | Prepares only safe type conversions. Fail generation if conversion is not possible. |
conv |
unsafe |
Prepares unsafe type conversions too. | |
conv |
assert |
Prepares safety assertions before type conversion. | |
column |
string |
Maps field to this column name. |
Also the shorthand tags are possible: when using tag without key:value
form,
tag with -
value is interpreted as field must be ignored; in other way it is
interpreted as the column name.
There are few additional options existing for flexibility purposes.
Previously only basic Go types were mentioned as ones that able to be converted
to ydb types. But it is possible generate code that maps defined type to YDB
type (actually to basic Go type and then to YDB type). To make so, such type
must provide two methods (when generation both getter and setters) – Get() (T, bool)
and Set(T)
, where T
is a basic Go type, and bool
is a flag that
indicates that value defined.
//go:generate ydbgen
//ydb:gen
type User struct {
Name OptString
}
type OptString struct {
Value string
Defined bool
}
func (s OptString) Get() (string, bool) {
return s.Value, s.Defined
}
func (s *OptString) Set(v string) {
*s = OptString{
Value: v,
Defined: true,
}
}
There is special package called ydb/opt
for this purposes:
package main
import "github.com/yandex-cloud/ydb-go-sdk/v2/opt"
//go:generate ydbgen
//ydb:gen
type User struct {
Name opt.String
}
There is additional feature that makes it easier to work with time.Time
values and their conversion to YDB types:
//go:generate ydbgen
//ydb:gen
type User struct {
Updated time.Time `ydb:"type:timestamp?"`
}
ydbgen
supports scanning and serializing container types such as List<T>
or Struct<T>
.
//go:generate ydbgen
//ydb:gen
type User struct {
Tags []string `ydb:"type:list<string>"`
}
Example above will interpret value for tags
column (or 0-th item, depending
on the seek
mode) as List<String>
.
Note that for String
type this is neccessary to inform ydbgen
that it is
not a container by setting type
field tag.
For more info please look at
ydb/examples/generation
folder.
More examples are listed in ydb/examples
directory.