YAML support for the Go language

This library has NO relation to the go-yaml/yaml library

Important

This library is developed from scratch to replace go-yaml/yaml. If you're looking for a better YAML library, this one should be helpful.

Why a new library?

As of this writing, there already exists a de facto standard library for YAML processing for Go: https://github.com/go-yaml/yaml. However, we believe that a new YAML library is necessary for the following reasons:

• Not actively maintained
• go-yaml/yaml has ported the libyaml written in C to Go, so the source code is not written in Go style
• There is a lot of content that cannot be parsed
• YAML is often used for configuration, and it is common to include validation along with it. However, the errors in go-yaml/yaml are not intuitive, and it is difficult to provide meaningful validation errors
• When creating tools that use YAML, there are cases where reversible transformation of YAML is required. However, to perform reversible transformations of content that includes Comments or Anchors/Aliases, manipulating the AST is the only option
• Non-intuitive Marshaler / Unmarshaler

By the way, libraries such as ghodss/yaml and sigs.k8s.io/yaml also depend on go-yaml/yaml, so if you are using these libraries, the same issues apply: they cannot parse things that go-yaml/yaml cannot parse, and they inherit many of the problems that go-yaml/yaml has.

Features

• No dependencies
• A better parser than go-yaml/yaml.
  • Support recursive processing
  • Higher coverage in the YAML Test Suite
    • YAML Test Suite consists of 402 cases in total, of which gopkg.in/yaml.v3 passes 295. In addition to passing all those test cases, goccy/go-yaml successfully passes nearly 60 additional test cases ( 2024/12/15 )
    • The test code is here
• Ease and sustainability of maintenance
  • The main maintainer is @goccy, but we are also building a system to develop as a team with trusted developers
  • Since it is written from scratch, the code is easy to read for Gophers
• An API structure that allows the use of not only Encoder/Decoder but also Tokenizer and Parser functionalities.
  • lexer.Tokenize
  • parser.Parse
• Filtering, replacing, and merging YAML content using YAML Path
• Reversible transformation without using the AST for YAML that includes Anchors, Aliases, and Comments
• Customize the Marshal/Unmarshal behavior for primitive types and third-party library types (RegisterCustomMarshaler, RegisterCustomUnmarshaler)
• Respects encoding/json behavior
  • Accept the json tag. Note that not all options from the json tag will have significance when parsing YAML documents. If both tags exist, yaml tag will take precedence.
  • json.Marshaler style marshaler
  • json.Unmarshaler style unmarshaler
  • Options for using MarshalJSON and UnmarshalJSON (UseJSONMarshaler, UseJSONUnmarshaler)
• Pretty format for error notifications
• Smart validation processing combined with go-playground/validator
  • example test code is here
• Allow referencing elements declared in another file via anchors

Users

The repositories that use goccy/go-yaml are listed here.

• https://github.com/goccy/go-yaml/wiki/Users

The source data is here. It is already being used in many repositories. Now it's your turn 😄

Installation

go get github.com/goccy/go-yaml

Synopsis

1. Simple Encode/Decode

Has an interface like go-yaml/yaml using reflect

var v struct {
	A int
	B string
}
v.A = 1
v.B = "hello"
bytes, err := yaml.Marshal(v)
if err != nil {
	//...
}
fmt.Println(string(bytes)) // "a: 1\nb: hello\n"

	yml := `
%YAML 1.2
---
a: 1
b: c
`
var v struct {
	A int
	B string
}
if err := yaml.Unmarshal([]byte(yml), &v); err != nil {
	//...
}

To control marshal/unmarshal behavior, you can use the yaml tag.

	yml := `---
foo: 1
bar: c
`
var v struct {
	A int    `yaml:"foo"`
	B string `yaml:"bar"`
}
if err := yaml.Unmarshal([]byte(yml), &v); err != nil {
	//...
}

For convenience, we also accept the json tag. Note that not all options from the json tag will have significance when parsing YAML documents. If both tags exist, yaml tag will take precedence.

	yml := `---
foo: 1
bar: c
`
var v struct {
	A int    `json:"foo"`
	B string `json:"bar"`
}
if err := yaml.Unmarshal([]byte(yml), &v); err != nil {
	//...
}

For custom marshal/unmarshaling, implement either Bytes or Interface variant of marshaler/unmarshaler. The difference is that while BytesMarshaler/BytesUnmarshaler behaves like encoding/json and InterfaceMarshaler/InterfaceUnmarshaler behaves like gopkg.in/yaml.v2.

Semantically both are the same, but they differ in performance. Because indentation matters in YAML, you cannot simply accept a valid YAML fragment from a Marshaler, and expect it to work when it is attached to the parent container's serialized form. Therefore when we receive use the BytesMarshaler, which returns []byte, we must decode it once to figure out how to make it work in the given context. If you use the InterfaceMarshaler, we can skip the decoding.

If you are repeatedly marshaling complex objects, the latter is always better performance wise. But if you are, for example, just providing a choice between a config file format that is read only once, the former is probably easier to code.

2. Reference elements declared in another file

testdata directory contains anchor.yml file:

├── testdata
   └── anchor.yml

And anchor.yml is defined as follows:

a: &a
  b: 1
  c: hello

Then, if yaml.ReferenceDirs("testdata") option is passed to yaml.Decoder, Decoder tries to find the anchor definition from YAML files the under testdata directory.

buf := bytes.NewBufferString("a: *a\n")
dec := yaml.NewDecoder(buf, yaml.ReferenceDirs("testdata"))
var v struct {
	A struct {
		B int
		C string
	}
}
if err := dec.Decode(&v); err != nil {
	//...
}
fmt.Printf("%+v\n", v) // {A:{B:1 C:hello}}

3. Encode with Anchor and Alias

3.1. Explicitly declared Anchor name and Alias name

If you want to use anchor, you can define it as a struct tag. If the value specified for an anchor is a pointer type and the same address as the pointer is found, the value is automatically set to alias. If an explicit alias name is specified, an error is raised if its value is different from the value specified in the anchor.

type T struct {
  A int
  B string
}
var v struct {
  C *T `yaml:"c,anchor=x"`
  D *T `yaml:"d,alias=x"`
}
v.C = &T{A: 1, B: "hello"}
v.D = v.C
bytes, err := yaml.Marshal(v)
if err != nil {
  panic(err)
}
fmt.Println(string(bytes))
/*
c: &x
  a: 1
  b: hello
d: *x
*/

3.2. Implicitly declared Anchor and Alias names

If you do not explicitly declare the anchor name, the default behavior is to use the equivalent of strings.ToLower($FieldName) as the name of the anchor. If the value specified for an anchor is a pointer type and the same address as the pointer is found, the value is automatically set to alias.

type T struct {
	I int
	S string
}
var v struct {
	A *T `yaml:"a,anchor"`
	B *T `yaml:"b,anchor"`
	C *T `yaml:"c"`
	D *T `yaml:"d"`
}
v.A = &T{I: 1, S: "hello"}
v.B = &T{I: 2, S: "world"}
v.C = v.A // C has same pointer address to A
v.D = v.B // D has same pointer address to B
bytes, err := yaml.Marshal(v)
if err != nil {
	//...
}
fmt.Println(string(bytes)) 
/*
a: &a
  i: 1
  s: hello
b: &b
  i: 2
  s: world
c: *a
d: *b
*/

3.3 MergeKey and Alias

Merge key and alias ( <<: *alias ) can be used by embedding a structure with the inline,alias tag.

type Person struct {
	*Person `yaml:",omitempty,inline,alias"` // embed Person type for default value
	Name    string `yaml:",omitempty"`
	Age     int    `yaml:",omitempty"`
}
defaultPerson := &Person{
	Name: "John Smith",
	Age:  20,
}
people := []*Person{
	{
		Person: defaultPerson, // assign default value
		Name:   "Ken",         // override Name property
		Age:    10,            // override Age property
	},
	{
		Person: defaultPerson, // assign default value only
	},
}
var doc struct {
	Default *Person   `yaml:"default,anchor"`
	People  []*Person `yaml:"people"`
}
doc.Default = defaultPerson
doc.People = people
bytes, err := yaml.Marshal(doc)
if err != nil {
	//...
}
fmt.Println(string(bytes))
/*
default: &default
  name: John Smith
  age: 20
people:
- <<: *default
  name: Ken
  age: 10
- <<: *default
*/

4. Pretty Formatted Errors

Error values produced during parsing have two extra features over regular error values.

First, by default, they contain extra information on the location of the error from the source YAML document, to make it easier to find the error location.

Second, the error messages can optionally be colorized.

If you would like to control exactly how the output looks like, consider using yaml.FormatError, which accepts two boolean values to control turning these features on or off.

5. Use YAMLPath

yml := `
store:
  book:
    - author: john
      price: 10
    - author: ken
      price: 12
  bicycle:
    color: red
    price: 19.95
`
path, err := yaml.PathString("$.store.book[*].author")
if err != nil {
  //...
}
var authors []string
if err := path.Read(strings.NewReader(yml), &authors); err != nil {
  //...
}
fmt.Println(authors)
// [john ken]

5.1 Print customized error with YAML source code

package main

import (
  "fmt"

  "github.com/goccy/go-yaml"
)

func main() {
  yml := `
a: 1
b: "hello"
`
  var v struct {
    A int
    B string
  }
  if err := yaml.Unmarshal([]byte(yml), &v); err != nil {
    panic(err)
  }
  if v.A != 2 {
    // output error with YAML source
    path, err := yaml.PathString("$.a")
    if err != nil {
      panic(err)
    }
    source, err := path.AnnotateSource([]byte(yml), true)
    if err != nil {
      panic(err)
    }
    fmt.Printf("a value expected 2 but actual %d:\n%s\n", v.A, string(source))
  }
}

output result is the following:

Tools

ycat

print yaml file with color

Installation

git clone https://github.com/goccy/go-yaml.git
cd ./cmd/ycat && go insatll .

For Developers

Note

In this project, we manage such test code under the testdata directory to avoid adding dependencies on libraries that are only needed for testing to the top go.mod file. Therefore, if you want to add test cases that use 3rd party libraries, please add the test code to the testdata directory.

Looking for Sponsors

I'm looking for sponsors this library. This library is being developed as a personal project in my spare time. If you want a quick response or problem resolution when using this library in your project, please register as a sponsor. I will cooperate as much as possible. Of course, this library is developed as an MIT license, so you can use it freely for free.

License

MIT