Generate embedded vocabulary maps lazily

[espadolini]

Maps in vars are created at init time, resulting in something like 2.5MiB of allocation at runtime that can be problematic for programs that only optionally use the tokenizer; this PR changes the vocab codegen to replace the map literal in the var with a map literal assignment in a fooInit function that can then get called with a sync.Once to only generate the map on first load.

[ghost]

Thanks for this. I'll look into this tomorrow.

[ghost]

Ciao, the code looks good. Before I merge it could you make sure the generated code passes gofmt?

(I think the change would be on the generators internal/cmd/vocab.go)

For example this:

var p50kBaseVocabOnce sync.Once
var p50kBaseVocab vocab
func p50kBaseVocabInit() {
p50kBaseVocab = vocab{
    "aasdsd": 1,
     ...
}

becomes

var p50kBaseVocabOnce sync.Once
var p50kBaseVocab vocab

func p50kBaseVocabInit() {
	p50kBaseVocab = vocab{
		"!":  0,
		"\"": 1,
               ....,
	}
}

[espadolini]

@mariano-sb I didn't do that initially because I wanted to avoid touching the actual lines with the words (and because it wasn't gofmt-correct to begin with), but if you view the changes in github with "hide whitespace" enabled it's still easy to review.