cmd/vet: warn about string(int) type conversions

[ianlancetaylor]

#3939 proposes removing the explicit type conversion string(i) where i has an integer type other than rune. That is a backward incompatible change, so we cannot make it until we can count on having modules everywhere, with the support modules provide for determining the version of the language to use to compile code.

There is a step we can take today, which is to change go vet to report conversions of integer types other than rune to string. If we think that removing this feature from the language is a good idea, then a vet check for it is a good idea, and it is one that we can implement today.

[agnivade]

@alandonovan @mvdan

[networkimprov]

Pls also permit string(b) where b is a byte or uint8.

[mvdan]

This seems fine to me. The warning should probably point the user in the right direction.

[alanfo]

Pls also permit string(b) where b is a byte or uint8.

I think this is essential if this proposal (and eventually #3939) is to be adopted.

The string(byte) conversion, when you're dealing with ASCII characters only, is probably used at least as often as string(rune)in my experience.

[dolmen]

@networkimprov @alanfo You will still be able to use string([]byte{b}).

[networkimprov]

That would be a surprising requirement. And allocates a byte slice for a single statement?

[alanfo]

@dolmen

You will still be able to use string([]byte{b}).

Well string(rune(b)) would be a better alternative because you wouldn't then have to allocate a byte slice but it would still be a nuisance to have to update all my code to use this workaround :(

[smasher164]

I will take a crack at adding an analysis pass and vet check for this.

[alandonovan]

string([]byte{b}) is not equivalent to string(b) for a byte value b: the former yields the string whose bytes are just b, the latter returns the string that UTF-8-encodes the single rune b. If b is greater than 127, then the former will be a single-byte string but the latter will be a two-byte encoding.

it would still be a nuisance to have to update all my code to use this workaround :(

Michael Matloob recently presented at a conference the "suggested fixes" feature of the analysis Diagnostics API. The diagnostic for the new analyzer could replace (or offer to replace) the deprecated expressions by the preferred version.

[networkimprov]

string([]byte{b}) ... yields the string whose bytes are just b

That could be invalid UTF-8. So string(buf[x:y]) may be a bug.

To extract strings from buffers, should we generally use string(bytes.ToValidUTF8(buf, nil))?

[alandonovan]

That could be invalid UTF-8. So string(buf[x:y]) may be a bug.

It could be, but it very much depends on the situation.

To extract strings from buffers, should we generally use string(bytes.ToValidUTF8(buf, nil))?

I had never even heard of that function until now, and I'm more deliberate than most about string encodings. Slicing a string or byte slice may be perfectly fine, again depending on the situation.

[alanfo]

The diagnostic for the new analyzer could replace (or offer to replace) the deprecated expressions by the preferred version.

Be that as it may, it still seems anomalous to allow string(rune) but deprecate string(byte).

After all, string([]rune] and string([]byte) are both allowed but not conversions from slices of other integer types. It may therefore seem strange to people that both corresponding 'scalar' conversions are not allowed as well.

Indeed, @ianlancetaylor made a similar point in #3939 itself.

[smasher164]

The diagnostic for the new analyzer could replace (or offer to replace) the deprecated expressions by the preferred version.

What would be the suggested fix for an integer that overflows the size of the rune? Although this shouldn't occur in practice, simply changing it to rune(i) would break pre-existing code.

package main

import "fmt"

func main() {
	i := int64(1 << 33)
	fmt.Printf("%q %q\n", string(i), string(rune(i))) // prints "�" "\x00"
}

[gopherbot]

Change https://golang.org/cl/212919 mentions this issue: go/analysis: add pass to check string(int) conversions

[ianlancetaylor]

@smasher164 Do you have any examples of correct code that has this problem?

The correct fix is going to be to add a condition comparing the value to utf8.MaxRune, and when it is larger using utf8.RuneError.

[smasher164]

@ianlancetaylor Running the tool over my GOPATH revealed 16 cases where an int, uint, int64, or uint64 were converted to a string, and did not do bounds checking to ensure that the converted value doesn't overflow MaxRune. However, most of these cases depend on these functions' call sites, which could potentially be passing only valid input.

github.com/antlr/antlr4/runtime/Go/antlr/dfa_serializer.go:115:15: conversion from int to string
func (l *LexerDFASerializer) getEdgeLabel(i int) string {
    return "'" + string(i) + "'"
}

github.com/antlr/antlr4/runtime/Go/antlr/interval_set.go:235:34: conversion from int to string
github.com/antlr/antlr4/runtime/Go/antlr/interval_set.go:238:30: conversion from int to string
github.com/antlr/antlr4/runtime/Go/antlr/interval_set.go:238:53: conversion from int to string
type Interval struct {
    start int
    stop  int
}
type IntervalSet struct {
    intervals []*Interval
    readOnly  bool
}
for j := 0; j < len(i.intervals); j++ {
    v := i.intervals[j]
    if v.stop == v.start+1 {
        if v.start == TokenEOF {
            names = append(names, "<EOF>")
        } else {
            names = append(names, ("'" + string(v.start) + "'"))
        }
    } else {
        names = append(names, "'"+string(v.start)+"'..'"+string(v.stop-1)+"'")
    }
}

github.com/antlr/antlr4/runtime/Go/antlr/lexer_atn_simulator.go:633:15: conversion from int to string
func (l *LexerATNSimulator) GetTokenName(tt int) string {
    if tt == -1 {
        return "EOF"
    }

    return "'" + string(tt) + "'"
}

github.com/antlr/antlr4/runtime/Go/antlr/transition.go:239:15: conversion from int to string
github.com/antlr/antlr4/runtime/Go/antlr/transition.go:239:42: conversion from int to string
type RangeTransition struct {
    *BaseTransition

    start, stop int
}
func (t *RangeTransition) String() string {
    return "'" + string(t.start) + "'..'" + string(t.stop) + "'"
}

github.com/cznic/ql/etc.go:464:10: conversion from int to string
switch i := int(o); i {
case andand:
    return "&&"
case andnot:
    return "&^"
case lsh:
    return "<<"
case le:
    return "<="
case eq:
    return "=="
case ge:
    return ">="
case neq:
    return "!="
case oror:
    return "||"
case rsh:
    return ">>"
default:
    return string(i)
}

github.com/cznic/ql/etc.go:901:11: conversion from idealInt to string
github.com/cznic/ql/etc.go:905:11: conversion from idealUint to string
github.com/cznic/ql/etc.go:918:11: conversion from int64 to string
github.com/cznic/ql/etc.go:928:11: conversion from uint64 to string
case qString:
switch x := val.(type) {
//case nil:
//case idealComplex:
//case idealFloat:
case idealInt:
    return string(x), nil
case idealRune:
    return string(x), nil
case idealUint:
    return string(x), nil
//case bool:
//case complex64:
//case complex128:
//case float32:
//case float64:
case int8:
    return string(x), nil
case int16:
    return string(x), nil
case int32:
    return string(x), nil
case int64:
    return string(x), nil
case string:
    return x, nil
case uint8:
    return string(x), nil
case uint16:
    return string(x), nil
case uint32:
    return string(x), nil
case uint64:
    return string(x), nil
case []byte:
    return string(x), nil
case *big.Int:
    return x.String(), nil
case time.Time:
    return x.String(), nil
case time.Duration:
    return x.String(), nil
default:
    return invConv(val, typ)
}

github.com/cznic/ql/etc.go:1698:15: conversion from uint64 to string
rec[i] = string(y)

github.com/cznic/ql/ql.go:820:17: conversion from int to string
for _, f := range f {
    a = append(a, string(f.typ)+f.name)
}

github.com/go-delve/delve/pkg/proc/eval.go:535:9: conversion from int64 to string
b, _ := constant.Int64Val(argv.Value)
s := string(b)

github.com/unidoc/unidoc/pdf/internal/cmap/cmap.go:396:36: conversion from uint64 to string
target := hexToUint64(v)
i := uint64(0)
for sc := srcCodeFrom; sc <= srcCodeTo; sc++ {
    r := target + i
    cmap.codeMap[numBytes-1][sc] = string(r)
    i++
}

[ianlancetaylor]

Thanks for the list. The antlr and ql cases all look like incorrect uses of string(int), where they really meant to use strconv.Itoa(int). If I'm right, the warning is serving its purpose for those cases. The unicode case probably really does mean to use string(r), so that code would have to change. I'm not sure about the go-delve case.

[smasher164]

I'm not sure about the go-delve case.

The go-delve case is implementing eval for type conversion, so I would ignore that case.

FWIW, if the goal of backwards compatibility in the suggested fixes is to ensure that the fix results in the same string as string(int), then comparing against utf8.MaxRune isn't the only option:

func conv(v uint64) uint64 {
    if v > utf8.MaxRune {
        return utf8.RuneError
    }
    return v
}

When a rune is converted to a string, if bit 22 is set, then the string will print "�". Using this fact, the conversion can be made branchless:

func conv(v uint64) uint64 {
    return v | (1-((v>>22|v>>21&1)-1)>>63)<<21
}

This checks if any of the top 43 bits are nonzero, and uses that truth to set bit 22.

[ianlancetaylor]

The goal of the warning is to tell people when their code is not acting as they expect. As far as I can see the fix for cases like antlr is not to add a condition; it's to change the code to use strconv.Itoa instead of string(int), so that the right thing happens.

[smasher164]

That's fair. But we'd have to detect and distinguish those cases from cases where the conversion is correctly made and stays within the bounds of a rune. For instance:

/Users/akhil/work/src/golang.org/x/net/webdav/internal/xml/xml.go:1043:14: conversion from uint64 to string
if err == nil && n <= unicode.MaxRune {
    text = string(n)
    haveText = true
}

/Users/akhil/work/src/golang.org/x/net/webdav/internal/xml/xml.go:1066:15: conversion from int to string
var entity = map[string]int{
    "lt":   '<',
    "gt":   '>',
    "amp":  '&',
    "apos": '\'',
    "quot": '"',
}
if r, ok := entity[s]; ok {
    text = string(r)
    haveText = true
} else if d.Entity != nil {
    text, haveText = d.Entity[s]
}

/Users/akhil/work/src/golang.org/x/text/unicode/cldr/base.go:101:10: conversion from int64 to string
if s[1] == '#' {
    r, _ := strconv.ParseInt(s[3:len(s)-1], 16, 32)
    return string(r)
}

/Users/akhil/work/src/golang.org/x/text/message/fmt_test.go:1454:9: conversion from int to string
for i := 0; i < 128; i++ {
    if f.Flag(i) {
        s += string(i)
    }
}

/Users/akhil/work/src/golang.org/x/exp/errors/fmt/scan.go:607:50: conversion from int64 to string
r := int64(s.getRune())
n := uint(bitSize)
x := (r << (64 - n)) >> (64 - n)
if x != r {
    s.errorString("overflow on character value " + string(r))
}

/Users/akhil/work/src/github.com/docker/docker/vendor/github.com/golang/protobuf/proto/text_parser.go:315:10: conversion from uint64 to string
if i > utf8.MaxRune {
    return "", "", fmt.Errorf(`\%c%s is not a valid Unicode code point`, r, ss)
}
return string(i), s, nil

/Users/akhil/work/src/github.com/docker/docker/pkg/chrootarchive/archive_test.go:103:32: conversion from int to string
for i := 0; i < 65534; i++ {
    excludes[i] = strings.Repeat(string(i), 64)
}

/Users/akhil/work/src/github.com/rogpeppe/godef/vendor/9fans.net/go/plan9/dir.go:173:9: conversion from int to string
/Users/akhil/work/src/github.com/rogpeppe/godef/vendor/9fans.net/go/plan9/dir.go:177:10: conversion from int to string
var permChars = []permChar{
    permChar{DMDIR, 'd'},
    permChar{DMAPPEND, 'a'},
    permChar{DMAUTH, 'A'},
    permChar{DMDEVICE, 'D'},
    permChar{DMSOCKET, 'S'},
    permChar{DMNAMEDPIPE, 'P'},
    permChar{0, '-'},
    permChar{DMEXCL, 'l'},
    permChar{DMSYMLINK, 'L'},
    permChar{0, '-'},
    permChar{0400, 'r'},
    permChar{0, '-'},
    permChar{0200, 'w'},
    permChar{0, '-'},
    permChar{0100, 'x'},
    permChar{0, '-'},
    permChar{0040, 'r'},
    permChar{0, '-'},
    permChar{0020, 'w'},
    permChar{0, '-'},
    permChar{0010, 'x'},
    permChar{0, '-'},
    permChar{0004, 'r'},
    permChar{0, '-'},
    permChar{0002, 'w'},
    permChar{0, '-'},
    permChar{0001, 'x'},
    permChar{0, '-'},
}
for _, pc := range permChars {
    if p&pc.bit != 0 {
        did = true
        s += string(pc.c)
    }
    if pc.bit == 0 {
        if !did {
            s += string(pc.c)
        }
        did = false
    }
}

/Users/akhil/work/src/github.com/miekg/dns/msg_test.go:70:13: conversion from int64 to string
n, _ := strconv.ParseInt(escape[1:], 10, 8)
return string(n)

/Users/akhil/work/src/github.com/google/skylark/library.go:406:16: conversion from int to string
i, err := AsInt32(args[0])
if err != nil {
    return nil, fmt.Errorf("chr: got %s, want int", args[0].Type())
}
if i < 0 {
    return nil, fmt.Errorf("chr: Unicode code point %d out of range (<0)", i)
}
if i > unicode.MaxRune {
    return nil, fmt.Errorf("chr: Unicode code point U+%X out of range (>0x10FFFF)", i)
}
return String(string(i)), nil

/Users/akhil/work/src/github.com/cznic/ql/all_test.go:2949:9: conversion from int to string
for _, v := range rand.Perm(32) {
    s += string('0' + v)
}

/Users/akhil/work/src/upspin.io/client/clientutil/all_test.go:129:40: conversion from int to string
for i := 0; i < 10; i++ {
    loc := upspin.Location{
        Endpoint:  inProcess,
        Reference: upspin.Reference("ref" + string(i)),
    }
    tLocs = append(tLocs, loc)
}