diff --git a/design/33974-add-public-lockedfile-pkg.md b/design/33974-add-public-lockedfile-pkg.md new file mode 100644 index 00000000..e54c6b3b --- /dev/null +++ b/design/33974-add-public-lockedfile-pkg.md @@ -0,0 +1,207 @@ +# Proposal: make the internal [lockedfile](https://godoc.org/github.com/golang/go/src/cmd/go/internal/lockedfile/) package public + +Author(s): [Adrien Delorme] + +Last updated: 2019-10-15 + +Discussion at https://golang.org/issue/33974. + +## Abstract + +Move already existing code residing in +`golang/go/src/cmd/go/internal/lockedfile` to `x/sync`. + +## Background + +A few open source Go projects are implementing file locking mechanisms but they +do not seem to be maintained anymore: +* https://github.com/gofrs/flock : This repo has lastly accepted PRs in March + 2019, so this implementation may be maintained and we could argue that the + `lockedfile` package API is more ergonomic. Incompatibilities with AIX, + Solaris and Illumos are preventing file locking on both projects, but it + looks like the go team is addressing for `lockedfile`. +* https://github.com/juju/fslock : Note that this implementation is both + unmaintained and LGPL-licensed, so even folks who would like to use it might + not be able to. Also not that this repo [was selected for removal in + 2017](https://github.com/juju/fslock/issues/4) + + +As a result some major projects are doing +their own version of it; ex: +[terraform](https://github.com/hashicorp/terraform/blob/1ff9a540202b8c36e33db950374bbb4495737d8f/states/statemgr/filesystem_lock_unix.go), +[boltdb](https://github.com/boltdb/bolt/search?q=flock&unscoped_q=flock). After +some researches it seemed to us that the already existing and maintained +[lockedfile](https://godoc.org/github.com/golang/go/src/cmd/go/internal/lockedfile/) +package is the best 'open source' version. + +File-locking interacts pretty deeply with the `os` package and the system call +library in `x/sys`, so it makes sense for (a subset of) the same owners to +consider the evolution of those packages together. +We think it would benefit the mass to make such a package public: since it's +already being part of the go code and therefore being maintained; it should be +made public. + +## Proposal + +We propose to copy the golang/go/src/cmd/go/internal/lockedfile to `x/exp`. To +make it public. Not changing any of the named types for now. + +Exported names and comments as can be currently found in +[07b4abd](https://github.com/golang/go/tree/07b4abd62e450f19c47266b3a526df49c01ba425/src/cmd/go/internal/lockedfile): + +``` +// Package lockedfile creates and manipulates files whose contents should only +// change atomically. +package lockedfile + +// A File is a locked *os.File. +// +// Closing the file releases the lock. +// +// If the program exits while a file is locked, the operating system releases +// the lock but may not do so promptly: callers must ensure that all locked +// files are closed before exiting. +type File struct { + // contains unexported fields +} + +// Create is like os.Create, but returns a write-locked file. +// If the file already exists, it is truncated. +func Create(name string) (*File, error) + +// Edit creates the named file with mode 0666 (before umask), +// but does not truncate existing contents. +// +// If Edit succeeds, methods on the returned File can be used for I/O. +// The associated file descriptor has mode O_RDWR and the file is write-locked. +func Edit(name string) (*File, error) + +// Transform invokes t with the result of reading the named file, with its lock +// still held. +// +// If t returns a nil error, Transform then writes the returned contents back to +// the file, making a best effort to preserve existing contents on error. +// +// t must not modify the slice passed to it. +func Transform(name string, t func([]byte) ([]byte, error)) (err error) + +// Open is like os.Open, but returns a read-locked file. +func Open(name string) (*File, error) + +// OpenFile is like os.OpenFile, but returns a locked file. +// If flag implies write access (ie: os.O_TRUNC, os.O_WRONLY or os.O_RDWR), the +// file is write-locked; otherwise, it is read-locked. +func OpenFile(name string, flag int, perm os.FileMode) (*File, error) + +// Read reads up to len(b) bytes from the File. +// It returns the number of bytes read and any error encountered. +// At end of file, Read returns 0, io.EOF. +// +// File can be read-locked or write-locked. +func (f *File) Read(b []byte) (n int, err error) + +// ReadAt reads len(b) bytes from the File starting at byte offset off. +// It returns the number of bytes read and the error, if any. +// ReadAt always returns a non-nil error when n < len(b). +// At end of file, that error is io.EOF. +// +// File can be read-locked or write-locked. +func (f *File) ReadAt(b []byte, off int64) (n int, err error) + +// Write writes len(b) bytes to the File. +// It returns the number of bytes written and an error, if any. +// Write returns a non-nil error when n != len(b). +// +// If File is not write-locked Write returns an error. +func (f *File) Write(b []byte) (n int, err error) + +// WriteAt writes len(b) bytes to the File starting at byte offset off. +// It returns the number of bytes written and an error, if any. +// WriteAt returns a non-nil error when n != len(b). +// +// If file was opened with the O_APPEND flag, WriteAt returns an error. +// +// If File is not write-locked WriteAt returns an error. +func (f *File) WriteAt(b []byte, off int64) (n int, err error) + +// Close unlocks and closes the underlying file. +// +// Close may be called multiple times; all calls after the first will return a +// non-nil error. +func (f *File) Close() error + +// A Mutex provides mutual exclusion within and across processes by locking a +// well-known file. Such a file generally guards some other part of the +// filesystem: for example, a Mutex file in a directory might guard access to +// the entire tree rooted in that directory. +// +// Mutex does not implement sync.Locker: unlike a sync.Mutex, a lockedfile.Mutex +// can fail to lock (e.g. if there is a permission error in the filesystem). +// +// Like a sync.Mutex, a Mutex may be included as a field of a larger struct but +// must not be copied after first use. The Path field must be set before first +// use and must not be change thereafter. +type Mutex struct { + // Path to the well-known lock file. Must be non-empty. + // + // Path must not change on a locked mutex. + Path string + // contains filtered or unexported fields +} + +// MutexAt returns a new Mutex with Path set to the given non-empty path. +func MutexAt(path string) *Mutex + +// Lock attempts to lock the Mutex. +// +// If successful, Lock returns a non-nil unlock function: it is provided as a +// return-value instead of a separate method to remind the caller to check the +// accompanying error. (See https://golang.org/issue/20803.) +func (mu *Mutex) Lock() (unlock func(), err error) + +// String returns a string containing the path of the mutex. +func (mu *Mutex) String() string +``` + +## Rationale + +* The `lockedfile.File` implements a subset of the `os.File` but with file + locking protection. + +* The `lockedfile.Mutex` does not implement `sync.Locker`: unlike a + `sync.Mutex`, a `lockedfile.Mutex` can fail to lock (e.g. if there is a + permission error in the filesystem). + +* `lockedfile` adds an `Edit` and a `Transform` function; `Edit` is not + currently part of the `file` package. Edit exists to make it easier to + implement locked read-modify-write operation. `Transform` simplifies the act + of reading and then writing to a locked file. + + +* Making this package public will make it more used. A tiny surge of issues + might come in the beginning; at the benefits of everyone. (Unless it's bug + free !!). + +* There exists a https://godoc.org/github.com/rogpeppe/go-internal package that + exports a lot of internal packages from the go repo. But if go-internal + became wildly popular; in order to have a bug fixed or a feature introduced + in; a user would still need to open a PR on the go repo; then the author of + go-internal would need to update the package. + +## Compatibility + +There are no retro-compatibility issues since this will be a code addition but +ideally we don't want to maintain two copies of this package going forward, and +we probably don't want to vendor `x/exp` into the `cmd` module. + + + +Perhaps that implies that this should go in the `x/sys` or `x/sync` repo instead? + +## Implementation + +Adrien Delorme plans to do copy the exported types in the proposal section from + `cmd/go/internal/lockedfile` to `x/sync`. + +Adrien Delorme plans to change the references to the `lockedfile` package in +`cmd`. \ No newline at end of file