When building programs to GOOS=wasip1, the program does not have the
guarantees that the underlying directories will come from a file system
where a zero inode value indicates that the entry was deleted but not
yet removed from the directory. The host runtime may be running on
windows or may be exposing virtual user-space file systems that do not
have the concept of inodes. In those setup, we assume that the host
runtime is in charge of dealing with edge cases such as skipping
directory entries with zero inodes when needed, and the guest
application should trust the list of entries that it sees;
therefore, we disable skipping over zero inodes on wasip1.
Change-Id: I99aa562441cdb4182965f270af054cf3cf7f8f20
Reviewed-on: https://go-review.googlesource.com/c/go/+/507915
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
For #58141
Co-authored-by: Richard Musiol <neelance@gmail.com>
Co-authored-by: Achille Roussel <achille.roussel@gmail.com>
Co-authored-by: Julien Fabre <ju.pryz@gmail.com>
Co-authored-by: Evan Phoenix <evan@phx.io>
Change-Id: I52e3e161f81dcbe8605570e47d732992979c4d34
Reviewed-on: https://go-review.googlesource.com/c/go/+/479623
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
And then revert the bootstrap cmd directories and certain testdata.
And adjust tests as needed.
Not reverting the changes in std that are bootstrapped,
because some of those changes would appear in API docs,
and we want to use any consistently.
Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories
when preparing the bootstrap copy.
A few files changed as a result of running gofmt -w
not because of interface{} -> any but because they
hadn't been updated for the new //go:build lines.
Fixes#49884.
Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09
Reviewed-on: https://go-review.googlesource.com/c/go/+/368254
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
When these packages are released as part of Go 1.18,
Go 1.16 will no longer be supported, so we can remove
the +build tags in these files.
Ran go fix -fix=buildtag std cmd and then reverted the bootstrapDirs
as defined in src/cmd/dist/buildtool.go, which need to continue
to build with Go 1.4 for now.
Also reverted src/vendor and src/cmd/vendor, which will need
to be updated in their own repos first.
Manual changes in runtime/pprof/mprof_test.go to adjust line numbers.
For #41184.
Change-Id: Ic0f93f7091295b6abc76ed5cd6e6746e1280861e
Reviewed-on: https://go-review.googlesource.com/c/go/+/344955
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
(*File).readdir allocates a fixed-size 8KiB buffer on unix systems that
cannot be reused. While it accounts for just a single allocation, it's
more than large enough to show up in profiles and make things quite a
bit slower.
Instead of allocating so often, use a sync.Pool to allow these buffers to
be reused. This has a large impact on readdir heavy workloads.
Package os benchmarks:
name old time/op new time/op delta
Readdirname-12 35.6µs ± 5% 18.1µs ± 4% -49.00% (p=0.000 n=10+10)
Readdir-12 142µs ± 1% 121µs ± 0% -14.87% (p=0.000 n=10+9)
ReadDir-12 44.0µs ± 6% 28.4µs ± 8% -35.58% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
Readdirname-12 14.4kB ± 0% 6.2kB ± 0% -57.08% (p=0.000 n=10+10)
Readdir-12 41.6kB ± 0% 33.4kB ± 0% -19.77% (p=0.000 n=10+9)
ReadDir-12 21.9kB ± 0% 13.7kB ± 0% -37.39% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Readdirname-12 131 ± 0% 130 ± 0% -0.76% (p=0.000 n=10+10)
Readdir-12 367 ± 0% 366 ± 0% -0.27% (p=0.000 n=10+10)
ReadDir-12 249 ± 0% 248 ± 0% -0.40% (p=0.000 n=10+10)
A clunky benchmark I threw together that calls filepath.WalkDir on $GOMODCACHE:
name old time/op new time/op delta
WalkDir-12 91.2ms ±19% 48.7ms ± 0% -46.54% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
WalkDir-12 54.0MB ± 0% 7.6MB ± 0% -85.92% (p=0.000 n=8+9)
name old allocs/op new allocs/op delta
WalkDir-12 136k ± 0% 130k ± 0% -4.15% (p=0.000 n=8+8)
Change-Id: I00e4d48726da0e46c528ab205409afd03127b844
Reviewed-on: https://go-review.googlesource.com/c/go/+/302169
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
Make all our package sources use Go 1.17 gofmt format
(adding //go:build lines).
Part of //go:build change (#41184).
See https://golang.org/design/draft-gobuild
Change-Id: Ia0534360e4957e58cd9a18429c39d0e32a6addb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/294430
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Necessary to move PathError to io/fs.
For #41190.
Change-Id: I05e87675f38a22f0570d4366b751b6169f7a1b13
Reviewed-on: https://go-review.googlesource.com/c/go/+/243900
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
ReadDir provides a portable, efficient way to read a directory
and discover the type of directory entries.
This enables a more efficient file system walk, yet to be added.
See #41467 for the proposal review for the API.
Fixes#41467.
Change-Id: I461a526793ae46df48821aa448b04f1705546739
Reviewed-on: https://go-review.googlesource.com/c/go/+/261540
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Previously, Readdirnames returned a *PathError on Windows and Plan 9,
but a *SyscallError on POSIX systems.
In contrast, similar methods (such as Stat) return a *PathError on all platforms.
Fixes#38923
Change-Id: I26395905b1e723933f07b792c7aeee7c335949cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/233917
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When we seek on the underlying FD, discard any directory entries
we've already read and cached. This makes sure we won't return
the same entry twice.
We already fixed this for Darwin in CL 209961.
Fixes#37161
Change-Id: I20e1ac8d751443135e67fb4c43c18d69befb643b
Reviewed-on: https://go-review.googlesource.com/c/go/+/219143
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The first Readdirnames calls opendir and caches the result.
The behavior of that cached opendir result isn't specified on a seek
of the underlying fd. Free the opendir result on a seek so that
we'll allocate a new one the next time around.
Also fix wasm behavior in this regard, so that a seek to the
file start resets the Readdirnames position, regardless of platform.
p.s. I hate the Readdirnames API.
Fixes#35767.
Change-Id: Ieffb61b3c5cdd42591f69ab13f932003966f2297
Reviewed-on: https://go-review.googlesource.com/c/go/+/209961
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
You were a useful port and you've served your purpose.
Thanks for all the play.
A subsequent CL will remove amd64p32 (including assembly files and
toolchain bits) and remaining bits. The amd64p32 removal will be
separated into its own CL in case we want to support the Linux x32 ABI
in the future and want our old amd64p32 support as a starting point.
Updates #30439
Change-Id: Ia3a0c7d49804adc87bf52a4dea7e3d3007f2b1cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/199499
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Getdirentries is implemented with the __getdirentries64 function
in libSystem.dylib. That function works, but it's on Apple's
can't-be-used-in-an-app-store-application list.
Implement Getdirentries using the underlying fdopendir/readdir_r/closedir.
The simulation isn't faithful, and could be slow, but it should handle
common cases.
Don't use Getdirentries in the stdlib, use fdopendir/readdir_r/closedir
instead (via (*os.File).readdirnames).
Fixes#30933
Update #28984
RELNOTE=yes
Change-Id: Ia6b5d003e5bfe43ba54b1e1d9cfa792cc6511717
Reviewed-on: https://go-review.googlesource.com/c/go/+/168479
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The getdirentries syscall is considered private API on iOS and is
rejected by the App Store submission checks. Replace it with the
fdopendir/readdir_r/closedir syscalls.
Fixes#28984
Change-Id: I73341b124310e9cb34834a95f946769f337ec5b7
Reviewed-on: https://go-review.googlesource.com/c/153338
Reviewed-by: Keith Randall <khr@golang.org>
Reportedly CIFS on RHEL 7 can fail to report files if directories are
read in 4K increments. While this seems to be a CIFS or RHEL bug,
reportedly CIFS does not return more than 5760 bytes in a block, so
reading in 8K increments should hide the problem from users with
minimal cost.
Fixes#24015
Change-Id: Iaf9f00ffe338d379c819ed9edcd4cc9834e3b0f7
Reviewed-on: https://go-review.googlesource.com/121756
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This commit adds the js/wasm architecture to the os package.
Access to the actual file system is supported through Node.js.
Updates #18892
Change-Id: I6fa642fb294ca020b2c545649d4324d981aa0408
Reviewed-on: https://go-review.googlesource.com/109977
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This changes the os package to use the runtime poller for file I/O
where possible. When a system call blocks on a pollable descriptor,
the goroutine will be blocked on the poller but the thread will be
released to run other goroutines. When using a non-pollable
descriptor, the os package will continue to use thread-blocking system
calls as before.
For example, on GNU/Linux, the runtime poller uses epoll. epoll does
not support ordinary disk files, so they will continue to use blocking
I/O as before. The poller will be used for pipes.
Since this means that the poller is used for many more programs, this
modifies the runtime to only block waiting for the poller if there is
some goroutine that is waiting on the poller. Otherwise, there is no
point, as the poller will never make any goroutine ready. This
preserves the runtime's current simple deadlock detection.
This seems to crash FreeBSD systems, so it is disabled on FreeBSD.
This is issue 19093.
Using the poller on Windows requires opening the file with
FILE_FLAG_OVERLAPPED. We should only do that if we can remove that
flag if the program calls the Fd method. This is issue 19098.
Update #6817.
Update #7903.
Update #15021.
Update #18507.
Update #19093.
Update #19098.
Change-Id: Ia5197dcefa7c6fbcca97d19a6f8621b2abcbb1fe
Reviewed-on: https://go-review.googlesource.com/36800
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
In the case of a file being deleted while Readdir was running, it was
possible for File.Readdir to return an empty slice and a nil error,
counter to its documentation.
Fixes#16919
Change-Id: If0e42882eea52fbf5530317a1895f3829ea8e67b
Reviewed-on: https://go-review.googlesource.com/28056
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Code movement only.
If someone finds function 'foo' in "foo_linux.go",
they will expect that the Window version of 'foo' exists in "foo_windows.go".
Current code doesn't follow this manner.
For example, 'sameFile' exists in "file_unix.go",
"stat_plan9.go" and "types_windows.go".
The CL address that problem by following rules:
* readdir family => dir.go, dir_$GOOS.go
* stat family => stat.go, stat_$GOOS.go
* path-functions => path_$GOOS.go
* sameFile => types.go, types_$GOOS.go
* process-functions => exec.go, exec_$GOOS.go
* hostname => sys.go, sys_$GOOS.go
Change-Id: Ic3c64663ce0b2a364d7a414351cd3c772e70187b
Reviewed-on: https://go-review.googlesource.com/27035
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
