Follow up CL 156379.
Updates #19093
Change-Id: I5ea3177fc5911d3af71cbb32584249e419e9d4a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/172937
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
WriteAt use pwrite syscall on *nix or WriteFile on Windows.
On Linux/Windows, these system calls always write to end of file in
append mode, regardless of offset parameter.
It is hard (maybe impossible) to make WriteAt work portably.
Making WriteAt returns an error if file is opened in append mode, we
guarantee to get consistent behavior between platforms, also prevent
user from accidently corrupting their data.
Fixes#30716
Change-Id: If83d935a22a29eed2ff8fe53d13d0b4798aa2b81
Reviewed-on: https://go-review.googlesource.com/c/go/+/166578
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Support for FreeBSD 10 will be dropped with Go 1.13, so revert the
workaround introduced in CL 157099.
Updates #29633
Updates #27619
Change-Id: I1a2e50d3f807a411389f3db07c0f4535a590da02
Reviewed-on: https://go-review.googlesource.com/c/go/+/165801
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
windows-arm TMP directory live inside such link (see
https://github.com/golang/go/issues/29746#issuecomment-456526811 for
details), so symlinks like that will be common at least on windows-arm.
This CL builds on current syscall.Readlink implementation. Main
difference between the two is how new code handles symlink targets,
like \??\Volume{ABCD}\.
New implementation uses Windows CreateFile API with
FILE_FLAG_OPEN_REPARSE_POINT flag to get \??\Volume{ABCD}\ file handle.
And then it uses Windows GetFinalPathNameByHandle with VOLUME_NAME_DOS
flag to convert that handle into standard Windows path.
FILE_FLAG_OPEN_REPARSE_POINT flag ensures that symlink is not followed
when CreateFile opens the file.
Fixes#30463
Change-Id: I33b18227ce36144caed694169ef2e429fd995fb4
Reviewed-on: https://go-review.googlesource.com/c/164201
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
- add EINTR loop on Darwin
- return PathError on error
- call newFile rather than NewFile
This tries to minimize the possibility of any future changes.
It would be nice to put openFdAt in the same file as openFileNolog,
but build tags forbid.
Updates #29983
Change-Id: I866002416d6473fbfd80ff6ef09b2bc4607f2934
Reviewed-on: https://go-review.googlesource.com/c/160181
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
The kqueue based netpoller always registers file descriptors with EVFILT_READ and EVFILT_WRITE.
However only EVFILT_READ notification is supported for regular files.
On FreeBSD a regular file is always reported as ready for writing, resulting in a busy wait.
On Darwin, Dragonfly, NetBSD and OpenBSD, a regular file is reported as ready for both reading and writing only once.
Updates #19093
Change-Id: If284341f60c6c2332fb5499637d4cfa7a4e26b7b
Reviewed-on: https://go-review.googlesource.com/c/156379
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The getdirentries syscall is considered private API on iOS and is
rejected by the App Store submission checks. Replace it with the
fdopendir/readdir_r/closedir syscalls.
Fixes#28984
Change-Id: I73341b124310e9cb34834a95f946769f337ec5b7
Reviewed-on: https://go-review.googlesource.com/c/153338
Reviewed-by: Keith Randall <khr@golang.org>
The Darwin kqueue implementation doesn't report any event when the
last writer for a fifo is closed.
Fixes#24164
Change-Id: Ic2c47018ef1284bf2e26379f8dd7646edaad4d05
Reviewed-on: https://go-review.googlesource.com/118566
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Each URL was manually verified to ensure it did not serve up incorrect
content.
Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df
Reviewed-on: https://go-review.googlesource.com/115798
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
This commit adds the js/wasm architecture to the os package.
Access to the actual file system is supported through Node.js.
Updates #18892
Change-Id: I6fa642fb294ca020b2c545649d4324d981aa0408
Reviewed-on: https://go-review.googlesource.com/109977
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
If NewFile is called with a file descriptor that is already set to
non-blocking mode, it tries to return a pollable file (one for which
SetDeadline methods work) by adding the filedes to the poll/netpoll
mechanism. If called with a filedes in blocking mode, it returns a
non-pollable file, as it always did.
Fixes#22939
Updates #24331
Change-Id: Id54c8be1b83e6d35e14e76d7df0e57a9fd64e176
Reviewed-on: https://go-review.googlesource.com/100077
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Instead of calling Chmod directly on perm, stat the created file/dir to extract the
actual permission bits which can be different from perm due to umask.
Fixes#23120.
Change-Id: I3e70032451fc254bf48ce9627e98988f84af8d91
Reviewed-on: https://go-review.googlesource.com/84477
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Otherwise, on systems for which syscall does not implement Getwd,
a lot of unnecessary files and directories get added to the testlog,
right up the root directory. This was causing tests on such systems
to fail to cache in practice.
Updates #22593
Change-Id: Ic8cb3450ea62aa0ca8eeb15754349f151cd76f85
Reviewed-on: https://go-review.googlesource.com/83455
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When we write a cached test result, we now also write a log of the
environment variables and files inspected by the test run,
along with a hash of their content. Before reusing a cached test result,
we recompute the hash of the content specified by the log, and only
use the result if that content has not changed.
This makes test caching behave correctly for tests that consult
environment variables or stat or read files or directories.
Fixes#22593.
Change-Id: I8608798e73c90e0c1911a38bf7e03e1232d784dc
Reviewed-on: https://go-review.googlesource.com/81895
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The full truth seems too complicated to write in this method's doc, so
I'm going with a simple half truth.
The full truth is that Fd returns the descriptor in blocking mode,
because that is historically how it worked, and existing programs
would be surprised if the descriptor is suddenly non-blocking. On Unix
systems whether a file is non-blocking or not is a property of the
underlying file description, not of a particular file descriptor, so
changing the returned descriptor to blocking mode also changes the
existing File to blocking mode. Blocking mode works fine, althoug I/O
operations now take up a thread. SetDeadline and friends rely on the
runtime poller, and the runtime poller only works if the descriptor is
non-blocking. So it's correct that calling Fd disables SetDeadline.
The other half of the truth is that if the program is willing to work
with a non-blocking descriptor, it could call
syscall.SetNonblock(descriptor, true) to change the descriptor, and
the original File, to non-blocking mode. At that point SetDeadline
would start working again. I tried to write that in a way that is
short and comprehensible but failed. Since deadlines mostly work on
pipes, and there isn't much reason to call Fd on a pipe, and few
people use SetDeadline, I decided to punt.
Fixes#22934
Change-Id: I2e49e036f0bcf71f5365193831696f9e4120527c
Reviewed-on: https://go-review.googlesource.com/81636
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Add SetDeadline, SetReadDeadline, and SetWriteDeadline methods to os.File,
just as they exist today for the net package.
Fixes#22114
Change-Id: I4d390d739169b991175baba676010897dc8568fa
Reviewed-on: https://go-review.googlesource.com/71770
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Instead record in the File whether it is stdout/stderr. This avoids a
race between a call to epipecheck and closing the file.
Fixes#21994
Change-Id: Ic3d552ffa83402136276bcb5029ec3e6691042c2
Reviewed-on: https://go-review.googlesource.com/65750
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Catch all the cases where a file operation might return ErrFileClosing,
and convert to ErrClosed. Use a new method for the conversion, which
permits us to remove some KeepAlive calls.
Change-Id: I584178f297efe6cb86f3090b2341091b412f1041
Reviewed-on: https://go-review.googlesource.com/41793
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
In the past we returned "use of closed network connection" when using
a closed network descriptor in some way. In CL 36799 that was changed
to return "use of closed file or network connection". Because programs
have no access to a value of this error type (see issue #4373) they
resort to doing direct string comparisons (see issue #19252). This CL
restores the old error string so that we don't break programs
unnecessarily with the 1.9 release.
This adds a test to the net package for the expected string.
For symmetry check that the os package returns the expected error,
which for os already exists as os.ErrClosed.
Updates #4373.
Fixed#19252.
Change-Id: I5b83fd12cfa03501a077cad9336499b819f4a38b
Reviewed-on: https://go-review.googlesource.com/39997
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Now that the os package uses internal/poll on Unix and Windows systems,
it can rely on internal/poll reference counting to ensure that the
file descriptor is not closed until all I/O is complete.
That was already working. This CL completes the job by not trying to
modify the Sysfd field when it might still be used by the I/O routines.
Fixes#7970
Change-Id: I7a3daa1a6b07b7345bdce6f0cd7164bd4eaee952
Reviewed-on: https://go-review.googlesource.com/41674
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This changes the os package to use the runtime poller for file I/O
where possible. When a system call blocks on a pollable descriptor,
the goroutine will be blocked on the poller but the thread will be
released to run other goroutines. When using a non-pollable
descriptor, the os package will continue to use thread-blocking system
calls as before.
For example, on GNU/Linux, the runtime poller uses epoll. epoll does
not support ordinary disk files, so they will continue to use blocking
I/O as before. The poller will be used for pipes.
Since this means that the poller is used for many more programs, this
modifies the runtime to only block waiting for the poller if there is
some goroutine that is waiting on the poller. Otherwise, there is no
point, as the poller will never make any goroutine ready. This
preserves the runtime's current simple deadlock detection.
This seems to crash FreeBSD systems, so it is disabled on FreeBSD.
This is issue 19093.
Using the poller on Windows requires opening the file with
FILE_FLAG_OVERLAPPED. We should only do that if we can remove that
flag if the program calls the Fd method. This is issue 19098.
Update #6817.
Update #7903.
Update #15021.
Update #18507.
Update #19093.
Update #19098.
Change-Id: Ia5197dcefa7c6fbcca97d19a6f8621b2abcbb1fe
Reviewed-on: https://go-review.googlesource.com/36800
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Windows has a limit of 260 characters on normal paths, but it's possible
to use longer paths by using "extended-length paths" that begin with
`\\?\`. This commit attempts to transparently convert an absolute path
to an extended-length path, following the subtly different rules those
paths require. It does not attempt to handle relative paths, which
continue to be passed to the operating system unmodified.
This adds a new test, TestLongPath, to the os package. This test makes
sure that it is possible to write a path at least 400 characters long
and runs on every platform. It also tests symlinks and hardlinks, though
symlinks are not testable with our builder configuration.
HasLink is moved to internal/testenv so it can be used by multiple tests.
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx
has Microsoft's documentation on extended-length paths.
Fixes#3358.
Fixes#10577.
Fixes#17500.
Change-Id: I4ff6bb2ef9c9a4468d383d98379f65cf9c448218
Reviewed-on: https://go-review.googlesource.com/32451
Run-TryBot: Quentin Smith <quentin@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Unix rejects this when new is a non-empty directory.
Other systems reject this when new is a directory, empty or not.
Make Unix reject empty directory too.
Fixes#14527.
Change-Id: Ice24b8065264c91c22cba24aa73e142386c29c87
Reviewed-on: https://go-review.googlesource.com/31358
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Code movement only.
If someone finds function 'foo' in "foo_linux.go",
they will expect that the Window version of 'foo' exists in "foo_windows.go".
Current code doesn't follow this manner.
For example, 'sameFile' exists in "file_unix.go",
"stat_plan9.go" and "types_windows.go".
The CL address that problem by following rules:
* readdir family => dir.go, dir_$GOOS.go
* stat family => stat.go, stat_$GOOS.go
* path-functions => path_$GOOS.go
* sameFile => types.go, types_$GOOS.go
* process-functions => exec.go, exec_$GOOS.go
* hostname => sys.go, sys_$GOOS.go
Change-Id: Ic3c64663ce0b2a364d7a414351cd3c772e70187b
Reviewed-on: https://go-review.googlesource.com/27035
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The tree's pretty inconsistent about single space vs double space
after a period in documentation. Make it consistently a single space,
per earlier decisions. This means contributors won't be confused by
misleading precedence.
This CL doesn't use go/doc to parse. It only addresses // comments.
It was generated with:
$ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])')
$ go test go/doc -update
Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7
Reviewed-on: https://go-review.googlesource.com/20022
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Dave Day <djd@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Old behavior: 10 consecutive EPIPE errors on any descriptor cause the
program to exit with a SIGPIPE signal.
New behavior: an EPIPE error on file descriptors 1 or 2 cause the
program to raise a SIGPIPE signal. If os/signal.Notify was not used to
catch SIGPIPE signals, this will cause the program to exit with SIGPIPE.
An EPIPE error on a file descriptor other than 1 or 2 will simply be
returned from Write.
Fixes#11845.
Update #9896.
Change-Id: Ic85d77e386a8bb0255dc4be1e4b3f55875d10f18
Reviewed-on: https://go-review.googlesource.com/18151
Reviewed-by: Russ Cox <rsc@golang.org>
Include syscall.Stat_t on unix to the
unexported fileStat structure rather than
accessing it though an interface.
Additionally add a benchmark for Readdir
(and Readdirnames).
Tested on linux, freebsd, netbsd, openbsd
darwin, solaris, does not touch windows
stuff. Does not change the API, as
discussed on golang-dev.
E.g. on linux/amd64 with a directory of 65 files:
benchmark old ns/op new ns/op delta
BenchmarkReaddir-4 67774 66225 -2.29%
benchmark old allocs new allocs delta
BenchmarkReaddir-4 334 269 -19.46%
benchmark old bytes new bytes delta
BenchmarkReaddir-4 25208 24168 -4.13%
Change-Id: I44ef72a04ad7055523a980f29aa11122040ae8fe
Reviewed-on: https://go-review.googlesource.com/16423
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In particular, don't use goto and do restrict the behavior to darwin.
This addresses comments from http://golang.org/cl/14484.
Change-Id: I5b99e1762d1c5b27fdd12b72a5c6d981f6a92f0f
Reviewed-on: https://go-review.googlesource.com/14673
Reviewed-by: Rob Pike <r@golang.org>
The man page for sigaction(2) on OS X doesn't guarantee that SA_RESTART
will work for open(2) on regular files:
The affected system calls include open(2), read(2), write(2),
sendto(2), recvfrom(2), sendmsg(2) and recvmsg(2) on a
communications channel or a slow device (such as a terminal, but not
a regular file) and during a wait(2) or ioctl(2).
I've never observed EINTR from open(2) for a traditional file system
such as HFS+, but it's easy to observe with a fuse file system that is
slightly slow (cf. https://goo.gl/UxsVgB). After this change, the
problem can no longer be reproduced when calling os.OpenFile.
Fixes#11180.
Change-Id: I967247430e20a7d29a285b3d76bf3498dc4773db
Reviewed-on: https://go-review.googlesource.com/14484
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Old style. Make it compliant with our code review comments document.
Also, make WriteString's return parameter named 'n', not 'ret', for
consistency.
Noticed during another documentation review.
Change-Id: Ie88910c5841f8353bc5c0152e2168b497578e15e
Reviewed-on: https://go-review.googlesource.com/12324
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Rename now uses MoveFileEx which was previously not available to
use because it is not supported on Windows 2000.
Change-Id: I583d029c4467c9be6d1574a790c423559b441e87
Reviewed-on: https://go-review.googlesource.com/6140
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Mkdir and OpenFile call Chmod internally on *BSD and Solaris,
because these OSes don't handle the sticky bit correctly.
However Chmod's error should be ignored. It shouldn't hide
the fact that a file itself is created.
Fixes#8383
Change-Id: Ia2e0b2ba72712d73a0a48ba5a263432e0fff31a5
Reviewed-on: https://go-review.googlesource.com/2057
Reviewed-by: Russ Cox <rsc@golang.org>
