This patch fixes up a bug in the inliner's special case code for
coverage counter updates, which was not properly working for
-covermode=atomic compilations.
Updates #56044.
Change-Id: I9e309312b123121c3df02862623bdbab1f6c6a4b
Reviewed-on: https://go-review.googlesource.com/c/go/+/441858
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
The linker needs FuncInfo metadata for all inlined functions. This is
typically handled by gc.enqueueFunc calling ir.InitLSym for all function
declarations in typecheck.Target.Decls (ir.UseClosure adds all closures
to Decls).
However, non-trivial closures in Decls are ignored, and are insteaded
enqueued when walk of the calling function discovers them.
This presents a problem for direct calls to closures. Inlining will
replace the entire closure definition with its body, which hides the
closure from walk and thus suppresses symbol creation.
Explicitly create a symbol early in this edge case to ensure we keep
this metadata.
InitLSym needs to move out of ssagen to avoid a circular dependency (it
doesn't have anything to do with ssa anyway). There isn't a great place
for it, so I placed it in ir, which seemed least objectionable.
The added test triggers one of these inlined direct non-trivial closure
calls, though the test needs CL 429637 to fail, which adds a FuncInfo
assertion to the linker. Note that the test must use "run" instead of
"compile" since the assertion is in the linker, and "compiler" doesn't
run the linker.
Fixes#54959.
Change-Id: I0bd1db4f3539a78da260934cd968372b7aa92546
Reviewed-on: https://go-review.googlesource.com/c/go/+/436240
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Add a new "coverage counter" classification for variables to be used
for storing code coverage counter values (somewhat in the same way
that we identify fuzzer counters). Tagging such variables allows us to
aggregate them in the linker, and to treat updates specially.
Updates #51430.
Change-Id: Ib49fb05736ffece98bcc2f7a7c37e991b7f67bbb
Reviewed-on: https://go-review.googlesource.com/c/go/+/401235
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Go 1.19 introduce new append-like APIs in package encoding/binary, this
change teaches the inliner to treat calls to these methods as cheap, so
that code using them will be more inlineable.
Updates #42958
Change-Id: Ie3dd4906e285430f435bdedbf8a11fdffce9302d
Reviewed-on: https://go-review.googlesource.com/c/go/+/431015
Auto-Submit: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
Reviewed-by: Jenny Rakoczy <jenny@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org>
Reviewed-by: Keith Randall <khr@google.com>
In go.dev/cl/419674 I added a mechanism to the inliner to allow
inlining to fail gracefully when a function body is missing, but I
missed we already have a mechanism for that: typecheck.HaveInlineBody.
This CL makes it overridable so that unified IR can plug in its
appropriate logic, like it does with the logic for building the
ir.InlinedCallExpr node.
While here, rename inline.NewInline to inline.InlineCall, because the
name "NewInline" is now a misnomer since we initialize it to oldInline
(now named oldInlineCall).
Change-Id: I4e65618d3725919f69e6f43cf409699d20fb797c
Reviewed-on: https://go-review.googlesource.com/c/go/+/427234
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
This CL changes the inliner to process transitive inlining iteratively
after the AST has actually been edited, rather than recursively and
immediately. This is important for handling indirect function calls
correctly, because ir.reassigned walks the function body looking for
reassignments; whereas previously the inlined reassignments might not
have been actually added to the AST yet.
Fixes#54632.
Change-Id: I0dd69813c8a70b965174e0072335bc00afedf286
Reviewed-on: https://go-review.googlesource.com/c/go/+/425257
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
The non-unified frontend had repeated issues with inlining and
generics (#49309, #51909, #52907), which led us to substantially
restrict inlining when shape types were present.
However, these issues are evidently not present in unified IR's
inliner, and the safety restrictions added for the non-unified
frontend can simply be disabled in unified mode.
Fixes#54497.
Change-Id: I8e6ac9f3393c588bfaf14c6452891b9640a9d1bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/424775
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
ir.ClosureExpr implements ir.InitNode, so ir.InitExpr can prepend init
statements to it. However, CalleeEffects wasn't aware of this and
could cause the init statements to get dropped when inlining a call to
a closure.
This isn't an issue today, because we don't create closures with init
statements. But I ran into this within unified IR.
Easy and robust solution: just take advantage that ir.TakeInit can
handle any node.
Change-Id: Ica05fbf6a8c5be4b11927daf84491a1140da5431
Reviewed-on: https://go-review.googlesource.com/c/go/+/422196
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
This CL changes GOEXPERIMENT=unified to insert implicit conversions
for multi-valued expressions.
Unfortunately, IR doesn't have strong, first-class support for
multi-valued expressions, so this CL takes the approach of spilling
them to temporary variables, which can then be implicitly converted.
This is the same approach taken by walk, but doing it this early does
introduce some minor complications:
1. For select case clauses with comma-ok assignments (e.g., `case x,
ok := <-ch:`), the compiler middle end wants to see the OAS2RECV
assignment is the CommClause.Comm statement. So when constructing
select statements, we need to massage this around a little.
2. The extra temporary variables and assignments skew the existing
inlining heuristics. As mentioned, the temporaries/assignments will
eventually be added (and often optimized away again) anyway, but now
they're visible to the inliner. So this CL also kludges the inlining
heuristics in this case to keep things comparable.
Change-Id: I3e3ea756ad92472ebe28bae3963be61ed7684a75
Reviewed-on: https://go-review.googlesource.com/c/go/+/415244
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
CL 395854 made inline pass to not inlining function with shape params,
but pass no shape arguments. This is intended to be the reverse case of
CL 361260.
However, CL 361260 is using wider condition than necessary. Though it
only needs to check against function parameters, it checks whether the
function type has no shape. It does not cause any issue, because
!fn.Type().HasShape() implies !fn.Type().Params().HasShape().
But for the reverse case, it's not true. Function may have shape type,
but has no shape arguments. Thus, we must tighten the condition to
explicitly check against the function parameters only.
Fixes#52907
Change-Id: Ib87e87ff767c31d99d5b36aa4a6c1d8baf32746d
Reviewed-on: https://go-review.googlesource.com/c/go/+/406475
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
CL 395854 made inline pass to not inlining function with shape params,
but pass no shape arguments. But it does not consider the case where
function has shape params, but passing zero arguments. In this case, the
un-safe interface conversion that may be applied to a shape argument can
not happen, so it's safe to inline the function.
Fixes#52907
Change-Id: Ifa7b23709bb47b97e27dc1bf32343d92683ef783
Reviewed-on: https://go-review.googlesource.com/c/go/+/406176
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
This is the same fix as CL 36126, but for the reverse case, function
with shape params but passed no shape arg. The same conversion problem
may occur in this case, see details explanation there.
Fixes#51909Fixes#51925
Change-Id: Ib0c1973c7511d85b4918a252c80060f1864180cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/395854
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
No longer needed now that IR construction uses types2.
Change-Id: If8b7aff80cd8472be7d87fd3a36da911a5df163c
Reviewed-on: https://go-review.googlesource.com/c/go/+/403839
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Now, 1.17 is the least supported version, the compiler always write
type information when exporting function bodies. So we can get rid of
go117ExportTypes constant and all its conditional checking codes.
Change-Id: I9ac616509c30601e94f99426049d814328253395
Reviewed-on: https://go-review.googlesource.com/c/go/+/402974
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This CL exports the existing ir.UintptrKeepAlive via the new directive
//go:uintptrkeepalive. This makes the compiler insert KeepAlives for
pointers converted to uintptr in calls, keeping them alive for the
duration of the call.
//go:uintptrkeepalive requires //go:nosplit, as stack growth can't
handle these arguments (it cannot know which are pointers). We currently
check this on the immediate function, but the actual restriction applies
to all transitive calls.
The existing //go:uintptrescapes is an extension of
//go:uintptrkeepalive which forces pointers to escape to the heap, thus
eliminating the stack growth issue.
This pragma is limited to the standard library.
For #51087
Change-Id: If9a19d484d3561b4219e5539b70c11a3cc09391e
Reviewed-on: https://go-review.googlesource.com/c/go/+/388095
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
[This CL is part of a sequence implementing the proposal #51082.
The design doc is at https://go.dev/s/godocfmt-design.]
Run the updated gofmt, which reformats doc comments,
on the main repository. Vendored files are excluded.
For #51082.
Change-Id: I7332f099b60f716295fb34719c98c04eb1a85407
Reviewed-on: https://go-review.googlesource.com/c/go/+/384268
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This patch revises the fix for issue 46234, fixing a bug that was
accidentally introduced by CL 320913. When inlining a chunk of code
with a closure expression, we want to avoid updating the source
positions in the function being closed over, but we do want to update
the position for the ClosureExpr itself (since it is part of the
function we are inlining). CL 320913 unintentionally did away with the
closure expr source position update; here we restore it again.
Updates #46234.
Fixes#49171.
Change-Id: Iaa51bc498e374b9e5a46fa0acd7db520edbbbfca
Reviewed-on: https://go-review.googlesource.com/c/go/+/366494
Trust: Than McIntosh <thanm@google.com>
Trust: Dan Scales <danscales@google.com>
Reviewed-by: Dan Scales <danscales@google.com>
Currently, we rely on a "crawling" step during export to identify
function and method bodies that need to be exported or re-exported so
we can trim out unnecessary ones and reduce build artifact sizes. To
catch cases where we expect a function to be inlinable but we failed
to export its body, we made this condition a fatal compiler error.
However, with generics, it's much harder to perfectly identify all
function bodies that need to be exported; and several attempts at
tweaking the algorithm have resulted in still having failure cases.
So for now, this CL changes a missing inline body into a graceful
failure instead.
Change-Id: I04b0872d0dcaae9c3de473e92ce584e4ec6fd782
Reviewed-on: https://go-review.googlesource.com/c/go/+/361403
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Dan Scales <danscales@google.com>
Don't inline a function fn that has no shape parameters, but is passed
at least one shape arg. This means we must be inlining a non-generic
function fn that was passed into a generic function, and can be called
with a shape arg because it matches an appropriate type parameter. But
fn may include an interface conversion (that may be applied to a shape
arg) that was not apparent when we first created the instantiation of
the generic function. We can't handle this if we actually do the
inlining, since we want to know all interface conversions immediately
after stenciling. So, we avoid inlining in this case.
Fixes#49309.
Change-Id: I7b8ab7b13e58fdb0111db91bc92a91d313f7c2c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/361260
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
CL 357649 fixes inlining labeled FOR/RANGE loops,
we should do same translation for inlined SWITCH's label
Fixes#49145
Change-Id: I9a6f365f57e974271a1eb279b38e81f9b5148788
Reviewed-on: https://go-review.googlesource.com/c/go/+/358315
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
There is already a mechanism using inlgen to rename labels insided
inlined functions so that they are unique and don't clash with loops in
the outer function. This is used for OLABEL and OGOTO. Now that we are
doing inlining of OFOR loops, we need to do this translation for OBREAK,
OCONTINUE, and OFOR. I also added the translation for ORANGE loops, in
anticipation of a CL that will allow inlining of ORANGE for loops.
Fixes#49100
Change-Id: I2ccddc3350370825c386965f4a1e4bc54d3c369b
Reviewed-on: https://go-review.googlesource.com/c/go/+/357649
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Trust: Dan Scales <danscales@google.com>
After CL 349012 and CL 350911, we can fully handle these
labeled statements, so we can allow them when inlining.
Updates #14768
Change-Id: I0ab3fd3f8d7436b49b1aedd946516b33c63f5747
Reviewed-on: https://go-review.googlesource.com/c/go/+/355497
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Dan Scales <danscales@google.com>
Reviewed-by: David Chase <drchase@google.com>
Trust: Dan Scales <danscales@google.com>
The encoding/binary little- and big-endian load and store routines are
frequently used in performance sensitive code. They look fairly complex
to the inliner. Though the routines themselves can be inlined,
code using them typically cannot be.
Yet they typically compile down to an instruction or two
on architectures that support merging such loads.
This change teaches the inliner to treat calls to these methods as cheap,
so that code using them will be more inlineable.
It'd be better to teach the inliner that this pattern of code is cheap,
rather than these particular methods. However, that is difficult to do
robustly when working with the IR representation. And the broader project
of which that would be a part, namely to model the rest of the compiler
in the inliner, is probably a non-starter. By way of contrast, imperfect
though it is, this change is an easy, cheap, and useful heuristic.
If/when we base inlining decisions on more accurate information obtained
later in the compilation process, or on PGO/FGO, we can remove this
and other such heuristics.
Newly inlineable functions in the standard library:
crypto/cipher.gcmInc32
crypto/sha512.appendUint64
crypto/md5.appendUint64
crypto/sha1.appendUint64
crypto/sha256.appendUint64
vendor/golang.org/x/crypto/poly1305.initialize
encoding/gob.(*encoderState).encodeUint
vendor/golang.org/x/text/unicode/norm.buildRecompMap
net/http.(*http2SettingsFrame).Setting
net/http.http2parseGoAwayFrame
net/http.http2parseWindowUpdateFrame
Benchmark impact for encoding/gob (the only package I measured):
name old time/op new time/op delta
EndToEndPipe-8 2.25µs ± 1% 2.21µs ± 3% -1.79% (p=0.000 n=28+27)
EndToEndByteBuffer-8 93.3ns ± 5% 94.2ns ± 5% ~ (p=0.174 n=30+30)
EndToEndSliceByteBuffer-8 10.5µs ± 1% 10.6µs ± 1% +0.87% (p=0.000 n=30+30)
EncodeComplex128Slice-8 1.81µs ± 0% 1.75µs ± 1% -3.23% (p=0.000 n=28+30)
EncodeFloat64Slice-8 900ns ± 1% 847ns ± 0% -5.91% (p=0.000 n=29+28)
EncodeInt32Slice-8 1.02µs ± 0% 0.90µs ± 0% -11.82% (p=0.000 n=28+26)
EncodeStringSlice-8 1.16µs ± 1% 1.04µs ± 1% -10.20% (p=0.000 n=29+26)
EncodeInterfaceSlice-8 28.7µs ± 3% 29.2µs ± 6% ~ (p=0.067 n=29+30)
DecodeComplex128Slice-8 7.98µs ± 1% 7.96µs ± 1% -0.27% (p=0.017 n=30+30)
DecodeFloat64Slice-8 4.33µs ± 1% 4.34µs ± 1% +0.24% (p=0.022 n=30+29)
DecodeInt32Slice-8 4.18µs ± 1% 4.18µs ± 0% ~ (p=0.074 n=30+28)
DecodeStringSlice-8 13.2µs ± 1% 13.1µs ± 1% -0.64% (p=0.000 n=28+28)
DecodeStringsSlice-8 31.9µs ± 1% 31.8µs ± 1% -0.34% (p=0.001 n=30+30)
DecodeBytesSlice-8 8.88µs ± 1% 8.84µs ± 1% -0.48% (p=0.000 n=30+30)
DecodeInterfaceSlice-8 64.1µs ± 1% 64.2µs ± 1% ~ (p=0.173 n=30+28)
DecodeMap-8 74.3µs ± 0% 74.2µs ± 0% ~ (p=0.131 n=29+30)
Fixes#42958
Change-Id: Ie048b8976fb403d8bcc72ac6bde4b33e133e2a47
Reviewed-on: https://go-review.googlesource.com/c/go/+/349931
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
For certain type of method wrappers we used to generate a tail
call. That was disabled in CL 307234 when register ABI is used,
because with the current IR it was difficult to generate a tail
call with the arguments in the right places. The problem was that
the IR does not contain a CALL-like node with arguments; instead,
it contains an OAS node that adjusts the receiver, than an
OTAILCALL node that just contains the target, but no argument
(with the assumption that the OAS node will put the adjusted
receiver in the right place). With register ABI, putting
arguments in registers are done in SSA. The assignment (OAS)
doesn't put the receiver in register.
This CL changes the IR of a tail call to take an actual OCALL
node. Specifically, a tail call is represented as
OTAILCALL (OCALL target args...)
This way, the call target and args are connected through the OCALL
node. So the call can be analyzed in SSA and the args can be passed
in the right places.
(Alternatively, we could have OTAILCALL node directly take the
target and the args, without the OCALL node. Using an OCALL node is
convenient as there are existing code that processes OCALL nodes
which do not need to be changed. Also, a tail call is similar to
ORETURN (OCALL target args...), except it doesn't preserve the
frame. I did the former but I'm open to change.)
The SSA representation is similar. Previously, the IR lowers to
a Store the receiver then a BlockRetJmp which jumps to the target
(without putting the arg in register). Now we use a TailCall op,
which takes the target and the args. The call expansion pass and
the register allocator handles TailCall pretty much like a
StaticCall, and it will do the right ABI analysis and put the args
in the right places. (Args other than the receiver are already in
the right places. For register args it generates no code for them.
For stack args currently it generates a self copy. I'll work on
optimize that out.) BlockRetJmp is still used, signaling it is a
tail call. The actual call is made in the TailCall op so
BlockRetJmp generates no code (we could use BlockExit if we like).
This slightly reduces binary size:
old new
cmd/go 14003088 13953936
cmd/link 6275552 6271456
Change-Id: I2d16d8d419fe1f17554916d317427383e17e27f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/350145
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: David Chase <drchase@google.com>
If the condition is a bool constant, there's no need to walk both
branches.
Passes toolstash -cmp.
Change-Id: I4ee5e3553ce07c2213efba0d33d869b4a1b57783
Reviewed-on: https://go-review.googlesource.com/c/go/+/347911
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Trust: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
ORANGE is still not inlineable now. This CL is correct only when the range statement is statically dead, and thus not counted during the inline budget check.
If we support range statements in inlining closures in the future, may require additional processing.
Fixes#48033.
Change-Id: I28f5755c28cfa27e41daef9eff2ae332059909bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/345436
Trust: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
The Func.ClosureCalled flag is an optimization used by escape analysis
to detect closures that were directly called, so we know we have
visibility of the result flows. It's not needed by any other phases of
the compiler, so we might as well calculate it within escape analysis
too.
This saves some trouble during IR construction and trying to maintain
the ClosureCalled flag through inlining and copying.
Passes toolstash -cmp.
Change-Id: Ic53cecb7ac439745c0dfba2cd202b9cc40f1e47c
Reviewed-on: https://go-review.googlesource.com/c/go/+/332691
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Trust: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Inlining replaces inlined calls with OINLCALL nodes, and then somewhat
clumsily tries to rewrite these in place without messing up
order-of-evaluation rules.
But handling these rules cleanly is much easier to do during order,
and escape analysis is the only major pass between inlining and
order. It's simpler to teach escape analysis how to analyze OINLCALL
nodes than to try to hide them from escape analysis.
Does not pass toolstash -cmp, but seems to just be line number
changes.
Change-Id: I1986cea39793e3e1ed5e887ba29d46364c6c532e
Reviewed-on: https://go-review.googlesource.com/c/go/+/332649
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Matthew Dempsky <mdempsky@google.com>
Go spec call them "method values", not "partial calls". Note that
we use "OMETHVALUE" (as opposed to "OMETHODVALUE") to be consistent
with "OMETHEXPR".
Change-Id: I1efd985d4b567a1b4b20aeb603eb82db579edbd5
Reviewed-on: https://go-review.googlesource.com/c/go/+/330837
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
CL 330671 move rewriting method call to method expression to escape
analysis. This CL move the rewriting up further, into typecheck. It
helps simplify the code for dowstream passes, as they now only have to
deal with OCALLFUNC.
There're two notes:
- For -G=3, we can't rewrite d.M() where d is an instantiated receiver
in transformCall, but let irgen.stencil to rewrite it.
- Escape analysis still have to check for rewriting method calls, as
the devirtualization pass can still generate OCALLMETH.
Does not pass toolstash, since when the export data now contains method
expression calls instead of method calls.
Change-Id: I77465ef04d50dc4efedddca7eb55b3fc9483db0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/330831
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
I initially made NewClosureFunc take an "outerfn *Func" parameter
because I was planning on having it handle closure naming, until
remembering that naming needs to wait until typecheck for noder.
We don't actually need the *Func yet, just to know whether it's
non-nil. So change the parameter to a bool, which simplifies callers a
little.
Change-Id: Ie83ee4a1ed0571ac6d3879ffd8474c6c3c1a9ff9
Reviewed-on: https://go-review.googlesource.com/c/go/+/327450
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
typecheck.tcClosure is complicated with many code flows because all of
its callers setup the closure funcs in slightly different ways. E.g.,
it's non-obvious who's responsible for setting the underlying func's
Sym or adding it to target.Decls, or how to write new code that
constructs a closure without interfering with existing code.
This CL refactors everything to use three common functions in package
ir: NewClosureFunc (which handle creating the Func, Name, and
ClosureExpr and wiring them together), NameClosure (which generates
and assigns its unique Sym), and UseClosure (which handles adding the
Func to target.Decls).
Most IR builders can actually name the closure right away, but the
legacy noder+typecheck path may not yet know the name of the enclosing
function. In particular, for methods declared with aliased receiver
parameters, we need to wait until after typechecking top-level
declarations to know the method's true name. So they're left anonymous
until typecheck.
UseClosure does relatively little work today, but it serves as a
useful spot to check that the code setting up closures got it right.
It may also eventually serve as an optimization point for early
lifting of trivial closures, which may or may not ultimately be
beneficial.
Change-Id: I7da1e93c70d268f575b12d6aaeb2336eb910a6f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/327051
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
This CL reorganizes export writing in preparation for unified IR:
1. It moves dumpexport into noder as noder.WriteExports so that it can
be extended to include unified IR's export data.
2. Adds an "extensions" flag to typecheck.WriteExports to control
whether the compiler-only extension data (e.g., function bodies and
linker symbol info) is included in the exports.
3. It moves the gc.exporter type into typecheck and renames it to
"crawler". The type originated as the implementation of
the (pre-iexport) binary exporter, but since the removal of bexport
it's been relegated to simply crawling the exported functions/bodies
graph to identify which inline bodies need to be included.
4. It changes inline.Inline_Flood into the method crawler.markInlBody.
Inline_Flood doesn't actually have anything to do with the rest of
inlining; its current name and location are just historical quirks.
Passes toolstash -cmp.
Change-Id: I6445e2de9d3ce500a3aded5a8e20b09f46d23dbc
Reviewed-on: https://go-review.googlesource.com/c/go/+/325212
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
This CL refactors mkinlcall by extracting the core InlinedCallExpr
construction code into a new "oldInline" function, and adds a new
"NewInline" hook point that can be overriden with a new inliner
implementation that only needs to worry about the details of
constructing the InlinedCallExpr.
It also moves the delayretvars optimization check into CanInline, so
it's performed just once per inlinable function rather than once for
each inlined call.
Finally, it skips printing the function body about to be inlined (and
updates the couple of regress tests that expected this output). We
already report the inline body as it was saved, and this diagnostic is
only applicable to the current inliner, which clones existing function
body instances. In the unified IR inliner, we'll directly construct
inline bodies from the serialized representation.
Change-Id: Ibdbe617da83c07665dcbda402cc8d4d4431dde2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/323290
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
This CL makes to minor changes motivated by making it easier to make
large-scale changes to the inliner while satisfying toolstash -cmp:
1. When creating inlining variables, make sure to preserve the
AutoTemp flag. This is necessary so that temporary variables
introduced by rewriting f(g()) calls during typecheck stay autotemp
after inlining and are (correctly) omitted from DWARF debugging
information.
2. When sorting variables for stack frame layout, use a stable
sort. This ensures that layout is insensitive to whether deadcode
elimination happens before or after inlining.
Change-Id: I672e752a873c7e16749b9873fd6573607e074309
Reviewed-on: https://go-review.googlesource.com/c/go/+/323011
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
We already have and use FixVariadicCall to normalize non-dotted calls
to variadic functions elsewhere in the compiler to simplify rewriting
of function calls. This CL updates inl.go to use it too.
A couple tests need to be updated to (correctly) expect diagnostics
about "... argument" instead of a slice literal. This is because
inl.go previously failed to set Implicit on the slice literal node.
Change-Id: I76bd79b95ae1f16e3b26ff7e9e1c468f538fd1f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/323009
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
Merge List:
+ 2021-05-25 f22ec51deb doc: add Go 1.17 release note about inlining functions with closures
+ 2021-05-25 8b462d7567 cmd/go: add a -compat flag to 'go mod tidy'
+ 2021-05-24 c89f1224a5 net: verify results from Lookup* are valid domain names
+ 2021-05-24 08a8fa9c47 misc/wasm: ensure correct stack pointer in catch clauses
+ 2021-05-24 32b73ae180 cmd/go: align checks of module path during initialization.
+ 2021-05-24 15d9d4a009 cmd/go: add tests illustrating what happens when Go 1.16 is used in a Go 1.17 main module
+ 2021-05-24 873401df5b cmd/compile: ensure equal functions don't do unaligned loads
+ 2021-05-24 b83610699a cmd/compile: record regabi status in DW_AT_producer
+ 2021-05-24 a22e317220 cmd/compile: always include underlying type for map types
+ 2021-05-24 4356e7e85f runtime: account for spill slots in Windows callback compilation
+ 2021-05-24 52d7033ff6 cmd/go/internal/modload: set the default GoVersion in a single location
+ 2021-05-24 05819bc104 cmd/go/internal/modcmd: factor out a type for flags whose arguments are Go versions
+ 2021-05-22 cca23a7373 cmd/compile: revert CL/316890
+ 2021-05-21 f87194cbd7 doc/go1.17: document changes to net/http package
+ 2021-05-21 217f5dd496 doc: document additional atomic.Value methods
+ 2021-05-21 3c656445f1 cmd/go: in TestScript/mod_replace, download an explicit module path
+ 2021-05-21 76b2d6afed os: document that StartProcess puts files into blocking mode
+ 2021-05-21 e4d7525c3e cmd/dist: display first class port status in json output
+ 2021-05-21 4fb10b2118 cmd/go: in 'go mod download' without args, don't save module zip sums
+ 2021-05-21 4fda54ce3f doc/go1.17: document database/sql changes for Go 1.17
+ 2021-05-21 8876b9bd6a doc/go1.17: document io/fs changes for Go 1.17
+ 2021-05-21 5fee772c87 doc/go1.17: document archive/zip changes for Go 1.17
+ 2021-05-21 3148694f60 cmd/go: remove warning from module deprecation notice printing
+ 2021-05-21 7e63c8b765 runtime: wait for Go runtime to initialize in Windows signal test
+ 2021-05-21 831573cd21 io/fs: added an example for io/fs.WalkDir
+ 2021-05-20 baa934d26d cmd: go get golang.org/x/tools/analysis@49064d23 && go mod vendor
+ 2021-05-20 7c692cc7ea doc/go1.17: document changes to os package
+ 2021-05-20 ce9a3b79d5 crypto/x509: add new FreeBSD 12.2+ trusted certificate folder
+ 2021-05-20 f8be906d74 test: re-enable test on riscv64 now that it supports external linking
+ 2021-05-20 def5360541 doc/go1.17: add release notes for OpenBSD ports
+ 2021-05-20 ef1f52cc38 doc/go1.17: add release note for windows/arm64 port
+ 2021-05-20 bb7495a46d doc/go1.17: document new math constants
+ 2021-05-20 f07e4dae3c syscall: document NewCallback and NewCallbackCDecl limitations
+ 2021-05-20 a8d85918b6 misc/cgo/testplugin: skip TestIssue25756pie on darwin/arm64 builder
+ 2021-05-19 6c1c055d1e cmd/internal/moddeps: use filepath.SkipDir only on directories
+ 2021-05-19 658b5e66ec net: return nil UDPAddr from ReadFromUDP
+ 2021-05-19 15a374d5c1 test: check portable error message on issue46234.go
+ 2021-05-18 eeadce2d87 go/build/constraint: fix parsing of "// +build" (with no args)
+ 2021-05-18 6d2ef2ef2a cmd/compile: don't emit inltree for closure within body of inlined func
+ 2021-05-18 048cb4ceee crypto/x509: remove duplicate import
Change-Id: Ib0442e3555493805f2aa1df26dfd6898df989a37
Replace the raw Node copy with the creation of a new node, and the
copying of the needed flags and fields.
Change-Id: I636bf228ba28c0d5dc25f8366d82379d86ecd731
Reviewed-on: https://go-review.googlesource.com/c/go/+/322189
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Trust: Dan Scales <danscales@google.com>
When inlining functions with closures, ensure that we don't mark the
body of the closure with a src.Pos marker that reflects the inline,
since this will result in the generation of an inltree table for the
closure itself (as opposed to the routine that the func-with-closure
was inlined into).
Fixes#46234.
Change-Id: I348296de6504fc4745d99adab436640f50be299a
Reviewed-on: https://go-review.googlesource.com/c/go/+/320913
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Than McIntosh <thanm@google.com>
This is only changes to comments, so should be fine to go into 1.17.
Change-Id: I01e28dc76b03fb3ca846d976f8ac84bc2acb2ea2
Reviewed-on: https://go-review.googlesource.com/c/go/+/318009
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
The inlsubst already does the same thing for OLABEL, so we must do the
same thing for OGOTO. Otherwise, new inlined OGOTO node will be
associated with non-existed label.
Fixes#45947
Change-Id: I40eef095f57fd3438c38a0b5d9751d5d7ebf759e
Reviewed-on: https://go-review.googlesource.com/c/go/+/316931
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
Within clovar, n.Defn can also be *ir.TypeSwitchGuard. The proper fix
here would be to populate m.Defn and have it filled in too, but we
already leave it nil in inlvar. So for consistency, this CL does the
same in clovar too.
Eventually inl.go should be rewritten to fully respect IR invariants.
Fixes#45743.
Change-Id: I8b38e5d8b2329ad242de97670f2141f713954d28
Reviewed-on: https://go-review.googlesource.com/c/go/+/313289
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
Trust: Dan Scales <danscales@google.com>
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Currently, when copying definition node of an inlined var, we do not
update var Defn field to point to new copied node. That causes all
inlined vars point to the same Defn, and ir.StaticValue can not find
inlined var in the lhs of its definition.
clovar creates new ONAME node for local variables or params of closure
inside inlined function, by copying most of the old node fields. So the
new Node.Defn is not modified, its lhs still refer to old node
instead of new one.
To fix this, we need to do two things:
- In subst.clovar, set a dummy Defn node for inlvar
- During subst.node, when seeing OAS/OAS2 nodes, after substituting, we
check if any node in lhs has the dummy Defn, then set it to the current
OAS/OAS2 node.
Fixes#45606
Change-Id: Ib517b753a7643756dcd61d36deae60f1a0fc53c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/312630
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
For the new export/import of node types, we were just missing setting
the types of the closure variables (which have the same types as the
captured variables) and the OCLOSURE node itself (which has the same
type as the Func node).
Re-enabled inlining of functions with closures.
Change-Id: I687149b061f3ffeec3244ff02dc6e946659077a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/308974
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Include type information on exported function bodies, so that the
importer does not have to re-typecheck the body. This involves
including type information in the encoded output, as well as
avoiding some of the opcode rewriting and other changes that the
old exporter did assuming there would be a re-typechecking pass.
This CL could be considered a cleanup, but is more important than that
because it is an enabling change for generics. Without this CL, we'd
have to upgrade the current typechecker to understand generics. With
this CL, the current typechecker can mostly go away in favor of the
types2 typechecker.
For now, inlining of functions that contain closures is turned off.
We will hopefully resolve this before freeze.
Object files are only 0.07% bigger.
Change-Id: I85c9da09f66bfdc910dc3e26abb2613a1831634d
Reviewed-on: https://go-review.googlesource.com/c/go/+/301291
Trust: Keith Randall <khr@golang.org>
Trust: Dan Scales <danscales@google.com>
Reviewed-by: Dan Scales <danscales@google.com>
