cc1as: change AssemblerInvocation.Triple from std::string to
llvm::Triple, matching upstream LLVM 22 and fixing deprecation warnings
for lookupTarget, createMCRegInfo, createMCAsmInfo, and
createMCSubtargetInfo.
interp: always use ConstNamedStruct in rawValue.toLLVMValue instead of
falling back to ConstStruct for unnamed structs. LLVM 22 uses anonymous
identified struct types where previous versions used literal structs;
ConstStruct creates a literal type that doesn't match, causing an
"initializer type mismatch" panic for globals like fmt.ppFree.
compileopts: add build-tag-guarded feature patching for pre-LLVM 20
(strip +bulk-memory-opt and +call-indirect-overlong from wasm features)
and restructure into three files covering all LLVM version ranges.
targets: strip redundant negative features from RISC-V target JSONs
(esp32c3, esp32c6, fe310, k210, riscv-qemu, tkey). These ~190 negative
features per target listed every extension LLVM knows about that the
target doesn't use, but LLVM disables them by default. They became stale
across LLVM versions, causing "not a recognized feature" warnings. Keep
only positive features and -relax.
Signed-off-by: deadprogram <ron@hybridgroup.com>
Written entirely by Claude (Anthropic's Claude Code), at the request of
and under the direction of dgryski, on the dgryski/llvm23 branch (LLVM
22 support, in preparation for the eventual LLVM 23 release; the
corresponding go-llvm branch of the same name adds the matching
binding support this depends on).
Found while running real-world Go test suites through a tinygo built
against LLVM 22 (as a smoke test of the earlier captures(none)/
nocapture and lifetime-intrinsic fixes on this branch): `tinygo test`
on github.com/dgryski/go-rc5 panicked at compile time with "unknown
value" inside interp.(*runner).getValue, while compiling
(*sync.Once).Do. Bisecting against LLVM 21 (unaffected) narrowed it to
a change introduced in LLVM 22 specifically, unrelated to the earlier
captures/lifetime work.
Root cause: LLVM 22 stopped exposing switch-instruction case values as
regular instruction operands -- only the condition and
destination-block operands remain. interp/compiler.go's `case
llvm.Switch:` handling walked raw operands assuming the old
alternating (value, label) layout, so under LLVM 22 it read a
destination *block* where it expected an integer case value,
eventually panicking deep inside a getValue call it couldn't resolve.
This only manifested on functions actually compiled by the interp
package (TinyGo's compile-time evaluator for global initializers) that
happen to reach a switch-based dispatch, such as sync.Once's
defer-based Do method -- hence it needed a real, moderately complex
package (crypto/cipher via go-rc5) to surface, and never showed up in
smaller smoke tests.
Fix: use the new version-independent Value.SuccessorsCount()/
Value.Successor(i) and Value.GetSwitchCaseValue(i) helpers just added
to go-llvm, instead of raw Operand() indexing.
Also applies the same captures(none)/nocapture golden-IR normalization
already used in transform/transform_test.go and compiler/compiler_test.go
to interp/interp_test.go's separate copy of the same comparator helper,
which was missed in the earlier pass and started failing two of its
subtests once actually run against LLVM 22.
Verified: `tinygo test` on go-rc5 now passes reliably (repeated runs)
against both LLVM 20 (default) and LLVM 22; full transform/compiler/
interp/cgo/goenv test suites pass on both versions. The `builder`
package has pre-existing, environment-related test failures (stale
GOROOT cache, local clang/target-feature drift) confirmed identical
against an unmodified LLVM 20 build, so unrelated to this change.
Instead of referring to an unused global, use a constant value. This is
safe even when using `-gc=none` (since no actual memory gets allocated)
which wasn't the case before. It should also reduce binary size by a few
bytes for most programs.
When the interpreter encountered a load that was out of bounds of the
object, it panicked with "interp: load out of bounds", crashing the
compiler. This can happen for valid Go programs, for example when
dereferencing the pointer returned by unsafe.SliceData on a
zero-capacity slice, which points to a zero-sized object:
package main
import "unsafe"
var p = unsafe.SliceData([]int{})
var v = *p
func main() {}
Return nil for an out-of-bounds load, the same as for an external
global, so the caller defers the load to runtime instead of crashing.
This matches what regular Go does, where the load reads from the
runtime zero-base at runtime.
Fixes#4214
The optimization to skip cloning the destination object on partial
stores when the buffer is already owned by the current memory view
mutated obj.buffer.buf in place. The previous v.buf clone only handled
the case where the store value itself aliased that buffer; it did not
cover the more common case where an earlier partial load had returned
a slice into the same buffer. The in-place store would then corrupt
the previously loaded value, breaking code such as cloudflare/bn256
where the gfP arithmetic loads, computes, and stores within the same
global during package initialization.
Fix this in load instead: when the source object is owned by the
current memory view, copy the loaded slice so the returned value is
independent of the live buffer. The v.buf clone in store is no longer
needed and is removed.
Updates the regression test added in the previous commit to reflect
the corrected behavior.
Adds a third init function to the store testdata: an initial partial
store puts the buffer into the current memory view, then a partial load
is followed by another partial store at the same offset, and finally the
loaded value is written to a separate global.
The expected output captures the current (incorrect) behavior of the
in-place partial store optimization: the loaded value gets corrupted by
the subsequent in-place store. The next commit fixes this.
memoryView.store cloned the entire destination object on every partial
store. Compiling a program that pulls in golang.org/x/text/collate hit
this hard: its generated tables are initialized one element at a time,
and each element store copied the whole global, making interp quadratic
in the table size.
Switch to copy-on-first-write per memory view: clone the object the
first time this view writes to it, then mutate that private copy in
place on later partial stores. Rollback is unaffected because revert
still discards the view's objects wholesale.
Two extra safety tweaks fall out of this:
- Full overwrites now clone the incoming value, so the stored buffer
can never alias state held by the caller.
- Partial in-place stores snapshot the source buffer, so a store
whose source is a load from the same object (overlapping or not)
still sees the pre-store bytes.
Add an interp golden test exercising both the overlapping load/store
case and a cross-object aliased load/store case.
* compiler, runtime, reflect: generate type-specific hash/equal for composite map keys
For map keys that are not trivially binary-comparable, the compiler now
generates type-specific hash and equal functions as LLVM IR instead of
going through the interface+reflection path. This covers comparable
types: strings, floats, complex numbers, interfaces, channels, and
composites containing any mix of these.
Previously, maps with composite keys containing strings or floats
converted the key to interface{}, hashed via reflection, and compared
through interface equality. Now the compiler walks struct fields and
array elements directly, dispatching to the right runtime helper for
each field type and storing keys at their actual type.
Struct keys are always handled field-by-field so padding bytes do not
affect equality or hashing. Blank fields are ignored, matching Go
equality. Generated hash/equal function names use canonical underlying
type structure so structurally identical key types can share generated
functions. Padding zeroing before map operations is no longer needed
because structs no longer use the binary key path.
Also fix reflect map iteration for interface-keyed maps: MapIter.Key
returns an interface Value for map[interface{}] keys instead of
unpacking to the concrete key kind.
* compiler: generate loops for array map key hash/equal
Previously, array key hash and equal functions were unrolled at compile
time, generating one block of IR per element. For large arrays like
[1000]int inside a struct with non-binary fields, this caused code
explosion.
Now, binary-element arrays dispatch directly to hash32/memequal for the
whole array. Non-binary-element arrays generate an LLVM IR loop. The
equal loop short-circuits on the first mismatch.
Small arrays are still unrolled instead of looping, keeping the simple
cases compact.
* reflect: fix at-runtime map issues from review, and more found locally
Maps created through reflect.MakeMap need hash/equal behavior that
matches compiler-created maps. Add hashmapMakeReflect for composite key
types, using runtime closures that reconstruct interface{} values from
raw key bytes and delegate to the interface hash and equality paths.
Interface-keyed maps are already stored as interface values, so use the
existing interface hash/equal helpers directly for those. This keeps
reflect insert, lookup, delete, and compiled lookup paths consistent.
Also fix addressable small values used as interface map keys or
interface map values. loadSmallValue puts small indirect values back in
the pointer-sized interface data field the same way valueInterfaceUnsafe
does.
* compiler, interp, reflect: fix pointer map literals; remove interface fallback
Package-level map literals with pointer keys (both *T and
unsafe.Pointer) crash the compiler: the interp pass panics when trying
to hash pointer data as raw bytes, because pointer values in the interp
memory model are symbolic identities that do not fit in a byte.
Fix this by setting a recoverable error flag instead of panicking. The
interp detects the error after each instruction and defers the map
insert to runtime init code, where real addresses are available for
hashing. This matches how the interp already handles other operations
it cannot evaluate at compile time.
With this fix, unsafe.Pointer can also be classified as a binary map
key, which was the last type requiring the interface-based fallback.
Since all comparable types now use either the binary or the
compiler-generated hash/equal path, remove the interface fallback from
the compiler and reflect packages.
* compiler, transform: always pass hash/equal function pointers to hashmapMakeGeneric
The compiler now always resolves the hash and equal functions at compile
time and passes them directly to hashmapMakeGeneric, instead of passing
an algorithm enum to hashmapMake and resolving at runtime. For string
keys, the runtime hashmapStringPtrHash/hashmapStringEqual functions are
referenced directly. For binary keys, hash32/memequal are referenced.
The old hashmapMake with alg enum is retained for reflect, which still
needs runtime resolution when creating maps dynamically.
The OptimizeMaps transform pass is updated to handle both hashmapMake
and hashmapMakeGeneric, and to recognize hashmapGenericSet in addition
to hashmapBinarySet and hashmapStringSet. The now-unused
hashmapCanGenerateHashEqual helper is removed.
* runtime: store large map keys and values indirectly
When a map key or value exceeds 128 bytes, the bucket now stores a
pointer to separately allocated memory instead of the data inline. This
matches Go's MapMaxKeyBytes/MapMaxElemBytes threshold and prevents
bucket sizes from exploding for large key/value types.
For example, map[[256]byte]int previously used 2128 bytes per bucket
(16 header + 256*8 keys + 8*8 values); now it uses 144 bytes per bucket
(16 header + 8*8 pointers + 8*8 values).
The indirection is fully encapsulated in the runtime via helper
functions. Store the computed key and value slot sizes on the hashmap so
all runtime and reflect paths use the same bucket layout, including
non-indirect keys and values.
Add big-key golden coverage and benchmarks. Make the benchmark vary
enough key bytes to exercise hashing.
* interp: bail out of loops that iterate too many times
The existing loop guard (errLoopUnrolled) only fires when a loop body
emits runtime instructions. Loops that are fully evaluable at compile
time, such as inserting thousands of entries into a map, were not
caught and could hang the compiler.
Add a per-basic-block iteration counter that triggers a recoverable
error (errLoopTooLong) when any block is entered more than 1000 times
in a single function call. This defers the init function to runtime,
which is the same behavior as other interp bailouts.
Profiling showed that 83% of CPU time was spent in GC, caused by
allocation pressure from the interp memory cloning on each map
mutation. The iteration limit avoids this entirely by bailing out
before the quadratic cost becomes significant.
Performance on the reproducer from #2090 (map init with strconv.Itoa):
entries before after
5,000 7.4s 2.1s
10,000 17.5s 2.1s
20,000 48.0s 2.8s
65,536 >180s (OOM) 3.2s
* Add -interp-loop-limit
* reflect: implement method-set based AssignableTo and Implements
Based on the design from #4376 by aykevl.
Fixes#4277, fixes#3580.
Co-authored-by: Ayke van Laethem <aykevanlaethem@gmail.com>
* builder: update expected binary sizes for reflect changes
* Make interface checks similar to invoke, allowing typeImplementsMethodSet and method info to be dropped when reflect is not present
* Add more tests that BigGo reflect tests
* Even more pruning
* Add go/token and net/url to passing tests
* Prune even further, I am less happy with this, though
* Update size test now that we are smaller
* Skip some tests
* elide method lists
* format, oops
* fix tests
* Add a panic, pull out constant to keep in sync
* Add debug info
* Remove code that was leftover from a previous refactor
---------
Co-authored-by: Ayke van Laethem <aykevanlaethem@gmail.com>
Go 1.26 changed Darwin's rawSyscall/Syscall from assembly stubs to Go
wrappers that call variadic rawsyscalln/syscalln. When the interp tries
to evaluate syscall.init() at compile time (which calls Getrlimit →
rawSyscall → rawsyscalln), it encounters a ptrtoint of a function
pointer and fails with 'ptrtoint integer size does not equal pointer
size'.
Make this a recoverable error so the interp reverts the init
interpretation and defers it to runtime, matching the behavior for other
uninterpretable init functions.
The compiler now implements the copy builtin directly instead of calling sliceCopy.
The length is calculated with the llvm.umax.* intrinsics, and the move is performed by llvm.memmove.*.
Both of these operations are easily understood by LLVM's optimization passes.
The type's alignment is also provided to llvm.memmove.*, which is useful when rewriting the move.
Interp no longer needs to reimplement sliceCopy.
Some edge case handling was implemented by sliceCopy but not llvm.memmove.*/llvm.memcpy.*.
I copied this over, so copies of external slices should work now.
Volatile moves/copies are now run at runtime by interp.
There is a 4-byte size increase due to some confusing length logic in sendUSBPacket.
I will look at sendUSBPacket in a future PR.
When debugging is enabled for interp, print better errors in a specific
case.
Before:
!! revert because of error: interp: unsupported instruction (to be emitted at runtime)
After:
!! revert because of error: /usr/local/go1.24.0/src/regexp/syntax/parse.go:927:27: interp: unsupported instruction (to be emitted at runtime)
So this adds error location information, which can be quite useful.
This switches the Espressif fork from LLVM 19 to LLVM 20, so we can use
the improvements made between those LLVM versions. It also better aligns
with the system-LLVM build method, which currently also defaults to LLVM
20.
Note that this disables the machine outliner for RISC-V. It appears
there's a bug in there somewhere, with the machine outliner enabled the
crypto/elliptic package tests fail with -target=riscv-qemu.
This should ideally be investigated and reported upstream.
Previously, if a function couldn't be interpreted in the interp pass, it
would just be run at runtime and the parameters would be marked as
potentially being modified. However, this is incorrect: the function
itself can also modify memory. So the function itself also needs to be
marked (recursively).
This fixes a number of regressions while upgrading to Go 1.24.
This results in a significant size regression that is mostly worked
around in the next commit.
Use the alignment from the align attribute of the runtime.alloc call.
This is going to be a more accurate alignment, and is typically smaller
than the default.
The interp package was assuming that all targets were little-endian. But
that's not true: we now have a big-endian target (GOARCH=mips).
This fixes the interp package to use the appropriate byte order for a
given target.
TODO: Remove the go.mod/go.sum in internal/tools once doing so doesn't break CI (e.g. once we drop support for go 1.19)
* builder/cc1as.h: fix typo found by 'make spell'
* GNUmakefile: remove exception for inbetween, fix instance now found by 'make spell'
* GNUmakefile: remove exception for programmmer, fix instance now found by 'make spell'
* go.mod: use updated misspell. GNUmakefile: add spellfix target, use it.
* ignore directories properly when invoking spellchecker.
* make spell: give internal/tools its own go.mod, as misspell requires newer go
* make lint: depend on tools and run the installed revive
(which was perhaps implied by the change that added revive to internal/tools,
but not required in GNUmakefile until we gave internal/tools its own temporary go.mod)
* .github: now that 'make spell' works well, run it from CI
* GNUmakefile: make spell now aborts if it finds misspelt words, so what it finds doesn't get lost in CI logs
* GNUmakefile: tools: avoid -C option on go generate to make test-llvm15-go119 circleci job happy, see
https://cs.opensource.google/go/go/+/2af48cbb7d85e5fdc635e75b99f949010c607786
* internal/tools/go.mod: fix format of go version to leave out patchlevel, else go complains.
getelementptr offsets are signed, not unsigned. Yet they were used as
unsigned integers in interp.
Somehow this worked most of the time, until finally there was some code
that did a getelementptr with a negative index.
The old LLVM pass manager is deprecated and should not be used anymore.
Moreover, the pass manager builder (which we used to set up a pass
pipeline) is actually removed from LLVM entirely in LLVM 17:
https://reviews.llvm.org/D145387https://reviews.llvm.org/D145835
The new pass manager does change the binary size in many cases: both
growing and shrinking it. However, on average the binary size remains
more or less the same.
This is needed as a preparation for LLVM 17.
This is a big change: apart from removing LLVM 14 it also removes typed
pointer support (which was only fully supported in LLVM up to version
14). This removes about 200 lines of code, but more importantly removes
a ton of special cases for LLVM 14.
The old traceback would look like this:
# internal/godebug
/usr/local/go/src/internal/godebug/godebug.go:101:11: interp: test
call <2> 0 <3> 0
traceback:
/usr/local/go/src/internal/godebug/godebug.go:101:11:
call <2> 0 <3> 0
/usr/local/go/src/internal/godebug:
call <1> 0
With this patch, it looks like this:
# io/fs
/usr/local/go/src/io/fs/fs.go:144:45: interp: test
%0 = load %runtime._interface, ptr @"internal/oserror.ErrInvalid", align 8, !dbg !316
traceback:
/usr/local/go/src/io/fs/fs.go:144:45:
%0 = load %runtime._interface, ptr @"internal/oserror.ErrInvalid", align 8, !dbg !316
/usr/local/go/src/io/fs/fs.go:137:28:
%0 = call %runtime._interface @"io/fs.errInvalid"(ptr undef), !dbg !317
For developers (like me) who are familiar with LLVM, this is probably
easier to read. For users, I'm not sure: the instructions have quite a
lot of distracting fluff in them. But at least it contains the function
names that are called (which are not currently present in the old
traceback).
...that said, having the LLVM instructions in a bug report is probably
going to be easier for people who are familar with LLVM.
This was bug https://github.com/tinygo-org/tinygo/issues/3890.
See the diff for details. Essentially, it means that `copy` in the
interpreter was copying data that wasn't known yet, or copying data into
a slice that could be read externally after the interp pass.
If a pointer value was xor'ed with a value other than 0, it would not
have been run at runtime but instead would fall through to the generic
integer operations. This would likely result in a "cannot convert
pointer to integer" panic.
This commit fixes this subtle case.
This is a big commit that changes the way runtime type information is stored in
the binary. Instead of compressing it and storing it in a number of sidetables,
it is stored similar to how the Go compiler toolchain stores it (but still more
compactly).
This has a number of advantages:
* It is much easier to add new features to reflect support. They can simply
be added to these structs without requiring massive changes (especially in
the reflect lowering pass).
* It removes the reflect lowering pass, which was a large amount of hard to
understand and debug code.
* The reflect lowering pass also required merging all LLVM IR into one
module, which is terrible for performance especially when compiling large
amounts of code. See issue 2870 for details.
* It is (probably!) easier to reason about for the compiler.
The downside is that it increases code size a bit, especially when reflect is
involved. I hope to fix some of that in later patches.
This implements the block-based GC as a partially precise GC. This means
that for most heap allocations it is known which words contain a pointer
and which don't. This should in theory make the GC faster (because it
can skip non-pointer object) and have fewer false positives in a GC
cycle. It does however use a bit more RAM to store the layout of each
object.
Right now this GC seems to be slower than the conservative GC, but
should be less likely to run out of memory as a result of false
positives.
This makes it much easier to read the value at runtime, as pointer
indices are naturally little endian. It should not affect anything else
in the program.
These instructions sometimes pop up in LLVM 15, but they never occured
in LLVM 14. Implementing them is relatively straightforward: simply
generalize the code that already exists for llvm.ICmp interpreting.
A number of llvm.Const* functions (in particular extractvalue and
insertvalue) were removed in LLVM 15, so we have to use a builder
instead. This builder will create the same constant values, it simply
uses a different API.
deadprogram
Damian Gryski
Jake Bailey
Ayke van Laethem
iamrajiv
Nia Waldvogel
Dan Kegel
dkegel-fastly
Marco Manino
Kenneth Bell
Elliott Sales de Andrade
shivay