Add compiler/testdata/go1.27.go, gated behind Go >= 1.27 (the version
that promoted generic methods out of the GenericMethods experiment),
covering both fixes from the previous two commits:
- genericMethod has a regular method and a generic method (its own
type parameter); boxing it into an interface must only include the
regular method in the runtime method set instead of panicking in
getTypeCodeName.
- onlyGenericMethod's sole method is generic, so its type code must
have hasMethodSet == false and no methodSet field at all, not an
empty one.
Verified this test panics on the pre-fix compiler/interface.go and
passes after it.
hasMethodSet was computed from the raw, unfiltered method set length
(ms.Len() != 0), before generic methods were excluded from numMethods
and the method set value. For a type whose only method is generic
(e.g. "func (t T) M[X int](n X) X"), this left hasMethodSet true even
though the actual (filtered) method set is empty, causing an
unnecessary empty method-set global and methodSet field to be emitted
for that type's type descriptor.
Compute hasMethodSet from the same filtered loop that produces
numMethods, so it's true only when at least one non-generic method
exists.
Go 1.27 promoted "generic methods" (methods with their own type
parameters, independent of any type parameters on the receiver) out
of the GenericMethods experiment, e.g.:
func (r *Rand) N[Int intType](n Int) Int
Boxing a value of a type with such a method into an interface caused
tinygo to panic while building the runtime type's method table:
getTypeCode -> getMethodSetValue -> getTypeCodeName
getTypeCodeName's type switch has no case for *types.TypeParam, and a
generic method's Signature carries the method's own type parameters
in its parameter/result types (e.g. "Int" above), so encoding it into
a type code name panicked with "unknown type: <param name>".
This affected any package using math/rand/v2.Rand.N, including much of
the math/rand/v2 test suite, since printing or otherwise boxing a
*Rand into an interface is common.
Upstream reflect deliberately excludes generic methods from
Type.NumMethod()/Type.Method() (they can't be instantiated implicitly,
and a generic method can never satisfy an interface method), so mirror
that behavior: add isGenericMethod, which checks whether a method's
Signature has its own type parameters, and skip such methods when
building the runtime method count, method set value, and method set
in compiler/interface.go.
Fixes a panic isolated to:
package main
type T struct{}
func (t T) M[X int](n X) X { return n }
func main() {
var t T
var i interface{} = t
_ = i
}
Goexit cannot run deferred functions on architectures without panic
unwinding. Query runtime.supportsRecover before calling it so unsupported
targets report an error instead of leaving the test runner blocked forever.
LLVM ComputeValueVTs recursively expands arrays and structs into one
value type per scalar leaf. SelectionDAG call lowering allocates data
structures proportional to this count, which makes very large values
exhaust memory or crash LLVM.
Count scalar leaves and use pointers for internal parameters and results
when the count exceeds 1024. A result pointer is the first parameter,
and aggregate parameters point to read-only memory. Exported function
types are unchanged.
Keep these SSA values in memory and copy them with memcpy when needed.
Handle calls, interfaces, maps, channels, selects, defers, goroutines,
phis, and multiple results. Update the expected compiler IR and re-enable
the native compress/flate tests.
Add compiler coverage for large aggregate parameters and results,
including function values, interfaces, maps, channels, selects, deferred
calls, goroutines, phis, and multiple results.
createInstruction records each LLVM value in locals, emits stack object
tracking, and constructs function returns inline.
Move these operations to setValue and createReturn. This keeps the
instruction switch from having to know how an SSA value or function
result is emitted.
createFunctionCall and createGo currently lower arguments before they
determine whether the call is direct, an interface invoke, or through a
function value.
Resolve the callee, function type, and context first, then append the
arguments in the same order as before. This does not change the
generated LLVM IR.
getFunction and getLLVMFunctionType duplicate the construction of LLVM
result types for functions with zero, one, or multiple results.
Move this code to getLLVMResultType. Also split the existing parameter
expansion code into expandDirectFormalParamType so callers can request
the current flattened parameter types directly.
Map, channel, index, and goroutine lowering each create allocas, store
SSA values into them, load results, and emit lifetime intrinsics.
Add helpers for these operations and use runtimeValueResult for runtime
calls that write a value and an optional comma-ok result. The generated
LLVM IR is unchanged.
createDefer builds an LLVM struct containing the deferred function and
its arguments. createRunDefers separately reconstructs the same struct
type before loading the fields.
Keep each LLVM value together with its type while building a deferred
call record, and load all argument fields through one helper. This
removes the duplicate lists of field types and field loads.
Written entirely by Claude (Anthropic's Claude Code), at the request of
and under the direction of dgryski, as part of an effort to get TinyGo
building against upcoming LLVM releases (this branch currently targets
LLVM 22, verified against real LLVM 22.1.8; a corresponding go-llvm
branch of the same name adds the matching binding support).
LLVM 21 replaced the boolean 'nocapture' enum attribute with the more
expressive 'captures' int attribute, where captures(none) (value 0) is
the equivalent of the old nocapture. This matters for
transform.OptimizeAllocs, which relies on reading this attribute for
its interprocedural escape analysis, and for compiler/symbol.go, which
emits it on a number of runtime/generated functions. Confirmed
empirically (via `opt -passes=function-attrs`) that the cutoff is
LLVM 20 emits/expects nocapture, LLVM 21+ emits/expects
captures(none). Since TinyGo must keep working with LLVM 20, both
sites now go through new version-gated helpers in
compiler/llvmutil (NoCaptureAttrName/IsNoCapture) rather than switching
unconditionally.
Also fixes a second, unrelated but load-bearing break found while
testing against LLVM 22: llvm.lifetime.start/end dropped their i64
size argument (confirmed via `opt -passes=verify`, the cutoff here is
one version later, at LLVM 22). compiler/llvmutil now builds the
right call signature based on version.
Adds llvm21 and llvm22 build-tag config files to the cgo package,
which parses cgo fragments via libclang and had never been updated
past LLVM 20 even though the compiler package itself already gained
LLVM 21 support previously -- a latent gap that would have caused a
version mismatch between the cgo preprocessor and the rest of the
compiler when building with -tags llvm21 or llvm22.
Finally, updates the golden-IR test comparators in
transform/transform_test.go and compiler/compiler_test.go to
normalize a few cosmetic LLVM 21/22 output differences (the
captures(none) rename/reordering, a new 'nocreateundeforpoison'
intrinsic attribute, and the lifetime intrinsic arity change) so a
single golden file continues to match output from either LLVM
version.
Verified by building a full (non-byollvm) tinygo binary against real
LLVM 22.1.8 and running a compiled Go program end-to-end (exercising
OptimizeAllocs' stack-allocation path), and by running the
transform/compiler/cgo test suites against both LLVM 20 (default) and
LLVM 22.
Not yet addressed: the byollvm embedded-clang/lld build path hits
separate, unrelated Clang C++ API breakage against LLVM 22
(DiagnosticOptions reference-to-pointer change, missing headers) --
that is a larger follow-up effort.
The interface lowering pass used a separate formatter for method
signatures. Same-named local aliases could therefore make distinct generic
methods compare equal. Reuse getTypeCodeName so interface checks preserve
type identity.
x/tools SSA names and go/types strings can preserve the source spelling of
type arguments, so aliases can make distinct instances collide. Use the
canonical type encoding for function names, synthetic local type owners,
instantiated named types, and method sets.
Add cases where same-named local aliases instantiate generic functions,
local types, and methods with float32 and float64. Record the current
collisions and incorrect results.
Go 1.27 introduced the //go:linknamestd directive, a standard-library
variant of //go:linkname that does not require importing "unsafe". The
iter package switched to it for referencing runtime.newcoro and
runtime.coroswitch, which caused "linker could not find symbol
iter.newcoro / iter.coroswitch" errors when building with Go 1.27.
Handle //go:linknamestd the same as //go:linkname, bypassing the unsafe
import requirement, and add test coverage in the pragma compiler test.
Signed-off-by: deadprogram <ron@hybridgroup.com>
Modern golang.org/x/sys/unix (v0.36+) declares its linknames
detached from function declarations, e.g.:
func syscall_syscall(...)
//go:linkname syscall_syscall syscall.syscall
TinyGo's pragma parser only inspected function doc comments and
therefore missed these, producing link errors like:
undefined symbol: _golang.org/x/sys/unix.syscall_syscall
Extend parsePragmas to also walk the enclosing *ast.File's
free-standing comments for //go:linkname directives matching the
function's name. Function-attached directives still take
precedence. The existing 'unsafe' import gate is preserved.
Fixes#4395, #5365
Treat panicState as a bitmask so a recovered panic during Goexit
can clear the panic bit while preserving the pending Goexit bit.
This removes the separate deferFrame Goexit field and restores the
previous defer frame size.
Keep a pending Goexit separate from the active panic state so recovered
deferred panics do not cancel the original Goexit unwind. Goexit should
also ignore -panic=trap, since it is not a panic.
Wire testing.FailNow and SkipNow through runtime.Goexit, and run tests
and benchmarks in goroutines. This lets Goexit terminate the user
function without skipping test bookkeeping. Add Goexit/recover coverage
and enable crypto/ecdh in the Linux stdlib test list.
Instead of referring to an unused global, use a constant value. This is
safe even when using `-gc=none` (since no actual memory gets allocated)
which wasn't the case before. It should also reduce binary size by a few
bytes for most programs.
I think this failed in the past, but presumably those failures have been
fixed by now. So these intrinsics can now be used.
Using these intrinsics instead of the native Go implementations helps
LLVM to reason about them: it can for example evaluate the value at
compile time or do optimizations like convert
`float32(math.Sin(float64(x)))` into a 32-bit sin operation. If the math
operation is not available on the target platform the C library
implementation will be used instead.
* compiler: make getTypeCodeName a method on compilerContext
* testdata: add regression tests for function-local named types
* compiler: disambiguate function-local named types
* Some PR feedback
Change hashmap set operations and channel send/close from
createRuntimeCall to createRuntimeInvoke. This places a setjmp
checkpoint before each call, allowing runtimePanicAt to safely
longjmp when these operations panic.
Emit fault checkpoints around compiler-generated runtime assertions so
runtimePanicAt can unwind through the existing defer/recover machinery
instead of aborting. This lets panics from bounds checks, type checks,
and other compiler-inserted runtime checks be recovered by deferred
functions.
Mark functions that call recover as noinline. Inlining such a function
into a deferred closure can make recover observe the wrong call context
and report success when it should return nil.
Addresses tinygo-org/tinygo issues 2759 and 3510.
This means runtime.alloc isn't allocated every time but instead only
when it is actually needed. It also just cleans up the compiler code a
little bit, and makes the next commit a bit nicer.
* compiler, runtime, reflect: generate type-specific hash/equal for composite map keys
For map keys that are not trivially binary-comparable, the compiler now
generates type-specific hash and equal functions as LLVM IR instead of
going through the interface+reflection path. This covers comparable
types: strings, floats, complex numbers, interfaces, channels, and
composites containing any mix of these.
Previously, maps with composite keys containing strings or floats
converted the key to interface{}, hashed via reflection, and compared
through interface equality. Now the compiler walks struct fields and
array elements directly, dispatching to the right runtime helper for
each field type and storing keys at their actual type.
Struct keys are always handled field-by-field so padding bytes do not
affect equality or hashing. Blank fields are ignored, matching Go
equality. Generated hash/equal function names use canonical underlying
type structure so structurally identical key types can share generated
functions. Padding zeroing before map operations is no longer needed
because structs no longer use the binary key path.
Also fix reflect map iteration for interface-keyed maps: MapIter.Key
returns an interface Value for map[interface{}] keys instead of
unpacking to the concrete key kind.
* compiler: generate loops for array map key hash/equal
Previously, array key hash and equal functions were unrolled at compile
time, generating one block of IR per element. For large arrays like
[1000]int inside a struct with non-binary fields, this caused code
explosion.
Now, binary-element arrays dispatch directly to hash32/memequal for the
whole array. Non-binary-element arrays generate an LLVM IR loop. The
equal loop short-circuits on the first mismatch.
Small arrays are still unrolled instead of looping, keeping the simple
cases compact.
* reflect: fix at-runtime map issues from review, and more found locally
Maps created through reflect.MakeMap need hash/equal behavior that
matches compiler-created maps. Add hashmapMakeReflect for composite key
types, using runtime closures that reconstruct interface{} values from
raw key bytes and delegate to the interface hash and equality paths.
Interface-keyed maps are already stored as interface values, so use the
existing interface hash/equal helpers directly for those. This keeps
reflect insert, lookup, delete, and compiled lookup paths consistent.
Also fix addressable small values used as interface map keys or
interface map values. loadSmallValue puts small indirect values back in
the pointer-sized interface data field the same way valueInterfaceUnsafe
does.
* compiler, interp, reflect: fix pointer map literals; remove interface fallback
Package-level map literals with pointer keys (both *T and
unsafe.Pointer) crash the compiler: the interp pass panics when trying
to hash pointer data as raw bytes, because pointer values in the interp
memory model are symbolic identities that do not fit in a byte.
Fix this by setting a recoverable error flag instead of panicking. The
interp detects the error after each instruction and defers the map
insert to runtime init code, where real addresses are available for
hashing. This matches how the interp already handles other operations
it cannot evaluate at compile time.
With this fix, unsafe.Pointer can also be classified as a binary map
key, which was the last type requiring the interface-based fallback.
Since all comparable types now use either the binary or the
compiler-generated hash/equal path, remove the interface fallback from
the compiler and reflect packages.
* compiler, transform: always pass hash/equal function pointers to hashmapMakeGeneric
The compiler now always resolves the hash and equal functions at compile
time and passes them directly to hashmapMakeGeneric, instead of passing
an algorithm enum to hashmapMake and resolving at runtime. For string
keys, the runtime hashmapStringPtrHash/hashmapStringEqual functions are
referenced directly. For binary keys, hash32/memequal are referenced.
The old hashmapMake with alg enum is retained for reflect, which still
needs runtime resolution when creating maps dynamically.
The OptimizeMaps transform pass is updated to handle both hashmapMake
and hashmapMakeGeneric, and to recognize hashmapGenericSet in addition
to hashmapBinarySet and hashmapStringSet. The now-unused
hashmapCanGenerateHashEqual helper is removed.
* runtime: store large map keys and values indirectly
When a map key or value exceeds 128 bytes, the bucket now stores a
pointer to separately allocated memory instead of the data inline. This
matches Go's MapMaxKeyBytes/MapMaxElemBytes threshold and prevents
bucket sizes from exploding for large key/value types.
For example, map[[256]byte]int previously used 2128 bytes per bucket
(16 header + 256*8 keys + 8*8 values); now it uses 144 bytes per bucket
(16 header + 8*8 pointers + 8*8 values).
The indirection is fully encapsulated in the runtime via helper
functions. Store the computed key and value slot sizes on the hashmap so
all runtime and reflect paths use the same bucket layout, including
non-indirect keys and values.
Add big-key golden coverage and benchmarks. Make the benchmark vary
enough key bytes to exercise hashing.
* reflect: implement method-set based AssignableTo and Implements
Based on the design from #4376 by aykevl.
Fixes#4277, fixes#3580.
Co-authored-by: Ayke van Laethem <aykevanlaethem@gmail.com>
* builder: update expected binary sizes for reflect changes
* Make interface checks similar to invoke, allowing typeImplementsMethodSet and method info to be dropped when reflect is not present
* Add more tests that BigGo reflect tests
* Even more pruning
* Add go/token and net/url to passing tests
* Prune even further, I am less happy with this, though
* Update size test now that we are smaller
* Skip some tests
* elide method lists
* format, oops
* fix tests
* Add a panic, pull out constant to keep in sync
* Add debug info
* Remove code that was leftover from a previous refactor
---------
Co-authored-by: Ayke van Laethem <aykevanlaethem@gmail.com>
Go 1.26 changed all Windows syscall wrappers in zsyscall_windows.go
to use SyscallN instead of fixed-argument Syscall/Syscall6/etc. The
SyscallN function now has a body that calls an unexported syscalln
function (provided by runtime via //go:linkname).
TinyGo's existing createSyscall compiler builtin used call.Args[2:]
to extract syscall arguments, but for variadic SyscallN the SSA
representation passes args as a slice value (not individual args),
causing call.Args[2:] to be empty -- resulting in zero arguments
being passed to Windows API calls and 0xc0000005 access violations.
Fix this by:
1. Excluding syscall.SyscallN from builtin interception, letting
Go 1.26's function body compile normally (it calls syscalln)
2. Adding a new createSyscalln compiler builtin that intercepts
syscall.syscalln and correctly handles the variadic slice:
- Generates a switch on the arg count n (0-18 cases)
- Each case loads args from the slice via GEP/Load
- Wraps calls with SetLastError(0)/GetLastError() as before
- Handles i386 stdcall conventions
3. Adding runtime stubs for both Go versions:
- go1.26: syscall.syscalln stub (body intercepted by compiler)
- pre-go1.26: syscall.SyscallN stub (linker satisfaction)
This simplifies the process of constructing and encoding layout bitmaps.
Instead of creating big integers and merging them, we can create a pre-sized bitmap and set positions within it.
This also changes the encoding logic to allow larger layouts to be encoded inline.
We would previously not encode a layout inline unless the size was less than the width of the data field.
This is overly conservative.
A layout can be encoded inline as long as:
1. The size fits within the size field.
2. All set bits in the bitmap fit into the data field.
The compiler now implements the copy builtin directly instead of calling sliceCopy.
The length is calculated with the llvm.umax.* intrinsics, and the move is performed by llvm.memmove.*.
Both of these operations are easily understood by LLVM's optimization passes.
The type's alignment is also provided to llvm.memmove.*, which is useful when rewriting the move.
Interp no longer needs to reimplement sliceCopy.
Some edge case handling was implemented by sliceCopy but not llvm.memmove.*/llvm.memcpy.*.
I copied this over, so copies of external slices should work now.
Volatile moves/copies are now run at runtime by interp.
There is a 4-byte size increase due to some confusing length logic in sendUSBPacket.
I will look at sendUSBPacket in a future PR.
This fixes two edge cases for min/max:
min/max(+0.0, -0.0) = -0.0, +0.0
min/max(number, NaN) = NaN, NaN
The compare and select method does not work here.
I switched to the llvm.minimum/llvm.maximum intrinsics which match the intended behavior.
The integer min/max were also swapped over to using intrinsics.
The compare and select path is now only used by strings.
Writing the pointer of a buffer to memory-mapped I/O will normally cause
it to escape, which forces the compiler to heap-allocate the buffer. But
we do know how long the value stays alive, so we can tell the compiler
to keep it alive exactly until it is not needed anymore - and tell it to
not treat the pointer-to-uintptr cast as escaping.
The compiler needs to know whether a defer is in a loop to determine whether to allocate stack or heap memory.
Previously, this performed a DFS of the CFG every time a defer was found.
This resulted in time complexity jointly proportional to the number of defers and the number of blocks in the function.
Now, the compiler will instead use Tarjan's strongly connected components algorithm to find cycles in linear time.
The search is performed lazily, so this has minimal performance impact on functions without defers.
In order to implement Tarjan's SCC algorithm, additional state needed to be attached to the blocks.
I chose to merge all of the per-block state into a single slice to simplify memory management.
See: https://github.com/tinygo-org/tinygo/issues/4959
This makes sure to emit an error instead of generating inline assembly,
which leads to hard-to-debug linker errors. Instead, it will now produce
errors like the following:
# golang.org/x/sys/unix
../../../../pkg/mod/golang.org/x/sys@v0.0.0-20220722155257-8c9f86f7a55f/unix/affinity_linux.go:20:23: system calls are not supported: target emulates a linux/arm system on wasm32
../../../../pkg/mod/golang.org/x/sys@v0.0.0-20220722155257-8c9f86f7a55f/unix/fcntl.go:17:29: system calls are not supported: target emulates a linux/arm system on wasm32
../../../../pkg/mod/golang.org/x/sys@v0.0.0-20220722155257-8c9f86f7a55f/unix/fcntl.go:32:24: system calls are not supported: target emulates a linux/arm system on wasm32
- Do not use make([]byte, ...) to allocate, instead call the allocator
directly with a nil (undefined) layout. This makes sure the precise
GC will scan the contents of the allocation, since C could very well
put pointers in there.
- Simplify the map to use the pointer as the key and the size as the
value, instead of storing the slices directly in the map.
Jake Bailey
Damian Gryski
Pat Whittingslow
Nia Waldvogel
deadprogram
Evan Wies
Ayke van Laethem