* all: clean up code with Go 1.24+ in mind
* all: more old go cleanup
* all: even remove rand_fastrand64
* runtime: revert rand changes
* runtime: re-drop rand_fastrand64
* runtime,syscall,internal/poll,os: wasip1 poll_oneoff scheduler integration + net.FileListener
On wasip1 today every syscall.Read/Write blocks the entire wasm module — the
cooperative scheduler invokes poll_oneoff only for sleep/timer wakeups, and
there's no path from the net package to a working TCP server. This change
fixes both: it threads poll_oneoff through the scheduler's idle path so a
goroutine doing FD I/O parks instead of blocking the module, and it provides
enough internal/poll / os / syscall surface that upstream Go's
net.FileListener / net.FileConn works on a host-pre-opened TCP socket.
* runtime: keep scheduler_cooperative idle-wait calls direct
TestBinarySize/hifive1b/examples/echo regressed by 32 bytes after the
previous commit routed the scheduler's idle wait through a
schedulerIdleWait helper. The extra call frame + branch landed on every
non-wasip1 cooperative target, where the original direct sleepTicks /
waitForEvents calls compile to a single inlined call.
I was seeing bad error messages... but only some of the time. So I went
digging. Eventually I traced it to this one bug, probably introduced in
https://github.com/tinygo-org/tinygo/pull/5290.
This is not just a visual issue, this is actually a bug in TinyGo that
will result in non-reproducible binaries and random behavior
differences.
When a string variable already had a source-level default value, the -ldflags
"-X" flag was silently ignored and the variable remained the default value
at runtime.
Fix by stripping the InitOrder entry for each -X variable before
LoadSSA() is called. With no entry in InitOrder, go/ssa emits no init
store, so the global stays zero-valued in the IR. makeGlobalsModule
still injects the actual -X values at final link time.
The -X values remain out of the per-package build cache with only the
variable names appearing in the cache key.
Signed-off-by: deadprogram <ron@hybridgroup.com>
Go 1.26 added a CPU jitter-based SP 800-90B entropy source for FIPS 140-3
compliance (crypto/internal/entropy/v1.0.0). It declares a 32 MiB global
ScratchBuffer ([1<<25]byte) used for memory access timing noise. On systems
with virtual memory this stays in .noptrbss and is lazily paged, but on
baremetal targets it becomes static RAM, causing fatal overflow (e.g.
pybadge with 192 KB SRAM reports 33 MB overflow).
Since FIPS jitter entropy is never used on TinyGo targets (fips140.Enabled
is always false, so drbg.Read falls through to sysrand.Read), provide a
TinyGo overlay that replaces ScratchBuffer with [0]byte and stubs out all
entropy functions with panics.
Go 1.26 added SWAR optimizations to unicode/utf8 that use:
const ptrSize = 4 << (^uintptr(0) >> 63)
const hiBits = 0x8080808080808080 >> (64 - 8*ptrSize)
This formula only distinguishes 32-bit and 64-bit architectures.
On AVR (16-bit uintptr), ptrSize computes as 4, and hiBits becomes
0x80808080 (2155905152) which overflows the 16-bit uintptr type.
Fix by providing a patched unicode/utf8 overlay for Go 1.26+ that
uses a ptrSize formula handling all three sizes (16/32/64-bit):
const ptrSize = 1 << (^uintptr(0)>>15&1 + ^uintptr(0)>>31&1 + ^uintptr(0)>>63&1)
This evaluates to 2 on AVR, 4 on 32-bit, and 8 on 64-bit. The
word() helper also gains a ptrSize==2 case for 16-bit loads.
The "all:" prefix in //go:embed directives instructs the compiler to
include hidden files (starting with "." or "_") when embedding a
directory. Previously, the prefix was passed literally to path.Match,
which would never match.
Introduce an embedPattern struct that parses and validates the pattern
at construction time via newEmbedPattern(). The "all:" prefix is
stripped once and stored along with the glob pattern, avoiding repeated
string prefix checks. Pattern validation is absorbed into the
constructor, guaranteeing that any embedPattern value is valid and
eliminating the need for separate validation loops.
Fixes embedding with patterns like "//go:embed all:static".
Fixes#4884 by upgrading the ssa package.
The fix is in v0.26.0, which also bumps the minimum Go to 1.22. The
latest x/tools module still depending on 1.22 is v0.30.0.
The upstream one assumes it's running on a Unix system (which makes
sense), but this package is also used on baremetal. So replace it on
systems that need a replaced syscall package.
This requires an API introduced in MacOS 11. I think that's fine, since
the version before that (MacOS 10.15) is EOL since 2022. Though if
needed, we could certainly work around it by using an older and slightly
less nice API.
Making this work on all targets was interesting but there's now a test
in place to make sure this works on all targets that have the CGo test
enabled (which is almost all targets).
Use a single package for certain constants that must be the same between
the compiler and the runtime.
While just using the same values in both places works, this is much more
obvious and harder to mess up. It also avoids the need for comments
pointing to the other location the constant is defined. And having it in
code makes it possible for IDEs to analyze the source.
In the future, more such constants and maybe algorithms can be added.
This issue was originally reported here:
https://github.com/NixOS/nixpkgs/pull/341170#issuecomment-2359237471
The fix here isn't a great fix, it turns the error message from this:
# runtime/interrupt
into this:
# runtime/interrupt
package requires newer Go version go1.23
...so not great, because it doesn't show the real error message (which
is that TinyGo wasn't compiled with the right Go version). But at least
it gives a hint in the right direction.
It's difficult to test for this specific case, so I've left out testing
in this case (boo!)
This required a few compiler and runtime tricks to work, but I ran a
bunch of tests and it seems fine. (CI will of course do more exhaustive
testing).
The main benefit here is that we don't need to maintain the darwin
version of the syscall package, and reduce extra risks for bugs (because
we reuse the well-tested syscall package). For example, Go 1.23 needed a
bunch of new constants in the syscall package. That would have been
avoided if we had used the native syscall package on MacOS.
This package can never be a full version as seen in upstream Go, because
TinyGo is very different. But it is necessary to define so that no code
can accidentally use this package (now or in the future).
It currently defines:
- NoEscape which is needed by strings.Builder since Go 1.23.
- FuncPCABI* which is needed by internal/syscall/unix on MacOS.
This version probably isn't as fast as the upstream version, but it is
good enough for now. It also doesn't free unreferenced handles like the
upstream version.
I tried implementing enough CGo support to get the native os/user
package to work. But I hit a few bugs, probably in CGo itself. Then I
realized I could just as well set the osusergo build tag to disable CGo
for this specific case.
This actually gets the os/user package to work correctly on Linux (I
confirmed it returns my name/uid/homedir etc). On other systems, it
probably just returns an error if it can't determine these kinds of
things. But that's no worse than the current behavior which just doesn't
do anything at all.
- add internal/wasm-tools/go.mod file to depend on wasm-tools-go
- copy package cm into src/internal/cm
- remove wasm-tools-go "vendor" submodule
internal/tools: fix typo
go.{mod,sum}, internal/tools: add wit-bindgen-go to tools
GNUmakefile: use go run for wit-bindgen-go
GNUmakefile: add tools target to go:generate tools binaries in internal/tools
GNUmakefile: add .PHONY for lint and spell
GNUmakefile, internal/cm: vendor package cm into internal/cm
go.{mod,sum}: update wasm-tools-go to v0.1.4
internal/wasi: use internal/cm package
remove submodule src/vendor/github.com/ydnar/wasm-tools-go
GNUmakefile: add comment documenting what wasi-cm target does
go.{mod,sum}: remove toolchain; go mod tidy
go.mod: revert to Go 1.19
go.mod: go 1.19
go.{mod,sum}, internal/{tools,wasm-tools}: revert root go.mod file to go1.19
Create a wasm-tools specific module that can require go1.22 for wasm-tools-go.
The encoding/binary package in Go 1.23 imports the slices package, which
results in circular import when imported from the machine package.
Therefore, encoding/binary cannot be used in the machine package.
This commit fixes that by introducing a new internal/binary package that
is just plain Go code without dependencies. It can be safely used
anywhere (including the runtime if needed).
CGo is needed for the rp2040 and for macOS (GOOS=darwin), without it
these targets just won't work. And there really isn't a benefit from
disabling CGo: we don't need any external linkers for example.
This avoids a somewhat common issue of people having CGO_ENABLED=0
somewhere in their environment and not understanding why things don't
work. See for example: https://github.com/tinygo-org/tinygo/issues/3450
This fixes the new loop variable behavior in Go 1.22.
Specifically:
* The compiler (actually, the x/tools/go/ssa package) now correctly
picks up the Go version.
* If a module doesn't specify the Go version, the current Go version
(from the `go` tool and standard library) is used. This fixes
`go run`.
* The tests in testdata/ that use a separate directory are now
actually run in that directory. This makes it possible to use a
go.mod file there.
* There is a test to make sure old Go modules still work with the old
Go behavior, even on a newer Go version.
This PR adds a network device driver model called netdev. There will be a companion PR for TinyGo drivers to update the netdev drivers and network examples. This PR covers the core "net" package.
An RFC for the work is here: #tinygo-org/drivers#487. Some things have changed from the RFC, but nothing major.
The "net" package is a partial port of Go's "net" package, version 1.19.3. The src/net/README file has details on what is modified from Go's "net" package.
Most "net" features are working as they would in normal Go. TCP/UDP/TLS protocol support is there. As well as HTTP client and server support. Standard Go network packages such as golang.org/x/net/websockets and Paho MQTT client work as-is. Other packages are likely to work as-is.
Testing results are here (https://docs.google.com/spreadsheets/d/e/2PACX-1vT0cCjBvwXf9HJf6aJV2Sw198F2ief02gmbMV0sQocKT4y4RpfKv3dh6Jyew8lQW64FouZ8GwA2yjxI/pubhtml?gid=1013173032&single=true).
Set -resource-dir in a central place instead of passing the header path
around everywhere and adding it using the `-I` flag. I believe this is
closer to how Clang is intended to be used.
This change was inspired by my attempt to add a Nix flake file to
TinyGo.
Hector Chu
Jake Bailey
Ayke van Laethem
Achille
deadprogram
Damian Gryski
Abhishek Goyal
Elias Naur
Randy Reddig
Dan Kegel
dkegel-fastly
Scott Feldman
ginglis13
Justin A. Wilson
José Carlos Chávez
Elliott Sales de Andrade