See the code comments for details. But in short, this implements a futex
for the cooperative scheduler (that is single threaded and
non-reentrant). Similar implementations can be made for basically every
other operating system, and even WebAssembly with the threading
(actually: atomics) proposal.
Using this basic futex implementation means we can use the same
implementation for synchronisation primitives on cooperative and
multicore systems.
For more information on futex across operating systems:
https://outerproduct.net/futex-dictionary.html
This adds some non-atomic types that have the same interface as the ones
in sync/atomic.
We currently don't need these to be atomic (because the scheduler is
entirely cooperative), but once we add support for a scheduler with
multiple threads and/or preemptive scheduling we can trivially add some
type aliases under a different build tag in the future when real
atomicity is needed for threading.
Previously the assembler was reordering this code:
jal tinygo_scanstack
move $a0, $sp
Into this:
jal tinygo_scanstack
nop
move $a0, $sp
So it was "helpfully" inserting a branch delay slot, even though this
was already being taken care of.
Somehow this didn't break, but it does break in the WIP threading branch
(https://github.com/tinygo-org/tinygo/pull/4559) where this bug leads to
a crash.
This uses uint32 instead of uint64. The reason for this is that uint64
atomic operations aren't universally available (especially on 32-bit
architectures). We could also use uintptr, but that seems needlessly
complicated: it's unlikely real-world programs will use more than a
billion select states (2^30).
This moves all scheduler code into a separate file that is only compiled
when there's a scheduler in use (the tasks or asyncify scheduler, which
are both cooperative). The main goal of this change is to make it easier
to add a new "scheduler" based on OS threads.
It also fixes a few subtle issues with `-gc=none`:
- Gosched() panicked. This is now fixed to just return immediately
(the only logical thing to do when there's only one goroutine).
- Timers aren't supported without a scheduler, but the relevant code
was still present and would happily add a timer to the queue. It
just never ran. So now it exits with a runtime error, similar to any
blocking operation.
With a few small modifications, all the problems with `-gc=precise` in
WebAssembly seem to have been fixed.
I didn't do any performance measurements, but this is supposed to
improve GC performance.
This adds linux/mipsle (little endian Mips) support to TinyGo.
It also adds experimental linux/mips (big-endian) support. It doesn't
quite work yet, some parts of the standard library (like the reflect
package) currently seem to assume a little-endian system.
I managed to get CGo sort-of working in VSCode (meaning that it will
typecheck code in the IDE itself) using a few crude hacks, but it
requires a few minor changes to the TinyGo standard library.
I intend to eventually add this support in the TinyGo extension for
VSCode directly, but for now I've manually updated .vscode/settings.json
to get IDE support. In any case, it would be nice to have this for when
I hopefully add this to the TinyGo extension eventually.
The old code was broken and led to a HardFault in a rather convoluted
way:
1. The CFA offset was incorrect, in fact it was not aligned (the stack
is supposed to always be aligned to 4 bytes at least).
2. This unaligned size was then used for stack size calculations.
3. A stack that wasn't a multiple of 4 was allocated.
4. The calleeSavedRegs struct (in `(internal/task.state).archInit`) was
not correctly aligned.
5. Writing to this struct resulted in a HardFault.
Blocking inside an interrupt is always unsafe and will lead to all kinds
of bad behavior (even if it might appear to work sometimes). So disallow
it, just like allocating heap memory inside an interrupt is not allowed.
I suspect this will usually be caused by channel sends, like this:
ch <- someValue
The easy workaround is to make it a non-blocking send instead:
select {
case ch <- someValue:
default:
}
This does mean the application might become a bit more complex to be
able to deal with this case, but the alternative (undefined behavior) is
IMHO much worse.
Some vector registers must be preserved across calls, but this wasn't
happening on Linux and MacOS. When I added support for windows/arm64, I
saw that it required these vector registers to be preserved and assumed
this was Windows deviating from the standard calling convention. But
actually, Windows was just implementing the standard calling convention
and the bug was on Linux and MacOS.
This commit fixes the bug on Linux and MacOS and at the same time merges
the Go and assembly files as they no longer need to be separate.
This was actually surprising once I got TinyGo to build on Windows 11
ARM64. All the changes are exactly what you'd expect for a new
architecture, there was no special weirdness just for arm64.
Actually getting TinyGo to build was kind of involved though. The very
short summary is: install arm64 versions of some pieces of software
(like golang, cmake) instead of installing them though choco. In
particular, use the llvm-mingw[1] toolchain instead of using standard
mingw.
[1]: https://github.com/mstorsjo/llvm-mingw/releases
This implements the block-based GC as a partially precise GC. This means
that for most heap allocations it is known which words contain a pointer
and which don't. This should in theory make the GC faster (because it
can skip non-pointer object) and have fewer false positives in a GC
cycle. It does however use a bit more RAM to store the layout of each
object.
Right now this GC seems to be slower than the conservative GC, but
should be less likely to run out of memory as a result of false
positives.
--allow-undefined can be a problem: it allows compiling code that will
fail when loaded. This change makes sure that if some symbols are
undefined, they are reported as an error by the linker.
Previously, people could get away with importing a function that was not
defined, like this:
func add(int a, int b) int
func test() {
println(add(3, 5))
}
This was always unintended but mostly worked. With this change, it isn't
possible anymore. Now every function needs to be marked with //export
explicitly:
//export add
func add(int a, int b) int
func test() {
println(add(3, 5))
}
As before, functions will be placed in the `env` module with the name
set from the `//export` tag. This can be overridden with
`//go:import-module`:
//go:import-module math
//export add
func add(int a, int b) int
func test() {
println(add(3, 5))
}
For the syscall/js package, I needed to give a list of symbols that are
undefined. This list is based on the JavaScript functions defined in
targets/wasm_exec.js.
Go 1.19 started reformatting code in a way that makes it more obvious
how it will be rendered on pkg.go.dev. It gets it almost right, but not
entirely. Therefore, I had to modify some of the comments so that they
are formatted correctly.
Some source code wasn't part of `FMT_PATHS` so wasn't checked for
correct formatting. This change includes all this source code and
excludes cgo/testdata because it contains files that can't be parsed.
If an interrupt happens between the writes to SPL and SPH, the stack
pointer is inconsistent and terrible things will happen. Therefore,
disable interrupts while updating the stack pointer.
Interrupts are restored _before_ the write to SPH. This is safe, because
interrupts are re-enabled with a one cycle delay. The avr-gcc and Clang
compilers do the same thing when they need to update the stack pointer.
It's almost impossible to test for this bug, but it should make firmware
just a little bit more reliable.
This patch adds support for generating GOOS=darwin GOARCH=arm64
binaries. This means that it will become possible to run `go test` on
recent Macs, for example.
This adds support for building with `-tags=llvm13` and switches to LLVM
13 for tinygo binaries that are statically linked against LLVM.
Some notes on this commit:
* Added `-mfloat-abi=soft` to all Cortex-M targets because otherwise
nrfx would complain that floating point was enabled on Cortex-M0.
That's not the case, but with `-mfloat-abi=soft` the `__SOFTFP__`
macro is defined which silences this warning.
See: https://reviews.llvm.org/D100372
* Changed from `--sysroot=<root>` to `-nostdlib -isystem <root>` for
musl because with Clang 13, even with `--sysroot` some system
libraries are used which we don't want.
* Changed all `-Xclang -internal-isystem -Xclang` to simply
`-isystem`, for consistency with the above change. It appears to
have the same effect.
* Moved WebAssembly function declarations to the top of the file in
task_asyncify_wasm.S because (apparently) the assembler has become
more strict.
This change swaps the stack chain when switching goroutines, ensuring that the chain is maintained consistently.
This is only really currently necessary with asyncify on wasm.
The assembly symbols were not marked as hidden and so were exported,
leading to unreferenced symbols.
Example error message:
Error: failed to run main module `/tmp/tinygo3961039405/main`
Caused by:
0: failed to instantiate "/tmp/tinygo3961039405/main"
1: unknown import: `asyncify::stop_rewind` has not been defined
This commit fixes this issue.
This change implements a new "scheduler" for WebAssembly using binaryen's asyncify transform.
This is more reliable than the current "coroutines" transform, and works with non-Go code in the call stack.
runtime (js/wasm): handle scheduler nesting
If WASM calls into JS which calls back into WASM, it is possible for the scheduler to nest.
The event from the callback must be handled immediately, so the task cannot simply be deferred to the outer scheduler.
This creates a minimal scheduler loop which is used to handle such nesting.
This allows the assembly routines in these files to be stripped as dead
code if they're not referenced. This solves the link issues on MacOS
when the `leaking` garbage collector or the `coroutines` scheduler
are selected.
Fixes#2081
Previously we used the i386 target, probably with all optional features
disabled. However, the Pentium 4 has been released a _long_ time ago and
it seems reasonable to me to take that as a minimum requirement.
Upstream Go now also seems to move in this direction:
https://github.com/golang/go/issues/40255
The main motivation for this is that there were floating point issues
when running the tests for the math package:
GOARCH=386 tinygo test math
I haven't investigated what's the issue, but I strongly suspect it's
caused by the weird x87 80-bit floating point format. This could perhaps
be fixed in a different way (by setting the FPU precision to 64 bits)
but I figured that just setting the minimum requirement to the Pentium 4
would probably be fine. If needed, we can respect the GO386 environment
variable to support these very old CPUs.
To support this newer CPU, I had to make sure that the stack is aligned
to 16 bytes everywhere. This was not yet always the case.
This results in smaller and likely more efficient code. It does require
some architecture specific code for each architecture, but I've kept the
amount of code as small as possible.
This distinction was useful before when reflect wasn't properly
supported. Back then it made sense to only include method sets that were
actually used in an interface. But now that it is possible to get to
other values (for example, by extracting fields from structs) and it is
possible to turn them back into interfaces, it is necessary to preserve
all method sets that can possibly be used in the program in a type
assert, interface assert or interface method call.
In the future, this logic will need to be revisited again when
reflect.New or reflect.Zero gets implemented.
Code size increases a bit in some cases, but usually in a very limited
way (except for one outlier in the drivers smoke tests). The next commit
will improve the situation significantly.
This commit switches from the previous behavior of compiling the whole
program at once, to compiling every package in parallel and linking the
LLVM bitcode files together for further whole-program optimization.
This is a small performance win, but it has several advantages in the
future:
- There are many more things that can be done per package in parallel,
avoiding the bottleneck at the end of the compiler phase. This
should speed up the compiler futher.
- This change is a necessary step towards a non-LTO build mode for
fast incremental builds that only rebuild the changed package, when
compiler speed is more important than binary size.
- This change refactors the compiler in such a way that it will be
easier to inspect the IR for one package only. Inspecting this IR
will be very helpful for compiler developers.
I have chosed to call this implementation `esp8266` instead of `xtensa`
as it has been written specifically for the ESP8266 and there are no
other Xtensa chips with the CALL0 ABI (no windowing) that I know of. The
only other related chip is the ESP32, which does implement register
windowing and thus needs a very different implementation.
This has been a *lot* of work, trying to understand the Xtensa windowed
registers ABI. But in the end I managed to come up with a very simple
implementation that so far seems to work very well.
I tested this with both blinky examples (with blinky2 slightly edited)
and ./testdata/coroutines.go to verify that it actually works.
Most development happened on the ESP32 QEMU fork from Espressif
(https://github.com/espressif/qemu/wiki) but I also verified that it
works on a real ESP32.
The Cortex-M target isn't much changed, but much of the logic for the
AVR stack switcher that was previously in assembly has now been moved to
Go to make it more maintainable and in fact smaller in code size. Three
functions (tinygo_getCurrentStackPointer, tinygo_switchToTask,
tinygo_switchToScheduler) have been changed to one: tinygo_swapTask.
This reduction in assembly code should make the code more maintainable
and should make it easier to port stack switching to other
architectures.
I've also moved the assembly files to src/internal/task, which seems
like a more appropriate location to me.
This is a big change that will determine the stack size for many
goroutines automatically. Functions that aren't recursive and don't call
function pointers can in many cases have an automatically determined
worst case stack size. This is useful, as the stack size is usually much
lower than the previous hardcoded default of 1024 bytes: somewhere
around 200-500 bytes is common.
A side effect of this change is that the default stack sizes (including
the stack size for other architectures such as AVR) can now be changed
in the config JSON file, making it tunable per application.
While adding some code to clear the Next field when popping from a task stack for safety reasons, the clear was placed outside of a nil pointer check.
As a result, (*internal/task.Stack).Pop panicked when the Stack is empty.
Ayke van Laethem
Dan Kegel
dkegel-fastly
waj334
Justin A. Wilson
Anuraag Agrawal
Yurii Soldak
Nia Waldvogel
Damian Gryski
Kenneth Bell
deadprogram
Jaden Weiss