Don't block forever if there's nothing to receive. On the other hand,
process the entire FIFO, not just a single byte.
This fixes an issue where the rp2350 would hang after programming through
openocd, where the UART0 interrupt would be spuriously pending.
Run a range of tests in CI, to make sure browser wasm and baremetal
don't regress too badly.
I have intentionally filtered out tests, so that newly added tests to
TEST_PACKAGES_FAST will be added here as well (and can be excluded if
needed).
It looks like we didn't have any tests for wasm. Having one tests is not
much, but it proves that the infrastructure works and it actually
verifies a fix to https://github.com/tinygo-org/tinygo/issues/4777.
We should add more packages to this list in the future.
Apparently this package imports `runtime.getRandomData` from the gojs
module. This is not yet implemented, but simply allowing this package to
do such imports gets crypto/sha256 to compile.
Right now it doesn't compile, with errors like the following:
# machine
/app/tinygo/src/machine/board_pico.go:7:13: undefined: GPIO0
/app/tinygo/src/machine/board_pico.go:8:13: undefined: GPIO1
/app/tinygo/src/machine/board_pico.go:9:13: undefined: GPIO2
/app/tinygo/src/machine/board_pico.go:10:13: undefined: GPIO3
[...etc...]
This patch should fix that.
* machine: add support for core voltage adjustments to rp2040
In preparation for bumping the core frequency of the rp2040, this
change implements the required core voltage adjustment logic.
* machine: bump rp2040 to 200MHz
Follow-up to #4728 which implemented the algorithm for finding the
dividers.
The calculation is computed at compile time by interp, as verified by
building example/blinky1 for -target pico.
The upstream one assumes it's running on a Unix system (which makes
sense), but this package is also used on baremetal. So replace it on
systems that need a replaced syscall package.
We can't use the bytes package in Go 1.24 since it would result in an
import cycle. Therefore, use bytealg.Index instead.
(I'm not sure this code is correct - just searching for a range of bytes
seems brittle. But at least this commit shouldn't change the code).
This mangles CGo identifier names to something like "_Cgo_foo" instead
of using literal identifiers like "C.foo". This works around
https://github.com/golang/go/issues/71777.
I don't like this solution, but I hope we'll find a better solution in
the future. In that case we can revert this commit.
older behavior for wasi modules to not return an exit code as if they were reactors.
See #4726 for some details on what this is intended to address.
Signed-off-by: deadprogram <ron@hybridgroup.com>
This ensures:
1. The xorshift state is initialized during interp.
2. The xorshift state gets initialized to a real random number on
hardware that supports it at runtime.
This fixes a big binary size regression from the previous commit. It's
still not perfect: most programs increase binary size by a few bytes.
But it's not nearly as bad as before.
Previously, if a function couldn't be interpreted in the interp pass, it
would just be run at runtime and the parameters would be marked as
potentially being modified. However, this is incorrect: the function
itself can also modify memory. So the function itself also needs to be
marked (recursively).
This fixes a number of regressions while upgrading to Go 1.24.
This results in a significant size regression that is mostly worked
around in the next commit.
Modify the ID used for looking up the reference, based on suggestion made by @prochac
Also use console.error in the caase that the reference is not found, since it is now actually
known to be an error.
`tinygo build -o /path/to/out .` expected `/path/to/out` to already exist, which is different behaviour to `go build`. This change ensures tinygo creates any directories needed to be able to build to the specified output dir.
The mutex implementation needs a different implementation once support
for threading lands. This implementation just moves code to the
internal/task package to centralize these algorithms.
The latest golang.org/x/net websocket package v0.33.0 needs
Dialer.DailContext in crypto/tls. This PR adds it.
Apps using golang.org/x/net are encouraged to use v0.33.0 to address:
CVE-2024-45338: Non-linear parsing of case-insensitive content in golang.org/x/net/html
CVE-2023-45288: net/http, x/net/http2: close connections when receiving too many headers
Tested with examples/net/websocket/dial.
If `ld.lld` is a version-specific binary (e.g., `ld.lld-18`), then its
error messages include the version. The parsing previously incorrectly
assumed it would be unversioned.
Show which files cause a binary size increase. This makes it easier to
see where the size is going: for example, this makes it easy to see how
much the GC contributes to code size compared to other runtime parts.
This is not a big change over the existing size report with -size=full,
but it is a bit more readable.
More information will be added in subsequent commits.
* initial implementation for Tillitis TKey device
* add UART implementation for TKey
* add Pin interface implementation for TKey touch sensor
* add RNG interface implementation for TKey
* add helpful machine package functions to return identifiers such as name and version for TKey
* use built-in timer for sleep timing on TKey
* modify UART implementation for TKey to implement Serialer interface
* implement BLAKE2s ROM function call for TKey device
* handle abort by triggering TKey device fault using illegal instruction to halt CPU
* simplify TKey implementation by inheriting from existing riscv32 target
* return error for trying to configure invalid baudrates on UART
* add tkey to builder test
* be very specific for features passed to LLVM for specific config in use for TKey
* handle feedback items from TKey device code review
Signed-off-by: deadprogram <ron@hybridgroup.com>
This actually simplifies the code and avoids a heap allocation in the
call to Wait. Instead, it uses the Data field of the task to store
information on whether a task was signalled early.
See the code comments for details. But in short, this implements a futex
for the cooperative scheduler (that is single threaded and
non-reentrant). Similar implementations can be made for basically every
other operating system, and even WebAssembly with the threading
(actually: atomics) proposal.
Using this basic futex implementation means we can use the same
implementation for synchronisation primitives on cooperative and
multicore systems.
For more information on futex across operating systems:
https://outerproduct.net/futex-dictionary.html
This adds some non-atomic types that have the same interface as the ones
in sync/atomic.
We currently don't need these to be atomic (because the scheduler is
entirely cooperative), but once we add support for a scheduler with
multiple threads and/or preemptive scheduling we can trivially add some
type aliases under a different build tag in the future when real
atomicity is needed for threading.
I don't think this is used anywhere right now, and it would need to be
updated to work with multithreading. So instead of fixing it, I think we
can remove it.
My original intention was to have something like this that could be used
in the machine package, but since this is in the runtime package (and
the runtime package imports the machine package on baremetal) it can't
actually be used that way.
I checked the TinyGo repo and the drivers repo, and `runtime.Cond` isn't
used anywhere except in that one test.
Previously the assembler was reordering this code:
jal tinygo_scanstack
move $a0, $sp
Into this:
jal tinygo_scanstack
nop
move $a0, $sp
So it was "helpfully" inserting a branch delay slot, even though this
was already being taken care of.
Somehow this didn't break, but it does break in the WIP threading branch
(https://github.com/tinygo-org/tinygo/pull/4559) where this bug leads to
a crash.
This ensures that calls to print/println happening in different threads
are not interleaved. It's a task.PMutex, so this should only change
things when threading is used.
This matches the Go compiler, which does the same thing:
https://godbolt.org/z/na5KzE7en
The locks are not recursive, which means that we need to be careful to
not call `print` or `println` inside a runtime.print* implementation,
inside putchar (recursively), and inside signal handlers. Making them
recursive might be useful to do in the future, but it's not really
necessary.
This uses uint32 instead of uint64. The reason for this is that uint64
atomic operations aren't universally available (especially on 32-bit
architectures). We could also use uintptr, but that seems needlessly
complicated: it's unlikely real-world programs will use more than a
billion select states (2^30).
This uses the task.PMutex parallel-only-mutex type to make the leaking
GC parallelism safe. The task.PMutex type is currently a no-op but will
become a real mutex once we add true parallelism.
Use the alignment from the align attribute of the runtime.alloc call.
This is going to be a more accurate alignment, and is typically smaller
than the default.
This requires an API introduced in MacOS 11. I think that's fine, since
the version before that (MacOS 10.15) is EOL since 2022. Though if
needed, we could certainly work around it by using an older and slightly
less nice API.
This really is the only sane way to register a signal. If this flag is
not set, many syscalls will return EINTR (and not complete their
operation) which will be a massive source of hard-to-debug bugs.
This rewrite simplifies the channel implementation considerably, with
34% less LOC. Perhaps the most important change is the removal of the
channel state, which made sense when we had only send and receive
operations but only makes things more compliated when multiple select
operations can be pending on a single channel.
I did this rewrite originally to make it possible to make channels
parallelism-safe. The current implementation is not parallelism-safe,
but it will be easy to make it so (the main additions will be a channel
lock, a global select lock, and an atomic compare-and-swap in
chanQueue.pop).
This only works on declarations, not definitions. This is intentional:
it follows the upstream Go implemetation.
However, we might want to loosen this requirement at some point: TinyGo
sometimes stores pointers in memory mapped I/O knowing they won't
actually escape, but the compiler doesn't know about this.
This moves all scheduler code into a separate file that is only compiled
when there's a scheduler in use (the tasks or asyncify scheduler, which
are both cooperative). The main goal of this change is to make it easier
to add a new "scheduler" based on OS threads.
It also fixes a few subtle issues with `-gc=none`:
- Gosched() panicked. This is now fixed to just return immediately
(the only logical thing to do when there's only one goroutine).
- Timers aren't supported without a scheduler, but the relevant code
was still present and would happily add a timer to the queue. It
just never ran. So now it exits with a runtime error, similar to any
blocking operation.
Making this work on all targets was interesting but there's now a test
in place to make sure this works on all targets that have the CGo test
enabled (which is almost all targets).
This fixes a bug where output would not actually be written to stdout
before exiting the process, leading to a flaky Windows CI. Exiting using
`exit(0)` appears to fix this.
For some background, see: https://github.com/tinygo-org/tinygo/pull/4589
* Move environment functions to their own files.
* Rewrite the WASIp2 version of environment variables to be much
simpler (don't go through C functions).
These should make sure basic functionality is still working.
Using the `-short` flag to avoid taking too long to run all tests (and
to install all the necessary emulators), and because some targets might
not work in older Go/LLVM versions (such as WASI).
This does _not_ run tests and checks against expected IR, because LLVM
IR changes a lot across versions.
It looks like this breaks on Windows:
https://github.com/tinygo-org/tinygo/issues/4557
I haven't confirmed this is indeed the problem, but it would make sense.
And passing through the temporary directory seems like a good idea
regardless, there's not much that could break due to that.
Use a single package for certain constants that must be the same between
the compiler and the runtime.
While just using the same values in both places works, this is much more
obvious and harder to mess up. It also avoids the need for comments
pointing to the other location the constant is defined. And having it in
code makes it possible for IDEs to analyze the source.
In the future, more such constants and maybe algorithms can be added.
This is similar to https://github.com/tinygo-org/tinygo/pull/3899, but
smaller and hopefully just as efficient.
Thanks to @HattoriHanzo031 for starting this work, benchmarking, and for
improving the performance of the code even further.
The git hash that's part of `tinygo version` is now read from the binary
itself instead of embedding it with a `-ldflags` flag. This means it is
also present when building TinyGo using `go build` or `go install`.
* internal/wasm-tools, internal/cm: update wasm-tools-go to v0.3.1 and regenerate bindings
* syscall: use new (cm.Result).Result() method instead of OK() and Err()
The alignment wasn't set, so defaulted to 4 (for a 32-bit int). LLVM saw
this, and therefore assumed that a ptrtoint of the pointer would have
had the lowest bits unset. That's an entirely valid optimization, except
that we are using these globals for arbitrary values (and aren't
actually using these globals).
Fixed by setting alignment to 1. It works, though long-term we should
maybe find a different solution for this.
Volatile loads/stors are only useful for communication with interrupts
or for memory-mapped I/O. They do not provide any sort of safety for
sync.Mutex, while making it *appear* as if it is more safe.
* `sync.Mutex` cannot be used safely inside interrupts, because any
blocking calls (including `Lock`) will cause a runtime panic.
* For multithreading, `volatile` is also the wrong choice. Atomic
operations should be used instead, and the current code would not
work for multithreaded programs anyway.
Every time we overflow the stack, we have to do a full rescan of the heap. Making this larger
means fewer overflows and thus fewer secondary+ heap scans.
With a few small modifications, all the problems with `-gc=precise` in
WebAssembly seem to have been fixed.
I didn't do any performance measurements, but this is supposed to
improve GC performance.
compiler: align with current wasm types proposal
https://github.com/golang/go/issues/66984
- Remove int and uint as allowed types in params, results, pointers, or struct fields
- Only allow small integers in pointers, arrays, or struct fields
- enforce structs.HostLayout usage per wasm types proposal
https://github.com/golang/go/issues/66984
- require go1.23 for structs.HostLayout
- use an interface to check if GoVersion() exists
This permits TinyGo to compile with Go 1.21.
- use goenv.Compare instead of WantGoVersion
- testdata/wasmexport: use int32 instead of int
- compiler/testdata: add structs.HostLayout
- compiler/testdata: improve tests for structs.HostLayout
goenv: parse patch version, add func Compare to compare two Go version strings
* Parse tests
* add Compare function to compare two Go version strings
* goenv, builder: parse patch version in Go version string
While there are some browser tests, Node.js is just a lot better for
testing this kind of stuff because it's much faster and we don't need a
browser for this.
This issue was originally reported here:
https://github.com/NixOS/nixpkgs/pull/341170#issuecomment-2359237471
The fix here isn't a great fix, it turns the error message from this:
# runtime/interrupt
into this:
# runtime/interrupt
package requires newer Go version go1.23
...so not great, because it doesn't show the real error message (which
is that TinyGo wasn't compiled with the right Go version). But at least
it gives a hint in the right direction.
It's difficult to test for this specific case, so I've left out testing
in this case (boo!)
This often doesn't work because there might not be a current task to
push the defer frame to. It will instead show an unhelpful nil pointer
dereference panic.
We could make this work with a separate defer stack for interrupts (as
if they were newly started goroutines) but that is difficult with
multiple interrupts happening at the same time (we shouldn't jump to a
previous interrupt in `panic()`!). So instead, disable defer altogether
in interrupts and adjust panic/recover accordingly.
This is a problem on Guix, which sets C_INCLUDE_PATH that is not
affected by `-nostdlibinc`. And therefore it affects the build in
unintended ways.
Removing all environmental variables fixes this issue, and perhaps also
other issues caused by Clang being affected by environment variables.
This adds support for //go:wasmexport with `-target=wasm` (in the
browser). This follows the //go:wasmexport proposal, meaning that
blocking functions are not allowed.
Both `-buildmode=default` and `-buildmode=c-shared` are supported. The
latter allows calling exported functions after `go.run()` has returned.
This should fix some CI issues we're currently having.
Thanks to @sylv-io for discovering that we need to remove LLVM to avoid
an Xtensa backend linker error.
This adds support for the `//go:wasmexport` pragma as proposed here:
https://github.com/golang/go/issues/65199
It is currently implemented only for wasip1 and wasm-unknown, but it is
certainly possible to extend it to other targets like GOOS=js and
wasip2.
Don't rename them when -work is set, instead update result.Binary each
time and leave result.Executable be the linker output (as intended:
result.Executable should be the unmodified linker output).
This matches upstream Go. Example:
$ go test -ldflags='-extldflags=-foobar' os
# os.test
/usr/local/go1.23.1/pkg/tool/linux_arm64/link: running gcc failed: exit status 1
/usr/bin/gcc -s -o $WORK/b001/os.test -rdynamic /tmp/go-link-914594215/go.o /tmp/go-link-914594215/000000.o /tmp/go-link-914594215/000001.o /tmp/go-link-914594215/000002.o /tmp/go-link-914594215/000003.o /tmp/go-link-914594215/000004.o /tmp/go-link-914594215/000005.o /tmp/go-link-914594215/000006.o /tmp/go-link-914594215/000007.o /tmp/go-link-914594215/000008.o /tmp/go-link-914594215/000009.o /tmp/go-link-914594215/000010.o /tmp/go-link-914594215/000011.o /tmp/go-link-914594215/000012.o /tmp/go-link-914594215/000013.o /tmp/go-link-914594215/000014.o /tmp/go-link-914594215/000015.o /tmp/go-link-914594215/000016.o /tmp/go-link-914594215/000017.o /tmp/go-link-914594215/000018.o /tmp/go-link-914594215/000019.o /tmp/go-link-914594215/000020.o /tmp/go-link-914594215/000021.o -O2 -g -O2 -g -lresolv -O2 -g -lpthread -foobar
gcc: error: unrecognized command-line option ‘-foobar’
FAIL os [build failed]
FAIL
And TinyGo, with this patch:
$ tinygo test -ldflags='-extldflags=-foobar' os
FAIL os 0.000s
ld.lld: error: unknown argument '-foobar'
Also note that Go doesn't support the `-extldflags` directly (which was
previously the case with TinyGo):
$ go test -extldflags='-foobar' os
flag provided but not defined: -extldflags
[...]
The values were stored in the passed object as the values itself (not
expanded like is common in the calling convention), and read back after
assuming they were expanded. This often works for simple parameters
(int, pointer, etc), but not for more complex parameters. Especially
when there's padding.
Found this while working on `//go:wasmexport`.
The reflect package needs to know the endianness of the system in a few
places. Before this patch, it assumed little-endian systems. But with
GOARCH=mips we now have a big-endian system which also needs to be
supported. So this patch fixes the reflect package to work on big-endian
systems.
Also, I've updated the tests for MIPS: instead of running the
little-endian tests, I've changed it to run the big-endian tests
instead. The two are very similar except for endianness so this should
be fine. To be sure we won't accidentally break little-endian support,
I've kept a single MIPS little-endian test (the CGo test, which doesn't
yet work on big-endian systems anyway).
The interp package was assuming that all targets were little-endian. But
that's not true: we now have a big-endian target (GOARCH=mips).
This fixes the interp package to use the appropriate byte order for a
given target.
This should widen compatibility a bit, so that older CPUs can also
execute programs built by TinyGo. The performance may be lower, if
that's an issue we can look into implementing the proposal here:
https://github.com/golang/go/issues/60072
This still wouldn't make programs usable on MIPS II CPUs, I suppose we
can lower compatiblity down to that CPU if needed.
I tried setting the -cpu flag in the QEMU command line to be able to
test this, but it looks like there are no QEMU CPU models that are
mips32r1 and have a FPU. So it's difficult to test this.
This required a few compiler and runtime tricks to work, but I ran a
bunch of tests and it seems fine. (CI will of course do more exhaustive
testing).
The main benefit here is that we don't need to maintain the darwin
version of the syscall package, and reduce extra risks for bugs (because
we reuse the well-tested syscall package). For example, Go 1.23 needed a
bunch of new constants in the syscall package. That would have been
avoided if we had used the native syscall package on MacOS.
This was still at jammy (22.04), while the CI container was noble
(24.04). Somehow this didn't break, but it certainly isn't ideal to
install packages across Ubuntu versions!
This is needed for Go 1.23 support.
These functions should ideally get implemented. Until that's done, just
panic here. Apparently panicking in internal/abi.FuncPCABI0 is enough to
fix all linker errors for mime/quotedprintable.
This package can never be a full version as seen in upstream Go, because
TinyGo is very different. But it is necessary to define so that no code
can accidentally use this package (now or in the future).
It currently defines:
- NoEscape which is needed by strings.Builder since Go 1.23.
- FuncPCABI* which is needed by internal/syscall/unix on MacOS.
This version probably isn't as fast as the upstream version, but it is
good enough for now. It also doesn't free unreferenced handles like the
upstream version.
I tried implementing enough CGo support to get the native os/user
package to work. But I hit a few bugs, probably in CGo itself. Then I
realized I could just as well set the osusergo build tag to disable CGo
for this specific case.
This actually gets the os/user package to work correctly on Linux (I
confirmed it returns my name/uid/homedir etc). On other systems, it
probably just returns an error if it can't determine these kinds of
things. But that's no worse than the current behavior which just doesn't
do anything at all.
Print a message for SIGBUS, SIGSEGV, and SIGILL when they happen.
These signals are always fatal, but it's very useful to know which of
them happened.
Also, it prints the location in the binary which can then be parsed by
`tinygo run` (see https://github.com/tinygo-org/tinygo/pull/4383).
While this does add some extra binary size, it's for Linux and MacOS
(systems that typically have plenty of RAM/storage) and could be very
useful when debugging some low-level crash such as a runtime bug.
This adds softfloat support to GOARM, as proposed here:
https://github.com/golang/go/issues/61588
This is similar to https://github.com/tinygo-org/tinygo/pull/4189 but
with a few differences:
* It is based on the work to support MIPS softfloat.
* It fixes the issue that the default changed to softfloat everywhere.
This PR defaults to softfloat on ARMv5 and hardfloat on ARMv6 and
ARMv7.
* It also compiles the C libraries (compiler-rt, musl) using the same
soft/hard floating point support. This is important because
otherwise a GOARM=7,softfloat binary could still contain hardware
floating point instructions.
This more accurately describes the libc we are using.
This also adds the environment to _all_ Linux systems, not just ARM. And
selects the correct ABI (soft/hardfloat) that's in use.
I don't know whether it has any practical implications, but LLVM appears
to be using this information so it seems wise to use the correct values.
I jumped through quite a few hoops to get test-llvm15-go119 to work, but this last hoop seems not worth jumping through:
cd internal/tools && go generate -tags tools ./
/go/pkg/mod/github.com/golangci/misspell@v0.6.0/mime.go:10:2: package slices is not in GOROOT (/usr/local/go/src/slices)
tools.go:12: running "go": exit status 1
make: *** [GNUmakefile:952: tools] Error 1
I mean, we could patch misspell to not use slices, or to use the prerelease slices, but...
TODO: Remove the go.mod/go.sum in internal/tools once doing so doesn't break CI (e.g. once we drop support for go 1.19)
* builder/cc1as.h: fix typo found by 'make spell'
* GNUmakefile: remove exception for inbetween, fix instance now found by 'make spell'
* GNUmakefile: remove exception for programmmer, fix instance now found by 'make spell'
* go.mod: use updated misspell. GNUmakefile: add spellfix target, use it.
* ignore directories properly when invoking spellchecker.
* make spell: give internal/tools its own go.mod, as misspell requires newer go
* make lint: depend on tools and run the installed revive
(which was perhaps implied by the change that added revive to internal/tools,
but not required in GNUmakefile until we gave internal/tools its own temporary go.mod)
* .github: now that 'make spell' works well, run it from CI
* GNUmakefile: make spell now aborts if it finds misspelt words, so what it finds doesn't get lost in CI logs
* GNUmakefile: tools: avoid -C option on go generate to make test-llvm15-go119 circleci job happy, see
https://cs.opensource.google/go/go/+/2af48cbb7d85e5fdc635e75b99f949010c607786
* internal/tools/go.mod: fix format of go version to leave out patchlevel, else go complains.
Example output:
$ make help
clean Remove build directory
fmt Reformat source
fmt-check Warn if any source needs reformatting
gen-device Generate microcontroller-specific sources
llvm-source Get LLVM sources
llvm-build Build LLVM
lint Lint source tree
spell Spellcheck source tree
Might even work on windows, since git for windows comes with awk.
This was needed in the past because LLVM used typed pointers and there
was a mismatch between pointer types. But we've dropped support for
typed pointers a while ago so now we can remove these wrappers.
This is just a cleanup, it shouldn't have any practical effect.
Previously, the compiler would default to hardfloat. This is not
supported by some MIPS CPUs.
This took me much longer than it should have because of a quirk in the
LLVM Mips backend: if the target-features string is not set (like during
LTO), the Mips backend picks the first function in the module and uses
that. Unfortunately, in the case of TinyGo this first function is
`llvm.dbg.value`, which is an LLVM intrinsic and doesn't have the
target-features string. I fixed it by adding a `-mllvm -mattr=` flag to
the linker.
Move triple calculation into defaultTarget. It was separate for historic
reasons that no longer apply. Merging the code makes future changes
easier (in particular, softfloat support).
The new code is actually shorter than the old code even though it has
more comments.
This shows nicely formatted error messages for missing symbol names and
for out-of-flash, out-of-RAM conditions (on microcontrollers with
limited flash/RAM).
Unfortunately the missing symbol name errors aren't available on Windows
and WebAssembly because the linker doesn't report source locations yet.
This is something that I could perhaps improve in LLD.
The C printf function is sometimes needed for C files included using
CGo. This commit makes sure they're available on all systems where CGo
is fully supported (that is, everywhere except on AVR).
For baremetal systems using picolibc, I've picked the integer-only
version of printf to save on flash size. We might want to consider
providing a way to pick the floating point version instead, if needed.
It's possible to detect the architecture from the target triple, but
there are a number of exceptions that make it unpleasant to use for this
purpose. There are just too many weird exceptions (like mips vs mipsel,
and armv6m vs thumv6m vs arm64 vs aarch64) so it's better to centralize
these to canonical architecture names.
I picked the architecture names that happen to match the musl
architecture names, because those seem the most natural to me.
This adds the same panic locations that are already present for
`tinygo flash -monitor`, but for `tinygo run` and `tinygo test`.
For example, this is the output that I get while working on some GC
code. It now shows the source location instead of just an address:
$ tinygo test -v archive/zip
=== RUN TestReader
=== RUN TestReader/test.zip
panic: runtime error at 0x000000000024d9b4: goroutine stack overflow
[tinygo: panic at /home/ayke/src/tinygo/tinygo/src/internal/task/task_stack.go:58:15]
FAIL archive/zip 0.139s
(This particular location isn't all that useful, but it shows that the
feature works).
These libraries will be automatically built when needed and cached.
The main reason these were needed is for play.tinygo.org, but I've now
prebuilt them there directly (so they don't need to be built for every
tarball).
* reflect: rawFieldByNameFunc: copy index slice to avoid later overwrites
* runtime: Simplify slice growing/appending code
Refactor the slice appending function to rely on the slice growing
function, and remove branches/loops to use a branchfree variant.
Signed-off-by: L. Pereira <l.pereira@fastly.com>
* runtime: Remove one branch in sliceAppend()
Both branches were equivalent, so guard the overall logic in
sliceAppend() with the more general condition.
Signed-off-by: L. Pereira <l.pereira@fastly.com>
* runtime: Simplify slice growing calculation
Use `bits.Len()` rather than `32 - bits.LeadingZeros32()`. They're
equivalent, but the Len version is a bit easier to read.
Signed-off-by: L. Pereira <l.pereira@fastly.com>
* reflect: Always call sliceGrow() in extendSlice()
sliceGrow() will return the old slice if its capacity is large enough.
Signed-off-by: L. Pereira <l.pereira@fastly.com>
---------
Signed-off-by: L. Pereira <l.pereira@fastly.com>
Co-authored-by: Damian Gryski <damian@gryski.com>
For some reason, the type kind name is "unsafe.Pointer" while the
.Name() method just returns "Pointer".
Not sure why this difference exists, but to be able to test .Name() in
testdata/reflect.go it needs to match.
This improves compilation performance by about 5% in my quick test,
while increasing binary size on average by 0.13% when comparing the
smoke tests in the drivers repo (and about two thirds of that 0.13% is
actually caused by a single smoke test).
I think this is a good idea because it aligns the TinyGo optimization
sequence with what ThinLTO expects.
This adds linux/mipsle (little endian Mips) support to TinyGo.
It also adds experimental linux/mips (big-endian) support. It doesn't
quite work yet, some parts of the standard library (like the reflect
package) currently seem to assume a little-endian system.
Previously, the error messages could be shown out of order.
With this patch, the errors are sorted before being shown to the user.
This makes it easier to read the error messages, and matches what the
`go` toolchain does.
This is a refactor, which should (in theory) not change the behavior of
the compiler. But since this is a pretty large change, there is a chance
there will be some regressions. For that reason, the previous commits
added a bunch of tests to make sure most error messages will not be
changed due to this refactor.
This is just one optimizer pass that's easy to test for. There are
others, but I'm ignoring them for now.
The one other that would be reasonable to test is when starting a
goroutine with -gc=none, but I'm leaving that one for another time.
Match the error messages in testdata/errors/*.go like LLVM FileCheck,
which includes regular expressions.
I believe this is much more flexible, and LLVM uses it in nearly all of
their tests so it must work quite well for compilers.
- add internal/wasm-tools/go.mod file to depend on wasm-tools-go
- copy package cm into src/internal/cm
- remove wasm-tools-go "vendor" submodule
internal/tools: fix typo
go.{mod,sum}, internal/tools: add wit-bindgen-go to tools
GNUmakefile: use go run for wit-bindgen-go
GNUmakefile: add tools target to go:generate tools binaries in internal/tools
GNUmakefile: add .PHONY for lint and spell
GNUmakefile, internal/cm: vendor package cm into internal/cm
go.{mod,sum}: update wasm-tools-go to v0.1.4
internal/wasi: use internal/cm package
remove submodule src/vendor/github.com/ydnar/wasm-tools-go
GNUmakefile: add comment documenting what wasi-cm target does
go.{mod,sum}: remove toolchain; go mod tidy
go.mod: revert to Go 1.19
go.mod: go 1.19
go.{mod,sum}, internal/{tools,wasm-tools}: revert root go.mod file to go1.19
Create a wasm-tools specific module that can require go1.22 for wasm-tools-go.
Compilers like GCC keep adding new checks that produce new warnings.
Sometimes it can be false positives.
Do not treat such warnings in binaryen library as errors. tinygo won't
be able to provide zero warnings in its dependencies.
Closes#4332
This is done at a later time anyway, so doesn't need to be done in the
cgo package. In fact, _not_ doing it there makes it easier to print
correct relative packages.
This is needed for some improvements I'm going to make next.
This commit also refactors error handling slightly to make it more
easily testable, this should hopefully not result in any actual changes
in behavior.
Go code might sometimes want to use preprocessor macros that were passed
on the command line. This wasn't working before and resulted in the
following error:
internal error: could not find file where macro is defined
This is now supported, though location information isn't available
(which makes sense: the command line is not a file).
I had to use the `clang_tokenize` API for this and reconstruct the
original source location. Apparently this is the only way to do it:
https://stackoverflow.com/a/19074846/559350
In the future we could consider replacing our own tokenization with the
tokenizer that's built into Clang directly. This should reduce the
possibility of bugs a bit.
Uses a vendored "internal/diff" from Big Go to show the differences
between the expected and actual outputs when tests fail.
Signed-off-by: L. Pereira <l.pereira@fastly.com>
Wasm linear memory is always initialized to zero by definition,
so there's no need to waste time zeroing out this allocation. This
is the case for freshly-obtained memory from mmap().
Signed-off-by: L. Pereira <l.pereira@fastly.com>
* internal/wasi: update to wasm-tools-go@v0.1.1
See https://github.com/ydnar/wasm-tools-go/releases/tag/v0.1.1 for additional information.
* internal/wasi: update to wasm-tools-go@v0.1.2
* internal/wasi: regenerate with wasm-tools-go@v0.1.3
It assumed the maximum alignment was equal to sizeof(void*), which is
definitely not the case. So this only worked more or less by accident
previously.
It now uses the alignment as specified by the frontend, or else
`unsafe.Alignof(complex128)` which is typically the maximum alignment of
a given platform (though this shouldn't really happen in practice: the
optimizer should keep the 'align' attribute in place).
This adds something like 'align 4' to the runtime.alloc calls, so that
the compiler knows the alignment of the allocation and can optimize
accordingly.
The optimization is very small, only 0.01% in my small test. However, my
plan is to read the value in the heap-to-stack transform pass to make it
more correct and let it produce slightly more optimal code.
This means it's possible to test just a particular OS/arch with a
command like this:
go test -run=Build -target=linux/arm
I found it useful while working on MIPS support.
See: https://github.com/tinygo-org/tinygo/issues/4314
The os package isn't particularly useful on wasm-unknown, but just like
on baremetal systems it's imported by a lot of packages so it should at
least be possible to import this package.
The encoding/binary package in Go 1.23 imports the slices package, which
results in circular import when imported from the machine package.
Therefore, encoding/binary cannot be used in the machine package.
This commit fixes that by introducing a new internal/binary package that
is just plain Go code without dependencies. It can be safely used
anywhere (including the runtime if needed).
We previously picked a work-in-progress patch, but this is the proper
fix for this race condition. I think we should use that instead of
relying on the previous work-in-progress patch.
The machine package wasn't tested for every board. Therefore, add a new
serial-like test that also tries to import the machine package. This
should highlight potential issues in the future.
Go 1.18 has been unsupported for quite a while now (the oldest supported
version is Go 1.21). But more importantly, the golang.org/x/tools module
now requires Go 1.19 or later. So we'll drop this older version.
There's no need to keep looping if one of the uses makes it impossible
to convert a call to `runtime.stringToBytes()` with a raw pointer.
Signed-off-by: L. Pereira <l.pereira@fastly.com>
They should, but we weren't testing this.
I discovered this while working on
https://github.com/tinygo-org/macos-minimal-sdk/pull/4 which will likely
make math.h not work anymore. So I wanted to make sure we have a test in
place before we update that dependency.
Some devices, such as Seeed Studio XIAO with Adafruit bootloader, use complex names for UF2 device. With this fix applied, /proc/mounts should be parsed correctly
The current list of targets does not build wasm-ld.
wasm-ld is a symlink created in ./llvm-build/bin pointing to ./lld.
Add the "lld" build target to get wasm-ld into ./llvm-build/bin.
Fixes a build failure where wasm-ld is not found.
Signed-off-by: Christian Stewart <christian@aperture.us>
- Add ReadRegister and WriteRegister (because they're still used by
some packages).
- Fix out-of-bounds panic in I2C.Tx when either w or r has a length of
zero (or is nil).
Both of my development boards exhibit stability problems when
programming at the default 4000 kHz speed of the target/stmf32d4x
OpenOCD configuration.
Note that the speed limit must be set by an event handler, because it
is hard-coded by a similar event handler in stmf32f4x.cfg:
$_TARGETNAME configure -event reset-init {
# Configure PLL to boost clock to HSI x 4 (64 MHz)
...
# Boost JTAG frequency
adapter speed 8000 <-- resolves to 4000 kHz by the SWD interface
}
While here, replace the reference to the deprecated "stlink-v2"
configuration.
Appending to a slice can lead to a race condition if the capacity of the
slice is larger than the length (and therefore the returned slice will
overwrite some fields in the underlying array).
This fixes that race condition. I don't know how severe this particular
one is. It shouldn't be that severe, but it is a bug and it makes it
easier to hunt for possibly more serious race conditions.
sleepTicks calls machineLightSleep with the duration cast to an unsigned
integer. This underflow for negative durations.
Signed-off-by: Elias Naur <mail@eliasnaur.com>
* lint: expand to src/{os,reflect}, fix or suppress what it found
* internal/tools/tools.go: the tools idiom requires a build tag guard to avoid go test complaints
The openocd-commands option cannot refer to commands defined by the
target configuration. Lift this restriction by moving the commands to
after target loading.
See https://github.com/tinygo-org/tinygo/issues/4225
Runs in both circleci and github, circleci is run on branch push, github is run on PR
Revive builds so fast, don't bother installing it; saves us wondering which one we get
Uses tools.go idiom to give control over linter versions to go.mod.
Also pacifies linter re AppendToGlobal as a token first fix.
TODO: gradually expand the number of directories that are linted,
uncomment more entries in revive.toml, and fix or suppress the
warnings lint finds.
TODO: add linters "go vet" and staticcheck
NOT TODO: don't add metalinters like golangci-lint that pull in
lots of new of dependencies; we'd rather not clutter go.mod that
much, let alone open ourselves up to the additional attack surface.
This is much, _much_ faster than __tinygo_spi_transfer which can only
transfer a single byte at a time. This is especially important for
simulated displays.
I've already implemented the browser side of this on the playground and
have used this patch for local testing where it massively speeds up
display operations.
This makes all rp2040 boards available for simulation using
-tags=<board_name>. Importantly, this includes the Gopher Badge which
I'm working on to add to the TinyGo Playground.
This is makes it easier to use these in simulation, plus it avoids
duplication. The only downside I see is that more UARTs get defined than
a board supports, but no existing code should break (no UARTs get
renamed for example).
This eliminates the 'wasi' build tag in favor of 'GOOS=wasip1', introduced in Go 1.21.
For backwards compatablity, -target=wasi is a synonym for -target=wasip1.
It's not generally needed. It was added in
https://github.com/tinygo-org/tinygo/pull/3958 to fix an issue with
binaryen that has since been fixed in a different way, so we don't need
the googletest dependency anymore.
This way you can for example run `make test GOTESTPKGS=./builder` to
only test the builder package.
I've often done this by manually modifying the Makefile, so having a
make parameter available would make this much easier.
Support for `-panic=trap` was previously a pass in the optimization
pipeline. This change moves it to the compiler and runtime, which in my
opinion is a much better place.
As a side effect, it also fixes
https://github.com/tinygo-org/tinygo/issues/4161 by trapping inside
runtime.runtimePanicAt and not just runtime.runtimePanic.
This change also adds a test for the list of imported functions. This is
a more generic test where it's easy to add more tests for WebAssembly
file properties, such as exported functions.
getelementptr offsets are signed, not unsigned. Yet they were used as
unsigned integers in interp.
Somehow this worked most of the time, until finally there was some code
that did a getelementptr with a negative index.
This reduces the size of the Docker image from 13GB to 1.71GB, and
should therefore make CI of the drivers package much faster. It should
hopefully also avoid the out-of-space problem we currently have when
building the smoke tests for the drivers repo.
This size reduction is done by using multistage builds and only copying
the necessary files in the final stage.
This updates lib/cmsis-svd, pulling in the updates to the Espressif SVD
files here:
https://github.com/cmsis-svd/cmsis-svd-data/pull/3
This is needed for wifi/BLE support on the ESP32-C3 (the older SVD files
were missing some necessary interrupts).
CGo is needed for the rp2040 and for macOS (GOOS=darwin), without it
these targets just won't work. And there really isn't a benefit from
disabling CGo: we don't need any external linkers for example.
This avoids a somewhat common issue of people having CGO_ENABLED=0
somewhere in their environment and not understanding why things don't
work. See for example: https://github.com/tinygo-org/tinygo/issues/3450
I managed to get CGo sort-of working in VSCode (meaning that it will
typecheck code in the IDE itself) using a few crude hacks, but it
requires a few minor changes to the TinyGo standard library.
I intend to eventually add this support in the TinyGo extension for
VSCode directly, but for now I've manually updated .vscode/settings.json
to get IDE support. In any case, it would be nice to have this for when
I hopefully add this to the TinyGo extension eventually.
Node.js 16 is no longer supported, so we can drop support for it as
well.
This also means updating a whole lot of GitHub Actions versions, because
they were updated to work on Node.js 20 instead. For most actions this
should be a relatively small change, but the upload-aftifact action has
had some major changes (which should generally improve things a lot).
This means we can finally release native arm64 builds of TinyGo on
macOS!
Also update from macOS 11 to macOS 12, because macOS 11 is not supported
anymore.
Without this change, my tinygo builds crash with
SIGSEGV: segmentation violation
PC=0x10884f87c m=16 sigcode=2
signal arrived during cgo execution
goroutine 378 [syscall]:
runtime.cgocall(0x100631e44, 0x1401755be88)
/nix/store/4hf287252ilsdf2w36mfm8fa0rvbf33w-go-1.21.4/share/go/src/runtime/cgocall.go:157 +0x44 fp=0x1401755be50 sp=0x1401755be10 pc=0x1002a5084
tinygo.org/x/go-llvm._Cfunc_LLVMGoWriteThinLTOBitcodeToMemoryBuffer(0x123614160)
_cgo_gotypes.go:6078 +0x34 fp=0x1401755be80 sp=0x1401755be50 pc=0x1004b66b4
...
and reports using LLVM 16:
$ tinygo version
tinygo version 0.31.0-dev darwin/arm64 (using go version go1.21.4 and LLVM version 16.0.6)
I don't know why libcxx-16 is being included, but explicitly specifying
llvmPackages_17.libcxx fixes the crash and version skew.
These functions are used by the binary blobs that implement wifi/BLE on
the ESP32-C3. While this is a lot of lines of linker script, I think
it's a good idea to add them here (instead of in a separate library)
because they're part of the ESP32-C3 hardware.
The IsZero function was changed slightly in Go 1.22, so we'll need to
update it.
While this could in theory break existing programs that rely on the old
behavior, they'd also break with the imminent Go 1.22 release so I don't
think this is a real problem.
This fixes the new loop variable behavior in Go 1.22.
Specifically:
* The compiler (actually, the x/tools/go/ssa package) now correctly
picks up the Go version.
* If a module doesn't specify the Go version, the current Go version
(from the `go` tool and standard library) is used. This fixes
`go run`.
* The tests in testdata/ that use a separate directory are now
actually run in that directory. This makes it possible to use a
go.mod file there.
* There is a test to make sure old Go modules still work with the old
Go behavior, even on a newer Go version.
This function is needed for Go 1.22, and is used from various packages
like math/rand.
When there is no random number generator available, it falls back to a
static sequence of numbers. I think this is fine, because as far as I
can see it is only used for non-cryptographic needs.
A few non-generic functions like HashStrBytes have been kept for
backwards compatibility, so this package should continue to work with
older Go versions (as long as they support generics).
The right-hand grove port on the wioterminal can be used as UART, using
D0/D1 pins. The pins D0/D1 are tied to SERCOM4, so this patch exposes a
UART4 for sercom4 access.
RX = A0/D0 = PB08/SERCOM4.0 (port 4 pad 0)
TX = A1/D1 = PB09/SERCOM4.1 (port 4 pad 1)
Tested with Lora E5 UART.
uart : = machine.UART4
tx := machine.D0
rx := machine.D1
Note: must also cross Tx/Rx wires in grove cable. See
https://www.lucadentella.it/en/2022/01/29/wio-terminal-porta-grove-di-destra-e-moduli-uart/
* stub out more types/funcs to compile against golang.org/x/net/internal/socket
These are changes need to compile github.com/domainr/dnsr/ with TinyGo.
See issue https://github.com/tinygo-org/net/issues/14.
These change are mostly to fix missing symbols in src/crypto/tls and
src/net. Missing types and functions are cut-and-pasted from go1.21.4.
Functions are stubbed out returning errors.New("not implemented").
DNRS is compiled by running tinygo test:
sfeldma@nuc:~/work/dnsr$ tinygo test -target=wasi
With this patch, and a corresponding patch for tinygo-org/net to fixup
src/net, you should get a clean compile.
Thanks to @aykevl for actually finding and providing this fix, I really just
reported the problem and tested the fix.
Signed-off-by: deadprogram <ron@hybridgroup.com>
This PR adds a network device driver model called netdev. There will be a companion PR for TinyGo drivers to update the netdev drivers and network examples. This PR covers the core "net" package.
An RFC for the work is here: #tinygo-org/drivers#487. Some things have changed from the RFC, but nothing major.
The "net" package is a partial port of Go's "net" package, version 1.19.3. The src/net/README file has details on what is modified from Go's "net" package.
Most "net" features are working as they would in normal Go. TCP/UDP/TLS protocol support is there. As well as HTTP client and server support. Standard Go network packages such as golang.org/x/net/websockets and Paho MQTT client work as-is. Other packages are likely to work as-is.
Testing results are here (https://docs.google.com/spreadsheets/d/e/2PACX-1vT0cCjBvwXf9HJf6aJV2Sw198F2ief02gmbMV0sQocKT4y4RpfKv3dh6Jyew8lQW64FouZ8GwA2yjxI/pubhtml?gid=1013173032&single=true).
The default on MacOS is `md5`, while Nix only has `md5sum` available.
Therefore, make it possible to override the variable via the environment
so that flake.nix can set the correct binary name.
Hopefully this won't break anybody: while all tests still pass, there
could in theory be systems where not supplying those libraries leads to
linker errors.
When TinyGo is installed using `go install` or `go build`, it uses the
Clang resource directory from the host. This works for Linux, but
doesn't work anymore on macOS with a recent change I made.
This re-adds support for Darwin in that case (with much, much simpler
code than there used to be).
This adds a flake.nix file that makes it possible to quickly create a
development environment.
You can download Nix here, for use on your Linux or macOS system:
https://nixos.org/download.html
After you have installed Nix, you can enter the development environment
as follows:
nix develop
This drops you into a bash shell, where you can install TinyGo simply
using the following command:
go install
That's all! Assuming you've set up your $PATH correctly, you can now use
the tinygo command as usual:
tinygo version
You can also do many other things from this environment. Building and
flashing should work as you're used to: it's not a VM or container so
there are no access restrictions.
Set -resource-dir in a central place instead of passing the header path
around everywhere and adding it using the `-I` flag. I believe this is
closer to how Clang is intended to be used.
This change was inspired by my attempt to add a Nix flake file to
TinyGo.
This allows us to test and use LLVM 17, now that it is available in
Homebrew.
Full support for LLVM 17 (including using it by default) will have to
wait until Espressif rebases their Xtensa fork of LLVM.
These parts aren't critical and lead to crashes on small chips without
long jumps (like the attiny85) with LLVM 17. (Older LLVM versions would
emit long jumps regardless, even if the chip didn't support those).
For more information, see: https://github.com/llvm/llvm-project/issues/67042
This happens with `tinygo test` for example when testing multiple
packages at the same time. I found this because the compiler crashed in
`make tinygo-test-fast`:
fatal error: concurrent map writes
fatal error: concurrent map read and map write
goroutine 15 [running]:
github.com/tinygo-org/tinygo/builder.Build({0x40002d0be0, 0xa}, {0x0, 0x0}, {0x4000398048, 0x14}, 0x40003b0000)
/home/ayke/src/tinygo/tinygo/builder/build.go:131 +0x388
main.buildAndRun({0x40002d0be0, 0xa}, 0x40003b0000, {0x8bc178, 0x40003a4090}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}, ...)
/home/ayke/src/tinygo/tinygo/main.go:856 +0x45c
main.Test({0x40002d0be0, 0xa}, {0x8bc118, 0x40000b3a08}, {0x0?, 0x0?}, 0x40001e6000, {0x0, 0x0})
/home/ayke/src/tinygo/tinygo/main.go:270 +0x758
main.main.func3()
/home/ayke/src/tinygo/tinygo/main.go:1718 +0xe4
created by main.main in goroutine 1
/home/ayke/src/tinygo/tinygo/main.go:1712 +0x34ec
My solution is essentially to copy the map over instead of modifying it
directly.
I've also moved the code up a little, because I think that's a more
sensible place (out of the way of the whole package compile logic).
Previously all (except one!) usage of goenv.Version manually added the
git sha1 hash, leading to duplicate code. I've moved this to do it all
in one place, to avoid this duplication.
The old LLVM pass manager is deprecated and should not be used anymore.
Moreover, the pass manager builder (which we used to set up a pass
pipeline) is actually removed from LLVM entirely in LLVM 17:
https://reviews.llvm.org/D145387https://reviews.llvm.org/D145835
The new pass manager does change the binary size in many cases: both
growing and shrinking it. However, on average the binary size remains
more or less the same.
This is needed as a preparation for LLVM 17.
This is a big change: apart from removing LLVM 14 it also removes typed
pointer support (which was only fully supported in LLVM up to version
14). This removes about 200 lines of code, but more importantly removes
a ton of special cases for LLVM 14.
Prior to this commit, the build instructions were encoded in Makefiles
that contained GNU extensions that are unique to GNU Make and
incompatible with older implementations.
Thie commit renames all Makefiles to GNUmakefile which clearly denotes
that the file contains GNU extensions.
GNU Make actually first looks for a GNUmakefile, then makefile, THEN
Makefile so in addition to simply being more correct and portable, it
saves a few unnecessary failed attempts to open the correct build
file.
Prior to this commit, our Makefiles assumed the name of the make program
was simply "make".
Since we use GNU make extensions, this isn't always the case --
operating systems like FreeBSD come with their own implementation named
make which can be incompatible with the extensions used in by tinygo.
To run a compatible make on some of these systems, we have explicitly
call GNU make by running gmake.
This commit changes references to the command "make" to "$(MAKE)" which
is a variable that contains the name of the executable invoked to
process the Makefile.
This allow the Makefiles to be uniformly processed by the same make
program.
This fixes https://github.com/tinygo-org/tinygo/issues/3926.
While working on this I've found another bug: if C.bool is referenced
from within Go, it isn't available anymore on the C side. This is an
existing bug that also applies to float and double, but may be less
likely to be triggered there.
This bug is something to be fixed at a later time (it has something to
do with preprocessor defines).
Previously, this pass would convert any read-only use of a
runtime.stringToBytes call to use the original string buffer instead.
This is incorrect: if there are any writes to the resulting buffer, none
of the slice buffer pointers can be converted to use the original
read-only string buffer.
This commit fixes that bug and adds a test to prove the new (correct)
behavior.
The old traceback would look like this:
# internal/godebug
/usr/local/go/src/internal/godebug/godebug.go:101:11: interp: test
call <2> 0 <3> 0
traceback:
/usr/local/go/src/internal/godebug/godebug.go:101:11:
call <2> 0 <3> 0
/usr/local/go/src/internal/godebug:
call <1> 0
With this patch, it looks like this:
# io/fs
/usr/local/go/src/io/fs/fs.go:144:45: interp: test
%0 = load %runtime._interface, ptr @"internal/oserror.ErrInvalid", align 8, !dbg !316
traceback:
/usr/local/go/src/io/fs/fs.go:144:45:
%0 = load %runtime._interface, ptr @"internal/oserror.ErrInvalid", align 8, !dbg !316
/usr/local/go/src/io/fs/fs.go:137:28:
%0 = call %runtime._interface @"io/fs.errInvalid"(ptr undef), !dbg !317
For developers (like me) who are familiar with LLVM, this is probably
easier to read. For users, I'm not sure: the instructions have quite a
lot of distracting fluff in them. But at least it contains the function
names that are called (which are not currently present in the old
traceback).
...that said, having the LLVM instructions in a bug report is probably
going to be easier for people who are familar with LLVM.
This is a small release just in time for GopherCon US.
Some of the big changes of this release are:
- switch to LLVM 16
- fixes for two separate hard-to-reproduce compiler crashes
- added support for the Adafruit Gemma M0
This usually works by chance, but leads to crashes. So we should never
ever do this.
I'm pretty sure this is the crash behind this issue: https://github.com/tinygo-org/tinygo/issues/3894
It may also have caused this crash: https://github.com/tinygo-org/tinygo/issues/3874
I have a suspicion this is also behind the rather crash-prone CircleCI
jobs, that we haven't been able to find the source of. But we'll find
out soon enough once this fix is merged.
To avoid hitting this issue again in the future, I've created a PR to
remove these dangerous functions altogether from the go-llvm API:
https://github.com/tinygo-org/go-llvm/pull/54
This commit adds support for LLVM 16 and switches to it by default. That
means three LLVM versions are supported at the same time: LLVM 14, 15,
and 16.
This commit includes work by QuLogic:
* Part of this work was based on a PR by QuLogic:
https://github.com/tinygo-org/tinygo/pull/3649
But I also had parts of this already implemented in an old branch I
already made for LLVM 16.
* QuLogic also provided a CGo fix here, which is also incorporated in
this commit:
https://github.com/tinygo-org/tinygo/pull/3869
The difference with the original PR by QuLogic is that this commit is
more complete:
* It switches to LLVM 16 by default.
* It updates some things to also make it work with a self-built LLVM.
* It fixes the CGo bug in a slightly different way, and also fixes
another one not included in the original PR.
* It does not keep compiler tests passing on older LLVM versions. I
have found this to be quite burdensome and therefore don't generally
do this - the smoke tests should hopefully catch most regressions.
This was bug https://github.com/tinygo-org/tinygo/issues/3890.
See the diff for details. Essentially, it means that `copy` in the
interpreter was copying data that wasn't known yet, or copying data into
a slice that could be read externally after the interp pass.
such as program changes and pitch bend. Also add error handling for invalid
parameter values such as MIDI channel. This however makes a somewhat breaking
change to the current implementation, in that we now use the typical MIDI user
system of counting MIDI channels from 1-16 instead of from 0-15 as the lower
level USB-MIDI API itself expects.
Also add constant values for continuous controller messages, rename SendCC
function, and add SysEx function.
Signed-off-by: deadprogram <ron@hybridgroup.com>
Instead of markGlobals calling markRoots unconditionally (which doesn't
make sense for -gc=none and -gc=leaking), provide markRoots as a
callback function.
This is in preparation for -gc=boehm, where the previous design is even
more awkward and a callback makes far more sense.
I've tested the size impact using `make smoketest XTENSA=0`. There is
none, except for two cases:
* One with `-opt=0` so const-propagation for the callback didn't take
place.
* One other on AVR, I don't know why but as it's only 16 bytes in a
very specific case I'm going to assume it's just a random change in
compiler output that caused a size difference.
This adds true GOOS=wasip1 support in addition to our existing
-target=wasi support. The old support for WASI isn't removed, but should
be treated as deprecated and will likely be removed eventually to reduce
the test burden.
It looks like this on my system, for example:
{
"target": {
"llvm-target": "aarch64-unknown-linux",
"cpu": "generic",
"features": "+neon",
"goos": "linux",
"goarch": "arm64",
"build-tags": [
"linux",
"arm64"
],
"gc": "precise",
"scheduler": "tasks",
"linker": "ld.lld",
"rtlib": "compiler-rt",
"libc": "musl",
"default-stack-size": 65536,
"ldflags": [
"--gc-sections"
],
"extra-files": [
"src/runtime/asm_arm64.S",
"src/internal/task/task_stack_arm64.S"
],
"gdb": [
"gdb"
],
"flash-1200-bps-reset": "false"
},
"goroot": "/home/ayke/.cache/tinygo/goroot-23c311bcaa05f188affa3c42310aba343acc82562d5e5f04dea9d5b79ac35f7e",
"goos": "linux",
"goarch": "arm64",
"goarm": "6",
...
}
This can be very useful while working on the automatically generated
target object for example (in my case, GOOS=wasip1).
This should not happen under normal circumstances. It can still happen
when there is a mismatch between TinyGo version and the associated
runtime, or while developing the compiler package.
When a function is exported using //export, but also had a
//go:wasm-module pragma, the //export name was ignored. The
//go:wasm-module doesn't actually do anything besides breaking the
export (exported functions don't have a module name).
I've refactored and cleaned up the code, and in the process removed this
weird edge case.
This commit adds I2C timeouts for nrf51 and nrf52 (but not yet for
others like nrf52840).
Tested on the PineTime, where I now got a timeout instead of hanging and
resetting due to a watchdog reset.
* add gosched calls to UART
* add UART.flush() stubs for all supported architectures
* add comment un uart.go on flush functionality
* uart.writeByte as base of UART usage
* fix NXP having duplicate WriteByte
* fix writeByte not returning error on some platforms
* add flush method for fe310 device
* check for error in WriteByte call to writeByte
This replaces our own manual detection of various variables (GOROOT,
GOPATH, Go version) with a simple call to `go env`.
If the `go` command is not found:
error: could not find 'go' command: executable file not found in $PATH
If the Go version is too old:
error: requires go version 1.18 through 1.20, got go1.17
If the Go tool itself outputs an error (using GOROOT=foobar here):
go: cannot find GOROOT directory: foobar
This does break the case where `go` wasn't available in $PATH but we
would detect it anyway (via some hardcoded OS-dependent paths). I'm not
sure we want to fix that: I think it's better to tell users "make sure
`go version` prints the right value" than to do some automagic detection
of Go binary locations.
This is a rewrite of how filesystems are detected. Specifically, it
fixes an issue on Linux where the location of the FAT filesystem can
vary between distributions (for example, we supported most distros by
checking two different paths, but NixOS uses a different path): it now
uses the data in /proc/mounts instead which should be universal.
This adds FieldByNameFunc, which some libraries like reflect2 need.
For my usecase I could also just stub FieldByNameFunc to panic, but
figured that it would work OK to just make it work. I'm not sure if
the overhead to FieldByName using a closure is acceptable.
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Found while working on the PineTime. For some reason it still kind of
works in most cases, but I was hitting this issue when interacting with
two different I2C devices (the touch sensor and the BMA421).
Browsers previously didn't support the WebAssembly i64 type, so we had
to work around that limitation by converting the LLVM i64 type to
something else. Some people used a pair of i32 values, but we used a
pointer to a stack allocated i64.
Now however, all major browsers and Node.js do support WebAssembly
BigInt integration so that i64 values can be passed back and forth
between WebAssembly and JavaScript easily. Therefore, I think the time
has come to drop support for this workaround.
For more information: https://v8.dev/features/wasm-bigint (note that
TinyGo has used a slightly different way of passing i64 values between
JS and Wasm).
For information on browser support: https://webassembly.org/roadmap/
The Linux artifacts have clear names (linux-amd64-double-zipped etc),
but the MacOS and Windows ones didn't. This patch renames these artifact
names to be more readable, especially when downloading the artifacts.
These are some major or breaking changes:
- Reflect support got improved a lot.
- Interrupts became more strict: heap allocations an blocking
operations, which have always been undefined behavior, now result in
a panic.
- `//go:wasmimport` was added
- various new flags were added to `tinygo test`
- the source location for panics is printed in the `-monitor` output
The regular port access is around 4 cycles, instead of the usual 2
cycles for a store instruction on Cortex-M0+. The IOBUS however is
faster, I didn't measure exactly but I guess it's 2 cycles as expected.
This fixes a bug in the WS2812 driver that only happens on samd21 chips:
https://github.com/tinygo-org/drivers/issues/540
I didn't add this method in the initial PR.
Also, I found that a few of my assumptions were incorrect. I've changed
the code that configures the pin to make input (floating and pullup)
actually work. These chips really are quite different from all the older
AVRs.
If a pointer value was xor'ed with a value other than 0, it would not
have been run at runtime but instead would fall through to the generic
integer operations. This would likely result in a "cannot convert
pointer to integer" panic.
This commit fixes this subtle case.
This is just support for the chip, no boards are currently supported.
However, you can use this target on a custom board.
Notes:
- This required a new runtime and machine implementation, because the
hardware is actually very different (and much nicer than older
AVRs!).
- I had to update gen-device-avr to support this chip. This also
affects the generated output of other AVRs, but I checked all chips
we support and there shouldn't be any backwards incompatible
changes.
- I did not implement peripherals like UART, I2C, SPI, etc because I
don't need them. That is left to do in the future.
You can flash these chips with only a UART and a 1kOhm resistor, which
is really nice (no special hardware needed). Here is the program I've
used for this purpose: https://pypi.org/project/pymcuprog/
This refactors gen-device-avr to output two different formats: one for
all the existing AVR chips (that don't really have the concept of a
peripheral, just a bunch of registers), and one for all the new chips
like the ATtiny1616 (tinyAVR 1-series and 2-series) that have
peripherals like the Cortex-M chips with type structs and instances.
I checked the generated code for all the AVR chips we have support for
(atmega1280, atmega1284p, atmega2560, atmega328p, atmega32u4, attiny85)
and while the generated Go code did change, it looks safe to me.
This is a small change that's not really important in itself, but it
avoids duplicate errors in a future commit that adds error messages to
//go:wasmimport.
This basically reverts https://github.com/tinygo-org/tinygo/pull/3357
and replaces it with a different mechanism to get to the same goal.
I do not think filtering tags like this is a good idea: it's the wrong
part of the compiler to be concerned with such tags (that part sets
tags, but doesn't modify existing tags). Instead, I've written the
//go:build lines in such a way that it has the same effect: WASI
defaults to leveldb, everything else defaults to fnv, and it's possible
to override the default using build tags.
It could be expensive to call Size() three times, and it is unnecessary.
Instead, do it only once.
This results in a very small reduction of binary size if Zero() is used.
This makes reviewing PRs a lot easier because I don't have to run this
myself :)
This only uses the drivers repo so far, which is a good starting point
but doesn't include binary size changes for WebAssembly for example. A
future change could add some real-world programs to get a better idea of
the real-world impact.
To be clear: the intention is not to just look at the number at the
bottom. It is important to look at the actual size difference to see the
overall pattern (like, the difference may be due to a few outlier).
This avoids a dependency on nrfutil. I have verified that it creates
equivalent zip files to a wasp-os DFU zip file I downloaded here:
https://github.com/wasp-os/wasp-os/releases/
I have also tested that it produces valid DFU files that can be uploaded
using the dfu.py program here to my PineTime:
https://github.com/wasp-os/ota-dfu-python/tree/3d6fd30d33c2b20bc86ff6b9269fddf4a1d4c7c6
There are some minor differences in the generated file that should not
matter in practice (JSON whitespace, firmware file name, zip
compression).
* Initialize the ADC in Configure() (instead of in Get()).
* Do not set all channels to "not connected" - that's already the
reset value.
* Don't disable the ADC after use. It's not necessary to disable
(current consumption appears to remain the same whether enabled or
disabled).
* Rename pinetime-devkit0 to pinetime because the production device is
almost the same hardware (the only noticeable difference is a
different accelerometer, which isn't part of the board file).
* Remove the UART and set serial to none. The UART uses a lot of
current by default, so it seems better to disable it.
This is a breaking change, but honestly I think I'm the only one who has
ever actually used TinyGo for the PineTime and I'm fine with this
change :)
This tool can be very useful to compare binary sizes as output by
`-size=short`.
This is a tool I wrote a while ago. It's not perfect (we should probably
use a geomean) but it works well enough to get a good idea on the binary
size impact of a change.
The old code was broken and led to a HardFault in a rather convoluted
way:
1. The CFA offset was incorrect, in fact it was not aligned (the stack
is supposed to always be aligned to 4 bytes at least).
2. This unaligned size was then used for stack size calculations.
3. A stack that wasn't a multiple of 4 was allocated.
4. The calleeSavedRegs struct (in `(internal/task.state).archInit`) was
not correctly aligned.
5. Writing to this struct resulted in a HardFault.
The previous commit started printing the instruction address for runtime
panics. This commit starts using this address to print the source
location.
Here is an example where this feature is very useful. There is a heap
allocation in the Bluetooth package, but we don't know where exactly.
Printing the instruction address of the panic is already useful, but
what is even more useful is looking up this address in the DWARF debug
information that's part of the binary:
$ tinygo flash -target=circuitplay-bluefruit -monitor ./examples/heartrate
Connected to /dev/ttyACM0. Press Ctrl-C to exit.
tick 00:00.810
tick 00:01.587
tick 00:02.387
tick 00:03.244
panic: runtime error at 0x00027c4d: alloc in interrupt
[tinygo: panic at /home/ayke/src/tinygo/bluetooth/adapter_sd.go:74:4]
To be clear, this path isn't stored on the microcontroller. It's stored
as part of the build, and `-monitor` just looks up the path from the
panic message.
Possible enhancements:
- Print such an address for regular panics as well. I'm not sure
that's so useful, as it's usually a lot easier to look up panics
just by their message.
- Use runtimePanicAt (instead of runtimePanic) in other locations, if
that proves to be beneficial.
- Print the TinyGo-generated output in some other color, to
distinguish it from the regular console output.
- Print more details when panicking (registers, stack values), and
print an actual backtrace.
This is not very useful in itself, but makes it possible to detect this
address in the output. See the next commit.
This adds around 50 bytes to each binary (except for AVR and wasm). This
is unfortunate, but I think this feature is quite useful still.
A future enhancement might be to create a build tag for extended panic
information that's not set by default.
Multisampling/averaging (using the Samples configuration property) was
returning incorrect values. When I investigated this, I found that the
samd51 gives erratic values when using multisampling together with fewer
than 16 bits resolution.
I fixed this by forcing 16 bit resolution when multisampling, and
adjusting the output to account for multisampling.
Found while reading the battery value on a pybadge, which gave
non-sensible values with Samples set to a value larger than 1.
This improves slightly. It also is some groundwork for better DMA
support in TinyGo in the future.
I'm not entirely sure why it improves performance (in theory the old
code should already saturate the SPI bus) but it does, so 🤷
The SPI frequency was rounded up, not rounded down. This meant that if
you wanted to configure 15MHz for example, it would pick the next
available frequency (24MHz). That's unsafe, the safe option is to round
down and the SPI support for most other chips also rounds down for this
reason.
In addition, I've improved SPI clock selection so that it will pick the
best clock of the two, widening the available frequencies. See the
comments in the patch for details.
I find myself consistently running tests, seeing them panic, and then
immediately running them again with this environment variable set. It's
easier to just have tinygo do this for me.
The only thing that's different between all these chips is the flash
size, which can easily be passed as a linker flag instead. This removes
a bunch of duplicate code in an uncommon language (linker script).
I've also fixed a few boards with incorrect flash sizes:
* nano-rp2040 has 16MB instead of 2MB
* macropad-rp2040 has 8MB instead of 2MB
* gopher-badge has 8MB instead of 1MB
These functions can be implemented more efficiently using LLVM
intrinsics. That makes them the Go equivalent of functions like
__builtin_clz which are also implemented using these LLVM intrinsics.
I believe the Go compiler does something very similar: IIRC it converts
calls to these functions into optimal instructions for the given
architecture.
I tested these by running `tinygo test math/bits` after uncommenting the
tests that would always fail (the *PanicZero and *PanicOverflow tests).
As discussed on Slack, I believe this property does more harm than good:
* I don't think it's used anywhere. None of the drivers use it.
* It is not fully implemented. While values <= 8 might work fine,
values larger than 8 result in extra zero bits (instead of anything
sensible).
* Worse, it doesn't return an error when it's out of range. This is
not an optional property: if the SPI peripheral doesn't support a
particular number of bits, it should return an error instead of
silently limiting the number of bits. This will be confusing to
users.
Therefore, I propose we drop it. Maybe there are good uses for it
(perhaps for displays that use big endian 16-bit values?), but without a
good use case like a driver in tinygo.org/x/drivers, I think it's more
trouble than it's worth.
The runtime.stringFromBytesTyped and runtime.stringToBytesTyped
functions aren't really necessary, because they have the same LLVM IR
signature. Therefore, remove them and link directly to the functions
that the compiler uses internally.
This gives a small improvement now, and is needed to be able to use the
Heap2Stack transform that's available in the Attributor pass. This
Heap2Stack transform could replace our custom OptimizeAllocs pass.
Most of the changes are just IR that changed, the actual change is
relatively small.
To give an example of why this is useful, here is the code size before
this change:
$ tinygo build -o test -size=short ./testdata/stdlib.go
code data bss | flash ram
95620 1812 968 | 97432 2780
$ tinygo build -o test -size=short ./testdata/stdlib.go
code data bss | flash ram
95380 1812 968 | 97192 2780
That's a 0.25% reduction. Not a whole lot, but nice for such a small
patch.
There was a very subtle bug in the ADC read code: it stores a pointer to
a variable in a register, waits for the hardware to complete the read,
and then reads the value again from the local variable. Unfortunately,
the compiler doesn't know there is some form of synchronization
happening in between.
This can be fixed in roughly two ways:
* Introduce some sort of synchronization.
* Do a volatile read from the variable.
I chose the second one as it is probably the least intrusive. We
certainly don't need atomic instructions (the chip is single threaded),
we just need to tell the compiler the value could have changed by making
the read volatile.
It is possible to create function-local named types:
func foo() any {
type named int
return named(0)
}
This patch makes sure they don't alias with named types declared at the
package scope.
Bug originally found by Damian Gryski while working on reflect support.
Previously we only supported recursive types in structs. But there can
be other kinds of recursive types, like slices:
type RecursiveSlice []RecursiveSlice
This doesn't involve structs, so it led to infinite recursion in the
compiler. This fix avoids recursion at the proper level: at the place
where the named type is defined.
This test only applies when using the built-in LLVM version. This way,
we have a stable LLVM version to test against. Distribution versions of
LLVM (especially Debian) tend to be patched in a way that affect the
results.
This has two benefits:
1. It attributes these bytes to the internal/task package (in
-size=full), instead of (unknown).
2. It makes it possible to print the stack sizes variable in GDB.
This is what it might look like in GDB:
(gdb) p 'internal/task.stackSizes'
$13 = {344, 120, 80, 2048, 360, 112, 80, 120, 2048, 2048}
Previously we were using a really weird calculation to determine the
alignment in bits - written by me (no idea what I was thinking at the
time, it's obviously incorrect). Just replace it with the much more
obviously correct multiplication by 8 to get bits from bytes.
Found while working on properly dealing with alignment in `-size=full`.
The compiler now uses DW_OP_plus_uconst in the address attribute of
DWARF variables. To be able to understand these addresses, we need to be
able to parse this opcode.
This commit should also improve the reliability of address parsing a
bit.
Previously the code size of the compiler-rt library might be displayed
like this:
```
1050 0 0 0 | 1050 0 | /home/ayke/src/tinygo/tinygo/llvm-project/compiler-rt/lib/builtins
146 0 0 0 | 146 0 | /home/ayke/src/tinygo/tinygo/llvm-project/compiler-rt/lib/builtins/arm
```
With this change, it is displayed nicely like this:
```
1196 0 0 0 | 1196 0 | C compiler-rt
```
This should fix a number of concurrency/threading issues.
I had to force-disable concurrency in the linker using a hack. I'm not
entirely sure what the cause is, possibly the MinGW version (version 12
appears to work for me, while version 11 as used on the GitHub runner
image seems to be broken).
There are a few ways to fix this in a better way:
* Fix the underlying cause (possibly by upgrading to MinGW-w64 12).
* Add the `--threads` flag to the LLD MinGW linker, so we can use a
regular parameter instead of this hack.
This is an attempt to figure out why the Windows CI keeps crashing. It
should be removed before the next release, by which point we should know
whether this has helped or not.
There was a mostly benign race condition in the compiler. The issue was
that there is a check for type aliases (which can alias types in another
function), but this check was _after_ accessing a property of the
function that might not have been completed.
I don't think this can have any serious effects, as the function is
skipped anyway, but such bugs should certainly be fixed.
If there is no source location, at least use the file (but not line) for
debug information.
This attributes almost 1kB of string data for ./testdata/stdlib.go when
compiling for WASI.
This is helpful for WebAssembly: it makes it possible to attribute many
more data to locations in the source code, which makes `-size=full` more
useful.
Removes usage of AsmFull which required an optimization pass to remove the map parameter passed into it. This caused issues when compiling with -opt=0 where memory for the map was being allocated as an unintended side-effect of using AsmFull with no optimizations enabled.
I have checked this conversion is not needed anymore after the previous
commit, by running various smoke tests of which none triggered this
optimization. The only case where the optimization would have kicked in
is in syscall/syscall_windows.go:76 of the Go standard library.
Therefore, I prefer to remove it to reduce code complexity.
We have an optimization for this specific pattern, but it's really just
a hack. With the addition of unsafe.Add in Go 1.17 we can directly
specify the intent instead and eventually remove this special case.
The code is also easier to read.
This metadata is emitted by Clang and I found it is important for source
level debugging on MacOS. This patch does not get source level debugging
to work yet (for that, it seems like packages need to be built
separately), but it is a step in the right direction.
machine/stm32, nrf: implement machine.Flash
Implements the machine.Flash interface using the same definition as the tinyfs BlockDevice.
This implementation covers the stm32f4, stm32l4, stm32wlx, nrf51, nrf52, and nrf528xx processors.
Blocking inside an interrupt is always unsafe and will lead to all kinds
of bad behavior (even if it might appear to work sometimes). So disallow
it, just like allocating heap memory inside an interrupt is not allowed.
I suspect this will usually be caused by channel sends, like this:
ch <- someValue
The easy workaround is to make it a non-blocking send instead:
select {
case ch <- someValue:
default:
}
This does mean the application might become a bit more complex to be
able to deal with this case, but the alternative (undefined behavior) is
IMHO much worse.
Add the timers test because they now work correctly on AVR, probably as
a result of the reflect refactor: https://github.com/tinygo-org/tinygo/pull/2640
I've also updated a few of the other tests to indicate the new status
and why they don't work. It's no longer because of compiler errors, but
because of linker or runtime errors (which is at least some progress).
For example, I found that testdata/reflect.go works if you disable
`testAppendSlice` and increase the stack size.
It can be difficult to find what went wrong in a test. Omitting -v
should make it easier to see the failing tests and the output for them
(note that output is still printed for tests that fail).
Some vector registers must be preserved across calls, but this wasn't
happening on Linux and MacOS. When I added support for windows/arm64, I
saw that it required these vector registers to be preserved and assumed
this was Windows deviating from the standard calling convention. But
actually, Windows was just implementing the standard calling convention
and the bug was on Linux and MacOS.
This commit fixes the bug on Linux and MacOS and at the same time merges
the Go and assembly files as they no longer need to be separate.
Running binaries in QEMU (when debugging on Linux for example) did not
work correctly as qemu-user expects the `-g` flag to be first on the
command line before the program name. Putting it after will make it a
command line parameter for the emulated program, which is not what we
want.
I don't think this ever worked correctly.
This is a big commit that changes the way runtime type information is stored in
the binary. Instead of compressing it and storing it in a number of sidetables,
it is stored similar to how the Go compiler toolchain stores it (but still more
compactly).
This has a number of advantages:
* It is much easier to add new features to reflect support. They can simply
be added to these structs without requiring massive changes (especially in
the reflect lowering pass).
* It removes the reflect lowering pass, which was a large amount of hard to
understand and debug code.
* The reflect lowering pass also required merging all LLVM IR into one
module, which is terrible for performance especially when compiling large
amounts of code. See issue 2870 for details.
* It is (probably!) easier to reason about for the compiler.
The downside is that it increases code size a bit, especially when reflect is
involved. I hope to fix some of that in later patches.
This was actually surprising once I got TinyGo to build on Windows 11
ARM64. All the changes are exactly what you'd expect for a new
architecture, there was no special weirdness just for arm64.
Actually getting TinyGo to build was kind of involved though. The very
short summary is: install arm64 versions of some pieces of software
(like golang, cmake) instead of installing them though choco. In
particular, use the llvm-mingw[1] toolchain instead of using standard
mingw.
[1]: https://github.com/mstorsjo/llvm-mingw/releases
Using ThinLTO manages to optimize binaries quite significantly. The
exact amount varies a lot by program but it's about 10-15% usually.
Don't remove non-ThinLTO support yet. It would not surprise me if this
triggered some unintended side effect. Eventually, non-ThinLTO support
should be removed though.
This allows you to expand {tmpDir} in the json "emulator" field, and
uses it in wasmtime instead of custom TMPDIR mapping logic.
Before, we had custom logic for wasmtime to create a separate tmpDir
when running go tests. This overwrite the TMPDIR variable when running,
after making a mount point. A simpler way to accomplish the end goal of
writing temp files is to use wasmtime's map-dir instead. When code is
compiled to wasm with the wasi target, tempDir is always /tmp, so we
don't need to add variables (since we know what it is). Further, the
test code is the same between normal go and run through wasmtime. So, we
don't need to make a separate temp dir first, and avoiding that reduces
logic, as well makes it easier to swap out the emulator (for wazero
which has no depedencies). To map the correct directory, this introduces
a {tmpDir} token whose value is the host-specific value taken from
`os.TempDir()`.
The motivation I have for this isn't so much to clean up the wasmtime
code, but allow wazero to execute the same tests. After this change, the
only thing needed to pass tests is to change the emulator, due to
differences in how wazero deals with relative lookups (they aren't
restricted by default, so there's not a huge amount of custom logic
needed).
In other words, installing wazero from main, `make tinygo-test-wasi`
works with no other changes except this PR and patching
`targets/wasi.json`.
```json
"emulator": "wazero run -mount=.:/ -mount={tmpDir}:/tmp {}",
```
On that note, if there's a way to override the emulator via arg or env,
this would be even better, but in any case patching json is fine.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
I found that when I enable ThinLTO, a miscompilation triggers that had
been hidden all the time previously. The bug appears to happen as
follows:
1. TinyGo generates a function with a runtime.trackPointer call, but
without an alloca (or the alloca gets optimized away).
2. LLVM sees that no alloca needs to be kept alive across the
runtime.trackPointer call, and therefore it adds the 'tail' flag.
One of the effects of this flag is that it makes it undefined
behavior to keep allocas alive across the call (which is still safe
at that point).
3. The GC lowering pass adds a stack slot alloca and converts
runtime.trackPointer calls into alloca stores.
The last step triggers the bug: the compiler inserts an alloca where
there was none before but that's not valid as long as the 'tail' flag is
present.
This patch fixes the bug in a somewhat dirty way, by always creating a
dummy alloca so that LLVM won't do the optimization in step 2 (and
possibly other optimizations that rely on there being no alloca
instruction).
This is purely a refactor, to make the next change simpler.
The wordpack functionality used to be necessary in transform passes, but
now it isn't anymore so let's not complicate this more than necessary.
This implements the block-based GC as a partially precise GC. This means
that for most heap allocations it is known which words contain a pointer
and which don't. This should in theory make the GC faster (because it
can skip non-pointer object) and have fewer false positives in a GC
cycle. It does however use a bit more RAM to store the layout of each
object.
Right now this GC seems to be slower than the conservative GC, but
should be less likely to run out of memory as a result of false
positives.
This makes it much easier to read the value at runtime, as pointer
indices are naturally little endian. It should not affect anything else
in the program.
Most of the code of the conservative GC can be reused for the precise
GC. So before adding precise GC support, this commit just moves code
around to make the next commit cleaner. It is a non-functional change.
Proof: https://godbolt.org/z/as4EM3713
Essentially, this means that there are objects on arm64 that have a
16-byte alignment and so we have to respect that when we allocate things
on the heap.
This function provides a mechanism to watch for changes to the GODEBUG
environment variable. For now, we'll not implement it. It might be
useful in the future, when it can always be added.
Before this patch, a compile error would prevent the 'ok' or 'FAIL' line
to be printed. That's unexpected. This patch changes the code in such a
way that it's obvious a test result line is printed in all cases.
To be able to also print the package name, I had to make sure the build
result is passed through everywhere even on all the failure paths. This
results in a bit of churn, but it's all relatively straightforward.
Found while working on Go 1.20.
There were two types that could result in a compiler stack overflow.
This is difficult to fix in LLVM 14, so I won't even bother. However,
this is trivial to fix with opaque pointers in LLVM 15. Therefore, this
fix is for LLVM 15 only.
Fixes: https://github.com/tinygo-org/tinygo/issues/3341
wasmtime by default will assume the subcommand is "run" vs one of its
others, but being explicit helps clarify the actual command invoked.
For example, we pass similar looking args to wasmtime and also wasi.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
We don't support these yet so let's just put them in a central location.
Once these functions are supported we can think about how to structure
the code again.
This gets rid of the following messages:
ld.lld: warning: duplicate /export option: hypot
ld.lld: warning: duplicate /export option: nextafter
I've wanted to wait for the next release but that may take a long while,
so I've simply set the submodule to the commit that fixes this message.
ThinLTO results in a small code size reduction, which is nice
(especially on these very small chips). It also brings us one step
closer to using ThinLTO everywhere.
LLVM wasn't aware that runtime.stackChainStart must be kept live and
can't be optimized away. With this hack, it is forced to consider
stackChainStart live at the time of the stack scan.
This fixes the corruption in https://github.com/tinygo-org/tinygo/issues/3277
This reverts commit 0b3a7280fa and updates
the documentation a little bit to explain the purpose of -gc=none. (I'm
thinking about the attiny10 by the way where defaulting to -gc=none
makes sense).
I found that .data is not correctly initialized with firmware images
that use over 64kB of flash. The problem is that the data in .data
(which is stored in flash, and copied to RAM at reset) is beyond the
64kB limit and must therefore be loaded using the elpm instruction
instead of the lpm instruction.
I encountered this issue while getting testdata/math.go to work for AVR.
The following command mostly works with this patch, while it prints
garbage withtout it:
tinygo run -target=simavr -size=short -scheduler=none ./testdata/math.go
(This also requires a patch to picolibc to work, see
https://github.com/picolibc/picolibc/pull/371)
It still doesn't work entirely with this patch: some of the math
operations have an incorrect result. But at least it's an improvement as
it won't print garbage anymore.
- Use compiler-rt and picolibc instead of avr-libc.
- Use ld.lld instead of avr-ld (or avr-gcc).
This makes it much easier to get started with TinyGo on AVR because
installing these extra tools (gcc-avr, avr-libc) can be a hassle.
It also opens the door for future improvements such as ThinLTO.
There is a code size increase but I think it's worth it in the long run.
The code size increase can hopefully be reduced with improvements to the
LLVM AVR backend and to compiler-rt.
Before this patch, `tinygo lldb path/to/package` would result in an
error:
(lldb) target create --arch=arm64-unknown-macosx10.12.0 "/var/folders/17/btmpymwj0wv6n50cmslwyr900000gn/T/tinygo2731663853/main"
error: the specified architecture 'arm64-unknown-macosx10.12.0' is not compatible with 'arm64-apple-macosx10.12.0' in '/var/folders/17/btmpymwj0wv6n50cmslwyr900000gn/T/tinygo2731663853/main'
This patch fixes this error.
Unfortunately, it doesn't get debug information to work yet. I still
haven't figured out what's going wrong here. But it's progress, I guess.
This is now possible because we're using the LLVM linker. It results in
some very minor code size reductions. The main benefit however is
consistency: eventually, all targets will support ThinLTO at which point
we can remove support for GNU linkers and simplify the compiler.
The Espressif fork of LLVM now has Xtensa support in the linker LLD.
(This support was written mosly by me). This means we don't have to use
the Espressif GNU toolchain anymore and makes installing TinyGo simpler.
In the future, this also paves the way for ThinLTO support. Right now it
is mostly just a way to simplify TinyGo installation and speed up CI
slightly.
This flag controls whether to convert external i64 parameters for use in
a browser-like environment.
This flag was needed in the past because back then we only supported
wasm on browsers but no WASI. Now, I can't think of a reason why anybody
would want to change the default. For `-target=wasm` (used for
browser-like environments), the wasm_exec.js file expects this
i64-via-stack ABI. For WASI, there is no limitation on i64 values and
`-wasm-abi=generic` is the default.
Anonymous enums (often used in typedefs) triggered a problem that was
already solved for structs but wasn't yet solved for enums. So this
patch generalizes the code to work for both structs and enums, and adds
testing for both.
Running `go install` on MacOS produces the following warning:
# github.com/tinygo-org/tinygo
ld: warning: directory not found for option '-L/opt/homebrew/opt/libffi/lib'
It doesn't look like libffi is used anywhere, so I simply removed it.
Not sure why it was included in the first place.
(I updated the Makefile for consistency, but we really should be
removing that Makefile especially because the Go bindings are removed in
upstream LLVM).
This should hopefully fix the following issue:
DW_FORM_rnglistx index pointing outside of .debug_rnglists offset array [in module /tmp/tinygo4013272868/main]
This command didn't work anymore since the refactor in #3211.
The reason it doesn't work anymore is that before the refactor, the code
in the callback wasn't executed for .ll, .bc and .o files but after the
refactor it is, which causes a spurious error.
This commit fixes this oversight.
Example that didn't work anymore and is fixed with this change:
tinygo build -o test.ll examples/serial
These instructions sometimes pop up in LLVM 15, but they never occured
in LLVM 14. Implementing them is relatively straightforward: simply
generalize the code that already exists for llvm.ICmp interpreting.
This flag is necessary in LLVM 15 because it appears that LLVM 15 has
changed the default target ABI from lp64 to lp64d. This results in a
linker failure. Setting the "target-abi" forces the RISC-V backend to
use the intended target ABI.
There was a bug in the wasm ABI lowering pass (found using
AddressSanitizer on LLVM 15) that resulted in a rather subtle memory
corruption. This commit fixes this issues.
A number of llvm.Const* functions (in particular extractvalue and
insertvalue) were removed in LLVM 15, so we have to use a builder
instead. This builder will create the same constant values, it simply
uses a different API.
Scanning of allocas was entirely broken on WebAssembly. The code
intended to do this was never run. There were also no tests.
Looking into this further, I found that it is actually not really
necessary to do that: the C stack can be scanned conservatively and in
fact this was already done for goroutine stacks (because they live on
the heap and are always referenced). It wasn't done for the system stack
however.
With these fixes, I believe code should be both faster *and* more
correct.
I found this in my work to get opaque pointers supported in LLVM 15,
because the code that was never reached now finally got run and was
actually quite buggy.
* Add test.batch flag so standalone tests print PASS on pass
* Add comment; use single-dash flag style to match usage elsewhere
* Don't add test.batch for wasmtime
* Skip test.batch unless emulator name is blank
* Remove test.batch flag; buffer test output and show only on verbose or failure
* Remove FAIL when all tests fail to match go test output
Old message:
error: failed to reset port /tmp/tinygo1441085170/main.uf2: opening port: Permission denied
new message:
error: failed to reset port /dev/ttyACM0: opening port: Permission denied
The only reason a callback was used, was so that the temporary directory
gets removed once `Build` returns. But that is honestly a really bad
reason: the parent function can simply create a temporary function and
remove it when it returns. It wasn't worth the code complexity that this
callback created.
This change should not cause any observable differences in behavior (it
should be a non-functional change).
I have no reason to do this now, but this unclean code has been bugging
me and I just wanted to get it fixed.
This prefix isn't actually used and only adds noise, so remove it.
It may have been useful on Linux that makes a distinction between
/dev/ttyACM* and /dev/ttyUSB* but it isn't now. Also, it's unlikely that
the same vid/pid pair will be shared between an acm and usb driver
anyway.
The needed stack size is hard to determine by the compiler. It will try,
but will fail in many common cases. Therefore, the runtime will pick a
fixed stack size.
There is a tradeoff between avoiding stack overflows and wasting RAM.
This tradeoff depends on the application: some don't need large stack
sizes but do need a lot of memory, while others need deep stacks but
aren't so memory constrained. That's why I've added a flag to do this on
the command line: https://github.com/tinygo-org/tinygo/pull/3159
It may be reasonable to use a different stack size per chip, for example
chips with lots of RAM could default to a larger stack size. But I don't
think it's a good idea to do this per board.
This sanitizer is useful to detect use-after-free, double free, buffer
overflows, and more such errors.
I've found it useful lately to detect some bugs in TinyGo, and having a
single flag to enable it makes it much easier to enable
AddressSanitizer.
Unfortunately the calling convention for variadic functions is different from
the calling convention of regular functions on darwin/arm64, and open happens
to be such a variadic function. The syscall package treated it like a regular
function, which resulted in buggy behavior.
This fix introduces a wrapper function. This is the cleanest change I could
come up with.
The -x flag prints commands as they are executed. Unfortunately, for the link
command, they were printed too early: before the ThinLTO flags were added.
This is somewhat confusing while debugging.
They are not necessary in TinyGo because they always map to float32 and
float64, but it's a good idea to add them regardless for compatibility
with existing software.
(Now I think about it, perhaps it would have been better to require
explicit casts here just in case we want to support some really weird C
system, but then again we even force 64-bit double on AVR even though
avr-gcc defaults to 32-bit double).
Before, on the baremetal target or MacOS, we errored if the user
provided configuration to strip debug info.
Ex.
```bash
$ $ tinygo build -o main.go -scheduler=none --no-debug main.go
error: cannot remove debug information: MacOS doesn't store debug info in the executable by default
```
This is a poor experience which results in having OS-specific CLI
behavior. Silently succeeding is good keeping with the Linux philosophy
and less distracting than logging the same without failing.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
In some cases, regular integers were used. But we have a constant to
explicitly say these pins are undefined: `NoPin`. So use this.
A better solution would be to not require these constants, like with the
proposal in https://github.com/tinygo-org/tinygo/issues/3152. This
change is just a slight improvement over the current state.
* cgo: fixes panic when FuncType.Results is nil
FuncType.Results can be nil. This fixes the comparison and backfills
relevant tests.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Ayke <aykevanlaethem@gmail.com>
This is a constant for internal use only, but was (unintentionally?)
exported. In addition, it doesn't follow the Go naming convention.
This change simply renames the constant so that it is unexported.
This removes level-triggered interrupts.
While working on https://github.com/tinygo-org/tinygo/pull/3170, I found
these level triggered interrupt constants. Apart from them being
inconsistent with each other (PinLowLevel vs PinLevelLow) I don't think
they are actually used anywhere. In addition, I removed the
PinNoInterrupt constant on the esp32c3. This makes the esp32c3 pass the
tests in #3170.
I looked into level-triggered interrupts and I really couldn't find a
good justification for them:
- They were added to the esp32c3 and the rp2040 together with other
pin interrupt types, meaning they were probably just added because
the chip supports the feature and not because they were actually
needed.
- Level interrupts aren't supported in TinyGo for any other chip, and
I haven't seen anybody ask for this feature.
- They aren't supported in the nrf series chips _at all_, and with a
quick search I found only very little demand for them in general.
- I tried to see whether there is any good use case for them, but I
couldn't really find one (where an edge triggered interrupt wouldn't
work just as well). If there is one where level triggered interrupts
are a real advantage over edge triggered interrupts, please let me
know.
Of course, we shouldn't remove a feature lightly. But in this case, I
can't think of an advantage of having this feature. I can think of
downsides: more maintenance and having to specify their behavior in the
machine package documentation.
In general, I would like to keep the machine package clean and only
support things that have a proven use case.
Similar to the rp2040, the nrf has an on-board temperature sensor.
The main reason why I added this function was to show how this new API
is more general and can be used on multiple chips.
Replace ADCChannel.ReadTemperature() with a simple ReadTemperature
function.
Not all chips will have a temperature sensor that is read by sampling an
ADC channel. The replacement ReadTemperature is simpler and more generic
to other chip families.
This breaks chips that were relying on the previous ReadTemperature
method. I hope it won't break a lot of existing code. If it does, a
fallback can be added.
This makes the code a bit cleaner because ErrTxInvalidSliceSize isn't
redefined in every file that uses SPI and Mode0/Mode1/Mode2/Mode3 is
defined for every target that uses SPI.
There are two main issues with these constants:
* They don't follow the Go naming convention.
* They call themselves "TWI", while it makes a lot more sense to refer
to the actual name which is I2C (or I²C).
I have not removed them but just deprecated them. Perhaps we can remove
them when we move towards version 1.0.
Some targets used capital PullUp/PullDown, while the documented standard
is Pullup/Pulldown. This commit fixes this mismatch, while preserving
compatibility with aliases that are marked deprecated.
These constants are for internal use only, so should not have been
exported. In addition, they didn't follow the Go naming convention
before this change.
All nrf chips have a cryptographically secure RNG on board. Therefore,
I've made the code more portable so that it works on all nrf chips.
I did remove a number of exported functions. I am of the opinion that
these should only be made available once we have an agreed upon API for
multiple chips. People who want to have greater control over the RNG
should use the device/nrf package directly instead.
I have also changed the behavior to always enable digital error
correction. Enabling it seems like a more conservative (and secure)
default to me. Again, people who would like to have it disabled can use
the device/nrf package directly.
The crypto/rand package is used for sensitive cryptographic operations.
Do not use the rp2040 RNG for this purpose, because it's not strong
enough for cryptography.
I think it is _possible_ to make use of the RP2040 RNG to create
cryptographically secure pseudo-random numbers, but it needs some
entropy calculation and secure hashing (blake2s or so) to make them
truly unpredictable.
The GC was originally designed for systems with a fixed amount of
memory, like baremetal systems. Therefore, it just used what it could
and ran a GC cycle when out of memory.
Other systems (like Linux or WebAssembly) are different. In those
systems, it is possible to grow the amount of memory on demand. But the
GC only actually grew the heap when it was really out of memory, not
when it was getting very close to being out of memory.
This patch fixes this by ensuring there is at least 33% headroom for the
GC. This means that programs can allocate around 50% more than what was
live in the last GC cycle. It should fix a performance cliff when a
program is almost, but not entirely, out of memory and the GC has to run
almost every heap allocation.
This documents memory constants. Somewhere, we should document what the
default memory size is (seems 2 pages so 128KB), as that determines the
initial heap size (which is a portion of that).
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This fixes https://github.com/tinygo-org/tinygo/issues/3146 by using a
prebuilt Docker image. I don't remember why I used `setup-go` but
probably to make it faster (setup-go usually uses cached binaries).
This updates to setup-go@v3 which is the only updated version (v2 last
updated in feb), and employs its cache to simplify workflow
configuration.
Notably, we can't do this for Alpine until #3146
Signed-off-by: Adrian Cole <adrian@tetrate.io>
--allow-undefined can be a problem: it allows compiling code that will
fail when loaded. This change makes sure that if some symbols are
undefined, they are reported as an error by the linker.
Previously, people could get away with importing a function that was not
defined, like this:
func add(int a, int b) int
func test() {
println(add(3, 5))
}
This was always unintended but mostly worked. With this change, it isn't
possible anymore. Now every function needs to be marked with //export
explicitly:
//export add
func add(int a, int b) int
func test() {
println(add(3, 5))
}
As before, functions will be placed in the `env` module with the name
set from the `//export` tag. This can be overridden with
`//go:import-module`:
//go:import-module math
//export add
func add(int a, int b) int
func test() {
println(add(3, 5))
}
For the syscall/js package, I needed to give a list of symbols that are
undefined. This list is based on the JavaScript functions defined in
targets/wasm_exec.js.
This adds a summary of each wasm example, as before it was a bit unclear
how to do so. This also fixes the callback example which was broken.
Fixes#2568
Signed-off-by: Adrian Cole <adrian@tetrate.io>
I found that some packages do in fact run on Windows, so I've added them
where possible. I've also updated the description of which packages fail
tests and why.
This target was added purely for running tests, and it is currently
unused. When I try to use it, it causes runtime exceptions.
The replacement riscv-qemu is much better behaved.
This doesn't drop support for any actual hardware, the HiFive 1 B will
remain supported.
This commit fixes two related issues:
1. CanInterface was unimplemented. It now uses the same check as is
used in Interface() itself.
This issue led to https://github.com/tinygo-org/tinygo/issues/3033
2. Allow making an interface out of a string char element.
Doing this in one commit (instead of two) because they are shown to be
correct with the same tests.
This is just a papercut, and not really something important. But I
noticed something weird:
$ GOOS=windows GOARCH=arm tinygo info ""
LLVM triple: armv7-unknown-windows-gnueabihf-gnu
GOOS: windows
GOARCH: arm
That -gnueabihf-gnu ending is weird, it should pick one of the two. I've
fixed it as follows:
$ GOOS=windows GOARCH=arm tinygo info ""
LLVM triple: armv7-unknown-windows-gnu
GOOS: windows
GOARCH: arm
[...]
We're probably never going to support windows/arm (this is 32-bit arm,
not arm64) so it doesn't really matter which one we pick. And this patch
shouldn't affect any other system.
I think it is more confusing than helpful because it is only relevant
when compiling an actual linux/arm binary (and in that case, it is also
included in the LLVM triple).
See: https://github.com/tinygo-org/tinygo/issues/3034
This is really a few more-or-less separate changes:
* Remove all math aliases that were used in Go 1.16 and below (the
math.[A-Z] aliases).
* Replace math aliases with an assembly implementation (the math.arch*
aliases) with a LLVM intrinsic, where one is available.
* Include missing math functions in picolibc build.
This leaves just four math aliases:
* math.archHypot and math.archModf do not have a LLVM builtin
equivalent. They could be replaced with calls to libm, and I think
that would be a good idea in the long term.
* math.archMax and math.archMin do have a LLVM builtin equivalent
(llvm.maximum.f64, llvm.minimum.f64), but unfortunately they crash
when used. Apparently these exact operations are not yet widely
supported in hardware and they don't have a libm equivalent either.
There are more LLVM builtins that we could use for the math package
(such as FMA), but I will leave that to a future change. It could
potentially speed up some math operations.
Instead of changing the calls, replace the function bodies themselves.
This is useful for a number of reasons, see
https://github.com/tinygo-org/tinygo/pull/2920 for more information.
I have removed the math intrinsics tests because they are no longer
useful. Instead, I think `tinygo test math` should suffice.
This gives some more optimization opportunities to LLVM, because it
understands these intrinsics. For example, it might convert
llvm.sqrt.f64 to llvm.sqrt.f32 if possible.
This commit adds support for time.NewTimer and time.NewTicker. It also
adds support for the Stop() method on time.Timer, but doesn't (yet) add
support for the Reset() method.
The implementation has been carefully written so that programs that
don't use these timers will normally not see an increase in RAM or
binary size. None of the examples in the drivers repo change as a result
of this commit. This comes at the cost of slightly more complex code and
possibly slower execution of the timers when they are used.
For some reason, the type of a function parameter can sometimes be of
interface type, while it should be the underlying type. This might be a
bug in the x/tools/go/ssa package but this is a simple workaround.
ed25519vectors_test.go still fails because:
* It relies on "go mod download" which doesn't work, as well as fork/exec.
* It relies on JSON parsing which has problems with reflection.
But, with the vectors hard coded in the test file the tests *do* succeed, so the encryption is working.
The Go tools only consider lowercase .s files to be assembly files. By
renaming these to uppercase .S files they won't be discovered by the Go
toolchain and listed as the SFiles to be assembled.
There is a difference between .s and .S: only uppercase .S will be
passed through the preprocessor. Doing that is normally safe, and
definitely safe in the case of these files.
Go 1.19 started reformatting code in a way that makes it more obvious
how it will be rendered on pkg.go.dev. It gets it almost right, but not
entirely. Therefore, I had to modify some of the comments so that they
are formatted correctly.
This should add support for things like quotes around tags, if they are
ever needed.
Only making this change now because I happened to stumble across
buildutil.TagsFlag.
For some reason, these aren't lowered when a generic function is
instantiated by the SSA package.
I've left unsafe.Offsetof to be implemented later, it's a bit difficult
to do correctly the way the code is currently structured.
Without this change, the compiler would probably have worked just fine
but the generated types would look odd.
You can see in the test case that it now doesn't use `main.Point` but
rather the correct `main.Poin[float32]` etc.
For write-only operations (in SPI displays for example), the transmit
speed is doubled with this relatively small change.
In the future, we should try to use DMA instead for larger buffers. But
this is already a significant improvement and will always be an
improvement for small buffer sizes.
These are reserved registers on MacOS so don't need to be clobbered, and
result in a compiler waring:
ld.lld-14: warning: inline asm clobber list contains reserved registers: X18, FP
Reserved registers on the clobber list may not be preserved across the asm statement, and clobbering them may lead to undefined behaviour.
This makes nrf51 consistent with nrf52 and other chips, which do provide
constants for hardware pin numbers.
I've also added the microbit to the smoketest because it is used on
play.tinygo.org. And removed PCA10040 and PCA10056 because they aren't
provided on play.tinygo.org anymore.
This makes it easier to move the TinyGo compiler between Linux versions
because it doesn't depend on any system libraries anymore. For example,
binaries should be able to run on old Linux versions and on
distributions without glibc (such as Alpine Linux).
Without extra flags, we would try to use LLVM 13 for cgo and LLVM 14 for
other things since 873412b43a. That isn't
great. So fix this by only using LLVM 14 in the cgo package.
The MachO file format is a bit weird and doesn't store the DWARF debug
information directly in the file. Instead, it has to be looked up in the
original object file. This makes reading the DWARF debug information for
code size usage a bit more difficult. However, it works with this
change.
This has always been unsupported on MacOS and has in fact been removed
from upstream Go a few releases ago. So do the same for TinyGo.
Linux seems to be the only supported OS with a stable syscall interface.
This changes the compiler from treating calls to sync/atomic.* functions
as special calls (emitted directly at the call site) to actually
defining their declarations when there is no Go SSA implementation. And
rely on the inliner to inline these very small functions.
This works a bit better in practice. For example, this makes it possible
to use these functions in deferred function calls.
This commit is a bit large because it also needs to refactor a few
things to make it possible to define such intrinsic functions.
I don't understand why this wasn't caught in CI. It should have. In any
case, because the llvm-features string was updated, these IR outputs
were updated.
This commit will start to use a few more WebAssembly features, such as
bulk memory operations. This results in a significant code size saving.
How much it saves varies a lot but it's typically around 1300 bytes.
This change is possible by bumping our minimum Node.js version to 14.
The previous LTS version (12) has been marked end of life, so we can
start to depend on features in the current oldest LTS version, which is
version 14. Browsers have been supporting these features for a long time
now, it's just Node.js that prevented us doing this before.
Some source code wasn't part of `FMT_PATHS` so wasn't checked for
correct formatting. This change includes all this source code and
excludes cgo/testdata because it contains files that can't be parsed.
You can see that it works with the following command:
tinygo run -target=simavr ./testdata/recover.go
This also gets the following tests to pass again:
go test -run=Build -target=simavr -v
Adding support for AVR was a bit more compliated because it's also
necessary to save and restore the Y register.
This is a small change to make it easier to support architectures that
need to restore more than just the sp and pc registers. In particular,
it is needed for the AVR architecture that needs to restore the frame
pointer (Y register).
If an interrupt happens between the writes to SPL and SPH, the stack
pointer is inconsistent and terrible things will happen. Therefore,
disable interrupts while updating the stack pointer.
Interrupts are restored _before_ the write to SPH. This is safe, because
interrupts are re-enabled with a one cycle delay. The avr-gcc and Clang
compilers do the same thing when they need to update the stack pointer.
It's almost impossible to test for this bug, but it should make firmware
just a little bit more reliable.
This adds early Go 1.19 support. There are a number of things that don't
work yet, but the smoke tests pass so it's at least working for a
significant subset of programs.
This change also switches from CircleCI convenience images to upstream
Go images. This makes it a bit easier to use the latest Go versions.
Also, the convenience images are not updated anymore.
For example, this commit moves the 'throw' branch of an assertion (nil
check, slice index check, etc) to the end of the function while
inserting the "continue" branch right after the insert location. This
makes the resulting IR easier to follow.
For some reason, this also reduces code size a bit on average. The
TinyGo smoke tests saw a reduction of 0.22%, mainly from WebAssembly.
The drivers repo saw little average change in code size (-0.01%).
This commit also adds a few compiler tests for the defer keyword.
llvm.AddBasicBlock should never be used. Instead, we should use the
AddBasicBlock method of the current LLVM context.
This didn't lead to any bugs... yet. But probably would, eventually.
Previously we used to scan between _edata and _end. This is not correct:
the .data section starts *before* _edata.
Fixing this would mean changing _edata to _etext, but that didn't quite
work either. It appears that there are inaccessible pages between _etext
and _end on ARM. Therefore, a different solution was needed.
What I've implemented is similar to Windows and MacOS: namely, finding
writable segments by parsing the program header of the currently running
program. It's a lot more verbose, but it should be correct on all
architectures. It probably also reduces the globals to scan to those
that _really_ need to be scanned.
This bug didn't result in issues in CI, but did result in a bug in the
recover branch: https://github.com/tinygo-org/tinygo/pull/2331. This
patch fixes this bug.
Show the correct error message when trying to strip debug information.
Also, remove the special case for GOOS=linux that was probably dead
code: it was only reachable on baremetal systems which were already
checked before.
tests/testing/recurse has two directories with tests;
"make smoketest" now does "tinygo test ./..." in that directory
and fails if it does not run both directories' tests.
Previously, the MakeGCStackSlots pass would attempt to pop the stack chain before a tail call.
This resulted in use-after-free bugs when the tail call allocated memory and used a value allocated by its caller.
Instead of trying to move the stack chain pop, remove the tail flag from the call.
This change triggers a revert whenever a basic block runs instructions at runtime twice.
As a result, a loop body with runtime-only instructions will no longer be unrolled.
This should help some extreme cases where loops can be expanded into hundreds or thousands of instructions.
Precise globals require a whole program optimization pass that is hard
to support when building packages separately. This patch removes support
for these globals by converting the last use (Linux) to use
linker-defined symbols instead.
For details, see: https://github.com/tinygo-org/tinygo/issues/2870
This shrinks transform.Optimize() a little bit, working towards the goal
of https://github.com/tinygo-org/tinygo/issues/2870. I ran the smoke
tests and there is no practical downside: one test got smaller (??) and
one had a different .hex hash, but other than that there was no
difference.
This should also make TinyGo a liiitle bit faster but it's probably not
even measurable.
The transform package is the more appropriate location for package-level
optimizations, to match `transform.Optimize` for whole-program
optimizations.
This is just a refactor, to make later changes easier to read.
Move it from *builder to *compilerContext, so that it can be called in
more places. This is necessary to create a string value (for the file
name) in createEmbedGlobal.
This commit moves the calculation of the package action ID (cache key)
into a separate job. At the moment, this won't have a big effect but
this change is necessary for some future changes I want to make.
This replaces "precise" global scanning in LLVM with conservative
scanning of writable MachO segments. Eventually I'd like to get rid of
the AddGlobalsBitmap pass, and this is one step towards that goal.
This is necessary for the next commit. The next commit would otherwise
cause an issue with the following constant operation:
i64 add (i64 ptrtoint (%runtime.machHeader* @_mh_execute_header to i64), i64 32)
Simplify the interrupt-based timer code in a few ways:
- Do not recalibrate the timer every 100ms. Instead, rely on the fact
that the machine package will calbrate the timer if necessary if it
makes changes to Timer0.
- Do not configure Timer0 and then set nanosecondsInTick based on that
value. Instead, use a fixed value.
These two changes together mean that in code that doesn't use PWM,
nanosecondsInTick will be constant which makes the TIMER0_OVF interrupt
handler a lot smaller.
Together this reduces the code size of AVR binaries by about 1200 bytes,
making it pretty close to the pre-timer code size (only about 250 bytes
larger).
It also somehow fixes a problem with
tinygo.org/x/drivers/examples/ws2812 on the Arduino Uno. I'm not quite
sure what was going wrong, but bisecting pointed towards the timer code
(https://github.com/tinygo-org/tinygo/pull/2428) and with this
simplification the bug appears to be gone.
Scan globals conservatively by reading writable sections from the PE
header.
I'd like to get rid of needing to precisely scan globals eventually, and
this brings us one step closer. It also avoids a bug with ThinLTO on
Windows.
This is just basic support. It doesn't add support for reading DWARF,
because that's a bit complicated on MacOS (it isn't stored in the file
itself but separately in the object files). But at least this change
makes it possible to easily print executable sizes by section type like
for other operating systems.
Bug:
1. fn.locals[v.value] returns 0 (the default value) if v.value is not
part of the fn.locals map.
2. locals[fn.locals[v.value]] then returns the first local value, which
is usually non-nil
3. This incorrect value is then used as the operand value.
The manifestation of this convoluted bug was
https://github.com/tinygo-org/tinygo/issues/2842. It didn't occur more
often probably because it only seems to happen in practice with inline
assembly.
Fixes https://github.com/tinygo-org/tinygo/issues/2842
This patch adds support for generating GOOS=darwin GOARCH=arm64
binaries. This means that it will become possible to run `go test` on
recent Macs, for example.
In the go protobuf code, a pattern is used to statically prevent
comparable structs by embedding:
```
type DoNotCompare [0]func()
type message struct {
DoNotCompare
data *uint32
}
```
Previously, sizezof(message{}) is 2 words large, but it only needs to be
1 byte. Making it be 1 byte allows protobufs to compile slightly more
(though not all the way).
This change adds support for compiler-rt, which supports float64 (unlike
libgcc for AVR). This gets a number of tests to pass that require
float64 support.
We're still using libgcc with this change, but libgcc will probably be
removed eventually once AVR support in compiler-rt is a bit more mature.
I've also pushed a fix for a small regression in our
xtensa_release_14.0.0-patched LLVM branch that has also been merged
upstream. Without it, a floating point comparison against zero always
returns true which is certainly a bug. It is necessary to correctly
print floating point values.
This patch changes two things:
1. It changes the default stack size. Without this change, the
goroutine.go test doesn't pass (apparently there's some memory
corruption).
2. It moves the excluded tests so that they are skipped with a regular
`-target=simavr`, not just when running all tests (without
`-target`).
This is a small change that makes sure not to force a rebuild of LLVM
every time.
For example, I might run:
make llvm-source
make llvm-build ASSERT=1
And then I might make some temporary changes to LLVM to test out a patch
for example. So I run:
make llvm-build
...and my whole build cache gets destroyed.
This commit addresses this issue by not forcing a re-run of CMake with
every `make llvm-build` invocation.
This is a large refactor of the cgo package. It should fix a number of
smaller problems and be a bit more strict (like upstream CGo): it for
example requires every Go file in a package to include the header files
it needs instead of piggybacking on imports in earlier files.
The main benefit is that it should be a bit more maintainable and easier
to add new features in the future (like static functions).
This breaks the tinygo.org/x/bluetooth package, which should be updated
before this change lands.
1.15 specific files deleted.
1.16 specific files folded carefully into generic files, with goal of reducing diff with upstream.
Follows upstream 1.16 in making PathError etc. be aliases for the same errors in io/fs.
This fixes#2817 and lets us add io/ioutil to "make test-tinygo" on linux and mac.
This matches the flash-command and is generally a bit easier to work
with.
This commit also prepares for allowing multiple formats to be used in
the emulator command, which is necessary for the esp32.
See the comment in the source for details.
Also see the discussion in
https://github.com/tinygo-org/tinygo/pull/2755, which originally
triggered this bug.
Somewhat surprising, this results in a slight code size decrease for ARM
targets of a few bytes.
Switch over to LLVM 14 for static builds. Keep using LLVM 13 for regular
builds for now.
This uses a branch of the upstream Espressif branch to fix an issue,
see: https://github.com/espressif/llvm-project/pull/59
Memory references (`*m` in LLVM IR inline assembly) need a pointer type
starting in LLVM 14. This is a bit inconvenient and requires a new API
in the go-llvm package.
Instead of doing that, I'd like to remove support for memory references
from AsmFull (and possibly AsmFull entirely if possible: it's hard to
use correctly).
This breaks tinygo.org/x/drivers/ws2812 for AVR, ARM, and RISC-V which
need to be updated. Probably using CGo.
This means that we don't need duplicate code to pass parameters to
wasmtime and that the following actually produces verbose output (it
didn't before this commit):
tinygo test -v -target=cortex-m-qemu math
Refactor the code that runs a binary. With this change, the slightly
duplicated code between `tinygo run` and `TestBuild` is merged into one.
Apart from deduplication (which doesn't even gain much in terms of lines
removed), it makes it much easier to maintain this code. In particular,
passing command line arguments to programs to run now becomes trivial.
A future change might also merge `buildAndRun` and `runPackageTest`,
which currently have some overlap. In particular, flags like `-test.v`
don't need to be special-cased for wasmtime.
Without this patch, the include directory isn't found and picolibc.h
(used indirectly by stdint.h for example) can't be found.
I would like to add tests for this but we currently don't run Xtensa
tests. This should be possible however using https://github.com/espressif/qemu/wiki
(see also: https://github.com/tinygo-org/tinygo/pull/2780).
In https://github.com/tinygo-org/tinygo/issues/2777, a poison value
ended up in `runtime.alloc`. This shouldn't happen, especially not for
well written code. So I'm not sure why it happens. But here is a fix
anyway.
Evidently when you read from pipes in Go, you have to adjust your slice length to reflect the number of bytes read.
Verified upstream behaves the same way.
There used to be a difference between `byte` and `uint8` in interface
methods. These are aliases, so they should be treated the same.
This patch introduces a custom serialization format for types,
circumventing the `Type.String()` method that is slightly wrong for our
purposes.
This also fixes an issue with the `any` keyword in Go 1.18, which
suffers from the same problem (but this time actually leads to a crash).
Grepped straight out of the appropriate signal.h, with order preserved.
Makes 1.18 tests happier.
See comment on discrepancy for SIGCHLD. Since wasi doesn't really support signals, this may not matter.
Makes 1.18 tests a little happier.
Works around this error:
$ make test GOTESTFLAGS="-run TestTest/EmulatedCortexM3/Pass"
...
main_test.go:520: test error: could not compile: /usr/local/go/src/internal/fuzz/sys_posix.go:19:18: PROT_READ not declared by package syscall
Exporting symbols seems to embed them in the WASM exports section which
causes wasmtime to fail: https://github.com/bytecodealliance/wasmtime/issues/2587
As a workaround, it is possible to specify the `--allow-unknown-exports`
flag on wasmtime.
But as discussed in the above linked issue, this seems to only be a
workaround. For the Rust compiler the fix was to remove the
`--export-dynamic` linker flag when targeting `wasm32-wasi`:
https://github.com/rust-lang/rust/pull/81255
Which is waht this commit does for Tinygo too.
This adds the `Version()` function of the `runtime` package which embeds
the go version that was used to build tinygo.
For programs that are compiled with tinygo the version can be overriden
via the:
`tinygo build -ldflags="-X 'runtime.buildVersion=abc'"` flag.
Otherwise it will continue to use the go version with which tinygo was
compiled.
Moving and exporting this variable from the main to the goenv package
allows us to use it from both the main and the builder package.
This is done in preparation to include the value in `tinygo build`
linker flags, so that we can embed the version and git sha into binaries
built with tinygo.
This test creates a new temp file and writes some bytes into it.
It then opens a new file for the file descriptor of the temp file and
tries to read some bytes out of it.
I didn't see how to run it easily from main_test.go, though I didn't try too hard.
And it doesn't really have a good place to go in Makefile.
So I added a new target tinygo-baremetal, and invoke it from CI at the end of assert-test-linux.
It only adds 7 seconds to the run, should be ok.
These are used in Go 1.18's `testing/internal/testdeps`. Though the
comment says they should exist _everywhere_, there is still a build
constraint, but that seems to be fine.
Previously, the wrong permission bits were emitted by
`tinygo build-library`. This commit fixes that, by `chmod`'ing to
reasonable default permission bits.
Updating them to libclang-13-dev was a good change, but we can go even
further:
* The suggestion didn't apply to MacOS.
* The suggestion would need to be updated with every LLVM release,
which is a maintenance burden.
* The suggestion is wrong when compiling with `-tags=llvm12` for
example to choose a different LLVM version.
Therefore, link to the build documentation instead.
ThinLTO optimizes across LLVM modules at link time. This means that
optimizations (such as inlining and const-propagation) are possible
between C and Go. This makes this change especially useful for CGo, but
not just for CGo. By doing some optimizations at link time, the linker
can discard some unused functions and this leads to a size reduction on
average. It does increase code size in some cases, but that's true for
most optimizations.
I've excluded a number of targets for now (wasm, avr, xtensa, windows,
macos). They can probably be supported with some more work, but that
should be done in separate PRs.
Overall, this change results in an average 3.24% size reduction over all
the tinygo.org/x/drivers smoke tests.
TODO: this commit runs part of the pass pipeline twice. We should set
the PrepareForThinLTO flag in the PassManagerBuilder for even further
reduced code size (0.7%) and improved compilation speed.
The machine outliner introduces a few new opcodes that weren't
previously emitted by LLVM. We need to support these.
This doesn't trigger very often, but it sometimes does. It triggers a
lot more often with ThinLTO enabled.
Update interrupt_esp32c3.go:
make callHandler inline
save and restore MSTATUS along with MEPC
save and restore actual threshold value and call fence
print additional data during exception
This commit adds stubs for the `ExitError` struct of the `exec` package
and `ProcessState.ExitCode()` of the `os` package.
Since the `os/exec` is listed as unsupported, stubbing these methods
and structs should be enough to get programs that use these to compile.
This adds a stub for the `NumCPU()` function from the `runtime`
package.
This change allows code to compile that tries to access this function.
I guess for most hardware boards and WASM setting this value to `1` is
fine. And as far as I can see it shouldn't break or change existing
applications, because the function previously did not exist at all.
This adds the necessary structs and the `ReadBuildInfo()` function to
the runtime/debug module to allow to compile code that tries to access
it.
The stub itself returns ok=false, so that calling code should not try
to read from the nil pointer that is returned instead of the actual
BuildInfo struct.
This function is used by the hash/maphash package.
Unfortunately, I couldn't get the hash/maphash package to pass tests
because it's too slow. I think it hits the quadratic complexity of the
current map implementation.
This appears to have been how it is upstream for about 10 years. Using
an interface breaks if a file descriptor number is passed directly to
`NewFile`, e.g., `NewFile(3, "fuzz_in")` as used in Go 1.18 fuzzing
code.
I see no reason why it isn't possible to run `tinygo test -c` with
multiple packages. It'll just create multiple test outputs. I think the
intended flag was the `-o` flag, which indeed doesn't make much sense
with multiple packages.
The register r0 was used unconditionally. This is a bug: the compiler
doesn't know it is clobbered and might consider it alive across the
inline assembly expression.
The fix is simple. It could probably be optimized, but this should at
least fix the bug.
This means that it will be possible to generate a Darwin binary on any
platform (Windows, Linux, and MacOS of course), including CGo. Of
course, the resulting binaries can only run on MacOS itself.
The binary links against libSystem.dylib, which is a shared library. The
macos-minimal-sdk repository contains open source header files and
generated symbol stubs so we can generate a stub libSystem.dylib without
copying any closed source code.
It should only be added at the point that it is needed, for example when
using libclang or using the built-in Clang. It isn't needed when running
an external tool.
Do it all at once in preparation for Go 1.18 support.
To make this commit, I've simply modified the `fmt-check` Makefile
target to rewrite files instead of listing the differences. So this is a
fully mechanical change, it should not have introduced any errors.
This would conflict with our own heap. We previously defined all those
functions to make sure it's not used, but with a more recent wasi-libc
version (https://github.com/WebAssembly/wasi-libc/pull/250) we can
simply not compile the wasi-libc heap, which is the proper fix.
It's wafer-thin :-)
Includes smoke test from upstream.
TODO: once t.TempDir is implemented, add io/fs to the list of standard library tests to run; that's a better test.
readdir is disabled on linux for 386 and arm until syscall.seek is implemented there.
windows is hard, so leaving that for later.
File src/os/dir_other_go115.go can be deleted when we drop support for go 1.15.
Also adds TestReadNonDir, which was helpful while debugging.
This subcommand has been broken for a while, since libraries also use
the CPU flag. This commit fixes this.
Previously, libraries were usable for most Cortex-M cores. But with the
addition of the CPU field, I've limited it to three popular cores: the
Cortex-M0 (microbit), Cortex-M0+ (atsamd21), and Cortex-M4 (atsamd21,
nrf52, and many others).
In the future we might consider also building libraries for the current
OS/arch so that libraries like musl are already precompiled.
On Ubuntu, using standard go, both go and gnu buildid sections are present.
On Alpine, the gnu buildid section is absent, which caused tinygo to abort early.
It is possible that we could hit a situation where only the gnu
buildid section is present, so accept either one just in case.
Fixes https://github.com/tinygo-org/tinygo/issues/2580
This replaces an earlier kludge which was at the wrong level
and caused "GOARCH=386 tinygo test os" to fail to compile on linux.
Stubbing just the one missing function, syscall.seek, lets
os tests compile on linux 386, and skipping tests of seek and Fstat
(which has a cryptic dependency on syscall.Seek via time)
lets os tests pass on linux 386.
The stub can be removed once tinygo implements go assembly and picks up the real definition.
- rename Bitfield to Constant
- add methods to the exiting types to set/get bitfields
- integrate clustered registers
- add cluster size to properly add filler at the end of the structure
- fix structures with leading filler (i.e. for FICR_Type.INFO in nfr9160)
- shorten the function name when prefix and suffix are identical. i.e. GetSTATE_STATE vs GetSTATE
Previously, a type assertion on a nil interface would result in an out-of-bounds operand access, as we assumed it was a ptrtoint.
This would usually result in an undefined value which happens not to have the same name as the asserted type (and therefore the assertion fails as expected).
However, with an LLVM build with asserts, LLVM throws an assertion error:
```
interp.test: /home/niaow/go/src/github.com/tinygo-org/tinygo/llvm-project/llvm/include/llvm/IR/User.h:170: llvm::Value* llvm::User::getOperand(unsigned int) const: Assertion `i < NumUserOperands && "getOperand() out of range!"' failed.
```
This change handles a type code of 0 specially.
Test currently enabled on pybadge (chosen at random)
TODO:
- enable test on arduino; currently fails with "interp: ptrtoint integer size..." (#2389)
- enable test on nintendoswitch; currently fails with many missing definitions (#2530)
This is File.Stat from https://github.com/tinygo-org/tinygo/pull/2371,
plus the windows bits,
plus a smoke test more or less from upstream,
all pulled together and rebased by dkegel-fastly.
This removes the parentHandle argument from the internal calling convention.
It was formerly used to implment coroutines.
Now that coroutines have been removed, it is no longer necessary.
When a package only uses runtime.trackPointer to create interface packs, the compiler fails to find runtime.trackPointer.
This change predeclares it alongside runtime.alloc and updates the tests to use runtime.trackPointer when the test's target uses it.
There are now a large number of paths that need to be searched, and this started to get a little bit unwieldy.
Additionally, brew paths were searched unconditionally, resulting in warnings every time the Makefile was run.
This reorganizes the detection paths into a parameterized list of search paths by version, which is appended to on Mac.
This list is then expanded across all versions.
The loop and filtering has been moved into the detect function.
Additionally, a helpful error message is displayed upon use of a missing tool:
Makefile:204: *** failed to locate llvm-blah at any of: llvm-build/bin/llvm-blah llvm-blah-13 llvm-blah-12 llvm-blah-11 llvm-blah. Stop.
The chromedp context was not cancelled, so resources may have been leaking.
Additionally this waits for the browser to start before the timer starts, and extends the timeout to 20 seconds.
Logging from chromedp has also been enabled, which may help identify possible issues?
With this, 'tinygo test' in github.com/pkg/errors at least compiles and passes a few tests:
$ git clone github.com/pkg/errors
$ cd errors
$ tinygo test -c
$ for a in $(go test -list Test | grep Test); do ./errors.test -test.run $a -test.v > $a.log 2>&1; done
$ grep -l PASS *.log | wc -l
19
$ grep -l FAIL *.log | wc -l
11
For https://github.com/tinygo-org/tinygo/issues/2445
Also fix typo in error message in sub_test.go from upstream,
and move a few members from B to common where they belonged.
Note that testdata/testing.go seems to be pushing the edge of what
the emulated cortex-m3 target can handle; using regexp in that test
causes it to fail on that target with an out of memory error.
TODO: once tinygo supports runtime.Goexit, consider just using upstream's testing directory...
Only trivial functional changes:
- gets rid of mistaken extra "no tests" warning (whoops)
- matches upstream's exit code better
In preparation for switching to fancy test filtering.
The scoop and brew package managers now bundle up-to-date copies of binaryen.
As a result, there is no longer a strong need for us to build and package our own copy.
On wasi, O_RDWR is a bitwise or of read and write mode.
As a result, the bit test result was incorrect, and rewrote it to read-write mode.
However, the bit tests are not necessary (and upstream Go does not use them).
This passes the flags through directly.
This change updates the test runner to use exec.CommandContext for timeout handling.
The timeout has been raised to 1 minute to handle slow machines and (hopefully) Windows.
The test run also now acquires the semaphore to reserve CPU time for the test and (hopefully?????) reduce the number of timeouts in Windows CI.
The AVR backend has several critical atomics bugs.
This change invokes libcalls for all atomic operations on AVR.
Now `testdata/atomic.go` compiles and runs correctly.
Some clang builds (e.g., Fedora's) enable unwind tables by default. As
tinygo does not need nor support them, that leads to undefined symbols
when linking.
Arch Linux stores the clang executable seperately from its data, so the search based on the executable does not work.
This change searches /usr/lib as a backup.
Arch Linux has turned on the stack protector by default.
This causes a crash in libc init because the stack protector uses TLS before it is initialized.
This adds support for building with `-tags=llvm13` and switches to LLVM
13 for tinygo binaries that are statically linked against LLVM.
Some notes on this commit:
* Added `-mfloat-abi=soft` to all Cortex-M targets because otherwise
nrfx would complain that floating point was enabled on Cortex-M0.
That's not the case, but with `-mfloat-abi=soft` the `__SOFTFP__`
macro is defined which silences this warning.
See: https://reviews.llvm.org/D100372
* Changed from `--sysroot=<root>` to `-nostdlib -isystem <root>` for
musl because with Clang 13, even with `--sysroot` some system
libraries are used which we don't want.
* Changed all `-Xclang -internal-isystem -Xclang` to simply
`-isystem`, for consistency with the above change. It appears to
have the same effect.
* Moved WebAssembly function declarations to the top of the file in
task_asyncify_wasm.S because (apparently) the assembler has become
more strict.
Also fix a couple os tests that wrote to current directory to write to os.TempDir() instead.
After this, os tests pass in wasi, so add them to the list run by "make tinygo-test-wasi".
This matches what upstream Go does. This also means len(b) == 0 successfully
reads 0 bytes without any extra logic. The tests in archive/zip test for this
behaviour.
Large object layouts don't fit in a pointer-sized integer and therefore
need to be stored in a global instead. However, the way the data was
stored in these globals was not correct for buffers that don't have
pointers near the end. This commit fixes this issue by using math/big
FillBytes() instead of Bytes().
This gets the unicode package to compile on AVR.
This adds proper debug locations to interp errors. For example, when
trying to use the unicode package on AVR (which currently doesn't work),
the following error is shown with this commit:
/usr/local/go1.17/src/unicode/casetables.go:13:31: interp: ptrtoint integer size does not equal pointer size
Before this commit, that error was a lot less helpful:
unicode/<init>:13:31: interp: ptrtoint integer size does not equal pointer size
In the early days of TinyGo, the idea of `postinit` was to enable
interrupts only after initializers have run. Which kind of makes
sense... except that `time.Sleep` is allowed in init code and
`time.Sleep` requires interrupts to be enabled. Therefore, interrupts
must be enabled while initializers are being run.
This commit simply moves the enabling of interrupts to a point right
before running package initializers. It also removes `runtime.postinit`,
which is not necessary anymore (and was only used on AVR).
The STM32F469 can use the same initialization as the existing STM32F407
with a few frequency tweaks. This change splits the generic
initialization code into a separate runtime_stm32f4.go file, leaving
only the 407 board specific constants in the existing
runtime_stm32f407.go file.
Note that runtime_stm32f405.go initialization seems semantically similar
to the 407, but I don't have enough confidence in merging 405 with 407
in this change.
The only differences are a more general SPI.getBaudRate and a different
frequency limit in I2C.getFreqRange.
This is a first step towards adding stm32f469 support: a follow-up
merges machine_stm32f407.go and machine_stm32f405.go, another adds
frequency tweaks for stm32f469.
This change adds an additional semaphore to tinygo test that limits the number of tests being processed simultaneously (in addition to the existing limit on build jobs and runs).
When running a large number of tests, this limits the number of copies of per-test data stored in memory (avoiding an OOM in CI).
Switching to a shared semaphore allows multi-build operations (compiler tests, package tests, etc.) to use the expected degree of parallelism efficiently.
While refactoring the job runner, the time complexity was also reduced from O(n^2) to O(n+m) (where n is the number of jobs, and m is the number of dependencies).
Interrupt based time. Adjust tick cost every 1 minute and when timer-0 is reconfigured (the time precision affected when timer-0 reconfigured). Keep all time in nanoseconds.
Instead of storing an increasing version number in relevant packages
(compiler.Version, interp.Version, cgo.Version, ...), read the build ID
from the currently running executable. This has several benefits:
* All changes relevant to the compiled packages are caught.
* No need to bump the version for each change to these packages.
This avoids merge conflicts.
* During development, `go install` is enough. No need to run
`tinygo clean` all the time.
Of course, the drawback is that it might be updated a bit more often
than necessary but I think the overall benefit is big.
Regular release users shouldn't see any difference. Because the tinygo
binary stays the same, the cache works well.
This change breaks the merged goroot creation process into 2 steps:
1. List all overrides
2. Construct a goroot with the specified overrides
Now step 2 is cached using a hash of the results from step 1.
This eliminates cache inconsistency, but means that step 1 needs to be run on every build.
This is relatively acceptable, as step 1 only takes about 3 ms (assuming the directory tree is in the OS filesystem cache).
This change implements __sync atomic polyfill libcalls by disabling interrupts.
This was previously done in a limited capacity on some targets, but this change uses a go:generate to emit all of the calls on all microcontroller targets.
This change prevents interp from trying to execute goroutine starts or checks.
This fixes a bug where a goroutine started by an init function would run before the init function.
This change uses flock (when available) to acquire locks for build operations.
This allows multiple tinygo processes to run concurrently without building the same thing twice.
This bug can be triggered by the following series of events:
A acquires a write lock
B starts waiting for a read lock
C starts waiting for a read lock
A releases the write lock
After this, both B and C are supposed to be resumed as a read-lock is available.
However, with the previous implementation, only C would be resumed immediately.
Other goroutines could immediately acquire the read lock, but B would not be resumed until C released the read lock.
The extalloc collector has been broken for a while, and it doesn't seem reasonable to fix right now.
In addition, after a recent change it no longer compiles.
In the future similar functionality can hopefully be reintroduced, but for now this seems to be the most reasonable option.
This change swaps the stack chain when switching goroutines, ensuring that the chain is maintained consistently.
This is only really currently necessary with asyncify on wasm.
When a library is built concurrently by multiple TinyGo processes, they may sometimes both build the headers.
In that case a directory rename may fail due to conflict.
This change detects and handles the conflict similar to how GOROOT construction does.
This fixes 2 bugs in the GC scan bounds:
1. On AVR, the GC could sometimes read one byte past the end of a block due to the difference between pointer size and alignment.
2. On WASM, the linker does not properly align the marker for the end of the globals section. A manual alignment operation has been added to markGlobals to work around this.
This change fixes the edge case where a negative sleep time is provided.
When this happens, the call now returns immediately (as specified by the docs for time.Sleep).
Add direct test for the problem to make the fix commit clearer.
Noticed while implementing MkdirTemp on mac; the upstream tests for MkdirTemp fail without this.
When using the latest wasi-libc I experienced a
panic on an attempt to call realloc. My first attempt to
add it to arch_tinygowasm.go was obviously not good (PR #2194). So here
is another suggestion.
When I wrote the code originally, I didn't know about SetAlignment so I
hacked a way around it by allocating [...]uintptr types. However, this
allocates a few too many bytes in some cases.
This commit changes this to only allocate the space that we actually
need.
The code size effect is mixed, but generally positive. The combined
average is reduced by 0.27% with more programs being reduced in size
than are increasing in size.
Operations on nil maps are accepted and shouldn't
panic. The base hashmapGet/hashmapDelete handled
nil-maps correctly, but the hashmapBinary versions
could segfault accessing the nil map while trying
to hash the key.
Fixes#2341
Some map keys are hard to compare, such as floats. They are stored as if
the map keys are of interface type instead of the key type itself. This
makes working with them in the runtime package easier: they are compared
as regular interfaces.
Iterating over maps didn't care about this special case though. It just
returns the key, value pair as it is stored in the map. This is buggy,
and this commit fixes this bug.
This allows positive and negative zero to hash to the same value,
as required by Go.
This is not perfect, but the best I could do without
revamping all the hash funtions to take a seed.
Fixes#2356
File.Stat is left as a stub for now.
Tests are a bit stubbed down because os.ReadDir, os.Symlink, and t.TempDir are not yet (fully) implemented.
TODO: reimport tests from upstream as those materialize.
This is necessary for the following:
- to make sure os/exec can be imported
- to make sure internal/testenv can be imported
The internal/testenv package (which imports os/exec) is used by a lot of
tests. By adding support for it, more tests can be run.
This commit adds a bunch of new packages that now pass all tests.
FreeBSD support has been broken for a long time, probably since
https://github.com/tinygo-org/tinygo/pull/1860 (merged in May). Nobody
has complained yet, so I am going to assume nobody uses it.
This doesn't remove support for FreeBSD entirely: the code necessary to
build TinyGo on FreeBSD is still there. It just removes the code
necessary to build binaries targetting FreeBSD. But again, it could very
well be broken as we don't test it.
If anybody wants to re-enable support for FreeBSD, they would be welcome
to do that. But I think it would at the very least need a smoke test of
some sort.
This patch adds //go: pragmas directly to declared functions and
globals found during CGo processing. This simplifies the logic in the
compiler: it no longer has to consider special "C." prefixed function
names. It also makes the cgo pass more flexible in the pragmas it emits
for functions and global variables.
These wasm tests weren't passing in GitHub Actions and also weren't
passing on my laptop. I'm not sure why, I think there are a few race
conditions that are going on.
This commit attempts to fix this at least to a degree:
- The context deadline is increased from 5 seconds to 10 seconds.
- The tests are not running in parallel anymore.
- Some `Sleep` calls were removed, they do not appear to be necessary
(and if they were, sleeping is the wrong solution to solve race
conditions).
Overall the tests are taking a few seconds more, but on the other hand
they seem to be passing more reliable. At least for me, on my laptop
(and hopefully also in CI).
The wrong path was used to cache binaryen, so it wasn't actually getting
cached. Therefore, wasm-opt was rebuilt on every new PR (slowing down
the "Build TinyGo release tarball" a lot).
> There are two hard things in computer science: cache invalidation,
> naming things, and off-by-one errors.
Because of this bug, sometimes the last object in a section might not be
attributed correctly to a source location.
Previously, -scheduler=none wasn't possible for WASM targets:
$ tinygo run -target=wasm -scheduler=none ./testdata/stdlib.go
src/runtime/runtime_wasm_js.go:34:2: attempted to start a goroutine without a scheduler
With this commit, it works just fine:
$ tinygo run -target=wasm -scheduler=none ./testdata/stdlib.go
stdin: /dev/stdin
stdout: /dev/stdout
stderr: /dev/stderr
pseudorandom number: 1298498081
strings.IndexByte: 2
strings.Replace: An-example-string
Supporting `-scheduler=none` has some benefits:
* it reduces file size a lot compared to having a scheduler
* it allows JavaScript to call exported functions
The idea here is as follows:
- Run all Linux and cross compilation tests in the asser-test-linux
job.
- Only run native tests on MacOS and Windows.
This reduces testing time on MacOS and Windows, which are generally more
expensive in CI. Also, by not duplicating tests in Windows and MacOS we
can reduce overall CI usage a bit.
I've also changed the assert-test-linux job a bit to so that the tests
that are more likely to break and the tests that are only run in
assert-test-linux are run first.
Split building the release and smoke-testing the release in two, and
don't redo some tests that are already done by assert-test-linux.
Some benefits:
- Lower overall CI time because tests aren't done multiple times.
- TinyHCI can run earlier because the build-linux job is finished as
soon as the build artifact is ready.
It does however have the downside of an extra job, which costs a few
seconds to spin up and a few seconds to push and pull the workspace. But
even with this, overall CI time is down by a few minutes per workflow
run.
Instead of doing lots of repetitive tests in test-llvm11-go115 and
test-llvm11-go116, do those tests only once in assert-test-linux and
only run smoke tests for older Go versions.
Benefits:
- This should reduce total CI time, because these jobs don't do tests
that are done elsewere anyway. They only do the minimal work
necessary to prove that the given Go/LLVM version works.
- Doing all tests in assert-test-linux hopefully catches bugs that
might not be found in regular LLVM builds.
This fixes a small mistake when calculating binary size for an Xtensa
file. Previously it would exit with the following error:
$ tinygo build -o test.elf -size=short -target=esp32-mini32 examples/serial
panic: runtime error: index out of range [65521] with length 18
Now it runs as expected:
$ tinygo build -o test.elf -size=short -target=esp32-mini32 examples/serial
code data bss | flash ram
2897 0 4136 | 2897 4136
internal/itoa wasn't around back in go 1.12 days when tinygo's syscall/errno.go was written.
It was only added as of go 1.17 ( https://github.com/golang/go/commit/061a6903a232cb868780b )
so we have to have an internal copy for now.
The internal copy should be deleted when tinygo drops support for go 1.16.
FWIW, the new version seems nicer.
It uses no allocations when converting 0,
and although the optimizer might make this moot, uses
a multiplication x 10 instead of a mod operation.
The assembly symbols were not marked as hidden and so were exported,
leading to unreferenced symbols.
Example error message:
Error: failed to run main module `/tmp/tinygo3961039405/main`
Caused by:
0: failed to instantiate "/tmp/tinygo3961039405/main"
1: unknown import: `asyncify::stop_rewind` has not been defined
This commit fixes this issue.
This environment variable can be set to 5, 6, or 7 and controls which
ARM version (ARMv5, ARMv6, ARMv7) is used when compiling for GOARCH=arm.
I have picked the default value ARMv6, which I believe is supported on
most common single board computers including all Raspberry Pis. The
difference in code size is pretty big.
We could even go further and support ARMv4 if anybody is interested. It
should be pretty simple to add this if needed.
This change implements a new "scheduler" for WebAssembly using binaryen's asyncify transform.
This is more reliable than the current "coroutines" transform, and works with non-Go code in the call stack.
runtime (js/wasm): handle scheduler nesting
If WASM calls into JS which calls back into WASM, it is possible for the scheduler to nest.
The event from the callback must be handled immediately, so the task cannot simply be deferred to the outer scheduler.
This creates a minimal scheduler loop which is used to handle such nesting.
This PR fixes two bugs at once:
1. Indices were incorrectly extended to a bigger type. Specifically,
unsigned integers were sign extended and signed integers were zero
extended. This commit swaps them around.
2. The getelementptr instruction was given the raw index, even if it
was a uint8 for example. However, getelementptr assumes the indices
are signed, and therefore an index of uint8(200) was interpreted as
an index of int8(-56).
The implementation has been mostly copied from the Go reference
implementation with some small changes to fit TinyGo.
Source: https://github.com/golang/go/blob/77a11c05d6a6f766c75f804ea9b8796f9a9f85a3/src/reflect/deepequal.go
In addition, this commit also contains the following:
- A set of tests copied from the Go reflect package.
- An increased stack size for the riscv-qemu and hifive1-qemu targets
(because they otherwise fail to run the tests). Because these
targets are only used for testing, this seems fine to me.
In the case where:
- Value.Index() was called on an array
- that array was bigger than a pointer
- the element type fits in a pointer
- the 'indirect' flag isn't set
the Value.Index() method would still (incorrectly) load the value.
This commit fixes that.
The next commit adds a test which would have triggered this bug so works
as a regression test.
v.Interaface() could construct an interface in interface value if v was
of type interface. This is not correct, and doesn't follow upstream Go
behavior. Instead, it should return the interface value itself.
Constant globals can't have been modified, even if a pointer is passed
externally. Therefore, don't treat it as such in hasExternalStore.
In addition, it doesn't make sense to update values of constant globals
after the interp pass is finished. So don't do this.
TODO: track whether objects are actually modified and only update the
globals if this is the case.
Previously the cache would be stale for every new branch.
With this change, PRs use the cache from the base branch and therefore
don't need to rebuild LLVM from scratch.
This matches Clang, and with that, it adds support for inlining between
Go and C because LLVM only allows inlining if the "target-cpu" and
"target-features" string attributes match.
For example, take a look at the following code:
// int add(int a, int b) {
// return a + b;
// }
import "C"
func main() {
println(C.add(3, 5))
}
The 'add' function is not inlined into the main function before this
commit, but after it, it can be inlined and trivially be optimized to
`println(8)`.
This makes sure that the LLVM target features match the one generated by
Clang:
- This fixes a bug introduced when setting the target CPU for all
targets: Cortex-M4 would now start using floating point operations
while they were disabled in C.
- This will make it possible in the future to inline C functions in Go
and vice versa. This will need some more work though.
There is a code size impact. Cortex-M4 targets are increased slightly in
binary size while Cortex-M0 targets tend to be reduced a little bit.
Other than that, there is little impact.
With this fix, `cflags` in the target JSON files is correctly ordered.
Previously, the cflags of a parent JSON file would come after the ones
in the child JSON file, which makes it hard to override properties in
the child JSON file.
Specifically, this fixes the case where targets/riscv32.json sets
`-march=rv32imac` and targets/esp32c3.json wants to override this using
`-march=rv32imc` but can't do this because its `-march` comes before the
riscv32.json one.
This is fake debug info. It doesn't point to a source location because
there is no source location. However, it helps to correctly attribute
code size usage to particular packages.
I've also updated builder/sizes.go with some debugging helpers.
Instead of doing everything in the interrupt lowering pass, generate
some more code in gen-device to declare interrupt handler functions and
do some work in the compiler so that interrupt lowering becomes a lot
simpler.
This has several benefits:
- Overall code is smaller, in particular the interrupt lowering pass.
- The code should be a bit less "magical" and instead a bit easier to
read. In particular, instead of having a magic
runtime.callInterruptHandler (that is fully written by the interrupt
lowering pass), the runtime calls a generated function like
device/sifive.InterruptHandler where this switch already exists in
code.
- Debug information is improved. This can be helpful during actual
debugging but is also useful for other uses of DWARF debug
information.
For an example on debug information improvement, this is what a
backtrace might look like before this commit:
Breakpoint 1, 0x00000b46 in UART0_IRQHandler ()
(gdb) bt
#0 0x00000b46 in UART0_IRQHandler ()
#1 <signal handler called>
[..etc]
Notice that the debugger doesn't see the source code location where it
has stopped.
After this commit, breaking at the same line might look like this:
Breakpoint 1, (*machine.UART).handleInterrupt (arg1=..., uart=<optimized out>) at /home/ayke/src/github.com/tinygo-org/tinygo/src/machine/machine_nrf.go:200
200 uart.Receive(byte(nrf.UART0.RXD.Get()))
(gdb) bt
#0 (*machine.UART).handleInterrupt (arg1=..., uart=<optimized out>) at /home/ayke/src/github.com/tinygo-org/tinygo/src/machine/machine_nrf.go:200
#1 UART0_IRQHandler () at /home/ayke/src/github.com/tinygo-org/tinygo/src/device/nrf/nrf51.go:176
#2 <signal handler called>
[..etc]
By now, the debugger sees an actual source location for UART0_IRQHandler
(in the generated file) and an inlined function.
The target triples have to match mostly to be able to link LLVM modules.
Linking LLVM modules is already possible (the triples already match),
but testing becomes much easier when they match exactly.
For macOS, I picked "macosx10.12.0". That's an old and unsupported
version, but I had to pick _something_. Clang by default uses
"macos10.4.0", which is much older.
This generally means that code size is reduced, especially when the os
package is not imported.
Specifically:
- On Linux (which currently statically links musl), it avoids calling
malloc, which avoids including the musl C heap for small programs
saving around 1.6kB.
- On WASI, it avoids initializing the args slice when the os package
is not used. This reduces binary size by around 1kB.
Previously, libclang was run on each fragment (import "C") separately.
However, in regular Go it's possible for later fragments to refer to
types in earlier fragments so they must have been parsed as one.
This commit changes the behavior to run only one C parser invocation for
each Go file.
WriteString just does the simple and and converts the passed string
to a byte-slice. This can be made zero-copy later with unsafe, if needed.
WriteAt returns ErrNotImplemented, to match Seek() and ReadAt().
Fixes#2157
This commit adds support for musl-libc and uses it by default on Linux.
The main benefit of it is that binaries are always statically linked
instead of depending on the host libc, even when using CGo.
Advantages:
- The resulting binaries are always statically linked.
- No need for any tools on the host OS, like a compiler, linker, or
libc in a release build of TinyGo.
- This also simplifies cross compilation as no cross compiler is
needed (it's all built into the TinyGo release build).
Disadvantages:
- Binary size increases by 5-6 kilobytes if -no-debug is used. Binary
size increases by a much larger margin when debugging symbols are
included (the default behavior) because musl is built with debugging
symbols enabled.
- Musl does things a bit differently than glibc, and some CGo code
might rely on the glibc behavior.
- The first build takes a bit longer because musl needs to be built.
As an additional bonus, time is now obtained from the system in a way
that fixes the Y2038 problem because musl has been a bit more agressive
in switching to 64-bit time_t.
MacOS X 10.14 has a soft limit of 256 open files by default, at least on
CircleCI. So don't keep object files open while writing the ar file to
reduce the number of open files at once.
Context: the musl libc has more than 256 object files in the .a file.
This resulted in the error "too many open files" on MacOS X 10.14 when
running in CircleCI.
This is really just a preparatory commit for musl support. The idea is
to store not just the archive file (.a) but also an include directory.
This is optional for picolibc but required for musl, so the main purpose
of this commit is the refactor needed for this change.
This brings a bit more consistency to libc configuration. It seems
better to me to set the header flags all in the same place, instead of
some in Go code and some in JSON target files (depending on the target).
GitHub Actions is faster and much better integrated into GitHub than
Azure Pipelines, and is in general easier to use. Therefore, switch to
GitHub Actions for our Windows builds and tests.
This is for consistency with Clang, which always adds a CPU flag even if
it's not specified in CFLAGS.
This commit also adds some tests to make sure the Clang target-cpu
matches the CPU property in the JSON files.
This does have an effect on the generated binaries. The effect is very
small though: on average just 0.2% increase in binary size, apparently
because Cortex-M3 and Cortex-M4 are compiled a bit differently. However,
when rebased on top of https://github.com/tinygo-org/tinygo/pull/2218
(minsize), the difference drops to -0.1% (a slight decrease on average).
These functions are defined in compiler-rt in assembly and therefore
don't have stack size information. However, they're often called so
these missing functions often inhibit stack size calculation.
Example, before:
$ tinygo build -o test.elf -target=cortex-m-qemu -print-stacks ./testdata/float.go
function stack usage (in bytes)
Reset_Handler unknown, __aeabi_memclr does not have stack frame information
runtime.run$1 unknown, __aeabi_dcmpgt does not have stack frame information
After:
$ tinygo build -o test.elf -target=cortex-m-qemu -print-stacks ./testdata/float.go
function stack usage (in bytes)
Reset_Handler 260
runtime.run$1 224
This commit improves accuracy of the -size=full flag in a big way.
Instead of relying on symbol names to figure out by which package
symbols belong, it will instead mostly use DWARF debug information
(specifically, debug line tables and debug information for global
variables) relying on symbols only for some specific things. This is
much more accurate: it also accounts for inlined functions.
For example, here is how it looked previously when compiling a personal
project:
code rodata data bss | flash ram | package
1902 333 0 0 | 2235 0 | (bootstrap)
46 256 0 0 | 302 0 | github
0 454 0 0 | 454 0 | handleHardFault$string
154 24 4 4 | 182 8 | internal/task
2498 83 5 2054 | 2586 2059 | machine
0 16 24 130 | 40 154 | machine$alloc
1664 32 12 8 | 1708 20 | main
0 0 0 200 | 0 200 | main$alloc
2476 79 0 36 | 2555 36 | runtime
576 0 0 0 | 576 0 | tinygo
9316 1277 45 2432 | 10638 2477 | (sum)
11208 - 48 6548 | 11256 6596 | (all)
And here is how it looks now:
code rodata data bss | flash ram | package
------------------------------- | --------------- | -------
1509 0 12 23 | 1521 35 | (unknown)
660 0 0 0 | 660 0 | C compiler-rt
58 0 0 0 | 58 0 | C picolibc
0 0 0 4096 | 0 4096 | C stack
174 0 0 0 | 174 0 | device/arm
6 0 0 0 | 6 0 | device/sam
598 256 0 0 | 854 0 | github.com/aykevl/ledsgo
320 24 0 4 | 344 4 | internal/task
1414 99 24 2181 | 1537 2205 | machine
726 352 12 208 | 1090 220 | main
3002 542 0 36 | 3544 36 | runtime
848 0 0 0 | 848 0 | runtime/volatile
70 0 0 0 | 70 0 | time
550 0 0 0 | 550 0 | tinygo.org/x/drivers/ws2812
------------------------------- | --------------- | -------
9935 1273 48 6548 | 11256 6596 | total
There are some notable differences:
* Odd packages like main$alloc and handleHardFault$string are gone,
instead their code is put in the correct package.
* C libraries and the stack are now included in the list, they were
previously part of the (bootstrap) pseudo-package.
* Unknown bytes are slightly reduced. It should be possible to reduce
it significantly more in the future: most of it is now caused by
interface invoke wrappers.
* Inlined functions are now correctly attributed. For example, the
runtime/volatile package is normally entirely inlined.
* There is no difference between (sum) and (all) anymore. A better
code size algorithm now counts the code/data sizes correctly.
* And last (but not least) there is a stylistic change: the table now
looks more like a table. Especially the summary should be clearer
now.
Future goals:
* Improve debug information so that the (unknown) pseudo-package is
reduced in size or even eliminated altogether.
* Add support for other file formats, most importantly WebAssembly.
* Perhaps provide a way to expand this report per file, or in a
machine-readable format like JSON or CSV.
This matches the behavior of Clang, which uses optsize for -Os and adds
minsize for -Oz.
The code size change is all over the map, but using a hacked together
size comparison tool I've found that there is a slight reduction in
binary size overall (-1.6% with the tinygo smoke tests and -0.8% for the
drivers smoke test).
This commit has a few related changes:
* It sets the optsize attribute immediately in the compiler instead of
adding it to each function afterwards in a loop. This seems to me
like the more appropriate way to do it.
* It centralizes setting the optsize attribute in the transform
package, to make later changes easier.
* It sets the optsize in a few more places: to runtime.initAll and to
WebAssembly i64 wrappers.
This commit does not affect the binary size of any of the smoke tests,
so should be risk-free.
This commit will use the memory layout information for heap allocations
added in the previous commit to determine LLVM types, instead of
guessing their types based on the content. This fixes a bug in which
recursive data structures (such as doubly linked lists) would result in
a compiler stack overflow due to infinite recursion.
Not all heap allocations have a memory layout yet, but this can be
incrementally fixed in the future. So far, this commit should fix
(almost?) all cases of this stack overflow issue.
Instead of doing lots of complicated calculations to get the shortest
GEP, I'll just cast it to i8*, do the GEP, and optionally cast to the
requested type.
This currently produces ugly constant expressions, but once LLVM
switches to opaque pointer types all of this shouldn't matter anymore.
This is uncommon, but it does happen if the source pointer is a bitcast
of a global. For example, if a struct is cast to an i8*, it's possible
to index beyond what would appear to be the size of the pointer (i8*).
This commit adds object layout information to new heap allocations. It
is not yet used anywhere: the next commit will make use of it.
Object layout information will eventually be used for a (mostly) precise
garbage collector. This is what the data is made for. However, it is
also useful in the interp package which can work better if it knows the
memory layout and thus the approximate LLVM type of heap-allocated
objects.
This layout parameter is currently always nil and ignored, but will
eventually contain a pointer to a memory layout.
This commit also adds module verification to the transform tests, as I
found out that it didn't (and therefore didn't initially catch all
bugs).
This is necessary to display error messages on Windows. For example,
this command invocation is not correct (esp32 doesn't define
machine.LED, you need esp32-coreboard-v2 for example):
tinygo run -target=esp32 examples/blinky1
It results in the following hard-to-read error message:
# examples/blinky1
..\..\..\..\..\AppData\Local\tinygo\goroot-go1.16-24cb853b66a5367bf6d65bc08b2cb665c75bd9971f0be8f8b73f69d1a33e04a1-syscall\src\examples\blinky1\blinky1.go:11:17: LED not declared by package machine
With this commit, this error message becomes much easier to read:
# examples/blinky1
C:\Users\Ayke\go\src\github.com\tinygo-org\tinygo\src\examples\blinky1\blinky1.go:11:17: LED not declared by package machine
This commit simplifies the IR a little bit: instead of calling
pseudo-functions runtime.interfaceImplements and
runtime.interfaceMethod, real declared functions are being called that
are then defined in the interface lowering pass. This should simplify
the interaction between various transformation passes. It also reduces
the number of lines of code, which is generally a good thing.
This adds support for a construct like this:
type foo func(fn foo)
Unfortunately, LLVM cannot create function pointers that look like this.
LLVM only supports named types for structs (not for pointers) and thus
can't add a pointer to a function type of the same type to a parameter
of that function type.
The fix is simple: cast all function pointers to a void function, in
LLVM IR:
void ()*
Raw function pointers are cast to this type before storing, and cast
back to the regular function type before calling. This means that
function parameters will never refer to its own type because raw
function types are fixed at that one type.
Somehow, this does have an effect on binary size in some cases. The
effect is small and goes both ways. On top of that, there is work
underway in LLVM which would make all pointer types opaque (without a
pointee type). This would make this whole commit useless and therefore
should fix any size increases that might happen.
https://llvm.org/docs/OpaquePointers.html
The division and remainder operations were lowered directly to LLVM IR.
This is wrong however because the Go specification defines exactly what
happens on a divide by zero or signed integer overflow and LLVM IR
itself treats those cases as undefined behavior. Therefore, this commit
implements divide by zero and signed integer overflow according to the
Go specification.
This does have an impact on the generated code, but it is surprisingly
small. I've used the drivers repo to test the code before and after, and
to my surprise most driver smoke tests are not changed at all. Those
that are, have only a small increase in code size. At the same time,
this change makes TinyGo more compliant to the Go specification.
This adds support for stdio in picolibc and fixes wasm_exec.js so that
it can also support C puts. With this, C stdout works on all supported
platforms.
There is no need to put these in the board files as the I2S is the same
on all Microchip SAM D21 chips. This simplifies the code and avoids some
special *_baremetal.go files.
This change does not change the resulting binaries.
This has practically no effect on the resulting binaries, the only
difference I could find was for the flash/console/spi driver example.
I'm not sure how to test that one, but I think it's very unlikely that
code will have changed in any meaningful way (apart from reordering some
globals).
This commit changes the I2C declarations so that the objects are
instantiated in each chip file (e.g. machine_atsamd21e18.go) and used to
define I2C0 (and similar) in the board file (e.g. board_qtpy.go). This
should make it easier to define new board files, and reduces the need
for separate *_baremetal.go files.
I have tested this the following way:
- With the LIS3DH driver example on the Circuit Playground Express and
the PyBadge.
- With the LSM6DS3 driver example on the Arduino Nano 33 IoT.
They both still work fine.
Instead of defining them separately for each board, define them once in
the chip definition and later simply use &sercomUART1 etc. to refer to
them. This is simpler and less error-prone.
I found two bugs while working on this:
- The P1AM-100 board mixed SERCOM 5 and SERCOM 3. It looks like SERCOM
5 was intended, based on the used pins.
- The Adafruit Matrix Portal appears to have configured the wrong
interrupt.
Unfortunately, I can't test these fixes. However, they make it clear
that such a change is important to avoid bugs.
I tested this commit on the PyBadge and the Circuit Playground Express.
This attribute is also set by Clang when it compiles C source files
(unless -fexceptions is set). The advantage is that no unwind tables are
emitted on Linux (and perhaps other systems). It also avoids
__aeabi_unwind_cpp_pr0 on ARM when using the musl libc.
This chip can run so much faster! Let's update the default frequency.
Also, change the UART implementation to be more fexible regarding the
clock frequency.
This commit changes `tinygo test` to always look at the exit code of the
running test, instead of looking for a "PASS" string at the end of the
output. This is possible now that the binaries running under
qemu-system-arm or qemu-system-riscv32 will signal the correct exit code
when they exit.
As a side effect, this also makes it possible to avoid the "PASS" line
between successful tests. Before:
$ tinygo test container/heap container/list
PASS
ok container/heap 0.001s
PASS
ok container/list 0.001s
After:
$ tinygo test container/heap container/list
ok container/heap 0.001s
ok container/list 0.001s
The new behavior is more in line with upstream Go:
go test container/heap container/list
ok container/heap 0.004s
ok container/list 0.004s
There were a few issues that were causing qemu-system-arm and
qemu-system-riscv to give the wrong exit codes. They are in fact capable
of exiting with 0 or 1 signalled from the running application, but this
functionality wasn't used. This commit changes this in the following
ways:
* It fixes SemiHosting codes, which were incorrectly written in
decimal while they should have been written in hexadecimal (oops!).
* It modifies all the baremetal main functions (aka reset handlers) to
exit with `exit(0)` instead of `abort()`.
* It changes `syscall.Exit` to call `exit(code)` instead of `abort()`
on baremetal targets.
* It adds these new exit functions where necessary, implemented in a
way that signals the correct exit status if running under QEMU.
All in all, this means that `tinygo test` doesn't have to look at the
output of a test to determine the outcome. It can simply look at the
exit code.
LLDB mostly works on most platforms, but it is still lacking in some
features. For example, it doesn't seem to support RISC-V yet (coming in
LLVM 12), it only partially supports AVR (no stacktraces), and it
doesn't seem to support the Ctrl-C keyboard command when running a
binary for another platform (e.g. with GOOS=arm64). However, it does
mostly work, even on baremetal systems.
Somehow this is accepted by QEMU. I'm doing this so that tests for
-target=hifive1-qemu still work with the RISC-V tasks scheduler (with a
stack size of 2048 bytes).
This is necessary to support the ESP32-C3, which lacks the A (atomic)
extension and thus requires these 32-bit atomic operations.
With this commit, flashing ./testdata/atomic.go to the ESP32-C3 works
correctly and produces the expected output on the serial console.
It is better to use environment variables (GOOS and GOARCH) for
consistency instead of providing two slightly incompatible ways. This
-target flag should only be used to specify a .json file (either
directly or in the TinyGo targets directory). Previously it was possible
to specify the LLVM target as well but that was never really fully
supported.
So:
- To specify a different OS/arch like you would in regular Go, use
GOOS and GOARCH.
- To specify a microcontroller chip or board, use the -target flag.
Also remove the old `os.Setenv` which might have had a purpose long ago
but doesn't have a purpose now.
... instead of setting a special -target= value. This is more robust and
makes sure that the test actually tests different arcitectures as they
would be compiled by TinyGo. As an example, the bug of the bugfix in the
previous commit ("arm: use armv7 instead of thumbv7") would have been
caught if this change was applied earlier.
I've decided to put GOOS/GOARCH in compileopts.Options, as it makes
sense to me to treat them the same way as command line parameters.
At the moment, thumbv7 is crashing. I'm not exactly sure why, but it
appears that there is an unknown instruction in __aeabi_uldivmod
(probably from libgcc).
I've fixed this by switching to armv7, which is also somewhat modern.
Maybe we can switch back to Thumb2 (aka thumbv7) once we start using
musl and compiler-rt. In the meantime, this does fix a miscompilation
(illegal instruction).
You can now debug the ESP32-C3 from the TinyGo command line, like this:
tinygo flash -target=esp32c3 examples/serial
tinygo gdb -target=esp32c3 examples/serial
It's important to flash before running `tinygo gdb`, because loading a
new firmware from GDB has not yet been implemented.
Probably the easiest way to connect to the ESP32-C3 is by using the
built-in JTAG connection. See:
https://docs.espressif.com/projects/esp-idf/en/latest/esp32c3/api-guides/jtag-debugging/configure-builtin-jtag.html
You will need to make sure that the `openocd` command in your $PATH is
the one from Espressif. Otherwise GDB will hang. You can debug this by
supplying the -ocd-output flag:
$ tinygo gdb -target=esp32c3 -ocd-output examples/serial
Open On-Chip Debugger 0.10.0
openocd: Licensed under GNU GPL v2
openocd: For bug reports, read
openocd: http://openocd.org/doc/doxygen/bugs.html
openocd: embedded:startup.tcl:60: Error: Can't find interface/esp_usb_jtag.cfg
openocd: in procedure 'script'
openocd: at file "embedded:startup.tcl", line 60
Make sure to configure OpenOCD correctly, until you get the correct
version (that includes the string "esp32"):
$ openocd --version
Open On-Chip Debugger v0.10.0-esp32-20210721 (2021-07-21-13:33)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
If you are on Linux, you may also get the following error:
$ tinygo gdb -target=esp32c3 -ocd-output examples/serial
Open On-Chip Debugger v0.10.0-esp32-20210721 (2021-07-21-13:33)
openocd: Licensed under GNU GPL v2
openocd: For bug reports, read
openocd: http://openocd.org/doc/doxygen/bugs.html
openocd: Info : only one transport option; autoselect 'jtag'
openocd: adapter speed: 40000 kHz
openocd:
openocd: Warn : Transport "jtag" was already selected
openocd: Info : Listening on port 6666 for tcl connections
openocd: Info : Listening on port 4444 for telnet connections
openocd: Error: libusb_open() failed with LIBUSB_ERROR_ACCESS
openocd: Error: esp_usb_jtag: could not find or open device!
The error LIBUSB_ERROR_ACCESS means that there is a permission error.
You can fix this by creating the following file:
$ cat /etc/udev/rules.d/50-esp.rules
# ESP32-C3
SUBSYSTEMS=="usb", ATTRS{idVendor}=="303a", ATTRS{idProduct}=="1001", MODE="0666"
For more details, see:
https://docs.espressif.com/projects/esp-idf/en/latest/esp32c3/api-guides/jtag-debugging/index.html
Hopefully this will fix the CI breakage after curl and wget refuse to
download anything from wasmtime.dev (which is signed by Let's Encrypt).
- wget needs and updated libgnutls30
- curl needs and updated libssl1.0.2
This is just a first step. It's not complete, but it gets some real
world C code to parse.
This signature, from the ESP-IDF:
esp_err_t esp_wifi_get_mac(wifi_interface_t ifx, uint8_t mac[6]);
Was previously converted to something like this (pseudocode):
C.esp_err_t esp_wifi_get_mac(ifx C.wifi_interface_t, mac [6]uint8)
But this is not correct. C array parameters will decay. The array is
passed by reference instead of by value. Instead, this would be the
correct signature:
C.esp_err_t esp_wifi_get_mac(ifx C.wifi_interface_t, mac *uint8)
So that it can be called like this (using CGo):
var mac [6]byte
errCode := C.esp_wifi_get_mac(C.ESP_IF_WIFI_AP, &mac[0])
This stores the result in the 6-element array mac.
For example, the following did not work before but does work with this
change:
// int add(int a, int b) {
// return a + b;
// }
import "C"
func main() {
println("add:", C.add(3, 5))
}
Even better, the functions in the header are compiled together with the
rest of the Go code and so they can be optimized together! Currently,
inlining is not yet allowed but const-propagation across functions
works. This should be improved in the future.
This commit changes a target triple like "armv6m-none-eabi" to
"armv6m-unknown-unknow-eabi". The reason is that while the former is
correctly parsed in Clang (due to normalization), it wasn't parsed
correctly in LLVM meaning that the environment wasn't set to EABI.
This change normalizes all target triples and uses the EABI environment
(-eabi in the triple) for Cortex-M targets.
This change also drops the `--target=` flag in the target JSON files,
the flag is now added implicitly in `(*compileopts.Config).CFlags()`.
This removes some duplication in target JSON files.
Unfortunately, this change also increases code size for Cortex-M
targets. It looks like LLVM now emits calls like __aeabi_memmove instead
of memmove, which pull in slightly more code (they basically just call
the regular C functions) and the calls themself don't seem to be as
efficient as they could be. Perhaps this is a LLVM bug that will be
fixed in the future, as this is a very common occurrence.
This brings some consistency to the CFlags and fixes the issue that on
some platforms (Linux, MacOS), no optimization level was set and
therefore C files in packages were not optimized at all.
This is a loose collection of small fixes flagged by staticcheck:
- dead code
- regexp expressions not using backticks (`foobar` / "foobar")
- redundant types of slice and map initializers
- misc other fixes
Not all of these seem very useful to me, but in particular dead code is
nice to fix. I've fixed them all just so that if there are problems,
they aren't hidden in the noise of less useful issues.
This change fixes a bug in which `alloca` memory lifetimes would not extend past the suspend of an asynchronous tail call.
This would typically manifest as memory corruption, and could happen with or without normal suspending calls within the function.
Instead of keeping a slice of jobs to run, let the runJobs function
determine which jobs should be run by investigating all dependencies.
This has two benefits:
- The code is somewhat cleaner, as no 'jobs' slice needs to be
maintained while constructing the dependency graph.
- Eventually, some jobs might not be required by any dependency.
While it's possible to avoid adding them to the slice, the simpler
solution is to build a new slice from the dependencies which will
only include required dependencies by design.
This change adds support for the ESP32-C3, a new chip from Espressif. It
is a RISC-V core so porting was comparatively easy.
Most peripherals are shared with the (original) ESP32 chip, but with
subtle differences. Also, the SVD file I've used gives some
peripherals/registers a different name which makes sharing code harder.
Eventually, when an official SVD file for the ESP32 is released, I
expect that a lot of code can be shared between the two chips.
More information: https://www.espressif.com/en/products/socs/esp32-c3
TODO:
- stack scheduler
- interrupts
- most peripherals (SPI, I2C, PWM, etc)
If all of the Go files presented to the compiler have syntax errors,
cgo.Process gets an empty files slice and will panic:
panic: runtime error: index out of range [0] with length 0
goroutine 1 [running]:
github.com/tinygo-org/tinygo/cgo.Process({0x0, 0x4e8e36, 0x0}, {0xc000024104, 0x18}, 0xc000090fc0, {0xc000899780, 0x7, 0xc00018ce68})
/home/ayke/src/github.com/tinygo-org/tinygo/cgo/cgo.go:186 +0x22ee
github.com/tinygo-org/tinygo/loader.(*Package).parseFiles(0xc0001ccf00)
/home/ayke/src/github.com/tinygo-org/tinygo/loader/loader.go:400 +0x51e
This is simple to work around: just don't try to run CGo when there are
no files to process. It just means there are bugs to fix before CGo can
properly run.
(This is perhaps not the nicest solution but certainly the simplest).
At startup, a large chunk of virtual memory is used up by the heap. This
works fine in emulation (qemu-arm), but doesn't work so well on an
actual Raspberry Pi. Therefore, this commit reduces the requested amount
until a heap size is found that works on the system.
This can certainly be improved, but for now it's an important fix
because it allows TinyGo built binaries to actually run on a Raspberry
Pi with just 1GB RAM.
This reduces binary size substantially, for two reasons:
- It switches to a much more architecture ARMv4 vs ARMv7.
- It switches to Thumb2, which is a lot denser than regular ARM.
Practically all modern and not-so-modern ARM chips support Thumb2, so
this seems like a safe change to me.
The size in numbers:
- Code size for testdata/stdlib.go is reduced by about 35%.
- Binary size for testdata/stdlib.go (when compiling with -no-debug to
strip debug information) is reduced by about 16%.
Normalization was required because previously we supported Go 1.13 and
Go 1.14 at the same time. Now we've dropped support for both so this
normalization is not necessary anymore.
CGo support remains the same. It's just the test outputs that aren't
normalized anymore.
For example, in this code:
type kv struct {
v float32
}
func foo(a *kv) {
type kv struct {
v byte
}
}
Both 'kv' types would be given the same LLVM type, even though they are
different types! This is fixed by only creating a LLVM type once per Go
type (types.Type).
As an added bonus, this change gives a performance improvement of about
0.4%. Not that much, but certainly not nothing for such a small change.
This allows the assembly routines in these files to be stripped as dead
code if they're not referenced. This solves the link issues on MacOS
when the `leaking` garbage collector or the `coroutines` scheduler
are selected.
Fixes#2081
Populate the GBA ROM header so that emulators and physical Game Boy
Advance consoles recognize the ROM as a valid game.
Note: The reserve space at the end of the header was hand-tuned. Why
this magic value?
heapptr is assinged to heapStart (which is 0) when it's declared, but preinit()
may have moved the heap somewhere else. Set heapptr to the proper value
of heapStart when we initialize the heap properly.
This allows the leaking allocator to work on unix.
First look at the VERSION file, only then look at
src/runtime/internal/sys/zversion.go. This makes it possible to
correctly detect the Go version for release candidates.
This commit improves make([]T, len) to be closer to upstream Go. The
difference is unlikely to have much real-world effect, but previously
certain make([]T, len) expressions would not result in a slice out of
bounds error in TinyGo while they would have done such a thing in Go
proper. In practice, available RAM is likely to be a bigger limiting
factor.
This flag is passed automatically with the (new) -v flag for TinyGo. For
example, this prints all the test outputs:
$ tinygo test -v crypto/md5
=== RUN TestGolden
--- PASS: TestGolden
=== RUN TestGoldenMarshal
--- PASS: TestGoldenMarshal
=== RUN TestLarge
--- PASS: TestLarge
=== RUN TestBlockGeneric
--- PASS: TestBlockGeneric
=== RUN TestLargeHashes
--- PASS: TestLargeHashes
PASS
ok crypto/md5 0.002s
This prints just a summary:
$ tinygo test crypto/md5
PASS
ok crypto/md5 0.002s
(The superfluous 'PASS' message may be removed in the future).
This is especially useful when testing a large number of packages:
$ tinygo test crypto/md5 crypto/sha1 crypto/sha256 crypto/sha512
PASS
ok crypto/md5 0.002s
PASS
ok crypto/sha1 0.043s
PASS
ok crypto/sha256 0.002s
PASS
ok crypto/sha512 0.003s
At the moment, the -test.v flag is not supplied to binaries running in
emulation. I intend to fix this after
https://github.com/tinygo-org/tinygo/pull/2038 lands by refactoring
runPackageTest, Run, and runTestWithConfig in the main package which all
do something similar.
This is mainly useful to be able to run `tinygo test`, for example:
tinygo test -target=cortex-m-qemu -v math
This is not currently supported, but will be in the future.
This commit adds support for the following packages:
- crypto/md5
- crypto/sha1
- crypto/sha256
- crypto/sha512
They would normally need assembly implementations, but with these
aliases they already work everywhere.
This makes them more flexible, especially with Go 1.17 making the
situation more complicated (see
https://github.com/golang/go/commit/1d20a362d0ca4898d77865e314ef6f73582daef0).
It also makes it possible to do the same for many other functions, such
as assembly implementations of cryptographich functions which are
similarly dependent on the architecture.
Previously we used the i386 target, probably with all optional features
disabled. However, the Pentium 4 has been released a _long_ time ago and
it seems reasonable to me to take that as a minimum requirement.
Upstream Go now also seems to move in this direction:
https://github.com/golang/go/issues/40255
The main motivation for this is that there were floating point issues
when running the tests for the math package:
GOARCH=386 tinygo test math
I haven't investigated what's the issue, but I strongly suspect it's
caused by the weird x87 80-bit floating point format. This could perhaps
be fixed in a different way (by setting the FPU precision to 64 bits)
but I figured that just setting the minimum requirement to the Pentium 4
would probably be fine. If needed, we can respect the GO386 environment
variable to support these very old CPUs.
To support this newer CPU, I had to make sure that the stack is aligned
to 16 bytes everywhere. This was not yet always the case.
The math package failed the package tests on arm64 and wasm:
GOARCH=arm64 tinygo test math
Apparently the builtins llvm.maximum.f64 and llvm.minimum.f64 have
slightly different behavior on arm64 and wasm compared to what Go
expects.
This doesn't have the potential blocking issue of the getentropy call
(which calls WASI random_get when using wasi-libc) and should therefore
be a lot faster.
For context, this is what the random_get documentation says:
> Write high-quality random data into a buffer. This function blocks
> when the implementation is unable to immediately provide sufficient
> high-quality random data. This function may execute slowly, so when
> large mounts of random data are required, it's advisable to use this
> function to seed a pseudo-random number generator, rather than to
> provide the random data directly.
Bug 1790 ("musttail call must precede a ret with an optional bitcast")
is caused by the GC stack slot pass inserting a store instruction
between a musttail call and a return instruction. This is not allowed in
LLVM IR.
One solution would be to remove the musttail. That would probably work,
but 1) the go-llvm API doesn't support this and 2) this might have
unforeseen consequences. What I've done in this commit is to move the
store instruction to a position earlier in the basic block, just after
the last access to the GC stack slot alloca.
Thanks to @fgsch for a very small repro, which I've used as a regression
test.
Stripping debug information at link time also allows relocation
compression (aka linker relaxations). Keeping debug information at
compile time and optionally stripping it at link time has some
advantages:
* Automatic stack sizes on Cortex-M rely on the presence of debug
information.
* Some parts of the compiler now rely on the presence of debug
information for proper diagnostics.
* It works better with the cache: there is no distinction between
debug and no-debug builds.
* It makes it easier (or possible at all) to enable debug information
in the wasi-libc library without big downsides.
This commit fixes two things:
* It changes the alignment to 16 bytes (from 4), to match max_align_t
in C.
* It manually aligns heapStart on WebAssembly, to work around a bug in
wasm-ld with --stack-first (see https://reviews.llvm.org/D106499).
This function previously returned the atomic time, that isn't affected
by system time changes but also has a time base at some arbitrary time
in the past. This makes sense for baremetal platforms (which typically
don't know the wall time) but it gives surprising results on Linux and
macOS: time.Now() usually returns a time somewhere near the start of
1970.
This commit fixes this by obtaining both time values: the monotonic time
and the wall clock time. This is also how the Go runtime implements the
time.now function.
These two heaps conflict with each other, so that if any function uses
the dlmalloc heap implementation it will eventually result in memory
corruption.
This commit fixes this by implementing all heap-related functions. This
overrides the functions that are implemented in wasi-libc. That's why
all of them are implemented (even if they just panic): to make sure no
program accidentally uses the wrong one.
Static libraries should be added at the end of the linker command, after
all object files. If that isn't done, that's _usually_ not a problem,
unless there are duplicate symbols. In that case, weird dependency
issues can arise. To solve that, object files (that may include symbols
to override symbols in the library) should be listed first on the
command line and then the static libraries should be listed.
This fixes an issue with overriding some symbols in wasi-libc.
Make sure that if a package initializer cannot be run, later package
initializers won't try to access any global variables touched by the
uninterpretable package initializer.
Previously, a package initializer that could not be reverted correctly
would be called at runtime. But the initializer would be called in the
wrong order: after later packages are initialized.
This commit fixes this oversight and adds a test to verify the new
behavior.
This fixes https://github.com/tinygo-org/tinygo/issues/1884.
My original plan to fix this was much more complicated, but then I
realized that the output type doesn't matter anyway and I can simply
cast the type to an *i8 and perform a GEP on that pointer.
On ARM, the stack has to be aligned to 8 bytes on function calls, but
not necessarily within a function. Leaf functions can take advantage of
this by not keeping the stack aligned so they can avoid pushing one
register. However, because regular functions might expect an aligned
stack, the interrupt controller will forcibly re-align the stack when an
interrupt happens in such a leaf function (controlled by the STKALIGN
flag, defaults to on). This means that stack size calculation (as used
in TinyGo) needs to make sure this extra space for stack re-alignment is
available.
This commit fixes this by aligning the stack size that will be used for
new goroutines.
Additionally, it increases the stack canary size from 4 to 8 bytes, to
keep the stack aligned. This is not strictly necessary but is required
by the AAPCS so let's do it anyway just to be sure.
This can be very useful for some purposes:
* It makes it possible to disable the UART in cases where it is not
needed or needs to be disabled to conserve power.
* It makes it possible to disable the serial output to reduce code
size, which may be important for some chips. Sometimes, a few kB can
be saved this way.
* It makes it possible to override the default, for example you might
want to use an actual UART to debug the USB-CDC implementation.
It also lowers the dependency on having machine.Serial defined, which is
often not defined when targeting a chip. Eventually, we might want to
make it possible to write `-target=nrf52` or `-target=atmega328p` for
example to target the chip itself with no board specific assumptions.
The defaults don't change. I checked this by running `make smoketest`
before and after and comparing the results.
This commit implements various process related functions like
os.Getuid() and os.Getpid(). It also implements or improves this support
in the syscall package if it isn't available yet.
This patch adds a new pragma for functions and globals to set the
section name. This can be useful to place a function or global in a
special device specific section, for example:
* Functions may be placed in RAM to make them run faster, or in flash
(if RAM is the default) to not let them take up RAM.
* DMA memory may only be placed in a special memory area.
* Some RAM may be faster than other RAM, and some globals may be
performance critical thus placing them in this special RAM area can
help.
* Some (large) global variables may need to be placed in external RAM,
which can be done by placing them in a special section.
To use it, you have to place a function or global in a special section,
for example:
//go:section .externalram
var externalRAMBuffer [1024]byte
This can then be placed in a special section of the linker script, for
example something like this:
.bss.extram (NOLOAD) : {
*(.externalram)
} > ERAM
These pragmas weren't really tested anywhere, except that some code
might break if they are not properly applied.
These tests make it easy to see they work correctly and also provide a
logical place to add new pragma tests.
I've also made a slight change to how functions and globals are created:
with the change they're also created in the IR even if they're not
referenced. This makes testing easier.
The wasm build tag together with GOARCH=arm was causing problems in the
internal/cpu package. In general, I think having two architecture build
tag will only cause problems (in this case, wasm and arm) so I've
removed the wasm build tag and replaced it with tinygo.wasm.
This is similar to the tinygo.riscv build tag, which is used for older
Go versions that don't yet have RISC-V support in the standard library
(and therefore pretend to be GOARCH=arm instead).
This package provides access to an operating system resource
(cryptographic numbers) and so needs to be replaced with a TinyGo
version that does this in a different way.
I've made the following choices while adding this feature:
- I'm using the getentropy call whenever possible (most POSIX like
systems), because it is easier to use and more reliable. Linux is
the exception: it only added getentropy relatively recently.
- I've left bare-metal implementations to a future patch. This because
it's hard to reliably get cryptographically secure random numbers on
embedded devices: most devices do not have a hardware PRNG for this
purpose.
This was triggered by the following code:
var smallPrimesProduct = new(big.Int).SetUint64(16294579238595022365)
It is part of the new TinyGo version of the crypto/rand package.
This makes it possible to flash a board even when there are multiple
different kinds of boards attached, e.g. an Arduino Uno and a Circuit
Playground Express. You can find the VID/PID pair in several ways:
1. By running `lsusb` before and after attaching the board and looking
at the new USB device.
2. By grepping for `usb_PID` and `usb_VID` in the TinyGo source code.
3. By checking the Arduino IDE boards.txt from the vendor.
Note that one board may have multiple VID/PID pairs:
* The bootloader and main program may have a different PID, so far
I've seen that the main program generally has the bootloader PID
with 0x8000 added.
* The software running on the board may have an erroneous PID, for
example from a different board. I've seen this happen a few times.
* A single board may have had some revisions which changed the PID.
This is particularly true for the Arduino Uno.
As a fallback, if the given VID/PID pair isn't found, the whole set of
serial ports will be used.
There are many boards which I haven't included yet simply because I
couldn't test them.
Previously it was 1024 bytes, which occasionally ran into a stack
overflow. I hope that 2048 bytes will be enough for most purposes.
I've also removed some 2048-byte stack size settings in JSON files,
which are unnecessary now that the parent (cortex-m.json) sets them.
This only works with a custom bossac build from Arduino, not with the
upstream version. It avoids needing the manual "double tap" to enter
bootloader mode before flashing firmware.
Int in Go and C are two different types (hence why CGo has C.int). The
code in syscall assumed they were of the same type, which led to a bug:
https://github.com/tinygo-org/tinygo/issues/1957
While the C standard makes no guarantees on the size of int, in most
modern operating systems it is 32-bits so Go int32 would be the correct
choice.
This commit includes two changes:
* It makes unexported interface methods package-private, so that it's
not possible to type-assert on an unexported method in a different
package.
* It makes the globals used to identify interface methods defined
globals, so that they can (eventually) be left in the program for an
eventual non-LTO build mode.
Previously, flash-command would assume it could execute a command
straight via /bin/sh, at least on non-Windows systems. Otherwise it
would just split the command using `strings.Split`. This is all a bit
hacky, so I've replaced it with a proper solution: splitting the command
_before_ substituting various paths using a real shell splitter
(shlex.Split, from Google). This solves a few things:
* It guards against special characters in path names. This can be an
issue on Windows where the temporary path may contain spaces (this
is uncommon on POSIX systems).
* It is more portable, by disallowing the use of a shell. That way, it
doesn't differentiate between Windows and non-Windows anymore.
Other chips support explicit control of pull-up vs pull-down for GPIO input. Support that with bluepill also. PinInputPullUpDown is maintained for back-compat. It is implicit pull-down.
This was broken because multiple packages in the program were named
'main', even one that was imported (by the generated main package).
This fixes tests for main packages.
Previously a command like the following would incorrectly print FAIL:
tinygo test -c math
This commit fixes this issue by defaulting to a passing test (the test
is marked as passed if it isn't run).
Because arm.SVCall1 lets pointers escape, the return value of
sd_softdevice_is_enabled (passed as a pointer in a parameter) will
escape and thus this value will be heap allocated.
Use a global variable for this purpose instead to avoid the heap
allocation. This is safe as waitForEvent may only be called outside of
interrupts.
Do not store the context parameter (which is used for closures and
function pointers) in the goroutine start parameter bundle for direct
functions that don't need a context parameter. This avoids storing the
(undef) context parameter and thus makes the IR to start a new goroutine
simpler in most cases.
This reduces code size in the channel.go and goroutines.go tests.
Surprisingly, all test cases (when compiled with -target=microbit) have
a changed binary, I haven't investigated why but I suppose the codegen
is slightly different for the runtime.run function (which starts the
main goroutine).
Closure variables are allocated in a parent function and are thus never
nil. Don't do a nil check before reading or modifying the value.
This commit results in a slight reduction in code size in some test
cases: calls.go, channel.go, goroutines.go, json.go, sort.go -
presumably wherever closures are used.
Not sure why you would ever do this, but it appears to be allowed by the
Go specification and previously TinyGo would crash with an unhelpful
error message when you would do this. I don't see any practical use of
it.
The implementation simply runs the builtin directly.
This commit adds a test for both WebAssembly and Cortex-M targets (which
use a different way of goroutine lowering) to show how they lower
goroutines. It makes it easier to show how the output changes in future
commits.
While LLVM coroutines are one implementation of goroutines, it is not
the only one. Therefore, rename the tests to 'goroutines' to better
describe what they're for.
Move the code from the compiler.go file to the goroutine.go file, which
is a more appropriate place. This keeps all the goroutine related code
in one file, to make it easier to find.
These variants uses an unsafe.Pointer instead of uintptr so that the
pointer/non-pointer fields match those of real slices and strings. This
may be necessary in the future once we switch to a precise garbage
collector.
This converts the existing const parser to the basics of a Pratt parser,
following the book "Writing An Interpreter In Go" by Thorsten Ball. It
doesn't really do anything interesting yet, it simply converts the
existing code (with existing tests) to the new structure.
The markExternal function is used when a global (function or global
variable) is somehow run at runtime. All the other globals it refers to
are from then on no longer known at compile time, so can't be used by
the interp package anymore.
This can also include inline assembly. While it is possible to modify
globals that way, it is only possible to modify exported globals:
similar to calling an undefined function (in C for example).
This commit disables the Clang static analyzer and ARCMigrate components
of Clang. These aren't used at the moment in TinyGo so don't need to be
enabled. This reduces the build by 200 files (2909 -> 2709).
The idea comes from here (via LLVM weekly):
https://www.cambus.net/speedbuilding-llvm-clang-in-5-minutes/
Previously, the machine.UART0 object had two meanings:
- it was the first UART on the chip
- it was the default output for println
These two meanings conflict, and resulted in workarounds like:
- Defining UART0 to refer to the USB-CDC interface (atsamd21,
atsamd51, nrf52840), even though that clearly isn't an UART.
- Defining NRF_UART0 to avoid a conflict with UART0 (which was
redefined as a USB-CDC interface).
- Defining aliases like UART0 = UART1, which refer to the same
hardware peripheral (stm32).
This commit changes this to use a new machine.Serial object for the
default serial port. It might refer to the first or second UART
depending on the board, or even to the USB-CDC interface. Also, UART0
now really refers to the first UART on the chip, no longer to a USB-CDC
interface.
The changes in the runtime package are all just search+replace. The
changes in the machine package are a mixture of search+replace and
manual modifications.
This commit does not affect binary size, in fact it doesn't affect the
resulting binary at all.
This means that machine.UART0, machine.UART1, etc are of type
*machine.UART, not machine.UART. This makes them easier to pass around
and avoids surprises when they are passed around by value while they
should be passed around by reference.
There is a small code size impact in some cases, but it is relatively
minor.
Make the USBCDC use a pointer receiver everywhere. This makes it easier
to pass around the object in the future.
This commit sometimes changes code size, but not significantly (a few
bytes) and usually in a positive way.
My eventual goal is the following:
- Declare `machine.USB` (or similar, name TBD) as a pointer receiver
for the USB-CDC interface.
- Let `machine.UART0` always point to an UART, never actually to a
USBCDC object.
- Define `machine.Serial`, which is either a real UART or an USB-CDC,
depending on the board.
This way, if you want a real UART you can use machine.UARTx and if you
just want to print to the default serial port, you can use
machine.Serial.
This change does have an effect on code size and memory consumption.
There is often a small reduction (-8 bytes) in RAM consumption and an
increase in flash consumption.
Make the GC globals scan phase conservative instead of precise on
WebAssembly. This reduces code size at the risk of introducing some
false positives.
This is a stopgap measure to mitigate an issue with the precise scanning
of globals that doesn't track all pointers. It works for regular globals
but globals created in the interp package don't always have a type and
therefore may be missed by the AddGlobalsBitmap pass.
The same issue is present on Linux and macOS, but is not as noticeable
there.
This results in smaller and likely more efficient code. It does require
some architecture specific code for each architecture, but I've kept the
amount of code as small as possible.
The next commit will change the implementation of func values on Linux
as a result of switching to a task-based scheduler. To keep the
compiler/testdata/func.go test working as expected, switch to
WebAssembly tests.
There is no reason to specialize this per chip as it is only ever used
for JavaScript. Not only that, it is causing confusion and is yet
another quirk to learn when porting the runtime to a new
microcontroller.
This commit improves the timers on various microcontrollers to better
deal with counter wraparound. The result is a reduction in RAM size of
around 12 bytes and a small effect (sometimes positive, sometimes
negative) on flash consumption. But perhaps more importantly: getting
the current time is now interrupt-safe (it previously could result in a
race condition) and the timer will now be correct when the timer isn't
retrieved for a long duration. Before this commit, a call to `time.Now`
more than 8 minutes after the previous call could result in an incorrect
time.
For more details, see:
https://www.eevblog.com/forum/microcontrollers/correct-timing-by-timer-overflow-count/msg749617/#msg749617
This commit makes the output of `tinygo test` similar to that of `go
test`. It changes the following things in the process:
* Running multiple tests in a single command is now possible. They
aren't paralellized yet.
* Packages with no test files won't crash TinyGo, instead it logs it
in the same way the Go toolchain does.
With this is possible to enable e.g., SIMD in WASM using -llvm-features
+simd128. Multiple features can be specified separated by comma,
e.g., -llvm-features +simd128,+tail-call
With help from @deadprogram and @aykevl.
It is always implemented exactly the same way (as an uint8) so there is
no reason to implement it in each target separately.
This also makes it easier to add some documentation to it.
Instead, leave args at its default value (which provides a fake argv[0] as it has for a long time).
linux and mac do not seem affected.
Fixes#1862 (tinygo apps after v0.17.0-113-g7b761fa crash if run without argv[0])
This commit does two things:
1. It makes it possible to grow the heap on Linux and MacOS by
allocating 1GB of virtual memory on startup and then slowly using it
as necessary, when running out of available heap space.
2. It switches the default GC to be the conservative GC (previously
extalloc). This is good for consistency with other platforms that
all use this same GC.
This makes the extalloc GC unused by default.
This heap allocation would normally be optimized away, but with -opt=0
perhaps not. This is a problem if the conservative GC is used, because
the conservative GC needs to be initialized before use.
These two passes are related, but can definitely work independently.
Which is what this change does: it splits the two passes. This should
make it easier to change these two new passes in the future.
This change now also enables slightly better testing by testing these
two passes independently. In particular, the reflect lowering pass got
some actual tests: it was barely unit-tested before.
I have verified that this doesn't really change code size, at least not
on the microbit target. Two tests do change, but in a very minor way
(and in opposite direction).
The CircleCI macOS builds are failing, probably due to the old macOS
version that's used. This version (10.13 High Sierra) isn't supported
anymore on Homebrew so it seems best to me to simply bump the version.
I picked Xcode 11.1.0 because 10.3.0 is somehow triggering an error
while trying to install QEMU (the Python install fails).
Because of this newer Xcode version, I had to add an extra flag
(-isysroot) to the default command line for MacOS. The reason is that
this newer Xcode version no longer stores header files in /usr/local, an
SDK must be specified manually. With this change, the default SDK is
used.
Since 2018, Arduino Nanos and clones are sold with a new bootloader, which
requires programming at 115200 baud instead of the 57600 baud required
by the old one.
In many cases, position information is not stored in Go SSA instructions
because they don't exit directly in the source code. This includes
implicit type conversions, implicit returns at the end of a function,
the creation of a (hidden) slice when calling a variadic function, and
many other cases. I'm not sure where this information is supposed to
come from, but this patch takes the value (usually) from the value the
instruction refers to. This seems to work well for these implicit
conversions.
I've also added a few extra tests to the heap-to-stack transform pass,
of which one requires this improved position information.
On some boards the FPU is already enabled on startup, probably as part
of the bootloader. On other chips it was enabled as part of the runtime
startup code. In all these cases, enabling the FPU is currently
unsupported: the automatic stack sizing of goroutines assumes that the
processor won't need to reserve space for FPU registers. Enabling the
FPU therefore can lead to a stack overflow.
This commit either removes the code that enables the FPU, or simply
disables it in startup code. A future change should fully enable the FPU
so that operations on float32 can be performed by the FPU instead of in
software, greatly speeding up such code.
- Add some extra fields: FPUPresent, CPU and NVICPrioBits which may
come in handy at a later time (and are easy to add).
- Rename DEVICE to Device, to match Go style.
This is in preparation to the next commit, which requires the FPUPresent
flag.
This commit replaces most heap allocations in USB related code with
stack allocations. This is important for several reasons:
- It avoids running the GC unnecessarily.
- It reduces code size by 400-464 bytes.
- USB code might be called from interrupt handlers. The heap may be in
an inconsistent state when that happens if main thread code also
performs heap allocations.
The last one is by far the most important one: not doing heap
allocations in interrupts is critical for correctness. But the code size
reduction alone should be worth it.
There are two heap allocations in USB related code left: in the function
receiveUSBControlPacket (SAMD21 and SAMD51). This heap allocation must
also be removed because it runs in an interrupt, but I've left that for
a future change.
This allows better escape analysis even without being able to see the
entire program. This makes the stack allocation test case more complete
but probably won't have much of an effect outside of that (as the
compiler is able to infer these attributes in the whole-program
functionattrs pass).
This flag, if set, is a regexp for function names. If there are heap
allocations in the matching function names, these heap allocations will
be printed with an explanation why the heap allocation exists (and why
the object can't be stack allocated).
This allows for adding more advanced tests, for example tests that use
the compiler package so that test sources can be written in Go instead
of LLVM IR.
The interp package is in many cases able to execute map functions in the
runtime directly. This is probably slower than adding special support
for them in the interp package and also doesn't cover all cases (most
importantly, map keys that contain pointers) but removing this code also
removes a large amount of code that needs to be maintained and is
susceptible to hard-to-find bugs.
As a side effect, this resulted in different output of the
testdata/map.go test because the test relied on the existing iteration
order of TinyGo maps. I've updated the test to not rely on this test,
making the output compatible with what the Go toolchain would output.
I've discovered a bug in the implementation of the PHI instruction in
the interp package. This commit fixes the bug.
I've found this issue while investigating an issue with maps after
running interp per package.
At the moment, all targets use the Clang compiler to compile C and
assembly files. There is no good reason to make this configurable
anymore and in fact it will make future changes more complicated (and
thus more likely to have bugs). Therefore, I've removed support for
setting the compiler.
Note that the same is not true for the linker. While it makes sense to
standardize on the Clang compiler (because if Clang doesn't support a
target, TinyGo is unlikely to support it either), linkers will remain
configurable for the foreseeable future. One example is Xtensa, which is
supported by the Xtensa LLVM fork but doesn't have support in ld.lld
yet.
I've also fixed a bug in compileAndCacheCFile: it wasn't using the right
CFlags for caching purposes. This could lead to using stale caches. This
commit fixes that too.
This improves compatibility between the regular browser target
(-target=wasm) and the WASI target (-target=wasi). Specifically, it
allows running WASI tests like this:
tinygo test -target=wasi encoding/base32
This doesn't change the firmware, but it does make the disassembly of
the ELF files. Before:
Disassembly of section .text:
00000000 <(machine.UART).Write-0x100>:
0: 20001000 .word 0x20001000
4: 000009db .word 0x000009db
8: 00000f05 .word 0x00000f05
c: 00000f0b .word 0x00000f0b
10: 00000f05 .word 0x00000f05
After:
Disassembly of section .text:
00000000 <__isr_vector>:
0: 20001000 .word 0x20001000
4: 000009db .word 0x000009db
8: 00000f05 .word 0x00000f05
c: 00000f0b .word 0x00000f0b
10: 00000f05 .word 0x00000f05
The difference is that the disassembler will now use a proper symbol name
instead of using the closest by symbol (in this case, (machine.UART).Write).
This makes the disassembly easier to read.
This currently doesn't work with `tinygo flash` yet (even with
`-programmer=openocd`), you can use pyocd instead. For example, from the
Bluetooth package:
tinygo build -o test.hex -target=microbit-v2-s113v7 ./examples/advertisement/
pyocd flash --target=nrf52 test.hex
I intend to add support for pyocd to work around this issue, so that a simple
`tinygo flash` suffices.
There doesn't appear to be a user-controllable LED outside of the LED
matrix. In fact, the pin assigned for this was P13, which was connected
to the SPI SCK pin.
In this commit I've moved all core-specific flags to files for that
specific core. This is a bit of a cleanup (less duplicated JSON) but
should also help in the future when core-specific changes are made, such
as core specific build tags or when the FPU finally gets supported in
TinyGo.
Some notable specific changes:
- I've removed floating point flags from the Teensy 3.6 target. The
reason is that the FPU is not yet supported in TinyGo (in goroutine
stack switching for example) and floating point numbers would only
be supported by C files, not Go files (because the LLVM FPU feature
flags aren't used). This would create an ABI mismatch across CGo.
- I've added the "cpu":"cortex-m7" to the cortex-m7.json file to match
the configuration for the Teensy 4.0. This implies a change to the
nucleo-f722ze (because now it has its CPU field set). Somehow that
reduces the code size, so it looks like a good change.
I don't believe any of these changes should have any practical
consequences.
One issue I've found is in the Cortex-M33 target: it uses armv7m, which
is incorrect: it should be armv8m. But the chip is backwards compatible
so this should mostly work. Switching to armv8m led to a compilation
failure because PRIMASK isn't defined, this may be an actual bug.
The -Qunused-arguments flag disables the warning where some flags are
not relevant to a compilation. This commonly happens when compiling
assembly files (.s or .S files) because some flags are specific to C and
not relevant to assembly.
Because practically all baremetal targets need some form of assembly,
this flag is added to most CFlags. This creates a lot of noise. And it
is also added for compiling C code where it might hide bugs (by hiding
the fact a flag is actually unused).
This commit adds the flag to all assembly compilations and removes them
from all target JSON files.
See "STM32F40x and STM32F41x Errata sheet" - SPI CLK port must be 'fast' or 'very fast' to avoid data corruption on last bit (at the APB clocks we configure).
There is no good reason for func values to refer to interface type
codes. The only thing they need is a stable identifier for function
signatures, which is easily created as a new kind of globals. Decoupling
makes it easier to change interface related code.
This is basically just a golden test for the "switch" style of func
lowering. The next commit will make changes to this lowering, which will
be visible in the test output.
The LLVM CoroFrame pass appears to be tripping over this zero-sized
alloca. Therefore, do what the runtime would do: return a pointer to
runtime.zeroSizedAlloc. Or just don't deal with this case. But don't
emit a zero sized alloca to avoid this LLVM bug.
More information: https://bugs.llvm.org/show_bug.cgi?id=49916
The func-lowering pass has started to fail in the dev branch, probably
as a result of replacing the ConstPropagation pass with the IPSCCP pass.
This commit makes the code a bit more robust and should be able to
handle all possible cases (not just ptrtoint).
This commit implements replacing some global variables with a different
value, if the global variable has no initializer. For example, if you
have:
package main
var version string
you can replace the value with -ldflags="-X main.version=0.2".
Right now it only works for uninitialized globals. The Go tooling also
supports initialized globals (var version = "<undefined>") but that is a
bit hard to combine with how initialized globals are currently
implemented.
The current implementation still allows caching package IR files while
making sure the values don't end up in the build cache. This means
compiling a program multiple times with different values will use the
cached package each time, inserting the string value only late in the
build process.
Fixes#1045
Don't run the entire test suite for these options, as that would quickly
explode the testing time (making it less likely people actually run it).
Instead, run just one test for each configuration that should check for
most issues.
- Explicitly list all test cases. This makes it possible to store
tests in testdata/ that aren't tested on all platforms.
- Clean up filesystem and env test, by running them in a subtest and
deduplicating some code and removing the additionalArgs parameter.
Some errors were generated but never returned or never checked in the
test function. That's a problem. Therefore this commit fixes this
oversight (by me).
The constant propagation pass is removed in LLVM 12, so this pass needs
to be replaced anyway. The direct replacement would be the SCCP (sparse
conditional constant propagation) pass, but perhaps a better replacement
is the IPSCCP pass, which is an interprocedural version of the SCCP
pass and propagates constants across function calls if possible.
This is not always a code size reduction, but it appears to reduce code
size in a majority of cases. It certainly reduces code size in almost
all WebAssembly tests I did.
This should result in a small compile time reduction for incremental
builds, somewhere around 5-9%.
This commit, while small, required many previous commits to not regress
binary size. Right now binary size is basically identical with very few
changes in size (the only baremetal program that changed in size did so
with a 4 byte increase).
This commit is one extra step towards doing as much work as possible in
the parallel and cached package build step, out of the serial LTO phase.
Later improvements in this area have this change as a prerequisite.
A switch statement is not normally emitted by the compiler package, but
LLVM function passes may convert a series of if/else pairs to a switch
statement. A future change will run function passes in the package
compile phase, so the interp package (which is also run after all
modules are merged together) will need to deal with these new switch
statements.
This commit optimizes string literals and globals by setting the
appropriate alignment and using a nil pointer in zero-length strings.
- Setting the alignment for string values has a surprisingly large
effect, up to around 2% in binary size. I suspect that LLVM will
pick some default alignment for larger byte arrays if no alignment
has been specified and forcing an alignment of 1 will pack all
strings closer together.
- Using nil for zero-length strings also has a positive effect, but
I'm not sure why. Perhaps it makes some optimizations more trivial.
- Always setting the alignment on globals improves code size slightly,
probably for the same reasons setting the alignment of string
literals improves code size. The effect is much smaller, however.
This commit might have an effect on performance, but if it does this
should be tested separately and such a large win in binary size should
definitely not be ignored for small embedded systems.
Sometimes, LLVM may rename named structs when merging modules.
Therefore, we can't rely on typecodeID structs to retain their struct
names.
This commit changes the interface lowering pass to not rely on these
names. The interp package does however still rely on this name, but I
hope to fix that in the future.
This simplifies future changes. While the move itself is very simple, it
required some other changes to a few transforms that create new
functions to add the optsize attribute manually. It also required
abstracting away the optimization level flags (based on the -opt flag)
so that it can easily be retrieved from the config object.
This commit does not impact binary size on baremetal and WebAssembly.
I've seen a few tests on linux/amd64 grow slightly in size, but I'm not
too worried about those.
Instead of the regular build, it's the `make test` line that fails due
to OOM. This is because testing means that a lot of test binaries need
to be built while the regular build only needs to link one binary.
This improves https://github.com/tinygo-org/tinygo/pull/1774 and should
hopefully actually fix the OOM errors.
This commit refactors PWM support in the machine package to be more
flexible. The new API can be used to produce tones at a specific
frequency and control servos in a portable way, by abstracting over
counter widths and prescalers.
This job is causing OOM errors on CircleCI so limit it to just two jobs
(which should be fine on a 2CPU executor). Hopefully this fixes the
errors in CI that have occured recently.
This patch adds support for passing CFLAGS added in #cgo lines of the
CGo preprocessing phase to the compiler when compiling C files inside
packages. This is expected and convenient but didn't work before.
This probably won't speed up the build on multicore systems (the build
is still dominated by the whole-program optimization step) but should be
useful at a later date for other optimizations. For example, I intend to
eventually optimize each package individually including C files, which
should enable cross-language optimizations (inlining C functions into Go
functions, for example). For that to work, accurate dependency tracking
is important.
Add a 'result' member to the compileJob struct which is used by the link
job to get all the paths that should be linked together. This is not yet
necessary (the paths are fixed), but soon the paths are only known after
a linker dependency has run.
In rare cases the signature might change as a result of LLVM renaming
some named struct types when multiple LLVM modules are merged. The
easiest workaround is to detect such mismatched signatures and adding a
bitcast: this should be safe as the underlying data is effectively of
the same type.
This results in a significant speedup in some cases. For example, this
runs over twice as fast with a warm cache:
tinygo build -o test.elf ./testdata/stdlib.go
This should help a lot with edit-compile-test cycles, that typically
only modify a single package.
This required some changes to the interp package to deal with globals
created in a previous run of the interp package and to deal with
external globals (that can't be loaded from or stored to).
This commit replaces a number of panics with returning an error value as
a result of changing the toLLVMValue method signature. This should make
it easier to diagnose issues.
This makes it possible to assign I2C objects (machine.I2C0,
machine.I2C1, etc.) without needing to take a pointer.
This is important especially in the future when I2C may be driven using
DMA and the machine.I2C type needs to store some state.
These stubs don't really belong there: attiny currently doesn't directly
support I2C at all (although it has hardware to support a software
implementation).
This commit adds a new transform that converts reflect Implements()
calls to runtime.interfaceImplements. At the moment, the Implements()
method is not yet implemented (how ironic) but if the value passed to
Implements is known at compile time the method call can be optimized to
runtime.interfaceImplements to make it a regular interface assert.
This commit is the last change necessary to add basic support for the
encoding/json package. The json package is certainly not yet fully
supported, but some trivial objects can be converted to JSON.
This is important as golden test output and to verify that the output is
correct. Later improvements and bug fixes are clearly visible in the IR,
and unintentional changes will also be immediately spotted.
They both reversed the direction of the check, in a way that mostly
cancelled each other out. Of course they're still mostly unimplemented,
but it's better if they're not wrong.
This patch fixes a use of the global context. I've seen a few instances
of crashes in the llvm.ConstInt function when called from
makeStructTypeFields, which I believe are caused by this bug.
This fixes a type system loophole. The following program would
incorrectly run in TinyGo, while it would trigger a panic in Go:
package main
import "reflect"
func main() {
v := reflect.ValueOf(struct {
x int
}{})
x := v.Field(0).Interface()
println("x:", x.(int))
}
Playground link: https://play.golang.org/p/nvvA18XFqFC
The panic in Go is the following:
panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
I've shortened it in TinyGo to save a little bit of space.
These stub functions are necessary for the encoding/json package. They
don't seem to be called in trivial cases, so leave them as simple stubs
for now.
This matches the main Go implementation and (among others) fixes a
compatibility issue with the encoding/json package. The encoding/json
package compares reflect.Type variables against nil, which does not work
as long as reflect.Type is of integer type.
This also adds a reflect.RawType() function (like reflect.Type()) that
makes it easier to avoid working with interfaces in the runtime package.
It is internal only, but exported to let the runtime package use it.
This change introduces a small code size increase when working with the
reflect package, but I've tried to keep it to a minimum. Most programs
that don't make extensive use of the reflect package (and don't use
package like fmt) should not be impacted by this.
This is necessary so that when reflect.Type is converted from a concrete
type to an interface type, the errors package can still be interpreted.
Without this change, basically every program would grow in size by a few
bytes.
The errors package has a call like this in the package initializer. This
commit adds support for running it at compile time, avoiding the call at
runtime.
This doesn't always help (the call is already optimized away in many
small programs) but it does help to shave off some binary size in larger
programs. Perhaps more importantly, it will avoid a penalty in code size
when the reflect package will convert reflect.Type from a regular type
to an interface type.
Previously there was code to avoid impossible type asserts but it wasn't
great and in fact was too aggressive when combined with reflection.
This commit improves this by checking all types that exist in the
program that may appear in an interface (even struct fields and the
like) but without creating runtime.typecodeID objects with the type
assert. This has two advantages:
* As mentioned, it optimizes impossible type asserts away.
* It allows methods on types that were only asserted on (in
runtime.typeAssert) but never used in an interface to be optimized
away using GlobalDCE. This may have a cascading effect so that other
parts of the code can be further optimized.
This sometimes massively improves code size and mostly negates the code
size regression of the previous commit.
This distinction was useful before when reflect wasn't properly
supported. Back then it made sense to only include method sets that were
actually used in an interface. But now that it is possible to get to
other values (for example, by extracting fields from structs) and it is
possible to turn them back into interfaces, it is necessary to preserve
all method sets that can possibly be used in the program in a type
assert, interface assert or interface method call.
In the future, this logic will need to be revisited again when
reflect.New or reflect.Zero gets implemented.
Code size increases a bit in some cases, but usually in a very limited
way (except for one outlier in the drivers smoke tests). The next commit
will improve the situation significantly.
Previously we used the --export-all linker flag to export most
functions. However, this is not needed and possibly increases binary
size. Instead, we should be exporting the specific functions to be
exported.
An allocated object is never nil, so there is no need for a nil check.
This probably does not result in any better optimization (the nil check
is easily optimized away by LLVM because the result of runtime.alloc is
marked nonnull) but it makes the slice tests a bit cleaner.
It's difficult to create clean test cases while remaining compatible
with multiple LLVM versions. Most test outputs are much more readable
after an instcombine pass but instcombine rules change between LLVM
versions, leading to different (but semantically equivalent) test
outputs.
This reduces the test coverage a little bit (because old LLVM versions
aren't tested as well), but it als makes it easier to add more complex
tests.
In the future it might be a good idea to make the compiler output a bit
less messy so these workarounds are not needed.
This commit switches from the previous behavior of compiling the whole
program at once, to compiling every package in parallel and linking the
LLVM bitcode files together for further whole-program optimization.
This is a small performance win, but it has several advantages in the
future:
- There are many more things that can be done per package in parallel,
avoiding the bottleneck at the end of the compiler phase. This
should speed up the compiler futher.
- This change is a necessary step towards a non-LTO build mode for
fast incremental builds that only rebuild the changed package, when
compiler speed is more important than binary size.
- This change refactors the compiler in such a way that it will be
easier to inspect the IR for one package only. Inspecting this IR
will be very helpful for compiler developers.
The SimpleDCE pass was previously used to only compile the parts of the
program that were in use. However, lately the only real purpose has been
to speed up the compiler a bit by only compiling the necessary
functions.
This pass however is a problem for compiling (and caching) packages in
parallel. Therefore, this commit removes it as a preparatory step
towards that goal.
This is a leftover from a long time ago, when everything was still in
the global context. The fact that this uses the global context is most
certainly a bug.
I have seen occasional crashes in the build-packages-indepedently branch
(and PRs based on it) which I suspect are caused by this bug. I think
this is a long-dormant bug that only surfaced when doing the compilation
steps in parallel.
This optimization level wasn't working before because some passes expect
some globals to be cleaned up afterwards. Cleaning these globals is
easy, just add the pass necessary for it. This shouldn't reduce the
usefulness of the -opt=0 build flag as most optimizations are still
skipped.
2021-03-15 19:36:21 +01:00
1540 changed files with 123227 additions and 26581 deletions
We would like your help to make this project better, so we appreciate any contributions. See if one of the following descriptions matches your situation:
### New to TinyGo
We'd love to get your feedback on getting started with TinyGo. Run into any difficulty, confusion, or anything else? You are not alone. We want to know about your experience, so we can help the next people. Please open a Github issue with your questions, or you can also get in touch directly with us on our Slack channel at [https://gophers.slack.com/messages/CDJD3SUP6](https://gophers.slack.com/messages/CDJD3SUP6).
### Something in TinyGo is not working as you expect
Please open a Github issue with your problem, and we will be happy to assist.
### Something in Go that you want/need does not appear to be in TinyGo
We probably have not implemented it yet. Please take a look at our [Roadmap](https://github.com/tinygo-org/tinygo/wiki/Roadmap). Your pull request adding the functionality to TinyGo would be greatly appreciated.
Please open a Github issue. We want to help, and also make sure that there is no duplications of efforts. Sometimes what you need is already being worked on by someone else.
A long tail of small (and large) language features haven't been implemented yet. In almost all cases, the compiler will show a `todo:` error from `compiler/compiler.go` when you try to use it. You can try implementing it, or open a bug report with a small code sample that fails to compile.
### Some specific hardware you want to use does not appear to be in TinyGo
As above, we probably have not implemented it yet. Your contribution adding the hardware support to TinyGo would be greatly appreciated.
Please start by opening a Github issue. We want to help you to help us to help you.
Lots of targets/boards are still unsupported. Adding an architecture often requires a few compiler changes, but if the architecture is supported you can try implementing support for a new chip or board in `src/runtime`. For details, see [this wiki entry on adding archs/chips/boards](https://github.com/tinygo-org/tinygo/wiki/Adding-a-new-board).
Microcontrollers have lots of peripherals (I2C, SPI, ADC, etc.) and many don't have an implementation yet in the `machine` package. Adding support for new peripherals is very useful.
## How to use our Github repository
The `release` branch of this repo will always have the latest released version of TinyGo. All of the active development work for the next release will take place in the `dev` branch. TinyGo will use semantic versioning and will create a tag/release for each release.
Here is how to contribute back some code or documentation:
- Fork repo
- Create a feature branch off of the `dev` branch
- Make some useful change
- Make sure the tests still pass
- Submit a pull request against the `dev` branch.
- Be kind
## How to run tests
To run the tests:
```
make test
```
Please take a look at our [Contributing](https://tinygo.org/docs/guides/contributing/) page on our web site for details. Thank you.
@if [ ! -e lib/wasi-libc/Makefile ];thenecho"Submodules have not been downloaded. Please download them using:\n git submodule update --init";exit 1;fi
cd lib/wasi-libc && make -j4 WASM_CC=$(CLANG)WASM_AR=$(LLVM_AR)WASM_NM=$(LLVM_NM)
# Build the Go compiler.
tinygo:
@if [ ! -f "$(LLVM_BUILDDIR)/bin/llvm-config"];thenecho"Fetch and build LLVM first by running:";echo" make llvm-source";echo" make $(LLVM_BUILDDIR)";exit 1;fi
fpm -f -s dir -t deb -n tinygo -v $(shell grep "const Version = " goenv/version.go | awk '{print $$NF}') -m '@tinygo-org' --description='TinyGo is a Go compiler for small places.' --license='BSD 3-Clause' --url=https://tinygo.org/ --deb-changelog CHANGELOG.md -p build/release.deb -C ./build/release-deb
TinyGo is a Go compiler intended for use in small places such as microcontrollers, WebAssembly (Wasm), and command-line tools.
TinyGo is a Go compiler intended for use in small places such as microcontrollers, WebAssembly (wasm/wasi), and command-line tools.
It reuses libraries used by the [Go language tools](https://golang.org/pkg/go/) alongside [LLVM](http://llvm.org) to provide an alternative way to compile programs written in the Go programming language.
## Embedded
Here is an example program that blinks the built-in LED when run directly on any supported board with onboard LED:
```go
@@ -35,73 +37,64 @@ The above program can be compiled and run without modification on an Arduino Uno
tinygo flash -target arduino examples/blinky1
```
## WebAssembly
TinyGo is very useful for compiling programs both for use in browsers (WASM) as well as for use on servers and other edge devices (WASI).
TinyGo programs can run in [Fastly Compute](https://www.fastly.com/documentation/guides/compute/go/), [Fermyon Spin](https://developer.fermyon.com/spin/go-components), [wazero](https://wazero.io/languages/tinygo/) and many other WebAssembly runtimes.
Here is a small TinyGo program for use by a WASI host application:
```go
packagemain
//go:wasmexport add
funcadd(x,yuint32)uint32{
returnx+y
}
```
This compiles the above TinyGo program for use on any WASI Preview 1 runtime:
See the [getting started instructions](https://tinygo.org/getting-started/) for information on how to install TinyGo, as well as how to run the TinyGo compiler using our Docker container.
## Supported boards/targets
## Supported targets
You can compile TinyGo programs for microcontrollers, WebAssembly and Linux.
### Embedded
The following 55 microcontroller boards are currently supported:
You can compile TinyGo programs for over 94 different microcontroller boards.
Please take a look at our [CONTRIBUTING.md](./CONTRIBUTING.md) document for details.
Please take a look at our [Contributing](https://tinygo.org/docs/guides/contributing/) page on our web site for details.
## Project Scope
@@ -140,7 +133,6 @@ Goals:
Non-goals:
* Using more than one core.
* Be efficient while using zillions of goroutines. However, good goroutine support is certainly a goal.
* Be as fast as `gc`. However, LLVM will probably be better at optimizing certain things so TinyGo might actually turn out to be faster for number crunching.
returnnil,fmt.Errorf("cannot compile with Go toolchain version go%d.%d (TinyGo was built using toolchain version %s)",gorootMajor,gorootMinor,runtime.Version())
<p>How much space is used by Go packages, C libraries, and other bits to set up the program environment.</p>
<ul>
<li><strong>Code</strong> is the actual program code (machine code instructions).</li>
<li><strong>Read-only data</strong> are read-only global variables. On most microcontrollers, these are stored in flash and do not take up any RAM.</li>
<li><strong>Data</strong> are writable global variables with a non-zero initializer. On microcontrollers, they are copied from flash to RAM on reset.</li>
<li><strong>BSS</strong> are writable global variables that are zero initialized. They do not take up any space in the binary, but do take up RAM. On microcontrollers, this area is zeroed on reset.</li>
</ul>
<p>The binary size consists of code, read-only data, and data. On microcontrollers, this is exactly the size of the firmware image. On other systems, there is some extra overhead: binary metadata (headers of the ELF/MachO/COFF file), debug information, exception tables, symbol names, etc. Using <code>-no-debug</code> strips most of those.</p>
<h2>Program breakdown</h2>
<p>You can click on the rows below to see which files contribute to the binary size.</p>
matches:=regexp.MustCompile(`referenced by .* \(((.*):([0-9]+))\)`).FindStringSubmatch(line)
ifmatches!=nil{
parsedError=true
line,_:=strconv.Atoi(matches[3])
// TODO: detect common mistakes like -gc=none?
linkErrors=append(linkErrors,scanner.Error{
Pos:token.Position{
Filename:matches[2],
Line:line,
},
Msg:"linker could not find symbol "+symbolName,
})
}
}
}
// Check for flash/RAM overflow.
ifmatches:=regexp.MustCompile(`^ld.lld(-[0-9]+)?: error: section '(.*?)' will not fit in region '(.*?)': overflowed by ([0-9]+) bytes$`).FindStringSubmatch(message);matches!=nil{
region:=matches[3]
n,err:=strconv.ParseUint(matches[4],10,64)
iferr!=nil{
// Should not happen at all (unless it overflows an uint64 for some reason).
continue
}
// Check which area overflowed.
// Some chips use differently named memory areas, but these are by
// far the most common.
switchregion{
case"FLASH_TEXT":
ifn>flashOverflow{
flashOverflow=n
}
parsedError=true
case"RAM":
ifn>ramOverflow{
ramOverflow=n
}
parsedError=true
}
}
// If we couldn't parse the linker error: show the error as-is to
Msg:fmt.Sprintf("program too large for this chip (flash overflowed by %d bytes)\n\toptimization guide: https://tinygo.org/docs/guides/optimizing-binaries/",flashOverflow),
})
}
iframOverflow>0{
linkErrors=append(linkErrors,LinkerError{
Msg:fmt.Sprintf("program uses too much static RAM on this chip (RAM overflowed by %d bytes)",ramOverflow),
})
}
returnnewMultiError(linkErrors,"")
}
// LLD linker error that could not be parsed or doesn't refer to a source
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
deadprogram
Ayke van Laethem
Elias Naur
Randy Reddig
Egawa Takashi
soypat
Scott Feldman
Laurent Demailly
leongross
JP Hastings-Spital
m00874241
HattoriHanzo031
Yurii Soldak
milkpirate
vaaski
Roman Grudzinski
Volodymyr Pobochii
Elliott Sales de Andrade
sago35
Thomas Legris
Matt Mets
Damian Gryski
sivchari
Ben Krieger
Joonas Bergius
Daniel Esteban
BCG
archie2x
Warren Guy
Dan Kegel
Unrud
dkegel-fastly
Kobayashi Shunta
L. Pereira
Matthew Hiles
Tai Groot
Anatol Pomozov
Dmitry Shemin
leongross
diamondburned
frenkel26
Mateusz Nowak
Johann Freymuth
Marco Manino
Christian Stewart
hongkuang
Patrick Ting
Patrick Lindsay
Jonathan Böcker
Anders Savill
Akshay Pai
BCG
Xudong Zheng
=
Dmitriy
Rado M
Shane O'Donovan
Randy Reddig
Christian Ege
Kenneth Bell
Flavio Castelli
ginglis13
Anuraag Agrawal
ayan george
Timothy Rule
Pertti Erkkilä
Pierre Constantineau
Charlie Haley
Achille Roussel
Mansour Behabadi
Tyler Rockwood
Stepan Rakitin
ivoszz
Rajat Jindal
Daz Wilkin
Edoardo Vacchi
cui fliter
waj334
Justin A. Wilson
shivay
Damian Gryski
Christian Stewart
Federico G. Schwindt
Bjoern Poetzschke
John Clark
Andy Shinn
Anton D. Kachalov
Takeshi Yoneda
Olivier Fauchon
Adrian Cole
Marcus Sorensen
joey
Thomas Richner
irieda
Jesús Espino
Julia Ogris
goropikari
tachyonicbytes
Austin Vazquez
waj334
Matt Johnston
Fred Goya
Hrishi Hiraskar
Crypt Keeper
Matt Schultz
Isaac Rodman
Tim Schaub
Joe Shaw
Avi
Miguel Angel
Yeicor
Roman Volosatovs
Phil Kedy
ardnew
José Carlos Chávez
Nia Waldvogel
Krzysztof Jagiello
Olaf Flebbe
Steven Kabbes
Dylan Arbour
Dan Kegel
lincolngill
Sai Sasidhar Maddali
Aayush Attri
ZauberNerd
Samuel Stauffer
Kyle Conroy
Federico G. Schwindt
Pure White
Cameron Eagans
Nia
Tobias Theel
Arsen Musayelyan
kenbell
Alan Wang
Rouven Broszeit
jaap aarts
Ast-x64
learnforpractice
Mike Mogenson
Rajiv Kanchan
Raqbit
developer
Agurato
akif999