* Add test.batch flag so standalone tests print PASS on pass
* Add comment; use single-dash flag style to match usage elsewhere
* Don't add test.batch for wasmtime
* Skip test.batch unless emulator name is blank
* Remove test.batch flag; buffer test output and show only on verbose or failure
* Remove FAIL when all tests fail to match go test output
Go 1.19 started reformatting code in a way that makes it more obvious
how it will be rendered on pkg.go.dev. It gets it almost right, but not
entirely. Therefore, I had to modify some of the comments so that they
are formatted correctly.
Also fix typo in error message in sub_test.go from upstream,
and move a few members from B to common where they belonged.
Note that testdata/testing.go seems to be pushing the edge of what
the emulated cortex-m3 target can handle; using regexp in that test
causes it to fail on that target with an out of memory error.
TODO: once tinygo supports runtime.Goexit, consider just using upstream's testing directory...
Only trivial functional changes:
- gets rid of mistaken extra "no tests" warning (whoops)
- matches upstream's exit code better
In preparation for switching to fancy test filtering.
This commit changes `tinygo test` to always look at the exit code of the
running test, instead of looking for a "PASS" string at the end of the
output. This is possible now that the binaries running under
qemu-system-arm or qemu-system-riscv32 will signal the correct exit code
when they exit.
As a side effect, this also makes it possible to avoid the "PASS" line
between successful tests. Before:
$ tinygo test container/heap container/list
PASS
ok container/heap 0.001s
PASS
ok container/list 0.001s
After:
$ tinygo test container/heap container/list
ok container/heap 0.001s
ok container/list 0.001s
The new behavior is more in line with upstream Go:
go test container/heap container/list
ok container/heap 0.004s
ok container/list 0.004s
This flag is passed automatically with the (new) -v flag for TinyGo. For
example, this prints all the test outputs:
$ tinygo test -v crypto/md5
=== RUN TestGolden
--- PASS: TestGolden
=== RUN TestGoldenMarshal
--- PASS: TestGoldenMarshal
=== RUN TestLarge
--- PASS: TestLarge
=== RUN TestBlockGeneric
--- PASS: TestBlockGeneric
=== RUN TestLargeHashes
--- PASS: TestLargeHashes
PASS
ok crypto/md5 0.002s
This prints just a summary:
$ tinygo test crypto/md5
PASS
ok crypto/md5 0.002s
(The superfluous 'PASS' message may be removed in the future).
This is especially useful when testing a large number of packages:
$ tinygo test crypto/md5 crypto/sha1 crypto/sha256 crypto/sha512
PASS
ok crypto/md5 0.002s
PASS
ok crypto/sha1 0.043s
PASS
ok crypto/sha256 0.002s
PASS
ok crypto/sha512 0.003s
At the moment, the -test.v flag is not supplied to binaries running in
emulation. I intend to fix this after
https://github.com/tinygo-org/tinygo/pull/2038 lands by refactoring
runPackageTest, Run, and runTestWithConfig in the main package which all
do something similar.
This commit makes the output of `tinygo test` similar to that of `go
test`. It changes the following things in the process:
* Running multiple tests in a single command is now possible. They
aren't paralellized yet.
* Packages with no test files won't crash TinyGo, instead it logs it
in the same way the Go toolchain does.
For example, for running tests with -target=wasm or
-target=cortex-m-qemu. It looks at the output to determine whether tests
were successful in the absence of a status code.
There were a few problems with the go/packages package. While it is more
or less designed for our purpose, it didn't work quite well as it didn't
provide access to indirectly imported packages (most importantly the
runtime package). This led to a workaround that sometimes broke
`tinygo test`.
This PR contains a number of related changes:
* It uses `go list` directly to retrieve the list of packages/files to
compile, instead of relying on the go/packages package.
* It replaces our custom TestMain replace code with the standard code
for running tests (generated by `go list`).
* It adds a dummy runtime/pprof package and modifies the testing
package, to get tests to run again with the code generated by
`go list`.
Implements nearly all of the test logging methods for both T and B
structs. Majority of the code has been copied from:
golang.org/src/testing/testing.go
then updated to match the existing testing.go structure.
Code structure/function/method order mimics upstream.
Both FailNow() and SkipNow() cannot be completely implemented,
because they require an early exit from the goroutine. Instead,
they call Error() to report the limitation.
This incomplete implementation allows more detailed test logging and
increases compatiblity with upstream.
Ayke van Laethem
leongross
Laurent Demailly
Damian Gryski
Fred Goya
Elliott Sales de Andrade
Dan Kegel
Lucas Teske
Brad Erickson
Carolyn Van Slyck