Jake Bailey
18033ebc36
* compiler, runtime, reflect: generate type-specific hash/equal for composite map keys
For map keys that are not trivially binary-comparable, the compiler now
generates type-specific hash and equal functions as LLVM IR instead of
going through the interface+reflection path. This covers comparable
types: strings, floats, complex numbers, interfaces, channels, and
composites containing any mix of these.
Previously, maps with composite keys containing strings or floats
converted the key to interface{}, hashed via reflection, and compared
through interface equality. Now the compiler walks struct fields and
array elements directly, dispatching to the right runtime helper for
each field type and storing keys at their actual type.
Struct keys are always handled field-by-field so padding bytes do not
affect equality or hashing. Blank fields are ignored, matching Go
equality. Generated hash/equal function names use canonical underlying
type structure so structurally identical key types can share generated
functions. Padding zeroing before map operations is no longer needed
because structs no longer use the binary key path.
Also fix reflect map iteration for interface-keyed maps: MapIter.Key
returns an interface Value for map[interface{}] keys instead of
unpacking to the concrete key kind.
* compiler: generate loops for array map key hash/equal
Previously, array key hash and equal functions were unrolled at compile
time, generating one block of IR per element. For large arrays like
[1000]int inside a struct with non-binary fields, this caused code
explosion.
Now, binary-element arrays dispatch directly to hash32/memequal for the
whole array. Non-binary-element arrays generate an LLVM IR loop. The
equal loop short-circuits on the first mismatch.
Small arrays are still unrolled instead of looping, keeping the simple
cases compact.
* reflect: fix at-runtime map issues from review, and more found locally
Maps created through reflect.MakeMap need hash/equal behavior that
matches compiler-created maps. Add hashmapMakeReflect for composite key
types, using runtime closures that reconstruct interface{} values from
raw key bytes and delegate to the interface hash and equality paths.
Interface-keyed maps are already stored as interface values, so use the
existing interface hash/equal helpers directly for those. This keeps
reflect insert, lookup, delete, and compiled lookup paths consistent.
Also fix addressable small values used as interface map keys or
interface map values. loadSmallValue puts small indirect values back in
the pointer-sized interface data field the same way valueInterfaceUnsafe
does.
* compiler, interp, reflect: fix pointer map literals; remove interface fallback
Package-level map literals with pointer keys (both *T and
unsafe.Pointer) crash the compiler: the interp pass panics when trying
to hash pointer data as raw bytes, because pointer values in the interp
memory model are symbolic identities that do not fit in a byte.
Fix this by setting a recoverable error flag instead of panicking. The
interp detects the error after each instruction and defers the map
insert to runtime init code, where real addresses are available for
hashing. This matches how the interp already handles other operations
it cannot evaluate at compile time.
With this fix, unsafe.Pointer can also be classified as a binary map
key, which was the last type requiring the interface-based fallback.
Since all comparable types now use either the binary or the
compiler-generated hash/equal path, remove the interface fallback from
the compiler and reflect packages.
* compiler, transform: always pass hash/equal function pointers to hashmapMakeGeneric
The compiler now always resolves the hash and equal functions at compile
time and passes them directly to hashmapMakeGeneric, instead of passing
an algorithm enum to hashmapMake and resolving at runtime. For string
keys, the runtime hashmapStringPtrHash/hashmapStringEqual functions are
referenced directly. For binary keys, hash32/memequal are referenced.
The old hashmapMake with alg enum is retained for reflect, which still
needs runtime resolution when creating maps dynamically.
The OptimizeMaps transform pass is updated to handle both hashmapMake
and hashmapMakeGeneric, and to recognize hashmapGenericSet in addition
to hashmapBinarySet and hashmapStringSet. The now-unused
hashmapCanGenerateHashEqual helper is removed.
* runtime: store large map keys and values indirectly
When a map key or value exceeds 128 bytes, the bucket now stores a
pointer to separately allocated memory instead of the data inline. This
matches Go's MapMaxKeyBytes/MapMaxElemBytes threshold and prevents
bucket sizes from exploding for large key/value types.
For example, map[[256]byte]int previously used 2128 bytes per bucket
(16 header + 256*8 keys + 8*8 values); now it uses 144 bytes per bucket
(16 header + 8*8 pointers + 8*8 values).
The indirection is fully encapsulated in the runtime via helper
functions. Store the computed key and value slot sizes on the hashmap so
all runtime and reflect paths use the same bucket layout, including
non-indirect keys and values.
Add big-key golden coverage and benchmarks. Make the benchmark vary
enough key bytes to exercise hashing.
Partial evaluation of initialization code in Go
For several reasons related to code size and memory consumption (see below), it
is best to try to evaluate as much initialization code at compile time as
possible and only run unknown expressions (e.g. external calls) at runtime. This
is in practice a partial evaluator of the runtime.initAll function, which
calls each package initializer.
This package is a rewrite of a previous partial evaluator that worked directly on LLVM IR and used the module and LLVM constants as intermediate values. This newer version instead uses a mostly Go intermediate form. It compiles functions and extracts relevant data first (compiler.go), then executes those functions (interpreter.go) in a memory space that can be rolled back per function (memory.go). This means that it is not necessary to scan functions to see whether they can be run at compile time, which was very error prone. Instead it just tries to execute everything and if it hits something it cannot interpret (such as a store to memory-mapped I/O) it rolls back the execution of that function and runs the function at runtime instead. All in all, this design provides several benefits:
- Much better error handling. By being able to revert to runtime execution without the need for scanning functions, this version is able to automatically work around many bugs in the previous implementation.
- More correct memory model. This is not inherent to the new design, but the new design also made the memory model easier to reason about.
- Faster execution of initialization code. While it is not much faster for normal interpretation (maybe 25% or so) due to the compilation overhead, it should be a whole lot faster for loops as it doesn't have to call into LLVM (via CGo) for every operation.
As mentioned, this partial evaluator comes in three parts: a compiler, an interpreter, and a memory manager.
Compiler
The main task of the compiler is that it extracts all necessary data from
every instruction in a function so that when this instruction is interpreted,
no additional CGo calls are necessary. This is not currently done for all
instructions (runtime.alloc is a notable exception), but at least it does
so for the vast majority of instructions.
Interpreter
The interpreter runs an instruction just like it would if it were executed 'for real'. The vast majority of instructions can be executed at compile time. As indicated above, some instructions need to be executed at runtime instead.
Memory
Memory is represented as objects (the object type) that contains data that
will eventually be stored in a global and values (the value interface) that
can be worked with while running the interpreter. Values therefore are only
used locally and are always passed by value (just like most LLVM constants)
while objects represent the backing storage (like LLVM globals). Some values
are pointer values, and point to an object.
Importantly, this partial evaluator can roll back the execution of a function. This is implemented by creating a new memory view per function activation, which makes sure that any change to a global (such as a store instruction) is stored in the memory view. It creates a copy of the object and stores that in the memory view to be modified. Once the function has executed successfully, all these modified objects are then copied into the parent function, up to the root function invocation which (on successful execution) writes the values back into the LLVM module. This way, function invocations can be rolled back without leaving a trace.
Pointer values point to memory objects, but not to a particular memory object. Every memory object is given an index, and pointers use that index to look up the current active object for the pointer to load from or to copy when storing to it.
Rolling back a function should roll back everything, including the few instructions emitted at runtime. This is done by treating instructions much like memory objects and removing the created instructions when necessary.
Why is this necessary?
A partial evaluator is hard to get right, so why go through all the trouble of writing one?
The answer is that globals with initializers are much easier to optimize by LLVM than initialization code. Also, there are a few other benefits:
- Dead globals are trivial to optimize away.
- Constant globals are easier to detect. Remember that Go does not have global constants in the same sense as that C has them. Constants are useful because they can be propagated and provide some opportunities for other optimizations (like dead code elimination when branching on the contents of a global).
- Constants are much more efficient on microcontrollers, as they can be allocated in flash instead of RAM.
The Go SSA package does not create constant initializers for globals. Instead, it emits initialization functions, so if you write the following:
var foo = []byte{1, 2, 3, 4}
It would generate something like this:
var foo []byte
func init() {
foo = make([]byte, 4)
foo[0] = 1
foo[1] = 2
foo[2] = 3
foo[3] = 4
}
This is of course hugely wasteful, it's much better to create foo as a
global array instead of initializing it at runtime.
For more details, see this section of the documentation.