14 Commits

Author SHA1 Message Date
soypat 91a048fb1d add noslog build tag and small reference section to README 2026-09-01 13:12:51 -07:00
soypat c97a32f3c1 cull fmt package use and prevent aggressive DCE in MWE example with TinyGo 2026-09-01 09:39:10 -07:00
Ron Evans d219daa2c4 docs: remove Go Report Card since has been sunsetted (#195)
Signed-off-by: deadprogram <ron@hybridgroup.com>
2026-08-29 11:12:06 -03:00
Pat Whittingslow 75f1e02a20 dns: @hnw alternate proposal (#193)
* fix(dns): resolve CNAME chains and prevent invalid IP parsing

- Add automated in-band CNAME chain resolution to extract final A/AAAA IP addresses.
- Replace `MaxResponseAnswers` with `MaxIPs` and `MaxCNAMEs` to explicitly bound resource decoding and prevent memory exhaustion.
- Bound CNAME chain traversal to prevent infinite loops from cyclic records.

* refactor(dns): eagerly decode CNAME target into r.data, drop target field

Address review feedback on #189:

- Remove Resource.target: Resource.Decode expands CNAME RDATA in place
  into r.data (uncompressed wire format) while the full message is still
  available and updates header.Length, storing the name bytes exactly once.
- Add Resource.CNAMEView returning a length-bounded view of the expanded
  target; Message.WriteAnswers uses it.
- Rename matchesHost to ResourceHeader.pertainsTo.
- Make ResolveConfig.MaxCNAMEs explicit: drop the silent default in
  Client.StartResolve and let callers opt in (x/xnet).
- Reduce test suite to regression coverage only.

refs #189

* dns: simplify @hnw function and additional fixes

---------

Co-authored-by: Yoshio HANAWA <y@hnw.jp>
2026-08-28 21:39:12 -03:00
Pat Whittingslow ab9d3ee691 xnet: debug packet before ingressing (#192) 2026-08-26 11:36:54 -03:00
Pat Whittingslow e6a5be3628 ci: laxify codecoverage (#191)
* ci: laxify codecoverage

* reread coverage docs

* try this yaml out

* add project level codecov informational

* add a project wide coverage

* test project wide coverage status fail

* remove test code
2026-08-25 13:36:33 -03:00
Patricio Whittingslow 56f943ed30 ci: run 2026-08-24 22:03:42 -03:00
Derek den Haas c879d0497c fix(internal): keep the write position when a ring is read empty (#181)
Ring.onReadEnd and Ring.ReadDiscard call Reset() when the last readable
byte is consumed, which rewinds Off and End to 0. That is a valid empty
state, but it also moves the write position, and bytes staged past that
position with PeekWrite are addressed relative to it. After the rewind a
Commit hands back whatever bytes happen to live at the start of the
buffer instead of the staged ones.

tcp stages out-of-order segments exactly this way (see reassembly.store,
"the ring write pointer advances in lockstep with rcv.NXT, so the staged
bytes are always where seq implies"). Reading is driven by the
application, so a receiver that drains its stream while a gap is open
loses the staged tail silently: the stream arrives with the correct
length and the wrong contents.

Observed on a Sophgo SG2002 (100Mbit DWMAC, receive ring dropping frames
under load): a 16 MiB download arrived complete with a different
SHA-256, and TLS over the same path failed with "bad record MAC", while
the transmit path was bit-perfect.

Mark the ring empty without rewinding instead: End=0 is what empty
means, and the write position is then Off, which Ring.Write already
supports explicitly.

Two tests, both failing before the change:

  - internal: staged bytes survive the ring being read empty.
  - tcp: a reassembled stream is byte-identical when segments arrive
    reordered (segment K arrived as the contents of an earlier segment).

Co-authored-by: Derek den Haas <i.pestano@easyflor.nl>
2026-08-24 19:41:05 -03:00
Derek den Haas 6313b1570d fix(xnet): allocate ephemeral ports sequentially, not randomly (#180)
Ephemeral ports were drawn at random from the 16384-port dynamic range on
every dial. Under connection-per-request churn the birthday paradox reuses
a recently-released port after only a few dozen dials (~1% per dial at 200
outstanding-in-teardown), and a reused 4-tuple lands on whatever state the
previous conversation left along the path, such as the peer's TIME-WAIT
socket or a NAT's flow-table entry, which silently swallows the new SYN.
Measured end to end: an HTTP client with keep-alives off against a macOS
peer hit a dead dial after ~286 connections and stayed dark for ~30
seconds, exactly one 2MSL TIME-WAIT expiry.

Allocate sequentially from a per-stack random starting offset instead: a
port is only revisited after the full 16384-port cycle, and the random
start keeps a rebooted node off the ports its previous life just used.

TestEphemeralPortSequence pins the full-cycle-no-reuse property.

Co-authored-by: Derek den Haas <d.haas@directcode.com>
2026-08-24 19:39:46 -03:00
Derek den Haas 21f477b86e fix(tcp): retransmit unacknowledged data after a local close (#182)
ControlBlock.Send refused any outgoing segment carrying data in
FIN-WAIT-1, citing RFC 9293's "no further SENDs from the user will be
accepted by the TCP implementation". That rule bounds what the
application may queue, which Handler.Write already enforces, and not the
retransmission of data the connection has already accepted from it.

The consequence is that write-then-close, which is what nearly every
server does with a response, cannot recover from losing its last data
segment. The FIN occupies a sequence number above that data, so the peer
cannot cross the gap to process the close: it waits for bytes that are
never resent while the sender waits for an ACK that cannot arrive.
Neither side times out at the TCP layer.

FIN-WAIT-2 keeps the restriction and gains the reasoning: it is reached
by our FIN being acknowledged, which acknowledges everything below it, so
no unacknowledged data can remain there.

PendingSegment needs the same distinction, since it decides whether to
offer send-buffer data at all; it gets an unexported State predicate
rather than a new exported one.

Two tests, the second red before the change:

  - retransmission after RTO expiry, covering the Handler/LossRecovery
    seam that the RTO unit tests do not reach (they exercise the state
    machine in isolation, where it behaves correctly).
  - retransmission after a close with unacknowledged data.

Co-authored-by: Derek den Haas <i.pestano@easyflor.nl>
2026-08-24 19:36:48 -03:00
Yoshio HANAWA 263b1ecf11 fix(dns): decode up to four answer records (#186)
* fix(dns): decode up to four answer records

A single DNS question can return multiple answer records. Use an answer
decode limit of four in Client.StartResolve and add a regression test
covering a single-question response with multiple A records.

* fix(dns): add MaxResponseAnswers to ResolveConfig

This allows callers to explicitly declare the maximum number of answer
records to retain, removing the hardcoded limit in Client.StartResolve.

xnet.StackAsync is updated to set this limit to match the length of its
lookup-result buffer. This maintains the zero-allocation design while
fixing the issue where multiple A records were ignored.
2026-08-18 11:55:33 -07:00
Marvin 马维 Drees ab91d08f41 fix: elimite netdev test failure (#175)
Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-08-03 22:15:13 -07:00
Pat Whittingslow d05cd14018 httphi router refactor (#176)
* httphi: RouterConfig refactor to enable DefaultRouterConfig

* RequestHeader not case sensitive anymore

* httphi: use DefaultRouterConfig in examples

* httphi: remove gated stage complexity

Misusing Stage methods by calling them once header has been written is totally harmless as far as I can tell. We simplify the codebase on this occasion by removing the headerWritten check for all stage methods

* httphi: improve APIs

* httphi: remove status
2026-08-03 15:33:43 -07:00
Joel Wetzell 4517010070 use net.UDPAddrFromPort now available in tinygo (#95) 2026-07-31 15:14:10 -03:00
49 changed files with 1523 additions and 393 deletions
+9
View File
@@ -1,3 +1,12 @@
coverage:
status:
project:
default:
target: 62% # Ensure we don't accumulate too much debt.
patch:
default:
informational: true # Shows the patch metric but never turns red/fails the PR
ignore:
- "examples/**"
- "**/stringers.go"
+8 -1
View File
@@ -1,6 +1,5 @@
# lneto
[![go.dev reference](https://pkg.go.dev/badge/github.com/soypat/lneto)](https://pkg.go.dev/github.com/soypat/lneto)
[![Go Report Card](https://goreportcard.com/badge/github.com/soypat/lneto)](https://goreportcard.com/report/github.com/soypat/lneto)
[![codecov](https://codecov.io/gh/soypat/lneto/branch/main/graph/badge.svg)](https://codecov.io/gh/soypat/lneto)
[![Go](https://github.com/soypat/lneto/actions/workflows/ci.yaml/badge.svg)](https://github.com/soypat/lneto/actions/workflows/ci.yaml)
[![sourcegraph](https://sourcegraph.com/github.com/soypat/lneto/-/badge.svg)](https://github.com/soypat/lneto/network/dependents)
@@ -291,3 +290,11 @@ The document has moved
</BODY></HTML>
success
```
## Reference
### Build tags
- `debugheaplog`: All logging calls are enabled and all will print out heap information. Warning: Heavy cost on some TinyGo garbage collectors which do not cache the GC statistics
- `noslog`: All slog package logging calls omitted.
- `xnetdebug`: Packet capture printing to standard output enabled on `xnet.StackAsync` Ethernet and IP receive and send methods
+4 -20
View File
@@ -2,9 +2,6 @@ package arp
import (
"encoding/binary"
"fmt"
"net"
"net/netip"
"github.com/soypat/lneto"
"github.com/soypat/lneto/ethernet"
@@ -147,21 +144,8 @@ func (afrm Frame) ValidateSize(v *lneto.Validator) {
func (afrm Frame) String() string {
opstr := afrm.Operation().String()
hwt, _ := afrm.Hardware()
ptt, _ := afrm.Protocol()
sndhw, sndpt := afrm.Sender()
tgthw, tgtpt := afrm.Target()
var sndstr, tgtstr string
if ptt == ethernet.TypeIPv4 || ptt == ethernet.TypeIPv6 {
sender, _ := netip.AddrFromSlice(sndpt)
target, _ := netip.AddrFromSlice(tgtpt)
sndstr = sender.String()
tgtstr = target.String()
} else {
sndstr = net.HardwareAddr(sndpt).String()
tgtstr = net.HardwareAddr(tgtpt).String()
}
return fmt.Sprintf("ARP %s HW=(%d,SENDER=%s,TARGET=%s) PROTO=(%s,SENDER=%s,TARGET=%s)",
opstr, hwt, net.HardwareAddr(sndhw).String(), net.HardwareAddr(tgthw).String(),
ptt.String(), sndstr, tgtstr)
var rawbuf [11]byte
b := append(rawbuf[:0], "ARP "...)
b = append(b, opstr...)
return string(rawbuf[:len(b)])
}
+2 -3
View File
@@ -3,7 +3,6 @@ package dhcpv4
import (
"encoding/binary"
"errors"
"fmt"
"github.com/soypat/lneto"
"github.com/soypat/lneto/internal"
@@ -220,10 +219,10 @@ func (sv *Server) Demux(carrierData []byte, frameOffset int) error {
}
default:
err = fmt.Errorf("unhandled message type %s", msgType.String())
err = errors.New("unhandled message type: " + msgType.String())
}
if err != nil {
return fmt.Errorf("msgtype=%s client=%+v: %w", msgType.String(), client, err)
return errors.New("dhcpv4 server demux fail on " + msgType.String())
}
sv.hosts[clientIDRaw] = client
return nil
+11 -2
View File
@@ -24,6 +24,11 @@ type ResolveConfig struct {
Questions []Question
Additional []Resource
EnableRecursion bool
// MaxResponseAnswers limits how many answer records are decoded from the
// DNS response. If zero it defaults to the number of Questions. Answers
// are decoded in wire order regardless of type, so a response resolved
// through CNAMEs needs room for the CNAME records as well as the addresses.
MaxResponseAnswers uint16
}
func (sudp *Client) Protocol() uint64 { return uint64(lneto.IPProtoUDP) }
@@ -34,11 +39,15 @@ func (sudp *Client) ConnectionID() *uint64 { return &sudp.connID }
func (c *Client) StartResolve(localPort, txid uint16, cfg ResolveConfig) error {
nd := len(cfg.Questions)
if nd > math.MaxUint16 {
if nd > math.MaxUint16 || nd == 0 {
return lneto.ErrInvalidConfig
}
maxAns := cfg.MaxResponseAnswers
if maxAns == 0 {
maxAns = uint16(nd)
}
c.reset(localPort, txid, CQueryPending, cfg.EnableRecursion)
c.msg.LimitResourceDecoding(uint16(nd), uint16(nd), 0, 0)
c.msg.LimitResourceDecoding(uint16(nd), maxAns, 0, 0)
c.msg.AddQuestions(cfg.Questions)
c.msg.AddAdditionals(cfg.Additional)
return nil
+2
View File
@@ -197,6 +197,8 @@ const (
TypeALL Type = 255 // ALL
)
func (tp Type) IsIPAddr() bool { return tp == TypeA || tp == TypeAAAA }
// A Class is a type of network.
type Class uint16
+165 -19
View File
@@ -3,6 +3,7 @@ package dns
import (
"bytes"
"encoding/binary"
"encoding/hex"
"math"
"net/netip"
"slices"
@@ -99,6 +100,12 @@ func NamesEqual(a, b Name) bool {
return internal.BytesEqual(a.data, b.data)
}
// NamesEqualFold reports whether two DNS names are equal under ASCII case
// folding, which is how DNS labels compare per RFC 1035 section 2.3.3.
func NamesEqualFold(a, b Name) bool {
return internal.BytesEqualFoldASCII(a.data, b.data)
}
type ZFlags uint16
func NewResource(name Name, typ Type, class Class, ttl uint32, data []byte) Resource {
@@ -299,27 +306,57 @@ func (m *Message) AppendTo(buf []byte, txid uint16, flags HeaderFlags) (_ []byte
return buf, nil
}
// WriteAnswers writes the addresses answering host into dst, following the
// CNAME chain rooted at host. It returns the number of addresses written.
func (m *Message) WriteAnswers(dst []netip.Addr, host string) (n uint16, err error) {
for i := range m.Answers {
if int(n) >= len(dst) {
return n, lneto.ErrExhausted
// Each round resolves one CNAME, which consumes an answer. Bounding the
// walk by the answer count is thus enough to reach the addresses, and
// terminates on cyclic chains.
var alias Name // Canonical name reached so far; zero means host itself.
for range m.Answers {
var next Name
for i := range m.Answers {
ans := &m.Answers[i]
if !ans.header.ownedBy(alias, host) {
continue
}
switch {
case ans.header.Type.IsIPAddr():
if int(n) >= len(dst) {
return n, lneto.ErrExhausted
}
addr, ok := netip.AddrFromSlice(ans.RawData())
if !ok {
err = lneto.ErrInvalidAddr
continue
}
dst[n] = addr
n++
case ans.header.Type == TypeCNAME:
if cname := ans.CNAMEView(); cname.Len() != 0 {
next = cname
}
}
}
ans := &m.Answers[i]
hdr := ans.Header()
if !hdr.Name.EqualString(host) {
continue
}
var ok bool
dst[n], ok = netip.AddrFromSlice(ans.RawData())
if !ok {
err = lneto.ErrInvalidAddr
} else {
n++
if n > 0 || next.Len() == 0 {
break
}
alias = next
}
return n, err
}
// ownedBy reports whether the record's owner name is the name being resolved:
// the alias reached by following CNAMEs, or host at the root of the chain.
func (h *ResourceHeader) ownedBy(alias Name, host string) bool {
if alias.Len() == 0 {
return h.Name.EqualString(host)
}
// Fold: the server chooses the case of both the CNAME target and the owner
// name of the records it aliases, and may randomize it (DNS 0x20).
return NamesEqualFold(h.Name, alias)
}
func (m *Message) Len() uint16 {
return SizeHeader + m.lenResources()
}
@@ -390,10 +427,64 @@ func (m *Message) Reset() {
m.Additionals = m.Additionals[:0]
}
// AppendText appends a human readable representation of the Message's resources
// to b and returns the resulting slice. It implements [encoding.TextAppender].
func (m *Message) AppendText(b []byte) (_ []byte, err error) {
if len(m.Questions) > 0 {
b = append(b, "-- Questions\n"...)
for i := range m.Questions {
b, err = m.Questions[i].AppendText(b)
if err != nil {
return b, err
}
b = append(b, '\n')
}
}
b, err = appendResourcesText(b, "-- Answers\n", m.Answers)
if err != nil {
return b, err
}
b, err = appendResourcesText(b, "-- Authorities\n", m.Authorities)
if err != nil {
return b, err
}
return appendResourcesText(b, "-- Additionals\n", m.Additionals)
}
func appendResourcesText(b []byte, title string, resources []Resource) (_ []byte, err error) {
if len(resources) == 0 {
return b, nil
}
b = append(b, title...)
for i := range resources {
b, err = resources[i].AppendText(b)
if err != nil {
return b, err
}
b = append(b, '\n')
}
return b, nil
}
// String returns a string representation of the header.
func (h *ResourceHeader) String() string {
return h.Name.String() + " " + h.Type.String() + " " + h.Class.String() +
" ttl=" + strconv.FormatUint(uint64(h.TTL), 10) + " len=" + strconv.FormatUint(uint64(h.Length), 10)
b, _ := h.AppendText(make([]byte, 0, 64))
return string(b)
}
// AppendText appends a human readable representation of the header to b and
// returns the resulting slice. It implements [encoding.TextAppender].
func (h *ResourceHeader) AppendText(b []byte) ([]byte, error) {
b = h.Name.AppendDottedTo(b)
b = append(b, ' ')
b = append(b, h.Type.String()...)
b = append(b, ' ')
b = append(b, h.Class.String()...)
b = append(b, " ttl="...)
b = strconv.AppendUint(b, uint64(h.TTL), 10)
b = append(b, " len="...)
b = strconv.AppendUint(b, uint64(h.Length), 10)
return b, nil
}
func (r *Resource) Reset() {
@@ -411,6 +502,38 @@ func (r *Resource) RawData() []byte {
return r.data[:length]
}
// CNAMEView returns the canonical name held by a CNAME record, aliasing the
// Resource's buffer. It returns a zero Name for any other record type.
func (r *Resource) CNAMEView() Name {
if r.header.Type != TypeCNAME {
return Name{}
}
return Name{data: r.RawData()}
}
// String returns a string representation of the Resource: its header followed by
// the record's data.
func (r *Resource) String() string {
b, _ := r.AppendText(make([]byte, 0, 96))
return string(b)
}
// AppendText appends a human readable representation of the Resource to b: the
// header followed by the record's data, in dotted format for CNAME records and
// hexadecimal otherwise. It implements [encoding.TextAppender].
func (r *Resource) AppendText(b []byte) (_ []byte, err error) {
b, err = r.header.AppendText(b)
if err != nil {
return b, err
}
b = append(b, " data="...)
if r.header.Type == TypeCNAME {
cname := r.CNAMEView()
return cname.AppendDottedTo(b), nil
}
return hex.AppendEncode(b, r.RawData()), nil
}
func (q *Question) Reset() {
q.Name.Reset()
*q = Question{Name: q.Name} // Reuse Name's buffer.
@@ -449,7 +572,19 @@ func (q *Question) appendTo(buf []byte) (_ []byte, err error) {
// String returns a string representation of the Question with the Name in dotted format.
func (q *Question) String() string {
return q.Name.String() + " " + q.Type.String() + " " + q.Class.String()
b, _ := q.AppendText(make([]byte, 0, 32))
return string(b)
}
// AppendText appends a human readable representation of the Question to b with
// the Name in dotted format. It implements [encoding.TextAppender].
func (q *Question) AppendText(b []byte) ([]byte, error) {
b = q.Name.AppendDottedTo(b)
b = append(b, ' ')
b = append(b, q.Type.String()...)
b = append(b, ' ')
b = append(b, q.Class.String()...)
return b, nil
}
func (r *Resource) Decode(b []byte, off uint16) (uint16, error) {
@@ -460,8 +595,19 @@ func (r *Resource) Decode(b []byte, off uint16) (uint16, error) {
if r.header.Length > uint16(len(b[off:])) {
return off, errResourceLen
}
r.data = append(r.data[:0], b[off:off+r.header.Length]...)
return off + r.header.Length, nil
end := off + r.header.Length
if r.header.Type == TypeCNAME {
// CNAME data is a name which may use message compression. Expand it now
// since r.data is detached from b, leaving pointers unresolvable later.
cname := Name{data: r.data[:0]}
if _, derr := cname.Decode(b, off); derr == nil {
r.data = cname.data
r.header.Length = uint16(len(r.data))
return end, nil
}
}
r.data = append(r.data[:0], b[off:end]...)
return end, nil
}
func (r *Resource) appendTo(buf []byte) (_ []byte, err error) {
+233 -89
View File
@@ -1,7 +1,6 @@
package dns
import (
"fmt"
"net/netip"
"strings"
"testing"
@@ -195,33 +194,8 @@ func TestMessageAppendEncodeIncompleteOK(t *testing.T) {
}
func (m *Message) String() string {
// s := fmt.Sprintf("Message: %#v\n", &m.Header)
var s strings.Builder
if len(m.Questions) > 0 {
s.WriteString("-- Questions\n")
for _, q := range m.Questions {
s.WriteString(fmt.Sprintf("%#v\n", q))
}
}
if len(m.Answers) > 0 {
s.WriteString("-- Answers\n")
for _, a := range m.Answers {
s.WriteString(fmt.Sprintf("%#v\n", a))
}
}
if len(m.Authorities) > 0 {
s.WriteString("-- Authorities\n")
for _, ns := range m.Authorities {
s.WriteString(fmt.Sprintf("%#v\n", ns))
}
}
if len(m.Additionals) > 0 {
s.WriteString("-- Additionals\n")
for _, e := range m.Additionals {
s.WriteString(fmt.Sprintf("%#v\n", e))
}
}
return s.String()
b, _ := m.AppendText(nil)
return string(b)
}
func TestDecodeMessage(t *testing.T) {
@@ -239,80 +213,250 @@ func TestDecodeMessage(t *testing.T) {
}
}
func TestClient_ReceivesDNSResponse(t *testing.T) {
const hostname = "example.com"
const txid = uint16(12345)
// Regression test for CNAME-following: a response for www.yahoo.co.jp
// contains a CNAME record to edge12.g.yimg.jp (with compressed labels in its
// RDATA) followed by the A record for the canonical name. The CNAME RDATA
// must not be interpreted as an IP address and the A record must be returned.
func TestClient_CNAMEResponse(t *testing.T) {
const hostname = "www.yahoo.co.jp"
const txid = uint16(0x1234)
const clientPort = uint16(54321)
wantIP := [4]byte{93, 184, 216, 34}
// Build a DNS response message.
response := []byte{
// Header: txid 0x1234, QR|RD|RA, QD=1 AN=2 NS=0 AR=0.
0x12, 0x34, 0x81, 0x80, 0x00, 0x01, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00,
// Question: www.yahoo.co.jp A IN.
0x03, 'w', 'w', 'w', 0x05, 'y', 'a', 'h', 'o', 'o', 0x02, 'c', 'o', 0x02, 'j', 'p', 0x00,
0x00, 0x01, 0x00, 0x01,
// Answer 1: (ptr to question) CNAME IN ttl=842 rdlen=16
// rdata: edge12.g.yimg.jp with "jp" as compression pointer to offset 0x19.
0xc0, 0x0c, 0x00, 0x05, 0x00, 0x01, 0x00, 0x00, 0x03, 0x4a, 0x00, 0x10,
0x06, 'e', 'd', 'g', 'e', '1', '2', 0x01, 'g', 0x04, 'y', 'i', 'm', 'g', 0xc0, 0x19,
// Answer 2: (ptr into CNAME rdata) A IN ttl=36 rdlen=4 182.22.23.124.
0xc0, 0x2d, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00, 0x00, 0x24, 0x00, 0x04, 0xb6, 0x16, 0x17, 0x7c,
}
name := MustNewName(hostname)
responseMsg := Message{
var client Client
err := client.StartResolve(clientPort, txid, ResolveConfig{
Questions: []Question{{
Name: name,
Type: TypeA,
Class: ClassINET,
}},
Answers: []Resource{
NewResource(name, TypeA, ClassINET, 300, wantIP[:]),
},
}
// Response flags: QR=1 (response), RD=1, RA=1.
responseFlags := HeaderFlags(1<<15 | 1<<8 | 1<<7)
var buf [512]byte
dnsPayload, err := responseMsg.AppendTo(buf[:0], txid, responseFlags)
if err != nil {
t.Fatal("failed to build DNS response:", err)
}
// Set up the DNS client.
var client Client
client.StartResolve(clientPort, txid, ResolveConfig{
Questions: []Question{{
Name: MustNewName(hostname),
Type: TypeA,
Class: ClassINET,
}},
EnableRecursion: true,
EnableRecursion: true,
MaxResponseAnswers: 6,
})
// Simulate sending by calling Encapsulate (changes state to AwaitResponse).
var dummy [512]byte
client.Encapsulate(dummy[:], 0, 0)
// Call Demux with DNS payload.
err = client.Demux(dnsPayload, 0)
if err != nil {
t.Fatal("Client Demux error:", err)
t.Fatal("failed to start DNS resolve:", err)
}
var queryBuf [512]byte
_, err = client.Encapsulate(queryBuf[:], 0, 0)
if err != nil {
t.Fatal("failed to encapsulate DNS query:", err)
}
if err := client.Demux(response, 0); err != nil {
t.Fatal("failed to demux DNS response:", err)
}
// Check the client received the answer.
var addrs [4]netip.Addr
answers, err := client.ResponseAnswerLookup(addrs[:], hostname)
if answers != 1 {
t.Fatalf("expected 1 answer, got %d", answers)
}
addr := addrs[0]
if !addr.Is4() {
t.Fatalf("expected 4 bytes in answer, got %d", addr.BitLen()/8)
}
if addr.As4() != wantIP {
t.Errorf("expected IP %v, got %v", wantIP, addr.String())
}
// Test MessageCopyTo as well.
var lookup Message
lookup.LimitResourceDecoding(1, 1, 0, 0)
done, err := client.ResponseCopyTo(&lookup)
n, err := client.ResponseAnswerLookup(addrs[:], hostname)
if err != nil {
t.Fatal("MessageCopyTo error:", err)
t.Fatal("failed to look up DNS response answers:", err)
}
if !done {
t.Fatal("expected done=true")
if n != 1 {
t.Fatalf("expected 1 answer, got %d: %v", n, addrs[:n])
}
if len(lookup.Answers) != 1 {
t.Fatalf("MessageCopyTo: expected 1 answer, got %d", len(lookup.Answers))
if addrs[0] != (netip.AddrFrom4([4]byte{182, 22, 23, 124})) {
t.Fatalf("expected 182.22.23.124, got %v", addrs[0])
}
}
// Table-driven tests for Message.WriteAnswers covering answer reordering
// and cyclic CNAME aliases.
func TestMessage_WriteAnswers(t *testing.T) {
tests := []struct {
name string
host string
response []byte
want []netip.Addr
}{
{
name: "A record before its CNAME",
host: "www.yahoo.co.jp",
// Answer 1 is the A record for edge12.g.yimg.jp, spelled out with
// a trailing compression pointer to "jp" in the question. Answer 2
// is the CNAME from www.yahoo.co.jp whose RDATA is a single
// backward compression pointer to answer 1's owner name.
response: []byte{
// Header: txid 0x1234, QR|RD|RA, QD=1 AN=2 NS=0 AR=0.
0x12, 0x34, 0x81, 0x80, 0x00, 0x01, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00,
// Question: www.yahoo.co.jp A IN.
0x03, 'w', 'w', 'w', 0x05, 'y', 'a', 'h', 'o', 'o', 0x02, 'c', 'o', 0x02, 'j', 'p', 0x00,
0x00, 0x01, 0x00, 0x01,
// Answer 1: edge12.g.yimg.jp A IN ttl=36 rdlen=4 182.22.23.124.
0x06, 'e', 'd', 'g', 'e', '1', '2', 0x01, 'g', 0x04, 'y', 'i', 'm', 'g', 0xc0, 0x19,
0x00, 0x01, 0x00, 0x01, 0x00, 0x00, 0x00, 0x24, 0x00, 0x04, 0xb6, 0x16, 0x17, 0x7c,
// Answer 2: (ptr to question) CNAME IN ttl=842 rdlen=2, target
// is a pointer to answer 1's owner name at offset 0x21.
0xc0, 0x0c, 0x00, 0x05, 0x00, 0x01, 0x00, 0x00, 0x03, 0x4a, 0x00, 0x02, 0xc0, 0x21,
},
want: []netip.Addr{netip.AddrFrom4([4]byte{182, 22, 23, 124})},
},
{
name: "CNAME cycle terminates",
host: "a.com",
response: []byte{
// Header: txid 0xabcd, QR|RD|RA, QD=1 AN=2 NS=0 AR=0.
0xab, 0xcd, 0x81, 0x80, 0x00, 0x01, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00,
// Question: a.com A IN.
0x01, 'a', 0x03, 'c', 'o', 'm', 0x00, 0x00, 0x01, 0x00, 0x01,
// Answer 1: a.com CNAME b.com.
0x01, 'a', 0x03, 'c', 'o', 'm', 0x00, 0x00, 0x05, 0x00, 0x01, 0x00, 0x00, 0x00, 0x0a, 0x00, 0x07, 0x01, 'b', 0x03, 'c', 'o', 'm', 0x00,
// Answer 2: b.com CNAME a.com.
0x01, 'b', 0x03, 'c', 'o', 'm', 0x00, 0x00, 0x05, 0x00, 0x01, 0x00, 0x00, 0x00, 0x0a, 0x00, 0x07, 0x01, 'a', 0x03, 'c', 'o', 'm', 0x00,
},
want: nil,
},
{
name: "CNAME target case differs from owner name",
host: "a.com",
// A server picks the case of both the CNAME target and the owner
// name of the record it aliases, and may randomize it (DNS 0x20),
// so the two must compare under ASCII case folding.
response: []byte{
// Header: txid 0xabcd, QR|RD|RA, QD=1 AN=2 NS=0 AR=0.
0xab, 0xcd, 0x81, 0x80, 0x00, 0x01, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00,
// Question: a.com A IN.
0x01, 'a', 0x03, 'c', 'o', 'm', 0x00, 0x00, 0x01, 0x00, 0x01,
// Answer 1: a.com CNAME B.CoM.
0x01, 'a', 0x03, 'c', 'o', 'm', 0x00, 0x00, 0x05, 0x00, 0x01, 0x00, 0x00, 0x00, 0x0a, 0x00, 0x07, 0x01, 'B', 0x03, 'C', 'o', 'M', 0x00,
// Answer 2: b.com A IN ttl=10 rdlen=4 1.2.3.4.
0x01, 'b', 0x03, 'c', 'o', 'm', 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00, 0x00, 0x0a, 0x00, 0x04, 0x01, 0x02, 0x03, 0x04,
},
want: []netip.Addr{netip.AddrFrom4([4]byte{1, 2, 3, 4})},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var msg Message
msg.LimitResourceDecoding(1, 4, 0, 0)
_, incomplete, err := msg.Decode(tt.response)
if incomplete || err != nil {
t.Fatal("decode:", incomplete, err)
}
var addrs [4]netip.Addr
n, err := msg.WriteAnswers(addrs[:], tt.host)
if err != nil {
t.Fatal("write answers:", err)
}
if n != uint16(len(tt.want)) {
t.Fatalf("expected %d addresses, got %d: %v", len(tt.want), n, addrs[:n])
}
for i, want := range tt.want {
if addrs[i] != want {
t.Errorf("address %d: expected %v, got %v", i, want, addrs[i])
}
}
})
}
}
func TestClient_ReceivesDNSResponse(t *testing.T) {
const hostname = "example.com"
const txid = uint16(12345)
const clientPort = uint16(54321)
const maxAnswers = 4
allIPs := [5][4]byte{
{192, 0, 2, 1},
{192, 0, 2, 2},
{192, 0, 2, 3},
{192, 0, 2, 4},
{192, 0, 2, 5},
}
tests := []struct {
name string
responseIPs [][4]byte
wantAnswers int // Addresses returned by ResponseAnswerLookup and copied by ResponseCopyTo.
}{
{name: "single_answer", responseIPs: allIPs[:1], wantAnswers: 1},
{name: "multiple_answers", responseIPs: allIPs[:4], wantAnswers: 4},
{name: "answer_limit", responseIPs: allIPs[:5], wantAnswers: maxAnswers},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
name := MustNewName(hostname)
responseMsg := Message{
Questions: []Question{{
Name: name,
Type: TypeA,
Class: ClassINET,
}},
Answers: make([]Resource, len(tt.responseIPs)),
}
for i := range tt.responseIPs {
responseMsg.Answers[i] = NewResource(name, TypeA, ClassINET, 300, tt.responseIPs[i][:])
}
// Response flags: QR=1 (response), RD=1, RA=1.
responseFlags := HeaderFlags(1<<15 | 1<<8 | 1<<7)
var responseBuf [512]byte
dnsPayload, err := responseMsg.AppendTo(responseBuf[:0], txid, responseFlags)
if err != nil {
t.Fatal("failed to build DNS response:", err)
}
var client Client
err = client.StartResolve(clientPort, txid, ResolveConfig{
Questions: []Question{{
Name: name,
Type: TypeA,
Class: ClassINET,
}},
EnableRecursion: true,
MaxResponseAnswers: maxAnswers,
})
if err != nil {
t.Fatal("failed to start DNS resolve:", err)
}
// Encapsulate the query to move the client into the outstanding state.
var queryBuf [512]byte
_, err = client.Encapsulate(queryBuf[:], 0, 0)
if err != nil {
t.Fatal("failed to encapsulate DNS query:", err)
}
if err := client.Demux(dnsPayload, 0); err != nil {
t.Fatal("failed to demux DNS response:", err)
}
var addrs [maxAnswers]netip.Addr
answers, err := client.ResponseAnswerLookup(addrs[:], hostname)
if err != nil {
t.Fatal("failed to look up DNS response answers:", err)
}
if int(answers) != tt.wantAnswers {
t.Fatalf("expected %d answers, got %d", tt.wantAnswers, answers)
}
for i := 0; i < tt.wantAnswers; i++ {
addr := addrs[i]
if !addr.Is4() {
t.Errorf("answer %d: expected IPv4 address, got %v", i, addr)
continue
}
if addr.As4() != tt.responseIPs[i] {
t.Errorf("answer %d: expected IP %v, got %v", i, tt.responseIPs[i], addr)
}
}
var lookup Message
done, err := client.ResponseCopyTo(&lookup)
if err != nil {
t.Fatal("failed to copy DNS response:", err)
}
if !done {
t.Fatal("expected done=true")
}
if len(lookup.Answers) != tt.wantAnswers {
t.Fatalf("expected %d copied answers, got %d", tt.wantAnswers, len(lookup.Answers))
}
})
}
}
+7 -17
View File
@@ -18,14 +18,10 @@ import (
)
const (
kB = 1 << 10
listenPort = 8080
bufferSizes = 2 * kB
// A browser sends around twenty header fields; a request carrying more
// than this is answered 431 rather than parsed into memory it was not
// given. Each field costs 8 bytes of table.
numHeaderFields = 32
readTimeout = 2 * time.Second
kB = 1 << 10
listenPort = 8080
memoryPerConn = 4 * kB
readTimeout = 2 * time.Second
)
// Credentials the endpoints check. They are in the source on purpose: this is a
@@ -84,14 +80,8 @@ func run() error {
server.Handle("/echo", server.echo) // No method: any method matches.
var router httphi.Router
err = router.Configure(httphi.RouterConfig{
FixedNumGoroutines: *flagThreads,
RequestHeaderBufferSize: bufferSizes,
RequestNumHeaderKVCap: numHeaderFields,
ResponseHeaderMinBufferSize: bufferSizes,
Mux: &server.mux,
Logger: slog.Default(),
})
cfg := httphi.DefaultRouterConfig(*flagThreads, memoryPerConn, server.mux.MaxPathValues())
err = router.Configure(&server.mux, cfg)
if err != nil {
return err
}
@@ -435,7 +425,7 @@ func (sv *Server) upload(exch *httphi.Exchange) {
func (sv *Server) echo(exch *httphi.Exchange) {
s := sv.acquireScratch()
defer sv.releaseScratch(s)
body := append(s.out[:0], exch.RequestMethodRaw()...)
body := append(s.out[:0], exch.RequestMethodBytes()...)
body = append(body, ' ')
body = append(body, exch.RequestTarget()...)
body = append(body, '\n')
+5 -15
View File
@@ -38,12 +38,7 @@ var indexhtml string
// Router memory. The router allocates all of it on Configure and never again,
// so these are the whole cost of serving HTTP over the stack.
const (
// A browser sends around 700 bytes of header on a landing page request.
requestHeaderBuffer = 1024
// Response headers reuse whatever the request left unused on top of this,
// and the status line does not count towards it.
responseHeaderBuffer = 256
numHeaderFields = 16
httpConnMemoryUse = 4 * 1024
// One exchange is allocated per worker, and a worker holds its exchange for
// the whole request, so this is what bounds requests served at once.
numWorkers = 2
@@ -252,14 +247,9 @@ func run() (err error) {
server.handle("GET /stats", server.stats)
var router httphi.Router
err = router.Configure(httphi.RouterConfig{
FixedNumGoroutines: numWorkers,
RequestHeaderBufferSize: requestHeaderBuffer,
ResponseHeaderMinBufferSize: responseHeaderBuffer,
RequestNumHeaderKVCap: numHeaderFields,
Mux: &server.mux,
Logger: slog.Default(),
})
cfg := httphi.DefaultRouterConfig(numWorkers, httpConnMemoryUse, server.mux.MaxPathValues())
cfg.Logger = slog.Default()
err = router.Configure(&server.mux, cfg)
if err != nil {
return fmt.Errorf("configuring HTTP router: %w", err)
}
@@ -322,7 +312,7 @@ type httpServer struct {
func (sv *httpServer) handle(pattern string, handler httphi.HandlerFunc) {
sv.mux.Handle(pattern, func(exch *httphi.Exchange) {
sv.served.Add(1)
fmt.Printf("< %s %s\n", exch.RequestMethodRaw(), exch.RequestTarget())
fmt.Printf("< %s %s\n", exch.RequestMethodBytes(), exch.RequestTarget())
handler(exch)
})
}
+7 -17
View File
@@ -14,15 +14,11 @@ import (
)
const (
kB = 1 << 10
listenPort = 8080
bufferSizes = 2 * kB
// A browser sends around twenty header fields; a request carrying more
// than this is answered 431 rather than parsed into memory it was not
// given. Each field costs 8 bytes of table.
numHeaderFields = 32
numGoroutines = 4
readTimeout = 2 * time.Second
kB = 1 << 10
listenPort = 8080
connMemoryUse = 4 * kB
numGoroutines = 4
readTimeout = 2 * time.Second
)
func main() {
@@ -45,14 +41,8 @@ func run() error {
server.Handle("GET /", server.homepage)
var router httphi.Router
err = router.Configure(httphi.RouterConfig{
FixedNumGoroutines: numGoroutines,
RequestHeaderBufferSize: bufferSizes,
RequestNumHeaderKVCap: numHeaderFields,
ResponseHeaderMinBufferSize: bufferSizes,
Mux: &server.mux,
Logger: slog.Default(),
})
cfg := httphi.DefaultRouterConfig(numGoroutines, connMemoryUse, server.mux.MaxPathValues())
err = router.Configure(&server.mux, cfg)
if err != nil {
return err
}
+17 -13
View File
@@ -2,7 +2,7 @@ package main
import (
"context"
"fmt"
"errors"
"net"
"net/netip"
"os"
@@ -46,7 +46,7 @@ func main() {
var stack xnet.StackAsync
ctx := context.Background()
if err := run(ctx, &stack); err != nil {
fmt.Println(err)
os.Stdout.WriteString(err.Error())
os.Exit(1)
}
}
@@ -71,7 +71,7 @@ func run(ctx context.Context, stack *xnet.StackAsync) error {
HardwareAddress: hwaddr,
})
if err != nil {
return fmt.Errorf("configuring stack: %w", err)
return makeMsgErr("configuring stack", err)
}
ctx, cancel := context.WithCancel(ctx)
defer cancel()
@@ -82,18 +82,18 @@ func run(ctx context.Context, stack *xnet.StackAsync) error {
rstack := stack.StackRetrying(stackBackoff)
results, err := rstack.DoDHCPv4([4]byte{}, protoTimeout, protoRetries)
if err != nil {
return fmt.Errorf("doing DHCP: %w", err)
return makeMsgErr("doing DHCP", err)
}
err = stack.AssimilateDHCPResults(results)
if err != nil {
return fmt.Errorf("assimilating DHCP: %w", err)
return makeMsgErr("assimilating DHCP", err)
}
gateway, err := rstack.DoResolveHardwareAddress6(results.Router, protoTimeout, protoRetries)
if err != nil {
return fmt.Errorf("resolving router MAC: %w", err)
return makeMsgErr("resolving Router MAC", err)
}
stack.SetGatewayHardwareAddr(gateway)
berkstack := stack.StackBlocking(stackBackoff).StackGo(xnet.StackGoConfig{
gostack := stack.StackBlocking(stackBackoff).StackGo(xnet.StackGoConfig{
ListenerPoolConfig: xnet.TCPPoolConfig{
PoolSize: tcpConnPoolSize,
QueueSize: tcpPacketQueueSize,
@@ -109,16 +109,16 @@ func run(ctx context.Context, stack *xnet.StackAsync) error {
laddr := net.TCPAddrFromAddrPort(netip.AddrPortFrom(netip.AddrFrom4(results.AssignedAddr4), 80))
// raddr := net.TCPAddr{} // If active (client) connection then set raddr in which case a net.Conn type is returned.
const sockstream = 0x1
c, err := berkstack.Socket(ctx, "tcp", syscall.AF_INET, sockstream, laddr, nil)
c, err := gostack.Socket(ctx, "tcp", syscall.AF_INET, sockstream, laddr, nil)
if err != nil {
return fmt.Errorf("creating AF_INET stream socket: %w", err)
return makeMsgErr("creating AF_INET stream socket", err)
}
listener := c.(net.Listener)
for ctx.Err() == nil {
time.Sleep(pollTime)
conn, err := listener.Accept()
if err != nil {
fmt.Println("conn failed:", err)
return makeMsgErr("listener.Accept failed", err)
}
go handleConn(conn)
}
@@ -145,18 +145,18 @@ func stackLoop(ctx context.Context, stack *xnet.StackAsync) {
for ctx.Err() == nil {
nwrite, err := stack.EgressEthernet(buf[:])
if err != nil {
fmt.Println("encaps err:", err)
os.Stderr.WriteString(err.Error())
} else if nwrite > 0 {
network.SendEth(buf[:nwrite])
cap.PrintEthernet("OUT", buf[:nwrite])
}
nread, err := network.RecvEth(buf[:])
if err != nil {
fmt.Println("network read err:", err)
os.Stderr.WriteString(err.Error())
} else if nread > 0 {
err = stack.IngressEthernet(buf[:nread])
if err != nil && err != lneto.ErrPacketDrop {
fmt.Println("demux err:", err)
os.Stderr.WriteString(err.Error())
} else {
cap.PrintEthernet("IN ", buf[:nread])
}
@@ -191,3 +191,7 @@ func tcpBackoff(consecutiveBackoffs uint) time.Duration {
wait := min(shifted, maxWait)
return time.Duration(wait)
}
func makeMsgErr(msg string, err error) error {
return errors.New(msg + ": " + err.Error())
}
+37
View File
@@ -0,0 +1,37 @@
package main
import (
"math/rand"
"time"
"github.com/soypat/lneto/ethernet"
)
func init() {
mn := &mockNetwork{
rng: rand.New(rand.NewSource(time.Now().UnixNano())),
}
network = mn
}
type mockNetwork struct {
rng *rand.Rand
}
func (m *mockNetwork) SendEth(frame []byte) error {
return nil
}
func (m *mockNetwork) RecvEth(dst []byte) (int, error) {
n := m.rng.Int() % ethernet.MaxFrameLength
if n < ethernet.MinimumFrameLength {
return 0, nil
}
n, _ = m.rng.Read(dst[:min(len(dst), n)])
return n, nil
}
func (m *mockNetwork) HardwareAddress6() ([6]byte, error) {
return [6]byte{0xde, 0xad, 0xbe, 0xef, 0x00, 0x00}, nil
}
func (m *mockNetwork) MaxFrameLength() (int, error) {
return ethernet.MaxFrameLength, nil
}
+2 -7
View File
@@ -22,13 +22,8 @@ mux.Handle("GET /", func(ex *httphi.Exchange) {
})
var router httphi.Router
err := router.Configure(httphi.RouterConfig{
FixedNumGoroutines: 4, // 4 workers, 4 exchanges, allocated here and never again.
RequestHeaderBufferSize: 1024,
ResponseHeaderMinBufferSize: 32, // Shares the request buffer.
RequestNumHeaderKVCap: 32,
Mux: &mux,
})
cfg := httphi.DefaultRouterConfig(numWorkers, memoryPerConn, mux.MaxPathValues())
err := router.Configure(&mux, cfg)
if err != nil {
log.Fatal(err)
}
+4 -13
View File
@@ -4,7 +4,6 @@ import (
"fmt"
"io"
"log"
"log/slog"
"net"
"os"
@@ -15,23 +14,15 @@ import (
// ExampleRouter_linux goes over how to setup a linux server using raw linux connections.
// See [ExampleMuxSlice_query_forms_multipart] on how to define handlers for common HTTP processing.
func ExampleRouter() {
// Chrome tends to send ~700 bytes on a typical landing page request.
const requestBuffer = 1024
const numHeaderKV = requestBuffer / 32 //
const numWorkers = 8
const memoryPerConn = 2048
var mux httphi.MuxSlice
mux.Handle("GET /", func(ex *httphi.Exchange) {
ex.WriteBody([]byte("hello world"))
})
var router httphi.Router
err := router.Configure(httphi.RouterConfig{
FixedNumGoroutines: -1, // Unbounded goroutines and allocations.
RequestHeaderBufferSize: requestBuffer,
ResponseHeaderMinBufferSize: 32, // Shared buffer with Request, not strictly necessary, especially if not sending headers.
RequestNumHeaderKVCap: numHeaderKV,
NormalizeOutgoingKeys: true,
Mux: &mux,
Logger: slog.Default(),
})
cfg := httphi.DefaultRouterConfig(numWorkers, memoryPerConn, mux.MaxPathValues())
err := router.Configure(&mux, cfg)
if err != nil {
log.Fatal(err)
}
+17 -30
View File
@@ -75,10 +75,8 @@ type ExchangeConfig struct {
// Optional [any] cap holding the request header to RequestBufferLim rather than
// growing it. A header outgrowing it is answered 431, see [httpraw.ErrBufferExhausted].
NoRequestBufferGrowth bool
// Conditional [>=the most wildcards any one registered pattern binds] number of
// path values bindable, read back with [Exchange.PathValue]. A pattern binding
// more never matches, see [SetPathValues]. Zero suits a mux of literal patterns.
MaxPathValues int
// Conditional [len >=[Mux.MaxPathValues]] written to during [Mux.LookupHandler] in [Handle].
PathValuesBuf []PathValue
}
// HijackRaw is a low-level implementation of http.Hijacker interface.
@@ -124,8 +122,7 @@ func (exch *Exchange) Configure(cfg ExchangeConfig) {
exch.reqHdr.Reset(cfg.RawBuf[:0:cfg.RequestBufferLim], cfg.NumHeaderKVCap)
exch.reqHdr.ConfigBufferGrowth(!cfg.NoRequestBufferGrowth)
exch.normalizeKeys = cfg.NormalizeOutgoingKeys
internal.SliceReuse(&exch.pathValues, cfg.MaxPathValues)
exch.pathValues = exch.pathValues[:cfg.MaxPathValues]
exch.pathValues = cfg.PathValuesBuf
}
// Acquire claims the exchange for conn and resets it to serve a new request,
@@ -170,27 +167,20 @@ func (exch *Exchange) Release() {
// Does not return the buffer used for the response first line so can be safely
// written to and used without modifying the staged response first line.
//
// Staging headers will write to this buffer so use mindfully.
// To access only the request header buffer portion use [httpraw.HeaderV1.BufferRaw] limited
// to [httpraw.HeaderV1.BufferParsed] as returned by [Exchange.requestHeaderRaw].
// Writing to this section will not change the contents read by [Exchange.ReadBody].
// Writing to this aforementioned section will not change the contents read by [Exchange.ReadBody].
//
// In [Router] context, the size of this buffer is influenced directly by [RouterConfig] HeaderBufferSize fields.
func (exch *Exchange) UnsafeRawBuffer() []byte { return exch.rawbuf }
// RequestHeaderV1Raw returns the parsed request header for access beyond the
// Request* methods, such as [httpraw.HeaderV1.ForEach]. Valid until the exchange
// is released, and writing to it corrupts the response.
// RequestHeaderV1Raw returns the internal [Exchange] data structure used for HTTP/1.x requests.
func (exch *Exchange) RequestHeaderV1Raw() *httpraw.HeaderV1 { return &exch.reqHdr }
// StageHeader stages a response header field, written on the first
// [Exchange.FlushHeader], [Exchange.WriteHeader] or [Exchange.WriteBody].
// Returns false and drops the field if the response buffer cannot fit it.
// Has no effect once the header has been written.
func (exch *Exchange) StageHeader(key, value string) (enoughMemory bool) {
if exch.headerWritten {
return false
}
off := int(exch.respHeaderOff) + int(exch.respHeaderLen)
free := len(exch.rawbuf) - off
// Field costs key+':'+value+CRLF, plus the CRLF [Exchange.FlushHeader]
@@ -230,7 +220,7 @@ func (exch *Exchange) StageHeaderInt(key string, value int64) (enoughMemory bool
// base must be in the range 10..36; lower bases are dropped, no HTTP header
// field value is written below base 10.
func (exch *Exchange) StageHeaderIntBase(key string, value int64, base int) (enoughMemory bool) {
if exch.headerWritten || base < 10 || base > 36 {
if base < 10 || base > 36 {
return false
}
off := int(exch.respHeaderOff) + int(exch.respHeaderLen)
@@ -255,9 +245,9 @@ func (exch *Exchange) StageHeaderIntBase(key string, value int64, base int) (eno
// StageStatus prepares the status line for the given code without writing
// it, i.e: "HTTP/1.1 404 Not Found". Codes with no [StatusText] get an empty
// reason phrase. Has no effect once the header has been written.
// reason phrase.
func (exch *Exchange) StageStatus(code int) {
if code >= 1000 || exch.headerWritten {
if code >= 1000 {
return
} else if code == 200 {
// Common case.
@@ -278,11 +268,8 @@ func (exch *Exchange) StageStatus(code int) {
// WriteHeader sends the status line for code along with the staged header
// fields. Only the first call reaches the wire, as in http.ResponseWriter.
func (exch *Exchange) WriteHeader(code int) (n int, err error) {
if !exch.headerWritten {
exch.StageStatus(code)
n, err = exch.FlushHeader()
}
return n, err
exch.StageStatus(code)
return exch.FlushHeader()
}
// Respond writes a complete response in one call: Content-Type, a Content-Length
@@ -497,7 +484,7 @@ func (exch *Exchange) RequestParseCookie(dst *httpraw.Cookie, key string) error
func (exch *Exchange) RequestContentType() []byte {
// Folded: field names are case insensitive and HTTP/2 mandates lowercase, so
// a proxy translating h2 to h1 sends "content-type", RFC 9110 5.1.
return exch.RequestHeaderV1Raw().GetFold("Content-Type")
return exch.RequestHeader("Content-Type")
}
// RequestContentLength returns the body length declared by the request's
@@ -719,10 +706,10 @@ func (exch *Exchange) ReadMultiparts(dst []MultipartSink, buf []byte, newSink fu
}
// RequestHeader returns the value of the first request header field matching
// key, or nil if absent. Key matching is case sensitive.
// key, or nil if absent. Matching is not case sensitive.
func (exch *Exchange) RequestHeader(key string) []byte {
header := exch.RequestHeaderV1Raw()
return header.Get(key)
return header.GetFold(key)
}
// RequestTarget returns the request-target (URI) of the request line, i.e:
@@ -738,7 +725,7 @@ func (exch *Exchange) RequestPath() []byte {
}
// RequestQuery returns the request's query string as it appears on the wire.
// Iterate it with [httpraw.NextQueryPair]. See [httpraw.HeaderV1.RequestQuery].
// Iterate it with [httpraw.NextQueryPair].
func (exch *Exchange) RequestQuery() []byte {
return exch.RequestHeaderV1Raw().RequestQuery()
}
@@ -828,11 +815,11 @@ func (exch *Exchange) PathValueAppend(dst []byte, key string, decoded bool) ([]b
// RequestMethod returns the request's [Method] enum.
func (exch *Exchange) RequestMethod() Method {
return MethodFromBytes(exch.RequestMethodRaw())
return MethodFromBytes(exch.RequestMethodBytes())
}
// RequestMethod returns the request line's method as a []byte view, i.e: "GET".
func (exch *Exchange) RequestMethodRaw() []byte {
// RequestMethodBytes returns the request line's method as a []byte view, i.e: "GET".
func (exch *Exchange) RequestMethodBytes() []byte {
return exch.RequestHeaderV1Raw().Method()
}
+1 -1
View File
@@ -218,7 +218,7 @@ func TestHandleRequestFields(t *testing.T) {
var sm MuxSlice
route, _, _ := strings.Cut(test.wantURI, "?") // Mux matches on path.
sm.Handle(route, func(ex *Exchange) {
gotMethod = string(ex.RequestMethodRaw())
gotMethod = string(ex.RequestMethodBytes())
gotURI = string(ex.RequestTarget())
gotHost = string(ex.RequestHeader("Host"))
ex.WriteHeader(200)
+5 -4
View File
@@ -252,7 +252,7 @@ func (sm *MuxSlice) Reset(capacity int) {
// Every method this package does not name is [MethUnknown], so a request with an
// extension method matches a bare-path registration and any registration naming
// an extension method, whichever it names. Tell PROPFIND from MKCOL inside the
// handler with [Exchange.RequestMethodRaw].
// handler with [Exchange.RequestMethodBytes].
func (sm *MuxSlice) LookupHandler(method Method, path []byte, dstPathVals []PathValue) (matched string, _ HandlerFunc) {
best := -1
bestSpec := 0
@@ -423,9 +423,12 @@ func hasLowerASCII(s string) bool {
}
// Method is a HTTP request method, parsed by [MethodFrom].
// Method can only take standardized values and is set to [MethUnknown] for non-standard methods.
type Method uint8
const (
// MethUndefined returned by [MethodFrom] on an empty/missing method.
// Used by [MuxSlice] to denote an unset method kind for a request pattern.
MethUndefined Method = iota // undefined
MethGet // GET
// lol.
@@ -438,6 +441,7 @@ const (
MethConnect // CONNECT
MethOptions // OPTIONS
MethTrace // TRACE
// MethUnknown returned by [MethodFrom] on an non-standard method kind i.e: "get" and "FROBNICATE".
MethUnknown // unknown
)
@@ -475,9 +479,6 @@ func MethodFrom(meth string) (res Method) {
// MethodFromBytes is a [MethodFrom] wrapper with bytes argument instead of string.
func MethodFromBytes(meth []byte) (res Method) {
if len(meth) == 0 {
return MethUndefined
}
return MethodFrom(b2s(meth))
}
+4 -4
View File
@@ -183,7 +183,7 @@ func TestExchangePathValueClearedBetweenRequests(t *testing.T) {
exch := new(Exchange)
exch.Configure(ExchangeConfig{
RawBuf: make([]byte, 2048), RequestBufferLim: 1024,
NumHeaderKVCap: defaultNumHeaderKVCap, MaxPathValues: 4,
NumHeaderKVCap: defaultNumHeaderKVCap, PathValuesBuf: make([]PathValue, 4),
})
// First request binds id=42 off a wildcard pattern.
@@ -243,7 +243,7 @@ func TestMuxSliceNoStaleBindingsAcrossCandidates(t *testing.T) {
exch := new(Exchange)
exch.Configure(ExchangeConfig{
RawBuf: make([]byte, 2048), RequestBufferLim: 1024,
NumHeaderKVCap: defaultNumHeaderKVCap, MaxPathValues: sm.MaxPathValues(),
NumHeaderKVCap: defaultNumHeaderKVCap, PathValuesBuf: make([]PathValue, sm.MaxPathValues()),
})
conn := newConn("GET /a/1/b HTTP/1.1\r\nHost: h\r\n\r\n")
conn.Hangup()
@@ -292,7 +292,7 @@ func TestMuxSliceTrailingSlashPattern(t *testing.T) {
exch := new(Exchange)
exch.Configure(ExchangeConfig{
RawBuf: make([]byte, 2048), RequestBufferLim: 1024,
NumHeaderKVCap: defaultNumHeaderKVCap, MaxPathValues: sm.MaxPathValues(),
NumHeaderKVCap: defaultNumHeaderKVCap, PathValuesBuf: make([]PathValue, sm.MaxPathValues()),
})
conn := newConn("GET " + test.path + " HTTP/1.1\r\nHost: h\r\n\r\n")
conn.Hangup()
@@ -497,7 +497,7 @@ func TestMuxSliceZeroValueWildcardStillMatches(t *testing.T) {
exch := new(Exchange)
exch.Configure(ExchangeConfig{
RawBuf: make([]byte, 2048), RequestBufferLim: 1024,
NumHeaderKVCap: defaultNumHeaderKVCap, MaxPathValues: sm.MaxPathValues(),
NumHeaderKVCap: defaultNumHeaderKVCap, PathValuesBuf: make([]PathValue, sm.MaxPathValues()),
})
conn := newConn("GET " + test.path + " HTTP/1.1\r\nHost: h\r\n\r\n")
conn.Hangup()
+10 -27
View File
@@ -58,6 +58,7 @@ type Router struct {
mux Mux
globbuf []byte
globpath []PathValue
exchs []Exchange
freeList *Exchange
@@ -87,37 +88,19 @@ type RouterConfig struct {
// Required [>0] request header key/value pairs to parse before failing with
// [StatusRequestHeaderFieldsTooLarge].
RequestNumHeaderKVCap int
// Optional [any] normalization of response header field keys as they are
// staged, i.e: "content-type" becomes "Content-Type".
NormalizeOutgoingKeys bool
// Required [non-nil] resolver of each request's method and path to the handler
// serving it. Routes must be registered before Configure, see [Mux.MaxPathValues].
Mux Mux
// Optional [nil disables] sink for failed exchanges.
Logger *slog.Logger
}
const (
// minRequestHeaderBuffer is the smallest request buffer [httpraw.Header]
// accepts with buffer growth disabled, which is how exchanges are configured.
minRequestHeaderBuffer = 32
// minResponseHeaderBuffer is the room [Exchange.FlushHeader] needs for the
// CRLF closing the header block, written even when no field was staged.
minResponseHeaderBuffer = len("\r\n")
// maxExchangeBuffer bounds an exchange's whole buffer: [Exchange] indexes it
// with uint16 offsets, so a larger one would be addressed truncated.
maxExchangeBuffer = math.MaxUint16
)
// Validate returns a non-nil error if the configuration cannot be used to
// configure a [Router].
func (cfg RouterConfig) Validate() error {
workerMode := cfg.workerMode()
switch {
case cfg.Mux == nil,
!workerMode && cfg.FixedNumGoroutines != -1,
case !workerMode && cfg.FixedNumGoroutines != -1,
cfg.RequestNumHeaderKVCap <= 0,
cfg.RequestHeaderBufferSize < minRequestHeaderBuffer,
cfg.ResponseHeaderMinBufferSize < minResponseHeaderBuffer,
@@ -167,7 +150,7 @@ func (r *Router) shutdownLocked() {
// Configure may be called on a serving router, but since the exchange buffers
// are reused it waits for connections in flight to finish and fails with a
// non-nil error rather than reconfigure buffers still being served from.
func (r *Router) Configure(cfg RouterConfig) error {
func (r *Router) Configure(mux Mux, cfg RouterConfig) error {
if err := cfg.Validate(); err != nil {
return err
}
@@ -181,9 +164,9 @@ func (r *Router) Configure(cfg RouterConfig) error {
r.reqNumHeaderCap = cfg.RequestNumHeaderKVCap
r.reqBuf = cfg.RequestHeaderBufferSize
r.respBuf = cfg.ResponseHeaderMinBufferSize
r.mux = cfg.Mux
r.mux = mux
r.log = cfg.Logger
maxPathValues := cfg.Mux.MaxPathValues()
maxPathValues := mux.MaxPathValues()
if maxPathValues < 0 {
return errors.New("Mux paths must be registered before configuring Router")
}
@@ -211,20 +194,20 @@ func (r *Router) Configure(cfg RouterConfig) error {
r.exchs = r.exchs[:numgoro]
rawBuflen := cfg.RequestHeaderBufferSize + cfg.ResponseHeaderMinBufferSize
internal.SliceReuse(&r.globbuf, numgoro*rawBuflen)
internal.SliceReuse(&r.globpath, numgoro*maxPathValues)
for i := range numgoro {
// TODO exchange buffer alloc
goff := i * rawBuflen
// r.globbuf[goff:goff+rawBuflen], cfg.RequestHeaderBufferSize, cfg.RequestNumHeaderCap, cfg.NormalizeOutgoingKeys
poff := i * maxPathValues
r.exchs[i].Configure(ExchangeConfig{
RawBuf: r.globbuf[goff : goff+rawBuflen],
RequestBufferLim: cfg.RequestHeaderBufferSize,
NumHeaderKVCap: cfg.RequestNumHeaderKVCap,
NormalizeOutgoingKeys: cfg.NormalizeOutgoingKeys,
NoRequestBufferGrowth: true, // Hard memory limit.
MaxPathValues: maxPathValues,
PathValuesBuf: r.globpath[poff : poff+maxPathValues],
})
go r.goroWorker(gen, jobqueue, cfg.Mux)
go r.goroWorker(gen, jobqueue, mux)
}
r.pendingConns = jobqueue
r.numGoro = numgoro
@@ -388,7 +371,7 @@ func (r *Router) getExchLocked(conn conn) (exch *Exchange) {
NumHeaderKVCap: r.reqNumHeaderCap,
NormalizeOutgoingKeys: r.normalizeKeys,
NoRequestBufferGrowth: true,
MaxPathValues: r.maxPathValues,
PathValuesBuf: make([]PathValue, r.maxPathValues),
})
exch.Acquire(conn) // Fresh exchange, CAS cannot fail.
return exch
+6 -10
View File
@@ -150,9 +150,8 @@ func (r *rwconn) ViewWritten() string {
var _ Mux = (*MuxSlice)(nil)
func configSynchronousRouter(t *testing.T, router *Router, bufferSize int, mux Mux) {
err := router.Configure(RouterConfig{
err := router.Configure(mux, RouterConfig{
FixedNumGoroutines: -1,
Mux: mux,
RequestHeaderBufferSize: bufferSize,
RequestNumHeaderKVCap: 16,
ResponseHeaderMinBufferSize: bufferSize,
@@ -198,7 +197,7 @@ func TestRouterRequestVisibleToHandler(t *testing.T) {
var gotMethod, gotURI, gotHost string
var gotMethodEnum Method
sm.Handle("GET /index.html", func(ex *Exchange) {
gotMethod = string(ex.RequestMethodRaw())
gotMethod = string(ex.RequestMethodBytes())
gotMethodEnum = ex.RequestMethod()
gotURI = string(ex.RequestTarget())
gotHost = string(ex.RequestHeader("Host"))
@@ -388,9 +387,8 @@ func TestRouterHandleAfterTeardown(t *testing.T) {
router Router
)
sm.Handle("GET /", staticPage(t, "ok"))
err := router.Configure(RouterConfig{
err := router.Configure(&sm, RouterConfig{
FixedNumGoroutines: 2,
Mux: &sm,
RequestHeaderBufferSize: 512,
RequestNumHeaderKVCap: 16,
ResponseHeaderMinBufferSize: 512,
@@ -422,7 +420,6 @@ func TestRouterTeardownReleasesQueuedConns(t *testing.T) {
sm.Handle("GET /", staticPage(t, "ok"))
cfg := RouterConfig{
FixedNumGoroutines: numGoro,
Mux: &sm,
RequestHeaderBufferSize: 512,
RequestNumHeaderKVCap: 16,
ResponseHeaderMinBufferSize: 512,
@@ -436,7 +433,7 @@ func TestRouterTeardownReleasesQueuedConns(t *testing.T) {
// generation drops its connections, but it must not outlive it.
var err error
for range 100 {
if err = router.Configure(cfg); err == nil {
if err = router.Configure(&sm, cfg); err == nil {
break
}
time.Sleep(time.Millisecond)
@@ -468,12 +465,11 @@ func TestRouterConfigureDuringWorkerHandle(t *testing.T) {
sm.Handle("GET /", staticPage(t, "ok"))
cfg := RouterConfig{
FixedNumGoroutines: 2,
Mux: &sm,
RequestHeaderBufferSize: 512,
RequestNumHeaderKVCap: 16,
ResponseHeaderMinBufferSize: 512,
}
if err := router.Configure(cfg); err != nil {
if err := router.Configure(&sm, cfg); err != nil {
t.Fatal(err)
}
defer router.Shutdown()
@@ -495,7 +491,7 @@ func TestRouterConfigureDuringWorkerHandle(t *testing.T) {
for range 20 {
// errBusyExchanges is legitimate backpressure: the previous
// generation was still serving when the buffers were needed.
if err := router.Configure(cfg); err != nil && err != errBusyExchanges {
if err := router.Configure(&sm, cfg); err != nil && err != errBusyExchanges {
t.Error(err)
return
}
+122
View File
@@ -0,0 +1,122 @@
package httphi
import (
"math"
"unsafe"
"github.com/soypat/lneto/http/httpraw"
)
const (
// minRequestHeaderBuffer is the smallest request buffer [httpraw.Header]
// accepts with buffer growth disabled, which is how exchanges are configured.
minRequestHeaderBuffer = 32
// minResponseHeaderBuffer is the room [Exchange.FlushHeader] needs for the
// CRLF closing the header block, written even when no field was staged.
minResponseHeaderBuffer = len("\r\n")
// maxExchangeBuffer bounds an exchange's whole buffer: [Exchange] indexes it
// with uint16 offsets, so a larger one would be addressed truncated.
maxExchangeBuffer = math.MaxUint16
// sizeofExchange is the fixed cost of an exchange, which a Router pays per
// connection it can serve concurrently on top of the buffers it hands it.
// It dwarfs a small request buffer, so budgets must account for it.
sizeofExchange = int(unsafe.Sizeof(Exchange{}))
// sizeofPathValue is the per-wildcard cost of the path value table.
sizeofPathValue = int(unsafe.Sizeof(PathValue{}))
// sizeofJob is an exchange's slot in the queue connections wait on for a
// worker goroutine, sized to the goroutine count in worker mode.
sizeofJob = int(unsafe.Sizeof(job{}))
// bytesPerHeaderField is the request buffer [DefaultRouterConfig] budgets per
// parseable header field. Real fields run a little longer than this
// ("Accept-Encoding: gzip, deflate, br\r\n" is 35 bytes), so a request fills
// the buffer before it exhausts the field table, which is the cheaper of the
// two limits to hit: growing the table costs [httpraw.SizeKV] per field on top
// of the bytes the field already occupies.
bytesPerHeaderField = 32
// defaultResponseHeaderBuffer is the response header room
// [DefaultRouterConfig] reserves when the budget can afford it: enough for a
// Content-Type, a Content-Length and a Connection field with room to spare.
// It does not scale with the request buffer because what a response header
// costs depends on the fields a handler stages, not on the request's size.
defaultResponseHeaderBuffer = 128
)
// MemoryUsagePerConnection returns the heap bytes a [Router] configured with cfg
// reserves for each connection it can serve concurrently, maxPathValues being
// the [Mux.MaxPathValues] of the mux it is configured with. Goroutine stacks are
// not counted: those are the runtime's to size, not the router's.
//
// In worker mode this is exact and fixed, so a router's whole heap footprint is
// this times FixedNumGoroutines, plus the runtime's own header for the job
// queue. With FixedNumGoroutines -1 the router allocates one of these per
// connection in flight instead, so the total grows with peak concurrency.
//
// It is the inverse of [DefaultRouterConfig] and useful to check a hand written
// configuration against a memory budget.
func (cfg RouterConfig) MemoryUsagePerConnection(maxPathValues int) int {
if maxPathValues < 0 {
maxPathValues = 0 // Mux with no routes registered yet, see [Mux.MaxPathValues].
}
n := sizeofExchange + // Exchange itself, an element of the router's exchange store.
cfg.RequestHeaderBufferSize + cfg.ResponseHeaderMinBufferSize + // Its window into the raw buffer.
cfg.RequestNumHeaderKVCap*httpraw.SizeKV + // The request header's field table.
maxPathValues*sizeofPathValue // Its window into the path value store.
if cfg.workerMode() {
n += sizeofJob // Slot in the queue connections wait on for a worker.
}
return n
}
// DefaultRouterConfig is a general purpose configuration creator
// for small, medium, large, performant or embedded projects.
//
// Generalness is achieved with parameters that let the Configuration
// determine allocation buffer sizes based on typical usage for that
// number of goroutines and heap allocation on a per-connection basis.
func DefaultRouterConfig(numGoroutines, memoryPerConnectionBytes, maxPathValues int) RouterConfig {
// The budget is spent on the request header buffer first, since that is what
// decides which requests are answered at all, then on a field table sized to
// match it and a small response header reserve. A budget too small to fund the
// minimum viable exchange yields the minimum instead, so the returned config is
// always one [Router.Configure] accepts but may exceed a budget under roughly
// sizeofExchange + 200 bytes. Check it with MemoryUsagePerConnection when the
// bound has to hold.
if numGoroutines <= 0 {
numGoroutines = -1 // Unbounded mode, the only non-positive value Validate accepts.
}
// Everything the exchange costs before any buffer is sized: subtract it first
// so the buffers below divide up what is actually left to spend.
fixed := sizeofExchange + maxPathValues*sizeofPathValue
if numGoroutines > 0 {
fixed += sizeofJob
}
// A budget past what the buffers may grow to is only spendable up to the cap
// below, so clamp before the products: on a 32 bit target an unclamped
// multiply would overflow and wrap a generous budget into a tiny buffer.
const maxSpendable = (maxExchangeBuffer + defaultResponseHeaderBuffer) *
(bytesPerHeaderField + httpraw.SizeKV) / bytesPerHeaderField
avail := min(memoryPerConnectionBytes-fixed, maxSpendable)
// The response reserve is a floor rather than a share of the budget, but a
// budget this small cannot afford the full one without starving the request.
respBuf := min(defaultResponseHeaderBuffer, avail/4)
// Solve avail-respBuf = reqBuf + reqBuf/bytesPerHeaderField*httpraw.SizeKV for
// reqBuf, the field table growing with the buffer it parses.
reqBuf := (avail - respBuf) * bytesPerHeaderField / (bytesPerHeaderField + httpraw.SizeKV)
// Clamp to what [RouterConfig.Validate] accepts. Truncating division above
// keeps the result under budget; these floors are what can push it over.
respBuf = max(respBuf, minResponseHeaderBuffer)
reqBuf = max(reqBuf, minRequestHeaderBuffer)
reqBuf = min(reqBuf, maxExchangeBuffer-respBuf)
return RouterConfig{
FixedNumGoroutines: numGoroutines,
RequestHeaderBufferSize: reqBuf,
ResponseHeaderMinBufferSize: respBuf,
RequestNumHeaderKVCap: max(reqBuf/bytesPerHeaderField, 1),
NormalizeOutgoingKeys: false,
}
}
+231
View File
@@ -0,0 +1,231 @@
package httphi
import (
"testing"
"github.com/soypat/lneto/http/httpraw"
)
// budgetMux exposes a settable path value count so budget tests can sweep it
// without registering patterns that bind that many wildcards.
type budgetMux struct {
MuxSlice
maxPathValues int
}
func (m *budgetMux) MaxPathValues() int { return m.maxPathValues }
func newBudgetMux(maxPathValues int) *budgetMux {
mux := &budgetMux{maxPathValues: maxPathValues}
mux.Handle("GET /", func(*Exchange) {})
return mux
}
// TestDefaultRouterConfigHonorsBudget sweeps budgets and path value counts and
// checks the returned configuration both fits its budget and configures a
// router. The floor is documented: below it the minimum viable exchange comes
// back instead, which is the only case allowed to exceed the budget.
func TestDefaultRouterConfigHonorsBudget(t *testing.T) {
minCfg := RouterConfig{
FixedNumGoroutines: 1,
RequestHeaderBufferSize: minRequestHeaderBuffer,
ResponseHeaderMinBufferSize: minResponseHeaderBuffer,
RequestNumHeaderKVCap: 1,
}
for _, numGoro := range []int{-1, 1, 4} {
for _, maxPathValues := range []int{0, 1, 4, 32} {
floor := minCfg.MemoryUsagePerConnection(maxPathValues)
mux := newBudgetMux(maxPathValues)
for _, budget := range []int{0, 1, 64, 256, 512, 1024, 4096, 65536, 1 << 20} {
cfg := DefaultRouterConfig(numGoro, budget, maxPathValues)
if err := cfg.Validate(); err != nil {
t.Fatalf("goro=%d pathvals=%d budget=%d: %s", numGoro, maxPathValues, budget, err)
}
got := cfg.MemoryUsagePerConnection(maxPathValues)
if got > budget && budget >= floor {
t.Errorf("goro=%d pathvals=%d budget=%d: uses %d bytes, over budget",
numGoro, maxPathValues, budget, got)
}
var router Router
if err := router.Configure(mux, cfg); err != nil {
t.Fatalf("goro=%d pathvals=%d budget=%d: %s", numGoro, maxPathValues, budget, err)
}
router.Shutdown()
}
}
}
}
// TestDefaultRouterConfigSpendsBudget guards the other direction: a config that
// fits but leaves most of the budget unspent is as wrong as one that overruns,
// since the memory is reserved either way.
func TestDefaultRouterConfigSpendsBudget(t *testing.T) {
const maxPathValues = 2
for _, budget := range []int{1024, 2048, 4096, 16384} {
cfg := DefaultRouterConfig(4, budget, maxPathValues)
used := cfg.MemoryUsagePerConnection(maxPathValues)
if pct := used * 100 / budget; pct < 95 {
t.Errorf("budget=%d: spends only %d bytes (%d%%)", budget, used, pct)
}
}
}
// TestExchangeMemoryTerms pins each term of MemoryUsagePerConnection to the
// allocation it stands for, so a layout change downstream fails here rather than
// silently letting a router overrun its budget.
func TestExchangeMemoryTerms(t *testing.T) {
const maxPathValues = 4
cfg := RouterConfig{
FixedNumGoroutines: 2,
RequestHeaderBufferSize: 512,
ResponseHeaderMinBufferSize: 128,
RequestNumHeaderKVCap: 16,
}
want := sizeofExchange + 512 + 128 + 16*httpraw.SizeKV + maxPathValues*sizeofPathValue + sizeofJob
if got := cfg.MemoryUsagePerConnection(maxPathValues); got != want {
t.Errorf("worker mode: got %d want %d", got, want)
}
// Unbounded mode has no job queue to reserve a slot in.
cfg.FixedNumGoroutines = -1
if got := cfg.MemoryUsagePerConnection(maxPathValues); got != want-sizeofJob {
t.Errorf("unbounded mode: got %d want %d", got, want-sizeofJob)
}
// A mux with no routes registered reports -1, which must not subtract memory.
if got := cfg.MemoryUsagePerConnection(-1); got != cfg.MemoryUsagePerConnection(0) {
t.Errorf("unregistered mux: got %d want %d", got, cfg.MemoryUsagePerConnection(0))
}
}
// TestRouterSharesExchangeStores checks the invariant the memory accounting
// rests on: every exchange's buffer and path values are windows into the two
// stores the router allocates, non-overlapping and exactly the configured size.
// Measuring allocations would only observe this indirectly.
func TestRouterSharesExchangeStores(t *testing.T) {
const numGoro = 8
const maxPathValues = 3
mux := newBudgetMux(maxPathValues)
cfg := DefaultRouterConfig(numGoro, 1024, maxPathValues)
var router Router
if err := router.Configure(mux, cfg); err != nil {
t.Fatal(err)
}
defer router.Shutdown()
wantRaw := cfg.RequestHeaderBufferSize + cfg.ResponseHeaderMinBufferSize
if len(router.exchs) != numGoro {
t.Fatalf("got %d exchanges, want %d", len(router.exchs), numGoro)
}
if cap(router.globbuf) < numGoro*wantRaw {
t.Errorf("raw store holds %d bytes, want %d", cap(router.globbuf), numGoro*wantRaw)
}
if cap(router.globpath) < numGoro*maxPathValues {
t.Errorf("path store holds %d values, want %d", cap(router.globpath), numGoro*maxPathValues)
}
rawSeen := make(map[*byte]int, numGoro*wantRaw)
pathSeen := make(map[*PathValue]int, numGoro*maxPathValues)
for i := range router.exchs {
exch := &router.exchs[i]
if len(exch.rawbuf) != wantRaw {
t.Errorf("exchange %d: raw buffer is %d bytes, want %d", i, len(exch.rawbuf), wantRaw)
}
if len(exch.pathValues) != maxPathValues {
t.Errorf("exchange %d: %d path values, want %d", i, len(exch.pathValues), maxPathValues)
}
// Every byte must come from the shared store and belong to this exchange
// alone: an exchange allocating its own, or two sharing a window, would
// make the per-connection accounting a fiction.
for j := range exch.rawbuf {
p := &exch.rawbuf[j]
if owner, dup := rawSeen[p]; dup {
t.Fatalf("exchanges %d and %d share raw buffer byte %d", owner, i, j)
}
rawSeen[p] = i
}
for j := range exch.pathValues {
p := &exch.pathValues[j]
if owner, dup := pathSeen[p]; dup {
t.Fatalf("exchanges %d and %d share path value %d", owner, i, j)
}
pathSeen[p] = i
}
}
if len(rawSeen) != numGoro*wantRaw {
t.Errorf("exchanges cover %d raw bytes, want %d", len(rawSeen), numGoro*wantRaw)
}
}
// TestExchangeConfigureIsAllocationFree checks an exchange handed all of its
// memory allocates none of its own, which is what lets a router carve every
// exchange out of its two stores.
func TestExchangeConfigureIsAllocationFree(t *testing.T) {
cfg := ExchangeConfig{
RawBuf: make([]byte, 640),
RequestBufferLim: 512,
NumHeaderKVCap: 16,
NoRequestBufferGrowth: true,
PathValuesBuf: make([]PathValue, 4),
}
var exch Exchange
exch.Configure(cfg) // Field table allocates once, then settles.
allocs := testing.AllocsPerRun(100, func() {
exch.Configure(cfg)
})
if allocs != 0 {
t.Errorf("Exchange.Configure allocates %v times, want 0", allocs)
}
}
// TestMemoryUsagePerConnectionMatchesHeap checks the accounting against the heap
// a router actually takes, which is what makes the number worth budgeting
// against.
//
// It measures two budgets and compares the difference rather than either
// absolute figure. A router's heap carries costs the accounting does not claim
// and should not: size class rounding, the job queue's runtime header and the
// runtime's per-goroutine bookkeeping. Those are identical at both budgets, so
// subtracting cancels them and leaves only the buffers, whose growth is exactly
// what MemoryUsagePerConnection predicts. Goroutine stacks never enter into it,
// the runtime accounting them separately from the heap measured here.
func TestMemoryUsagePerConnectionMatchesHeap(t *testing.T) {
if testing.Short() {
t.Skip("measures heap over many Configure iterations")
}
const numGoro = 64
const maxPathValues = 3
mux := newBudgetMux(maxPathValues)
measure := func(budget int) (accounted, heap int) {
cfg := DefaultRouterConfig(numGoro, budget, maxPathValues)
res := testing.Benchmark(func(b *testing.B) {
b.ReportAllocs()
for range b.N {
var router Router
if err := router.Configure(mux, cfg); err != nil {
b.Fatal(err)
}
router.Shutdown()
}
})
return numGoro * cfg.MemoryUsagePerConnection(maxPathValues), int(res.AllocedBytesPerOp())
}
lowAcct, lowHeap := measure(2048)
highAcct, highHeap := measure(16384)
wantGrowth := highAcct - lowAcct
gotGrowth := highHeap - lowHeap
t.Logf("accounted %d->%d (+%d), heap %d->%d (+%d)",
lowAcct, highAcct, wantGrowth, lowHeap, highHeap, gotGrowth)
// What remains after cancelling is buffer growth, which the accounting covers
// term for term. Only size class rounding on the grown buffers is left over.
const tolerancePercent = 2
if diff := abs(gotGrowth - wantGrowth); diff*100 > wantGrowth*tolerancePercent {
t.Errorf("budget growth accounted %d bytes, heap grew %d (%+d)",
wantGrowth, gotGrowth, gotGrowth-wantGrowth)
}
}
func abs(x int) int {
if x < 0 {
return -x
}
return x
}
+1 -5
View File
@@ -3,7 +3,7 @@ package httphi
// StatusText returns a text for the HTTP status code. It returns the empty
// string if the code is unknown.
func StatusText(code int) string {
switch status(code) {
switch code {
case StatusContinue:
return "Continue"
case StatusSwitchingProtocols:
@@ -133,10 +133,6 @@ func StatusText(code int) string {
}
}
const ()
type status int
// HTTP status codes as registered with IANA.
// See: https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml
const (
+7 -19
View File
@@ -300,25 +300,7 @@ func (kvb *kvBuffer) getFoldIdx(key string) int {
// EqualFoldASCII reports whether a and b are equal under ASCII case folding.
// Unlike strings.EqualFold it does not fold non-ASCII runes, so no multi-byte
// rune such as U+212A KELVIN SIGN can alias a header key.
func EqualFoldASCII(a, b string) bool {
if len(a) != len(b) {
return false
}
const asciiCapDiff = 'a' - 'A'
for i := 0; i < len(a); i++ {
ca, cb := a[i], b[i]
if ca >= 'A' && ca <= 'Z' {
ca += asciiCapDiff
}
if cb >= 'A' && cb <= 'Z' {
cb += asciiCapDiff
}
if ca != cb {
return false
}
}
return true
}
func EqualFoldASCII(a, b string) bool { return internal.EqualFoldASCII(a, b) }
// reserve ensures need free bytes are available in the buffer, growing it when
// permitted. It accounts for the byte-0 reservation on an empty buffer (see
@@ -462,6 +444,12 @@ type pairKV struct {
value view // value start >0 means value is present.
}
// SizeKV is the heap cost of a single key/value field slot, as reserved by the
// numHeaderCapacity argument to [HeaderV1.Reset] and by [Form.Reset]. Callers
// budgeting a fixed memory pool up front, such as a Router sizing its
// exchanges, multiply it by the pair capacity to account the field table.
const SizeKV = int(unsafe.Sizeof(pairKV{}))
// isValid is for stores parsed in place, where offset 0 is the first key so
// only length can signal presence. Empty keys are valid: see valueless cookies.
func (pair pairKV) isValid() bool {
+2 -15
View File
@@ -253,20 +253,7 @@ func trimOWS(b []byte) []byte {
return b
}
// equalFold compares b to the ASCII lowercase key, case insensitively.
// equalFold compares b to key, case insensitively.
func equalFold(b []byte, key string) bool {
if len(b) != len(key) {
return false
}
const asciiCapDiff = 'a' - 'A'
for i := range b {
c := b[i]
if c >= 'A' && c <= 'Z' {
c += asciiCapDiff
}
if c != key[i] {
return false
}
}
return true
return EqualFoldASCII(b2s(b), key)
}
+1 -1
View File
@@ -18,7 +18,7 @@ var (
)
func LogEnabled(l *slog.Logger, lvl slog.Level) bool {
return true
return logEnabled
}
func logAttrsAndAllocs(allocmsg string, l *slog.Logger, level slog.Level, msg string, attrs ...slog.Attr) {
+2 -2
View File
@@ -10,14 +10,14 @@ import (
const HeapAllocDebugging = false
func LogEnabled(l *slog.Logger, lvl slog.Level) bool {
return l != nil && l.Handler().Enabled(context.Background(), lvl)
return logEnabled && l != nil && l.Handler().Enabled(context.Background(), lvl)
}
// LogAttrs is a helper function that is used by all package loggers and that
// can be switched out with the `debugheaplog` build tag for a non-allocating
// logger that prints out when heap allocations occur.
func LogAttrs(l *slog.Logger, level slog.Level, msg string, attrs ...slog.Attr) {
if l != nil {
if logEnabled && l != nil {
l.LogAttrs(context.Background(), level, msg, attrs...)
}
}
+5
View File
@@ -0,0 +1,5 @@
//go:build noslog
package internal
const logEnabled = false
+5
View File
@@ -0,0 +1,5 @@
//go:build !noslog
package internal
const logEnabled = true
+14 -2
View File
@@ -157,7 +157,7 @@ func (r *Ring) ReadDiscard(n int) error {
case n > buffered:
return errDiscardExceeds
case n == buffered:
r.Reset()
r.emptied()
case n+r.Off > len(r.Buf):
r.Off = n - (len(r.Buf) - r.Off)
default:
@@ -224,6 +224,18 @@ func (r *Ring) Reset() {
r.End = 0
}
// emptied marks the ring empty keeping the write position where it is, unlike
// [Ring.Reset] which rewinds it to index 0. Bytes staged past that position with
// [Ring.PeekWrite] are addressed relative to it, so moving it makes a later
// [Ring.Commit] hand back the wrong bytes.
func (r *Ring) emptied() {
off := r.End
if off == len(r.Buf) {
off = 0 // Tail exhausted, next write wraps.
}
r.Off, r.End = off, 0
}
// Size returns the capacity of the ring buffer.
func (r *Ring) Size() int {
return len(r.Buf)
@@ -306,7 +318,7 @@ func (r *Ring) onReadEnd(totalRead int) {
}
newOff := r.addOff(r.Off, totalRead)
if newOff == r.End {
r.Reset()
r.emptied()
} else if newOff == len(r.Buf) {
r.Off = 0 // Optimization case.
} else {
+37
View File
@@ -661,3 +661,40 @@ func TestRingPeekWriteRejects(t *testing.T) {
t.Error("Commit beyond free must error")
}
}
// TestRingPeekWriteSurvivesEmptyRead checks bytes staged with [Ring.PeekWrite]
// survive the ring being read empty, an event the stager does not control.
func TestRingPeekWriteSurvivesEmptyRead(t *testing.T) {
r := &Ring{Buf: make([]byte, 16)}
if _, err := r.Write([]byte("AAAA")); err != nil {
t.Fatal("first write:", err)
}
// Stage "CCCC" one 4-byte gap past the write position.
if !r.PeekWrite([]byte("CCCC"), 4) {
t.Fatal("PeekWrite should fit")
}
// Drain everything readable: ring goes empty, staged bytes still pending.
got := make([]byte, 16)
n, err := r.Read(got)
if err != nil || string(got[:n]) != "AAAA" {
t.Fatalf("drain read %q (%v), want AAAA", got[:n], err)
}
if !r.IsEmpty() {
t.Fatal("ring should be empty after draining")
}
// Fill the gap and commit the staged tail.
if _, err := r.Write([]byte("BBBB")); err != nil {
t.Fatal("gap write:", err)
}
if err := r.Commit(4); err != nil {
t.Fatal("commit:", err)
}
n, err = r.Read(got)
if err != nil {
t.Fatal("read:", err)
}
if string(got[:n]) != "BBBBCCCC" {
t.Fatalf("read %q, want BBBBCCCC: the staged bytes were committed from the wrong offset", got[:n])
}
testRingSanity(t, r)
}
+34
View File
@@ -14,6 +14,40 @@ func BytesEqual(a, b []byte) bool {
return unsafe.String(&a[0], len(a)) == unsafe.String(&b[0], len(b))
}
// EqualFoldASCII reports whether a and b are equal under ASCII case folding.
// Unlike [strings.EqualFold] it does not fold non-ASCII runes, so no multi-byte
// rune such as U+212A KELVIN SIGN can alias an ASCII key.
func EqualFoldASCII(a, b string) bool {
if len(a) != len(b) {
return false
}
const asciiCapDiff = 'a' - 'A'
for i := 0; i < len(a); i++ {
ca, cb := a[i], b[i]
if ca >= 'A' && ca <= 'Z' {
ca += asciiCapDiff
}
if cb >= 'A' && cb <= 'Z' {
cb += asciiCapDiff
}
if ca != cb {
return false
}
}
return true
}
// BytesEqualFoldASCII is the []byte form of [EqualFoldASCII]. Like [BytesEqual]
// it is heapless in tinygo, unlike [bytes.EqualFold] which also folds non-ASCII.
func BytesEqualFoldASCII(a, b []byte) bool {
if len(a) != len(b) {
return false
} else if len(a) == 0 {
return true
}
return EqualFoldASCII(unsafe.String(&a[0], len(a)), unsafe.String(&b[0], len(b)))
}
// IsZeroed returns true if all arguments are set to their zero value.
func IsZeroed[T comparable](a ...T) bool {
var z T
+10
View File
@@ -1,5 +1,15 @@
package internal
import "strconv"
// AppendStrDecimal appends pfx followed by value in base 10 to dst and returns
// the resulting slice. It condenses the prefixed-number pattern common to
// AppendString/AppendText methods, i.e. `internal.AppendStrDecimal(b, " len=", 4)`.
func AppendStrDecimal(dst []byte, pfx string, value int64) []byte {
dst = append(dst, pfx...)
return strconv.AppendInt(dst, value, 10)
}
// IntLen returns the number of bytes [strconv.AppendInt] emits for value in the
// given base, including a leading minus sign for negatives. Lets callers size a
// buffer, or test whether a value fits an existing slot, before writing a byte.
+3 -6
View File
@@ -987,16 +987,13 @@ func (frm Frame) AppendString(b []byte) []byte {
bitlen := frm.LenBits()
b = append(b, frm.Protocol...)
if bitlen%8 == 0 {
b = append(b, " len="...)
b = strconv.AppendInt(b, int64(bitlen/8), 10)
b = internal.AppendStrDecimal(b, " len=", int64(bitlen/8))
} else {
b = append(b, " bits="...)
b = strconv.AppendInt(b, int64(bitlen), 10)
b = internal.AppendStrDecimal(b, " bits=", int64(bitlen))
}
iopt, err := frm.FieldByClass(FieldClassOptions)
if err == nil {
b = append(b, " optlen="...)
b = strconv.AppendInt(b, int64((frm.Fields[iopt].BitLength+7)/8), 10)
b = internal.AppendStrDecimal(b, " optlen=", int64((frm.Fields[iopt].BitLength+7)/8))
}
for _, err := range frm.Errors {
b = append(b, ' ')
+6 -12
View File
@@ -2,8 +2,6 @@ package ipv4
import (
"encoding/binary"
"fmt"
"net/netip"
"github.com/soypat/lneto"
)
@@ -238,14 +236,10 @@ func (ifrm Frame) ValidateExceptCRC(v *lneto.Validator) {
}
func (ifrm Frame) String() string {
dst := netip.AddrFrom4(*ifrm.DestinationAddr())
src := netip.AddrFrom4(*ifrm.SourceAddr())
hl := ifrm.HeaderLength()
tl := int(ifrm.TotalLength())
ttl := ifrm.TTL()
id := ifrm.ID()
proto := ifrm.Protocol()
tos := ifrm.ToS()
return fmt.Sprintf("IP %s SRC=%s DST=%s LEN=%d OPT=%d TTL=%d ID=%d ToS=0x%x", proto.String(), src.String(), dst.String(), tl, tl-hl, ttl, id, tos)
proto := ifrm.Protocol().String()
b := make([]byte, 0, 5+len(proto))
b = append(b, "IP ("...)
b = append(b, proto...)
b = append(b, ')')
return string(b)
}
+8 -5
View File
@@ -279,10 +279,9 @@ func (tcb *ControlBlock) PendingSegment(payloadLen int) (_ Segment, ok bool) {
// Optimist Strategy: retransmit oldest data once.
return Segment{SEQ: tcb.snd.UNA, DATALEN: Size(payloadLen), ACK: tcb.rcv.NXT, WND: tcb.rcv.WND, Flags: FlagACK}, true
}
established := tcb._state == StateEstablished
canSendData := established || tcb._state == StateCloseWait
canSendData := tcb._state.txQueuedDataOpen()
if !canSendData {
payloadLen = 0 // Can't send data if not established or close-wait.
payloadLen = 0 // No send-buffer data may go out in this state.
}
if pending == 0 && payloadLen == 0 {
return Segment{}, false // No pending segment.
@@ -522,8 +521,12 @@ func (tcb *ControlBlock) validateOutgoingSegment(seg Segment) (err error) {
err = errSeqNotInWindow
}
case seg.DATALEN > 0 && (tcb._state == StateFinWait1 || tcb._state == StateFinWait2):
err = errConnectionClosing // Case 1: No further SENDs from the user will be accepted by the TCP implementation.
case seg.DATALEN > 0 && tcb._state == StateFinWait2:
// FIN-WAIT-2 means our FIN was acknowledged, so no data below it can be
// unacknowledged and data here is a caller error. FIN-WAIT-1 is excluded:
// its FIN sits above data the peer may still be missing, which must go out
// for either side to make progress (RFC 9293 §3.10.8).
err = errConnectionClosing
case checkSeq && tcb.snd.WND == 0 && seg.DATALEN > 0 && seg.SEQ == tcb.snd.NXT:
err = errZeroWindow
+21 -4
View File
@@ -2,12 +2,12 @@ package tcp
import (
"errors"
"fmt"
"math/bits"
"strconv"
"unsafe"
"github.com/soypat/lneto"
"github.com/soypat/lneto/internal"
)
//go:generate stringer -type=State,OptionKind -linecomment -output stringers.go .
@@ -73,10 +73,19 @@ func (seg Segment) isFirstSYN() bool {
}
func (seg Segment) String() string {
if seg.DATALEN == 0 {
return fmt.Sprintf("SEG %s ACK=%d SEQ=%d WND=%d", seg.Flags, seg.ACK, seg.SEQ, seg.WND)
return string(seg.AppendString(nil))
}
func (seg Segment) AppendString(b []byte) []byte {
b = append(b, "SEG "...)
b = append(b, seg.Flags.String()...)
b = internal.AppendStrDecimal(b, " ACK=", int64(seg.ACK))
b = internal.AppendStrDecimal(b, " SEQ=", int64(seg.SEQ))
b = internal.AppendStrDecimal(b, " WND=", int64(seg.WND))
if seg.DATALEN > 0 {
b = internal.AppendStrDecimal(b, " DATALEN=", int64(seg.DATALEN))
}
return fmt.Sprintf("SEG %s ACK=%d SEQ=%d WND=%d DATALEN=%d", seg.Flags, seg.ACK, seg.SEQ, seg.WND, seg.DATALEN)
return b
}
// ClientSynSegment is a the first packet sent over a TCP connection to a server. Typically the client
@@ -329,6 +338,14 @@ func (s State) TxDataOpen() bool {
return s == StateEstablished || s == StateCloseWait
}
// txQueuedDataOpen returns true if already-queued send-buffer data may still be
// put on the wire. It stays true after a local close, where the FIN occupies a
// sequence above data the peer has not acknowledged: until that data is
// (re)transmitted the peer cannot reach the FIN. RFC 9293 §3.10.8.
func (s State) txQueuedDataOpen() bool {
return s.TxDataOpen() || s == StateFinWait1 || s == StateClosing || s == StateLastAck
}
// RxDataOpen returns true if the state allows the receiving of incoming data segments.
// Combine with [State.IsPreestablished] to know whether there is no more data to be received over the network.
func (s State) RxDataOpen() bool {
+7 -2
View File
@@ -2,10 +2,10 @@ package tcp
import (
"encoding/binary"
"fmt"
"math"
"github.com/soypat/lneto"
"github.com/soypat/lneto/internal"
)
const (
@@ -170,7 +170,12 @@ func (tfrm Frame) String() string {
src := tfrm.SourcePort()
dst := tfrm.DestinationPort()
seg := tfrm.Segment(len(tfrm.Payload()))
return fmt.Sprintf("TCP :%d -> :%d %s", src, dst, seg.String())
b := make([]byte, 0, 64)
b = append(b, "TCP "...)
b = internal.AppendStrDecimal(b, " src=", int64(src))
b = internal.AppendStrDecimal(b, " dst=", int64(dst))
b = append(b, ' ')
return string(seg.AppendString(b))
}
//
+122
View File
@@ -0,0 +1,122 @@
package tcp
import (
"math/rand"
"strconv"
"testing"
"github.com/soypat/lneto/ethernet"
)
// TestHandlerStreamIntegrityUnderReorder asserts byte identity of a reassembled
// stream whose segments arrive out of order: arrival order is randomised within
// each block of shuffleWindow segments, and nothing is lost or retransmitted, so
// several segments sit staged in the receive ring at once. Reordering may cost
// throughput; it may not change the bytes.
func TestHandlerStreamIntegrityUnderReorder(t *testing.T) {
const (
mtu = ethernet.MaxMTU
maxpackets = 8
segSize = 100
nsegs = 8 // per round; 800 bytes through a 1500-byte ring
rounds = 40 // enough for the ring to wrap many times
shuffleWindow = 4 // segments that may arrive in any order among themselves
)
rng := rand.New(rand.NewSource(3))
client, server := newHandler(t, mtu, maxpackets), newHandler(t, mtu, maxpackets)
setupClientServer(t, rng, client, server)
var rawbuf [mtu]byte
establish(t, client, server, rawbuf[:])
var want, got []byte
rb := make([]byte, mtu)
letter := byte('A')
for round := range rounds {
// Capture this round's segments on the wire, one segment per write.
segs := make([][]byte, 0, nsegs)
for range nsegs {
payload := make([]byte, segSize)
for j := range payload {
payload[j] = letter
}
letter++
if letter > 'Z' {
letter = 'A'
}
if n, err := client.Write(payload); err != nil || n != segSize {
t.Fatalf("round %d: client write: %d %v", round, n, err)
}
clear(rawbuf[:])
n, err := client.Send(rawbuf[:])
if err != nil {
t.Fatalf("round %d: client send: %v", round, err)
}
segs = append(segs, append([]byte(nil), rawbuf[:n]...))
want = append(want, payload...)
}
order := make([]int, 0, nsegs)
for i := 0; i < nsegs; i += shuffleWindow {
block := make([]int, 0, shuffleWindow)
for j := i; j < min(i+shuffleWindow, nsegs); j++ {
block = append(block, j)
}
rng.Shuffle(len(block), func(a, b int) { block[a], block[b] = block[b], block[a] })
order = append(order, block...)
}
for _, idx := range order {
if err := server.Recv(append([]byte(nil), segs[idx]...)); err != nil {
t.Logf("round %d segment %d refused: %v", round, idx, err)
}
// Drain as an application would, keeping the ring from filling.
for {
n, err := server.Read(rb)
if n > 0 {
got = append(got, rb[:n]...)
}
if n == 0 || err != nil {
break
}
}
// Feed ACKs back so the sender's window keeps opening; without this
// the test stalls on flow control instead of exercising reassembly.
clear(rawbuf[:])
if n, err := server.Send(rawbuf[:]); err == nil && n > 0 {
client.Recv(rawbuf[:n])
}
}
if string(got) != string(want) {
// Report the first divergence; later rounds only add noise.
i := 0
for i < len(got) && i < len(want) && got[i] == want[i] {
i++
}
t.Errorf("stream diverges in round %d at byte %d of %d; arrival order %v",
round, i, len(want), order)
lo := max(0, i-200)
t.Errorf("got %s", summarizeRuns(got[lo:min(len(got), i+200)]))
t.Errorf("want %s", summarizeRuns(want[lo:min(len(want), i+200)]))
t.FailNow()
}
}
t.Logf("%d bytes intact across %d rounds of reordering (window %d)", len(got), rounds, shuffleWindow)
}
// summarizeRuns renders a byte stream as run-length pairs ("A*100 B*100") so a
// duplicated or missing segment is visible at a glance.
func summarizeRuns(b []byte) string {
out := make([]byte, 0, 64)
for i := 0; i < len(b); {
j := i
for j < len(b) && b[j] == b[i] {
j++
}
out = append(out, b[i], '*')
out = append(out, strconv.Itoa(j-i)...)
out = append(out, ' ')
i = j
}
return string(out)
}
+120
View File
@@ -0,0 +1,120 @@
package tcp
import (
"math/rand"
"testing"
"time"
"github.com/soypat/lneto/ethernet"
)
// TestHandlerRetransmitsAfterRTO covers the seam between a Handler and its
// LossRecovery, which the RTO unit tests do not: a lost data segment must be
// resent once the timer expires, with nothing arriving to prompt it.
func TestHandlerRetransmitsAfterRTO(t *testing.T) {
const mtu = ethernet.MaxMTU
const maxpackets = 4
rng := rand.New(rand.NewSource(5))
client, server := newHandler(t, mtu, maxpackets), newHandler(t, mtu, maxpackets)
var now int64 // injected monotonic clock, in nanoseconds
client.SetLossRecovery(new(RTO), func() int64 { return now })
setupClientServer(t, rng, client, server)
var rawbuf [mtu]byte
establish(t, client, server, rawbuf[:])
data := []byte("hello")
if n, err := client.Write(data); err != nil || n != len(data) {
t.Fatal("client write:", n, err)
}
clear(rawbuf[:])
n, err := client.Send(rawbuf[:])
if err != nil || n == 0 {
t.Fatal("client send:", n, err)
}
// That frame is lost: it is never handed to the server.
// Nothing may come back before the timer expires.
var probe [mtu]byte
if n, err := client.Send(probe[:]); err != nil || n != 0 {
t.Fatalf("client sent %d bytes before the RTO expired (err %v)", n, err)
}
now += int64(3 * time.Second) // past the initial RTO and one backoff
clear(probe[:])
n, err = client.Send(probe[:])
if err != nil {
t.Fatal("client send after RTO:", err)
}
if n == 0 {
t.Fatal("no retransmission after the RTO expired: the loss-recovery directive is never applied")
}
if err := server.Recv(probe[:n]); err != nil {
t.Fatal("server refused the retransmission:", err)
}
got := make([]byte, 16)
nr, err := server.Read(got)
if err != nil || string(got[:nr]) != string(data) {
t.Fatalf("server read %q (%v), want %q", got[:nr], err, data)
}
}
// TestHandlerRetransmitsAfterCloseWithUnackedData is the write-then-close case
// every server performs. With the last data segment lost, the FIN behind it sits
// above a gap the peer cannot cross, so FIN-WAIT-1 must still retransmit that
// data or both sides wait forever.
func TestHandlerRetransmitsAfterCloseWithUnackedData(t *testing.T) {
const mtu = ethernet.MaxMTU
const maxpackets = 4
rng := rand.New(rand.NewSource(9))
client, server := newHandler(t, mtu, maxpackets), newHandler(t, mtu, maxpackets)
var now int64
client.SetLossRecovery(new(RTO), func() int64 { return now })
setupClientServer(t, rng, client, server)
var rawbuf [mtu]byte
establish(t, client, server, rawbuf[:])
data := []byte("last response bytes")
if n, err := client.Write(data); err != nil || n != len(data) {
t.Fatal("client write:", n, err)
}
clear(rawbuf[:])
n, err := client.Send(rawbuf[:]) // this frame is lost in transit
if err != nil || n == 0 {
t.Fatal("client send:", n, err)
}
// The application closes right after writing.
if err := client.Close(); err != nil {
t.Fatal("client close:", err)
}
var finbuf [mtu]byte
nfin, err := client.Send(finbuf[:]) // FIN (also lost, or simply unacked)
if err != nil {
t.Fatal("client send FIN:", err)
}
t.Logf("state after close: %s (FIN frame %d bytes)", client.State(), nfin)
now += int64(3 * time.Second) // past the RTO
var probe [mtu]byte
n, err = client.Send(probe[:])
if err != nil {
t.Fatal("client send after RTO:", err)
}
if n == 0 {
t.Fatalf("no retransmission in %s: unacknowledged data is stranded by the close", client.State())
}
if err := server.Recv(probe[:n]); err != nil {
t.Fatal("server refused the retransmission:", err)
}
got := make([]byte, 32)
nr, err := server.Read(got)
if err != nil || string(got[:nr]) != string(data) {
t.Fatalf("server read %q (%v), want %q", got[:nr], err, data)
}
}
+3 -4
View File
@@ -1,7 +1,6 @@
package udp
import (
"fmt"
"net"
"github.com/soypat/lneto"
@@ -117,7 +116,7 @@ func (h *Handler) Send(buf []byte) (int, error) {
dgram := internal.SliceDequeueFront(&h.txDgrams)
n, err := h.txRing.Read(buf[8 : 8+dgram.length])
if err != nil || n != int(dgram.length) {
panic(fmt.Sprintf("udp send handler failure %d %s", n, err))
panic("udp send handler failure")
}
ufrm.SetSourcePort(h.lport)
ufrm.SetDestinationPort(h.rport)
@@ -152,13 +151,13 @@ func (h *Handler) ReadNext(b []byte) (int, error) {
dgram := internal.SliceDequeueFront(&h.rxDgrams)
n, err := h.rxRing.Read(b[:min(len(b), int(dgram.length))])
if err != nil {
panic(fmt.Sprintf("udp read handler failure %d %s", n, err))
panic("udp readnext rx ring failure")
}
discard := int(dgram.length) - len(b)
if discard > 0 {
err = h.rxRing.ReadDiscard(discard)
if err != nil {
panic(fmt.Sprintf("udp readdiscard handler failure %d %s", n, err))
panic("udp readnext discard failure")
}
}
return n, nil
+1 -2
View File
@@ -1,7 +1,6 @@
package udp
import (
"fmt"
"math"
"net"
"net/netip"
@@ -226,7 +225,7 @@ func (mh *muxHandler) Encapsulate(carrierData []byte, ipOffset, frameOffset int)
n, err := mh.txRing.Read(buf[8 : 8+dgram.length])
if err != nil || n != int(dgram.length) {
panic(fmt.Sprintf("udp send handler failure %d %s", n, err))
panic("udp muxh encaps fail txring read")
}
ufrm.SetSourcePort(dgram.lport)
ufrm.SetDestinationPort(dgram.rport)
+1 -2
View File
@@ -2,7 +2,6 @@ package lneto
import (
"errors"
"fmt"
"strconv"
)
@@ -82,7 +81,7 @@ type BitPosErr struct {
}
func (bpe *BitPosErr) Error() string {
return fmt.Sprintf("%s at bits %d..%d", bpe.Err.Error(), bpe.BitStart, bpe.BitStart+bpe.BitLen)
return bpe.Err.Error()
}
func (bpe *BitPosErr) AppendError(dst []byte) []byte {
+35 -5
View File
@@ -107,9 +107,30 @@ func (bs *bufferSelect) numFree() (numFree int) {
}
// getRx returns the oldest published Rx frame, or nil if none is pending.
//
// The slot scan is not a consistent snapshot: goroPutRx may publish a frame
// into an already-scanned slot while this scan is in progress. Because the
// producer publishes frames in seq (arrival) order, any such straggler carries
// a lower seq than the candidate and must be delivered first to preserve
// arrival order. A confirming re-scan detects it; the loop retries until no
// older frame is observed, which terminates because the candidate seq strictly
// decreases and is bounded below by the true oldest pending frame.
func (bs *bufferSelect) getRx() []byte {
oldest := -1
var oldestSeq uint32
for {
oldest, oldestSeq := bs.scanOldest()
if oldest < 0 {
return nil
}
if !bs.hasPendingOlderThan(oldestSeq) {
return bs.bufs[oldest].buf[:bs.bufs[oldest].lenAcquire.Load()]
}
}
}
// scanOldest returns the index and seq of the pending Rx frame with the lowest
// arrival seq, or -1 if none is pending.
func (bs *bufferSelect) scanOldest() (oldest int, oldestSeq uint32) {
oldest = -1
for i := range bs.bufs {
n := bs.bufs[i].lenAcquire.Load()
if n > 0 && bs.bufs[i].isRx.Load() &&
@@ -118,10 +139,19 @@ func (bs *bufferSelect) getRx() []byte {
oldestSeq = bs.bufs[i].seq
}
}
if oldest < 0 {
return nil
return oldest, oldestSeq
}
// hasPendingOlderThan reports whether any pending Rx frame has a seq strictly
// less than seq, i.e. a frame that should be delivered before it.
func (bs *bufferSelect) hasPendingOlderThan(seq uint32) bool {
for i := range bs.bufs {
n := bs.bufs[i].lenAcquire.Load()
if n > 0 && bs.bufs[i].isRx.Load() && lessThan(bs.bufs[i].seq, seq) {
return true
}
}
return bs.bufs[oldest].buf[:bs.bufs[oldest].lenAcquire.Load()]
return false
}
func (bs *bufferSelect) release(buf []byte) {
+25 -1
View File
@@ -53,6 +53,10 @@ type StackAsync struct {
lookup dns.Message
dnssv netip.Addr
// ephPort drives sequential ephemeral-port allocation (see
// [StackAsync.ephemeralPort]); zero means not yet seeded.
ephPort uint32
ntpUDP internet.StackUDPPort
ntp ntp.Client
@@ -125,8 +129,8 @@ func (s *StackAsync) IngressEthernet(ethernetFrame []byte) error {
s.mu.Lock()
defer s.mu.Unlock()
s.stats.TotalReceived += uint64(len(ethernetFrame))
err := s.link.Demux(ethernetFrame, 0)
debugPacket("IN ", ethernetFrame)
err := s.link.Demux(ethernetFrame, 0)
if err == nil {
s.arpt.learnFromIngressEthernet(ethernetFrame)
}
@@ -352,6 +356,23 @@ func (s *StackAsync) Prand32() (randval uint32) {
return randval
}
// ephemeralPort returns the next port of the IANA dynamic range (49152-65535,
// RFC 6335 §6), allocated sequentially from a random per-stack start so a port is
// revisited only after the full 16384-port cycle. Random selection instead reuses
// a recent port at birthday-paradox rates, and a reused 4-tuple can collide with
// state the previous conversation left behind (a TIME-WAIT, a NAT flow entry)
// which swallows the new SYN.
func (s *StackAsync) ephemeralPort() uint16 {
s.mu.Lock()
if s.ephPort == 0 {
s.ephPort = s.prand32()%16384 | 1
}
port := 49152 + s.ephPort%16384
s.ephPort++
s.mu.Unlock()
return uint16(port)
}
func (s *StackAsync) prand32() uint32 {
/* Algorithm "xor" from p. 4 of Marsaglia, "Xorshift RNGs" */
seed := internal.Prand32(s.prng)
@@ -630,6 +651,9 @@ func (s *StackAsync) StartLookupIPType(host string, qtype dns.Type) error {
s.ednsopt,
},
EnableRecursion: true,
// Leave headroom above the address buffer for CNAME records, which
// occupy answer slots before the addresses they alias.
MaxResponseAnswers: uint16(len(s.addrbufnip)) + 8,
})
if err != nil {
return err
+6 -14
View File
@@ -102,8 +102,9 @@ func (s StackGo) SocketNetip(ctx context.Context, network string, family, sotype
isDial := raddr.IsValid() && !raddr.Addr().IsUnspecified()
if laddr.Port() == 0 {
// Auto-assign an ephemeral port for both outbound dials and for listeners
// that did not request a fixed port.
laddr = netip.AddrPortFrom(laddr.Addr(), uint16(49152+s.blk.async.Prand32()%16384))
// that did not request a fixed port. Sequential, not random: see
// [StackAsync.ephemeralPort] for why random selection breaks dial churn.
laddr = netip.AddrPortFrom(laddr.Addr(), s.blk.async.ephemeralPort())
}
if laddr.Addr().IsUnspecified() {
// Fill in the stack's configured address for the requested family.
@@ -162,11 +163,9 @@ func (s StackGo) SocketNetip(ctx context.Context, network string, family, sotype
return nil, err
}
uc := udpconn{
Conn: &conn,
// TODO: use udpaddr until UDPAddrFromAddrPort added to tinygo.
// https://github.com/tinygo-org/net/issues/45
localAddr: udpaddr(laddr),
raddr: udpaddr(raddr),
Conn: &conn,
localAddr: net.UDPAddrFromAddrPort(laddr),
raddr: net.UDPAddrFromAddrPort(raddr),
}
return uc, nil
case "tcp", "tcp4", "tcp6":
@@ -384,13 +383,6 @@ func (c udpconn) ReadFrom(b []byte) (int, net.Addr, error) {
func (c udpconn) WriteTo(b []byte, _ net.Addr) (int, error) {
return c.Conn.Write(b) // connected UDP: always writes to dialed remote
}
func udpaddr(addr netip.AddrPort) net.Addr {
return &net.UDPAddr{
IP: addr.Addr().AsSlice(),
Zone: addr.Addr().Zone(),
Port: int(addr.Port()),
}
}
// parseNetAddr converts a [net.Addr] to a [netip.AddrPort]. A nil or empty IP
// (e.g. ":22" from a listen address with no host) is treated as 0.0.0.0 so
+102
View File
@@ -156,6 +156,14 @@ func buildDNSResponsePacket(t *testing.T, txid uint16, dstPort uint16, hostname
dns.NewResource(name, dns.TypeA, dns.ClassINET, 300, addr.AsSlice()),
},
}
return buildDNSMsgResponsePacket(t, txid, dstPort, msg, srcIP, srcMAC, dstIP, dstMAC, buf)
}
// buildDNSMsgResponsePacket wraps a DNS response message into a complete
// Ethernet+IP+UDP packet with valid checksums.
func buildDNSMsgResponsePacket(t *testing.T, txid uint16, dstPort uint16, msg dns.Message,
srcIP netip.Addr, srcMAC [6]byte, dstIP netip.Addr, dstMAC [6]byte, buf []byte) ([]byte, error) {
t.Helper()
// Response flags: QR=1 (response), RD=1 (recursion desired), RA=1 (recursion available).
responseFlags := dns.HeaderFlags(1<<15 | 1<<8 | 1<<7)
@@ -237,3 +245,97 @@ var errBaseLenDNS = func() error {
}()
var errInvalidEtherType = errors.New("invalid ethernet type")
// TestDNS_CNAMEResponse verifies that a DNS response containing a CNAME record
// followed by an A record for the canonical name resolves to the A record's
// address: the CNAME RDATA must not be misinterpreted as an IP address.
func TestDNS_CNAMEResponse(t *testing.T) {
const seed = 9876
const MTU = ethernet.MaxMTU
client := new(StackAsync)
dnsServerAddr := netip.AddrFrom4([4]byte{8, 8, 8, 8})
clientAddr := netip.AddrFrom4([4]byte{10, 0, 0, 100})
clientMAC := [6]byte{0xde, 0xad, 0xbe, 0xef, 0x00, 0x01}
dnsServerMAC := [6]byte{0x00, 0x11, 0x22, 0x33, 0x44, 0x55}
err := client.Reset(StackConfig{
Hostname: "DNSClient",
RandSeed: seed,
StaticAddress4: clientAddr.As4(),
DNSServer: dnsServerAddr,
HardwareAddress: clientMAC,
MTU: uint16(MTU),
})
if err != nil {
t.Fatal("client Reset failed:", err)
}
client.SetGatewayHardwareAddr(dnsServerMAC)
const hostname = "www.example.com"
const alias = "cdn.example.net"
wantAddr := netip.MustParseAddr("192.0.2.200")
err = client.StartLookupIP(hostname)
if err != nil {
t.Fatal("StartLookupIP failed:", err)
}
const carrierDataSize = ethernet.MaxFrameLength
var buf [carrierDataSize]byte
// Client sends DNS query for www.example.com.
n, err := client.EgressEthernet(buf[:])
if err != nil || n == 0 {
t.Fatal("expected DNS query packet from client:", err, n)
}
txid, clientPort, err := extractDNSTxIDAndPort(buf[:n])
if err != nil {
t.Fatal("failed to extract DNS txid:", err)
}
// Respond with a CNAME record www.example.com -> cdn.example.net
// followed by the A record for cdn.example.net.
owner, err := dns.NewName(hostname)
if err != nil {
t.Fatal(err)
}
aliasName, err := dns.NewName(alias)
if err != nil {
t.Fatal(err)
}
aliasWire, err := aliasName.AppendTo(nil)
if err != nil {
t.Fatal(err)
}
msg := dns.Message{
Questions: []dns.Question{{
Name: owner,
Type: dns.TypeA,
Class: dns.ClassINET,
}},
Answers: []dns.Resource{
dns.NewResource(owner, dns.TypeCNAME, dns.ClassINET, 300, aliasWire),
dns.NewResource(aliasName, dns.TypeA, dns.ClassINET, 300, wantAddr.AsSlice()),
},
}
responsePkt, err := buildDNSMsgResponsePacket(t, txid, clientPort, msg,
dnsServerAddr, dnsServerMAC, clientAddr, clientMAC, buf[:])
if err != nil {
t.Fatal("failed to build CNAME response packet:", err)
}
if err = client.IngressEthernet(responsePkt); err != nil {
t.Fatal("client Demux of CNAME response failed:", err)
}
addrs, done, err := client.ResultLookupIP(hostname)
if err != nil {
t.Fatal("ResultLookupIP error:", err)
}
if !done {
t.Fatal("DNS lookup not done after receiving CNAME response")
}
if !slices.Contains(addrs, wantAddr) {
t.Errorf("expected address %s not found in result %v", wantAddr, addrs)
}
}
+36
View File
@@ -1196,3 +1196,39 @@ func TestEgressIP_TCPMSSAdvertisesMTU(t *testing.T) {
t.Errorf("advertised MSS = %d, want %d (MTU %d - 40)", gotMSS, wantMSS, mtu)
}
}
// TestEphemeralPortSequence checks no ephemeral port is reused before the whole
// 16384-port dynamic range has cycled, which is what keeps a redial off the
// teardown state (TIME-WAIT, NAT flow entries) of the conversation before it.
func TestEphemeralPortSequence(t *testing.T) {
s := new(StackAsync)
err := s.Reset(StackConfig{
Hostname: "eph",
RandSeed: 42,
StaticAddress4: [4]byte{10, 0, 0, 50},
MaxActiveTCPPorts: 1,
HardwareAddress: [6]byte{0xbe, 0xef, 0, 0, 0, 50},
MTU: ethernet.MaxMTU,
ICMPQueueLimit: 2,
})
if err != nil {
t.Fatal(err)
}
const cycle = 16384
var seen [cycle]bool
for i := range cycle {
port := s.ephemeralPort()
if port < 49152 {
t.Fatalf("port %d below dynamic range (RFC 6335)", port)
}
idx := port - 49152
if seen[idx] {
t.Fatalf("port %d reused after only %d allocations (want full %d cycle)", port, i, cycle)
}
seen[idx] = true
}
// The cycle is exhausted: the next allocation may legitimately reuse.
if got := s.ephemeralPort(); got < 49152 {
t.Fatalf("post-cycle port %d below dynamic range", got)
}
}