104 Commits

Author SHA1 Message Date
Patricio Whittingslow 98f104f67c remove backoff from form parsing 2026-07-27 18:10:22 -03:00
Patricio Whittingslow 659e7c7ba9 expose rawsock as experimental package (will use for external benchmarks) 2026-07-27 17:45:43 -03:00
Patricio Whittingslow ccb2762fc2 run go fix 2026-07-27 12:28:52 -03:00
Patricio Whittingslow a97959739e fix examples 2026-07-27 12:24:57 -03:00
Patricio Whittingslow 0d1f50fc48 fix tests after excplicit header alloc change 2026-07-27 12:22:20 -03:00
Patricio Whittingslow 0e2f487e9f explicit header key/value alloc and add ExchangeConfig 2026-07-27 11:51:02 -03:00
Patricio Whittingslow 75d2c0c46d add a pattern argument to Mux 2026-07-26 22:55:50 -03:00
Patricio Whittingslow e46e45238d apply go fix 2026-07-26 22:12:57 -03:00
Patricio Whittingslow 33ed289d30 simplify clanker slop 2026-07-26 22:08:57 -03:00
Patricio Whittingslow ebfd80c80c ai insists with backoffs 2026-07-26 21:29:02 -03:00
Patricio Whittingslow 0b6a71efa8 add Exchange.ReadMultiparts reimagining of clanker slop 2026-07-26 20:19:46 -03:00
Patricio Whittingslow 0c1a51150d begin adding readMultiPart 2026-07-26 18:04:34 -03:00
Patricio Whittingslow 4f1a178f29 first Multipart approach 2026-07-26 17:22:09 -03:00
Patricio Whittingslow dd7c4bb037 remove status type 2026-07-26 13:57:18 -03:00
Patricio Whittingslow cc7ecc0c19 finish rounding up multipart form parsing 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 4976cdf38e begin adding multipart form logic 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 92cf46e570 add streaming API distinct from Exchange 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 6d88506541 add raw buffer access 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 229acd3e68 fail on incomplete staging 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 69640a6235 minor doc nit 2026-07-26 13:57:18 -03:00
Patricio Whittingslow b79e58b812 add MethodFrom 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 10932ebf34 rework Mux interface to receive a string request path 2026-07-26 13:57:18 -03:00
Patricio Whittingslow c19fd2093c run go fix 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 5937070cc5 Router.Handle returns error after being torn down 2026-07-26 13:57:18 -03:00
Patricio Whittingslow dab5095a49 massive documentation push and code reordering in files 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 9faa0c2722 remove ForEach pattern, allocates in TinyGo 2026-07-26 13:57:18 -03:00
Patricio Whittingslow d83da243a0 add query handling 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 99f848f93f add benchmarks 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 6e332e7833 small nit 2026-07-26 13:57:18 -03:00
Patricio Whittingslow a7f598f975 several bugfixes, add internal.IntLen, round up http-linux example with new router API 2026-07-26 13:57:18 -03:00
Patricio Whittingslow d4666630f7 improve locking and acquisition of Exchanges in reconfiguring 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 962f6864be add Hijacker-like functionality 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 5b83b18da0 more tests, run go generate 2026-07-26 13:57:18 -03:00
Patricio Whittingslow e0dd19416c add low level Handle function and more tests 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 9397caf144 claude found neat bugs 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 0fd0ad8cbf begin adding httphi tests 2026-07-26 13:57:18 -03:00
Patricio Whittingslow 6e7cd94040 add http/httphi 2026-07-26 13:57:18 -03:00
Pat Whittingslow a3f2742abf DHCPv4+Debug (#170)
* DHCPv4+Debug: add RawData method and improve Logging semantics with configured logger instead of default

* add xnet.LogAllocs
2026-07-26 13:55:48 -03:00
Marvin 马维 Drees 8873b55e5d feat(tcp): add RFC 6298 RTO as a LossRecovery implementation (#169)
Provide a concrete packet-loss recovery algorithm for the LossRecovery
interface added in #168: the RFC 6298 round-trip-time estimator and single
retransmission timer.

RTO is a pure, reactive state machine. It derives RTT estimates and
retransmission decisions solely from the segments observed through the
LossRecovery hooks and the monotonic time handed in at each boundary, so it
holds no clock and allocates nothing (issue #140). It tracks a shadow of the
send sequence space (snd.UNA/snd.NXT) purely from observed segments, which is
how it manages the timer without reaching into the ControlBlock and how it
distinguishes retransmissions for Karn's algorithm.

Covered:
  - §2.2/§2.3 SRTT/RTTVAR/RTO smoothing (integer-shift form)
  - §3 Karn's algorithm: one sample in flight, never sample a retransmit
  - §5.1-§5.3 timer arm/restart/stop as data is sent and acknowledged
  - §5.4-§5.6 timeout response: exponential backoff + go-back-N retransmit
  - §5.7 backoff collapse on a valid RTT sample
  - RTO clamped to [rtoMin, rtoMax]

RTT introspection (SmoothedRTT, CurrentRTO, Running) lives on the concrete type,
not the interface, per the LossRecovery design. Install with new(RTO) on
ConnConfig.LossRecovery; the connection calls Reset on open so the zero value is
ready to use.

Generated with LLM assistance.

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-07-22 13:44:31 -03:00
Pat Whittingslow 41c7e3c445 add loss.go (#168) 2026-07-22 13:09:12 -03:00
Pat Whittingslow 1bbfd5eac3 @MDr164 trick to ignore no-extension files (#162)
* @MDr164 trick to ignore no-extension files

* catch changes to LICENSE
2026-07-21 16:14:53 -03:00
Pat Whittingslow 1a474154cb more httpraw bug fixes (#160)
* more httpraw bug fixes

* add http-linux example and add httpraw.Header.SetInt

* finish http-linux example

* fix CI vet
2026-07-19 11:09:09 -03:00
Patricio Whittingslow 347cef9ba5 README: update binary benchmark table 2026-07-17 12:03:28 -03:00
Pat Whittingslow 157a8537a2 testing.B.Loop() standardization and StackAsync.Debug improvement (#156)
* testing.B.Loop() standardization and StackAsync.Debug heap debugging improvement

* apply @MDr164 fixes

* @MDr164 great suggestions to prevent panic and print correct mallocs
2026-07-17 11:59:27 -03:00
Pat Whittingslow 766e03e90e httpraw: fix overrwrite/empty val bugs in Header.Set/SetBytes (#159) 2026-07-17 11:07:59 -03:00
Marvin Drees ab1a0c735a Add out-of-order segment reassembly (#148)
* feat(tcp): add out-of-order segment reassembly

Add an opt-in, bounded out-of-order reassembly buffer so a single lost
segment can be recovered by retransmitting the gap while later segments
are held and delivered once the gap fills.

The receiver also subtracts buffered out-of-order bytes from the
advertised receive window and avoids challenge-ACK aborts for in-window
future data. Reassembly is disabled by default.

Generated with LLM assistance.

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>

* implement review feedback around rx buffer reuse

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>

---------

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-07-10 10:36:15 -03:00
Marvin Drees 3c1f0e0281 Handle IA_NA/IA_PD requests and config options in DHCPv6 (#147)
* feat(dhcpv6): handle IA_NA/IA_PD requests and config options

Adds the DHCPv6 client request exchange with IA_NA/IA_PD handling,
reconfigure-renew (RFC 8415), DNS server and domain search options
(RFC 3646) and NTP server suboptions (RFC 5908).

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>

* fix(dhcpv6): bound server-supplied option counts to prevent OOM

A DHCPv6 server may place arbitrarily many DNS servers, search domains,
NTP entries and delegated prefixes in a single message (RFC 8415). The
client appended every entry into growable slices with no upper bound, so
a malicious or misconfigured server could drive unbounded allocation.

Add a configurable Limits to RequestConfig (with safe defaults) capping
each repeated option. The backing arrays are sized to their caps once in
BeginRequest and reused across resets, so parsing a server message now
performs no allocation and stored entries stay bounded.

Add tests asserting the caps are enforced against a flooded Reply and that
option parsing is allocation-free after BeginRequest.

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>

---------

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-07-09 14:12:30 -03:00
Pat Whittingslow 99e9d90a60 Netdev revamp (#145)
* begin working on netdev solution

* need to roll back some assumptions in next commit

* dual poll/async mode for netdev.Runner

* add wake on rx semantics

* remove TODO

* more reworking of Runner

* work on applying @MDr164 suggestions and a couple extra revamps

* add newline to end of test file
2026-07-09 14:04:10 -03:00
Marvin Drees 4fc53a84cd fix: make github action comment work right (#153)
It was previously just commenting the file name instead of the content,
this change attempts to fix it using json instead of the raw file

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-07-09 14:01:53 -03:00
Marvin Drees 104a2b0b26 feat(ci): add benchci for better visualization (#149)
This adds a small in-repo tool and updates the ci.yml file in order
to make benchmark and most importantly allocation results visible in a
PR. It also drops the 3rdparty action we had commented out.

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-07-02 20:21:49 -03:00
Patricio Whittingslow e5f8ac0687 add as of now unused af_inet6 for posterity 2026-07-01 21:09:28 -03:00
Patricio Whittingslow 3b5b010673 export xnet: AF_INET, SOCK_STREAM, SOCK_DGRAM 2026-07-01 21:07:21 -03:00
Pat Whittingslow 4d6363da00 bound MTU on Stack EgressIP (#151) 2026-07-01 18:46:24 -03:00
Pat Whittingslow cdc99fb9ff work on removing heap allocations (#150) 2026-07-01 13:26:05 -03:00
Pat Whittingslow 96a02e7d2c begin writing noipv6 test (#146) 2026-06-28 10:26:49 -03:00
Pat Whittingslow b795d6a437 use ltesto as package to contain scheduler/goroutine testing logic (#143)
* use ltesto as package to contain scheduler/goroutine testing logic

* add sched usage to another test

* testCloseTransmitsPending rewrite

* replace deadline with context
2026-06-27 10:37:46 -03:00
Ron Evans fa609ea54f fix: two fixes for dhcp/ap (#136)
* feature: ipv4 broadcast support

* stackip4 method receiver varname

* Incorporate strict test timing for TestStackGoTCPDialRetriesPendingControl (#135)

* rewrite tinygo failing test to be more real-time

* use channels to scheduler stack

* fix flakiness by increasing timeout and move implementation to top of file

* fix(dhcp): use chaddr as lookup key and patch Ethernet dst on Offer/Ack

- Client lookup in Demux() now keys on chaddr when no OptClientIdentifier
  is present. It previously used ciaddr which is 0.0.0.0 during initial lease
  acquisition, causing MsgRequest to fail to match any client (RFC 2131 §4.3.1)
- In Encapsulate(), overwrite Ethernet dst with client.hwaddr when packet is
  embedded in an IP frame (offsetToIP >= 14), per RFC 2131 §4.1. It previously
  sent to gwmac which clients without an ARP entry could not receive

Signed-off-by: deadprogram <ron@hybridgroup.com>

* fix: accept 255.255.255.255 aka bradcast dst in demux4

This fixes a problem with accept 255.255.255.255 dst in demux4 which previously dropped when stack had a static IP.

Signed-off-by: deadprogram <ron@hybridgroup.com>

---------

Signed-off-by: deadprogram <ron@hybridgroup.com>
Co-authored-by: Patricio Whittingslow <graded.sp@gmail.com>
2026-06-24 22:16:53 -03:00
Pat Whittingslow a42f0b404c feature: ipv4 broadcast support and bugfix for reset connection reset of IPv4+IPv6 stacks (#139)
* feature: ipv4 broadcast support

* stackip4 method receiver varname
2026-06-24 16:54:41 -03:00
Pat Whittingslow ef279863a9 Incorporate strict test timing for TestStackGoTCPDialRetriesPendingControl (#135)
* rewrite tinygo failing test to be more real-time

* use channels to scheduler stack

* fix flakiness by increasing timeout and move implementation to top of file
2026-06-24 13:07:19 -03:00
TuteMthCD d5ea2efdf4 Retransmit active-open SYN after RTO (#130)
* added: sync packet retransmission

* test: sync packet retransmission

* refactor: remove packet time from tcppool

* fix: return comment

* refactor: move retrying to stackRetrying

* added: retries & timeout to stackGoConfig

* test: test retries stack

* refactor: move StackRetrying to stackBlocking

* fix: line gofmt

---------

Co-authored-by: Pat Whittingslow <graded.sp@gmail.com>
2026-06-23 00:19:16 -03:00
Marvin Drees a0a5336108 Add xcurl port and debug flag (#129)
* feat: add xcurl port and debug flag

Update the docs and add new flags to xcurl to aid debugging under an OS
like Linux.

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>

* drop debugtcp flag from xcurl again

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>

---------

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-06-22 18:32:50 -03:00
Pat Whittingslow 28addf329a try to fix fmt msg (#133)
* try to fix fmt msg

* fix CI
2026-06-22 18:16:29 -03:00
Marvin Drees 860d6d00bb Add atomic id guard to prevent TOCTOU (#131)
* add atomic id guard to prevent TOCTOU

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>

* implement review feedback to tcp conn guard

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>

---------

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-06-22 09:56:38 -03:00
Pat Whittingslow 8f9d765cb0 Ack dropping - fix for #50 (#127)
* fix for #50

* fix remaining test

* revert changes and instead approach problem on full-duplex case

* fix merge messup

* fix @MDr164 note and upgrade documentation while at it

* gate full buffer exit on rx shutdown since no risk of full buffer error

* add log line to note issue is hit

* document CloseRead relationship with Close
2026-06-19 18:43:13 -03:00
Pat Whittingslow 6f5acd1948 persist issue #57 fix in test (#128)
* persist issue 57 fix in test

* have go fix output error on CI

* apply go fix required fix

* test no || for go fix command

* test no || for go fix command complete, it is needed

* forgot to fix
2026-06-18 18:23:15 -03:00
Marvin Drees e0ef681085 feat: add RFC 3927 support (#114)
Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-06-18 10:21:25 -03:00
Pat Whittingslow bef2137bad add zero os docs (#126)
* add zero os docs

* mention scheduling
2026-06-18 10:08:49 -03:00
Patricio Whittingslow 059f761856 CapturePrinter expose Ethernet and IP printers 2026-06-17 16:42:38 -03:00
Pat Whittingslow 5a674a4ad1 fix for #124 (#125) 2026-06-17 15:16:55 -03:00
Pat Whittingslow 60098ad8d0 More IPv6 support and race condition fixes (#123)
* changes in preparation of wireguard impl

* suggestions by @MDr164
2026-06-17 15:10:25 -03:00
Pat Whittingslow 813b7b5e57 Lay out device/stack interfaces - Netdev/Netlink (#92)
* good code today

* add netdev.Runner

* round off APIs more

* better interface method documentation

* improve DHCP netstack API

* add pico w netdev example

* remove temp file

* keep thinking about this. this is hard :/

* fix ci

* small fixer

* working dhcp

* fix rebase API mismatches

* begin adding espradio example

* keep working on espradio, icmp not working

* fix icmp by fixing dhcp
2026-06-17 14:18:08 -03:00
Marvin Drees 5f8ca45859 feat(x/nts): add NTS support (#88)
Implement NTS (RFC 8915) with KERecord zero-copy frame, PerformKE
(TLS 1.3 + ExportKeyingMaterial), DeriveKeys, and Client state
machine implementing lneto.StackNode. Client handles cookie pool
management, auth body codec with nonce/ciphertext, and two-exchange
NTP flow with UniqueID verification and AEAD authentication.

Add NTS Server wrapping ntp.Server with AEAD verification/sealing,
and HandleKE for server-side NTS Key Exchange over TLS 1.3.

Use internal.LogAttrs for non-allocating structured logging
throughout the NTS client, matching existing conventions.

Generated with LLM assistance.

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-06-15 11:42:10 -03:00
Marvin Drees 856ac373d5 docs: update README to aid Linux debugging (#122)
Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-06-15 11:02:49 -03:00
Patricio Whittingslow 82f9461548 add missing backoffs 2026-06-09 14:33:50 -03:00
Pat Whittingslow 6ba06acdcb lneto rework: Require explicit BackoffStrategy in all APIs (#116)
* make backoff explicit API

* fix tests to use explicit backoff

* fix examples with explicit tcp
2026-06-09 10:20:40 -03:00
Pat Whittingslow 3b82cefec3 exclude stringers.go files from code coverage (#119) 2026-06-09 10:13:55 -03:00
Pat Whittingslow 24d5d303bc modifications to xnet stacks in preparation of wireguard integration (#113)
* modifications to xnet stacks in preparation of wireguard integration

* gofmt files
2026-05-27 13:53:57 -03:00
Marvin Drees 46d4c06b9f fix: clean up misc TODO comments and improve UDP source filtering (#89)
- http/httpraw: cap header slice growth to pre-allocated capacity, fix
  benchmark to reuse Header across iterations (0 allocs/op)
- internal/ring: replace TODO panic comments with invariant explanations
- tcp/conn: document truncated-frame offset check
- tcp/handler: replace TODO with net.ErrClosed on RST-closed connection
- internet/stack-udpport: filter incoming packets by remote IP address
  when configured via SetStackNode; skip filter for IPv4 multicast
  destinations (class D) so mDNS and similar protocols work correctly

Generated with LLM assistance.

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-05-20 14:29:09 -03:00
Patricio Whittingslow edf302baf8 normalize net.Addr parsing in Gostack 2026-05-14 18:06:38 -03:00
Patricio Whittingslow 8efd810610 more API touch ups 2026-05-14 13:15:16 -03:00
Patricio Whittingslow d010e6a7e9 API touch ups 2026-05-14 12:59:57 -03:00
Pat Whittingslow 9fcb7e9b52 Add udp.PacketConn (#112)
* update README table and begin planning udp.PacketConn

* first attempt at packetconn implementation

* add udp.PacketConn to StackAsync and implement SocketNetip iface

* add tests for PacketConn
2026-05-14 12:48:07 -03:00
Pat Whittingslow b95eb03aa5 fix CI and rework package structure (#111) 2026-05-13 15:44:30 -03:00
Pat Whittingslow a2970b923d add ipv6 to xnet.StackAsync (#107)
* add ipv6 to xnet.StackAsync

* dns improvements

* improve DNS workings of StackAsync

* add tentative ICMPv6

* work on prefixes and fix some small bugs, plan UDP/TCP6

* fix bugs in StackAsync and ipv4.Prefix.Contains

* update arpsubtable

* completely remove legacy internet.StackIP for StackIPv4/v6

* ipv4/ipv6 tcp/udp

* add TCP6/UDP6 dialing APIs

* add xnet.Stack6 interface

* more ipv6 integration into StackAsync; various tweaks to lneto and documentation+TODOs

* add stack6 tests

* replace netip.Prefix with ipv4.Prefix where it makes sense
2026-05-13 15:31:18 -03:00
Pat Whittingslow 7d5830d7ab V2 Netbird integration - UDP MIMO/SIMO, ICMPv6, DHCPv6 implementations (#106)
* begin adding udp.MuxHandler

* add udp MuxHandlerSIMO/MIMO

* add tcp rx shutdown

* icmpv6 client

* icmpv6 Client shared NDP/Echo preparation

* icmpv6 client ndp/echo split

* icmpv6 client ndp/echo split done

* icmpv6 adjustments

* add dhcpv6 stubs

* dhcpv4 preliminary revision

* add dns.NextLabel

* dns label name tweaks

* dns begin work on TCP client

* add dnstcp package

* apply gofmt changes

* add udp mux tests

* clean up, remove StackBig for now

* remove dnstcp so as to merged confident parts and we continue dnstcp work elsewhere
2026-05-10 12:16:30 -03:00
Pat Whittingslow bdbd38ab44 ipv6: StackIP and StackAsync.Addr refactor (#105)
* apply StackIP changes and internet package test passing

* fix tests and examples

* remove old Reset method on StackIP

* use encapsulate for ipv6

* add TCP over IPv6 tests
2026-05-09 16:11:31 -03:00
Marvin Drees a430f6c40a remove trailing whitespaces and add support matrix (#102)
Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-04-27 13:52:22 -03:00
Marvin Drees bba913751a feat(ntp): add two-exchange client, server handler, and extension fields (#101)
Implement full two-exchange NTP client state machine (RFC 5905 §8).
First exchange stores offset/RTT, second averages both for improved
accuracy. Client places T1 in TransmitTime per RFC 5905 §8; server
echoes it as OriginTime in the response.

Add NTP extension field codec (RFC 7822) with NextExtField iterator
and AppendExtField builder. Add NTS extension type constants from
RFC 8915.

Add NTP Server (StackNode) that receives client requests via Demux
and builds server responses via Encapsulate with configurable
stratum, precision, reference ID, and pending request queue.

Add Frame accessor methods: RawData(), ExtensionFields(),
ValidateSize(), and Timestamp.Uint64().

Add ntp-client and ntp-server example programs with
CalculateSystemPrecision, retry limits, and backoff.

Use internal.LogAttrs pattern for non-allocating structured logging
in the client, matching the tcp/debug.go convention.

Generated with Claude assistance.

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-04-27 13:27:08 -03:00
Patricio Whittingslow cf94767133 seeded fuzz test adjustment 2026-04-26 12:23:24 -03:00
Pat Whittingslow aa77403a2b passive MAC learning and ARP cache revamp (#96)
* begin working on arp cache

* fix little things

* rework arp cache priority

* fix test

* keep working on ARP

* push changes before thinking about ip prefix issue

* add passive peer MAC setting

* patch egress mac with correct ethernet CRCs

* consolidate subnet learning in subnetTable type

* add tests and fix subnet table bug
2026-04-24 23:34:53 -03:00
Patricio Whittingslow 67c42aa136 add log warning to httptap 2026-04-24 23:30:50 -03:00
Pat Whittingslow 198db3789b Add DNS support to pcap (#100)
* begin adding dns pcap functionality

* improve question formatting

* work on capturing answers as well

* finalize DNS printing

* increase subfield limits

* add FlagContainer type

* prettier DNS printing
2026-04-24 19:41:10 -03:00
Joel Wetzell b2c3d11c4c cleanup connect function and combine connection slices (#94)
* combine conns into single slice

* reuse logic in connect function

Co-authored-by: Copilot <copilot@github.com>

* correct comment

---------

Co-authored-by: Copilot <copilot@github.com>
2026-04-24 09:21:09 -03:00
Joel Wetzell a46e40580c add basic support for connect and send for UDP sockfd's (#93)
Co-authored-by: Copilot <copilot@github.com>
2026-04-23 19:38:48 -03:00
Marvin Drees 34bbaa6eb4 chore: bump upload-artifact action version (#91)
I noticed some warning around Node v20 usage from the old version
so I bumped to the currently latest one (v7 moved to Node v24).

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-04-20 10:36:08 -03:00
Pat Whittingslow 363bec6793 fixes for CI (#90) 2026-04-17 15:58:55 -03:00
Patricio Whittingslow 6d0ffcf0ad add extra info to README 2026-04-17 15:42:21 -03:00
Patricio Whittingslow c020795303 add gvisor-mwe and add binary sizes to README 2026-04-17 15:41:17 -03:00
Marvin Drees f87761f777 chore: bump minimum Go version to 1.24 (#85)
Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-04-16 17:21:23 -03:00
Marvin Drees 9d436c1d86 ci: replace go.yml with comprehensive ci.yaml (#84)
- Rename go.yml to ci.yaml
- Pin all actions to full commit SHAs (checkout v6.0.2, setup-go v6.4.0,
  codecov v6.0.0, setup-tinygo v2.0.1, github-action-benchmark v1.22.0)
- Add dedicated benchmark job with regression detection via
  benchmark-action/github-action-benchmark (120% alert threshold)
- Set workflow-level permissions: read-all with per-job write grants

Signed-off-by: Marvin Drees <marvin.drees@9elements.com>
2026-04-16 13:58:55 -03:00
Pat Whittingslow 16c2c3de36 Stack backoff rework (#83)
* huge Stack backoff rework

* lneto bump: huge Stack backoff rework
2026-04-14 21:05:02 -03:00
Pat Whittingslow 75a812a8d7 Tcp buffer bug fix (#79)
* start working on tracking down tcp buffer bug

* mtu refactor

* modularize test

* more precise testing

* tests fail, but is it the failure we are looking for?

* fix typo in espradio link (#76)

* implement a new backoff abstraction (#75)

* rewrite backoff api

* rewrite tcp.Conn.Write

* keep fixing small things

* much better Conn.Read implementation

* fix critical overflow bug in internal.ConnRWBackoff

---------

Co-authored-by: Joel Wetzell <jwetzell@yahoo.com>
2026-04-14 19:11:29 -03:00
Pat Whittingslow 5bde7a9979 implement a new backoff abstraction (#75) 2026-04-14 16:11:03 -03:00
jwetzell 191b8ee4cc fix typo in espradio link (#76) 2026-04-13 17:49:31 -03:00
200 changed files with 24117 additions and 1930 deletions
+4
View File
@@ -0,0 +1,4 @@
ignore:
- "examples/**"
- "**/stringers.go"
- "**/*_stringers.go"
+90
View File
@@ -0,0 +1,90 @@
name: Benchmark Comment
# Rationale: This more privileged workflow runs in the base repo's
# context (with a write token) only after the 'untrusted' CI workflow finishes.
# It does NOT check out or execute PR code; it only consumes the benchmark
# report artifact as inert, validated data.
on:
workflow_run:
workflows: [CI]
types: [completed]
permissions:
contents: read
jobs:
comment:
# Only for PR-triggered CI runs that succeeded to suppress noise
if: >
github.event.workflow_run.event == 'pull_request' &&
github.event.workflow_run.conclusion == 'success'
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write # needed to comment
steps:
- name: Download generated benchmark report
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.1
with:
name: bench-report
path: bench-report
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: Validate report and upsert PR comment
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.workflow_run.pull_requests[0].number }}
HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
HEAD_OWNER: ${{ github.event.workflow_run.head_repository.owner.login }}
HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
run: |
set -euo pipefail
report="bench-report/bench-report.md"
marker='<!-- lneto-bench -->'
# Treat the artifact as untrusted input: it was produced by a job that
# ran PR code. Basic validation before posting. TODO: Make more robust.
test -f "$report"
size="$(wc -c < "$report")"
if [ "$size" -le 0 ] || [ "$size" -gt 1000000 ]; then
echo "::error::benchmark report has unexpected size: ${size} bytes"
exit 1
fi
if ! grep -qF "$marker" "$report"; then
echo "::error::benchmark report missing marker ${marker}"
exit 1
fi
# Prefer the PR number from the workflow_run payload, which ties this
# privileged workflow to the PR-triggered CI run. Fall back to the
# run's head SHA/branch only when GitHub does not populate it.
pr="$PR_NUMBER"
if [ -z "$pr" ]; then
pr="$(gh api "repos/${REPO}/commits/${HEAD_SHA}/pulls" \
--jq 'map(select(.state == "open")) | .[0].number // empty')"
fi
if [ -z "$pr" ]; then
pr="$(gh api --method GET "repos/${REPO}/pulls" \
-f state=open \
-f head="${HEAD_OWNER}:${HEAD_BRANCH}" \
--jq '.[0].number // empty')"
fi
if [ -z "$pr" ]; then
echo "No open PR found for ${HEAD_SHA}; nothing to comment."
exit 0
fi
comment_id="$(
gh api "repos/${REPO}/issues/${pr}/comments" --paginate \
--jq ".[] | select(.body | contains(\"${marker}\")) | .id" | head -n1
)"
if [ -n "$comment_id" ]; then
gh api --method PATCH "repos/${REPO}/issues/comments/${comment_id}" \
--input - < <(jq -n --rawfile body "$report" '{body: $body}')
else
gh api --method POST "repos/${REPO}/issues/${pr}/comments" \
--input - < <(jq -n --rawfile body "$report" '{body: $body}')
fi
+137
View File
@@ -0,0 +1,137 @@
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
permissions: read-all
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "1.26"
- name: gofmt
run: |
unformatted=$(gofmt -l -d . 2>&1) || true
if [ -n "$unformatted" ]; then
echo "::error::File(s) not formatted with gofmt:"
echo "$unformatted"
exit 1
fi
- name: go vet
run: go vet ./...
- name: go vet (tinygo tags)
run: go vet -tags=tinygo ./...
- name: go fix
run: |
diff=$(go fix -diff ./... 2>&1) || true
if [ -n "$diff" ]; then
echo "$diff"
echo "::error::Files need go tool fix. Run: go tool fix ./..."
exit 1
fi
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "1.26"
- name: Build
run: go build -v ./...
test:
needs: [build]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "1.26"
- name: Test
run: |
mkdir -p profiles
go test -v -shuffle=on -count=1 \
-coverprofile=profiles/coverage.profile -covermode=atomic -coverpkg=./... \
-timeout=10m \
./...
- name: Upload coverage
uses: codecov/codecov-action@3f20e214133d0983f9a10f3d63b0faf9241a3daa # v6.0.0
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: profiles/coverage.profile
slug: soypat/lneto
- name: Upload profiles
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: test-profiles
path: profiles/
retention-days: 14
race:
needs: [build]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "1.26"
- name: Test (race + shuffle)
run: go test -race -shuffle=on -count=1 -timeout=10m ./...
benchmark:
needs: [test]
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "1.26"
- name: Run benchmarks
run: go test -bench=. -benchmem -count=5 -shuffle=on -run='^$' -timeout=15m ./... | tee bench-results.txt
- name: Generate benchmark report
run: |
go run ./internal/benchci -current bench-results.txt -out bench-report.md
cat bench-report.md >> "$GITHUB_STEP_SUMMARY"
- name: Upload generated benchmark report
if: github.event_name == 'pull_request'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: bench-report
path: bench-report.md
retention-days: 30
- name: Upload raw benchmark results
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: bench-results
path: bench-results.txt
retention-days: 30
-43
View File
@@ -1,43 +0,0 @@
# This workflow will build a golang project
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-go
name: Go
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: 1.25
- name: Build
run: go build -v ./...
- name: TinyGo vet
run: go vet -tags=tinygo ./...
# - name: govulncheck # Getting false positives all the time.
# uses: golang/govulncheck-action@v1
# with:
# go-version-input: 1.21
# go-package: ./...
- name: Test
run: go test -v -coverprofile=coverage.txt -covermode=atomic -coverpkg=./... -race ./...
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
slug: soypat/lneto
+18 -1
View File
@@ -1,11 +1,26 @@
# Ignore files with no extensions trick:
# First ignore all.
*
# Unignore directories since they usually have no extensions
!*/
# Unignore all with extensions.
!*.*
!LICENSE
# Go workspaces.
go.work
go.work.sum
# Test binary, built with `go test -c`
*.test
# Test coverage profiles.
profiles/
# Dependency directories after running `go mod vendor`
vendor/
# Wifi credentials
*.credentials
# Binaries for programs and plugins
*.elf
*.uf2
@@ -15,7 +30,8 @@ vendor/
*.so
*.dylib
*.hex
*.wasm
/http-linux
# Profiling
*.pprof
@@ -31,6 +47,7 @@ vendor/
**__debug_bin*
# `__debug_bin` Debug binary generated in VSCode when using the built-in debugger.
*bin
/gen-binary-bench
/bridge
# IDE
+83 -11
View File
@@ -2,20 +2,24 @@
[![go.dev reference](https://pkg.go.dev/badge/github.com/soypat/lneto)](https://pkg.go.dev/github.com/soypat/lneto)
[![Go Report Card](https://goreportcard.com/badge/github.com/soypat/lneto)](https://goreportcard.com/report/github.com/soypat/lneto)
[![codecov](https://codecov.io/gh/soypat/lneto/branch/main/graph/badge.svg)](https://codecov.io/gh/soypat/lneto)
[![Go](https://github.com/soypat/lneto/actions/workflows/go.yml/badge.svg)](https://github.com/soypat/lneto/actions/workflows/go.yml)
[![sourcegraph](https://sourcegraph.com/github.com/soypat/lneto/-/badge.svg)](https://sourcegraph.com/github.com/soypat/lneto?badge)
[![Go](https://github.com/soypat/lneto/actions/workflows/ci.yaml/badge.svg)](https://github.com/soypat/lneto/actions/workflows/ci.yaml)
[![sourcegraph](https://sourcegraph.com/github.com/soypat/lneto/-/badge.svg)](https://github.com/soypat/lneto/network/dependents)
Userspace networking primitives.
Userspace networking primitives.
`lneto` is pronounced "L-net-oh", a.k.a. "El Neto"; a.k.a. "Don Networkio"; a.k.a "Neto, connector of worlds".
## Features
`lneto` provides the following features:
- Zero Operating System required. ***Zero***. All protocols (including TCP, excepting NTP) are time-independent.
- Explicit HAL needed. Lneto performs no `time.Sleep` nor `time.Now` calls unless configured by user.
- Zero scheduling required. No goroutines/channels use in Lneto. Can be run in event loop.
- Heapless packet processing
- [`httpraw`](https://github.com/soypat/lneto/tree/main/http/httpraw) is likely the most performant HTTP/1.1 processing package in the Go ecosystem. Based on [`fasthttp`](https://github.com/valyala/fasthttp) but simpler and more thoughtful memory use.
- [`httφ`](https://github.com/soypat/lneto/tree/main/http/httphi) - Heapless HTTP router and zero-copy response writing with Go's standard library API.
- Lean memory footprint
- HTTP header struct is 80 bytes with no runtime usage nor heap usage other than buffer
- Entire Ethernet+IPv4+UDP+DHCP+DNS+NTP stack in ~1kB.
- Entire Ethernet+IPv4+UDP+DHCP+DNS+NTP stack in ~2kB RAM.
- Empty go.mod file. No dependencies except for basic standard library packages such as `bytes`, `errors`, `io`.
- `net` only imported for `net.ErrClosed`.
- Can produce **very** small binaries. Ideal for embedded systems.
@@ -27,15 +31,26 @@ Userspace networking primitives.
## [`min-working-example`](./examples/min-working-example/) - Quick lneto showcase
Get a quick showcase of how lneto can be configured and how to get a TCP listening server up and running.
### Binary size comparisons
All examples include IPv4, ARP, ICMP, TCP and UDP functionality. Go and TinyGo default build flags used. **DNC**= Does Not Compile.
```sh
go run ./examples/gen/gen-binary-bench # generate table
```
| Program | Extra Protocols | Packet capture printing | amd64 Go | WASM Go | amd64 TinyGo | WASM TinyGo | Pico TinyGo |
|---|:---:|:---:|---|---|---|---|---|
| [Lneto MWE](./examples/min-working-example/) | DNS,NTP,DHCP | ✅ | 3.9MB | 4.5MB | 1.6MB | 1.2MB | 192kB |
| [Gvisor MWE w/ go-net](./examples/_import_examples/gvisor-mwe/) | None | ❌ | 6.6MB | 7.5MB | DNC | DNC | DNC |
## `xcurl` example
You may try lneto out on linux with the [xcurl example](./examples/xcurl/) which gets an HTTP page by doing all the low-level networking part using absolutely no standard library.
You may try lneto out on linux with the [xcurl example](./examples/xcurl/) which gets an HTTP page by doing all the low-level networking part using absolutely no standard library.
- DHCP client address lease
- ARP address resolution
- DNS address resolution of requested host
- HTTP over TCP/IPv4/Ethernet connection using
- NTP time check (optional)
- Print packet captures using lneto's [internet/pcap](./internet/pcap) package
- Print packet captures using lneto's [internet/pcap](./internet/pcap) package
- Optional fixed TCP source port with `-port`, useful when debugging raw-socket interactions with the host OS
See Developing section below for more information.
@@ -43,7 +58,7 @@ See Developing section below for more information.
lneto is currently being used primarily by embedded developers and those who need a lighter alternative to gVisor in terms of memory usage and binary size.
- [**soypat/cyw43439**](https://github.com/soypat/cyw43439): Enabling internet access on [Raspberry Pi Pico W](https://www.raspberrypi.com/products/raspberry-pi-pico/). See [`examples`](https://github.com/soypat/cyw43439/tree/main/examples).
- [**tinygo-org/espradio**](https://github.com/tinygo-rog/espradio): Enabling internet access on [Espressif's ESP32s](https://www.espressif.com/en/products/socs/esp32)
- [**tinygo-org/espradio**](https://github.com/tinygo-org/espradio): Enabling internet access on [Espressif's ESP32s](https://www.espressif.com/en/products/socs/esp32).
- [**TamaGo**](https://github.com/usbarmory/tamago): Enabling networking on Go baremetal projects. See [go-net project](https://github.com/usbarmory/go-net).
- [**netbird.io**](https://netbird.io/)(planned): Secure remote P2P access.
- [**soypat/lan8720**](https://github.com/soypat/lan8720): Enabling internet access with 100M ethernet PHY devices.
@@ -71,26 +86,62 @@ PASS
ok github.com/soypat/lneto/x/xnet 2.926s
```
## Protocol Support Matrix
> **Generated 2026-04-27.** This table may not always reflect the current state of the codebase.
> Allocation figures are derived from benchmark tests run on amd64.
> `—` denotes no standalone benchmark exists for that protocol; those packages follow the same zero-allocation design principle documented in the codebase.
| Protocol | RFC / Spec | Status | Package | Allocs/op | Notes |
|----------|-----------|:------:|---------|:---------:|-------|
| Ethernet II | IEEE 802.3 | ✅ | `ethernet` | 0 ¹ | Frame parsing, CRC-32 |
| ARP | RFC 826 | ✅ | `arp` | 0 ¹ | Request/reply, hardware address cache |
| IPv4 | RFC 791 | ✅ | `ipv4` | 0 ² | Frame parsing, ones-complement checksum |
| ICMPv4 | RFC 792 | ✅ | `ipv4/icmpv4` | — | Echo (ping) client handler |
| UDP | RFC 768 | ✅ | `udp` | — | Handler + thread-safe `Conn` |
| TCP | RFC 9293 | ✅ | `tcp` | 0 ²³ | Full state machine, SYN cookies, retransmit queue, `Conn`/`Listener` |
| DNS | RFC 1035 | ✅ | `dns` | — | Client (A/AAAA query) |
| DHCPv4 | RFC 2131 | ✅ | `dhcp/dhcpv4` | — | Client + Server |
| IPv4 Link-Local (APIPA) | RFC 3927 | ✅ | `ipv4/linklocal4` | 0 | Heapless claim-and-defend state machine for 169.254.x.x |
| NTP | RFC 5905 | ✅ | `ntp` | — | Client |
| NTS | RFC 8915 | ✅ | `x/nts` | — | Client + Server; key exchange over TLS 1.3, authenticated NTP. Requires caller-supplied AEAD_AES_SIV_CMAC_256 |
| mDNS | RFC 6762 | ✅ | `dns/mdns` | — | Client (service announcement + query) |
| HTTP/1.1 headers | RFC 9110, RFC 9112 | ✅ | `http/httpraw` | 2 ⁴ | Header parse/format; no field normalization |
| Ethernet PHY/MDIO | IEEE 802.3 cl.22/45 | ✅ | `phy` | — | Bare-metal PHY management via MDIO |
| IPv6 | RFC 8200 | ✅ | `ipv6` | — | Frame parsing and stack handling |
| ICMPv6 | RFC 4443 | ✅ | `ipv6/icmpv6` | — | Echo+NDP frame parsing and stack handling |
| DHCPv6 DNS Configuration | RFC 3646 | 🟡 | `dhcp/dhcpv6` | — | Recursive DNS server and domain search options parsed and exposed |
| DHCPv6 NTP Configuration | RFC 5908 | ✅ | `dhcp/dhcpv6` | — | NTP server address, multicast address, and FQDN suboptions parsed and exposed |
| DHCPv6 | RFC 8415 | 🟡 | `dhcp/dhcpv6` | — | Client IA_NA/IA_PD request handling and reconfigure-renew; no relay agent or dynamic server pools |
| TLS 1.3 | RFC 8446 | ❌ | — | — | Not implemented |
¹ `BenchmarkARPExchange` — full ARP request/response exchange over Ethernet: **0 B/op, 0 allocs/op**
² `BenchmarkTCPHandshake` — TCP 3-way handshake over Ethernet/IPv4: **0 B/op, 0 allocs/op**
³ `BenchmarkSYNCookie_Generate` / `BenchmarkSYNCookie_Validate` — SYN cookie generation and validation: **0 B/op, 0 allocs/op each**
`BenchmarkParseBytes` — HTTP/1.1 request header + body parsing: **240 B/op, 2 allocs/op**
## Packages
- `lneto`: Low-level Networking Operations, or "El Neto", the networking package. Zero copy network frame marshalling and unmarshalling.
- [`lneto/validation.go`](./validation.go): Packet validation utilities
- [`lneto/internet`](./internet): Userspace IP/TCP networking stack. This is where the magic happens. Integrates many of the listed packages.
- [`lneto/internet/pcap`](./internal/pcap): Packet capture and field breakdown utilities. Wireshark in the making.
- [`lneto/http/httpraw`](./http/httpraw/): Heapless HTTP header processing and validation. Does no implement header normalization.
- [`lneto/tcp`](./ntp): TCP implementation and low level logic.
- [`lneto/tcp`](./tcp): TCP implementation and low level logic.
- [`lneto/ipv4/linklocal4`](./ipv4/linklocal4): RFC 3927 IPv4 link-local (APIPA) address autoconfiguration. Heapless claim-and-defend state machine for self-assigned 169.254.x.x addresses when no DHCP server is available.
- [`lneto/dhcpv4`](./dhcpv4): DHCP version 4 protocol implementation and low level logic.
- [`lneto/dns`](./dns): DNS protocol implementation and low level logic.
- [`lneto/ntp`](./ntp): NTP implementation and low level logic. Includes NTP time primitives manipulation and conversion to Go native types.
- [`lneto/internal`](./internal): Lightweight and flexible ring buffer implementation and debugging primitives.
- [`lneto/x`](./x): Experimental packages.
- [`lneto/x/xnet`](./x/xnet/): `net` package like abstractions of stack implementations for ease of reuse. Still in testing phase and likely subject to breaking API change.
- [`lneto/x/nts`](./x/nts/): Network Time Security (RFC 8915). NTS-KE key exchange over TLS 1.3 and AEAD-authenticated NTP client/server. Ships no cryptography itself; the caller supplies the mandated `AEAD_AES_SIV_CMAC_256` `cipher.AEAD` implementation.
### Abstractions
The following interface is implemented by networking stack nodes and the stack themselves.
```go
type StackNode interface {
// Encapsulate receives a buffer the receiver must fill with data.
// Encapsulate receives a buffer the receiver must fill with data.
// The receiver's start byte is at carrierData[offsetToFrame].
Encapsulate(carrierData []byte, offsetToIP, offsetToFrame int) (int, error)
// Demux receives a buffer the receiver must decode and pass on to corresponding child StackNode(s).
@@ -116,11 +167,32 @@ go mod download github.com/soypat/lneto@latest
## Developing (linux)
When testing lneto through a Linux network interface, disable receive offloading if packets are unexpectedly dropped with checksum errors. Generic receive offload (GRO) can merge TCP frames before delivery to raw sockets without recalculating the full checksum.
```sh
sudo ethtool -K <network-interface> gro off
```
Depending on the interface and driver, other offloading features may also need to be disabled.
When running `xcurl` directly on a normal Linux interface (for example `-i wlan0` or `-i eth0`), lneto shares the host interface IP and MAC address. The Linux TCP stack also sees incoming packets for that IP. If xcurl's TCP handshake succeeds but the HTTP request is followed by a TCP RST from the server, the host kernel may have sent its own outbound RST for xcurl's raw TCP flow because no kernel socket owns that source port.
For a short validation test, use a fixed xcurl source port and temporarily drop kernel-generated outbound RSTs for that port:
```sh
go build -o examples/xcurl/xcurl ./examples/xcurl
sudo iptables -I OUTPUT -p tcp --sport 2300 --tcp-flags RST RST -j DROP
sudo ./examples/xcurl/xcurl -i <network-interface> -port 2300 -host example.com
sudo iptables -D OUTPUT -p tcp --sport 2300 --tcp-flags RST RST -j DROP
```
This is a debugging workaround, not the preferred operating mode. Prefer `-ihttp`, a TAP interface, a network namespace, or an otherwise isolated interface/IP when testing xcurl without host TCP stack interference.
- [`examples/httptap`](./examples/httptap) (linux only, root privilidges required) Program opens a TAP interface and assigns an IP address to it and exposes the interface via a HTTP interface. This program is run with root privilidges to facilitate debugging of lneto since no root privilidges are required to interact with the HTTP interface exposed.
- `POST http://127.0.0.1:7070/send`: Receives a POST with request body containing JSON string of data to send over TAP interface. Response contains only status code.
- `GET http://127.0.0.1:7070/recv`: Receives a GET request. Response contains a JSON string of oldest unread TAP interface packet. If string is empty then there is no more data to read.
- [`xcurl`](./examples/xcurl) Contains example of a application that uses lneto and can attach to a linux tap/bridge interface or a [httptap](./examples/httptap)(with -ihttp flag) to work. When using httptap can be run as non-root user to be debugged comfortably.
- [`xcurl`](./examples/xcurl) Contains example of a application that uses lneto and can attach to a linux tap/bridge interface or a [httptap](./examples/httptap)(with -ihttp flag) to work. When using httptap can be run as non-root user to be debugged comfortably.
- Example: `go run ./examples/xcurl -host google.com -ihttp`
### LLM Policy / AI Policy
@@ -218,4 +290,4 @@ The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
success
```
```
+5 -86
View File
@@ -2,8 +2,6 @@ package arp
import (
"bytes"
"log"
"slices"
"testing"
"github.com/soypat/lneto"
@@ -49,9 +47,9 @@ func TestHandler(t *testing.T) {
}
// Perform ARP exchange.
expectHWAddr := c2.ourHWAddr
expectHWAddr := c2.ourHWAddr[:]
queryAddr := c2.ourProtoAddr
err = c1.StartQuery(nil, queryAddr)
err = c1.StartQuery(queryAddr, false)
if err != nil {
t.Fatal(err)
}
@@ -85,11 +83,11 @@ func TestHandler(t *testing.T) {
if err != nil {
t.Fatal(err)
}
hwaddr, err := c1.QueryResult(queryAddr)
hwaddr, err := c1.CacheLookup(queryAddr)
if err != nil {
log.Fatal("expected query result:", err)
t.Fatal("expected query result:", err)
} else if !bytes.Equal(hwaddr, expectHWAddr) {
log.Fatalf("expected to get hwaddr %x!=%x", hwaddr, expectHWAddr)
t.Fatalf("expected to get hwaddr %x!=%x", hwaddr, expectHWAddr)
}
n, err = c1.Encapsulate(buf[:], -1, 0)
if err != nil {
@@ -105,85 +103,6 @@ func TestHandler(t *testing.T) {
}
}
func TestQueryCompaction(t *testing.T) {
var h Handler
startQuery := func(addr []byte) {
err := h.StartQuery(nil, addr)
if err != nil {
t.Fatal(err)
}
}
err := h.Reset(HandlerConfig{
HardwareAddr: []byte{0xde, 0xad, 0xbe, 0xef, 0x00, 0x00},
ProtocolAddr: []byte{192, 168, 1, 1},
MaxQueries: 5,
MaxPending: 1,
HardwareType: 1,
ProtocolType: ethernet.TypeIPv4,
})
if err != nil {
t.Fatal(err)
}
// Create multiple queries
addr1 := []byte{192, 168, 1, 10}
addr2 := []byte{192, 168, 1, 20}
addr3 := []byte{192, 168, 1, 30}
// Start 3 queries
startQuery(addr1)
startQuery(addr2)
startQuery(addr3)
if len(h.queries) != 3 {
t.Fatalf("expected 3 queries, got %d", len(h.queries))
}
// Discard the middle query (addr2)
if err := h.DiscardQuery(addr2); err != nil {
t.Fatal(err)
}
// Verify addr2 is marked as invalid
hasAddr2 := slices.ContainsFunc(h.queries, func(q queryResult) bool {
return bytes.Equal(q.protoaddr, addr2)
})
if hasAddr2 {
t.Fatal("addr2 query found after discard")
}
// Start new queries to trigger compaction
addr4 := []byte{192, 168, 1, 40}
addr5 := []byte{192, 168, 1, 50}
addr6 := []byte{192, 168, 1, 60}
startQuery(addr4)
startQuery(addr5)
startQuery(addr6)
// After compaction we should be left with 5 queries
expectedAddrs := [][]byte{addr1, addr3, addr4, addr5, addr6}
if len(h.queries) != len(expectedAddrs) {
t.Fatalf("after compaction: expected %d queries, got %d", len(expectedAddrs), len(h.queries))
}
gotAddrs := [][]byte{}
for _, q := range h.queries {
if !q.isInvalid() {
gotAddrs = append(gotAddrs, q.protoaddr)
} else {
t.Fatalf("invalid query %v should have been removed during compaction", q.protoaddr)
}
}
if !slices.EqualFunc(gotAddrs, expectedAddrs, bytes.Equal) {
t.Fatalf("expected %v, got %v", expectedAddrs, gotAddrs)
}
}
func validateARP(t *testing.T, buf []byte) {
t.Helper()
afrm, err := NewFrame(buf)
+167
View File
@@ -0,0 +1,167 @@
package arp
import (
"github.com/soypat/lneto"
"github.com/soypat/lneto/ethernet"
"github.com/soypat/lneto/internal"
)
type cache struct {
entries []entry
}
// entry is designed for compactness. size=class=24 bytes, same as a slice header on x86.
type entry struct {
addr [16]byte
mac [6]byte
age uint8
flags eflags
}
func (e *entry) use(mac [6]byte, proto []byte, flags eflags) {
e.flags = eflagInUse | flags
if copy(e.addr[:], proto) == 16 {
e.flags |= eflagIPv6
}
e.age = 0
e.mac = mac
}
func (e *entry) destroy() { *e = entry{} }
func (e *entry) put(frame, ourAddr []byte, ourMAC [6]byte, op Operation) (int, error) {
if len(ourMAC) != 6 {
return 0, lneto.ErrInvalidAddr
}
f, err := NewFrame(frame)
if err != nil {
return 0, err
}
f.SetHardware(1, 6)
f.SetOperation(op)
var n int
if e.flags&eflagIPv6 != 0 {
if len(ourAddr) != 16 {
return 0, lneto.ErrInvalidAddr
} else if len(frame) < sizeHeaderv6 {
return 0, lneto.ErrShortBuffer
}
f.SetProtocol(ethernet.TypeIPv6, 16)
hw, addr := f.Sender16()
*hw = ourMAC
copy(addr[:], ourAddr)
hw, addr = f.Target16()
copy(hw[:], e.mac[:])
copy(addr[:], e.addr[:])
n = sizeHeaderv6
} else {
if len(ourAddr) != 4 {
return 0, lneto.ErrInvalidAddr
}
f.SetProtocol(ethernet.TypeIPv4, 4)
hw, addr := f.Sender4()
*hw = ourMAC
copy(addr[:], ourAddr)
hw, addr = f.Target4()
copy(hw[:], e.mac[:])
copy(addr[:], e.addr[:])
n = sizeHeaderv4
}
return n, nil
}
func (flags eflags) hasAny(bits eflags) bool { return flags&bits != 0 }
type eflags uint8
// unset eflagInUse to signal the entry can be acquired for a new query.
const (
// eflagInUse set when in use. Discarded/unused entries have this bit unset.
eflagInUse eflags = 1 << iota
// set for IPv6 addressed entries.
eflagIPv6
// network device queried our address and we must respond to it.
// Both MAC and IP are valid in this case.
eflagPendingResponse
// user asked to query this address and query has yet to be answered. May or may not be sent.
eflagIncomplete
// user asked to query this address and the query has not been sent out yet.
// The MAC address is invalid in this case.
eflagIncompletePendingQuery
// eflagPriority set for prioritized cache entries. These entries are discarded last.
// i.e: set for user created queries, unset for external incoming network queries.
eflagPriority
// trigger callback, you know the drill.
eflagResolveTriggersCallback
)
func (c *cache) age() {
for i := range c.entries {
if c.entries[i].flags&eflagInUse != 0 && c.entries[i].age < 255 {
c.entries[i].age++
}
}
}
func (c *cache) reset(size int) {
internal.SliceReuse(&c.entries, size)
c.entries = c.entries[:cap(c.entries)] // maximize queries given allocation.
}
func (c *cache) getNextFlagged(entryHasFlags eflags) *entry {
for i := range c.entries {
flags := c.entries[i].flags
if flags&eflagInUse != 0 && flags.hasAny(entryHasFlags) {
return &c.entries[i]
}
}
return nil
}
func (c *cache) clearFlags(entryHasFlags, clrTheseFlagsIfMatch eflags) {
for i := range c.entries {
// Can clear flags on unused too, simpler.
if c.entries[i].flags&entryHasFlags != 0 {
c.entries[i].flags &^= clrTheseFlagsIfMatch
}
}
}
func (c *cache) Lookup(addr []byte) *entry {
n := len(addr)
for i := range c.entries {
if c.entries[i].flags&eflagInUse != 0 && internal.BytesEqual(c.entries[i].addr[:n], addr) {
return &c.entries[i]
}
}
return nil
}
// acquireNext gets next available entry for use. If all are in use evicts
// the oldest passive entry (learned from incoming requests) before touching
// active user queries or pending responses.
func (c *cache) acquireNext() *entry {
const priorityFlags = eflagPendingResponse | eflagIncomplete | eflagPriority
oldest, oldestPassive := 0, -1
for i := range c.entries {
if c.entries[i].flags&eflagInUse == 0 {
oldest = i
break
}
if !c.entries[i].flags.hasAny(priorityFlags) {
if oldestPassive < 0 || c.entries[i].age > c.entries[oldestPassive].age {
oldestPassive = i
}
}
if c.entries[i].age > c.entries[oldest].age {
oldest = i
}
}
if oldestPassive >= 0 && c.entries[oldest].flags&eflagInUse != 0 {
oldest = oldestPassive
}
c.age()
e := &c.entries[oldest]
e.destroy()
return e
}
+109 -180
View File
@@ -1,21 +1,21 @@
package arp
import (
"log/slog"
"github.com/soypat/lneto"
"github.com/soypat/lneto/ethernet"
"github.com/soypat/lneto/internal"
)
type Handler struct {
connID uint64
ourHWAddr []byte
ourProtoAddr []byte
htype uint16
protoType ethernet.Type
pendingResponse [][sizeHeaderv6]byte
queries []queryResult
connID uint64
cache cache
vld lneto.Validator
ourProtoAddr []byte
onresolve func(hw, proto []byte)
htype uint16
protoType ethernet.Type
ourHWAddr [6]byte
}
type HandlerConfig struct {
@@ -41,6 +41,10 @@ func (h *Handler) UpdateProtoAddr(protoAddr []byte) error {
return nil
}
func (h *Handler) SetOnResolveCallback(cb func(hwAddr, protoAddr []byte)) {
h.onresolve = cb
}
func (h *Handler) Reset(cfg HandlerConfig) error {
if len(cfg.HardwareAddr) == 0 || len(cfg.HardwareAddr) > 255 ||
len(cfg.ProtocolAddr) == 0 || len(cfg.ProtocolAddr) > 255 {
@@ -48,197 +52,120 @@ func (h *Handler) Reset(cfg HandlerConfig) error {
} else if cfg.MaxQueries <= 0 || cfg.MaxPending <= 0 {
return lneto.ErrInvalidConfig
}
if cfg.HardwareType != 1 || cfg.ProtocolType != ethernet.TypeIPv4 && cfg.ProtocolType != ethernet.TypeIPv6 {
return lneto.ErrUnsupported // We only support common for now.
}
*h = Handler{
connID: h.connID + 1,
ourHWAddr: h.ourHWAddr[:0],
ourProtoAddr: h.ourProtoAddr[:0],
htype: cfg.HardwareType,
protoType: cfg.ProtocolType,
pendingResponse: h.pendingResponse[:0],
queries: h.queries[:0],
connID: h.connID + 1,
ourHWAddr: h.ourHWAddr,
ourProtoAddr: h.ourProtoAddr[:0],
htype: cfg.HardwareType,
protoType: cfg.ProtocolType,
cache: h.cache,
}
h.ourHWAddr = append(h.ourHWAddr, cfg.HardwareAddr...)
h.cache.reset(cfg.MaxPending + cfg.MaxQueries)
h.ourHWAddr = [6]byte(cfg.HardwareAddr)
h.ourProtoAddr = append(h.ourProtoAddr, cfg.ProtocolAddr...)
if cap(h.pendingResponse) < cfg.MaxPending {
h.pendingResponse = make([][52]byte, cfg.MaxPending)[:0]
}
if cap(h.queries) < cfg.MaxQueries {
h.queries = make([]queryResult, cfg.MaxQueries)[:0]
}
return nil
}
type queryResult struct {
protoaddr []byte
hwaddr []byte
dstHw []byte
querysent bool
// inc tracks amount of times the query survived compaction.
// The queries with higher inc will be discarded first.
inc uint16
}
func (qr *queryResult) destroy() {
*qr = queryResult{protoaddr: qr.protoaddr[:0], hwaddr: qr.hwaddr[:0]}
}
func (qr *queryResult) response() []byte {
if len(qr.hwaddr) == 0 {
return nil
}
return qr.hwaddr[:]
}
func (qr *queryResult) isInvalid() bool { return len(qr.protoaddr) == 0 }
// AbortPending drops pending queries and incoming requests.
func (h *Handler) AbortPending() {
h.pendingResponse = h.pendingResponse[:0]
h.queries = h.queries[:0]
h.cache.clearFlags(eflagPendingResponse|eflagIncomplete, eflagInUse)
}
func (h *Handler) expectSize() int {
return sizeHeader + 2*len(h.ourHWAddr) + 2*len(h.ourProtoAddr)
// CacheSeed pre-populates the cache with a known proto→hardware mapping, making it
// immediately resolvable via [Handler.CacheLookup] without an ARP exchange.
// Seeded entries are evicted before active user queries when the cache is full.
func (h *Handler) CacheSeed(protoAddr, hwAddr []byte) error {
if len(hwAddr) != 6 {
return lneto.ErrUnsupported
}
e := h.cache.acquireNext()
e.use([6]byte(hwAddr), protoAddr, 0)
return nil
}
func (h *Handler) QueryResult(protoAddr []byte) (hwAddr []byte, err error) {
for i := range h.queries {
if internal.BytesEqual(protoAddr, h.queries[i].protoaddr) {
if !h.queries[i].querysent {
return nil, errQueryPending
}
mac := h.queries[i].response()
if mac == nil {
return nil, errQueryPending
}
return mac, nil
}
// CacheLookup returns the hardware address for protoAddr if it is resolved in the cache.
// Returns [errQueryPending] if a query is in flight, [errQueryNotFound] if no entry exists.
func (h *Handler) CacheLookup(protoAddr []byte) (hwAddr []byte, err error) {
e := h.cache.Lookup(protoAddr)
if e == nil {
return nil, errQueryNotFound
} else if e.flags.hasAny(eflagIncomplete) {
return nil, errQueryPending
}
return nil, errQueryNotFound
return e.mac[:], nil
}
func (h *Handler) DiscardQuery(protoAddr []byte) error {
for i := range h.queries {
q := &h.queries[i]
if internal.BytesEqual(protoAddr, q.protoaddr) {
q.destroy()
return nil
}
// CacheRemove cancels a pending query or evicts a cached entry for protoAddr.
func (h *Handler) CacheRemove(protoAddr []byte) error {
e := h.cache.Lookup(protoAddr)
if e == nil {
return errQueryNotFound
}
return errQueryNotFound
}
func (h *Handler) compactQueries() {
validOff := 0
maxIdx := -1
maxInc := uint16(0)
for i := 0; i < len(h.queries); i++ {
discard := h.queries[i].isInvalid()
if !discard {
h.queries[i].inc++
if h.queries[i].inc > maxInc {
maxInc = h.queries[i].inc
maxIdx = i
}
if i != validOff {
// We swap the queries here so that when `StartQuery` extends
// queries slice, we don't have sharing of the internal structures.
// An alternative would be to zero things, however that would incur
// an allocation cost.
h.queries[validOff], h.queries[i] = h.queries[i], h.queries[validOff]
}
validOff++
}
}
if validOff == len(h.queries) && maxIdx >= 0 {
h.queries[maxIdx].destroy() // Destroy oldest query if unable to compact.
}
h.queries = h.queries[:validOff]
e.destroy()
return nil
}
// StartQuery queues a query to perform over ARP for the protocol address `proto`.
// The user can additionally specify an dstHWAddr to write query result to on completion.
// If dstHWAddr is nil then query still occurs but no external buffer is written on query completion.
// dstHWAddr must be zeroed out (invalid MAC).
func (h *Handler) StartQuery(dstHWAddr, proto []byte) error {
if len(h.queries) == cap(h.queries) {
h.compactQueries()
if len(h.queries) == cap(h.queries) {
return lneto.ErrExhausted // Should never fail.
}
}
// Use [Handler.SetOnResolveCallback] to asynchronously set an ARP request result.
func (h *Handler) StartQuery(proto []byte, triggerCallback bool) error {
if len(proto) != len(h.ourProtoAddr) {
return lneto.ErrMismatchLen
} else if dstHWAddr != nil && len(dstHWAddr) != len(h.ourHWAddr) {
return lneto.ErrMismatchLen
} else if dstHWAddr != nil && !internal.IsZeroed(dstHWAddr...) {
return lneto.ErrInvalidConfig
}
q := internal.SliceReclaim(&h.queries)
*q = queryResult{
protoaddr: append(q.protoaddr[:0], proto...),
hwaddr: q.hwaddr[:0],
dstHw: dstHWAddr,
e := h.cache.acquireNext()
e.use([6]byte{}, proto, eflagIncomplete|eflagIncompletePendingQuery|eflagPriority)
if triggerCallback {
e.flags |= eflagResolveTriggersCallback
}
return nil
}
func (h *Handler) Encapsulate(carrierData []byte, offsetToIP, offsetToFrame int) (int, error) {
func (h *Handler) Encapsulate(carrierData []byte, _, offsetToFrame int) (int, error) {
b := carrierData[offsetToFrame:]
n := h.expectSize()
if len(b) < n {
return 0, errShortARP
afrm, err := h.newframe(b)
if err != nil {
return 0, err
}
if len(h.pendingResponse) > 0 {
// pop frame.
afrm, _ := NewFrame(h.pendingResponse[len(h.pendingResponse)-1][:])
h.pendingResponse = h.pendingResponse[:len(h.pendingResponse)-1]
afrm.SetOperation(OpReply)
afrm.SwapTargetSender()
hwsender, _ := afrm.Sender()
copy(hwsender, h.ourHWAddr)
n := copy(b, afrm.Clip().RawData())
tgt, _ := afrm.Target()
trySetEthernetDst(carrierData[:offsetToFrame], tgt)
return n, nil
op := OpReply
e := h.cache.getNextFlagged(eflagPendingResponse) // Prioritize responses.
if e == nil {
e = h.cache.getNextFlagged(eflagIncompletePendingQuery)
if e == nil {
return 0, nil // No action to perform
}
e.flags &^= eflagIncompletePendingQuery
op = OpRequest
} else {
e.flags &^= eflagPendingResponse
}
for i := range h.queries {
if h.queries[i].isInvalid() || h.queries[i].querysent {
continue
}
h.queries[i].querysent = true
afrm, _ := NewFrame(b)
afrm.SetHardware(h.htype, uint8(len(h.ourHWAddr)))
afrm.SetProtocol(h.protoType, uint8(len(h.ourProtoAddr)))
afrm.SetOperation(OpRequest)
hwSender, protoSender := afrm.Sender()
copy(hwSender, h.ourHWAddr)
copy(protoSender, h.ourProtoAddr)
hwTarget, protoTarget := afrm.Target()
copy(protoTarget, h.queries[i].protoaddr)
for j := range hwTarget {
hwTarget[j] = 0
}
// Write Request or Reply, depending on which entry we got.
n, err := e.put(b, h.ourProtoAddr, h.ourHWAddr, op)
if err != nil {
return 0, err
}
switch op {
case OpRequest:
broadcast := ethernet.BroadcastAddr()
trySetEthernetDst(carrierData[:offsetToFrame], broadcast[:])
return n, nil
case OpReply:
tgt, _ := afrm.Target()
trySetEthernetDst(carrierData[:offsetToFrame], tgt)
}
return 0, nil
return n, nil
}
func (h *Handler) Demux(ethFrame []byte, frameOffset int) error {
if len(h.pendingResponse) == cap(h.pendingResponse) {
return lneto.ErrExhausted
}
b := ethFrame[frameOffset:]
afrm, err := NewFrame(b)
afrm, err := h.newframe(b)
if err != nil {
return err
}
var vld lneto.Validator
afrm.ValidateSize(&vld)
if vld.HasError() {
return vld.ErrPop()
afrm.ValidateSize(&h.vld)
if h.vld.HasError() {
return h.vld.ErrPop()
}
htype, hlen := afrm.Hardware()
if htype != h.htype || int(hlen) != len(h.ourHWAddr) {
@@ -254,35 +181,37 @@ func (h *Handler) Demux(ethFrame []byte, frameOffset int) error {
if !internal.BytesEqual(protoaddr, h.ourProtoAddr) {
return nil // Not for us.
}
h.pendingResponse = h.pendingResponse[:len(h.pendingResponse)+1] // Extend pending buffer.
copy(h.pendingResponse[len(h.pendingResponse)-1][:], afrm.buf) // Set pending buffer.
hw, proto := afrm.Sender()
e := h.cache.acquireNext()
e.use([6]byte(hw), proto, eflagPendingResponse)
case OpReply:
hwaddr, protoaddr := afrm.Sender()
for i := range h.queries {
q := &h.queries[i]
mac := q.response()
if mac == nil && internal.BytesEqual(q.protoaddr, protoaddr) {
q.hwaddr = append(q.hwaddr, hwaddr...)
if q.dstHw != nil {
if !internal.IsZeroed(q.dstHw...) {
internal.LogAttrs(nil, slog.LevelError, "race-condition:ARP-reused-buffer")
}
// External write to user buffer.
// Copy data and free up this memory.
copy(q.dstHw, hwaddr)
q.inc = 10000
}
return nil
}
e := h.cache.Lookup(protoaddr)
if e == nil {
return nil
}
copy(e.mac[:], hwaddr)
e.flags &^= eflagIncomplete | eflagIncompletePendingQuery
if e.flags.hasAny(eflagResolveTriggersCallback) && h.onresolve != nil {
h.onresolve(e.mac[:], protoaddr)
}
default:
return errARPUnsupported
}
return nil
}
func (h *Handler) newframe(b []byte) (Frame, error) {
f, err := NewFrame(b)
if err != nil {
return f, err
} else if h.protoType == ethernet.TypeIPv6 && len(b) < sizeHeaderv6 {
return f, lneto.ErrMismatch
}
return f, nil
}
func trySetEthernetDst(ethFrame []byte, dst []byte) {
if len(ethFrame) >= 14 {
copy(ethFrame[:6], dst)
+4 -4
View File
@@ -17,9 +17,9 @@ const _Operation_name = "requestreply"
var _Operation_index = [...]uint8{0, 7, 12}
func (i Operation) String() string {
i -= 1
if i >= Operation(len(_Operation_index)-1) {
return "Operation(" + strconv.FormatInt(int64(i+1), 10) + ")"
idx := int(i) - 1
if i < 1 || idx >= len(_Operation_index)-1 {
return "Operation(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _Operation_name[_Operation_index[i]:_Operation_index[i+1]]
return _Operation_name[_Operation_index[idx]:_Operation_index[idx+1]]
}
+37
View File
@@ -0,0 +1,37 @@
package lneto
import (
"runtime"
"time"
)
// Flag return values for a BackoffStrategy.
const (
BackoffFlagGosched = time.Duration(-1)
BackoffFlagNop = time.Duration(-2)
)
// BackoffStrategy is the abstraction of a backoff strategy for retrying an operation.
// It returns the amount of time to sleep for or a flag value:
// - Returns [BackoffFlagNop]: Signal no yielding function should be called.
// Useful for when the BackoffStrategy implements its own yield.
// - Returns [BackoffFlagGosched]: Signal [runtime.Gosched] should be called.
//
// Despite the name(changes welcome) consecutiveBackoffs starts at 0 and
// increments by 1 every time the operation is retried.
// See internal/backoff.go for a implementation example.
type BackoffStrategy func(consecutiveBackoffs uint) (sleepOrFlag time.Duration)
// Do applies the backoff strategy by calling backoff(consecutiveBackoffs)
// and then the corresponding yield function for the returned value. See [BackoffStrategy].
func (backoff BackoffStrategy) Do(consecutiveBackoffs uint) {
sleep := backoff(consecutiveBackoffs)
switch sleep {
case BackoffFlagNop:
// No yield. Yield implemented by backoff.
case BackoffFlagGosched:
runtime.Gosched()
default:
time.Sleep(sleep)
}
}
+2 -2
View File
@@ -37,12 +37,12 @@ type StackNode interface {
// SetFlagPending(flagPending func(numPendingEncapsulations int))
}
//go:generate stringer -type=IPProto,errGeneric -linecomment -output stringers.go .
// IPProto represents the IP protocol number.
type IPProto uint8
// IP protocol numbers.
//
//go:generate stringer -type=IPProto,errGeneric -linecomment -output stringers.go .
const (
IPProtoHopByHop IPProto = 0 // IPv6 Hop-by-Hop Option [RFC8200]
IPProtoICMP IPProto = 1 // ICMP [RFC792]
+4 -8
View File
@@ -153,10 +153,7 @@ func (c *Client) Encapsulate(carrierData []byte, offsetToIP, offsetToFrame int)
numOpts += n
n, _ = EncodeOption(opts[numOpts:], OptParameterRequestList, defaultParamReqList...)
numOpts += n
maxlen := len(dst)
if maxlen > math.MaxUint16 {
maxlen = math.MaxUint16
}
maxlen := min(len(dst), math.MaxUint16)
n, _ = EncodeOption16(opts[numOpts:], OptMaximumMessageSize, uint16(maxlen))
numOpts += n
if c.reqIP.valid {
@@ -369,12 +366,11 @@ func (d *Client) DNSServerFirst() netip.Addr {
return d.dns[0]
}
func (d *Client) SubnetPrefix() netip.Prefix {
func (d *Client) SubnetPrefix() ipv4.Prefix {
if !d.offer.valid {
return netip.Prefix{}
return ipv4.Prefix{}
}
m, _ := netip.AddrFrom4(d.offer.addr).Prefix(int(d.SubnetCIDRBits()))
return m
return ipv4.PrefixFrom(d.offer.addr, d.SubnetCIDRBits())
}
func (d *Client) SubnetCIDRBits() uint8 {
@@ -1,10 +1,10 @@
package dhcpv4
import (
"net/netip"
"testing"
"github.com/soypat/lneto/internal"
"github.com/soypat/lneto/ipv4"
)
func TestClientServer(t *testing.T) {
@@ -29,7 +29,7 @@ func TestClientServer(t *testing.T) {
}
sv.Configure(ServerConfig{
ServerAddr: svAddr,
Subnet: netip.PrefixFrom(netip.AddrFrom4(svAddr), 24),
Subnet: ipv4.PrefixFrom(svAddr, 24),
})
// CLIENT DISCOVER.
assertClState(StateInit)
+3
View File
@@ -44,6 +44,9 @@ type Frame struct {
buf []byte
}
// RawData returns the underlying frame buffer.
func (frm Frame) RawData() []byte { return frm.buf }
// OptionsPayload returns the options portion of the DHCP frame. May be zero lengthed.
func (frm Frame) OptionsPayload() []byte {
return frm.buf[OptionsOffset:]
+42 -23
View File
@@ -4,18 +4,18 @@ import (
"encoding/binary"
"errors"
"fmt"
"net/netip"
"github.com/soypat/lneto"
"github.com/soypat/lneto/internal"
"github.com/soypat/lneto/ipv4"
)
var errOptionNotFit = errors.New("DHCPv4: options dont fit")
type Server struct {
connID uint64
nextAddr netip.Addr
prefix netip.Prefix
nextAddr [4]byte
subnet ipv4.Prefix
hosts map[[36]byte]serverEntry
vld lneto.Validator
pending int
@@ -35,7 +35,7 @@ type ServerConfig struct {
// DNS server address advertised to clients. Zero value omits the option.
DNS [4]byte
// Subnet defines the network prefix for address allocation and subnet mask responses.
Subnet netip.Prefix
Subnet ipv4.Prefix
// LeaseSeconds is the lease duration. Zero defaults to 3600.
LeaseSeconds uint32
// Port is the server listening port. Zero defaults to DefaultServerPort.
@@ -63,10 +63,9 @@ type serverEntry struct {
// The connection ID is incremented on each call to invalidate existing connections.
// The hosts map is reused across calls to avoid reallocation.
func (sv *Server) Configure(cfg ServerConfig) error {
svAddr := netip.AddrFrom4(cfg.ServerAddr)
if !cfg.Subnet.IsValid() {
return errors.New("dhcpv4 server: invalid subnet")
} else if !cfg.Subnet.Contains(svAddr) {
} else if !cfg.Subnet.Contains(cfg.ServerAddr) {
return errors.New("dhcpv4 server: server address outside subnet")
}
port := cfg.Port
@@ -90,10 +89,10 @@ func (sv *Server) Configure(cfg ServerConfig) error {
siaddr: cfg.ServerAddr,
gwaddr: cfg.Gateway,
dns: cfg.DNS,
prefix: cfg.Subnet,
subnet: cfg.Subnet,
port: port,
leaseSeconds: lease,
nextAddr: svAddr,
nextAddr: cfg.Subnet.Next(cfg.ServerAddr),
hosts: hosts,
}
return nil
@@ -153,7 +152,20 @@ func (sv *Server) Demux(carrierData []byte, frameOffset int) error {
var client serverEntry
var clientExists bool
if len(clientID) == 0 {
client, clientIDRaw, clientExists = sv.getClientByIP(*dfrm.CIAddr())
// No explicit client identifier: use chaddr as the stable lookup key.
// This lets the server correlate Discover→Offer→Request even when
// ciaddr=0.0.0.0 (client has no IP yet), which is the normal case
// for first-time lease acquisition (RFC 2131 §4.3.1).
chaddr := *dfrm.CHAddrAs6()
copy(clientIDRaw[:], chaddr[:])
client, clientExists = sv.getClient(clientIDRaw)
if !clientExists {
// Fallback: look up by ciaddr for clients that did send ciaddr.
ciaddr := *dfrm.CIAddr()
if ciaddr != ([4]byte{}) {
client, clientIDRaw, clientExists = sv.getClientByIP(ciaddr)
}
}
} else {
copy(clientIDRaw[:], clientID)
client, clientExists = sv.getClient(clientIDRaw)
@@ -263,8 +275,8 @@ func (sv *Server) Encapsulate(carrierData []byte, offsetToIP, offsetToFrame int)
n, _ = EncodeOption(optBuf[nopt:], OptRouter, sv.gwaddr[:]...)
nopt += n
}
if sv.prefix.IsValid() {
bits := uint(sv.prefix.Bits())
if sv.subnet.IsValid() {
bits := uint(sv.subnet.Bits())
mask := ^uint32(0) << (32 - bits)
var maskBuf [4]byte
binary.BigEndian.PutUint32(maskBuf[:], mask)
@@ -302,6 +314,14 @@ func (sv *Server) Encapsulate(carrierData []byte, offsetToIP, offsetToFrame int)
if err != nil {
return 0, err
}
// Per RFC 2131 §4.1: unicast Offer/Ack to chaddr because the client
// does not yet have the offered IP, so no ARP entry exists.
// The Ethernet layer sets dst=gwmac before calling us; overwrite it
// here so the frame reaches the client via its hardware address.
// offsetToIP==14 means the Ethernet header is at carrierData[0:14].
if offsetToIP >= 14 {
copy(carrierData[offsetToIP-14:offsetToIP-8], client.hwaddr[:])
}
}
client.state = futureState
@@ -317,18 +337,18 @@ func (sv *Server) Encapsulate(carrierData []byte, offsetToIP, offsetToFrame int)
// it is preferred. Returns false if the pool is exhausted.
func (sv *Server) allocAddr(reqAddr []byte) ([4]byte, bool) {
if len(reqAddr) == 4 {
candidate := netip.AddrFrom4([4]byte(reqAddr))
if sv.prefix.Contains(candidate) && candidate.As4() != sv.siaddr && !sv.isAddrAssigned(candidate) {
return candidate.As4(), true
candidate := [4]byte(reqAddr)
if sv.subnet.Contains(candidate) && candidate != sv.siaddr && !sv.isAddrAssigned(candidate) {
return candidate, true
}
}
sv.nextAddr = sv.nextAddr.Next()
if !sv.prefix.Contains(sv.nextAddr) {
return [4]byte{}, false
}
// Reject broadcast address (all host bits set).
a := sv.nextAddr.As4()
hostBits := uint(32 - sv.prefix.Bits())
a := sv.nextAddr
sv.nextAddr = sv.subnet.Next(sv.nextAddr)
if sv.nextAddr == sv.siaddr {
sv.nextAddr = sv.subnet.Next(sv.nextAddr)
}
hostBits := uint(32 - sv.subnet.Bits())
hostMask := ^uint32(0) >> (32 - hostBits)
if binary.BigEndian.Uint32(a[:])&hostMask == hostMask {
return [4]byte{}, false
@@ -336,10 +356,9 @@ func (sv *Server) allocAddr(reqAddr []byte) ([4]byte, bool) {
return a, true
}
func (sv *Server) isAddrAssigned(addr netip.Addr) bool {
a4 := addr.As4()
func (sv *Server) isAddrAssigned(addr [4]byte) bool {
for _, v := range sv.hosts {
if v.addr == a4 {
if v.addr == addr {
return true
}
}
+252
View File
@@ -0,0 +1,252 @@
package dhcpv4
// Tests covering the AP-mode DHCP server fixes:
// 1. Client lookup by chaddr when no ClientID option is present.
// 2. Ciaddr fallback lookup when chaddr lookup finds nothing.
// 3. Ethernet dst is patched to chaddr on Offer/Ack when Encapsulate is
// called with offsetToIP >= 14.
import (
"testing"
"github.com/soypat/lneto/ipv4"
)
// doraNoClientID runs a complete DORA exchange using raw DHCP frames
// (no ClientID option) with the given chaddr. The client uses only chaddr
// as its identity, matching normal AP-mode behaviour where mobile clients
// don't send OptClientIdentifier.
func doraNoClientID(t *testing.T, sv *Server, chaddr [6]byte, xid uint32) (assignedIP [4]byte) {
t.Helper()
var cl Client
err := cl.BeginRequest(xid, RequestConfig{
ClientHardwareAddr: chaddr,
// Intentionally no ClientID forces server to key on chaddr.
})
if err != nil {
t.Fatalf("BeginRequest: %v", err)
}
var buf [1024]byte
// DISCOVER
n, err := cl.Encapsulate(buf[:], -1, 0)
if err != nil || n == 0 {
t.Fatalf("discover encapsulate: n=%d err=%v", n, err)
}
if err = sv.Demux(buf[:n], 0); err != nil {
t.Fatalf("discover demux: %v", err)
}
// OFFER
n, err = sv.Encapsulate(buf[:], -1, 0)
if err != nil || n == 0 {
t.Fatalf("offer encapsulate: n=%d err=%v", n, err)
}
frm, _ := NewFrame(buf[:n])
assignedIP = *frm.YIAddr()
if err = cl.Demux(buf[:n], 0); err != nil {
t.Fatalf("offer demux: %v", err)
}
// REQUEST
n, err = cl.Encapsulate(buf[:], -1, 0)
if err != nil || n == 0 {
t.Fatalf("request encapsulate: n=%d err=%v", n, err)
}
if err = sv.Demux(buf[:n], 0); err != nil {
t.Fatalf("request demux: %v", err)
}
// ACK
n, err = sv.Encapsulate(buf[:], -1, 0)
if err != nil || n == 0 {
t.Fatalf("ack encapsulate: n=%d err=%v", n, err)
}
if err = cl.Demux(buf[:n], 0); err != nil {
t.Fatalf("ack demux: %v", err)
}
if cl.State() != StateBound {
t.Errorf("want StateBound, got %s", cl.State())
}
return assignedIP
}
// TestServerChaddrLookup_NoClientID verifies the server can complete a full DORA
// cycle when the client sends no OptClientIdentifier option. Before the fix the
// MsgRequest Demux call would fail with "request for non existing client" because
// the server tried to look up the entry by ciaddr (0.0.0.0) instead of chaddr.
func TestServerChaddrLookup_NoClientID(t *testing.T) {
svAddr := [4]byte{192, 168, 4, 1}
var sv Server
sv.Configure(ServerConfig{
ServerAddr: svAddr,
Gateway: svAddr,
Subnet: ipv4.PrefixFrom(svAddr, 24),
})
chaddr := [6]byte{0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff}
ip := doraNoClientID(t, &sv, chaddr, 0x12345678)
if ip[0] != 192 || ip[1] != 168 || ip[2] != 4 {
t.Errorf("unexpected subnet in assigned IP %v", ip)
}
}
// TestServerChaddrLookup_MultipleClientsNoClientID verifies that multiple clients
// without ClientID each get a unique address and successfully reach StateBound.
func TestServerChaddrLookup_MultipleClientsNoClientID(t *testing.T) {
svAddr := [4]byte{192, 168, 4, 1}
var sv Server
sv.Configure(ServerConfig{
ServerAddr: svAddr,
Gateway: svAddr,
Subnet: ipv4.PrefixFrom(svAddr, 24),
})
addrs := make([][4]byte, 4)
for i := range addrs {
chaddr := [6]byte{0x00, 0x11, 0x22, 0x33, 0x44, byte(i + 1)}
addrs[i] = doraNoClientID(t, &sv, chaddr, uint32(i+1))
}
for i := range addrs {
for j := i + 1; j < len(addrs); j++ {
if addrs[i] == addrs[j] {
t.Errorf("clients %d and %d got same address %v", i, j, addrs[i])
}
}
}
}
// TestServerCiaddrFallback exercises the ciaddr fallback path in Demux.
// An entry is registered under an explicit non-MAC ClientID. A subsequent
// RELEASE arrives with no ClientID (so chaddr lookup misses) but with
// ciaddr=assignedIP, which should let the server find and remove the entry.
func TestServerCiaddrFallback(t *testing.T) {
svAddr := [4]byte{192, 168, 4, 1}
chaddr := [6]byte{0xca, 0xfe, 0x00, 0x00, 0x01, 0x01}
const xid = uint32(0xaabbccdd)
var sv Server
sv.Configure(ServerConfig{
ServerAddr: svAddr,
Subnet: ipv4.PrefixFrom(svAddr, 24),
})
// DORA with an explicit ClientID — server keys the entry by that string.
var cl Client
if err := cl.BeginRequest(xid, RequestConfig{
ClientHardwareAddr: chaddr,
ClientID: "device-id",
}); err != nil {
t.Fatalf("BeginRequest: %v", err)
}
assignedIP := doraWithClient(t, &sv, &cl)
// RELEASE: no ClientID, ciaddr = assignedIP.
// chaddr lookup fails (entry is keyed by "device-id"), so server must
// fall back to getClientByIP.
var relBuf [512]byte
rfrm, _ := NewFrame(relBuf[:])
rfrm.ClearHeader()
rfrm.SetOp(OpRequest)
rfrm.SetHardware(1, 6, 0)
rfrm.SetXID(xid)
*rfrm.CIAddr() = assignedIP
copy(rfrm.CHAddrAs6()[:], chaddr[:])
rfrm.SetMagicCookie(MagicCookie)
opts := rfrm.OptionsPayload()
n, _ := EncodeOption(opts, OptMessageType, byte(MsgRelease))
opts[n] = byte(OptEnd)
n++
if err := sv.Demux(relBuf[:OptionsOffset+n], 0); err != nil {
t.Fatalf("release demux: %v", err)
}
if len(sv.hosts) != 0 {
t.Errorf("expected 0 hosts after release, got %d", len(sv.hosts))
}
}
// TestServerOfferPatchesEthernetDst verifies that Encapsulate overwrites the
// first 6 bytes of the carrier (Ethernet dst) with the client's chaddr when
// offsetToIP >= 14.
func TestServerOfferPatchesEthernetDst(t *testing.T) {
svAddr := [4]byte{192, 168, 4, 1}
clChaddr := [6]byte{0x02, 0x03, 0x04, 0x05, 0x06, 0x07}
var sv Server
sv.Configure(ServerConfig{
ServerAddr: svAddr,
Subnet: ipv4.PrefixFrom(svAddr, 24),
})
var cl Client
cl.BeginRequest(0x11223344, RequestConfig{ClientHardwareAddr: clChaddr})
var buf [1024]byte
n, err := cl.Encapsulate(buf[:], -1, 0)
if err != nil || n == 0 {
t.Fatalf("discover encapsulate: n=%d err=%v", n, err)
}
if err = sv.Demux(buf[:n], 0); err != nil {
t.Fatalf("discover demux: %v", err)
}
// Carrier: 14 bytes Ethernet header, then a minimal IPv4 header marker.
var carrier [1024]byte
carrier[14] = 0x45 // IPv4 version+IHL so SetIPAddrs succeeds.
offerN, err := sv.Encapsulate(carrier[:], 14, 14+20+8)
if err != nil || offerN == 0 {
t.Fatalf("offer encapsulate: n=%d err=%v", offerN, err)
}
var got [6]byte
copy(got[:], carrier[0:6])
if got != clChaddr {
t.Errorf("Ethernet dst: got %v, want %v", got, clChaddr)
}
}
// doraWithClient runs a complete DORA exchange using the given pre-configured
// Client and returns the assigned IP.
func doraWithClient(t *testing.T, sv *Server, cl *Client) (assignedIP [4]byte) {
t.Helper()
var buf [1024]byte
n, err := cl.Encapsulate(buf[:], -1, 0)
if err != nil || n == 0 {
t.Fatalf("discover encapsulate: n=%d err=%v", n, err)
}
if err = sv.Demux(buf[:n], 0); err != nil {
t.Fatalf("discover demux: %v", err)
}
n, err = sv.Encapsulate(buf[:], -1, 0)
if err != nil || n == 0 {
t.Fatalf("offer encapsulate: n=%d err=%v", n, err)
}
frm, _ := NewFrame(buf[:n])
assignedIP = *frm.YIAddr()
if err = cl.Demux(buf[:n], 0); err != nil {
t.Fatalf("offer demux: %v", err)
}
n, err = cl.Encapsulate(buf[:], -1, 0)
if err != nil || n == 0 {
t.Fatalf("request encapsulate: n=%d err=%v", n, err)
}
if err = sv.Demux(buf[:n], 0); err != nil {
t.Fatalf("request demux: %v", err)
}
n, err = sv.Encapsulate(buf[:], -1, 0)
if err != nil || n == 0 {
t.Fatalf("ack encapsulate: n=%d err=%v", n, err)
}
if err = cl.Demux(buf[:n], 0); err != nil {
t.Fatalf("ack demux: %v", err)
}
if cl.State() != StateBound {
t.Errorf("want StateBound, got %s", cl.State())
}
return assignedIP
}
@@ -1,14 +1,15 @@
package dhcpv4
import (
"net/netip"
"testing"
"github.com/soypat/lneto/ipv4"
)
func testServerConfig(svAddr [4]byte) ServerConfig {
return ServerConfig{
ServerAddr: svAddr,
Subnet: netip.PrefixFrom(netip.AddrFrom4(svAddr), 24),
Subnet: ipv4.PrefixFrom(svAddr, 24),
}
}
@@ -106,7 +107,7 @@ func TestServerMultipleClients(t *testing.T) {
}
// All assigned addresses must be unique.
for i := 0; i < nClients; i++ {
for i := range nClients {
for j := i + 1; j < nClients; j++ {
if assignedAddrs[i] == assignedAddrs[j] {
t.Errorf("clients %d and %d got same address %v", i, j, assignedAddrs[i])
@@ -123,7 +124,7 @@ func TestServerSequentialAddressAllocation(t *testing.T) {
sv.Configure(testServerConfig(svAddr))
// Build raw DISCOVER frames for two clients.
for i := byte(0); i < 2; i++ {
for i := range byte(2) {
var buf [512]byte
frm, _ := NewFrame(buf[:])
frm.ClearHeader()
@@ -147,7 +148,7 @@ func TestServerSequentialAddressAllocation(t *testing.T) {
// Encapsulate both OFFERs and verify addresses are in expected range.
var seen [2][4]byte
for i := byte(0); i < 2; i++ {
for i := range byte(2) {
var buf [512]byte
n, err := sv.Encapsulate(buf[:], -1, 0)
if err != nil {
@@ -180,7 +181,7 @@ func TestServerOfferContainsOptions(t *testing.T) {
ServerAddr: svAddr,
Gateway: gwAddr,
DNS: dnsAddr,
Subnet: netip.PrefixFrom(netip.AddrFrom4(svAddr), 24),
Subnet: ipv4.PrefixFrom(svAddr, 24),
LeaseSeconds: 7200,
})
@@ -295,7 +296,7 @@ func TestServerConfigValidation(t *testing.T) {
}
err = sv.Configure(ServerConfig{
ServerAddr: [4]byte{10, 0, 0, 1},
Subnet: netip.PrefixFrom(netip.AddrFrom4([4]byte{192, 168, 1, 0}), 24),
Subnet: ipv4.PrefixFrom([4]byte{192, 168, 1, 0}, 24),
})
if err == nil {
t.Error("expected error for server address outside subnet")
@@ -106,10 +106,11 @@ const _Op_name = "undefinedrequestreply"
var _Op_index = [...]uint8{0, 9, 16, 21}
func (i Op) String() string {
if i >= Op(len(_Op_index)-1) {
idx := int(i) - 0
if i < 0 || idx >= len(_Op_index)-1 {
return "Op(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _Op_name[_Op_index[i]:_Op_index[i+1]]
return _Op_name[_Op_index[idx]:_Op_index[idx+1]]
}
func _() {
// An "invalid array index" compiler error signifies that the constant values have changed.
@@ -131,10 +132,11 @@ const _MessageType_name = "undefineddiscoverofferrequestdeclineacknakreleaseinfo
var _MessageType_index = [...]uint8{0, 9, 17, 22, 29, 36, 39, 42, 49, 55}
func (i MessageType) String() string {
if i >= MessageType(len(_MessageType_index)-1) {
idx := int(i) - 0
if i < 0 || idx >= len(_MessageType_index)-1 {
return "MessageType(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _MessageType_name[_MessageType_index[i]:_MessageType_index[i+1]]
return _MessageType_name[_MessageType_index[idx]:_MessageType_index[idx+1]]
}
func _() {
// An "invalid array index" compiler error signifies that the constant values have changed.
@@ -155,9 +157,9 @@ const _ClientState_name = "initselectingrequestingboundrenewingrebindinginit-reb
var _ClientState_index = [...]uint8{0, 4, 13, 23, 28, 36, 45, 56, 65}
func (i ClientState) String() string {
i -= 1
if i >= ClientState(len(_ClientState_index)-1) {
return "ClientState(" + strconv.FormatInt(int64(i+1), 10) + ")"
idx := int(i) - 1
if i < 1 || idx >= len(_ClientState_index)-1 {
return "ClientState(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _ClientState_name[_ClientState_index[i]:_ClientState_index[i+1]]
return _ClientState_name[_ClientState_index[idx]:_ClientState_index[idx+1]]
}
+633
View File
@@ -0,0 +1,633 @@
package dhcpv6
import (
"encoding/binary"
"net"
"net/netip"
"github.com/soypat/lneto"
"github.com/soypat/lneto/dns"
"github.com/soypat/lneto/internal"
)
// Default caps for the repeated, server-supplied options the client retains.
// A DHCPv6 server may place arbitrarily many of each in a single message
// (RFC 8415), so the client bounds them to prevent a malicious or
// misconfigured server from driving unbounded allocation (an OOM vector).
const (
defaultMaxDNSServers = 4
defaultMaxDomainSearch = 6
defaultMaxNTPServers = 4
defaultMaxNTPMulticastServers = 2
defaultMaxNTPServerNames = 4
defaultMaxDelegatedPrefixes = 4
)
// Limits bounds how many entries of each repeated, server-supplied option the
// client retains from a single exchange. The backing arrays are sized to these
// caps once in [Client.BeginRequest] and reused across resets, so parsing a
// server message performs no further allocation and a hostile server cannot
// grow client memory without bound. A zero (or negative) field selects its
// default; see the default* constants.
type Limits struct {
MaxDNSServers int // OptDNSServers addresses.
MaxDomainSearch int // OptDomainList search names.
MaxNTPServers int // OptNTPServer suboption 1 addresses.
MaxNTPMulticastServers int // OptNTPServer suboption 2 addresses.
MaxNTPServerNames int // OptNTPServer suboption 3 FQDNs.
MaxDelegatedPrefixes int // OptIAPD/OptIAPrefix delegated prefixes.
}
// withDefaults returns a copy of l with every non-positive field replaced by
// its default cap, so the resolved limits are always usable.
func (l Limits) withDefaults() Limits {
if l.MaxDNSServers <= 0 {
l.MaxDNSServers = defaultMaxDNSServers
}
if l.MaxDomainSearch <= 0 {
l.MaxDomainSearch = defaultMaxDomainSearch
}
if l.MaxNTPServers <= 0 {
l.MaxNTPServers = defaultMaxNTPServers
}
if l.MaxNTPMulticastServers <= 0 {
l.MaxNTPMulticastServers = defaultMaxNTPMulticastServers
}
if l.MaxNTPServerNames <= 0 {
l.MaxNTPServerNames = defaultMaxNTPServerNames
}
if l.MaxDelegatedPrefixes <= 0 {
l.MaxDelegatedPrefixes = defaultMaxDelegatedPrefixes
}
return l
}
// RequestConfig holds the parameters for starting a DHCPv6 exchange.
type RequestConfig struct {
// ClientHardwareAddr is the client's Ethernet MAC address.
// It is used to construct the client DUID-LL and IAID.
ClientHardwareAddr [6]byte
// Limits bounds how many of each repeated server-supplied option the client
// stores. The zero value selects safe defaults for every field.
Limits Limits
}
// DelegatedPrefix is an IPv6 prefix delegated by a DHCPv6 server.
type DelegatedPrefix struct {
Prefix netip.Prefix
PreferredLifetime uint32
ValidLifetime uint32
}
// Client is a stateful DHCPv6 client implementing the [lneto.StackNode] interface.
// It manages the Solicit→Advertise→Request→Reply exchange (RFC 8415 §18).
//
// Typical usage:
//
// var cl Client
// cl.BeginRequest(xid, RequestConfig{ClientHardwareAddr: mac})
// // drive Encapsulate / Demux calls via the network stack
type Client struct {
connID uint64
state ClientState
xid uint32 // lower 24 bits used
// duid is the client's DUID-LL. Client owns the backing array; it is set
// once from the MAC in BeginRequest and carried across resets unchanged.
duid []byte
// serverDUID is the selected server's DUID. Client owns the backing array;
// it is cleared (len=0) on reset so capacity is reused without allocation.
serverDUID []byte
// dns accumulates DNS recursive name server addresses (OptDNSServers).
// Client owns the backing array; cleared on reset, capacity reused.
dns []netip.Addr
// domainSearch accumulates DNS domain search names (OptDomainList).
// Client owns the backing array; cleared on reset, capacity reused.
domainSearch []dns.Name
// ntps accumulates NTP server addresses (OptNTPServer, suboption 1).
// Client owns the backing array; cleared on reset, capacity reused.
ntps []netip.Addr
// ntpMulticast accumulates NTP multicast addresses (OptNTPServer, suboption 2).
// Client owns the backing array; cleared on reset, capacity reused.
ntpMulticast []netip.Addr
// ntpNames accumulates NTP server FQDNs (OptNTPServer, suboption 3).
// Client owns the backing array; cleared on reset, capacity reused.
ntpNames []dns.Name
// delegatedPrefixes accumulates IA_PD prefixes (OptIAPD/OptIAPrefix).
// Client owns the backing array; cleared on reset, capacity reused.
delegatedPrefixes []DelegatedPrefix
assignedAddr [16]byte
assignedAddrValid bool
// iaid is derived from the first 4 bytes of the client MAC.
iaid [4]byte
// IA_NA timers from the server's Advertise/Reply.
t1, t2 uint32
preferredLifetime uint32
validLifetime uint32
// IA_PD timers from the server's Advertise/Reply.
pdT1, pdT2 uint32
reconfigureMsg MsgType
reconfigureOK bool
clientMAC [6]byte
// limits is the resolved (defaults-applied) cap on each repeated option.
// It is set in BeginRequest and preserved across resets so the bounded
// backing arrays are reused without reallocation.
limits Limits
// auxbuf is a scratch buffer used during Encapsulate to avoid allocations.
auxbuf [128]byte
}
// BeginRequest initialises a new DHCPv6 exchange with the given 24-bit transaction ID.
// It must be called before any Encapsulate or Demux calls.
func (c *Client) BeginRequest(xid uint32, cfg RequestConfig) error {
if xid == 0 {
return lneto.ErrInvalidConfig
} else if c.state != StateInit && c.state != 0 {
return lneto.ErrInvalidConfig
} else if internal.IsZeroed(cfg.ClientHardwareAddr[:]...) {
return lneto.ErrInvalidConfig
}
c.clientMAC = cfg.ClientHardwareAddr
c.iaid = [4]byte(cfg.ClientHardwareAddr[:4])
c.xid = xid & 0xFFFFFF
c.limits = cfg.Limits.withDefaults()
c.reset()
// Size the per-option backing arrays to their caps once, here, so the
// parse path on the network-facing Demux never allocates and stored
// entries stay bounded across the connection's lifetime.
c.dns = ensureAddrCap(c.dns, c.limits.MaxDNSServers)
c.ntps = ensureAddrCap(c.ntps, c.limits.MaxNTPServers)
c.ntpMulticast = ensureAddrCap(c.ntpMulticast, c.limits.MaxNTPMulticastServers)
c.domainSearch = ensureNameCap(c.domainSearch, c.limits.MaxDomainSearch)
c.ntpNames = ensureNameCap(c.ntpNames, c.limits.MaxNTPServerNames)
c.delegatedPrefixes = ensurePrefixCap(c.delegatedPrefixes, c.limits.MaxDelegatedPrefixes)
c.duid = AppendDUIDLL(c.duid[:0], cfg.ClientHardwareAddr)
c.state = StateInit
return nil
}
// ensureAddrCap returns s truncated to length 0 with capacity at least n,
// allocating a new backing array only when the existing one is too small.
func ensureAddrCap(s []netip.Addr, n int) []netip.Addr {
if cap(s) < n {
return make([]netip.Addr, 0, n)
}
return s[:0]
}
func ensureNameCap(s []dns.Name, n int) []dns.Name {
if cap(s) < n {
return make([]dns.Name, 0, n)
}
return s[:0]
}
func ensurePrefixCap(s []DelegatedPrefix, n int) []DelegatedPrefix {
if cap(s) < n {
return make([]DelegatedPrefix, 0, n)
}
return s[:0]
}
// Reset clears the DHCPv6 exchange state and invalidates stack registrations.
func (c *Client) Reset() { c.reset() }
// reset clears exchange state while preserving slice backing arrays and the
// connection ID is incremented to invalidate any existing stack registrations.
func (c *Client) reset() {
*c = Client{
connID: c.connID + 1,
xid: c.xid,
clientMAC: c.clientMAC,
iaid: c.iaid,
limits: c.limits,
duid: c.duid,
serverDUID: c.serverDUID[:0],
dns: c.dns[:0],
domainSearch: c.domainSearch[:0],
ntps: c.ntps[:0],
ntpMulticast: c.ntpMulticast[:0],
ntpNames: c.ntpNames[:0],
delegatedPrefixes: c.delegatedPrefixes[:0],
}
}
// Encapsulate writes the next outgoing DHCPv6 message into carrierData[offsetToFrame:].
// Returns the number of bytes written or 0 if there is nothing to send in the current state.
// Implements [lneto.StackNode].
func (c *Client) Encapsulate(carrierData []byte, _, offsetToFrame int) (int, error) {
if c.isClosed() {
return 0, net.ErrClosed
}
dst := carrierData[offsetToFrame:]
if len(dst) < OptionsOffset+128 {
return 0, lneto.ErrShortBuffer
}
frm, err := NewFrame(dst)
if err != nil {
return 0, err
}
var numOpts int
var nextState ClientState
switch c.state {
case StateInit:
frm.SetMsgType(MsgSolicit)
frm.SetTransactionID(c.xid)
n, _ := EncodeOption(dst[OptionsOffset+numOpts:], OptClientID, c.duid...)
numOpts += n
n, _ = EncodeOption(dst[OptionsOffset+numOpts:], OptReconfAccept)
numOpts += n
n, _ = EncodeOptionIANA(dst[OptionsOffset+numOpts:], c.iaid, 0, 0, nil)
numOpts += n
n, _ = EncodeOption(dst[OptionsOffset+numOpts:], OptORO, defaultOptRequestList...)
numOpts += n
n, _ = EncodeOption16(dst[OptionsOffset+numOpts:], OptElapsedTime, 0)
numOpts += n
nextState = StateSoliciting
case StateRequesting:
frm.SetMsgType(MsgRequest)
frm.SetTransactionID(c.xid)
n, _ := EncodeOption(dst[OptionsOffset+numOpts:], OptClientID, c.duid...)
numOpts += n
n, _ = EncodeOption(dst[OptionsOffset+numOpts:], OptServerID, c.serverDUID...)
numOpts += n
n, _ = EncodeOption(dst[OptionsOffset+numOpts:], OptReconfAccept)
numOpts += n
auxN, _ := EncodeOptionIAAddr(c.auxbuf[:], c.assignedAddr, 0, 0)
n, _ = EncodeOptionIANA(dst[OptionsOffset+numOpts:], c.iaid, 0, 0, c.auxbuf[:auxN])
numOpts += n
n, _ = EncodeOption(dst[OptionsOffset+numOpts:], OptORO, defaultOptRequestList...)
numOpts += n
n, _ = EncodeOption16(dst[OptionsOffset+numOpts:], OptElapsedTime, 0)
numOpts += n
nextState = StateRequesting // retransmittable; Demux(Reply) advances to Bound
case StateRenewing:
frm.SetMsgType(MsgRenew)
frm.SetTransactionID(c.xid)
n, _ := EncodeOption(dst[OptionsOffset+numOpts:], OptClientID, c.duid...)
numOpts += n
n, _ = EncodeOption(dst[OptionsOffset+numOpts:], OptServerID, c.serverDUID...)
numOpts += n
auxN, _ := EncodeOptionIAAddr(c.auxbuf[:], c.assignedAddr, 0, 0)
n, _ = EncodeOptionIANA(dst[OptionsOffset+numOpts:], c.iaid, 0, 0, c.auxbuf[:auxN])
numOpts += n
n, _ = EncodeOption16(dst[OptionsOffset+numOpts:], OptElapsedTime, 0)
numOpts += n
nextState = StateRenewing
case StateRebinding:
frm.SetMsgType(MsgRebind)
frm.SetTransactionID(c.xid)
n, _ := EncodeOption(dst[OptionsOffset+numOpts:], OptClientID, c.duid...)
numOpts += n
// No OptServerID in Rebind (RFC 8415 §18.2.5).
auxN, _ := EncodeOptionIAAddr(c.auxbuf[:], c.assignedAddr, 0, 0)
n, _ = EncodeOptionIANA(dst[OptionsOffset+numOpts:], c.iaid, 0, 0, c.auxbuf[:auxN])
numOpts += n
n, _ = EncodeOption16(dst[OptionsOffset+numOpts:], OptElapsedTime, 0)
numOpts += n
nextState = StateRebinding
default:
return 0, nil // StateSoliciting, StateBound, or uninitialised.
}
c.state = nextState
return OptionsOffset + numOpts, nil
}
// Demux processes an incoming DHCPv6 message at carrierData[frameOffset:].
// It validates the transaction ID and advances the client state machine on success.
// Implements [lneto.StackNode].
func (c *Client) Demux(carrierData []byte, frameOffset int) error {
if c.isClosed() {
return net.ErrClosed
}
frm, err := NewFrame(carrierData[frameOffset:])
if err != nil {
return err
}
if frm.MsgType() == MsgReconfigure {
return c.handleReconfigure(frm)
}
if frm.TransactionID() != c.xid {
return lneto.ErrMismatch
}
msgType := frm.MsgType()
var nextState ClientState
switch c.state {
case StateSoliciting:
if msgType != MsgAdvertise {
return lneto.ErrPacketDrop
}
nextState = StateRequesting
case StateRequesting, StateRenewing, StateRebinding:
if msgType != MsgReply {
return lneto.ErrPacketDrop
}
nextState = StateBound
default:
return lneto.ErrPacketDrop
}
if err := c.setOptions(frm); err != nil {
return err
}
c.state = nextState
return nil
}
func (c *Client) handleReconfigure(frm Frame) error {
if !c.state.HasIP() {
return lneto.ErrPacketDrop
}
var clientOK, serverOK bool
var reconf MsgType
err := frm.ForEachOption(func(_ int, code OptCode, data []byte) error {
switch code {
case OptClientID:
clientOK = internal.BytesEqual(data, c.duid)
case OptServerID:
serverOK = len(c.serverDUID) == 0 || internal.BytesEqual(data, c.serverDUID)
case OptReconfMsg:
if len(data) != 1 {
return lneto.ErrInvalidLengthField
}
reconf = MsgType(data[0])
}
return nil
})
if err != nil {
return err
}
if !clientOK || !serverOK {
return lneto.ErrMismatch
}
c.reconfigureMsg = reconf
c.reconfigureOK = true
if reconf != MsgRenew {
return lneto.ErrUnsupported
}
c.state = StateRenewing
return nil
}
// setOptions parses all DHCPv6 options in frm and stores relevant values.
func (c *Client) setOptions(frm Frame) error {
return frm.ForEachOption(func(_ int, code OptCode, data []byte) error {
switch code {
case OptServerID:
c.serverDUID = append(c.serverDUID[:0], data...)
case OptIANA:
c.parseIANA(data)
case OptIAPD:
c.parseIAPD(data)
case OptDNSServers:
if len(c.dns) > 0 || len(data)%16 != 0 {
break // skip if already populated or malformed
}
for i := 0; i+16 <= len(data) && len(c.dns) < c.limits.MaxDNSServers; i += 16 {
c.dns = append(c.dns, netip.AddrFrom16([16]byte(data[i:i+16])))
}
case OptDomainList:
c.parseDomainSearch(data)
case OptNTPServer:
c.parseNTPServer(data)
}
return nil
})
}
func (c *Client) parseIAPD(data []byte) {
if len(c.delegatedPrefixes) > 0 {
return
}
if len(data) < 12 {
return
}
t1 := binary.BigEndian.Uint32(data[4:8])
t2 := binary.BigEndian.Uint32(data[8:12])
for ptr := 12; ptr+4 <= len(data); {
subCode := OptCode(binary.BigEndian.Uint16(data[ptr:]))
subLen := int(binary.BigEndian.Uint16(data[ptr+2:]))
if ptr+4+subLen > len(data) {
break
}
if subCode == OptIAPrefix && subLen >= 25 && len(c.delegatedPrefixes) < c.limits.MaxDelegatedPrefixes {
sub := data[ptr+4 : ptr+4+subLen]
bits := int(sub[8])
prefix := netip.PrefixFrom(netip.AddrFrom16([16]byte(sub[9:25])), bits)
if prefix.IsValid() {
c.delegatedPrefixes = append(c.delegatedPrefixes, DelegatedPrefix{
Prefix: prefix.Masked(),
PreferredLifetime: binary.BigEndian.Uint32(sub[:4]),
ValidLifetime: binary.BigEndian.Uint32(sub[4:8]),
})
}
}
ptr += 4 + subLen
}
if len(c.delegatedPrefixes) > 0 {
c.pdT1 = t1
c.pdT2 = t2
}
}
func (c *Client) parseNTPServer(data []byte) {
if len(c.ntps) > 0 || len(c.ntpMulticast) > 0 || len(c.ntpNames) > 0 {
return
}
for ptr := 0; ptr+4 <= len(data); {
subCode := binary.BigEndian.Uint16(data[ptr:])
subLen := int(binary.BigEndian.Uint16(data[ptr+2:]))
if ptr+4+subLen > len(data) {
break
}
subData := data[ptr+4 : ptr+4+subLen]
switch subCode {
case 1:
if subLen == 16 && len(c.ntps) < c.limits.MaxNTPServers {
c.ntps = append(c.ntps, netip.AddrFrom16([16]byte(data[ptr+4:ptr+20])))
}
case 2:
if subLen == 16 && len(c.ntpMulticast) < c.limits.MaxNTPMulticastServers {
c.ntpMulticast = append(c.ntpMulticast, netip.AddrFrom16([16]byte(data[ptr+4:ptr+20])))
}
case 3:
idx := len(c.ntpNames)
if idx >= c.limits.MaxNTPServerNames {
break // suboption switch: cap reached, drop further names.
}
if idx < cap(c.ntpNames) {
c.ntpNames = c.ntpNames[:idx+1]
} else {
c.ntpNames = append(c.ntpNames, dns.Name{})
}
next, err := c.ntpNames[idx].Decode(subData, 0)
if err != nil || next != uint16(len(subData)) {
c.ntpNames[idx].Reset()
c.ntpNames = c.ntpNames[:idx]
}
}
ptr += 4 + subLen
}
}
func (c *Client) parseDomainSearch(data []byte) {
if len(c.domainSearch) > 0 {
return
}
for off := uint16(0); off < uint16(len(data)) && len(c.domainSearch) < c.limits.MaxDomainSearch; {
idx := len(c.domainSearch)
if idx < cap(c.domainSearch) {
c.domainSearch = c.domainSearch[:idx+1]
} else {
c.domainSearch = append(c.domainSearch, dns.Name{})
}
next, err := c.domainSearch[idx].Decode(data, off)
if err != nil || next <= off {
c.domainSearch[idx].Reset()
c.domainSearch = c.domainSearch[:idx]
return
}
off = next
}
}
// parseIANA processes the payload of an OptIANA option, extracting the
// assigned address and lease timers from any embedded OptIAAddr sub-option.
func (c *Client) parseIANA(data []byte) {
if len(data) < 12 {
return
}
if [4]byte(data[:4]) != c.iaid {
return // not our Identity Association
}
t1 := binary.BigEndian.Uint32(data[4:8])
t2 := binary.BigEndian.Uint32(data[8:12])
// Iterate sub-options manually (same 4-byte TLV format).
ptr := 12
for ptr+4 <= len(data) {
subCode := OptCode(binary.BigEndian.Uint16(data[ptr:]))
subLen := int(binary.BigEndian.Uint16(data[ptr+2:]))
if ptr+4+subLen > len(data) {
break // malformed sub-option; stop safely
}
if subCode == OptIAAddr && subLen >= 24 {
sub := data[ptr+4 : ptr+4+subLen]
c.assignedAddr = [16]byte(sub[:16])
c.assignedAddrValid = true
c.preferredLifetime = binary.BigEndian.Uint32(sub[16:20])
c.validLifetime = binary.BigEndian.Uint32(sub[20:24])
}
ptr += 4 + subLen
}
if c.assignedAddrValid {
c.t1 = t1
c.t2 = t2
}
}
func (c *Client) isClosed() bool { return c.state == 0 || c.xid == 0 }
// State returns the current client state.
func (c *Client) State() ClientState { return c.state }
// LastReconfigure returns the last accepted Reconfigure message target.
func (c *Client) LastReconfigure() (MsgType, bool) { return c.reconfigureMsg, c.reconfigureOK }
// AssignedAddr returns the IPv6 address assigned by the server and whether it is valid.
func (c *Client) AssignedAddr() ([16]byte, bool) { return c.assignedAddr, c.assignedAddrValid }
// RebindingSeconds returns the IA_NA T2 timer from the server.
func (c *Client) RebindingSeconds() uint32 { return c.t2 }
// RenewalSeconds returns the IA_NA T1 timer from the server.
func (c *Client) RenewalSeconds() uint32 { return c.t1 }
// PreferredLifetimeSeconds returns the preferred lifetime of the assigned address.
func (c *Client) PreferredLifetimeSeconds() uint32 { return c.preferredLifetime }
// ValidLifetimeSeconds returns the valid lifetime of the assigned address.
func (c *Client) ValidLifetimeSeconds() uint32 { return c.validLifetime }
// PrefixDelegationRebindingSeconds returns the IA_PD T2 timer from the server.
func (c *Client) PrefixDelegationRebindingSeconds() uint32 { return c.pdT2 }
// PrefixDelegationRenewalSeconds returns the IA_PD T1 timer from the server.
func (c *Client) PrefixDelegationRenewalSeconds() uint32 { return c.pdT1 }
// AppendDelegatedPrefixes appends delegated IPv6 prefixes received from the server to dst.
func (c *Client) AppendDelegatedPrefixes(dst []DelegatedPrefix) []DelegatedPrefix {
return append(dst, c.delegatedPrefixes...)
}
// NumDelegatedPrefixes returns the number of delegated IPv6 prefixes received.
func (c *Client) NumDelegatedPrefixes() int { return len(c.delegatedPrefixes) }
// AppendDNSServers appends the DNS server addresses received from the server to dst.
func (c *Client) AppendDNSServers(dst []netip.Addr) []netip.Addr { return append(dst, c.dns...) }
// NumDNSServers returns the number of DNS server addresses received.
func (c *Client) NumDNSServers() int { return len(c.dns) }
// AppendDomainSearch appends the DNS domain search names received from the server to dst.
func (c *Client) AppendDomainSearch(dst []dns.Name) []dns.Name { return append(dst, c.domainSearch...) }
// NumDomainSearch returns the number of DNS domain search names received.
func (c *Client) NumDomainSearch() int { return len(c.domainSearch) }
// AppendNTPServers appends the NTP server addresses received from the server to dst.
func (c *Client) AppendNTPServers(dst []netip.Addr) []netip.Addr { return append(dst, c.ntps...) }
// NumNTPServers returns the number of NTP server addresses received.
func (c *Client) NumNTPServers() int { return len(c.ntps) }
// AppendNTPMulticastServers appends NTP multicast addresses received from the server to dst.
func (c *Client) AppendNTPMulticastServers(dst []netip.Addr) []netip.Addr {
return append(dst, c.ntpMulticast...)
}
// NumNTPMulticastServers returns the number of NTP multicast addresses received.
func (c *Client) NumNTPMulticastServers() int { return len(c.ntpMulticast) }
// AppendNTPServerNames appends NTP server FQDNs received from the server to dst.
func (c *Client) AppendNTPServerNames(dst []dns.Name) []dns.Name { return append(dst, c.ntpNames...) }
// NumNTPServerNames returns the number of NTP server FQDNs received.
func (c *Client) NumNTPServerNames() int { return len(c.ntpNames) }
// ConnectionID returns a pointer to the client's connection ID.
// The value increments on each reset; callers should discard registrations when it changes.
// Implements [lneto.StackNode].
func (c *Client) ConnectionID() *uint64 { return &c.connID }
// LocalPort returns the DHCPv6 client port (546).
// Implements [lneto.StackNode].
func (c *Client) LocalPort() uint16 { return ClientPort }
// Protocol returns the IP protocol number for UDP.
// Implements [lneto.StackNode].
func (c *Client) Protocol() uint64 { return uint64(lneto.IPProtoUDP) }
// defaultOptRequestList is the ORO payload (RFC 8415 §21.7) listing the options
// the client wants the server to include in its reply.
var defaultOptRequestList = []byte{
byte(OptDNSServers >> 8), byte(OptDNSServers), // 23
byte(OptDomainList >> 8), byte(OptDomainList), // 24
byte(OptNTPServer >> 8), byte(OptNTPServer), // 56
}
+720
View File
@@ -0,0 +1,720 @@
package dhcpv6
import (
"encoding/binary"
"net/netip"
"testing"
"github.com/soypat/lneto/dns"
)
// writeOpt6 encodes a single DHCPv6 option into dst using the 4-byte TLV header
// (2-byte code + 2-byte length) and returns the total bytes written.
// Used by tests to build server frames without depending on the stub EncodeOption.
func writeOpt6(dst []byte, code OptCode, data ...byte) int {
binary.BigEndian.PutUint16(dst[0:2], uint16(code))
binary.BigEndian.PutUint16(dst[2:4], uint16(len(data)))
copy(dst[4:], data)
return 4 + len(data)
}
// buildServerFrame constructs a minimal DHCPv6 Advertise or Reply frame
// containing OptServerID, and an OptIANA with an embedded OptIAAddr.
func buildServerFrame(msgType MsgType, xid uint32, serverDUID []byte, iaid [4]byte, addr [16]byte) []byte {
// IAAddr payload: addr(16) + preferred(4) + valid(4)
iaAddrPayload := make([]byte, 24)
copy(iaAddrPayload[:16], addr[:])
binary.BigEndian.PutUint32(iaAddrPayload[16:20], 3600)
binary.BigEndian.PutUint32(iaAddrPayload[20:24], 7200)
// Encode IAAddr as an option.
iaAddrOpt := make([]byte, 4+len(iaAddrPayload))
writeOpt6(iaAddrOpt, OptIAAddr, iaAddrPayload...)
// IA_NA payload: IAID(4) + T1(4) + T2(4) + IAAddr option.
iaNAPayload := make([]byte, 12+len(iaAddrOpt))
copy(iaNAPayload[:4], iaid[:])
binary.BigEndian.PutUint32(iaNAPayload[4:8], 1800)
binary.BigEndian.PutUint32(iaNAPayload[8:12], 3600)
copy(iaNAPayload[12:], iaAddrOpt)
buf := make([]byte, 1024)
buf[0] = byte(msgType)
buf[1] = byte(xid >> 16)
buf[2] = byte(xid >> 8)
buf[3] = byte(xid)
n := OptionsOffset
n += writeOpt6(buf[n:], OptServerID, serverDUID...)
n += writeOpt6(buf[n:], OptIANA, iaNAPayload...)
return buf[:n]
}
// TestFrameForEachOption verifies that ForEachOption correctly delivers
// the option code and data to the callback for a hand-built frame.
func TestFrameForEachOption(t *testing.T) {
buf := make([]byte, OptionsOffset+8)
buf[0] = byte(MsgSolicit)
buf[3] = 42 // XID low byte
n := writeOpt6(buf[OptionsOffset:], OptClientID, 'A', 'B')
frm, err := NewFrame(buf[:OptionsOffset+n])
if err != nil {
t.Fatal(err)
}
var gotCode OptCode
var gotData []byte
err = frm.ForEachOption(func(_ int, code OptCode, data []byte) error {
gotCode = code
gotData = append(gotData[:0], data...)
return nil
})
if err != nil {
t.Fatal("ForEachOption:", err)
}
if gotCode != OptClientID {
t.Errorf("option code: want %d (OptClientID), got %d", OptClientID, gotCode)
}
if string(gotData) != "AB" {
t.Errorf("option data: want %q, got %q", "AB", gotData)
}
}
// TestFrameValidateSize verifies that ValidateSize returns an error when an option's
// declared length extends past the end of the buffer.
func TestFrameValidateSize(t *testing.T) {
buf := make([]byte, OptionsOffset+6)
buf[0] = byte(MsgSolicit)
// Option code = OptClientID, claimed length = 100, actual data = 2 bytes.
binary.BigEndian.PutUint16(buf[OptionsOffset:], uint16(OptClientID))
binary.BigEndian.PutUint16(buf[OptionsOffset+2:], 100)
buf[OptionsOffset+4] = 'A'
buf[OptionsOffset+5] = 'B'
frm, err := NewFrame(buf)
if err != nil {
t.Fatal(err)
}
if err := frm.ValidateSize(); err == nil {
t.Error("ValidateSize: want error for truncated option, got nil")
}
}
// TestClientSolicitRequest exercises the full four-step DHCPv6 exchange:
// Solicit → (fabricated) Advertise → Request → (fabricated) Reply → Bound.
func TestClientSolicitRequest(t *testing.T) {
const xid = 0x112233
clientMAC := [6]byte{0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF}
serverDUID := []byte{0, 3, 0, 1, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66}
assignedAddr := [16]byte{0x20, 0x01, 0x0d, 0xb8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}
iaid := [4]byte{clientMAC[0], clientMAC[1], clientMAC[2], clientMAC[3]}
var cl Client
if err := cl.BeginRequest(xid, RequestConfig{ClientHardwareAddr: clientMAC}); err != nil {
t.Fatal("BeginRequest:", err)
}
if cl.State() != StateInit {
t.Fatalf("initial state: want StateInit, got %v", cl.State())
}
buf := make([]byte, 1024)
// CLIENT: send Solicit.
n, err := cl.Encapsulate(buf, -1, 0)
if err != nil {
t.Fatal("Encapsulate (Solicit):", err)
}
if n == 0 {
t.Fatal("Encapsulate (Solicit): wrote 0 bytes")
}
if !frameHasOption(t, buf[:n], OptReconfAccept) {
t.Fatal("Solicit missing Reconfigure Accept option")
}
if cl.State() != StateSoliciting {
t.Fatalf("after Solicit: want StateSoliciting, got %v", cl.State())
}
// SERVER: fabricated Advertise.
advFrame := buildServerFrame(MsgAdvertise, xid, serverDUID, iaid, assignedAddr)
if err := cl.Demux(advFrame, 0); err != nil {
t.Fatal("Demux (Advertise):", err)
}
if cl.State() != StateRequesting {
t.Fatalf("after Advertise: want StateRequesting, got %v", cl.State())
}
// CLIENT: send Request.
n, err = cl.Encapsulate(buf, -1, 0)
if err != nil {
t.Fatal("Encapsulate (Request):", err)
}
if n == 0 {
t.Fatal("Encapsulate (Request): wrote 0 bytes")
}
// SERVER: fabricated Reply.
replyFrame := buildServerFrame(MsgReply, xid, serverDUID, iaid, assignedAddr)
if err := cl.Demux(replyFrame, 0); err != nil {
t.Fatal("Demux (Reply):", err)
}
if cl.State() != StateBound {
t.Fatalf("after Reply: want StateBound, got %v", cl.State())
}
addr, valid := cl.AssignedAddr()
if !valid {
t.Fatal("AssignedAddr: not valid after bound")
}
if addr != assignedAddr {
t.Errorf("AssignedAddr: got %v, want %v", addr, assignedAddr)
}
}
func TestClientReconfigureRenew(t *testing.T) {
const xid = 0x112233
clientMAC := [6]byte{0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF}
serverDUID := []byte{0, 3, 0, 1, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66}
assignedAddr := [16]byte{0x20, 0x01, 0x0d, 0xb8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}
iaid := [4]byte{clientMAC[0], clientMAC[1], clientMAC[2], clientMAC[3]}
var cl Client
if err := cl.BeginRequest(xid, RequestConfig{ClientHardwareAddr: clientMAC}); err != nil {
t.Fatal(err)
}
var buf [1024]byte
if _, err := cl.Encapsulate(buf[:], -1, 0); err != nil {
t.Fatal(err)
}
if err := cl.Demux(buildServerFrame(MsgAdvertise, xid, serverDUID, iaid, assignedAddr), 0); err != nil {
t.Fatal(err)
}
if _, err := cl.Encapsulate(buf[:], -1, 0); err != nil {
t.Fatal(err)
}
if err := cl.Demux(buildServerFrame(MsgReply, xid, serverDUID, iaid, assignedAddr), 0); err != nil {
t.Fatal(err)
}
if cl.State() != StateBound {
t.Fatalf("state before reconfigure = %v, want %v", cl.State(), StateBound)
}
reconf := buildReconfigureFrame(0x445566, serverDUID, cl.duid, MsgRenew)
if err := cl.Demux(reconf, 0); err != nil {
t.Fatal(err)
}
if cl.State() != StateRenewing {
t.Fatalf("state after reconfigure = %v, want %v", cl.State(), StateRenewing)
}
msg, ok := cl.LastReconfigure()
if !ok || msg != MsgRenew {
t.Fatalf("LastReconfigure = %v, %v, want %v, true", msg, ok, MsgRenew)
}
n, err := cl.Encapsulate(buf[:], -1, 0)
if err != nil {
t.Fatal(err)
}
frm, err := NewFrame(buf[:n])
if err != nil {
t.Fatal(err)
}
if frm.MsgType() != MsgRenew {
t.Fatalf("Encapsulate after Reconfigure type = %v, want %v", frm.MsgType(), MsgRenew)
}
}
func buildReconfigureFrame(xid uint32, serverDUID, clientDUID []byte, msg MsgType) []byte {
buf := make([]byte, 128)
buf[0] = byte(MsgReconfigure)
buf[1] = byte(xid >> 16)
buf[2] = byte(xid >> 8)
buf[3] = byte(xid)
n := OptionsOffset
n += writeOpt6(buf[n:], OptServerID, serverDUID...)
n += writeOpt6(buf[n:], OptClientID, clientDUID...)
n += writeOpt6(buf[n:], OptReconfMsg, byte(msg))
return buf[:n]
}
func frameHasOption(t testing.TB, b []byte, want OptCode) bool {
t.Helper()
frm, err := NewFrame(b)
if err != nil {
t.Fatal(err)
}
found := false
if err := frm.ForEachOption(func(_ int, code OptCode, _ []byte) error {
if code == want {
found = true
}
return nil
}); err != nil {
t.Fatal(err)
}
return found
}
func TestClientParsesNTPServerOption(t *testing.T) {
const xid = 0x102030
clientMAC := [6]byte{0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF}
serverDUID := []byte{0, 3, 0, 1, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66}
assignedAddr := [16]byte{0x20, 0x01, 0x0d, 0xb8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}
ntpAddr := [16]byte{0x20, 0x01, 0x0d, 0xb8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0x7b}
ntpMulticast := [16]byte{0xff, 0x05, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0x01}
iaid := [4]byte{clientMAC[0], clientMAC[1], clientMAC[2], clientMAC[3]}
var cl Client
if err := cl.BeginRequest(xid, RequestConfig{ClientHardwareAddr: clientMAC}); err != nil {
t.Fatal("BeginRequest:", err)
}
var buf [1024]byte
if _, err := cl.Encapsulate(buf[:], -1, 0); err != nil {
t.Fatal("Encapsulate (Solicit):", err)
}
advFrame := buildServerFrame(MsgAdvertise, xid, serverDUID, iaid, assignedAddr)
if err := cl.Demux(advFrame, 0); err != nil {
t.Fatal("Demux (Advertise):", err)
}
if _, err := cl.Encapsulate(buf[:], -1, 0); err != nil {
t.Fatal("Encapsulate (Request):", err)
}
replyFrame := appendNTPServerOption(t, buildServerFrame(MsgReply, xid, serverDUID, iaid, assignedAddr), ntpAddr, ntpMulticast, "time.example.com")
if err := cl.Demux(replyFrame, 0); err != nil {
t.Fatal("Demux (Reply):", err)
}
var ntps []netip.Addr
ntps = cl.AppendNTPServers(ntps)
if len(ntps) != 1 || ntps[0] != netip.AddrFrom16(ntpAddr) {
t.Fatalf("AppendNTPServers = %v, want %v", ntps, netip.AddrFrom16(ntpAddr))
}
if n := cl.NumNTPServers(); n != 1 {
t.Fatalf("NumNTPServers = %d, want 1", n)
}
var multicasts []netip.Addr
multicasts = cl.AppendNTPMulticastServers(multicasts)
if len(multicasts) != 1 || multicasts[0] != netip.AddrFrom16(ntpMulticast) {
t.Fatalf("AppendNTPMulticastServers = %v, want %v", multicasts, netip.AddrFrom16(ntpMulticast))
}
if n := cl.NumNTPMulticastServers(); n != 1 {
t.Fatalf("NumNTPMulticastServers = %d, want 1", n)
}
var names []dns.Name
names = cl.AppendNTPServerNames(names)
if len(names) != 1 || !names[0].EqualString("time.example.com") {
t.Fatalf("AppendNTPServerNames = %v, want time.example.com", names)
}
if n := cl.NumNTPServerNames(); n != 1 {
t.Fatalf("NumNTPServerNames = %d, want 1", n)
}
}
func TestClientParsesDomainSearchOption(t *testing.T) {
const xid = 0x203040
clientMAC := [6]byte{0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF}
serverDUID := []byte{0, 3, 0, 1, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66}
assignedAddr := [16]byte{0x20, 0x01, 0x0d, 0xb8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}
iaid := [4]byte{clientMAC[0], clientMAC[1], clientMAC[2], clientMAC[3]}
var cl Client
if err := cl.BeginRequest(xid, RequestConfig{ClientHardwareAddr: clientMAC}); err != nil {
t.Fatal("BeginRequest:", err)
}
var buf [1024]byte
if _, err := cl.Encapsulate(buf[:], -1, 0); err != nil {
t.Fatal("Encapsulate (Solicit):", err)
}
advFrame := buildServerFrame(MsgAdvertise, xid, serverDUID, iaid, assignedAddr)
if err := cl.Demux(advFrame, 0); err != nil {
t.Fatal("Demux (Advertise):", err)
}
if _, err := cl.Encapsulate(buf[:], -1, 0); err != nil {
t.Fatal("Encapsulate (Request):", err)
}
replyFrame := appendDomainSearchOption(t, buildServerFrame(MsgReply, xid, serverDUID, iaid, assignedAddr), "example.com", "corp.example.com")
if err := cl.Demux(replyFrame, 0); err != nil {
t.Fatal("Demux (Reply):", err)
}
var domains []dns.Name
domains = cl.AppendDomainSearch(domains)
if len(domains) != 2 {
t.Fatalf("AppendDomainSearch len = %d, want 2", len(domains))
}
if !domains[0].EqualString("example.com") || !domains[1].EqualString("corp.example.com") {
t.Fatalf("AppendDomainSearch = %q, %q", domains[0].String(), domains[1].String())
}
if n := cl.NumDomainSearch(); n != 2 {
t.Fatalf("NumDomainSearch = %d, want 2", n)
}
}
func TestClientParsesDelegatedPrefix(t *testing.T) {
const xid = 0x304050
clientMAC := [6]byte{0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF}
serverDUID := []byte{0, 3, 0, 1, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66}
assignedAddr := [16]byte{0x20, 0x01, 0x0d, 0xb8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}
delegated := netip.MustParsePrefix("2001:db8:abcd::/48")
iaid := [4]byte{clientMAC[0], clientMAC[1], clientMAC[2], clientMAC[3]}
var cl Client
if err := cl.BeginRequest(xid, RequestConfig{ClientHardwareAddr: clientMAC}); err != nil {
t.Fatal("BeginRequest:", err)
}
var buf [1024]byte
if _, err := cl.Encapsulate(buf[:], -1, 0); err != nil {
t.Fatal("Encapsulate (Solicit):", err)
}
advFrame := buildServerFrame(MsgAdvertise, xid, serverDUID, iaid, assignedAddr)
if err := cl.Demux(advFrame, 0); err != nil {
t.Fatal("Demux (Advertise):", err)
}
if _, err := cl.Encapsulate(buf[:], -1, 0); err != nil {
t.Fatal("Encapsulate (Request):", err)
}
replyFrame := appendIAPDOption(buildServerFrame(MsgReply, xid, serverDUID, iaid, assignedAddr), iaid, delegated)
if err := cl.Demux(replyFrame, 0); err != nil {
t.Fatal("Demux (Reply):", err)
}
var prefixes []DelegatedPrefix
prefixes = cl.AppendDelegatedPrefixes(prefixes)
if len(prefixes) != 1 {
t.Fatalf("AppendDelegatedPrefixes len = %d, want 1", len(prefixes))
}
if prefixes[0].Prefix != delegated || prefixes[0].PreferredLifetime != 1800 || prefixes[0].ValidLifetime != 3600 {
t.Fatalf("delegated prefix = %+v, want %s preferred=1800 valid=3600", prefixes[0], delegated)
}
if n := cl.NumDelegatedPrefixes(); n != 1 {
t.Fatalf("NumDelegatedPrefixes = %d, want 1", n)
}
if cl.PrefixDelegationRenewalSeconds() != 900 || cl.PrefixDelegationRebindingSeconds() != 1800 {
t.Fatalf("IA_PD timers = renewal:%d rebind:%d, want 900/1800", cl.PrefixDelegationRenewalSeconds(), cl.PrefixDelegationRebindingSeconds())
}
}
func appendNTPServerOption(t testing.TB, frame []byte, addr, multicast [16]byte, fqdn string) []byte {
t.Helper()
var ntpPayload []byte
ntpPayload = appendNTPServerAddrSuboption(ntpPayload, 1, addr)
ntpPayload = appendNTPServerAddrSuboption(ntpPayload, 2, multicast)
name, err := dns.NewName(fqdn)
if err != nil {
t.Fatal(err)
}
nameStart := len(ntpPayload) + 4
ntpPayload = append(ntpPayload, 0, 3, 0, 0)
ntpPayload, err = name.AppendTo(ntpPayload)
if err != nil {
t.Fatal(err)
}
binary.BigEndian.PutUint16(ntpPayload[nameStart-2:nameStart], uint16(len(ntpPayload)-nameStart))
buf := append(frame[:len(frame):len(frame)], make([]byte, 4+len(ntpPayload))...)
writeOpt6(buf[len(frame):], OptNTPServer, ntpPayload...)
return buf
}
func appendNTPServerAddrSuboption(dst []byte, code uint16, addr [16]byte) []byte {
start := len(dst)
dst = append(dst, 0, 0, 0, 16)
binary.BigEndian.PutUint16(dst[start:start+2], code)
return append(dst, addr[:]...)
}
func appendDomainSearchOption(t testing.TB, frame []byte, domains ...string) []byte {
t.Helper()
var payload []byte
for _, domain := range domains {
name, err := dns.NewName(domain)
if err != nil {
t.Fatal(err)
}
payload, err = name.AppendTo(payload)
if err != nil {
t.Fatal(err)
}
}
buf := append(frame[:len(frame):len(frame)], make([]byte, 4+len(payload))...)
writeOpt6(buf[len(frame):], OptDomainList, payload...)
return buf
}
func appendIAPDOption(frame []byte, iaid [4]byte, prefix netip.Prefix) []byte {
var iaPrefix [29]byte
binary.BigEndian.PutUint16(iaPrefix[0:2], uint16(OptIAPrefix))
binary.BigEndian.PutUint16(iaPrefix[2:4], 25)
binary.BigEndian.PutUint32(iaPrefix[4:8], 1800)
binary.BigEndian.PutUint32(iaPrefix[8:12], 3600)
iaPrefix[12] = byte(prefix.Bits())
prefixAddr := prefix.Addr().As16()
copy(iaPrefix[13:], prefixAddr[:])
var iapd [41]byte
copy(iapd[:4], iaid[:])
binary.BigEndian.PutUint32(iapd[4:8], 900)
binary.BigEndian.PutUint32(iapd[8:12], 1800)
copy(iapd[12:], iaPrefix[:])
buf := append(frame[:len(frame):len(frame)], make([]byte, 4+len(iapd))...)
writeOpt6(buf[len(frame):], OptIAPD, iapd[:]...)
return buf
}
// TestClientEncapsulateSolicit verifies that the first Encapsulate call
// writes a Solicit message with at least OptClientID and OptIANA.
func TestClientEncapsulateSolicit(t *testing.T) {
var cl Client
if err := cl.BeginRequest(0xABCDEF, RequestConfig{
ClientHardwareAddr: [6]byte{1, 2, 3, 4, 5, 6},
}); err != nil {
t.Fatal(err)
}
buf := make([]byte, 512)
n, err := cl.Encapsulate(buf, -1, 0)
if err != nil {
t.Fatal(err)
}
if n == 0 {
t.Fatal("Encapsulate: wrote 0 bytes")
}
frm, err := NewFrame(buf[:n])
if err != nil {
t.Fatal(err)
}
if frm.MsgType() != MsgSolicit {
t.Errorf("MsgType: want MsgSolicit, got %v", frm.MsgType())
}
if frm.TransactionID() != 0xABCDEF {
t.Errorf("TransactionID: want 0xABCDEF, got 0x%X", frm.TransactionID())
}
var hasClientID, hasIANA bool
err = frm.ForEachOption(func(_ int, code OptCode, _ []byte) error {
switch code {
case OptClientID:
hasClientID = true
case OptIANA:
hasIANA = true
}
return nil
})
if err != nil {
t.Fatal("ForEachOption:", err)
}
if !hasClientID {
t.Error("Solicit: missing OptClientID")
}
if !hasIANA {
t.Error("Solicit: missing OptIANA")
}
}
// TestClientDoubleTapEncapsulate verifies that calling Encapsulate twice in the
// same state returns 0 bytes on the second call (idempotent, no duplicate messages).
func TestClientDoubleTapEncapsulate(t *testing.T) {
var cl Client
if err := cl.BeginRequest(1, RequestConfig{
ClientHardwareAddr: [6]byte{1, 2, 3, 4, 5, 6},
}); err != nil {
t.Fatal(err)
}
buf := make([]byte, 512)
n, err := cl.Encapsulate(buf, -1, 0)
if err != nil {
t.Fatal("first Encapsulate:", err)
}
if n == 0 {
t.Fatal("first Encapsulate: wrote 0 bytes")
}
n2, err := cl.Encapsulate(buf, -1, 0)
if err != nil {
t.Fatal("second Encapsulate:", err)
}
if n2 != 0 {
t.Errorf("second Encapsulate: want 0 bytes (idempotent), got %d", n2)
}
}
// driveToRequesting advances a fresh client through Solicit→Advertise→Request so
// the next Demux of a Reply runs the option-parsing path.
func driveToRequesting(t testing.TB, cl *Client, xid uint32, serverDUID []byte, iaid [4]byte, addr [16]byte) {
t.Helper()
var buf [1024]byte
if _, err := cl.Encapsulate(buf[:], -1, 0); err != nil {
t.Fatal("Encapsulate (Solicit):", err)
}
if err := cl.Demux(buildServerFrame(MsgAdvertise, xid, serverDUID, iaid, addr), 0); err != nil {
t.Fatal("Demux (Advertise):", err)
}
if _, err := cl.Encapsulate(buf[:], -1, 0); err != nil {
t.Fatal("Encapsulate (Request):", err)
}
}
// floodDNSServersOption appends an OptDNSServers carrying n distinct addresses.
func floodDNSServersOption(frame []byte, n int) []byte {
payload := make([]byte, 0, n*16)
for i := range n {
var a [16]byte
a[0], a[1], a[15] = 0x20, 0x01, byte(i+1)
payload = append(payload, a[:]...)
}
buf := append(frame[:len(frame):len(frame)], make([]byte, 4+len(payload))...)
writeOpt6(buf[len(frame):], OptDNSServers, payload...)
return buf
}
// floodIAPDOption appends an OptIAPD carrying n OptIAPrefix sub-options.
func floodIAPDOption(frame []byte, iaid [4]byte, n int) []byte {
payload := make([]byte, 12)
copy(payload[:4], iaid[:])
binary.BigEndian.PutUint32(payload[4:8], 900)
binary.BigEndian.PutUint32(payload[8:12], 1800)
for i := range n {
var iaPrefix [29]byte
binary.BigEndian.PutUint16(iaPrefix[0:2], uint16(OptIAPrefix))
binary.BigEndian.PutUint16(iaPrefix[2:4], 25)
binary.BigEndian.PutUint32(iaPrefix[4:8], 1800)
binary.BigEndian.PutUint32(iaPrefix[8:12], 3600)
iaPrefix[12] = 48
iaPrefix[13], iaPrefix[14], iaPrefix[15] = 0x20, 0x01, byte(i+1)
payload = append(payload, iaPrefix[:]...)
}
buf := append(frame[:len(frame):len(frame)], make([]byte, 4+len(payload))...)
writeOpt6(buf[len(frame):], OptIAPD, payload...)
return buf
}
// floodNTPOption appends an OptNTPServer carrying nAddr unicast (suboption 1),
// nMulticast (suboption 2) and nNames FQDN (suboption 3) entries.
func floodNTPOption(t testing.TB, frame []byte, nAddr, nMulticast, nNames int) []byte {
t.Helper()
var payload []byte
for i := range nAddr {
var a [16]byte
a[0], a[15] = 0x20, byte(i+1)
payload = appendNTPServerAddrSuboption(payload, 1, a)
}
for i := range nMulticast {
var a [16]byte
a[0], a[1], a[15] = 0xff, 0x05, byte(i+1)
payload = appendNTPServerAddrSuboption(payload, 2, a)
}
name, err := dns.NewName("ntp.example.com")
if err != nil {
t.Fatal(err)
}
for range nNames {
start := len(payload) + 4
payload = append(payload, 0, 3, 0, 0)
payload, err = name.AppendTo(payload)
if err != nil {
t.Fatal(err)
}
binary.BigEndian.PutUint16(payload[start-2:start], uint16(len(payload)-start))
}
buf := append(frame[:len(frame):len(frame)], make([]byte, 4+len(payload))...)
writeOpt6(buf[len(frame):], OptNTPServer, payload...)
return buf
}
func repeatStrings(s string, n int) []string {
out := make([]string, n)
for i := range out {
out[i] = s
}
return out
}
// TestClientLimitsCapServerData verifies that the client stores no more than
// the configured number of each repeated option, so a server flooding a Reply
// cannot drive unbounded allocation (the OOM concern raised in the PR review).
func TestClientLimitsCapServerData(t *testing.T) {
const xid = 0x515151
mac := [6]byte{0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF}
serverDUID := []byte{0, 3, 0, 1, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66}
iaid := [4]byte{mac[0], mac[1], mac[2], mac[3]}
addr := [16]byte{0x20, 0x01, 0x0d, 0xb8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}
limits := Limits{
MaxDNSServers: 2, MaxDomainSearch: 2, MaxNTPServers: 2,
MaxNTPMulticastServers: 1, MaxNTPServerNames: 2, MaxDelegatedPrefixes: 2,
}
var cl Client
if err := cl.BeginRequest(xid, RequestConfig{ClientHardwareAddr: mac, Limits: limits}); err != nil {
t.Fatal("BeginRequest:", err)
}
driveToRequesting(t, &cl, xid, serverDUID, iaid, addr)
// A Reply far exceeding every configured cap.
reply := buildServerFrame(MsgReply, xid, serverDUID, iaid, addr)
reply = floodDNSServersOption(reply, 8)
reply = floodIAPDOption(reply, iaid, 8)
reply = appendDomainSearchOption(t, reply, repeatStrings("example.com", 8)...)
reply = floodNTPOption(t, reply, 8, 8, 8)
if err := cl.Demux(reply, 0); err != nil {
t.Fatal("Demux (Reply):", err)
}
for _, tc := range []struct {
name string
got int
want int
}{
{"DNS servers", cl.NumDNSServers(), limits.MaxDNSServers},
{"domain search", cl.NumDomainSearch(), limits.MaxDomainSearch},
{"NTP servers", cl.NumNTPServers(), limits.MaxNTPServers},
{"NTP multicast", cl.NumNTPMulticastServers(), limits.MaxNTPMulticastServers},
{"NTP names", cl.NumNTPServerNames(), limits.MaxNTPServerNames},
{"delegated prefixes", cl.NumDelegatedPrefixes(), limits.MaxDelegatedPrefixes},
} {
if tc.got != tc.want {
t.Errorf("%s stored = %d, want capped at %d", tc.name, tc.got, tc.want)
}
}
}
// TestClientParseNoAllocs verifies that once BeginRequest has sized the option
// backing arrays, parsing a server message performs no further allocation: the
// bounded slices are reused, keeping the network-facing path off the heap.
func TestClientParseNoAllocs(t *testing.T) {
const xid = 0x123456
mac := [6]byte{0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF}
serverDUID := []byte{0, 3, 0, 1, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66}
iaid := [4]byte{mac[0], mac[1], mac[2], mac[3]}
addr := [16]byte{0x20, 0x01, 0x0d, 0xb8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}
var cl Client
if err := cl.BeginRequest(xid, RequestConfig{ClientHardwareAddr: mac}); err != nil {
t.Fatal("BeginRequest:", err)
}
reply := buildServerFrame(MsgReply, xid, serverDUID, iaid, addr)
reply = floodDNSServersOption(reply, cl.limits.MaxDNSServers)
reply = floodIAPDOption(reply, iaid, cl.limits.MaxDelegatedPrefixes)
reply = appendDomainSearchOption(t, reply, repeatStrings("example.com", cl.limits.MaxDomainSearch)...)
reply = floodNTPOption(t, reply, cl.limits.MaxNTPServers, cl.limits.MaxNTPMulticastServers, cl.limits.MaxNTPServerNames)
frm, err := NewFrame(reply)
if err != nil {
t.Fatal("NewFrame:", err)
}
clearStores := func() {
cl.serverDUID = cl.serverDUID[:0]
cl.dns = cl.dns[:0]
cl.domainSearch = cl.domainSearch[:0]
cl.ntps = cl.ntps[:0]
cl.ntpMulticast = cl.ntpMulticast[:0]
cl.ntpNames = cl.ntpNames[:0]
cl.delegatedPrefixes = cl.delegatedPrefixes[:0]
}
clearStores()
if err := cl.setOptions(frm); err != nil { // warm up the dns.Name backing arrays.
t.Fatal("setOptions:", err)
}
allocs := testing.AllocsPerRun(50, func() {
clearStores()
_ = cl.setOptions(frm)
})
if allocs != 0 {
t.Errorf("setOptions must not allocate after BeginRequest sizing, got %v allocs/op", allocs)
}
}
+179
View File
@@ -0,0 +1,179 @@
package dhcpv6
import (
"encoding/binary"
"github.com/soypat/lneto"
)
// ClientState transition table during request:
//
// StateInit -> | Send out Solicit | -> StateSoliciting
// StateSoliciting -> | Accept Advertise | -> StateRequesting
// StateRequesting -> | Receive Reply | -> StateBound
//go:generate stringer -type=MsgType,ClientState,OptCode,StatusCode,DUIDType -linecomment -output stringers.go
const (
// ClientPort is the UDP port DHCPv6 clients listen on.
ClientPort = 546
// ServerPort is the UDP port DHCPv6 servers listen on.
ServerPort = 547
// OptionsOffset is the byte offset where DHCPv6 options begin in a client-server message.
// Layout: MsgType(1) + TransactionID(3).
OptionsOffset = 4
)
// MsgType is the DHCPv6 message type (RFC 8415 §7.3).
type MsgType uint8
const (
MsgSolicit MsgType = 1 // solicit
MsgAdvertise MsgType = 2 // advertise
MsgRequest MsgType = 3 // request
MsgConfirm MsgType = 4 // confirm
MsgRenew MsgType = 5 // renew
MsgRebind MsgType = 6 // rebind
MsgReply MsgType = 7 // reply
MsgRelease MsgType = 8 // release
MsgDecline MsgType = 9 // decline
MsgReconfigure MsgType = 10 // reconfigure
MsgInformRequest MsgType = 11 // inform-request
MsgRelayForw MsgType = 12 // relay-forw
MsgRelayRepl MsgType = 13 // relay-repl
)
// ClientState is the DHCPv6 client DORA state machine state.
type ClientState uint8
const (
_ ClientState = iota
StateInit // init
StateSoliciting // soliciting
StateRequesting // requesting
StateBound // bound
StateRenewing // renewing
StateRebinding // rebinding
)
// HasIP returns true if the state indicates the Client has an IPv6 address assigned.
func (s ClientState) HasIP() bool {
return s == StateBound || s == StateRenewing || s == StateRebinding
}
// OptCode is a DHCPv6 option code (RFC 8415 §21), encoded as a 2-byte big-endian value.
type OptCode uint16
const (
OptClientID OptCode = 1 // client-id
OptServerID OptCode = 2 // server-id
OptIANA OptCode = 3 // ia-na
OptIATA OptCode = 4 // ia-ta
OptIAAddr OptCode = 5 // iaaddr
OptORO OptCode = 6 // oro
OptPreference OptCode = 7 // preference
OptElapsedTime OptCode = 8 // elapsed-time
OptRelayMsg OptCode = 9 // relay-msg
OptAuth OptCode = 11 // auth
OptUnicast OptCode = 12 // unicast
OptStatusCode OptCode = 13 // status-code
OptRapidCommit OptCode = 14 // rapid-commit
OptUserClass OptCode = 15 // user-class
OptVendorClass OptCode = 16 // vendor-class
OptVendorOpts OptCode = 17 // vendor-opts
OptInterfaceID OptCode = 18 // interface-id
OptReconfMsg OptCode = 19 // reconf-msg
OptReconfAccept OptCode = 20 // reconf-accept
OptDNSServers OptCode = 23 // dns-servers
OptDomainList OptCode = 24 // domain-list
OptIAPD OptCode = 25 // ia-pd
OptIAPrefix OptCode = 26 // iaprefix
OptNTPServer OptCode = 56 // ntp-server
)
// DUIDType is the DHCP Unique Identifier type (RFC 8415 §11).
type DUIDType uint16
const (
DUIDTypeLLT DUIDType = 1 // duid-llt
DUIDTypeEN DUIDType = 2 // duid-en
DUIDTypeLL DUIDType = 3 // duid-ll
)
// StatusCode is the DHCPv6 status code value (RFC 8415 §21.13).
type StatusCode uint16
const (
StatusSuccess StatusCode = 0 // success
StatusUnspecFail StatusCode = 1 // unspec-fail
StatusNoAddrsAvail StatusCode = 2 // no-addrs-avail
StatusNoBinding StatusCode = 3 // no-binding
StatusNotOnLink StatusCode = 4 // not-on-link
StatusUseMulticast StatusCode = 5 // use-multicast
)
// EncodeOption writes a DHCPv6 TLV option into dst.
// Format: code(2) + length(2) + data.
func EncodeOption(dst []byte, code OptCode, data ...byte) (int, error) {
if len(data) > 0xffff {
return 0, lneto.ErrInvalidLengthField
} else if len(dst) < 4+len(data) {
return 0, lneto.ErrShortBuffer
}
binary.BigEndian.PutUint16(dst[0:2], uint16(code))
binary.BigEndian.PutUint16(dst[2:4], uint16(len(data)))
copy(dst[4:], data)
return 4 + len(data), nil
}
// EncodeOption16 encodes a single uint16 value as a DHCPv6 option.
func EncodeOption16(dst []byte, code OptCode, v uint16) (int, error) {
return EncodeOption(dst, code, byte(v>>8), byte(v))
}
// EncodeOption32 encodes a single uint32 value as a DHCPv6 option.
func EncodeOption32(dst []byte, code OptCode, v uint32) (int, error) {
return EncodeOption(dst, code, byte(v>>24), byte(v>>16), byte(v>>8), byte(v))
}
// EncodeOptionIANA encodes an IA_NA option (RFC 8415 §21.4).
// Layout: code(2) + len(2) + IAID(4) + T1(4) + T2(4) + subOpts.
func EncodeOptionIANA(dst []byte, iaid [4]byte, t1, t2 uint32, subOpts []byte) (int, error) {
const fixedLen = 12 // IAID(4) + T1(4) + T2(4)
dataLen := fixedLen + len(subOpts)
if len(dst) < 4+dataLen {
return 0, lneto.ErrShortBuffer
}
binary.BigEndian.PutUint16(dst[0:2], uint16(OptIANA))
binary.BigEndian.PutUint16(dst[2:4], uint16(dataLen))
copy(dst[4:8], iaid[:])
binary.BigEndian.PutUint32(dst[8:12], t1)
binary.BigEndian.PutUint32(dst[12:16], t2)
copy(dst[16:], subOpts)
return 4 + dataLen, nil
}
// EncodeOptionIAAddr encodes an IAADDR option (RFC 8415 §21.6).
// Layout: code(2) + len(2) + addr(16) + preferred(4) + valid(4).
func EncodeOptionIAAddr(dst []byte, addr [16]byte, preferred, valid uint32) (int, error) {
const dataLen = 24 // addr(16) + preferred(4) + valid(4)
if len(dst) < 4+dataLen {
return 0, lneto.ErrShortBuffer
}
binary.BigEndian.PutUint16(dst[0:2], uint16(OptIAAddr))
binary.BigEndian.PutUint16(dst[2:4], dataLen)
copy(dst[4:20], addr[:])
binary.BigEndian.PutUint32(dst[20:24], preferred)
binary.BigEndian.PutUint32(dst[24:28], valid)
return 4 + dataLen, nil
}
// AppendDUIDLL appends a DUID-LL (type 3) for an Ethernet MAC to dst.
// Format: DUIDType(2) + hwtype=1(2) + mac(6).
func AppendDUIDLL(dst []byte, mac [6]byte) []byte {
return append(dst,
byte(DUIDTypeLL>>8), byte(DUIDTypeLL), // type 3
0, 1, // hardware type 1 = Ethernet
mac[0], mac[1], mac[2], mac[3], mac[4], mac[5],
)
}
+84
View File
@@ -0,0 +1,84 @@
package dhcpv6
import (
"encoding/binary"
"io"
"github.com/soypat/lneto"
)
// NewFrame returns a Frame backed by buf.
// Returns an error if buf is shorter than [OptionsOffset] bytes.
func NewFrame(buf []byte) (Frame, error) {
if len(buf) < OptionsOffset {
return Frame{}, lneto.ErrTruncatedFrame
}
return Frame{buf: buf}, nil
}
// Frame encapsulates the raw bytes of a DHCPv6 client-server message (RFC 8415 §8)
// and provides methods for accessing and modifying its fields.
//
// Layout:
//
// Byte 0: msg-type
// Bytes 1-3: transaction-id (24-bit big-endian)
// Bytes 4+: options (code(2) + length(2) + data)
type Frame struct {
buf []byte
}
// MsgType returns the message type field.
func (frm Frame) MsgType() MsgType { return MsgType(frm.buf[0]) }
// SetMsgType sets the message type field.
func (frm Frame) SetMsgType(t MsgType) { frm.buf[0] = byte(t) }
// TransactionID returns the 24-bit transaction ID as a uint32 (upper byte is always zero).
func (frm Frame) TransactionID() uint32 {
return uint32(frm.buf[1])<<16 | uint32(frm.buf[2])<<8 | uint32(frm.buf[3])
}
// SetTransactionID writes the lower 24 bits of id into bytes 13.
func (frm Frame) SetTransactionID(id uint32) {
frm.buf[1] = byte(id >> 16)
frm.buf[2] = byte(id >> 8)
frm.buf[3] = byte(id)
}
// Options returns the options section of the frame (bytes from [OptionsOffset] onward).
func (frm Frame) Options() []byte { return frm.buf[OptionsOffset:] }
// ForEachOption iterates over all DHCPv6 options in the frame's options section.
// Each option is passed to fn as (byteOffset, optionCode, optionData).
// Iteration stops early if fn returns [io.EOF]; any other non-nil error is returned directly.
// If fn is nil, the function only validates the structure.
func (frm Frame) ForEachOption(fn func(off int, code OptCode, data []byte) error) error {
buf := frm.buf
ptr := OptionsOffset
for ptr+4 <= len(buf) {
code := OptCode(binary.BigEndian.Uint16(buf[ptr:]))
optlen := int(binary.BigEndian.Uint16(buf[ptr+2:]))
if ptr+4+optlen > len(buf) {
return lneto.ErrInvalidLengthField
}
if fn != nil {
err := fn(ptr, code, buf[ptr+4:ptr+4+optlen])
if err == io.EOF {
return nil
}
if err != nil {
return err
}
}
ptr += 4 + optlen
}
if ptr != len(buf) {
// 13 trailing bytes that cannot form a valid option header.
return lneto.ErrTruncatedFrame
}
return nil
}
// ValidateSize validates the structure of the options section without invoking a callback.
func (frm Frame) ValidateSize() error { return frm.ForEachOption(nil) }
+162
View File
@@ -0,0 +1,162 @@
// Code generated by "stringer -type=MsgType,ClientState,OptCode,StatusCode,DUIDType -linecomment -output stringers.go"; DO NOT EDIT.
package dhcpv6
import "strconv"
func _() {
// An "invalid array index" compiler error signifies that the constant values have changed.
// Re-run the stringer command to generate them again.
var x [1]struct{}
_ = x[MsgSolicit-1]
_ = x[MsgAdvertise-2]
_ = x[MsgRequest-3]
_ = x[MsgConfirm-4]
_ = x[MsgRenew-5]
_ = x[MsgRebind-6]
_ = x[MsgReply-7]
_ = x[MsgRelease-8]
_ = x[MsgDecline-9]
_ = x[MsgReconfigure-10]
_ = x[MsgInformRequest-11]
_ = x[MsgRelayForw-12]
_ = x[MsgRelayRepl-13]
}
const _MsgType_name = "solicitadvertiserequestconfirmrenewrebindreplyreleasedeclinereconfigureinform-requestrelay-forwrelay-repl"
var _MsgType_index = [...]uint8{0, 7, 16, 23, 30, 35, 41, 46, 53, 60, 71, 85, 95, 105}
func (i MsgType) String() string {
idx := int(i) - 1
if i < 1 || idx >= len(_MsgType_index)-1 {
return "MsgType(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _MsgType_name[_MsgType_index[idx]:_MsgType_index[idx+1]]
}
func _() {
// An "invalid array index" compiler error signifies that the constant values have changed.
// Re-run the stringer command to generate them again.
var x [1]struct{}
_ = x[StateInit-1]
_ = x[StateSoliciting-2]
_ = x[StateRequesting-3]
_ = x[StateBound-4]
_ = x[StateRenewing-5]
_ = x[StateRebinding-6]
}
const _ClientState_name = "initsolicitingrequestingboundrenewingrebinding"
var _ClientState_index = [...]uint8{0, 4, 14, 24, 29, 37, 46}
func (i ClientState) String() string {
idx := int(i) - 1
if i < 1 || idx >= len(_ClientState_index)-1 {
return "ClientState(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _ClientState_name[_ClientState_index[idx]:_ClientState_index[idx+1]]
}
func _() {
// An "invalid array index" compiler error signifies that the constant values have changed.
// Re-run the stringer command to generate them again.
var x [1]struct{}
_ = x[OptClientID-1]
_ = x[OptServerID-2]
_ = x[OptIANA-3]
_ = x[OptIATA-4]
_ = x[OptIAAddr-5]
_ = x[OptORO-6]
_ = x[OptPreference-7]
_ = x[OptElapsedTime-8]
_ = x[OptRelayMsg-9]
_ = x[OptAuth-11]
_ = x[OptUnicast-12]
_ = x[OptStatusCode-13]
_ = x[OptRapidCommit-14]
_ = x[OptUserClass-15]
_ = x[OptVendorClass-16]
_ = x[OptVendorOpts-17]
_ = x[OptInterfaceID-18]
_ = x[OptReconfMsg-19]
_ = x[OptReconfAccept-20]
_ = x[OptDNSServers-23]
_ = x[OptDomainList-24]
_ = x[OptIAPD-25]
_ = x[OptIAPrefix-26]
_ = x[OptNTPServer-56]
}
const (
_OptCode_name_0 = "client-idserver-idia-naia-taiaaddroropreferenceelapsed-timerelay-msg"
_OptCode_name_1 = "authunicaststatus-coderapid-commituser-classvendor-classvendor-optsinterface-idreconf-msgreconf-accept"
_OptCode_name_2 = "dns-serversdomain-listia-pdiaprefix"
_OptCode_name_3 = "ntp-server"
)
var (
_OptCode_index_0 = [...]uint8{0, 9, 18, 23, 28, 34, 37, 47, 59, 68}
_OptCode_index_1 = [...]uint8{0, 4, 11, 22, 34, 44, 56, 67, 79, 89, 102}
_OptCode_index_2 = [...]uint8{0, 11, 22, 27, 35}
)
func (i OptCode) String() string {
switch {
case 1 <= i && i <= 9:
i -= 1
return _OptCode_name_0[_OptCode_index_0[i]:_OptCode_index_0[i+1]]
case 11 <= i && i <= 20:
i -= 11
return _OptCode_name_1[_OptCode_index_1[i]:_OptCode_index_1[i+1]]
case 23 <= i && i <= 26:
i -= 23
return _OptCode_name_2[_OptCode_index_2[i]:_OptCode_index_2[i+1]]
case i == 56:
return _OptCode_name_3
default:
return "OptCode(" + strconv.FormatInt(int64(i), 10) + ")"
}
}
func _() {
// An "invalid array index" compiler error signifies that the constant values have changed.
// Re-run the stringer command to generate them again.
var x [1]struct{}
_ = x[StatusSuccess-0]
_ = x[StatusUnspecFail-1]
_ = x[StatusNoAddrsAvail-2]
_ = x[StatusNoBinding-3]
_ = x[StatusNotOnLink-4]
_ = x[StatusUseMulticast-5]
}
const _StatusCode_name = "successunspec-failno-addrs-availno-bindingnot-on-linkuse-multicast"
var _StatusCode_index = [...]uint8{0, 7, 18, 32, 42, 53, 66}
func (i StatusCode) String() string {
idx := int(i) - 0
if i < 0 || idx >= len(_StatusCode_index)-1 {
return "StatusCode(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _StatusCode_name[_StatusCode_index[idx]:_StatusCode_index[idx+1]]
}
func _() {
// An "invalid array index" compiler error signifies that the constant values have changed.
// Re-run the stringer command to generate them again.
var x [1]struct{}
_ = x[DUIDTypeLLT-1]
_ = x[DUIDTypeEN-2]
_ = x[DUIDTypeLL-3]
}
const _DUIDType_name = "duid-lltduid-enduid-ll"
var _DUIDType_index = [...]uint8{0, 8, 15, 22}
func (i DUIDType) String() string {
idx := int(i) - 1
if i < 1 || idx >= len(_DUIDType_index)-1 {
return "DUIDType(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _DUIDType_name[_DUIDType_index[idx]:_DUIDType_index[idx+1]]
}
+23 -24
View File
@@ -4,6 +4,7 @@ import (
"log/slog"
"math"
"net"
"net/netip"
"github.com/soypat/lneto"
"github.com/soypat/lneto/internal"
@@ -15,7 +16,7 @@ type Client struct {
lport uint16
msg Message
respFlags HeaderFlags
state clientState
state StateClientQuery
enableRecursion bool
}
@@ -36,7 +37,7 @@ func (c *Client) StartResolve(localPort, txid uint16, cfg ResolveConfig) error {
if nd > math.MaxUint16 {
return lneto.ErrInvalidConfig
}
c.reset(localPort, txid, dnsSendQuery, cfg.EnableRecursion)
c.reset(localPort, txid, CQueryPending, cfg.EnableRecursion)
c.msg.LimitResourceDecoding(uint16(nd), uint16(nd), 0, 0)
c.msg.AddQuestions(cfg.Questions)
c.msg.AddAdditionals(cfg.Additional)
@@ -46,7 +47,7 @@ func (c *Client) StartResolve(localPort, txid uint16, cfg ResolveConfig) error {
func (c *Client) Encapsulate(carrierData []byte, offsetToIP, offsetToFrame int) (int, error) {
if c.isClosed() {
return 0, net.ErrClosed
} else if c.state != dnsSendQuery {
} else if c.state != CQueryPending {
return 0, nil
}
@@ -64,7 +65,7 @@ func (c *Client) Encapsulate(carrierData []byte, offsetToIP, offsetToFrame int)
internal.LogAttrs(nil, slog.LevelError, "dns:unexpected-write", slog.Int("got", len(data)), slog.Int("want", int(msglen)))
return 0, lneto.ErrBug
}
c.state = dnsAwaitResponse
c.state = CQueryOutstanding
// Unset don't frag since DNS requests go through LOTS of nodes.
// if frameOffset >= 28 {
// version := carrierData[0] >> 4
@@ -78,7 +79,7 @@ func (c *Client) Encapsulate(carrierData []byte, offsetToIP, offsetToFrame int)
func (c *Client) Demux(carrierData []byte, frameOffset int) error {
if c.isClosed() {
return net.ErrClosed
} else if c.state != dnsAwaitResponse {
} else if c.state != CQueryOutstanding {
return nil
}
frame := carrierData[frameOffset:]
@@ -91,7 +92,7 @@ func (c *Client) Demux(carrierData []byte, frameOffset int) error {
return nil // Not meant for our client.
}
c.respFlags = flags
c.state = dnsDone
c.state = CQueryDone
msg := &c.msg
_, incompleteButOK, err := msg.Decode(frame)
if err != nil && !incompleteButOK {
@@ -101,10 +102,10 @@ func (c *Client) Demux(carrierData []byte, frameOffset int) error {
}
func (c *Client) isClosed() bool {
return c.state == dnsClosed || c.state == dnsAborted
return c.state == CQueryIdle || c.state == CQueryAborted
}
func (c *Client) MessageCopyTo(dst *Message) (done bool, err error) {
func (c *Client) ResponseCopyTo(dst *Message) (done bool, err error) {
if !c.respFlags.IsResponse() {
return false, nil
}
@@ -116,18 +117,26 @@ func (c *Client) MessageCopyTo(dst *Message) (done bool, err error) {
return true, nil
}
func (c *Client) Answers() []Resource {
if c.state != dnsDone {
return nil
func (c *Client) ResponseAnswerLookup(dst []netip.Addr, host string) (uint16, error) {
if !c.respFlags.IsResponse() {
return 0, nil
}
return c.msg.Answers
rcode := c.respFlags.ResponseCode()
if rcode != 0 {
return 0, rcode
}
return c.msg.WriteAnswers(dst, host)
}
func (c *Client) ResponseFlags() (HeaderFlags, bool) {
return c.respFlags, c.respFlags.IsResponse()
}
func (c *Client) Abort() {
c.reset(0, 0, 0, false)
c.reset(0, 0, CQueryAborted, false)
}
func (c *Client) reset(lport, txid uint16, state clientState, enableRecursion bool) {
func (c *Client) reset(lport, txid uint16, state StateClientQuery, enableRecursion bool) {
*c = Client{
connID: c.connID + 1,
lport: lport,
@@ -138,13 +147,3 @@ func (c *Client) reset(lport, txid uint16, state clientState, enableRecursion bo
}
c.msg.Reset()
}
type clientState uint8
const (
dnsClosed clientState = iota
dnsSendQuery
dnsAwaitResponse
dnsDone
dnsAborted
)
+11
View File
@@ -238,6 +238,17 @@ const (
RCodeRefused RCode = 5 // refused
)
// StateClientQuery is the lifecycle state of a single DNS query.
type StateClientQuery uint8
const (
CQueryIdle StateClientQuery = iota // no active query (zero value)
CQueryPending // query built, not yet transmitted
CQueryOutstanding // transmitted; awaiting response (RFC 7766 §9.3)
CQueryDone // response received and decoded
CQueryAborted // query abandoned (connection error or caller abort)
)
func b2u8(b bool) uint8 {
if b {
return 1
+129 -48
View File
@@ -4,6 +4,7 @@ import (
"bytes"
"encoding/binary"
"math"
"net/netip"
"slices"
"strconv"
"strings"
@@ -26,6 +27,11 @@ const (
MaxSizeUDP = 512
)
// Message is a convenience type for decoding DNS messages and storing results in a single object.
// Message is designed for ease of memory reuse. All internal buffers in a Message are reused in methods:
// - [Message.Decode]: Limited in decode size by [Message.LimitResourceDecoding] which must be called beforehand.
// - [Message.CopyFrom]
// - [Message.AddQuestions]
type Message struct {
Questions []Question
Answers []Resource
@@ -54,10 +60,38 @@ type ResourceHeader struct {
Length uint16
}
// Name is a wire representation of a DNS name.
type Name struct {
data []byte
}
// EqualString checks if the name receiver matches the strname string (non-wire formatted) name.
func (n Name) EqualString(strname string) bool {
data := n.data
for len(data) > 0 {
labelLen := int(data[0])
if labelLen == 0 {
return strname == "" || strname == "."
}
if len(data) < 1+labelLen {
return false
}
label := data[1 : 1+labelLen]
var seg string
before, after, ok := strings.Cut(strname, ".")
if !ok {
seg, strname = strname, ""
} else {
seg, strname = before, after
}
if len(seg) != len(label) || seg != string(label) {
return false
}
data = data[1+labelLen:]
}
return false
}
// NamesEqual reports whether two DNS names are equal by comparing
// their wire-format representations directly. This is case-sensitive;
// for case-insensitive comparison use [NamesEqualFold].
@@ -265,6 +299,27 @@ func (m *Message) AppendTo(buf []byte, txid uint16, flags HeaderFlags) (_ []byte
return buf, nil
}
func (m *Message) WriteAnswers(dst []netip.Addr, host string) (n uint16, err error) {
for i := range m.Answers {
if int(n) >= len(dst) {
return n, lneto.ErrExhausted
}
ans := &m.Answers[i]
hdr := ans.Header()
if !hdr.Name.EqualString(host) {
continue
}
var ok bool
dst[n], ok = netip.AddrFromSlice(ans.RawData())
if !ok {
err = lneto.ErrInvalidAddr
} else {
n++
}
}
return n, err
}
func (m *Message) Len() uint16 {
return SizeHeader + m.lenResources()
}
@@ -346,6 +401,8 @@ func (r *Resource) Reset() {
r.data = r.data[:0]
}
func (r *Resource) Header() ResourceHeader { return r.header }
func (r *Resource) RawData() []byte {
length := r.header.Length
if int(length) > len(r.data) {
@@ -489,7 +546,7 @@ func NewName(domain string) (Name, error) {
// Returns an empty Name if n exceeds the number of labels.
func (n Name) TrimLabels(skip int) Name {
off := 0
for i := 0; i < skip; i++ {
for range skip {
if off >= len(n.data) {
return Name{}
}
@@ -596,8 +653,6 @@ func append32(b []byte, v uint32) []byte {
}
func visitAllLabels(msg []byte, off uint16, fn func(b []byte), allowCompression bool) (uint16, error) {
// currOff is the current working offset.
currOff := off
if len(msg) > math.MaxUint16 {
return off, errResTooLong
}
@@ -608,60 +663,86 @@ func visitAllLabels(msg []byte, off uint16, fn func(b []byte), allowCompression
// the usage of this name.
var newOff = off
LOOP:
for {
if currOff >= uint16(len(msg)) {
return off, lneto.ErrTruncatedFrame
}
c := uint16(msg[currOff])
currOff++
switch c & 0xc0 {
case 0x00: // String label (segment).
if c == 0x00 {
break LOOP // Nominal end of name, always ends with null terminator.
}
endOff := currOff + c
if endOff > uint16(len(msg)) {
return off, errCalcLen
}
// Reject names containing dots. See issue golang/go#56246
if bytes.IndexByte(msg[currOff:endOff], '.') >= 0 {
return off, errInvalidName
}
fn(msg[currOff:endOff])
currOff = endOff
case 0xc0: // Pointer.
// https://cs.opensource.google/go/x/net/+/refs/tags/v0.19.0:dns/dnsmessage/message.go;l=2078
if !allowCompression {
return off, errCompressedSRV
}
if currOff >= uint16(len(msg)) {
return off, errInvalidPtr
}
c1 := msg[currOff]
currOff++
start, end, isPtr, err := NextLabel(msg[off:])
if err != nil {
return off, err
} else if start == end {
if ptr == 0 {
newOff = currOff
newOff = off + 1 // advance past the null terminator byte
}
// Don't follow too many pointers, maybe there's a loop.
if ptr++; ptr > 10 {
return off, errTooManyPtr
break
} else if isPtr {
if !allowCompression {
return newOff, errCompressedSRV
}
currOff = (c^0xC0)<<8 | uint16(c1)
default:
// Prefixes 0x80 and 0x40 are reserved.
return off, errReserved
if ptr == 0 {
newOff = off + 2 // next record follows the 2-byte pointer
}
off = start
if int(off) >= len(msg) {
return newOff, errInvalidPtr
} else if ptr++; ptr > 10 {
return newOff, errTooManyPtr
}
} else {
// Is normal label; start/end are relative to msg[off:].
fn(msg[off+start : off+end])
off += end
}
}
if ptr == 0 {
newOff = currOff
}
return newOff, nil
}
// NextLabel parses the first control byte of data and returns the position and extent of next DNS label.
//
// For a normal string label (RFC 1035 §3.1), isPointer==false and start/end are
// byte indices into data: data[start:end] holds the raw label bytes.
// A null terminator (c==0) signals the end of the name: start==end==1, err==nil.
//
// For a compression pointer (RFC 1035 §4.1.4), isPointer==true:
// - start is the absolute target offset within the full DNS message to jump to.
// - end==0 (sentinel; not a data range).
//
// Returns [lneto.ErrTruncatedFrame] if data is too short to read the full label or pointer.
// Returns errReserved for the 0x40 and 0x80 reserved prefix classes.
func NextLabel(data []byte) (start_RelOrAbs, endRel uint16, isAbsPointer bool, err error) {
// Default invalid values
start_RelOrAbs, endRel = 0, 0
if len(data) == 0 {
return start_RelOrAbs, endRel, false, lneto.ErrTruncatedFrame
}
c := uint16(data[0])
switch c & 0xc0 {
case 0:
start_RelOrAbs = 1
// String label segment.
if c == 0 {
return start_RelOrAbs, start_RelOrAbs, false, nil // Null terminator. String ended.
}
endRel = start_RelOrAbs + c
if int(endRel) > len(data) {
return start_RelOrAbs, endRel, false, lneto.ErrTruncatedFrame
}
// Reject names containing dots. See issue golang/go#56246
if bytes.IndexByte(data[start_RelOrAbs:endRel], '.') >= 0 {
return start_RelOrAbs, endRel, false, errInvalidName
}
// Correct label!
case 0xc0:
// Pointer. Start is absolute index in DNS message.
isAbsPointer = true
if len(data) < 2 {
return start_RelOrAbs, endRel, isAbsPointer, lneto.ErrTruncatedFrame // Need more data to fully read pointer.
}
c1 := uint16(data[1])
start_RelOrAbs = (c^0xC0)<<8 | c1
default:
err = errReserved
}
return start_RelOrAbs, endRel, isAbsPointer, err
}
func (dst *Message) CopyFrom(m Message) {
internal.SliceReuse(&dst.Questions, len(m.Questions))
internal.SliceReuse(&dst.Answers, len(m.Answers))
+21 -20
View File
@@ -2,6 +2,7 @@ package dns
import (
"fmt"
"net/netip"
"strings"
"testing"
)
@@ -195,32 +196,32 @@ func TestMessageAppendEncodeIncompleteOK(t *testing.T) {
func (m *Message) String() string {
// s := fmt.Sprintf("Message: %#v\n", &m.Header)
var s string
var s strings.Builder
if len(m.Questions) > 0 {
s += "-- Questions\n"
s.WriteString("-- Questions\n")
for _, q := range m.Questions {
s += fmt.Sprintf("%#v\n", q)
s.WriteString(fmt.Sprintf("%#v\n", q))
}
}
if len(m.Answers) > 0 {
s += "-- Answers\n"
s.WriteString("-- Answers\n")
for _, a := range m.Answers {
s += fmt.Sprintf("%#v\n", a)
s.WriteString(fmt.Sprintf("%#v\n", a))
}
}
if len(m.Authorities) > 0 {
s += "-- Authorities\n"
s.WriteString("-- Authorities\n")
for _, ns := range m.Authorities {
s += fmt.Sprintf("%#v\n", ns)
s.WriteString(fmt.Sprintf("%#v\n", ns))
}
}
if len(m.Additionals) > 0 {
s += "-- Additionals\n"
s.WriteString("-- Additionals\n")
for _, e := range m.Additionals {
s += fmt.Sprintf("%#v\n", e)
s.WriteString(fmt.Sprintf("%#v\n", e))
}
}
return s
return s.String()
}
func TestDecodeMessage(t *testing.T) {
@@ -288,23 +289,23 @@ func TestClient_ReceivesDNSResponse(t *testing.T) {
}
// Check the client received the answer.
answers := client.Answers()
if len(answers) != 1 {
t.Fatalf("expected 1 answer, got %d", len(answers))
var addrs [4]netip.Addr
answers, err := client.ResponseAnswerLookup(addrs[:], hostname)
if answers != 1 {
t.Fatalf("expected 1 answer, got %d", answers)
}
data := answers[0].RawData()
if len(data) != 4 {
t.Fatalf("expected 4 bytes in answer, got %d", len(data))
addr := addrs[0]
if !addr.Is4() {
t.Fatalf("expected 4 bytes in answer, got %d", addr.BitLen()/8)
}
if [4]byte(data) != wantIP {
t.Errorf("expected IP %v, got %v", wantIP, data)
if addr.As4() != wantIP {
t.Errorf("expected IP %v, got %v", wantIP, addr.String())
}
// Test MessageCopyTo as well.
var lookup Message
lookup.LimitResourceDecoding(1, 1, 0, 0)
done, err := client.MessageCopyTo(&lookup)
done, err := client.ResponseCopyTo(&lookup)
if err != nil {
t.Fatal("MessageCopyTo error:", err)
}
+6 -4
View File
@@ -119,10 +119,11 @@ const _RCode_name = "successformat errorserver failurename errornot implementedr
var _RCode_index = [...]uint8{0, 7, 19, 33, 43, 58, 65}
func (i RCode) String() string {
if i >= RCode(len(_RCode_index)-1) {
idx := int(i) - 0
if i < 0 || idx >= len(_RCode_index)-1 {
return "RCode(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _RCode_name[_RCode_index[i]:_RCode_index[i+1]]
return _RCode_name[_RCode_index[idx]:_RCode_index[idx+1]]
}
func _() {
// An "invalid array index" compiler error signifies that the constant values have changed.
@@ -138,8 +139,9 @@ const _OpCode_name = "Standard queryInverse queryServer status request"
var _OpCode_index = [...]uint8{0, 14, 27, 48}
func (i OpCode) String() string {
if i >= OpCode(len(_OpCode_index)-1) {
idx := int(i) - 0
if i < 0 || idx >= len(_OpCode_index)-1 {
return "OpCode(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _OpCode_name[_OpCode_index[i]:_OpCode_index[i+1]]
return _OpCode_name[_OpCode_index[idx]:_OpCode_index[idx+1]]
}
+4 -1
View File
@@ -17,16 +17,19 @@ const (
ErrMismatch // mismatch
ErrMismatchLen // mismatched length
ErrInvalidConfig // invalid configuration
_ // little stitious
ErrInvalidField // invalid field
ErrInvalidLengthField // invalid length field
ErrExhausted // resource exhausted
ErrAlreadyRegistered // protocol already registered
ErrTruncatedFrame // truncated frame
ErrMissingHALConfig // missing HAL configuration
ErrBadState // operation invalid in current state
// Below are potentially good future error additions
// based on one or two encountered use cases, example use case included.
/*
- ErrUnregistered/ErrAborted // connection unregistered. i.e: ICMP client aborted during active ping, ping process returns this.
- ErrInvalidArgs // invalid func arguments i.e: different from Config since this refers to non-config arguments. usually nil values.
*/
)
+13 -2
View File
@@ -5,13 +5,24 @@ import (
)
const (
// MaxMTU defines the maximum payload of a standard ethernet frame. Does NOT include ethernet header, FCS and VLAN tag.
// Ethernet frames can be larger but this is out of the 802.3 standard and going into jumbo frame territory.
MaxMTU = 1500
// MaxFrameLength (1522) defines the maximum length of a standard ethernet frame including headers, FCS and VLAN if present.
MaxFrameLength = MaxMTU + MaxOverheadSize
// MaxOverheadSize is the maximum overhead a packet can incur
// from transmitting an ethernet frame. Includes:
// - 14 bytes of Ethernet header containing MAC addresses and ethernet type, always present
// - 4 bytes of VLAN tag, if present.
// - 4 bytes of VLAN tag, if present. See 802.1Q.
// - 4 bytes of the 32 bit trailing CRC, if required by PHY.
MaxOverheadSize = 14 + 4 + 4
MaxOverheadSize = sizeHeaderNoVLAN + fcsOverhead + vlanOverhead
vlanOverhead = 4
fcsOverhead = 4
sizeHeaderNoVLAN = 14
// MinimumFrameLength as defined by IEEE 802.3 standards. Includes ethernet header.
MinimumFrameLength = 64
// Does not include VLAN/FCS for more conservative minimum MTU.
MinimumMTU = MinimumFrameLength - sizeHeaderNoVLAN
)
// AppendAddr appends the text representation of the hardware address to the destination buffer.
+8
View File
@@ -0,0 +1,8 @@
# import-examples
This folder contains examples that need external imports to work.
This is done to avoid adding dependencies to lneto's go.mod file.
- Trivial Auditing
- No dependency analysis needed for vulnerability scanning
- Eliminate a whole set of attack vectors for lneto importers
- Ensures nothing outside Lneto gets compiled maintaining binary sizes small
@@ -0,0 +1,14 @@
module espradio-netdev
go 1.25.7
// These replace directives point to local checkouts during development.
// Remove them when using as a standalone program with published releases.
replace github.com/soypat/lneto => ../../../.
replace tinygo.org/x/espradio => ../../../../espradio
require (
github.com/soypat/lneto v0.1.1-0.20260425023453-aa77403a2b32
tinygo.org/x/espradio v0.1.0
)
@@ -0,0 +1,223 @@
// Example showing how to use the ESP32 WiFi radio through lneto's netdev
// package. This mirrors the picow-netdev example and demonstrates the
// standardised DevEthernet+Netlink interface that works across hardware targets.
//
// Build and flash:
//
// tinygo flash -target xiao-esp32c3 \
// -ldflags="-X main.ssid=YourSSID -X main.password=YourPassword" \
// -monitor ./examples/esp32-netdev
package main
import (
"context"
_ "embed"
"net/netip"
"strings"
"time"
"github.com/soypat/lneto"
"github.com/soypat/lneto/x/netdev"
"github.com/soypat/lneto/x/xnet"
"tinygo.org/x/espradio"
)
var (
//go:embed wifi.credentials
credentials string
// remove windows CRLF "\r\n" and trailing newline.
credentialsNormalized = strings.TrimSuffix(strings.ReplaceAll(credentials, "\r\n", "\n"), "\n")
globConnectParams espradio.STAConfig
poolCfg = xnet.TCPPoolConfig{
PoolSize: 4,
QueueSize: 4,
TxBufSize: 2048,
RxBufSize: 512,
NewBackoff: func() lneto.BackoffStrategy { return backoff },
NanoTime: func() int64 { return time.Now().UnixNano() },
EstablishedTimeout: 5 * time.Second,
ClosingTimeout: 3 * time.Second,
}
)
func main() {
time.Sleep(time.Second)
ssid, password, ok := strings.Cut(credentialsNormalized, "\n")
if !ok {
fail("must write newline separated ssid/password in wifi.credentials", nil)
}
globConnectParams.SSID = ssid
globConnectParams.Password = password
var dev EspDev
dev.radioConfig = espradio.Config{Logging: espradio.LogLevelError}
// LinkConnect runs Enable+Start+Connect+StartNetDev. It must complete
// before HardwareAddr6 is called because the MAC is only readable after
// Enable() initialises the WiFi hardware.
err := dev.LinkConnect(globConnectParams)
failIfErr("wifi connect", err)
hw, err := dev.HardwareAddr6()
failIfErr("hardware addr", err)
var stack xnet.Netstack
err = stack.Reset(xnet.StackConfig{
RandSeed: time.Now().UnixNano() | 1,
Hostname: "esp32-lneto",
MaxActiveTCPPorts: 4,
MaxActiveUDPPorts: 4,
ICMPQueueLimit: 1,
MTU: 1500,
PassivePeers: 4,
HardwareAddress: hw,
}, backoff, poolCfg)
failIfErr("stack reset", err)
err = stack.EnableICMP(true)
failIfErr("icmp enable", err)
dev.LinkNotify(userNotify)
var iface netdev.Interface[espradio.STAConfig]
err = iface.Init(&dev, &dev, netdev.InterfaceConfig{})
failIfErr("init iface", err)
var runner netdev.Runner[espradio.STAConfig]
go func() {
// EthPoll drains the C ring buffer and delivers frames through the
// receive handler, so the device is async but still needs the pump.
err := runner.Configure(netdev.RunnerConfig[espradio.STAConfig]{
Buffers: iface.RunnerBuffers(2),
Backoff: backoff,
Flags: netdev.RunnerInterfaceAsync | netdev.RunnerInterfacePoll,
})
if err != nil {
failIfErr("runnerconfig", err)
}
if err := runner.Run(context.Background(), &iface, &stack); err != nil {
failIfErr("runner", err)
}
}()
assigned, gatewayRt, subnetBits, err := stack.EnableDHCP(context.Background(), true, netip.Addr{})
failIfErr("enable dhcp", err)
println("assigned=", assigned.String(), "gateway=", gatewayRt.String(), "subnet=", subnetBits)
select {}
}
// compile-time interface checks.
var _ lneto.BackoffStrategy = backoff
var _ netdev.Stack = (*xnet.Netstack)(nil)
var _ netdev.DevEthernet = (*EspDev)(nil)
var _ netdev.Netlink[espradio.STAConfig] = (*EspDev)(nil)
func backoff(consecutiveBackoffs uint) time.Duration {
return 5 * time.Millisecond
}
func userNotify(connected bool) (retries int, reconnectParams espradio.STAConfig) {
if !connected {
return 1, globConnectParams
}
return 0, espradio.STAConfig{}
}
// EspDev adapts [espradio.NetDev] to [netdev.DevEthernet] and wraps the
// ESP32 WiFi bring-up sequence (Enable/Start/Connect/StartNetDev) as [netdev.Netlink].
//
// The same struct implements both interfaces so a single pointer can be passed
// to [netdev.Interface.Init] for both the netlink and device arguments, matching
// the pattern used by the picow-netdev example.
type EspDev struct {
nd *espradio.NetDev
radioConfig espradio.Config
notifyCb netdev.NotifyCallback[espradio.STAConfig]
}
// LinkConnect implements [netdev.Netlink].
// Runs Enable→Start→Connect→StartNetDev. nd is nil until this returns successfully.
func (d *EspDev) LinkConnect(cfg espradio.STAConfig) error {
if err := espradio.Enable(d.radioConfig); err != nil {
return err
}
if err := espradio.Start(); err != nil {
return err
}
if err := espradio.Connect(cfg); err != nil {
return err
}
nd, err := espradio.StartNetDev()
if err != nil {
return err
}
d.nd = nd
return nil
}
// LinkDisconnect implements [netdev.Netlink].
func (d *EspDev) LinkDisconnect() {}
// LinkNotify implements [netdev.Netlink].
func (d *EspDev) LinkNotify(cb netdev.NotifyCallback[espradio.STAConfig]) {
d.notifyCb = cb
}
// HardwareAddr6 implements [netdev.DevEthernet].
func (d *EspDev) HardwareAddr6() ([6]byte, error) {
return d.nd.HardwareAddr6()
}
// SendOffsetEthFrame implements [netdev.DevEthernet].
// ESP32 has no frame prefix offset, so buf is passed directly to [espradio.NetDev.SendEthFrame].
func (d *EspDev) SendOffsetEthFrame(buf []byte) error {
return d.nd.SendEthFrame(buf)
}
// SetEthRecvHandler implements [netdev.DevEthernet].
// Adapts the error-less netdev handler signature to espradio's error-returning one.
func (d *EspDev) SetEthRecvHandler(handler func(rxEthframe []byte)) {
if handler == nil {
d.nd.SetEthRecvHandler(nil)
return
}
d.nd.SetEthRecvHandler(func(pkt []byte) error {
handler(pkt)
return nil
})
}
// EthPoll implements [netdev.DevEthernet].
//
// espradio.NetDev.EthPoll both pops a frame from the C ring buffer into buf
// AND synchronously calls the registered rxHandler with that frame. Returning
// the non-zero byte count here as well would trigger the Runner's "device uses
// both paths" guard. We therefore drain the ring (calling the handler) and
// return (0, 0, err) so the Runner sees only the handler-based receive path.
func (d *EspDev) EthPoll(buf []byte) (ethFrameOff, ethernetBytes int, err error) {
_, err = d.nd.EthPoll(buf)
return 0, 0, err
}
// MaxFrameSizeAndOffset implements [netdev.DevEthernet].
// ESP32 transmits frames with no prefix offset.
func (d *EspDev) MaxFrameSizeAndOffset() (maxFrameSize int, frameOff int) {
return d.nd.MaxFrameSize(), 0
}
func failIfErr(msg string, err error) {
if err != nil {
fail(msg, err)
}
println(msg, "PASS")
}
func fail(msg string, err error) {
var errstr string
if err != nil {
errstr = err.Error()
}
for {
println("FAIL:", msg, errstr)
time.Sleep(time.Second)
}
}
@@ -0,0 +1,11 @@
module gvisormwe
go 1.25.7
require (
github.com/google/btree v1.1.2 // indirect
github.com/usbarmory/go-net v0.0.0-20260416163630-1078311e0956 // indirect
golang.org/x/sys v0.26.0 // indirect
golang.org/x/time v0.7.0 // indirect
gvisor.dev/gvisor v0.0.0-20250911055229-61a46406f068 // indirect
)
@@ -0,0 +1,10 @@
github.com/google/btree v1.1.2 h1:xf4v41cLI2Z6FxbKm+8Bu+m8ifhj15JuZ9sa0jZCMUU=
github.com/google/btree v1.1.2/go.mod h1:qOPhT0dTNdNzV6Z/lhRX0YXUafgPLFUh+gZMl761Gm4=
github.com/usbarmory/go-net v0.0.0-20260416163630-1078311e0956 h1:ZjhVXLMT/Ogxs1GIy1g8UJk7yi9G8kt90D0ElAPbaCc=
github.com/usbarmory/go-net v0.0.0-20260416163630-1078311e0956/go.mod h1:+6WiKCFJtJQZdNM2VpwQsYGo/aBJ39pN7nWx6Td3Z8s=
golang.org/x/sys v0.26.0 h1:KHjCJyddX0LoSTb3J+vWpupP9p0oznkqVk/IfjymZbo=
golang.org/x/sys v0.26.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/time v0.7.0 h1:ntUhktv3OPE6TgYxXWv9vKvUSJyIFJlyohwbkEwPrKQ=
golang.org/x/time v0.7.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
gvisor.dev/gvisor v0.0.0-20250911055229-61a46406f068 h1:95kdltF/maTDk/Wulj7V81cSLgjB/Mg/6eJmOKsey4U=
gvisor.dev/gvisor v0.0.0-20250911055229-61a46406f068/go.mod h1:K16uJjZ+hSqDVsXhU2Rg2FpMN7kBvjZp/Ibt5BYZJjw=
@@ -0,0 +1,93 @@
package main
import (
"context"
"fmt"
"net"
"net/netip"
"os"
"syscall"
"time"
gnet "github.com/usbarmory/go-net"
)
const pollTime = 5 * time.Millisecond
var networkDevice NetworkDevice
// NetworkDevice implements gnet.NetworkDevice for bridging with raw Ethernet I/O.
type NetworkDevice struct {
send func([]byte) error
recv func([]byte) (int, error)
}
func (d *NetworkDevice) Transmit(buf []byte) error { return d.send(buf) }
func (d *NetworkDevice) Receive(buf []byte) (int, error) { return d.recv(buf) }
func main() {
ctx := context.Background()
if err := run(ctx); err != nil {
fmt.Println(err)
os.Exit(1)
}
}
func run(ctx context.Context) error {
// Create gVisor-based networking stack.
stack := gnet.NewGVisorStack(1)
// Configure stack with MAC, IP prefix, and gateway.
err := stack.Configure(
"aa:bb:cc:dd:ee:ff",
netip.MustParsePrefix("192.168.1.10/24"),
netip.MustParseAddr("192.168.1.1"),
)
if err != nil {
return fmt.Errorf("configuring stack: %w", err)
}
err = stack.EnableICMP()
if err != nil {
return fmt.Errorf("enabling ICMP: %w", err)
}
// Bridge the stack with a network device for packet I/O.
iface := &gnet.Interface{
Stack: stack,
NetworkDevice: &networkDevice,
HandleStackErr: func(err error, tx bool) {
dir := "rx"
if tx {
dir = "tx"
}
fmt.Printf("stack %s err: %v\n", dir, err)
},
}
// Start the packet processing loop in a goroutine.
go iface.Start()
// Create a TCP listener on port 80 using the gVisor stack.
const sockStream = 0x1
laddr := net.TCPAddrFromAddrPort(netip.AddrPortFrom(netip.MustParseAddr("192.168.1.10"), 80))
c, err := stack.Socket(ctx, "tcp", syscall.AF_INET, sockStream, laddr, nil)
if err != nil {
return fmt.Errorf("creating AF_INET stream socket: %w", err)
}
listener := c.(net.Listener)
for ctx.Err() == nil {
time.Sleep(pollTime)
conn, err := listener.Accept()
if err != nil {
fmt.Println("conn failed:", err)
continue
}
go handleConn(conn)
}
return nil
}
func handleConn(conn net.Conn) {
defer conn.Close()
conn.Write([]byte("Hello from gVisor stack!"))
}
@@ -0,0 +1,21 @@
module piconetdev
go 1.25.7
require (
github.com/soypat/cyw43439 v0.1.1
github.com/soypat/lneto v0.1.1-0.20260425023453-aa77403a2b32
)
require (
github.com/soypat/seqs v0.0.0-20250124201400-0d65bc7c1710 // indirect
github.com/tinygo-org/pio v0.2.0 // indirect
golang.org/x/exp v0.0.0-20240808152545-0cdaa3abc0fa // indirect
)
// This is an example taken grom github.com/soypat/lneto
// Remove this replace directive when using as own program.
replace github.com/soypat/lneto => ../../../.
// Local cyw43439 with the poll-based EthPoll API.
replace github.com/soypat/cyw43439 => ../../../../cyw43439
@@ -0,0 +1,6 @@
github.com/soypat/seqs v0.0.0-20250124201400-0d65bc7c1710 h1:Y9fBuiR/urFY/m76+SAZTxk2xAOS2n85f+H1CugajeA=
github.com/soypat/seqs v0.0.0-20250124201400-0d65bc7c1710/go.mod h1:oCVCNGCHMKoBj97Zp9znLbQ1nHxpkmOY9X+UAGzOxc8=
github.com/tinygo-org/pio v0.2.0 h1:vo3xa6xDZ2rVtxrks/KcTZHF3qq4lyWOntvEvl2pOhU=
github.com/tinygo-org/pio v0.2.0/go.mod h1:LU7Dw00NJ+N86QkeTGjMLNkYcEYMor6wTDpTCu0EaH8=
golang.org/x/exp v0.0.0-20240808152545-0cdaa3abc0fa h1:ELnwvuAXPNtPk1TJRuGkI9fDTwym6AYBu0qzT8AcHdI=
golang.org/x/exp v0.0.0-20240808152545-0cdaa3abc0fa/go.mod h1:akd2r19cwCdwSwWeIdzYQGa/EZZyqcOdwWiwj5L5eKQ=
@@ -0,0 +1,189 @@
package main
import (
"context"
_ "embed"
"net"
"net/netip"
"strings"
"time"
"github.com/soypat/cyw43439"
"github.com/soypat/lneto"
"github.com/soypat/lneto/x/netdev"
"github.com/soypat/lneto/x/xnet"
)
var (
//go:embed wifi.credentials
credentials string
// remove windows CRLF "\r\n" and trailing newline.
credentialsNormalized = strings.TrimSuffix(strings.ReplaceAll(credentials, "\r\n", "\n"), "\n")
globConnectParams ConnectParams
poolCfg = xnet.TCPPoolConfig{
PoolSize: 5,
QueueSize: 5,
TxBufSize: 2048,
RxBufSize: 2048,
NewBackoff: func() lneto.BackoffStrategy { return backoff },
NanoTime: func() int64 { return time.Now().UnixNano() },
EstablishedTimeout: 5 * time.Second,
ClosingTimeout: 3 * time.Second,
}
)
func main() {
time.Sleep(time.Second)
ssid, password, ok := strings.Cut(credentialsNormalized, "\n")
if !ok {
fail("must write newline separated ssid/password in wifi.credentials", nil)
}
globConnectParams.SSID = ssid
globConnectParams.Passphrase = password
dev := Netdev{
dev: cyw43439.NewPicoWDevice(),
}
err := dev.dev.Init(cyw43439.DefaultWifiConfig())
failIfErr("init cyw43439", err)
hw, _ := dev.HardwareAddr6()
var stack xnet.Netstack
err = stack.Reset(xnet.StackConfig{
RandSeed: time.Now().UnixNano() | 1,
Hostname: "lneto-pico",
MaxActiveTCPPorts: 4,
MaxActiveUDPPorts: 4,
ICMPQueueLimit: 1,
MTU: 1500,
HardwareAddress: hw,
}, backoff, poolCfg)
failIfErr("stack reset", err)
err = stack.EnableICMP(true)
failIfErr("icmp enable", err)
err = dev.LinkConnect(globConnectParams)
failIfErr("wifi join", err)
var iface netdev.Interface[ConnectParams]
var runner netdev.Runner[ConnectParams]
dev.LinkNotify(userNotify)
err = iface.Init(&dev, &dev, netdev.InterfaceConfig{})
failIfErr("init iface", err)
go func() {
err := runner.Configure(netdev.RunnerConfig[ConnectParams]{
Buffers: iface.RunnerBuffers(1),
Backoff: backoff,
Flags: netdev.RunnerInterfaceAsync | netdev.RunnerInterfacePoll,
})
if err != nil {
failIfErr("runnerconfig", err)
}
if err := runner.Run(context.Background(), &iface, &stack); err != nil {
failIfErr("runner", err)
}
}()
assigned, gatewayRt, subnetBits, err := stack.EnableDHCP(context.Background(), true, netip.Addr{})
failIfErr("enable dhcp", err)
println("assigned=", assigned.String(), "gateway=", gatewayRt.String(), "subnet", subnetBits)
for {
runner.PrintDebug()
time.Sleep(2 * time.Second)
}
}
// compile-time guarantee of interface implementation.
var _ lneto.BackoffStrategy = backoff
var _ netdev.Stack = (*xnet.Netstack)(nil)
func backoff(consecutiveBackoffs uint) (sleepOrFlag time.Duration) {
return 5 * time.Millisecond
}
func userNotify(connected bool) (retries int, connectParams ConnectParams) {
if !connected {
return 1, globConnectParams
}
return 0, ConnectParams{}
}
var _ netdev.DevEthernet = (*Netdev)(nil)
var _ netdev.Netlink[ConnectParams] = (*Netdev)(nil)
type Netdev struct {
dev *cyw43439.Device
notifyCb netdev.NotifyCallback[ConnectParams]
}
type ConnectParams struct {
SSID string
cyw43439.JoinOptions
}
func (nl *Netdev) Netflags() (flags net.Flags) {
flags |= net.FlagUp
if nl.dev.IsLinkUp() {
flags |= net.FlagRunning
}
return flags
}
// LinkConnect implements [netdev.Netlink].
func (nl *Netdev) LinkConnect(connectParams ConnectParams) error {
return nl.dev.Join(connectParams.SSID, connectParams.JoinOptions)
}
// LinkDisconnect implements [netdev.Netlink].
func (nl *Netdev) LinkDisconnect() {
// Not implemented by cyw43439 package.
}
// LinkNotify implements [netdev.Netlink].
func (nl *Netdev) LinkNotify(cb netdev.NotifyCallback[ConnectParams]) {
nl.notifyCb = cb
}
// HardwareAddr6 implements [netdev.DevEthernet].
func (d *Netdev) HardwareAddr6() ([6]byte, error) {
return d.dev.HardwareAddr6()
}
// SendEthFrameOffset implements [netdev.DevEthernet].
func (d *Netdev) SendOffsetEthFrame(offsetTxEthFrame []byte) error {
return d.dev.SendEth(offsetTxEthFrame)
}
// SetRecvHandler implements [netdev.DevEthernet].
//
// The cyw43439 delivers received Ethernet frames through this callback whenever
// the bus is serviced (including ioctl/control transactions outside EthPoll), so
// the runner must drive it in async mode. EthPoll is still used to pump the device.
func (d *Netdev) SetEthRecvHandler(handler func(rxEthframe []byte)) {
d.dev.RecvEthHandle(handler)
}
// EthPoll implements [netdev.DevEthernet].
func (d *Netdev) EthPoll(buf []byte) (ethFrameOff, ethernetBytes int, err error) {
return d.dev.EthPoll(buf)
}
// MaxFrameSizeAndOffset implements [netdev.DevEthernet].
func (d *Netdev) MaxFrameSizeAndOffset() (maxFrameSize int, frameOff int) {
return 2048, 0
}
func failIfErr(msg string, err error) {
if err != nil {
fail(msg, err)
}
println("PASS", msg)
}
func fail(msg string, err error) {
var errstr string
if err != nil {
errstr = err.Error()
}
for {
println(msg, errstr)
time.Sleep(time.Second)
}
}
+38 -15
View File
@@ -27,6 +27,7 @@ import (
"github.com/soypat/lneto/internal"
"github.com/soypat/lneto/internal/ltesto"
"github.com/soypat/lneto/internet/pcap"
"github.com/soypat/lneto/ipv4"
"github.com/soypat/lneto/tcp"
"github.com/soypat/lneto/x/xnet"
)
@@ -134,8 +135,9 @@ func run() error {
pfbuf = fmt.Appendf(pfbuf[:0], "%-3s %3d", context, len(pkt))
pfbuf = append(pfbuf, ' ', '[')
pfbuf, err = pf.FormatFrames(pfbuf, frames, pkt)
pfbuf = bytes.ReplaceAll(pfbuf, stack.Addr().AppendTo(nil), []byte("us"))
pfbuf = bytes.ReplaceAll(pfbuf, ethernet.AppendAddr(nil, stack.HardwareAddress()), []byte("us"))
addr := stack.Addr4()
pfbuf = bytes.ReplaceAll(pfbuf, ipv4.AppendFormatAddr(nil, addr), []byte("us"))
pfbuf = bytes.ReplaceAll(pfbuf, ethernet.AppendAddr(nil, stack.HardwareAddr()), []byte("us"))
pfbuf = append(pfbuf, ']', '\n')
if err != nil {
return err
@@ -158,11 +160,11 @@ func run() error {
} else if n != nwrite {
log.Fatalf("mismatch written bytes %d!=%d", nwrite, n)
}
if flagMockClient && mockStack.Addr().IsValid() {
if flagMockClient && mockStack.Addr4() != ([4]byte{}) {
mockStack.IngressEthernet(buf[:nwrite])
}
}
if flagMockClient && mockStack.Addr().IsValid() {
if flagMockClient && mockStack.Addr4() != ([4]byte{}) {
n, _ := mockStack.EgressEthernet(buf[:])
if n > 0 {
stack.IngressEthernet(buf[:n])
@@ -199,7 +201,7 @@ func run() error {
}()
// Create blocking + Berkeley stack
blocking := stack.StackBlocking(5 * time.Millisecond)
blocking := stack.StackBlocking(stackBackoff)
berkeley := blocking.StackGo(xnet.StackGoConfig{
ListenerPoolConfig: xnet.TCPPoolConfig{
PoolSize: uint16(flagPoolSize),
@@ -208,11 +210,12 @@ func run() error {
RxBufSize: mtu,
EstablishedTimeout: 5 * time.Second,
ClosingTimeout: 5 * time.Second,
NewBackoff: func() lneto.BackoffStrategy { return tcpBackoff },
},
})
// Perform DHCP to get address.
rstack := stack.StackRetrying(5 * time.Millisecond)
rstack := stack.StackRetrying(stackBackoff)
const dhcpTimeout = 6 * time.Second
const dhcpRetries = 2
results, err := rstack.DoDHCPv4([4]byte{192, 168, 1, 96}, dhcpTimeout, dhcpRetries)
@@ -222,7 +225,7 @@ func run() error {
if err = stack.AssimilateDHCPResults(results); err != nil {
return fmt.Errorf("assimilating DHCP results: %w", err)
}
slog.Info("dhcp-complete", slog.String("assignedIP", results.AssignedAddr.String()), slog.String("routerIP", results.Router.String()))
slog.Info("dhcp-complete", slog.String("assignedIP", string(ipv4.AppendFormatAddr(nil, results.AssignedAddr4))), slog.String("routerIP", results.Router.String()))
// Resolve router HW and set gateway
routerHw, err := rstack.DoResolveHardwareAddress6(results.Router, 2*time.Second, 2)
@@ -230,7 +233,7 @@ func run() error {
return fmt.Errorf("ARP resolution of router failed: %w", err)
}
// Set gateway on the async stack (exported API).
stack.SetGateway6(routerHw)
stack.SetGatewayHardwareAddr(routerHw)
// Create Berkeley listener via SocketNetip
laddr := netip.AddrPortFrom(netip.IPv4Unspecified(), uint16(flagPort))
@@ -246,7 +249,7 @@ func run() error {
}
defer ln.Close()
fmt.Printf("Listening (Berkeley) on %s:%d\n", stack.Addr().String(), flagPort)
fmt.Printf("Listening (Berkeley) on %s:%d\n", string(ipv4.AppendFormatAddr(nil, stack.Addr4())), flagPort)
// Optionally run an in-memory mock client that dials the berkeley listener
if flagMockClient {
@@ -285,7 +288,7 @@ func handleConnNet(conn net.Conn) error {
}
}
method := string(hdr.Method())
uri := string(hdr.RequestURI())
uri := string(hdr.RequestTarget())
fmt.Printf("< %s %s\n", method, uri)
var resp httpraw.Header
@@ -326,11 +329,11 @@ func tryPoll(iface ltesto.Interface, poll time.Duration) (dataMayBeReady bool, _
}
func mockClient(stack *xnet.StackAsync, port uint16, subnet netip.Prefix) {
target := netip.AddrPortFrom(stack.Addr(), port)
target := netip.AddrPortFrom(netip.AddrFrom4(stack.Addr4()), port)
err := mockStack.Reset(xnet.StackConfig{
StaticAddress: subnet.Addr().Next(),
StaticAddress4: subnet.Addr().Next().As4(),
MaxActiveTCPPorts: 1,
HardwareAddress: stack.Gateway6(),
HardwareAddress: stack.GatewayHardwareAddr(),
Hostname: "the-other",
MTU: uint16(stack.MTU()),
RandSeed: int64(stack.Prand32()),
@@ -344,6 +347,7 @@ func mockClient(stack *xnet.StackAsync, port uint16, subnet netip.Prefix) {
TxBuf: make([]byte, 2048),
TxPacketQueueSize: 4,
Logger: slog.Default(),
RWBackoff: tcpBackoff,
})
if err != nil {
panic(err.Error())
@@ -366,9 +370,9 @@ func mockClient(stack *xnet.StackAsync, port uint16, subnet netip.Prefix) {
var hdr httpraw.Header
hdr.SetMethod("GET")
hdr.SetRequestURI("/")
hdr.SetRequestTarget("/")
hdr.SetProtocol("HTTP/1.1")
hdr.Set("Host", stack.Addr().String())
hdr.Set("Host", string(ipv4.AppendFormatAddr(nil, stack.Addr4())))
hdr.Set("User-Agent", "lneto-mock")
hdr.Set("Connection", "close")
req, err := hdr.AppendRequest(nil)
@@ -399,3 +403,22 @@ func mockClient(stack *xnet.StackAsync, port uint16, subnet netip.Prefix) {
fmt.Println("mockclient: received response:\n", string(page))
mockConn.Close()
}
func stackBackoff(consecutiveBackoffs uint) time.Duration {
if consecutiveBackoffs < 10 {
return time.Millisecond
}
return 10 * time.Millisecond
}
func tcpBackoff(consecutiveBackoffs uint) time.Duration {
const (
minWait = uint32(time.Microsecond)
maxWait = 5 * uint32(time.Millisecond)
maxShift = 22
_overflowCheck = minWait << maxShift
)
shifted := minWait << min(consecutiveBackoffs, maxShift)
wait := min(shifted, maxWait)
return time.Duration(wait)
}
+189
View File
@@ -0,0 +1,189 @@
//go:build !tinygo && linux
package main
import (
"flag"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"time"
)
type program struct {
Name string
Link string // relative path for README link
Dir string // directory to build from, relative to repo root
ExtraProtocols string
PacketCapture bool
}
type buildTarget struct {
Name string
ext string // output file extension (determines format for tinygo)
build func(dir, outFile string) error
}
type result struct {
BinarySize int64
CompileTime time.Duration
DNC bool // Does Not Compile
Err error
}
func (r result) sizeString() string {
if r.DNC {
return "DNC"
}
if r.Err != nil {
return "ERR"
}
return formatSize(r.BinarySize)
}
func formatSize(n int64) string {
const mb = 1024 * 1024
if n >= mb {
return fmt.Sprintf("%.1fMB", float64(n)/mb)
}
return fmt.Sprintf("%dkB", (n+512)/1024)
}
func goBuild(goos, goarch string) func(dir, out string) error {
return func(dir, out string) error {
cmd := exec.Command("go", "build", "-o", out, ".")
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOOS="+goos, "GOARCH="+goarch)
if out, err := cmd.CombinedOutput(); err != nil {
return fmt.Errorf("%w: %s", err, out)
}
return nil
}
}
func tinygoBuild(target string) func(dir, out string) error {
return func(dir, out string) error {
args := []string{"build"}
if target != "" {
args = append(args, "-target="+target)
}
args = append(args, "-o", out, ".")
cmd := exec.Command("tinygo", args...)
cmd.Dir = dir
if out, err := cmd.CombinedOutput(); err != nil {
return fmt.Errorf("%w: %s", err, out)
}
return nil
}
}
var buildTargets = []buildTarget{
{Name: "amd64 Go", ext: ".elf", build: goBuild("linux", "amd64")},
{Name: "WASM Go", ext: ".wasm", build: goBuild("wasip1", "wasm")},
{Name: "amd64 TinyGo", ext: ".elf", build: tinygoBuild("")},
{Name: "WASM TinyGo", ext: ".wasm", build: tinygoBuild("wasm")},
{Name: "Pico TinyGo", ext: ".bin", build: tinygoBuild("pico")},
}
var programs = []program{
{
Name: "Lneto MWE",
Link: "./examples/min-working-example/",
Dir: "examples/min-working-example",
ExtraProtocols: "DNS,NTP,DHCP",
PacketCapture: true,
},
{
Name: "Gvisor MWE w/ go-net",
Link: "./examples/_import_examples/gvisor-mwe/",
Dir: "examples/_import_examples/gvisor-mwe",
ExtraProtocols: "None",
PacketCapture: false,
},
}
func measure(dir, outFile string, fn func(dir, out string) error) result {
start := time.Now()
err := fn(dir, outFile)
elapsed := time.Since(start)
if err != nil {
return result{DNC: true, CompileTime: elapsed, Err: err}
}
fi, err := os.Stat(outFile)
if err != nil {
return result{Err: err, CompileTime: elapsed}
}
size := fi.Size()
os.Remove(outFile)
return result{BinarySize: size, CompileTime: elapsed}
}
func main() {
root := flag.String("root", ".", "path to repository root")
flag.Parse()
repoRoot, err := filepath.Abs(*root)
if err != nil {
panic(err)
}
tmpDir, err := os.MkdirTemp("", "binbench-*")
if err != nil {
panic(err)
}
defer os.RemoveAll(tmpDir)
type row struct {
prog program
results []result
}
rows := make([]row, len(programs))
for i, prog := range programs {
dir := filepath.Join(repoRoot, prog.Dir)
results := make([]result, len(buildTargets))
for j, bt := range buildTargets {
outFile := filepath.Join(tmpDir, fmt.Sprintf("p%d_t%d%s", i, j, bt.ext))
fmt.Fprintf(os.Stderr, "building %s for %s...\n", prog.Name, bt.Name)
r := measure(dir, outFile, bt.build)
if r.DNC {
fmt.Fprintf(os.Stderr, " DNC: %v\n", r.Err)
}
results[j] = r
}
rows[i] = row{prog: prog, results: results}
}
// Print markdown table.
headers := []string{"Program", "Extra Protocols", "Packet capture printing"}
for _, bt := range buildTargets {
headers = append(headers, bt.Name)
}
fmt.Printf("| %s |\n", strings.Join(headers, " | "))
aligns := make([]string, len(headers))
aligns[0] = "---"
aligns[1] = ":---:"
aligns[2] = ":---:"
for i := 3; i < len(aligns); i++ {
aligns[i] = "---"
}
fmt.Printf("|%s|\n", strings.Join(aligns, "|"))
for _, r := range rows {
pcap := "❌"
if r.prog.PacketCapture {
pcap = "✅"
}
cols := []string{
fmt.Sprintf("[%s](%s)", r.prog.Name, r.prog.Link),
r.prog.ExtraProtocols,
pcap,
}
for _, res := range r.results {
cols = append(cols, res.sizeString())
}
fmt.Printf("| %s |\n", strings.Join(cols, " | "))
}
}
+111
View File
@@ -0,0 +1,111 @@
//go:build !tinygo && linux
package main
import (
"log/slog"
"os"
"strconv"
"sync/atomic"
"time"
"github.com/soypat/lneto/http/httphi"
"github.com/soypat/lneto/x/rawsock"
)
const (
kB = 1 << 10
listenPort = 8080
bufferSizes = 2 * kB
// A browser sends around twenty header fields; a request carrying more
// than this is answered 431 rather than parsed into memory it was not
// given. Each field costs 8 bytes of table.
numHeaderFields = 32
numGoroutines = 4
readTimeout = 2 * time.Second
)
func main() {
if err := run(); err != nil {
println("Error:", err.Error())
os.Exit(1)
}
println("DONE")
}
func run() error {
var ln rawsock.Listener
err := ln.Listen(listenPort)
if err != nil {
return err
}
defer ln.Close()
print("listening on http://localhost:", listenPort, "\n")
var mux httphi.MuxSlice
mux.Handle("GET /", homepage)
var router httphi.Router
err = router.Configure(httphi.RouterConfig{
FixedNumGoroutines: numGoroutines,
RequestHeaderBufferSize: bufferSizes,
RequestNumHeaderKVCap: numHeaderFields,
ResponseHeaderMinBufferSize: bufferSizes,
MaxAwaitingConns: 256,
Backoff: func(consecutiveBackoffs uint) (sleepOrFlag time.Duration) {
return min(time.Second, time.Millisecond*time.Duration(consecutiveBackoffs))
},
Mux: &mux,
Logger: slog.Default(),
})
if err != nil {
return err
}
defer router.TeardownGoroutines()
for {
conn := new(rawsock.Conn)
err := ln.AcceptConn(conn)
if err != nil {
return err
}
// The connection owns the idle policy: a peer that opens a socket and
// then stalls fails its read instead of holding a router goroutine.
conn.SetReadTimeout(readTimeout)
err = router.Handle(conn)
if err != nil {
// Every goroutine is busy and the queue is full. Dropping the
// connection is the backpressure: memory stays bounded.
slog.Warn("dropped connection", slog.String("remote", conn.RemoteAddr().String()), slog.String("err", err.Error()))
conn.Close()
}
}
}
const (
htmlHead = `<html><body bgcolor="#000080" text="#00FF00"><center>` +
`<marquee><font face="Comic Sans MS" size="5" color="#FFFF00">` +
`*** WELCOME TO MY HOMEPAGE ***</font></marquee>` +
`<h1><blink>YOU ARE VISITOR #`
htmlTail = `!</blink></h1>` +
`<font color="#FF00FF">Sign my guestbook!</font>` +
`<br><hr>Best viewed in Netscape Navigator</center></body></html>`
// maxPage bounds the rendered page: both halves plus the visitor number.
maxPage = len(htmlHead) + 20 + len(htmlTail)
)
// visits counts served requests. Handlers run on the router's goroutines, so
// every visitor gets their own number.
var visits atomic.Uint64
func homepage(exch *httphi.Exchange) {
var page [maxPage]byte
n := copy(page[:], htmlHead)
n += len(strconv.AppendUint(page[n:n], visits.Add(1), 10))
n += copy(page[n:], htmlTail)
exch.StageHeader("Content-Type", "text/html")
exch.StageHeaderInt("Content-Length", int64(n), 10)
exch.WriteHeader(int(httphi.StatusOK))
exch.WriteBody(page[:n])
}
+2 -2
View File
@@ -27,7 +27,7 @@ func run() error {
// Prepare GET request.
var hdr httpraw.Header
hdr.SetMethod("GET")
hdr.SetRequestURI("/")
hdr.SetRequestTarget("/")
hdr.SetProtocol("HTTP/1.1")
req, err := hdr.AppendRequest(nil)
if err != nil {
@@ -50,7 +50,7 @@ func run() error {
}()
fmt.Println(time.Now().Format("15:04:05.000"), "writing...")
conn.Write(req)
hdr.Reset(nil)
hdr.Reset(nil, 0) // No preallocation. Both key/value and raw buffer will grow and allocate.
var needMore bool = true
for needMore {
_, err = hdr.ReadFromLimited(conn, 1024)
+30 -8
View File
@@ -26,6 +26,7 @@ import (
"github.com/soypat/lneto/internal"
"github.com/soypat/lneto/internal/ltesto"
"github.com/soypat/lneto/internet/pcap"
"github.com/soypat/lneto/ipv4"
"github.com/soypat/lneto/tcp"
"github.com/soypat/lneto/x/xnet"
)
@@ -134,8 +135,9 @@ func run() (err error) {
pfbuf = fmt.Appendf(pfbuf[:0], "%-3s %3d", context, len(pkt))
pfbuf = append(pfbuf, ' ', '[')
pfbuf, err = pf.FormatFrames(pfbuf, frames, pkt)
pfbuf = bytes.ReplaceAll(pfbuf, stack.Addr().AppendTo(nil), []byte("us"))
pfbuf = bytes.ReplaceAll(pfbuf, ethernet.AppendAddr(nil, stack.HardwareAddress()), []byte("us"))
pfbuf = bytes.ReplaceAll(pfbuf, ipv4.AppendFormatAddr(nil, stack.Addr4()), []byte("us"))
pfbuf = bytes.ReplaceAll(pfbuf, ethernet.AppendAddr(nil, stack.HardwareAddr()), []byte("us"))
pfbuf = append(pfbuf, ']', '\n')
if err != nil {
return err
@@ -190,7 +192,7 @@ func run() (err error) {
}
}()
rstack := stack.StackRetrying(5 * time.Millisecond)
rstack := stack.StackRetrying(stackBackoff)
const (
dhcpTimeout = 6 * time.Second
@@ -206,7 +208,7 @@ func run() (err error) {
if err != nil {
return fmt.Errorf("assimilating DHCP results: %w", err)
}
slog.Info("dhcp-complete", slog.String("assignedIP", results.AssignedAddr.String()), slog.String("routerIP", results.Router.String()))
slog.Info("dhcp-complete", slog.String("assignedIP", string(ipv4.AppendFormatAddr(nil, results.AssignedAddr4))), slog.String("routerIP", results.Router.String()))
const (
arpTimeout = 2 * time.Second
@@ -218,10 +220,10 @@ func run() (err error) {
return fmt.Errorf("ARP resolution of router failed: %w", err)
}
timeResolveRouterHW()
stack.SetGateway6(routerHw)
stack.SetGatewayHardwareAddr(routerHw)
svPort := uint16(flagPort)
fmt.Printf("Listening on %s:%d\n", stack.Addr().String(), svPort)
fmt.Printf("Listening on %s:%d\n", ipv4.AppendFormatAddr(nil, stack.Addr4()), svPort)
// Serve connections in a loop.
for {
@@ -230,8 +232,9 @@ func run() (err error) {
RxBuf: make([]byte, mtu),
TxBuf: make([]byte, mtu),
TxPacketQueueSize: 3,
RWBackoff: tcpBackoff,
})
err = stack.ListenTCP(&conn, svPort)
err = stack.ListenTCP4(&conn, svPort)
if err != nil {
return fmt.Errorf("listen TCP: %w", err)
}
@@ -279,7 +282,7 @@ func handleConnection(conn *tcp.Conn) error {
}
method := string(hdr.Method())
uri := string(hdr.RequestURI())
uri := string(hdr.RequestTarget())
fmt.Printf("< %s %s\n", method, uri)
// Build response body.
@@ -354,3 +357,22 @@ func tryPoll(iface ltesto.Interface, poll time.Duration) (dataMayBeReady bool, _
dataMayBeReady = true
return dataMayBeReady, nil
}
func stackBackoff(consecutiveBackoffs uint) time.Duration {
if consecutiveBackoffs < 10 {
return time.Millisecond
}
return 10 * time.Millisecond
}
func tcpBackoff(consecutiveBackoffs uint) time.Duration {
const (
minWait = uint32(time.Microsecond)
maxWait = 5 * uint32(time.Millisecond)
maxShift = 22
_overflowCheck = minWait << maxShift
)
shifted := minWait << min(consecutiveBackoffs, maxShift)
wait := min(shifted, maxWait)
return time.Duration(wait)
}
+12 -2
View File
@@ -5,13 +5,15 @@ package main
import (
"encoding/binary"
"fmt"
"log/slog"
"net/netip"
"sync"
"github.com/soypat/lneto"
"github.com/soypat/lneto/arp"
"github.com/soypat/lneto/dhcpv4"
"github.com/soypat/lneto/dhcp/dhcpv4"
"github.com/soypat/lneto/ethernet"
"github.com/soypat/lneto/internal"
"github.com/soypat/lneto/internal/ltesto"
"github.com/soypat/lneto/ipv4"
"github.com/soypat/lneto/udp"
@@ -53,7 +55,7 @@ type arpEntry struct {
// newDHCPInterceptor creates a dhcpInterceptor that wraps iface and serves
// DHCP from the given server address and subnet.
func newDHCPInterceptor(iface ltesto.Interface, svIP [4]byte, svMAC [6]byte, subnet netip.Prefix) (*dhcpInterceptor, error) {
func newDHCPInterceptor(iface ltesto.Interface, svIP [4]byte, svMAC [6]byte, subnet ipv4.Prefix) (*dhcpInterceptor, error) {
d := &dhcpInterceptor{
inner: iface,
svMAC: svMAC,
@@ -89,6 +91,12 @@ func (d *dhcpInterceptor) Write(b []byte) (int, error) {
return len(b), nil // Consumed by DHCP server, don't forward.
}
d.rewriteEthernetDst(b)
if len(b) >= sizeEthernet+sizeIPv4 &&
*(*[6]byte)(b[0:6]) == d.svMAC &&
binary.BigEndian.Uint16(b[12:14]) == uint16(ethernet.TypeIPv4) {
dstIP := *(*[4]byte)(b[sizeEthernet+16 : sizeEthernet+20])
internal.LogAttrs(slog.Default(), slog.LevelWarn, "ethernet-self-loop", internal.SlogAddr4("dstIP", &dstIP))
}
return d.inner.Write(b)
}
@@ -308,6 +316,8 @@ func (d *dhcpInterceptor) rewriteEthernetDst(b []byte) {
d.mu.Unlock()
if ok {
copy(b[0:6], mac[:])
} else {
internal.LogAttrs(slog.Default(), slog.LevelWarn, "gateway-rewrite-miss", internal.SlogAddr4("dstIP", &dstIP))
}
}
+6 -2
View File
@@ -18,6 +18,7 @@ import (
"github.com/soypat/lneto/internal"
"github.com/soypat/lneto/internal/ltesto"
"github.com/soypat/lneto/internet/pcap"
"github.com/soypat/lneto/ipv4"
)
func main() {
@@ -73,8 +74,10 @@ func run() error {
if err != nil {
return err
}
svIP := ipMask.Addr().As4()
iface, err = newDHCPInterceptor(iface, svIP, hwaddr, ipMask.Masked())
subnet := ipv4.PrefixFromNetip(ipMask)
iface, err = newDHCPInterceptor(iface, svIP, hwaddr, subnet.Masked())
if err != nil {
return fmt.Errorf("DHCP interceptor: %w", err)
}
@@ -85,8 +88,9 @@ func run() error {
}
defer sv.Close()
var cap pcap.PacketBreakdown
cap.SubfieldLimit = 30 // For big DNS or DHCP.
pf := pcap.Formatter{
FilterClasses: []pcap.FieldClass{pcap.FieldClassDst, pcap.FieldClassSrc, pcap.FieldClassSize, pcap.FieldClassFlags},
FilterClasses: []pcap.FieldClass{pcap.FieldClassDst, pcap.FieldClassSrc, pcap.FieldClassSize, pcap.FieldClassFlags, pcap.FieldClassDNSName},
}
var pfbuf []byte
sv.OnTransfer(func(channel int, pkt []byte) {
+26 -6
View File
@@ -79,7 +79,7 @@ func run(ctx context.Context, stack *xnet.StackAsync) error {
// Other option is to instead use async API which leads to more verbose
// and more stateful code.
go stackLoop(ctx, stack)
rstack := stack.StackRetrying(pollTime)
rstack := stack.StackRetrying(stackBackoff)
results, err := rstack.DoDHCPv4([4]byte{}, protoTimeout, protoRetries)
if err != nil {
return fmt.Errorf("doing DHCP: %w", err)
@@ -92,8 +92,8 @@ func run(ctx context.Context, stack *xnet.StackAsync) error {
if err != nil {
return fmt.Errorf("resolving router MAC: %w", err)
}
stack.SetGateway6(gateway)
berkstack := stack.StackBlocking(pollTime).StackGo(xnet.StackGoConfig{
stack.SetGatewayHardwareAddr(gateway)
berkstack := stack.StackBlocking(stackBackoff).StackGo(xnet.StackGoConfig{
ListenerPoolConfig: xnet.TCPPoolConfig{
PoolSize: tcpConnPoolSize,
QueueSize: tcpPacketQueueSize,
@@ -102,10 +102,11 @@ func run(ctx context.Context, stack *xnet.StackAsync) error {
NanoTime: nanotime,
EstablishedTimeout: tcpEstablishedTimeout,
ClosingTimeout: tcpCloseTimeout,
NewBackoff: func() lneto.BackoffStrategy { return tcpBackoff },
},
})
laddr := net.TCPAddrFromAddrPort(netip.AddrPortFrom(results.AssignedAddr, 80))
laddr := net.TCPAddrFromAddrPort(netip.AddrPortFrom(netip.AddrFrom4(results.AssignedAddr4), 80))
// raddr := net.TCPAddr{} // If active (client) connection then set raddr in which case a net.Conn type is returned.
const sockstream = 0x1
c, err := berkstack.Socket(ctx, "tcp", syscall.AF_INET, sockstream, laddr, nil)
@@ -147,7 +148,7 @@ func stackLoop(ctx context.Context, stack *xnet.StackAsync) {
fmt.Println("encaps err:", err)
} else if nwrite > 0 {
network.SendEth(buf[:nwrite])
cap.PrintPacket("OUT", buf[:nwrite])
cap.PrintEthernet("OUT", buf[:nwrite])
}
nread, err := network.RecvEth(buf[:])
if err != nil {
@@ -157,7 +158,7 @@ func stackLoop(ctx context.Context, stack *xnet.StackAsync) {
if err != nil && err != lneto.ErrPacketDrop {
fmt.Println("demux err:", err)
} else {
cap.PrintPacket("IN ", buf[:nread])
cap.PrintEthernet("IN ", buf[:nread])
}
}
if nwrite == 0 && nread == 0 {
@@ -171,3 +172,22 @@ func must(err error) {
panic(err)
}
}
func stackBackoff(consecutiveBackoffs uint) time.Duration {
if consecutiveBackoffs < 10 {
return time.Millisecond
}
return 10 * time.Millisecond
}
func tcpBackoff(consecutiveBackoffs uint) time.Duration {
const (
minWait = uint32(time.Microsecond)
maxWait = 5 * uint32(time.Millisecond)
maxShift = 22
_overflowCheck = minWait << maxShift
)
shifted := minWait << min(consecutiveBackoffs, maxShift)
wait := min(shifted, maxWait)
return time.Duration(wait)
}
+90
View File
@@ -0,0 +1,90 @@
// Command ntp-client performs a two-exchange NTP clock synchronization against
// a remote server and prints the corrected time, clock offset, and round-trip
// delay.
//
// Usage:
//
// go run ./examples/ntp-client/ -server pool.ntp.org:123
// go run ./examples/ntp-client/ -server 127.0.0.1:10123 -debug
//
// This tool uses the standard library net package for UDP transport instead of
// lneto's own networking stack. These examples exercise one protocol layer at a
// time in isolation, keeping the transport concern separate so failures are
// clearly attributable to the NTP codec and state machine rather than the
// full-stack IP/UDP path.
package main
import (
"flag"
"fmt"
"log/slog"
"net"
"os"
"time"
"github.com/soypat/lneto/ntp"
)
func main() {
if err := run(); err != nil {
fmt.Println(err)
os.Exit(1)
}
}
func run() error {
addr := flag.String("server", "pool.ntp.org:123", "NTP server address (host:port)")
debug := flag.Bool("debug", false, "enable debug logging")
flag.Parse()
conn, err := net.DialTimeout("udp", *addr, 5*time.Second)
if err != nil {
return fmt.Errorf("dial: %w", err)
}
defer conn.Close()
var precBuf [64]int64
sysprec := ntp.CalculateSystemPrecision(nil, precBuf[:])
var client ntp.Client
client.Reset(sysprec, time.Now)
if *debug {
client.SetLogger(slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{Level: slog.LevelDebug})))
}
const maxRetries = 10
var buf [1500]byte
for attempt := 0; !client.IsDone() && attempt < maxRetries; attempt++ {
n, err := client.Encapsulate(buf[:ntp.SizeHeader], 0, 0)
if err != nil {
return fmt.Errorf("encapsulate: %w", err)
}
if n == 0 {
return fmt.Errorf("encapsulate returned 0 bytes unexpectedly")
}
conn.SetDeadline(time.Now().Add(5 * time.Second))
if _, err = conn.Write(buf[:n]); err != nil {
return fmt.Errorf("write: %w", err)
}
rn, err := conn.Read(buf[:])
if err != nil {
return fmt.Errorf("read: %w", err)
}
if err = client.Demux(buf[:rn], 0); err != nil {
return fmt.Errorf("demux: %w", err)
}
}
if !client.IsDone() {
return fmt.Errorf("NTP exchange did not complete within %d attempts", maxRetries)
}
fmt.Printf("NTP time: %s\n", client.Now().Format(time.RFC3339Nano))
fmt.Printf("Offset: %s\n", client.Offset())
fmt.Printf("RTD: %s\n", client.RoundTripDelay())
fmt.Printf("Stratum: %s\n", client.ServerStratum())
return nil
}
+96
View File
@@ -0,0 +1,96 @@
// Command ntp-server is a minimal NTP server that listens for client requests
// on a UDP socket and responds with the current system time. It serves as an
// integration test target for the ntp-client example.
//
// Usage:
//
// go run ./examples/ntp-server/ -addr :10123
//
// The listen address defaults to :123 (requires root).
//
// This tool uses the standard library net package for UDP transport instead of
// lneto's own networking stack. These examples exercise one protocol layer at a
// time in isolation, keeping the transport concern separate so failures are
// clearly attributable to the NTP codec and state machine rather than the
// full-stack IP/UDP path.
package main
import (
"flag"
"fmt"
"net"
"os"
"time"
"github.com/soypat/lneto/ntp"
)
func main() {
if err := run(); err != nil {
fmt.Println(err)
os.Exit(1)
}
}
func run() error {
listenAddr := flag.String("addr", ":123", "UDP listen address (host:port)")
flag.Parse()
pc, err := net.ListenPacket("udp", *listenAddr)
if err != nil {
return fmt.Errorf("listen: %w", err)
}
defer pc.Close()
fmt.Printf("NTP server listening on %s\n", pc.LocalAddr())
var precBuf [64]int64
sysprec := ntp.CalculateSystemPrecision(nil, precBuf[:])
var handler ntp.Server
err = handler.Reset(ntp.ServerConfig{
Now: time.Now,
Stratum: ntp.StratumPrimary,
Precision: sysprec,
RefID: [4]byte{'G', 'O', 'L', 'N'},
MaxPending: 16,
})
if err != nil {
return fmt.Errorf("server reset: %w", err)
}
var buf [1500]byte
var backoff uint
for {
pc.SetReadDeadline(time.Now().Add(100 * time.Millisecond))
n, raddr, err := pc.ReadFrom(buf[:])
if err != nil {
if nerr, ok := err.(net.Error); ok && nerr.Timeout() {
backoff++
if backoff > 10 {
time.Sleep(time.Duration(backoff) * time.Millisecond)
}
continue
}
fmt.Printf("read error: %v\n", err)
continue
}
backoff = 0
if err = handler.Demux(buf[:n], 0); err != nil {
fmt.Printf("demux error from %s: %v\n", raddr, err)
continue
}
var resp [ntp.SizeHeader]byte
rn, err := handler.Encapsulate(resp[:], 0, 0)
if err != nil {
fmt.Printf("encapsulate error: %v\n", err)
continue
}
if rn > 0 {
if _, err = pc.WriteTo(resp[:rn], raddr); err != nil {
fmt.Printf("write error: %v\n", err)
}
}
}
}
+57 -9
View File
@@ -26,6 +26,7 @@ import (
"github.com/soypat/lneto/internal"
"github.com/soypat/lneto/internal/ltesto"
"github.com/soypat/lneto/internet/pcap"
"github.com/soypat/lneto/ipv4"
"github.com/soypat/lneto/tcp"
"github.com/soypat/lneto/x/xnet"
)
@@ -49,6 +50,7 @@ func run() (err error) {
flagUseHTTP = false
flagHostToResolve = ""
flagRequestedIP = ""
flagLocalTCPPort = 0
flagDoNTP = false
flagNoPcap = false
flagPprof = false
@@ -57,6 +59,7 @@ func run() (err error) {
flag.BoolVar(&flagUseHTTP, "ihttp", flagUseHTTP, "Use HTTP tap interface.")
flag.StringVar(&flagHostToResolve, "host", flagHostToResolve, "Hostname to resolve via DNS.")
flag.StringVar(&flagRequestedIP, "addr", flagRequestedIP, "IP address to request via DHCP.")
flag.IntVar(&flagLocalTCPPort, "port", flagLocalTCPPort, "Local TCP source port to use. Zero chooses a pseudo-random port.")
flag.BoolVar(&flagDoNTP, "ntp", flagDoNTP, "Do NTP round and print result time")
flag.BoolVar(&flagNoPcap, "nopcap", flagNoPcap, "Disable pcap logging.")
flag.BoolVar(&flagPprof, "pprof", flagPprof, "Enable CPU profiling.")
@@ -78,6 +81,21 @@ func run() (err error) {
flag.Usage()
return err
}
reqAddr := [4]byte{192, 168, 1, 96}
requestedAddrSet := false
if flagRequestedIP != "" {
addr, err := netip.ParseAddr(flagRequestedIP)
if err != nil || !addr.Is4() {
flag.Usage()
return fmt.Errorf("invalid requested IPv4 address %q", flagRequestedIP)
}
reqAddr = addr.As4()
requestedAddrSet = true
}
if flagLocalTCPPort < 0 || flagLocalTCPPort > math.MaxUint16 {
flag.Usage()
return fmt.Errorf("invalid local TCP port %d", flagLocalTCPPort)
}
fmt.Println("softrand", softRand)
var iface ltesto.Interface
if flagUseHTTP {
@@ -136,9 +154,11 @@ func run() (err error) {
lastAction := time.Now()
buf := make([]byte, math.MaxUint16) // Generic-receive Offload (GRO) can aggregate packets.
var cap pcap.PacketBreakdown
cap.SubfieldLimit = 30 // For chunky DNS.
var frames []pcap.Frame
pf := pcap.Formatter{
FilterClasses: []pcap.FieldClass{pcap.FieldClassFlags, pcap.FieldClassOperation, pcap.FieldClassDst, pcap.FieldClassSrc, pcap.FieldClassAddress, pcap.FieldClassTimestamp},
FilterClasses: []pcap.FieldClass{pcap.FieldClassFlags, pcap.FieldClassOperation, pcap.FieldClassDst, pcap.FieldClassSrc, pcap.FieldClassAddress, pcap.FieldClassTimestamp, pcap.FieldClassDNSName},
SubfieldLimit: 30,
}
var pfbuf []byte
logFrames := func(context string, pkt []byte) error {
@@ -154,8 +174,9 @@ func run() (err error) {
pfbuf = fmt.Appendf(pfbuf[:0], "%-3s %3d", context, len(pkt))
pfbuf = append(pfbuf, ' ', '[')
pfbuf, err = pf.FormatFrames(pfbuf, frames, pkt)
pfbuf = bytes.ReplaceAll(pfbuf, stack.Addr().AppendTo(nil), []byte("us"))
pfbuf = bytes.ReplaceAll(pfbuf, ethernet.AppendAddr(nil, stack.HardwareAddress()), []byte("us"))
addr := stack.Addr4()
pfbuf = bytes.ReplaceAll(pfbuf, addr[:], []byte("us"))
pfbuf = bytes.ReplaceAll(pfbuf, ethernet.AppendAddr(nil, stack.HardwareAddr()), []byte("us"))
pfbuf = append(pfbuf, ']', '\n')
if err != nil {
return err
@@ -212,24 +233,27 @@ func run() (err error) {
}
}()
rstack := stack.StackRetrying(5 * time.Millisecond)
rstack := stack.StackRetrying(stackBackoff)
const (
dhcpTimeout = 6 * time.Second
dhcpRetries = 2
)
timeDHCP := timer("DHCP request completed")
results, err := rstack.DoDHCPv4([4]byte{192, 168, 1, 96}, dhcpTimeout, dhcpRetries)
results, err := rstack.DoDHCPv4(reqAddr, dhcpTimeout, dhcpRetries)
if err != nil {
return fmt.Errorf("DHCP failed: %w", err)
}
timeDHCP()
if requestedAddrSet && results.AssignedAddr4 != reqAddr {
return fmt.Errorf("DHCP assigned %s, not requested %s", netip.AddrFrom4(results.AssignedAddr4), netip.AddrFrom4(reqAddr))
}
err = stack.AssimilateDHCPResults(results)
if err != nil {
return fmt.Errorf("assimilating DHCP results: %w", err)
}
slog.Info("dhcp-complete", slog.String("assignedIP", results.AssignedAddr.String()), slog.String("routerIP", results.Router.String()), slog.Any("DNS", results.DNSServers), slog.Any("subnet", results.Subnet.String()))
slog.Info("dhcp-complete", slog.String("assignedIP", string(ipv4.AppendFormatAddr(nil, results.AssignedAddr4))), slog.String("routerIP", results.Router.String()), slog.Any("DNS", results.DNSServers), slog.Any("subnet", results.Subnet.String()))
const (
arpTimeout = 2 * time.Second
arpRetries = 2
@@ -244,7 +268,7 @@ func run() (err error) {
return fmt.Errorf("ARP resolution of router failed: %w", err)
}
timeResolveRouterHW()
stack.SetGateway6(routerHw)
stack.SetGatewayHardwareAddr(routerHw)
if flagDoNTP {
timeLookupNTP := timer("NTP IP lookup")
const ntpHost = "pool.ntp.org"
@@ -277,12 +301,13 @@ func run() (err error) {
RxBuf: make([]byte, mtu),
TxBuf: make([]byte, mtu),
TxPacketQueueSize: 3,
RWBackoff: tcpBackoff,
})
timeHTTPCreate := timer("create HTTP GET request")
var hdr httpraw.Header
hdr.SetMethod("GET")
hdr.SetRequestURI("/")
hdr.SetRequestTarget("/")
hdr.SetProtocol("HTTP/1.1")
hdr.Set("Host", flagHostToResolve)
hdr.Set("User-Agent", "lneto")
@@ -297,7 +322,12 @@ func run() (err error) {
timeTCPDial := timer("TCP dial (handshake)")
const tcpDialTimeout = 8 * time.Second // Was 60 * time.Minute causing 3.6s sleep per iteration!
target := netip.AddrPortFrom(addrs[0], 80)
err = rstack.DoDialTCP(&conn, uint16(softRand&0xefff)+1024, target, tcpDialTimeout, internetRetries)
localPort := uint16(flagLocalTCPPort)
if localPort == 0 {
localPort = uint16(softRand&0xefff) + 1024
}
fmt.Printf("TCP target %s local-port=%d\n", target, localPort)
err = rstack.DoDialTCP(&conn, localPort, target, tcpDialTimeout, internetRetries)
if err != nil {
return fmt.Errorf("TCP failed: %w", err)
}
@@ -382,3 +412,21 @@ func tryPoll(iface ltesto.Interface, poll time.Duration) (dataMayBeReady bool, _
dataMayBeReady = true
return dataMayBeReady, nil
}
func stackBackoff(consecutiveBackoffs uint) time.Duration {
if consecutiveBackoffs < 10 {
return time.Millisecond
}
return 10 * time.Millisecond
}
func tcpBackoff(consecutiveBackoffs uint) time.Duration {
const (
minWait = uint32(time.Microsecond)
maxWait = 5 * uint32(time.Millisecond)
maxShift = 22
_overflowCheck = minWait << maxShift
)
shifted := minWait << min(consecutiveBackoffs, maxShift)
wait := min(shifted, maxWait)
return time.Duration(wait)
}
+1 -1
View File
@@ -1,3 +1,3 @@
module github.com/soypat/lneto
go 1.23.8
go 1.24
+134
View File
@@ -0,0 +1,134 @@
package httphi
import (
"io"
"testing"
"github.com/soypat/lneto/http/httpraw"
"github.com/soypat/lneto/internal"
)
// benchConn replays a fixed request and discards the response. It allocates
// nothing itself so benchmark alloc counts belong to the package under test.
type benchConn struct {
request string
read int
written int
}
func (c *benchConn) rewind() { c.read, c.written = 0, 0 }
func (c *benchConn) Read(b []byte) (int, error) {
if c.read >= len(c.request) {
return 0, io.EOF
}
n := copy(b, c.request[c.read:])
c.read += n
return n, nil
}
func (c *benchConn) Write(b []byte) (int, error) {
c.written += len(b)
return len(b), nil
}
func (c *benchConn) Close() error { return nil }
// benchBody is package level: converting a string literal to []byte inside the
// handler would allocate on every request and hide the router's own cost.
var benchBody = []byte("hello world")
func benchExchange(b *testing.B, conn conn) *Exchange {
b.Helper()
const bufferSize = 1024
const numHeaderCap = 2
exch := new(Exchange)
exch.Configure(ExchangeConfig{
RawBuf: make([]byte, 2*bufferSize),
RequestBufferLim: bufferSize,
NumHeaderKVCap: numHeaderCap,
})
if !exch.Acquire(conn) {
b.Fatal("fresh exchange failed to acquire connection")
}
return exch
}
// BenchmarkHandle measures a whole exchange: read, parse, mux and respond.
func BenchmarkHandle(b *testing.B) {
expect := []byte("123")
buf := make([]byte, 64)
for _, bb := range []struct {
name string
request string
handler HandlerFunc
}{
{
name: "GETWithHeadersAndQuery",
request: "GET /?abc=123 HTTP/1.1\r\nHost: tinygo.org\r\nUser-Agent: bench\r\nAccept: */*\r\nConnection: close\r\n\r\n",
handler: func(ex *Exchange) {
ex.StageHeader("Content-Type", "text/plain")
ex.StageHeaderInt("Content-Length", int64(len(benchBody)), 10)
data, present := ex.AppendQuery(buf[:0], "abc", true)
if !present || !internal.BytesEqual(data, expect) {
panic("invalid result")
}
ex.WriteBody(benchBody)
},
},
{
name: "NotFound",
request: "GET /nowhere HTTP/1.1\r\nHost: tinygo.org\r\n\r\n",
handler: nil, // Unregistered: exercises the 404 path.
},
} {
b.Run(bb.name, func(b *testing.B) {
var mux MuxSlice
if bb.handler != nil {
mux.Handle("GET /", bb.handler)
}
conn := &benchConn{request: bb.request}
exch := benchExchange(b, conn)
b.ReportAllocs()
b.SetBytes(int64(len(bb.request)))
b.ResetTimer()
for i := 0; i < b.N; i++ {
conn.rewind()
exch.Release()
exch.Acquire(conn)
Handle(exch, &mux, nopBackoff)
}
})
}
}
// benchForm is package level so the Form's pair slice is reused across requests,
// as a real handler holding one per goroutine would.
var benchForm httpraw.Form
// BenchmarkRequestParseForm measures reading and parsing a urlencoded body into
// a buffer the caller owns. Nothing on the path may allocate.
func BenchmarkRequestParseForm(b *testing.B) {
const request = "POST /f HTTP/1.1\r\nHost: tinygo.org\r\n" +
"Content-Type: application/x-www-form-urlencoded\r\nContent-Length: 27\r\n\r\n" +
"user=gopher&msg=hello+world"
buf := make([]byte, 64)
var mux MuxSlice
mux.Handle("POST /f", func(ex *Exchange) {
err := ex.RequestParseForm(&benchForm, buf)
if err != nil || benchForm.Len() != 2 {
panic("invalid result")
}
})
conn := &benchConn{request: request}
exch := benchExchange(b, conn)
b.ReportAllocs()
b.SetBytes(int64(len(request)))
b.ResetTimer()
for i := 0; i < b.N; i++ {
conn.rewind()
exch.Release()
exch.Acquire(conn)
Handle(exch, &mux, nopBackoff)
}
}
+638
View File
@@ -0,0 +1,638 @@
package httphi
import (
"io"
"net"
"slices"
"strconv"
"sync/atomic"
"github.com/soypat/lneto"
"github.com/soypat/lneto/http/httpraw"
"github.com/soypat/lneto/internal"
)
// maxStatusLine bounds the response status line: "HTTP/1.1 " + 3 digit code +
// " " + longest [StatusText] + CRLF.
const maxStatusLine = len("HTTP/1.1 ") + 3 + 1 + len("Network Authentication Required") + 2
// Exchange is a single request-response cycle over a connection, playing the
// part of both http.Request and http.ResponseWriter: Request* methods read the
// request, [Exchange.StageHeader] and [Exchange.WriteBody] produce the response.
// A [Router] owns a fixed pool of them, which is what bounds its memory.
//
// Request and response share one buffer, the response header being written over
// the bytes that follow the parsed request header. Read the request body with
// [Exchange.ReadBody] before setting response headers.
type Exchange struct {
used atomic.Bool
gen atomic.Uint32
respTopBuf [maxStatusLine]byte
respTopWritten uint8
rawbuf []byte
respHeaderOff uint16
respHeaderLen uint16
reqHdr httpraw.Header
hijacked bool
rw conn
matchedPattern string
respRemains int
respErr error // Sticky: response is unrecoverable once a write fails.
headerWritten bool
normalizeKeys bool
nextFree *Exchange
readErr error
}
type ExchangeConfig struct {
RawBuf []byte
RequestBufferLim int
NumHeaderKVCap int
NormalizeOutgoingKeys bool
NoRequestBufferGrowth bool
}
// HijackRaw is a low-level implementation of http.Hijacker interface.
// A Hijack method is not exposed due to heap allocation implications and correctness concerns.
// Below is what an actual implementation may look like:
//
// func (exch *Exchange) Hijack() (net.Conn, *bufio.ReadWriter, error) {
// conn, ok := exch.rw.(net.Conn)
// if !ok {
// return nil, nil, errors.New("net.Conn not implemented")
// }
// _, data, err := exch.HijackRaw(nil)
// if err != nil {
// return nil, nil, err
// }
// var rd *bufio.ReadWriter
// if len(data) > 0 {
// rd = &bufio.ReadWriter{Reader: bufio.NewReader(bytes.NewReader(data))}
// }
// return conn, rd, nil
// }
func (exch *Exchange) HijackRaw(dstBody []byte) (conn, []byte, error) {
data, err := exch.remainingSurplusBody()
if err != nil {
return nil, nil, err
}
exch.hijacked = true
dstBody = append(dstBody, data...)
return exch.rw, dstBody, nil
}
// Configure sets the memory the exchange works with for the rest of its life:
// rawbuf holds the request header, the response header and any surplus body,
// of which the first requestLim bytes are reserved for the request header.
// Panics if requestLim exceeds the buffer. Set normalizeKeys to normalize
// outgoing header keys, i.e: "content-type" to "Content-Type".
func (exch *Exchange) Configure(cfg ExchangeConfig) {
respSize := len(cfg.RawBuf) - cfg.RequestBufferLim
if respSize < 0 {
panic("request lim larger than buffer")
}
exch.rawbuf = cfg.RawBuf
exch.reqHdr.Reset(cfg.RawBuf[:0:cfg.RequestBufferLim], cfg.NumHeaderKVCap)
exch.reqHdr.ConfigBufferGrowth(!cfg.NoRequestBufferGrowth)
exch.normalizeKeys = cfg.NormalizeOutgoingKeys
}
// Acquire claims the exchange for conn and resets it to serve a new request,
// reusing the buffer set by [Exchange.Configure]. Returns false if the exchange
// is already serving, in which case conn is untouched.
func (exch *Exchange) Acquire(conn conn) bool {
if !exch.used.CompareAndSwap(false, true) {
return false
}
exch.matchedPattern = ""
exch.gen.Add(1)
exch.readErr = nil
exch.respErr = nil
exch.hijacked = false
exch.respTopWritten = 0
exch.respHeaderOff = 0
exch.respHeaderLen = 0
exch.respRemains = 0
exch.rw = conn
exch.headerWritten = false
exch.nextFree = nil
exch.reqHdr.Reset(nil, 0)
return true
}
// Release closes the exchange's connection and frees the exchange for a future
// [Exchange.Acquire]. The connection is left open if the handler took ownership
// of it with [Exchange.HijackRaw].
func (exch *Exchange) Release() {
if !exch.hijacked {
exch.rw.Close()
}
exch.rw = nil
exch.gen.Add(1)
exch.used.Store(false)
}
// UnsafeRawBuffer returns the contiguous buffer owned by [Exchange] being used for the request and response.
//
// Writing to it will mangle the entire request header+body and/or any staged response headers.
// Does not return the buffer used for the response first line so can be safely
// written to and used without modifying the staged response first line.
//
// Staging headers will write to this buffer so use mindfully.
// To access only the request header buffer portion use [httpraw.Header.BufferRaw] limited
// to [httpraw.Header.BufferParsed] as returned by [Exchange.RequestHeaderRaw].
// Writing to this section will not change the contents read by [Exchange.ReadBody].
//
// In [Router] context, the size of this buffer is influenced directly by [RouterConfig] HeaderBufferSize fields.
func (exch *Exchange) UnsafeRawBuffer() []byte { return exch.rawbuf }
// StageHeader stages a response header field, written on the first
// [Exchange.FlushHeader], [Exchange.WriteHeader] or [Exchange.WriteBody].
// Returns false and drops the field if the response buffer cannot fit it.
// Has no effect once the header has been written.
func (exch *Exchange) StageHeader(key, value string) (enoughMemory bool) {
if exch.headerWritten {
return false
}
off := int(exch.respHeaderOff) + int(exch.respHeaderLen)
free := len(exch.rawbuf) - off
// Field costs key+':'+value+CRLF, plus the CRLF [Exchange.FlushHeader]
// appends past the last field to close the header block.
if len(key)+len(value)+len(":\r\n")+len("\r\n") > free {
exch.respErr = lneto.ErrBufferFull // Omit writing header back to prevent incomplete response.
return false
}
n := copy(exch.rawbuf[off:], key)
if exch.normalizeKeys {
httpraw.NormalizeHeaderKey(exch.rawbuf[off : off+n])
}
exch.rawbuf[off+n] = ':'
n++
n += copy(exch.rawbuf[off+n:], value)
exch.rawbuf[off+n] = '\r'
exch.rawbuf[off+n+1] = '\n'
n += 2
exch.respHeaderLen += uint16(n)
return true
}
// StageHeaderInt is [Exchange.StageHeader] with an integer value, i.e: Content-Length.
// It formats the value directly into the response buffer without allocating.
// base must be in the range 10..36; lower bases are dropped, no HTTP header
// field value is written below base 10.
func (exch *Exchange) StageHeaderInt(key string, value int64, base int) (enoughMemory bool) {
if exch.headerWritten || base < 10 || base > 36 {
return false
}
off := int(exch.respHeaderOff) + int(exch.respHeaderLen)
free := len(exch.rawbuf) - off
if len(key)+internal.IntLen(value, base)+len(":\r\n")+len("\r\n") > free {
exch.respErr = lneto.ErrBufferFull // Omit writing header back to prevent incomplete response.
return false
}
n := copy(exch.rawbuf[off:], key)
if exch.normalizeKeys {
httpraw.NormalizeHeaderKey(exch.rawbuf[off : off+n])
}
exch.rawbuf[off+n] = ':'
n++
n += len(strconv.AppendInt(exch.rawbuf[off+n:off+n], value, base))
exch.rawbuf[off+n] = '\r'
exch.rawbuf[off+n+1] = '\n'
n += 2
exch.respHeaderLen += uint16(n)
return true
}
// StageStatus prepares the status line for the given code without writing
// it, i.e: "HTTP/1.1 404 Not Found". Codes with no [StatusText] get an empty
// reason phrase. Has no effect once the header has been written.
func (exch *Exchange) StageStatus(code int) {
if code >= 1000 || exch.headerWritten {
return
} else if code == 200 {
// Common case.
exch.respTopWritten = uint8(copy(exch.respTopBuf[:], "HTTP/1.1 200 OK\r\n"))
return
}
n := copy(exch.respTopBuf[:], "HTTP/1.1 ")
n += len(strconv.AppendInt(exch.respTopBuf[n:n], int64(code), 10))
text := StatusText(code)
exch.respTopBuf[n] = ' '
n++
n += copy(exch.respTopBuf[n:], text)
exch.respTopBuf[n] = '\r'
exch.respTopBuf[n+1] = '\n'
exch.respTopWritten = uint8(n + 2)
}
// WriteHeader sends the status line for code along with the staged header
// fields. Only the first call reaches the wire, as in http.ResponseWriter.
func (exch *Exchange) WriteHeader(code int) {
if !exch.headerWritten {
exch.StageStatus(code)
exch.FlushHeader()
}
}
// FlushHeader writes the status line and staged header fields to the connection
// and returns the bytes written, defaulting to a 200 status if none was staged.
// Does nothing if the header was already written.
func (exch *Exchange) FlushHeader() (int, error) {
if exch.respErr != nil {
return 0, exch.respErr
} else if exch.headerWritten {
return 0, nil
}
if exch.respTopWritten == 0 {
exch.StageStatus(200)
}
exch.headerWritten = true
ng, err := exch.rw.Write(exch.respTopBuf[:exch.respTopWritten])
if err != nil {
exch.respErr = err
return ng, err
}
off := int(exch.respHeaderOff)
headers := exch.rawbuf[off : off+int(exch.respHeaderLen)+2]
headers[len(headers)-1] = '\n'
headers[len(headers)-2] = '\r'
ng2, err := exch.rw.Write(headers)
exch.respErr = err
return ng + ng2, err
}
// ExchangeRW is an [io.ReadWriteCloser] view of an [Exchange] wrapping
// [Exchange.ReadBody] and [Exchange.WriteBody] methods.
//
// Exchanges are pooled and reused, so a handle records the exchange generation
// it was taken at and refuses to touch the connection once that exchange moves
// on to another request. Obtain one with [Exchange.ReadWriter].
type ExchangeRW struct {
gen uint32
exch *Exchange
}
// IsValid returns true while the handle still refers to the request it was
// taken from, i.e: false once the exchange was released or hijacked away.
func (rw *ExchangeRW) IsValid() bool {
return rw.gen == rw.exch.gen.Load() && rw.exch.used.Load()
}
func (rw *ExchangeRW) validate() error {
if !rw.IsValid() {
return net.ErrClosed
}
return nil
}
// Write writes response body bytes. See [Exchange.WriteBody].
// Fails with [net.ErrClosed] once the handle is no longer valid.
func (rw *ExchangeRW) Write(buf []byte) (int, error) {
if err := rw.validate(); err != nil {
return 0, err
}
return rw.exch.WriteBody(buf)
}
// Read reads request body bytes. See [Exchange.ReadBody].
// Fails with [net.ErrClosed] once the handle is no longer valid.
func (rw *ExchangeRW) Read(buf []byte) (int, error) {
if err := rw.validate(); err != nil {
return 0, err
}
return rw.exch.ReadBody(buf)
}
// Close invalidates this handle so later reads and writes fail. It does not
// close the connection nor end the exchange, both of which the [Router] owns.
func (rw *ExchangeRW) Close() error {
if err := rw.validate(); err != nil {
return err
}
rw.gen--
return nil
}
// ReadWriter fills dst with a stream view of the exchange, valid until the
// exchange is released. The caller owns dst, so a handler may keep one and
// refill it every request without allocating.
func (exch *Exchange) ReadWriter(dst *ExchangeRW) {
dst.gen = exch.gen.Load()
dst.exch = exch
}
// Write writes response body bytes, flushing the header first if the handler
// has not written it yet. Once a write to the connection fails the response is
// unrecoverable and every later write returns that same error, so a body never
// reaches the wire without its header.
func (exch *Exchange) WriteBody(buf []byte) (int, error) {
if exch.respErr != nil {
return 0, exch.respErr
} else if !exch.headerWritten {
_, err := exch.FlushHeader()
if err != nil {
return 0, err // Body must not reach the wire without its header.
}
}
if len(buf) == 0 {
return 0, nil
}
n, err := exch.rw.Write(buf)
exch.respErr = err
return n, err
}
// ReadBody reads the request body into dst, starting with the bytes that
// arrived in the same read as the header and continuing from the connection.
// The exchange does not know the body's length: use Content-Length or the
// transfer encoding to know when to stop reading.
func (exch *Exchange) ReadBody(dst []byte) (n int, _ error) {
if exch.respRemains > 0 {
toRead, err := exch.remainingSurplusBody()
if err != nil {
return 0, err
}
n = copy(dst, toRead)
exch.respRemains -= n
// hand over what already arrived since conn might have
// exhausted data and could block indefinetely.
return n, nil
}
return exch.rw.Read(dst)
}
func (exch *Exchange) remainingSurplusBody() ([]byte, error) {
_, err := exch.reqHdr.Body()
if err != nil {
return nil, err // Returns mangled buffer error if request header has been misused.
}
surplus := exch.rawbuf[exch.reqHdr.BufferParsed():exch.reqHdr.BufferReceived()]
toRead := surplus[len(surplus)-exch.respRemains:]
return toRead, nil
}
// MuxPattern returns the pattern [Mux] matched to the request.
func (exch *Exchange) MuxPattern() string {
return exch.matchedPattern
}
// RequestHeaderRaw returns the parsed request header for access beyond the
// Request* methods, such as [httpraw.Header.ForEach]. Valid until the exchange
// is released, and writing to it corrupts the response.
func (exch *Exchange) RequestHeaderRaw() *httpraw.Header {
return &exch.reqHdr
}
// RequestParseCookie parses the request's key header field into dst, i.e:
// "Cookie". The caller owns dst and its buffer, so it may be reused between
// requests.
func (exch *Exchange) RequestParseCookie(dst *httpraw.Cookie, key string) error {
value := exch.RequestHeader(key)
return dst.ParseBytes(value)
}
// RequestContentType returns the request's Content-Type field value as it
// appears on the wire, parameters included, nil if absent. Test it with
// [httpraw.MediaTypeIs] and pick parameters out with [httpraw.ContentParam].
func (exch *Exchange) RequestContentType() []byte {
return exch.RequestHeader("Content-Type")
}
// RequestContentLength returns the body length declared by the request's
// Content-Length field. An absent field is not a client error: such a request
// has no body at all, RFC 9112 6.3. Check for the error to answer 411 instead.
// See [httpraw.Header.ContentLength].
func (exch *Exchange) RequestContentLength() (int64, error) {
return exch.RequestHeaderRaw().ContentLength()
}
// RequestParseForm reads the request body into buf and parses it as
// "application/x-www-form-urlencoded" into dst. buf is the only storage used and
// the only limit: a body longer than buf is refused with [lneto.ErrBufferFull]
// before a single byte is read, leaving the caller free to answer 413. Pairs are
// left as they arrived, call [httpraw.Form.Decode] to decode them in place.
//
// Unlike http.Request.ParseForm the query string is not folded in, reach it with
// [Exchange.RequestQuery] or [Exchange.AppendQuery]. The body is consumed, so
// call this before [Exchange.ReadBody].
//
// A request with no Content-Length has no body, RFC 9112 6.3, and yields an
// empty form. Use [Exchange.RequestContentLength] to tell that apart from a body
// that arrived empty.
func (exch *Exchange) RequestParseForm(dst *httpraw.Form, buf []byte) error {
if !httpraw.MediaTypeIs(exch.RequestContentType(), "application/x-www-form-urlencoded") {
return errNotFormEncoded
} else if exch.RequestHeader("Transfer-Encoding") != nil {
// Chunked bodies are framed, so reading Content-Length bytes off the
// wire would parse chunk sizes as form data. httpraw does not decode them.
return errUnsupportedTransferCoding
}
length, err := exch.RequestContentLength()
if err != nil {
dst.Reset(buf[:0])
return dst.Parse() // No length is no body, RFC 9112 6.3.
} else if length > int64(len(buf)) {
return lneto.ErrBufferFull // Refuse before reading, caller may answer 413.
}
buf = buf[:length]
for read := 0; read < len(buf); {
n, err := exch.ReadBody(buf[read:])
read += n
if n == 0 && err != nil {
return err
}
}
dst.Reset(buf)
return dst.Parse()
}
// RequestMultipart returns a parser prepared from the boundary parameter of the
// request's Content-Type field. It reads no body: multipart parts declare no
// length, so the caller drives the loop with a buffer it owns and decides per
// part what to keep and when a part has grown too large. See
// [Exchange.ReadMultiparts] for that loop already written.
func (exch *Exchange) RequestMultipart() (mp httpraw.Multipart, err error) {
contentType := exch.RequestContentType()
if !httpraw.MediaTypeIs(contentType, "multipart/form-data") {
return mp, errNotMultipart
}
return mp, mp.SetContentType(contentType)
}
// MultipartSink is a part of a multipart body together with the writer its
// content was streamed to, as appended by [Exchange.ReadMultiparts].
type MultipartSink struct {
// Header identifies the part. Name and Filename are copies, so they
// outlive the read buffer; PartView does not, see [httpraw.MultipartHeader].
Header httpraw.MultipartHeader
// Sink received the part's content and was closed when the part ended,
// nil for a part newSink chose to discard.
Sink io.WriteCloser
}
// ReadMultiparts streams the request's "multipart/form-data" body, writing each
// part to a sink newSink returns for it and appending the pair to dst. buf is the
// only storage used and content is never held whole, so a part of any length
// streams through a buffer the caller sized. dst is appended to and returned, so
// a handler may hand back the slice of a previous request to reuse its parts.
//
// newSink is called once per part, before any of its content is read, and picks
// what to do with it from hdr.Name and hdr.Filename: return a writer to keep the
// part, or nil to discard its content and keep only the header. Each sink is
// closed as soon as its part ends, so Close reports the part arrived whole; on
// error the sink of the part being read is left open for the caller to deal with.
//
// A part header that does not fit buf is refused with [lneto.ErrShortBuffer],
// since reading more can never complete it, leaving the caller free to answer
// 413. The body is consumed, so call this before [Exchange.ReadBody].
func (exch *Exchange) ReadMultiparts(dst []MultipartSink, buf []byte, newSink func(hdr *httpraw.MultipartHeader) io.WriteCloser) (_ []MultipartSink, _ error) {
mp, err := exch.RequestMultipart()
if err != nil {
return dst, err
} else if newSink == nil || len(buf) <= len("\r\n--")+len(mp.Boundary) {
// A buffer that cannot outgrow a delimiter never makes progress.
return dst, lneto.ErrInvalidConfig
}
buflen := 0
for {
// Slot for the next part, given back when the body turns out to be
// over, so its Name and Filename buffers stay available for reuse.
part := internal.SliceReclaim(&dst)
var parsed int
for {
parsed, err = mp.NextHeader(&part.Header, buf[:buflen])
if err != nil {
dst = dst[:len(dst)-1]
if err == io.EOF {
err = nil // Closing delimiter, body done.
}
return dst, err
} else if parsed > 0 {
break // Delimiter and header block complete.
} else if buflen == len(buf) {
dst = dst[:len(dst)-1]
return dst, lneto.ErrShortBuffer // Header longer than buf.
}
// A read that both delivers and fails, as the last of the body
// followed by a hangup does, may still hold what the parser is
// waiting for: take the data and let the error surface on the
// next read.
n, readErr := exch.ReadBody(buf[buflen:])
buflen += n
if n == 0 && readErr != nil {
dst = dst[:len(dst)-1]
return dst, readErr
}
}
part.Sink = newSink(&part.Header)
buflen = copy(buf, buf[parsed:buflen])
for {
bodyLen, restOff, done := mp.NextBody(buf[:buflen])
if bodyLen > 0 && part.Sink != nil {
_, err = part.Sink.Write(buf[:bodyLen])
if err != nil {
return dst, err
}
}
buflen = copy(buf, buf[restOff:buflen])
if done {
break // Buffer now starts at the next part's delimiter.
}
n, readErr := exch.ReadBody(buf[buflen:])
buflen += n
if n == 0 && readErr != nil {
return dst, readErr // Body ended mid part.
}
}
if part.Sink != nil {
if err = part.Sink.Close(); err != nil {
return dst, err
}
}
}
}
// RequestHeader returns the value of the first request header field matching
// key, or nil if absent. Key matching is case sensitive.
func (exch *Exchange) RequestHeader(key string) []byte {
header := exch.RequestHeaderRaw()
return header.Get(key)
}
// RequestTarget returns the request-target (URI) of the request line, i.e:
// "/search?q=go". See [httpraw.Header.RequestTarget].
func (exch *Exchange) RequestTarget() []byte {
return exch.RequestHeaderRaw().RequestTarget()
}
// RequestPath returns the request-target (URI) up to the query string. This is
// what the [Mux] matches on, i.e: "/search" for a request to "/search?q=go".
func (exch *Exchange) RequestPath() []byte {
return exch.RequestHeaderRaw().RequestPath()
}
// RequestQuery returns the request's query string as it appears on the wire.
// Iterate it with [httpraw.NextQueryPair]. See [httpraw.Header.RequestQuery].
func (exch *Exchange) RequestQuery() []byte {
return exch.RequestHeaderRaw().RequestQuery()
}
// AppendQuery appends the value of the first query parameter matching key to
// dst and reports whether the parameter was present. A parameter with no value
// ("?debug") and one with an empty value ("?debug=") are both present with
// nothing appended.
//
// Keys are matched decoded, so key "a b" finds "a%20b" and "a+b". Values are
// appended raw unless decoded is set, in which case percent escapes and '+' are
// decoded. A parameter whose value fails to decode is reported absent, and a
// parameter whose key fails to decode is skipped.
//
// dst doubles as scratch space for decoding candidate keys, so AppendQuery only
// allocates when dst lacks the capacity to hold the longest key it inspects.
func (exch *Exchange) AppendQuery(dst []byte, key string, decoded bool) (valueAppended []byte, present bool) {
const plusAsSpace = true // Query strings are form encoded, unlike paths.
base := len(dst)
rawkey, rawval, rest := httpraw.NextQueryPair(exch.RequestQuery())
for ; rawkey != nil; rawkey, rawval, rest = httpraw.NextQueryPair(rest) {
if b2s(rawkey) != key {
// Key may be encoded: decode it over dst's free space and compare.
// A decoded key cannot appear raw, so this cannot alias a real key.
dst = slices.Grow(dst, len(rawkey))
scratch := dst[base : base+len(rawkey)]
n, err := httpraw.CopyDecodedPercentURL(scratch, rawkey, plusAsSpace)
if err != nil || b2s(scratch[:n]) != key {
continue // Malformed or different key, keep looking.
}
}
if len(rawval) == 0 {
return dst[:base], true // Flag or empty value, nothing to append.
}
dst = slices.Grow(dst, len(rawval))
if !decoded {
return append(dst[:base], rawval...), true
}
n, err := httpraw.CopyDecodedPercentURL(dst[base:base+len(rawval)], rawval, plusAsSpace)
if err != nil {
return dst[:base], false // Do not hand back half a decode.
}
return dst[:base+n], true
}
return dst[:base], false
}
// RequestMethod returns the request line's method, i.e: "GET". See
// [MethodFromBytes] to compare it against a [Method].
func (exch *Exchange) RequestMethod() []byte {
return exch.RequestHeaderRaw().Method()
}
// RequestConnectionClose returns true if the client asked for the connection to
// be closed after this exchange with a "Connection: close" header field.
func (exch *Exchange) RequestConnectionClose() bool {
return exch.RequestHeaderRaw().ConnectionClose()
}
File diff suppressed because it is too large Load Diff
+203
View File
@@ -0,0 +1,203 @@
package httphi
import (
"strings"
"unsafe"
"github.com/soypat/lneto"
"github.com/soypat/lneto/http/httpraw"
"github.com/soypat/lneto/internal"
)
// Handle is a extremely low-level HTTP handling method used internally in [Router].
// Requires exchange to be acquired and configured. Will panic if any argument is nil.
// Handle does not close the connection on any outcome: the caller owns it.
func Handle(exch *Exchange, mux Mux, backoff lneto.BackoffStrategy) error {
reqhdr := &exch.reqHdr
reqhdr.Reset(nil, 0) // Assume exchange has been configured and reuse memory.
var consecutiveBackoffs uint
for {
n, err := reqhdr.ReadFromLimited(exch.rw, reqhdr.BufferFree())
if err != nil {
exch.readErr = err
exch.handleError(err)
return err
} else if n == 0 {
backoff.Do(consecutiveBackoffs)
consecutiveBackoffs++
continue
}
consecutiveBackoffs = 0
const asRequest = false
needMore, err := reqhdr.TryParse(asRequest)
if needMore {
continue // Request header split across reads, accumulate the rest.
} else if err != nil {
exch.handleError(err)
return err
}
break // Done!
}
// Setup Exchange fields necessary for correct functioning.
parsed := reqhdr.BufferParsed()
exch.respRemains = reqhdr.BufferReceived() - parsed
exch.respHeaderOff = uint16(parsed)
exch.respHeaderLen = 0
if len(reqhdr.Protocol()) == 0 {
// Request line with no HTTP version is a HTTP/0.9 simple-request, which
// httpraw tolerates. It is not a valid HTTP/1.1 request-line, RFC 9112 3.
exch.WriteHeader(int(StatusBadRequest))
return errNoRequestProto
}
// Mux on the request path: the query string is the handler's business.
path := reqhdr.RequestPath()
meth := reqhdr.Method()
matchedPattern, handler := mux.LookupHandler(MethodFromBytes(meth), b2s(path))
if handler != nil {
exch.matchedPattern = matchedPattern
handler(exch)
exch.FlushHeader()
} else {
exch.WriteHeader(404)
}
// TODO write response from exchange here.
return nil
}
func (exch *Exchange) handleError(err error) {
if err == httpraw.ErrHeaderTooMany || err == httpraw.ErrSmallHeaderBuffer || exch.reqHdr.BufferFree() == 0 {
// The peer is owed an answer: no larger buffer is coming, so
// say so instead of dropping the connection, RFC 6585 5.
exch.StageHeader("Content-Length", "0")
exch.WriteHeader(int(StatusRequestHeaderFieldsTooLarge))
}
}
// HandlerFunc serves a single request, playing the part of http.Handler.
// The exchange is only valid for the duration of the call: it is released to
// the router's pool on return, so a handler must not retain it nor any slice it
// handed out.
type HandlerFunc func(ex *Exchange)
// Mux resolves a request to the handler that serves it. [Handle] calls
// LookupHandler with the request-target's path, not the whole target, and
// replies 404 when it returns nil.
type Mux interface {
// LookupHandler matches the requestPath and method to a handler and returns it and the
// pattern it matched.
LookupHandler(get Method, requestPath string) (matchedPattern string, handler HandlerFunc)
}
// MuxSlice is a [Mux] backed by a slice of registered endpoints, matched by
// exact path. Lookup is linear in the number of registrations.
type MuxSlice struct {
// TODO: binary search worth it?
_handlers []struct {
method Method
path string
handler HandlerFunc
}
}
// Reset discards all registered handlers, reusing the backing array and growing
// it to fit capacity registrations.
func (sm *MuxSlice) Reset(capacity int) {
internal.SliceReuse(&sm._handlers, capacity)
}
// LookupHandler returns the handler registered for request path, or nil if none matches.
// The first registration matching both method and uri wins.
func (sm *MuxSlice) LookupHandler(method Method, path string) (matched string, _ HandlerFunc) {
for _, endpoint := range sm._handlers {
if endpoint.method != MethUndefined && endpoint.method != method {
continue
}
// Method matches.
if path == endpoint.path {
return endpoint.path, endpoint.handler
}
}
return "", nil
}
// Handle registers handler for reg, either a bare path matching any method or a
// method and path separated by a space, i.e: "/health" or "GET /health".
// Handle does not check for duplicate registrations: the first one added wins.
func (sm *MuxSlice) Handle(optMethodAndPath string, handler HandlerFunc) {
v := internal.SliceReclaim(&sm._handlers)
method := MethUndefined
methodOrURL, url, methodFound := strings.Cut(optMethodAndPath, " ")
if methodFound {
method = MethodFrom(methodOrURL)
} else {
url = methodOrURL
}
v.method = method
v.path = url
v.handler = handler
}
// Method is a HTTP request method, parsed by [MethodFrom].
type Method uint8
const (
MethUndefined Method = iota // undefined
MethGet // GET
// lol.
MethHead // HEAD
MethPost // POST
MethPut // PUT
// RFC 5789
MethPatch // PATCH
MethDelete // DELETE
MethConnect // CONNECT
MethOptions // OPTIONS
MethTrace // TRACE
MethUnknown // unknown
)
// MethodFrom returns the [Method] matching meth, [MethUndefined] if meth is
// empty and [MethUnknown] if it names a method this package does not know.
// Comparison is case sensitive: methods are uppercase, RFC 9110 9.1.
func MethodFrom(meth string) (res Method) {
if len(meth) == 0 {
return MethUndefined
}
switch meth {
case "GET":
res = MethGet
case "HEAD":
res = MethHead
case "POST":
res = MethPost
case "PUT":
res = MethPut
case "PATCH":
res = MethPatch
case "DELETE":
res = MethDelete
case "CONNECT":
res = MethConnect
case "OPTIONS":
res = MethOptions
case "TRACE":
res = MethTrace
default:
res = MethUnknown
}
return res
}
// MethodFromBytes is a [MethodFrom] wrapper with bytes argument instead of string.
func MethodFromBytes(meth []byte) (res Method) {
if len(meth) == 0 {
return MethUndefined
}
return MethodFrom(b2s(meth))
}
// b2s converts byte slice to a string without memory allocation.
// See https://groups.google.com/forum/#!msg/Golang-Nuts/ENgbUzYvCuU/90yGx7GUAgAJ .
func b2s(b []byte) string {
return unsafe.String(unsafe.SliceData(b), len(b))
}
+362
View File
@@ -0,0 +1,362 @@
package httphi
import (
"errors"
"io"
"log/slog"
"sync"
"sync/atomic"
"time"
"github.com/soypat/lneto"
"github.com/soypat/lneto/internal"
)
//go:generate stringer -type Method -linecomment -output stringers.go
// reconfigureWait bounds how long [Router.Configure] waits for the previous
// generation to stop serving before reusing its exchange buffers.
const reconfigureWait = 10 * time.Millisecond
var (
errNoRequestProto = errors.New("httphi: request line with no HTTP version")
errBusyExchanges = errors.New("httphi: exchanges still serving, cannot reuse their buffers")
errRouterTornDown = errors.New("httphi: router torn down, configure it before serving")
errNotFormEncoded = errors.New("httphi: request body is not application/x-www-form-urlencoded")
errNotMultipart = errors.New("httphi: request body is not multipart/form-data")
errUnsupportedTransferCoding = errors.New("httphi: transfer coding not decoded, read the body directly")
)
type conn = io.ReadWriteCloser
// Router serves HTTP connections handed to it with [Router.Handle], routing
// each request to a handler found through its [Mux]. It plays the part of
// http.Server minus the listening: accepting connections is the caller's job,
// which is what lets the same router run over a TCP stack, a socket or a test
// pipe.
//
// A Router owns the exchanges and goroutines that serve connections and sizes
// both at [Router.Configure] time, so serving load costs no allocation and
// bounded memory. Connections arriving with nothing left to serve them are
// refused rather than queued, see [Router.Handle].
//
// Methods are safe for concurrent use. The zero value is not usable: configure
// it first.
type Router struct {
mu sync.Mutex
gen atomic.Uint32
numGoro int
reqBuf int
respBuf int
reqNumHeaderCap int
normalizeKeys bool
pendingConns chan job
mux Mux
globbuf []byte
exchs []Exchange
freeList *Exchange
backoff lneto.BackoffStrategy
log *slog.Logger
}
// job is a connection waiting on an exchange for a worker goroutine to serve it.
type job struct {
exch *Exchange
}
// RouterConfig configures a [Router]. See [Router.Configure].
type RouterConfig struct {
// FixedNumGoroutines must be set to either -1 (freely allocate new goroutines) or to the number of goroutines
// to spawn on [Router.Configure] being called.
FixedNumGoroutines int
// RequestHeaderBufferSize determines the buffer allocated
// for processing request HTTP headers including request-target (URI), protocol and key/value pairs.
RequestHeaderBufferSize int
// ResponseHeaderMinBufferSize determines buffer allocated for processing response headers.
// Response buffer will reuse unused request memory so this is not a strict limit.
// "HTTP/1.1 200 OK\r\n" does not count towards this memory, only actual Headers key/value pairs use this memory.
// After memory is fully consumed [Exchange.StageHeader] will not append more headers.
ResponseHeaderMinBufferSize int
// Number of request header key/value pairs to parse before failing and returning [StatusRequestHeaderFieldsTooLarge].
RequestNumHeaderKVCap int
// NormalizeOutgoingKeys normalizes response header field keys as they are
// staged, i.e: "content-type" becomes "Content-Type".
NormalizeOutgoingKeys bool
// MaxAwaitingConns is the depth of the queue connections wait in for a free
// goroutine. [Router.Handle] drops connections once it is full. Required and
// must be non-zero when running a fixed number of goroutines, unused otherwise.
MaxAwaitingConns int
// Backoff is consulted when a read off a connection yields no data, letting
// the caller decide whether to sleep, yield or spin. Required.
Backoff lneto.BackoffStrategy
// Mux resolves each request's method and path to the handler serving it. Required.
Mux Mux
// Logger receives failed exchanges. Optional, nil disables logging.
Logger *slog.Logger
}
// Validate returns a non-nil error if the configuration cannot be used to
// configure a [Router].
func (cfg RouterConfig) Validate() error {
workerMode := cfg.workerMode()
if workerMode && cfg.MaxAwaitingConns == 0 ||
cfg.Mux == nil ||
cfg.RequestNumHeaderKVCap <= 0 ||
!workerMode && cfg.FixedNumGoroutines != -1 {
return lneto.ErrInvalidConfig
} else if cfg.Backoff == nil {
return lneto.ErrMissingHALConfig
}
return nil
}
func (cfg RouterConfig) workerMode() bool {
return cfg.FixedNumGoroutines > 0
}
// TeardownGoroutines stops the router's fixed goroutines once they finish the
// exchanges they are serving. [Router.Configure] calls it before installing a
// new generation. A torn down router refuses connections with a non-nil error
// until it is configured again.
func (r *Router) TeardownGoroutines() {
r.mu.Lock()
defer r.mu.Unlock()
r.teardownGoroutinesLocked()
}
// teardownGoroutinesLocked closes the job queue fixed goroutines feed from.
// Requires r.mu held so that a concurrent [Router.Handle] cannot be enqueueing
// on the channel being closed.
func (r *Router) teardownGoroutinesLocked() {
r.gen.Add(1)
if r.pendingConns != nil {
close(r.pendingConns)
r.pendingConns = nil
}
}
// Configure prepares the router to serve connections, tearing down the previous
// generation of goroutines and exchanges first. In worker mode it spawns
// [RouterConfig.FixedNumGoroutines] goroutines and allocates their exchange
// buffers up front, so the router's memory use does not grow with load.
//
// Configure may be called on a serving router, but since the exchange buffers
// are reused it waits for connections in flight to finish and fails with a
// non-nil error rather than reconfigure buffers still being served from.
func (r *Router) Configure(cfg RouterConfig) error {
if err := cfg.Validate(); err != nil {
return err
}
r.mu.Lock()
defer r.mu.Unlock()
r.teardownGoroutinesLocked()
gen := r.gen.Load()
numgoro := cfg.FixedNumGoroutines
workerMode := cfg.workerMode()
r.reqNumHeaderCap = cfg.RequestNumHeaderKVCap
r.reqBuf = cfg.RequestHeaderBufferSize
r.respBuf = cfg.ResponseHeaderMinBufferSize
r.mux = cfg.Mux
r.log = cfg.Logger
r.normalizeKeys = cfg.NormalizeOutgoingKeys
if !workerMode {
r.backoff = cfg.Backoff
r.numGoro = 0
r.pendingConns = nil
return nil
}
if workerMode {
jobqueue := make(chan job, cfg.MaxAwaitingConns)
if gen > 1 {
// Exchange buffers below are reused: the previous generation must be
// done serving before they may be handed to the new one.
err := r.awaitIdleExchangesLocked(reconfigureWait)
if err != nil {
return err
}
}
r.freeList = nil // Freelist entries point into the buffers reused below.
internal.SliceReuse(&r.exchs, numgoro)
r.exchs = r.exchs[:numgoro]
rawBuflen := cfg.RequestHeaderBufferSize + cfg.ResponseHeaderMinBufferSize
internal.SliceReuse(&r.globbuf, numgoro*rawBuflen)
for i := range numgoro {
// TODO exchange buffer alloc
goff := i * rawBuflen
// r.globbuf[goff:goff+rawBuflen], cfg.RequestHeaderBufferSize, cfg.RequestNumHeaderCap, cfg.NormalizeOutgoingKeys
r.exchs[i].Configure(ExchangeConfig{
RawBuf: r.globbuf[goff : goff+rawBuflen],
RequestBufferLim: cfg.RequestHeaderBufferSize,
NumHeaderKVCap: cfg.RequestNumHeaderKVCap,
NormalizeOutgoingKeys: cfg.NormalizeOutgoingKeys,
NoRequestBufferGrowth: true, // Hard memory limit.
})
go r.goroWorker(gen, jobqueue, cfg.Backoff, cfg.Mux)
}
r.pendingConns = jobqueue
r.numGoro = numgoro
}
return nil
}
// awaitIdleExchangesLocked waits up to maxWait for exchanges of the previous
// generation to finish serving so their buffers may be reused. Requires r.mu
// held; the lock is released while waiting since [Router.freeExch] needs it to
// free the exchanges being waited on.
func (r *Router) awaitIdleExchangesLocked(maxWait time.Duration) error {
const pollInterval = time.Millisecond
for waited := time.Duration(0); ; waited += pollInterval {
busy := false
for i := range r.exchs {
if r.exchs[i].used.Load() {
busy = true
break
}
}
if !busy {
return nil
} else if waited >= maxWait {
return errBusyExchanges
}
r.mu.Unlock()
time.Sleep(pollInterval)
r.mu.Lock()
}
}
// Handle takes ownership of conn and serves one exchange on it, closing it when
// done. It does not block on the exchange: the connection is handed to a
// goroutine and Handle returns immediately.
//
// Handle returns [lneto.ErrExhausted] when no exchange is free,
// [lneto.ErrPacketDrop] when the queue of connections awaiting a goroutine is
// full, and an error when the router's goroutines have been torn down. On every
// one of them conn is left untouched and unclosed for the caller to dispose of:
// refusing connections is how a router with fixed memory applies backpressure.
func (r *Router) Handle(conn io.ReadWriteCloser) error {
// Exchange acquisition and the configuration it is served with must be read
// under the same lock: [Router.Configure] may run concurrently.
r.mu.Lock()
numGoro, backoff, mux := r.numGoro, r.backoff, r.mux
if numGoro > 0 && r.pendingConns == nil {
// Goroutines torn down: refuse before claiming an exchange.
r.mu.Unlock()
return errRouterTornDown
}
exch := r.getExchLocked(conn)
if exch == nil {
r.mu.Unlock()
return lneto.ErrExhausted
} else if numGoro == 0 {
r.mu.Unlock()
go r.goroHandle(exch, backoff, mux)
return nil
}
// Enqueue under the lock: [Router.Configure] closes pendingConns while
// holding it, so an unlocked send could land on a closed channel. The send
// never blocks, so holding the lock cannot stall a worker.
var enqueued bool
select {
case r.pendingConns <- job{exch: exch}:
enqueued = true
default:
// pendingConns cannot store another Conn, we drop and return error.
exch.used.Store(false) // release.
}
r.mu.Unlock()
if enqueued {
return nil
}
return lneto.ErrPacketDrop
}
func (r *Router) goroWorker(gen uint32, queue chan job, backoff lneto.BackoffStrategy, mux Mux) {
for job := range queue {
exch := job.exch
if gen != r.gen.Load() {
return
} else if exch == nil {
panic("httplo: unreachable nil job")
}
r.goroHandle(exch, backoff, mux)
}
}
func (r *Router) goroHandle(exch *Exchange, backoff lneto.BackoffStrategy, mux Mux) {
defer r.freeExch(exch)
err := Handle(exch, mux, backoff)
if err != nil {
if exch.readErr != nil {
r.error("goroHandle:ReadFromLimited", slog.String("err", err.Error()))
} else {
r.error("goroHandle:TryParse?", slog.String("err", err.Error()))
}
}
}
func (r *Router) freeExch(exch *Exchange) {
const freelistMaxDepth = 5
r.mu.Lock()
depth := 0
for node := r.freeList; node != nil && depth < freelistMaxDepth; node = node.nextFree {
depth++
}
if depth < freelistMaxDepth {
// Push at head: appending at the tail would drop every node past the
// depth limit instead of dropping the exchange we cannot store.
exch.nextFree = r.freeList
r.freeList = exch
} else {
exch.nextFree = nil // Freelist full, exchange is dropped.
}
exch.Release()
r.mu.Unlock()
}
// getExchLocked returns an exchange acquired on conn. Requires r.mu held.
func (r *Router) getExchLocked(conn conn) (exch *Exchange) {
if r.freeList != nil {
// Successor must be read before Acquire: Acquire clears nextFree, so
// popping afterwards would truncate the freelist to the popped node.
next := r.freeList.nextFree
if r.freeList.Acquire(conn) {
exch = r.freeList
r.freeList = next
return exch
}
}
for i := range r.exchs {
if r.exchs[i].Acquire(conn) {
return &r.exchs[i]
}
}
// Unbounded mode is stored as zero, see [Router.Configure]: a router that
// spawns a goroutine per connection allocates the exchange to go with it.
if r.numGoro == 0 {
exch := new(Exchange)
exch.Configure(ExchangeConfig{
RawBuf: make([]byte, r.respBuf+r.reqBuf),
RequestBufferLim: r.reqBuf,
NumHeaderKVCap: r.reqNumHeaderCap,
NormalizeOutgoingKeys: r.normalizeKeys,
NoRequestBufferGrowth: true,
})
exch.Acquire(conn) // Fresh exchange, CAS cannot fail.
return exch
}
return nil
}
func (r *Router) error(msg string, attrs ...slog.Attr) {
internal.LogAttrs(r.log, slog.LevelError, msg, attrs...)
}
func (r *Router) info(msg string, attrs ...slog.Attr) {
internal.LogAttrs(r.log, slog.LevelInfo, msg, attrs...)
}
+415
View File
@@ -0,0 +1,415 @@
package httphi
import (
"bytes"
"context"
"io"
"net"
"strings"
"sync"
"testing"
"time"
"github.com/soypat/lneto"
)
// rwconn is a in-memory conn. The router handles connections on another
// goroutine so every field is guarded; onClose lets tests await the handler.
//
// A drained rwconn reads (0,nil) as a live socket with no data pending would,
// so a partial request can be completed with AddReadable mid-handling. Tests
// that need the peer to hang up call Hangup.
type rwconn struct {
mu sync.Mutex
readable bytes.Buffer
segments []string
written bytes.Buffer
closed bool
hangup bool
failWr int
onClose chan struct{}
deadline time.Time
}
// newConn returns a conn preloaded with request and whose Close is observable
// with [rwconn.AwaitClose].
func newConn(request string) *rwconn {
r := &rwconn{onClose: make(chan struct{})}
r.AddReadable([]byte(request))
return r
}
// AddSegment queues data delivered on a later read, once everything already
// pending has been read. Models a request split over several TCP segments
// without depending on goroutine scheduling.
func (r *rwconn) AddSegment(b string) {
r.mu.Lock()
defer r.mu.Unlock()
r.segments = append(r.segments, b)
}
// FailWrites makes the next n writes fail, as a conn refusing further data
// would. Later writes succeed.
func (r *rwconn) FailWrites(n int) {
r.mu.Lock()
defer r.mu.Unlock()
r.failWr = n
}
// Hangup makes reads past the pending data return [io.EOF], as a peer that
// closed its side of the connection would.
func (r *rwconn) Hangup() {
r.mu.Lock()
defer r.mu.Unlock()
r.hangup = true
}
func (r *rwconn) Close() error {
r.mu.Lock()
defer r.mu.Unlock()
if !r.closed {
r.closed = true
if r.onClose != nil {
close(r.onClose)
}
}
return nil
}
// AwaitClose blocks until the connection is closed by its handler or timeout elapses.
func (r *rwconn) AwaitClose(t *testing.T, timeout time.Duration) {
t.Helper()
select {
case <-r.onClose:
case <-time.After(timeout):
t.Fatal("timed out awaiting connection close by handler")
}
}
func (r *rwconn) Read(b []byte) (int, error) {
r.mu.Lock()
defer r.mu.Unlock()
if r.closed {
return 0, net.ErrClosed
} else if r.deadlineExceeded() {
return 0, context.DeadlineExceeded
} else if r.readable.Len() == 0 {
if len(r.segments) > 0 {
r.readable.WriteString(r.segments[0])
r.segments = r.segments[1:]
return r.readable.Read(b)
}
if r.hangup {
return 0, io.EOF
}
return 0, nil // No data pending, handler backs off and retries.
}
return r.readable.Read(b)
}
func (r *rwconn) Write(b []byte) (int, error) {
r.mu.Lock()
defer r.mu.Unlock()
if r.closed {
return 0, net.ErrClosed
} else if r.deadlineExceeded() {
return 0, context.DeadlineExceeded
} else if r.failWr > 0 {
r.failWr--
return 0, io.ErrShortWrite
}
return r.written.Write(b)
}
func (r *rwconn) deadlineExceeded() bool {
return !r.deadline.IsZero() && time.Since(r.deadline) > 0
}
func (r *rwconn) AddReadable(b []byte) {
r.mu.Lock()
defer r.mu.Unlock()
r.readable.Write(b)
}
// SetDeadline makes reads and writes past t fail, as a conn with a read
// deadline set would.
func (r *rwconn) SetDeadline(t time.Time) {
r.mu.Lock()
defer r.mu.Unlock()
r.deadline = t
}
// IsClosed reports whether the connection was closed by its handler.
func (r *rwconn) IsClosed() bool {
r.mu.Lock()
defer r.mu.Unlock()
return r.closed
}
func (r *rwconn) ViewWritten() string {
r.mu.Lock()
defer r.mu.Unlock()
return r.written.String()
}
var _ Mux = (*MuxSlice)(nil)
func configSynchronousRouter(t *testing.T, router *Router, bufferSize int, mux Mux) {
err := router.Configure(RouterConfig{
FixedNumGoroutines: -1,
Mux: mux,
RequestHeaderBufferSize: bufferSize,
RequestNumHeaderKVCap: 16,
ResponseHeaderMinBufferSize: bufferSize,
Backoff: func(consecutiveBackoffs uint) (sleepOrFlag time.Duration) {
return lneto.BackoffFlagNop
},
})
if err != nil {
t.Fatal(err)
}
}
func TestRouterGet(t *testing.T) {
const bufferSize = 1024
const expectResponse = "its time"
var (
sm MuxSlice
router Router
)
sm.Handle("GET /", staticPage(t, expectResponse))
configSynchronousRouter(t, &router, bufferSize, &sm)
conn := newConn("GET / HTTP/1.1\r\nHost: tinygo.org\r\n\r\n")
err := router.Handle(conn)
if err != nil {
t.Fatal(err)
}
conn.AwaitClose(t, time.Second)
got := conn.ViewWritten()
if !strings.HasPrefix(got, "HTTP/1.1 200 OK\r\n") {
t.Errorf("want 200 status line, got %q", got)
}
if !strings.HasSuffix(got, expectResponse) {
t.Errorf("want body %q at end of response, got %q", expectResponse, got)
}
}
// The handler observes the request line and header fields the router parsed.
func TestRouterRequestVisibleToHandler(t *testing.T) {
const bufferSize = 1024
var (
sm MuxSlice
router Router
)
var gotMethod, gotURI, gotHost string
sm.Handle("GET /index.html", func(ex *Exchange) {
gotMethod = string(ex.RequestMethod())
gotURI = string(ex.RequestTarget())
gotHost = string(ex.RequestHeader("Host"))
ex.WriteHeader(200)
})
configSynchronousRouter(t, &router, bufferSize, &sm)
conn := newConn("GET /index.html HTTP/1.1\r\nHost: tinygo.org\r\n\r\n")
if err := router.Handle(conn); err != nil {
t.Fatal(err)
}
conn.AwaitClose(t, time.Second)
if gotMethod != "GET" {
t.Errorf("want method %q, got %q", "GET", gotMethod)
}
if gotURI != "/index.html" {
t.Errorf("want URI %q, got %q", "/index.html", gotURI)
}
if gotHost != "tinygo.org" {
t.Errorf("want Host %q, got %q", "tinygo.org", gotHost)
}
}
// Router must route on method and URI, and must not invoke a handler for
// requests it has no registration for.
func TestRouterMux(t *testing.T) {
const bufferSize = 1024
for _, test := range []struct {
name string
request string
want string // Response body, empty means no handler must run.
}{
{name: "get root", request: "GET / HTTP/1.1\r\nHost: h\r\n\r\n", want: "root"},
{name: "get page", request: "GET /page HTTP/1.1\r\nHost: h\r\n\r\n", want: "page"},
{name: "any method", request: "DELETE /any HTTP/1.1\r\nHost: h\r\n\r\n", want: "any"},
{name: "method mismatch", request: "POST / HTTP/1.1\r\nHost: h\r\n\r\n", want: ""},
{name: "unknown uri", request: "GET /nowhere HTTP/1.1\r\nHost: h\r\n\r\n", want: ""},
} {
t.Run(test.name, func(t *testing.T) {
var (
sm MuxSlice
router Router
)
sm.Handle("GET /", staticPage(t, "root"))
sm.Handle("GET /page", staticPage(t, "page"))
sm.Handle("/any", staticPage(t, "any")) // No method: matches any.
configSynchronousRouter(t, &router, bufferSize, &sm)
conn := newConn(test.request)
if err := router.Handle(conn); err != nil {
t.Fatal(err)
}
conn.AwaitClose(t, time.Second)
got := conn.ViewWritten()
if test.want == "" {
if strings.Contains(got, "root") || strings.Contains(got, "page") || strings.Contains(got, "any") {
t.Errorf("no handler must run, got response %q", got)
}
return
}
if !strings.HasSuffix(got, test.want) {
t.Errorf("want body %q, got response %q", test.want, got)
}
})
}
}
// A request arriving in pieces (TCP segmentation) must still be handled.
func TestRouterSplitRequest(t *testing.T) {
const bufferSize = 1024
const expectResponse = "split ok"
var (
sm MuxSlice
router Router
)
sm.Handle("GET /", staticPage(t, expectResponse))
configSynchronousRouter(t, &router, bufferSize, &sm)
conn := newConn("GET / HTTP/1.1\r\nHo")
conn.AddSegment("st: tinygo.org\r\n\r")
conn.AddSegment("\n") // Final CRLF lands in its own segment.
if err := router.Handle(conn); err != nil {
t.Fatal(err)
}
conn.AwaitClose(t, time.Second)
if got := conn.ViewWritten(); !strings.HasSuffix(got, expectResponse) {
t.Errorf("want body %q, got response %q", expectResponse, got)
}
}
func staticPage(t *testing.T, page string) HandlerFunc {
return func(ex *Exchange) {
var rw ExchangeRW // Streaming APIs take the ReadWriter view.
ex.ReadWriter(&rw)
n, err := io.WriteString(&rw, page)
// Handler runs on the router goroutine: Error, never Fatal.
if err != nil {
t.Error(err)
} else if n != len(page) {
t.Error("expected written ", len(page), "got", n)
}
}
}
// Configure writes the fields Handle reads; concurrent use must not race.
func TestRouterConfigureHandleRace(t *testing.T) {
const bufferSize = 1024
var (
sm MuxSlice
router Router
)
sm.Handle("GET /", staticPage(t, "ok"))
configSynchronousRouter(t, &router, bufferSize, &sm)
var wg sync.WaitGroup
wg.Add(2)
go func() {
defer wg.Done()
configSynchronousRouter(t, &router, bufferSize, &sm)
}()
go func() {
defer wg.Done()
conn := newConn("GET / HTTP/1.1\r\nHost: h\r\n\r\n")
if err := router.Handle(conn); err != nil {
t.Error(err)
}
}()
wg.Wait()
}
// Reconfiguring a running router tears down the job queue that Handle may be
// sending a connection on. Connections may be dropped, but never panic.
// A torn down router has nothing left to serve with: it must say so instead of
// dropping the connection as if it were merely busy.
func TestRouterHandleAfterTeardown(t *testing.T) {
var (
sm MuxSlice
router Router
)
sm.Handle("GET /", staticPage(t, "ok"))
err := router.Configure(RouterConfig{
FixedNumGoroutines: 2,
MaxAwaitingConns: 4,
Mux: &sm,
RequestHeaderBufferSize: 512,
RequestNumHeaderKVCap: 16,
ResponseHeaderMinBufferSize: 512,
Backoff: nopBackoff,
})
if err != nil {
t.Fatal(err)
}
router.TeardownGoroutines()
conn := newConn("GET / HTTP/1.1\r\nHost: h\r\n\r\n")
if err = router.Handle(conn); err != errRouterTornDown {
t.Errorf("want errRouterTornDown, got %v", err)
}
if conn.IsClosed() {
t.Error("refused connection must be left for the caller to dispose of")
}
}
func TestRouterConfigureDuringWorkerHandle(t *testing.T) {
var (
sm MuxSlice
router Router
)
sm.Handle("GET /", staticPage(t, "ok"))
cfg := RouterConfig{
FixedNumGoroutines: 2,
MaxAwaitingConns: 4,
Mux: &sm,
RequestHeaderBufferSize: 512,
RequestNumHeaderKVCap: 16,
ResponseHeaderMinBufferSize: 512,
Backoff: nopBackoff,
}
if err := router.Configure(cfg); err != nil {
t.Fatal(err)
}
defer router.TeardownGoroutines()
var wg sync.WaitGroup
wg.Add(2)
go func() {
defer wg.Done()
for range 300 {
conn := newConn("GET / HTTP/1.1\r\nHost: h\r\n\r\n")
conn.Hangup()
router.Handle(conn) // Drops are fine, panics are not.
}
}()
go func() {
defer wg.Done()
// Each Configure sleeps 5ms tearing down the previous generation, keep
// the count low and let the Handle loop supply the concurrency.
for range 20 {
// errBusyExchanges is legitimate backpressure: the previous
// generation was still serving when the buffers were needed.
if err := router.Configure(cfg); err != nil && err != errBusyExchanges {
t.Error(err)
return
}
}
}()
wg.Wait()
}
+273
View File
@@ -0,0 +1,273 @@
package httphi
// StatusText returns a text for the HTTP status code. It returns the empty
// string if the code is unknown.
func StatusText(code int) string {
switch status(code) {
case StatusContinue:
return "Continue"
case StatusSwitchingProtocols:
return "Switching Protocols"
case StatusProcessing:
return "Processing"
case StatusEarlyHints:
return "Early Hints"
case StatusOK:
return "OK"
case StatusCreated:
return "Created"
case StatusAccepted:
return "Accepted"
case StatusNonAuthoritativeInfo:
return "Non-Authoritative Information"
case StatusNoContent:
return "No Content"
case StatusResetContent:
return "Reset Content"
case StatusPartialContent:
return "Partial Content"
case StatusMultiStatus:
return "Multi-Status"
case StatusAlreadyReported:
return "Already Reported"
case StatusIMUsed:
return "IM Used"
case StatusMultipleChoices:
return "Multiple Choices"
case StatusMovedPermanently:
return "Moved Permanently"
case StatusFound:
return "Found"
case StatusSeeOther:
return "See Other"
case StatusNotModified:
return "Not Modified"
case StatusUseProxy:
return "Use Proxy"
case StatusTemporaryRedirect:
return "Temporary Redirect"
case StatusPermanentRedirect:
return "Permanent Redirect"
case StatusBadRequest:
return "Bad Request"
case StatusUnauthorized:
return "Unauthorized"
case StatusPaymentRequired:
return "Payment Required"
case StatusForbidden:
return "Forbidden"
case StatusNotFound:
return "Not Found"
case StatusMethodNotAllowed:
return "Method Not Allowed"
case StatusNotAcceptable:
return "Not Acceptable"
case StatusProxyAuthRequired:
return "Proxy Authentication Required"
case StatusRequestTimeout:
return "Request Timeout"
case StatusConflict:
return "Conflict"
case StatusGone:
return "Gone"
case StatusLengthRequired:
return "Length Required"
case StatusPreconditionFailed:
return "Precondition Failed"
case StatusRequestEntityTooLarge:
return "Request Entity Too Large"
case StatusRequestURITooLong:
return "Request URI Too Long"
case StatusUnsupportedMediaType:
return "Unsupported Media Type"
case StatusRequestedRangeNotSatisfiable:
return "Requested Range Not Satisfiable"
case StatusExpectationFailed:
return "Expectation Failed"
case StatusTeapot:
return "I'm a teapot"
case StatusMisdirectedRequest:
return "Misdirected Request"
case StatusUnprocessableEntity:
return "Unprocessable Entity"
case StatusLocked:
return "Locked"
case StatusFailedDependency:
return "Failed Dependency"
case StatusTooEarly:
return "Too Early"
case StatusUpgradeRequired:
return "Upgrade Required"
case StatusPreconditionRequired:
return "Precondition Required"
case StatusTooManyRequests:
return "Too Many Requests"
case StatusRequestHeaderFieldsTooLarge:
return "Request Header Fields Too Large"
case StatusUnavailableForLegalReasons:
return "Unavailable For Legal Reasons"
case StatusInternalServerError:
return "Internal Server Error"
case StatusNotImplemented:
return "Not Implemented"
case StatusBadGateway:
return "Bad Gateway"
case StatusServiceUnavailable:
return "Service Unavailable"
case StatusGatewayTimeout:
return "Gateway Timeout"
case StatusHTTPVersionNotSupported:
return "HTTP Version Not Supported"
case StatusVariantAlsoNegotiates:
return "Variant Also Negotiates"
case StatusInsufficientStorage:
return "Insufficient Storage"
case StatusLoopDetected:
return "Loop Detected"
case StatusNotExtended:
return "Not Extended"
case StatusNetworkAuthenticationRequired:
return "Network Authentication Required"
default:
return ""
}
}
const ()
type status int
// HTTP status codes as registered with IANA.
// See: https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml
const (
// RFC 9110, 15.2.1
StatusContinue = 100 // Continue
// RFC 9110, 15.2.2
StatusSwitchingProtocols = 101 // Switching Protocols
// RFC 2518, 10.1
StatusProcessing = 102 // Processing
// RFC 8297
StatusEarlyHints = 103 // Early Hints
// RFC 9110, 15.3.1
StatusOK = 200 // OK
// RFC 9110, 15.3.2
StatusCreated = 201 // Created
// RFC 9110, 15.3.3
StatusAccepted = 202 // Accepted
// RFC 9110, 15.3.4
StatusNonAuthoritativeInfo = 203 // Non-Authoritative Information
// RFC 9110, 15.3.5
StatusNoContent = 204 // No Content
// RFC 9110, 15.3.6
StatusResetContent = 205 // Reset Content
// RFC 9110, 15.3.7
StatusPartialContent = 206 // Partial Content
// RFC 4918, 11.1
StatusMultiStatus = 207 // Multi-Status
// RFC 5842, 7.1
StatusAlreadyReported = 208 // Already Reported
// RFC 3229, 10.4.1
StatusIMUsed = 226 // IM Used
// RFC 9110, 15.4.1
StatusMultipleChoices = 300 // Multiple Choices
// RFC 9110, 15.4.2
StatusMovedPermanently = 301 // Moved Permanently
// RFC 9110, 15.4.3
StatusFound = 302 // Found
// RFC 9110, 15.4.4
StatusSeeOther = 303 // See Other
// RFC 9110, 15.4.5
StatusNotModified = 304 // Not Modified
// RFC 9110, 15.4.6
StatusUseProxy = 305 // Use Proxy
// RFC 9110, 15.4.7 (Unused)
_ = 306
// RFC 9110, 15.4.8
StatusTemporaryRedirect = 307 // Temporary Redirect
// RFC 9110, 15.4.9
StatusPermanentRedirect = 308 // Permanent Redirect
// RFC 9110, 15.5.1
StatusBadRequest = 400 // Bad Request
// RFC 9110, 15.5.2
StatusUnauthorized = 401 // Unauthorized
// RFC 9110, 15.5.3
StatusPaymentRequired = 402 // Payment Required
// RFC 9110, 15.5.4
StatusForbidden = 403 // Forbidden
// RFC 9110, 15.5.5
StatusNotFound = 404 // Not Found
// RFC 9110, 15.5.6
StatusMethodNotAllowed = 405 // Method Not Allowed
// RFC 9110, 15.5.7
StatusNotAcceptable = 406 // Not Acceptable
// RFC 9110, 15.5.8
StatusProxyAuthRequired = 407 // Proxy Authentication Required
// RFC 9110, 15.5.9
StatusRequestTimeout = 408 // Request Timeout
// RFC 9110, 15.5.10
StatusConflict = 409 // Conflict
// RFC 9110, 15.5.11
StatusGone = 410 // Gone
// RFC 9110, 15.5.12
StatusLengthRequired = 411 // Length Required
// RFC 9110, 15.5.13
StatusPreconditionFailed = 412 // Precondition Failed
// RFC 9110, 15.5.14
StatusRequestEntityTooLarge = 413 // Request Entity Too Large
// RFC 9110, 15.5.15
StatusRequestURITooLong = 414 // Request URI Too Long
// RFC 9110, 15.5.16
StatusUnsupportedMediaType = 415 // Unsupported Media Type
// RFC 9110, 15.5.17
StatusRequestedRangeNotSatisfiable = 416 // Requested Range Not Satisfiable
// RFC 9110, 15.5.18
StatusExpectationFailed = 417 // Expectation Failed
// RFC 9110, 15.5.19 (Unused)
StatusTeapot = 418 // I'm a teapot
// RFC 9110, 15.5.20
StatusMisdirectedRequest = 421 // Misdirected Request
// RFC 9110, 15.5.21
StatusUnprocessableEntity = 422 // Unprocessable Entity
// RFC 4918, 11.3
StatusLocked = 423 // Locked
// RFC 4918, 11.4
StatusFailedDependency = 424 // Failed Dependency
// RFC 8470, 5.2.
StatusTooEarly = 425 // Too Early
// RFC 9110, 15.5.22
StatusUpgradeRequired = 426 // Upgrade Required
// RFC 6585, 3
StatusPreconditionRequired = 428 // Precondition Required
// RFC 6585, 4
StatusTooManyRequests = 429 // Too Many Requests
// RFC 6585, 5
StatusRequestHeaderFieldsTooLarge = 431 // Request Header Fields Too Large
// RFC 7725, 3
StatusUnavailableForLegalReasons = 451 // Unavailable For Legal Reasons
// RFC 9110, 15.6.1
StatusInternalServerError = 500 // Internal Server Error
// RFC 9110, 15.6.2
StatusNotImplemented = 501 // Not Implemented
// RFC 9110, 15.6.3
StatusBadGateway = 502 // Bad Gateway
// RFC 9110, 15.6.4
StatusServiceUnavailable = 503 // Service Unavailable
// RFC 9110, 15.6.5
StatusGatewayTimeout = 504 // Gateway Timeout
// RFC 9110, 15.6.6
StatusHTTPVersionNotSupported = 505 // HTTP Version Not Supported
// RFC 2295, 8.1
StatusVariantAlsoNegotiates = 506 // Variant Also Negotiates
// RFC 4918, 11.5
StatusInsufficientStorage = 507 // Insufficient Storage
// RFC 5842, 7.2
StatusLoopDetected = 508 // Loop Detected
// RFC 2774, 7
StatusNotExtended = 510 // Not Extended
// RFC 6585, 6
StatusNetworkAuthenticationRequired = 511 // Network Authentication Required
)
+34
View File
@@ -0,0 +1,34 @@
// Code generated by "stringer -type Method -linecomment -output stringers.go"; DO NOT EDIT.
package httphi
import "strconv"
func _() {
// An "invalid array index" compiler error signifies that the constant values have changed.
// Re-run the stringer command to generate them again.
var x [1]struct{}
_ = x[MethUndefined-0]
_ = x[MethGet-1]
_ = x[MethHead-2]
_ = x[MethPost-3]
_ = x[MethPut-4]
_ = x[MethPatch-5]
_ = x[MethDelete-6]
_ = x[MethConnect-7]
_ = x[MethOptions-8]
_ = x[MethTrace-9]
_ = x[MethUnknown-10]
}
const _Method_name = "undefinedGETHEADPOSTPUTPATCHDELETECONNECTOPTIONSTRACEunknown"
var _Method_index = [...]uint8{0, 9, 12, 16, 20, 23, 28, 34, 41, 48, 53, 60}
func (i Method) String() string {
idx := int(i) - 0
if i < 0 || idx >= len(_Method_index)-1 {
return "Method(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _Method_name[_Method_index[idx]:_Method_index[idx+1]]
}
+8 -4
View File
@@ -74,9 +74,11 @@ func (c *Cookie) Parse() error {
return nil
}
// ForEach iterates over the cookie's key-value pairs, stopping on the first
// error returned by cb and returning it.
func (c *Cookie) ForEach(cb func(key, value []byte) error) error {
nc := len(c.kvs)
for i := 0; i < nc; i++ {
for i := range nc {
kv := c.kvs[i]
key := tok2bytes(c.buf, kv.key)
value := tok2bytes(c.buf, kv.value)
@@ -91,7 +93,7 @@ func (c *Cookie) ForEach(cb func(key, value []byte) error) error {
// Get gets a cookie's value from its key. Use HasValueOrKey to check if a key or single-valued cookie is present in the cookie.
func (c *Cookie) Get(key string) []byte {
nc := len(c.kvs)
for i := 0; i < nc; i++ {
for i := range nc {
kv := c.kvs[i]
if b2s(tok2bytes(c.buf, kv.key)) == key {
return tok2bytes(c.buf, kv.value)
@@ -100,9 +102,11 @@ func (c *Cookie) Get(key string) []byte {
return nil
}
// HasKeyOrSingleValue returns true if the cookie contains a pair with the given
// key or a valueless attribute with the given text, i.e: "Secure" or "HttpOnly".
func (c *Cookie) HasKeyOrSingleValue(keyOrSingleValue string) bool {
nc := len(c.kvs)
for i := 0; i < nc; i++ {
for i := range nc {
kv := c.kvs[i]
if kv.key.len == 0 && b2s(tok2bytes(c.buf, kv.value)) == keyOrSingleValue ||
b2s(tok2bytes(c.buf, kv.key)) == keyOrSingleValue {
@@ -159,7 +163,7 @@ func (c *Cookie) String() string {
// AppendKeyValues appends the HTTP header value of the cookie expected after the "Cookie:" string. Does not include trailing \r\n's.
func (c *Cookie) AppendKeyValues(dst []byte) []byte {
nc := len(c.kvs)
for i := 0; i < nc; i++ {
for i := range nc {
kv := c.kvs[i]
key := tok2bytes(c.buf, kv.key)
value := tok2bytes(c.buf, kv.value)
+124
View File
@@ -0,0 +1,124 @@
package httpraw
// Form holds "application/x-www-form-urlencoded" key-value pairs, the encoding
// HTML forms use for POST bodies and query strings alike. Methods function
// similarly to eponymous [Cookie] methods.
//
// Pairs are stored as they appear on the wire, percent-encoded and with '+'
// undecoded, until [Form.Decode] rewrites them in place. The caller bounds the
// data: Form parses the buffer it is handed and reads nothing more.
type Form struct {
buf []byte
kvs []argsKV
}
// Reset discards parsed pairs and sets the buffer to parse in place.
// If buf is nil the current buffer is reused.
func (f *Form) Reset(buf []byte) {
if buf == nil {
buf = f.buf[:0]
}
*f = Form{
buf: buf,
kvs: f.kvs[:0],
}
}
// ParseBytes copies the argument bytes to the Form's underlying buffer and parses them.
func (f *Form) ParseBytes(b []byte) error {
f.Reset(nil)
f.buf = append(f.buf[:0], b...)
return f.Parse()
}
// Parse parses the form's buffer in place.
func (f *Form) Parse() error {
f.kvs = f.kvs[:0]
key, value, rest := NextQueryPair(f.buf)
for key != nil {
kv := argsKV{key: bytes2tok(f.buf, key)}
if value != nil {
kv.value = bytes2tok(f.buf, value)
}
f.kvs = append(f.kvs, kv)
key, value, rest = NextQueryPair(rest)
}
return nil
}
// Decode rewrites every key and value in place, replacing percent escapes and
// '+' with the bytes they encode. Decoding only shrinks, so no memory is added.
func (f *Form) Decode() error {
const plusAsSpace = true // Form encoded data, unlike a path.
for i := range f.kvs {
kv := &f.kvs[i]
n, err := CopyDecodedPercentURL(tok2bytes(f.buf, kv.key), tok2bytes(f.buf, kv.key), plusAsSpace)
if err != nil {
return err
}
kv.key.len = tokint(n)
if !kv.HasValue() {
continue
}
n, err = CopyDecodedPercentURL(tok2bytes(f.buf, kv.value), tok2bytes(f.buf, kv.value), plusAsSpace)
if err != nil {
return err
}
kv.value.len = tokint(n)
}
return nil
}
// Len returns the amount of key-value pairs parsed.
func (f *Form) Len() int { return len(f.kvs) }
// Pair returns the i'th key-value pair in wire order. The value is nil for a
// pair with no '=', i.e: "ok" in "ok&q=go", which distinguishes it from "ok="
// where the value is present and empty.
func (f *Form) Pair(i int) (key, value []byte) {
kv := f.kvs[i]
key = tok2bytes(f.buf, kv.key)
if kv.HasValue() {
value = tok2bytes(f.buf, kv.value)
}
return key, value
}
// Get returns the value of the first pair matching key, nil if absent or if the
// pair has no value. Bytes are compared as stored, so call [Form.Decode] first
// when keys may be encoded.
func (f *Form) Get(key string) []byte {
for i := range f.kvs {
gotKey, value := f.Pair(i)
if b2s(gotKey) == key {
return value
}
}
return nil
}
// Has returns true if key is present, with or without a value.
func (f *Form) Has(key string) bool {
for i := range f.kvs {
if b2s(tok2bytes(f.buf, f.kvs[i].key)) == key {
return true
}
}
return false
}
// AppendKeyValues appends the form's wire representation to dst and returns it.
func (f *Form) AppendKeyValues(dst []byte) []byte {
for i := range f.kvs {
key, value := f.Pair(i)
if i > 0 {
dst = append(dst, '&')
}
dst = append(dst, key...)
if value != nil {
dst = append(dst, '=')
dst = append(dst, value...)
}
}
return dst
}
+134
View File
@@ -0,0 +1,134 @@
package httpraw
import (
"strings"
"testing"
)
// render joins a form's pairs as "key=value", a valueless key as "key".
func render(f *Form) string {
var sb strings.Builder
for i := range f.Len() {
key, value := f.Pair(i)
if i > 0 {
sb.WriteByte('|')
}
sb.Write(key)
if value != nil {
sb.WriteByte('=')
sb.Write(value)
}
}
return sb.String()
}
func TestFormParse(t *testing.T) {
for _, test := range []struct {
body string
want string
}{
{body: "", want: ""},
{body: "q=go", want: "q=go"},
{body: "name=Jos%C3%A9+P%C3%A9rez&msg=hi+there&ok=on", want: "name=Jos%C3%A9+P%C3%A9rez|msg=hi+there|ok=on"},
{body: "ok", want: "ok"}, // Flag: no '=' at all.
{body: "ok=", want: "ok="}, // Present but empty.
{body: "&&q=go&", want: "q=go"}, // Empty sequences skipped.
{body: "tag=a&tag=b", want: "tag=a|tag=b"}, // Duplicates kept in order.
{body: "=v", want: "=v"}, // Empty name kept.
{body: "a=b=c", want: "a=b=c"}, // Only the first '=' splits.
} {
var f Form
if err := f.ParseBytes([]byte(test.body)); err != nil {
t.Fatalf("%q: %s", test.body, err)
}
if got := render(&f); got != test.want {
t.Errorf("%q: want %q, got %q", test.body, test.want, got)
}
}
}
// Decode rewrites keys and values in place: percent escapes and '+' as space.
func TestFormDecode(t *testing.T) {
var f Form
const body = "name=Jos%C3%A9+P%C3%A9rez&a%20b=c%2Bd&ok"
if err := f.ParseBytes([]byte(body)); err != nil {
t.Fatal(err)
}
if err := f.Decode(); err != nil {
t.Fatal(err)
}
const want = "name=José Pérez|a b=c+d|ok"
if got := render(&f); got != want {
t.Errorf("want %q, got %q", want, got)
}
if got := string(f.Get("a b")); got != "c+d" {
t.Errorf("want decoded key lookup %q, got %q", "c+d", got)
}
}
// A malformed escape must be reported, never silently passed through.
func TestFormDecodeMalformed(t *testing.T) {
for _, body := range []string{"q=%zz", "%zz=v", "q=%4"} {
var f Form
if err := f.ParseBytes([]byte(body)); err != nil {
t.Fatal(err)
}
if err := f.Decode(); err == nil {
t.Errorf("%q: want decode error, got nil", body)
}
}
}
func TestFormGetHas(t *testing.T) {
var f Form
if err := f.ParseBytes([]byte("tag=a&tag=b&ok&empty=")); err != nil {
t.Fatal(err)
}
if got := string(f.Get("tag")); got != "a" {
t.Errorf("want first value %q, got %q", "a", got)
}
if got := f.Get("ok"); got != nil {
t.Errorf("want nil value for valueless key, got %q", got)
}
if got := f.Get("nope"); got != nil {
t.Errorf("want nil for absent key, got %q", got)
}
if v := f.Get("empty"); v == nil || len(v) != 0 {
t.Errorf("want present empty value, got %v", v)
}
for _, key := range []string{"tag", "ok", "empty"} {
if !f.Has(key) {
t.Errorf("want Has(%q) true", key)
}
}
if f.Has("nope") {
t.Error("want Has(nope) false")
}
}
func TestFormAppendKeyValues(t *testing.T) {
const body = "name=go&ok&empty=&tag=a&tag=b"
var f Form
if err := f.ParseBytes([]byte(body)); err != nil {
t.Fatal(err)
}
if got := string(f.AppendKeyValues(nil)); got != body {
t.Errorf("want round trip %q, got %q", body, got)
}
}
// Parsing into a reused Form must not allocate: the pair storage is reused.
func TestFormParseReuseNoAlloc(t *testing.T) {
body := []byte("name=go&tag=a&tag=b&ok")
var f Form
if err := f.ParseBytes(body); err != nil { // Warm up the pair storage.
t.Fatal(err)
}
allocs := testing.AllocsPerRun(100, func() {
f.Reset(body)
f.Parse()
})
if allocs != 0 {
t.Errorf("reused Form allocated %v times, want 0", allocs)
}
}
+282 -69
View File
@@ -4,31 +4,38 @@ import (
"bytes"
"io"
"slices"
"unsafe"
"strconv"
)
const (
methodGet = "GET"
strHTTP11 = "HTTP/1.1"
strCRLF = "\r\n"
headerCookie = "Cookie"
headerConnection = "Connection"
strClose = "close"
methodGet = "GET"
strHTTP11 = "HTTP/1.1"
strCRLF = "\r\n"
headerCookie = "Cookie"
headerConnection = "Connection"
headerContentLength = "Content-Length"
strClose = "close"
)
type flags uint16
// Flags is a bitset of signals gathered while parsing or building a header,
// such as a status code having been set or the peer requesting connection
// close. See [Header.Flags].
type Flags uint16
const (
flagNoBufferGrow flags = 1 << iota
flagNoBufferGrow Flags = 1 << iota
flagDoneParsingHeader
flagOOMReached
flagConnClose
flagNoHTTP11
flagMangledBuffer // set when header fields appended to buffer via Add,Set calls
flagReaderEOF
// set if [Header.SetStatus] or [Header.SetStatusInt] has been called.
FlagStatusSet
)
func (f flags) hasAny(checkThese flags) bool {
// HasAny returns true if any of the argument flags are set.
func (f Flags) HasAny(checkThese Flags) bool {
return f&checkThese != 0
}
@@ -43,22 +50,27 @@ type Header struct {
hbuf headerBuf
// Request fields.
method headerSlice
requestURI headerSlice
proto headerSlice
method headerSlice
requestTarget headerSlice
proto headerSlice
// Response fields.
statusCode headerSlice
statusText headerSlice
flags flags
flags Flags
_ noCopy
}
// EnableBufferGrowth disables buffer growth during parsing if b is false. Is enabled by default.
// Disabling buffer growth prevents allocations but methods may throw errors on insufficient memory.
func (h *Header) EnableBufferGrowth(b bool) {
if !b {
// Flags returns [Flags] to signal status code has been set, Connection:Close or other useful signals provided by flags.
func (h *Header) Flags() Flags { return h.flags }
// ConfigBufferGrowth configures the memory the header may use. Setting
// outlives [Header.Reset]. Call before parsing/reading.
//
// enableBufferGrowth enables growing both the header buffer and the header key/value pair slice.
func (h *Header) ConfigBufferGrowth(enableBufferGrowth bool) {
if !enableBufferGrowth {
h.flags |= flagNoBufferGrow
} else {
h.flags &^= flagNoBufferGrow
@@ -67,7 +79,7 @@ func (h *Header) EnableBufferGrowth(b bool) {
// ParseBytes copies the bytes into buffer and parses the HTTP header. It fails if HTTP header data is incomplete.
func (h *Header) ParseBytes(asResponse bool, b []byte) error {
h.Reset(nil)
h.Reset(nil, 0)
h.hbuf.readFromBytes(b)
return h.parse(asResponse)
}
@@ -76,7 +88,7 @@ func (h *Header) ParseBytes(asResponse bool, b []byte) error {
// It fails if HTTP data is incomplete.
func (h *Header) Parse(asResponse bool) error {
debuglog("http:parse:reset")
h.Reset(h.hbuf.buf)
h.Reset(h.hbuf.buf, 0)
debuglog("http:parse:start")
return h.parse(asResponse)
}
@@ -98,24 +110,24 @@ func (h *Header) Parse(asResponse bool) error {
// return err
// }
func (h *Header) TryParse(asResponse bool) (needMoreData bool, err error) {
if h.flags.hasAny(flagDoneParsingHeader) {
if h.flags.HasAny(flagDoneParsingHeader) {
return false, errAlreadyParsed
} else if h.flags.hasAny(flagMangledBuffer) {
} else if h.flags.HasAny(flagMangledBuffer) {
return false, errMangledBuffer
}
if asResponse && h.statusCode.len == 0 || !asResponse && h.requestURI.start == 0 {
if asResponse && h.statusCode.len == 0 || !asResponse && h.requestTarget.start == 0 {
err = h.parseFirstLine(asResponse)
if err != nil {
return err == errNeedMore, err
return err == ErrNeedMoreData, err
}
}
err = h.parseNextHeaders()
return err == errNeedMore, err
err = h.parseNextHeaders(h.flags)
return err == ErrNeedMoreData, err
}
// ParsingSuccess returns true if TryParse was successful, that is to say it returned needMoreData==false and err==nil.
func (h *Header) ParsingSuccess() bool {
return h.flags.hasAny(flagDoneParsingHeader)
return h.flags.HasAny(flagDoneParsingHeader)
}
// ReadFromLimited reads at most maxBytesToRead from reader and appends them to underlying buffer.
@@ -123,19 +135,24 @@ func (h *Header) ParsingSuccess() bool {
// If read is successful (read length>0) and reader returns [io.EOF] then ReadFromLimited will return a nil error.
func (h *Header) ReadFromLimited(r io.Reader, maxBytesToRead int) (int, error) {
if maxBytesToRead <= 0 {
return 0, errSmallBuffer
} else if h.flags.hasAny(flagMangledBuffer) {
return 0, ErrSmallHeaderBuffer
} else if h.flags.HasAny(flagMangledBuffer) {
return 0, errMangledBuffer
} else if h.flags.HasAny(flagReaderEOF) {
return 0, io.EOF // Now we do return EOF.
}
free := h.BufferFree()
if free < maxBytesToRead {
if h.flags.hasAny(flagNoBufferGrow) {
return 0, errSmallBuffer
if h.flags.HasAny(flagNoBufferGrow) {
return 0, ErrSmallHeaderBuffer
}
h.hbuf.buf = slices.Grow(h.hbuf.buf, maxBytesToRead)
}
blen := len(h.hbuf.buf)
b := h.hbuf.buf[blen:min(blen+maxBytesToRead, cap(h.hbuf.buf))]
if len(b) == 0 {
return 0, ErrSmallHeaderBuffer
}
n, err := r.Read(b)
if err != nil && err == io.EOF {
h.flags |= flagReaderEOF
@@ -151,12 +168,12 @@ func (h *Header) ReadFromLimited(r io.Reader, maxBytesToRead int) (int, error) {
// Used to accumulate HTTP header for later parsing with [Header.TryParse].
func (h *Header) ReadFromBytes(b []byte) (int, error) {
if len(b) == 0 {
return 0, errSmallBuffer
return 0, ErrSmallHeaderBuffer
}
free := h.BufferFree()
if free < len(b) {
if h.flags.hasAny(flagNoBufferGrow) {
return 0, errSmallBuffer
if h.flags.HasAny(flagNoBufferGrow) {
return 0, ErrSmallHeaderBuffer
}
h.hbuf.buf = slices.Grow(h.hbuf.buf, len(b))
}
@@ -165,8 +182,9 @@ func (h *Header) ReadFromBytes(b []byte) (int, error) {
}
// BufferReceived returns the amoung of bytes read during calls to Read* methods.
// Returns 0 if buffer is invalid/mangled.
func (h *Header) BufferReceived() int {
if h.flags.hasAny(flagMangledBuffer | flagOOMReached) {
if h.flags.HasAny(flagMangledBuffer | flagOOMReached) {
return 0
}
return len(h.hbuf.buf)
@@ -176,18 +194,33 @@ func (h *Header) BufferReceived() int {
// If the Parse* method completed without error then BufferParsed returns the header's length including the final "\r\n\r\n" text.
// BufferParsed returns 0 if the buffer is invalid/mangled or if no header data has been parsed succesfully.
func (h *Header) BufferParsed() int {
if h.flags.hasAny(flagMangledBuffer | flagOOMReached) {
if h.flags.HasAny(flagMangledBuffer | flagOOMReached) {
return 0
}
return h.hbuf.off
}
// BufferRaw returns the undeerlying buffer as stored currently in memory.
// The length of the returned buffer is the used portion. Capacity of returned slice is [Header.BufferCapacity].
func (h *Header) BufferRaw() []byte { return h.hbuf.buf }
// BufferUsed returns the raw memory used.
//
// BufferUsed + BufferFree == BufferCapacity
func (h *Header) BufferUsed() int {
return len(h.hbuf.buf)
}
// BufferFree returns amount of bytes free in underlying buffer.
//
// BufferUsed + BufferFree == BufferCapacity
func (h *Header) BufferFree() int {
return h.hbuf.free()
}
// BufferCapacity returns the total capacity of the underlying buffer.
//
// BufferUsed + BufferFree == BufferCapacity
func (h *Header) BufferCapacity() int {
return cap(h.hbuf.buf)
}
@@ -199,7 +232,7 @@ func (h *Header) ForEach(cb func(key, value []byte) error) error {
func (hb *headerBuf) forEach(cb func(key, value []byte) error) error {
nh := len(hb.headers)
for i := 0; i < nh; i++ {
for i := range nh {
kv := hb.headers[i]
if !kv.isValid() {
continue
@@ -219,16 +252,16 @@ func (hb *headerBuf) forEach(cb func(key, value []byte) error) error {
// data with Reset to parse data in-place.
// If buf is nil then the current buffer is reused. There are 3 ways to use Reset:
//
// h.Reset(prealloc[:0]); h.ParseBytes(httpHeader) // Tell header to use a pre-allocated buffer capacity.
// h.Reset(httpHeader); h.Parse() // Parse bytes in place with no copying.
// h.Reset(prealloc[:0], 16); h.ParseBytes(httpHeader) // Tell header to use a pre-allocated buffer capacity.
// h.Reset(httpHeader, 16); h.Parse() // Parse bytes in place with no copying.
// h.Reset(nil) // Reuse buffer previously set in a call to Reset.
func (h *Header) Reset(buf []byte) {
if h.flags.hasAny(flagNoBufferGrow) && len(buf) < 32 {
panic("small buffer and flagNoBufferGrow set")
}
func (h *Header) Reset(buf []byte, numHeaderCapacity int) {
const persistentFlags = flagNoBufferGrow
debuglog("http:reset:hbuf")
h.hbuf.reset(buf)
h.hbuf.reset(buf, numHeaderCapacity)
if h.flags.HasAny(flagNoBufferGrow) && cap(h.hbuf.buf) < 32 {
panic("small buffer and flagNoBufferGrow set")
}
*h = Header{
hbuf: h.hbuf,
flags: h.flags & persistentFlags,
@@ -239,9 +272,9 @@ func (h *Header) Reset(buf []byte) {
// Body returns the surplus data following headers. It is only valid as long as Parse* or Reset methods are not called.
func (h *Header) Body() ([]byte, error) {
debuglog("http:body")
if h.flags.hasAny(flagMangledBuffer) {
if h.flags.HasAny(flagMangledBuffer) {
return nil, errMangledBuffer
} else if h.flags.hasAny(flagDoneParsingHeader) {
} else if h.flags.HasAny(flagDoneParsingHeader) {
return h.hbuf.buf[h.hbuf.off:], nil
}
return nil, errUnparsed
@@ -250,18 +283,45 @@ func (h *Header) Body() ([]byte, error) {
// SetBytes is equivalent to [Header.Set] but with a []byte value. Does not keep reference to value slice.
// Calling SetBytes Mangles the buffer.
func (h *Header) SetBytes(key string, value []byte) {
h.Set(key, unsafe.String(&value[0], len(value)))
h.Set(key, b2s(value))
}
// SetInt is equivalent to [Header.Set] but with an integer value i.e: Content-Length header key.
// base must be in the range 2..36 (as accepted by [strconv.AppendInt]); other bases are dropped.
// SetInt formats the value directly into the header buffer without heap allocation.
func (h *Header) SetInt(key string, value int64, base int) {
if base < 2 || base > 36 {
return // strconv.AppendInt only supports base 2..36.
}
useKv := h.takeReusableSlot(key)
if useKv == nil {
h.appendHeaderInt(key, value, base)
} else {
useKv.value = h.reuseOrAppendInt(useKv.value, value, base)
}
}
// Set sets a key-value pair in the HTTP header.
// Calling Set mangles the buffer.
func (h *Header) Set(key, value string) {
useKv := h.takeReusableSlot(key)
if useKv == nil {
h.appendHeader(key, value)
} else {
useKv.value = h.reuseOrAppend(useKv.value, value)
}
}
// takeReusableSlot returns the valid key-value entry for key with the largest
// value buffer (best candidate for in-place reuse) and invalidates any other
// entries sharing the key. Returns nil if the key is not present.
func (h *Header) takeReusableSlot(key string) *argsKV {
hb := &h.hbuf
var useKv *argsKV
for i := len(hb.headers); len(hb.headers) > 0 && i <= 0; i++ {
for i := 0; i < len(hb.headers); i++ {
// Search for key-value with largest buffer for value to store value reusing buffer.
gotkv := &hb.headers[i]
if b2s(hb.musttoken(gotkv.key)) == key {
if gotkv.isValid() && b2s(hb.musttoken(gotkv.key)) == key {
if useKv == nil {
useKv = gotkv
} else if gotkv.value.len > useKv.value.len {
@@ -272,11 +332,7 @@ func (h *Header) Set(key, value string) {
}
}
}
if useKv == nil {
h.appendHeader(key, value)
} else {
useKv.value = h.reuseOrAppend(useKv.value, value)
}
return useKv
}
// Get gets the first value of a key found in the headers. Use [Header.ForEach] to find multiple values corresponding to same key.
@@ -291,6 +347,28 @@ func (h *Header) Get(key string) []byte {
return nil
}
// ContentLength returns the body length declared by the Content-Length field.
// Fails with an error if the field is absent, which for a request means the
// message has no body at all unless a transfer coding applies, RFC 9112 6.3.
// The value must be digits only, so a negative or list-valued field is rejected
// rather than guessed at.
func (h *Header) ContentLength() (int64, error) {
kv := h.peekHeader(headerContentLength)
if !kv.isValid() {
return 0, errNoContentLength
}
value := trimOWS(h.hbuf.musttoken(kv.value))
if len(value) == 0 {
return 0, errBadContentLength
}
// Unsigned parse of 63 bits rejects a sign and anything past int64's range.
n, err := strconv.ParseUint(b2s(value), 10, 63)
if err != nil {
return 0, errBadContentLength // strconv's error allocates and is not comparable.
}
return int64(n), nil
}
// Add adds a new key-value pair to the HTTP header. Calling Add mangles the buffer.
func (h *Header) Add(key, value string) {
h.appendHeader(key, value)
@@ -306,14 +384,71 @@ func (h *Header) SetMethod(method string) {
h.method = h.reuseOrAppend(h.method, method)
}
// SetRequestURI sets RequestURI for the first HTTP request line.
func (h *Header) SetRequestURI(requestURI string) {
h.requestURI = h.reuseOrAppend(h.requestURI, requestURI)
// SetRequestTarget sets request-target (URI) for the first HTTP request line.
func (h *Header) SetRequestTarget(requestTarget string) {
h.requestTarget = h.reuseOrAppend(h.requestTarget, requestTarget)
}
// RequestURI returns RequestURI from the first HTTP request line.
func (h *Header) RequestURI() []byte {
return h.getNonEmptyValue(h.requestURI)
// RequestTarget returns a view of the request-target (URI) of the first HTTP request line.
// Called Request-URI in the obsolete RFC 2616, renamed request-target by RFC 9112.
func (h *Header) RequestTarget() []byte {
return h.getNonEmptyValue(h.requestTarget)
}
// RequestPath returns the request-target (URI) up to the query string, i.e: "/search"
// for "/search?q=go". Returns the whole target if it contains no query string.
func (h *Header) RequestPath() []byte {
target := h.RequestTarget()
before, _, ok := bytes.Cut(target, []byte{'?'})
if !ok {
return target
}
return before
}
// RequestQuery returns the request-target (URI) query string as it appears on the
// wire, percent-encoded and with '+' undecoded, i.e: "q=go" for "/search?q=go".
// Returns nil if the target has no query string. Iterate it with [NextQueryPair].
func (h *Header) RequestQuery() []byte {
target := h.RequestTarget()
_, after, ok := bytes.Cut(target, []byte{'?'})
if !ok {
return nil
}
return after
}
// NextQueryPair splits the leading key-value pair off a query string and returns
// what remains of it. Loop until rawkey is nil:
//
// rawkey, rawval, rest := httpraw.NextQueryPair(h.RequestQuery())
// for rawkey != nil {
// // use rawkey, rawval.
// rawkey, rawval, rest = httpraw.NextQueryPair(rest)
// }
//
// A pair with no '=' yields a nil rawval, i.e: "debug" in "?debug&q=go", which
// distinguishes it from "?debug=" where the value is present and empty. Empty
// sequences are skipped, so "?&&q=go&" yields a single pair. Only '&' separates
// pairs and only the first '=' splits a pair.
func NextQueryPair(query []byte) (rawkey, rawval, rest []byte) {
for len(query) > 0 {
pair := query
amp := bytes.IndexByte(query, '&')
if amp >= 0 {
pair, query = query[:amp], query[amp+1:]
} else {
query = nil
}
if len(pair) == 0 {
continue // Empty sequence, see WHATWG URL urlencoded parsing.
}
if before, after, ok := bytes.Cut(pair, []byte{'='}); ok {
return before, after, query
}
return pair, nil, query
}
return nil, nil, nil
}
// Protocol returns the request header's HTTP protocol. Usually "HTTP/1.1".
@@ -334,12 +469,20 @@ func (h *Header) Status() (code, statusText []byte) {
return h.hbuf.musttoken(h.statusCode), h.hbuf.musttoken(h.statusText)
}
// Status sets the response header's status code and status text. i.e: "200" "OK".
// SetStatus sets the response header's status code and status text. i.e: "200" "OK".
func (h *Header) SetStatus(code, statusText string) {
h.flags |= FlagStatusSet
h.statusCode = h.reuseOrAppend(h.statusCode, code)
h.statusText = h.reuseOrAppend(h.statusText, statusText)
}
// SetStatusInt is identical to [Header.SetStatus] but performs integer to text conversion for status code.
func (h *Header) SetStatusInt(code int64, statusText string) {
h.flags |= FlagStatusSet
h.statusCode = h.reuseOrAppendInt(h.statusCode, code, 10)
h.statusText = h.reuseOrAppend(h.statusText, statusText)
}
func (h *Header) getNonEmptyValue(s headerSlice) []byte {
if s.len == 0 {
return nil // If empty then value is invalid, return nil.
@@ -350,9 +493,9 @@ func (h *Header) getNonEmptyValue(s headerSlice) []byte {
// AppendRequest appends the request header representation to the buffer and returns the result.
func (h *Header) AppendRequest(dst []byte) ([]byte, error) {
proto := h.Protocol()
if h.flags.hasAny(flagOOMReached) {
if h.flags.HasAny(flagOOMReached) {
return dst, errOOM
} else if h.requestURI.len == 0 || h.method.len == 0 {
} else if h.requestTarget.len == 0 || h.method.len == 0 {
return dst, errNeedMethodURI
} else if len(proto) == 0 {
return dst, errNoProto
@@ -364,7 +507,7 @@ func (h *Header) AppendRequest(dst []byte) ([]byte, error) {
} else {
dst = append(dst, method...)
}
uri := h.RequestURI()
uri := h.RequestTarget()
dst = append(dst, ' ')
dst = append(dst, uri...)
@@ -379,8 +522,18 @@ func (h *Header) AppendRequest(dst []byte) ([]byte, error) {
// AppendResponse appends the response header representation to the buffer and returns the result.
func (h *Header) AppendResponse(dst []byte) ([]byte, error) {
dst, err := h.AppendResponseNoHeaders(dst)
if err != nil {
return dst, err
}
dst = h.AppendHeaders(dst)
return append(dst, strCRLF...), nil
}
// AppendResponseNoHeaders appends the first line of the response containing protocol and status code/text: i.e: "HTTP/1.1 200 OK\r\n"
func (h *Header) AppendResponseNoHeaders(dst []byte) ([]byte, error) {
proto := h.Protocol()
if h.flags.hasAny(flagOOMReached) {
if h.flags.HasAny(flagOOMReached) {
return dst, errOOM
} else if h.statusCode.len == 0 || h.statusText.len == 0 {
return dst, errBadStatusCodeTxt
@@ -395,10 +548,7 @@ func (h *Header) AppendResponse(dst []byte) ([]byte, error) {
dst = append(dst, ' ')
dst = append(dst, text...)
dst = append(dst, strCRLF...)
dst = h.AppendHeaders(dst)
return append(dst, strCRLF...), nil
return dst, nil
}
// AppendHeaders appends headers to buffer. Use AppendRequest and AppendResponse over this.
@@ -415,6 +565,9 @@ func (h *Header) AppendHeaders(dst []byte) []byte {
return dst
}
// String returns the header's wire representation, as a request if it has a
// request line and as a response otherwise. Returns the error text if neither
// can be built. Allocates, so it is meant for debugging and logging only.
func (h *Header) String() string {
buf, err := h.AppendRequest(nil)
if err != nil {
@@ -506,3 +659,63 @@ func CopyNormalizedHeaderValue(dst []byte, value []byte) (n int, modified bool)
}
return write, modified
}
// CopyDecodedPercentURL decodes percent-escapes in value into dst and returns bytes written.
// n < len(value) implies percent-escapes were decoded; the converse does not hold since
// '+' substitution preserves length. If plusAsSpace is set '+' decodes to ' ',
// which is correct for query and form-encoded data but NOT for path segments.
// On malformed escape returns n bytes written before the fault and a non-nil error.
// dst and value may only alias if &dst[0] == &value[0].
func CopyDecodedPercentURL(dst, value []byte, plusAsSpace bool) (n int, err error) {
if len(dst) < len(value) {
panic("httpraw.CopyDecodedPercentURL: dst buffer shorter than value")
}
read := 0
for {
escape := bytes.IndexByte(value[read:], '%')
if escape < 0 {
n += copyPlusDecoded(dst[n:], value[read:], plusAsSpace)
return n, nil
}
escape += read
n += copyPlusDecoded(dst[n:], value[read:escape], plusAsSpace)
if escape+2 >= len(value) {
return n, errBadPercentEncode // Truncated escape at end of value.
}
hi, okhi := unhexdigit(value[escape+1])
lo, oklo := unhexdigit(value[escape+2])
if !okhi || !oklo {
return n, errBadPercentEncode
}
// Write index n is always <= escape since decoding shrinks 3 bytes to 1,
// so writing here never clobbers an unread byte when dst aliases value.
dst[n] = hi<<4 | lo
n++
read = escape + 3
}
}
// copyPlusDecoded copies src to dst replacing '+' with ' ' if plusAsSpace set.
func copyPlusDecoded(dst, src []byte, plusAsSpace bool) int {
n := copy(dst, src)
if plusAsSpace {
for i := range n {
if dst[i] == '+' {
dst[i] = ' '
}
}
}
return n
}
func unhexdigit(c byte) (byte, bool) {
switch {
case c >= '0' && c <= '9':
return c - '0', true
case c >= 'a' && c <= 'f':
return c - 'a' + 10, true
case c >= 'A' && c <= 'F':
return c - 'A' + 10, true
}
return 0, false
}
+449 -7
View File
@@ -2,6 +2,7 @@ package httpraw
import (
"bytes"
"errors"
"fmt"
"net/http"
"strconv"
@@ -10,6 +11,8 @@ import (
"time"
)
const numHeaderCapacity = 16
func TestHeaderParseRequest(t *testing.T) {
const (
wantMethod = "GET"
@@ -50,8 +53,8 @@ func TestHeaderParseRequest(t *testing.T) {
if string(hdr.Method()) != wantMethod {
t.Errorf("want method %s, got %q", wantMethod, hdr.Method())
}
if !bytes.Equal(hdr.RequestURI(), []byte(wantURI)) {
t.Errorf("want request URI %q, got %q", wantURI, hdr.RequestURI())
if !bytes.Equal(hdr.RequestTarget(), []byte(wantURI)) {
t.Errorf("want request URI %q, got %q", wantURI, hdr.RequestTarget())
}
contentLength, _ := strconv.Atoi(string(hdr.Get("Content-Length")))
if contentLength != len(wantMessage) {
@@ -112,13 +115,19 @@ func BenchmarkParseBytes(b *testing.B) {
asRequest = false
)
req, _ := http.NewRequest(wantMethod, wantURI, strings.NewReader(wantMessage))
var buf bytes.Buffer
req.Write(&buf)
data := buf.Bytes()
var rawBuf bytes.Buffer
req.Write(&rawBuf)
data := rawBuf.Bytes()
// hdr is declared outside the loop so that ParseBytes can reuse the
// backing arrays on Reset (headers slice and data buffer) without
// allocating on every iteration. Declaring it inside the loop causes
// two allocs per iteration: one for the headers slice (make in reset)
// and one for the data buffer (append in readFromBytes).
var hdr Header
b.StartTimer()
for i := 0; i < b.N; i++ {
var hdr Header
for b.Loop() {
err := hdr.ParseBytes(asRequest, data)
if err != nil {
b.Fatal(err)
@@ -145,6 +154,121 @@ func strSameSite(mode http.SameSite) string {
}
}
func TestHeaderRequestPath(t *testing.T) {
for _, test := range []struct {
uri string
want string
}{
{uri: "/", want: "/"},
{uri: "/search?q=go", want: "/search"},
{uri: "/search?", want: "/search"},
{uri: "/a/b/c?x=1&y=2", want: "/a/b/c"},
{uri: "/?q=go", want: "/"},
} {
var h Header
err := h.ParseBytes(false, []byte("GET "+test.uri+" HTTP/1.1\r\nHost: h\r\n\r\n"))
if err != nil {
t.Fatal(err)
}
if got := string(h.RequestPath()); got != test.want {
t.Errorf("uri %q: want path %q, got %q", test.uri, test.want, got)
}
}
}
func TestHeaderContentLength(t *testing.T) {
for _, test := range []struct {
field string // Extra header lines, empty for an absent field.
want int64
wantErr error
}{
{field: "Content-Length: 0", want: 0},
{field: "Content-Length: 12", want: 12},
{field: "Content-Length: 12 ", want: 12}, // OWS around the value, RFC 9110 5.6.3.
{field: "Content-Length: 9223372036854775807", want: 9223372036854775807},
{field: "", wantErr: errNoContentLength}, // No body, RFC 9112 6.3.
{field: "Content-Length:", wantErr: errBadContentLength}, // Present but empty.
{field: "Content-Length: -1", wantErr: errBadContentLength}, // Digits only, RFC 9112 6.2.
{field: "Content-Length: 1 2", wantErr: errBadContentLength}, // Not a list.
{field: "Content-Length: 9223372036854775808", wantErr: errBadContentLength},
} {
var h Header
raw := "POST / HTTP/1.1\r\nHost: h\r\n"
if test.field != "" {
raw += test.field + "\r\n"
}
if err := h.ParseBytes(false, []byte(raw+"\r\n")); err != nil {
t.Fatal(err)
}
got, err := h.ContentLength()
if err != test.wantErr {
t.Errorf("%q: want error %v, got %v", test.field, test.wantErr, err)
} else if err == nil && got != test.want {
t.Errorf("%q: want %d, got %d", test.field, test.want, got)
}
}
}
func TestNextQueryPair(t *testing.T) {
for _, test := range []struct {
uri string
want string // "key=value" pairs joined by '|'; nil value shown as "key".
}{
{uri: "/", want: ""},
{uri: "/x?", want: ""},
{uri: "/x?q=go", want: "q=go"},
{uri: "/x?q=go&n=1", want: "q=go|n=1"},
{uri: "/x?debug&q=go", want: "debug|q=go"}, // No '=' yields a nil value.
{uri: "/x?q=", want: "q="}, // Empty but present value.
{uri: "/x?&&q=go&", want: "q=go"}, // Empty sequences skipped.
{uri: "/x?=v", want: "=v"}, // Empty name is kept.
{uri: "/x?a=1&a=2", want: "a=1|a=2"}, // Duplicates all yielded.
{uri: "/x?a%20b=c%20d", want: "a%20b=c%20d"}, // Raw, undecoded.
{uri: "/x?a=b=c", want: "a=b=c"}, // Only first '=' splits.
} {
var h Header
err := h.ParseBytes(false, []byte("GET "+test.uri+" HTTP/1.1\r\nHost: h\r\n\r\n"))
if err != nil {
t.Fatal(err)
}
var got []byte
rawkey, rawval, rest := NextQueryPair(h.RequestQuery())
for rawkey != nil {
if len(got) > 0 {
got = append(got, '|')
}
got = append(got, rawkey...)
if rawval != nil {
got = append(got, '=')
got = append(got, rawval...)
}
rawkey, rawval, rest = NextQueryPair(rest)
}
if string(got) != test.want {
t.Errorf("uri %q: want %q, got %q", test.uri, test.want, got)
}
}
}
// A nil key ends iteration, and stopping early is just not looping again.
func TestNextQueryPairEnd(t *testing.T) {
rawkey, rawval, rest := NextQueryPair([]byte("a=1&b=2"))
if string(rawkey) != "a" || string(rawval) != "1" || string(rest) != "b=2" {
t.Fatalf("want a=1 with rest b=2, got %q=%q rest %q", rawkey, rawval, rest)
}
rawkey, _, rest = NextQueryPair(rest)
if string(rawkey) != "b" || len(rest) != 0 {
t.Fatalf("want b with empty rest, got %q rest %q", rawkey, rest)
}
if rawkey, _, _ = NextQueryPair(rest); rawkey != nil {
t.Fatalf("want nil key at end of query, got %q", rawkey)
}
// Trailing separators yield no pair rather than an empty one.
if rawkey, _, _ = NextQueryPair([]byte("&&")); rawkey != nil {
t.Fatalf("want nil key for empty sequences, got %q", rawkey)
}
}
func TestHeaderNormalizeKey(t *testing.T) {
var tests = []struct {
key string
@@ -202,3 +326,321 @@ func TestCopyNormalizedHeaderValue(t *testing.T) {
}
}
}
func TestCopyDecodedPercentURL(t *testing.T) {
var tests = []struct {
value string
plusAsSpace bool
want string
wantErr bool
}{
{value: "", want: ""},
{value: "/plain/path", want: "/plain/path"},
{value: "/a%20b", want: "/a b"},
{value: "%41%42%43", want: "ABC"},
{value: "%2f%2F", want: "//"}, // lower and upper case hex digits.
{value: "%25", want: "%"}, // escaped percent must not re-trigger decoding.
{value: "%2525", want: "%25"}, // decoded output is not re-scanned.
{value: "a+b", want: "a+b"}, // plus is literal in path segments.
{value: "a+b", plusAsSpace: true, want: "a b"},
{value: "%20+%20", plusAsSpace: true, want: " "},
{value: "/x?q=%E2%82%AC", want: "/x?q=\xe2\x82\xac"}, // multi-byte UTF-8 sequence.
// Malformed escapes must error, never pass through silently.
{value: "%zz", want: "", wantErr: true},
{value: "ok%4", want: "ok", wantErr: true}, // truncated escape at end.
{value: "ok%", want: "ok", wantErr: true}, // bare percent at end.
{value: "a%2gb", want: "a", wantErr: true}, // second digit not hex.
{value: "a%g2b", want: "a", wantErr: true}, // first digit not hex.
}
dst := make([]byte, 256)
for _, test := range tests {
value := []byte(test.value)
n, err := CopyDecodedPercentURL(dst[:len(value)], value, test.plusAsSpace)
if test.wantErr && err == nil {
t.Errorf("%q: want error, got nil", test.value)
} else if !test.wantErr && err != nil {
t.Errorf("%q: unexpected error %s", test.value, err)
}
if got := string(dst[:n]); got != test.want {
t.Errorf("%q: want %q got %q", test.value, test.want, got)
}
// n<len(value) implies percent-escapes were decoded. The converse does not
// hold: '+'->' ' substitution preserves length.
if !test.wantErr && n < len(test.value) && test.want == test.value {
t.Errorf("%q: n=%d signals decoding but value unchanged", test.value, n)
}
if !test.wantErr && !test.plusAsSpace && test.want != test.value && n >= len(test.value) {
t.Errorf("%q: n=%d does not signal decoding of %q", test.value, n, test.want)
}
}
}
// In-place decoding (dst aliasing value at offset 0) must yield the same result.
func TestCopyDecodedPercentURLInPlace(t *testing.T) {
const value, want = "/a%20b%2Fc%25", "/a b/c%"
buf := []byte(value)
n, err := CopyDecodedPercentURL(buf, buf, false)
if err != nil {
t.Fatal(err)
}
if got := string(buf[:n]); got != want {
t.Fatalf("want %q got %q", want, got)
}
}
func TestHeaderSetOverwrite(t *testing.T) {
var h Header
h.Reset(nil, numHeaderCapacity)
h.SetMethod("GET")
h.SetRequestTarget("/")
h.SetProtocol("HTTP/1.1")
h.Set("Host", "first.example.com")
h.Set("Host", "second.example.com")
got := string(h.Get("Host"))
if got != "second.example.com" {
t.Errorf("want Host %q, got %q", "second.example.com", got)
}
req, err := h.AppendRequest(nil)
if err != nil {
t.Fatal(err)
}
if n := strings.Count(string(req), "Host:"); n != 1 {
t.Errorf("want 1 Host field in request, got %d:\n%s", n, req)
}
}
func TestHeaderSetBytesEmptyValue(t *testing.T) {
var h Header
h.Reset(nil, numHeaderCapacity)
h.SetBytes("X-Empty", nil)
if got := h.Get("X-Empty"); len(got) != 0 {
t.Errorf("want empty value, got %q", got)
}
}
// buffer/offset past uint16 range silently corrupts or panics.
// A header block > 64KiB pushes a field's offset past 65535. tokint truncation
// makes Get return the wrong window (silent corruption) or panic on slice bounds.
func TestHeader_LargeBufferOverflow(t *testing.T) {
const wantVal = "the-canary-value"
// Pad with a big header so the canary field lands past offset 65535.
pad := strings.Repeat("x", 70000)
raw := "GET / HTTP/1.1\r\n" +
"X-Pad: " + pad + "\r\n" +
"X-Canary: " + wantVal + "\r\n" +
"\r\n"
var h Header
err := h.ParseBytes(false, []byte(raw))
if err != nil {
// Clean rejection of the oversized header is the intended behavior:
// no panic, no silent corruption.
return
}
// If it did parse, the value must be correct (never a truncated-offset window).
got := string(h.Get("X-Canary"))
if got != wantVal {
t.Fatalf("overflow corruption: want X-Canary %q, got %q", wantVal, got)
}
}
// a complete but malformed header line with no colon must be a hard error,
// not ErrNeedMoreData (which makes a streaming parser wait forever).
func TestHeader_ColonlessLineIsHardError(t *testing.T) {
raw := "GET / HTTP/1.1\r\nBadHeaderNoColon\r\n\r\n"
var h Header
err := h.ParseBytes(false, []byte(raw))
if err == nil {
t.Fatal("want error on colonless header line, got nil")
}
if err == ErrNeedMoreData {
t.Fatalf("colonless line reported as ErrNeedMoreData (parser would hang); want a hard error like errInvalidName")
}
}
// Regression for a TCP split landing BEFORE the colon (no newline yet) must
// stay "need more data" and parse once the rest arrives. This is the case the
// Colonless fix must NOT turn into a hard error.
func TestHeader_SplitBeforeColonStillParses(t *testing.T) {
const part1 = "GET / HTTP/1.1\r\nHost" // split mid-key, before colon+newline
const part2 = ": example.com\r\n\r\n"
var h Header
h.Reset(nil, numHeaderCapacity)
if _, err := h.ReadFromBytes([]byte(part1)); err != nil {
t.Fatal(err)
}
needMore, err := h.TryParse(false)
if err != nil && err != ErrNeedMoreData {
t.Fatalf("split before colon: want ErrNeedMoreData/nil, got %v", err)
}
if !needMore {
t.Fatal("want needMoreData=true after partial input")
}
if _, err := h.ReadFromBytes([]byte(part2)); err != nil {
t.Fatal(err)
}
needMore, err = h.TryParse(false)
if err != nil {
t.Fatalf("after full input: %v", err)
}
if needMore {
t.Fatal("want needMoreData=false after full input")
}
if got := string(h.Get("Host")); got != "example.com" {
t.Fatalf("want Host %q, got %q", "example.com", got)
}
}
// appendHeader must reserve the +1 byte that mustAppendSlice consumes on an
// empty buffer. Force cap == len(key)+len(value) to defeat allocator rounding.
func TestHeader_AppendHeaderExactCapNoPanic(t *testing.T) {
const key, value = "K", "V"
buf := make([]byte, 0, len(key)+len(value)) // exact cap, no slack.
var h Header
h.Reset(buf, numHeaderCapacity)
defer func() {
if r := recover(); r != nil {
t.Fatalf("appendHeader panicked on exact-cap buffer: %v", r)
}
}()
h.Add(key, value)
if got := string(h.Get(key)); got != value {
t.Fatalf("want %q, got %q", value, got)
}
}
// Add/Set on a full buffer with growth disabled must drop gracefully (flag OOM),
// never panic. Panicking is unacceptable for this package.
func TestHeader_AddFullBufferNoPanic(t *testing.T) {
buf := make([]byte, 0, 40) // Small cap; enough for Reset (len 0) but not the field below.
var h Header
h.Reset(buf, numHeaderCapacity)
h.ConfigBufferGrowth(false)
h.SetMethod("GET")
h.SetRequestTarget("/")
h.SetProtocol("HTTP/1.1")
defer func() {
if r := recover(); r != nil {
t.Fatalf("Add on full no-grow buffer panicked: %v", r)
}
}()
h.Add("X-Very-Long-Header-Key", "a-value-that-cannot-possibly-fit-in-the-buffer")
// Graceful drop: request marshalling reports OOM instead of emitting bad data.
if _, err := h.AppendRequest(nil); err == nil {
t.Fatal("want OOM error after dropped Add, got nil")
}
}
func TestHeader_SetInt(t *testing.T) {
for _, tc := range []struct {
name string
value int64
base int
want string
}{
{"decimal", 1234, 10, "1234"},
{"zero", 0, 10, "0"},
{"negative", -42, 10, "-42"},
{"maxint64", 9223372036854775807, 10, "9223372036854775807"},
{"minint64", -9223372036854775808, 10, "-9223372036854775808"},
{"hex", 255, 16, "ff"},
} {
t.Run(tc.name, func(t *testing.T) {
var h Header
h.Reset(nil, numHeaderCapacity)
h.SetInt("Content-Length", tc.value, tc.base)
if got := string(h.Get("Content-Length")); got != tc.want {
t.Fatalf("want %q, got %q", tc.want, got)
}
})
}
}
// SetInt on an existing key must reuse the slot in place (single field, latest value).
func TestHeader_SetIntOverwrite(t *testing.T) {
var h Header
h.Reset(nil, numHeaderCapacity)
h.SetMethod("GET")
h.SetRequestTarget("/")
h.SetProtocol("HTTP/1.1")
h.SetInt("Content-Length", 100, 10)
h.SetInt("Content-Length", 5, 10) // shorter, must fit in old slot.
if got := string(h.Get("Content-Length")); got != "5" {
t.Fatalf("want Content-Length %q, got %q", "5", got)
}
req, err := h.AppendRequest(nil)
if err != nil {
t.Fatal(err)
}
if n := strings.Count(string(req), "Content-Length:"); n != 1 {
t.Errorf("want 1 Content-Length field, got %d:\n%s", n, req)
}
}
// SetInt must not heap-allocate: it must format directly into the header buffer.
func TestHeader_SetIntNoAlloc(t *testing.T) {
buf := make([]byte, 0, 256)
var h Header
h.Reset(buf, numHeaderCapacity)
h.ConfigBufferGrowth(false)
h.Add("Content-Length", "0000000000000000000000") // pre-size a reusable slot.
allocs := testing.AllocsPerRun(100, func() {
h.SetInt("Content-Length", 1234567890, 10)
})
if allocs != 0 {
t.Fatalf("SetInt allocated %v times, want 0", allocs)
}
if got := string(h.Get("Content-Length")); got != "1234567890" {
t.Fatalf("want %q, got %q", "1234567890", got)
}
}
// A browser sends upwards of twenty header fields and an API client with a few
// custom fields is not far behind. The field table must be sized from the
// buffer the caller handed over, not fixed at a count that a real request
// exceeds.
func TestHeader_FieldTableSizedFromBuffer(t *testing.T) {
const wantVal = "the-canary-value"
var raw strings.Builder
raw.WriteString("GET / HTTP/1.1\r\nHost: lneto.test\r\n")
for i := range 40 {
raw.WriteString("X-Field-" + strconv.Itoa(i) + ": value-of-a-realistic-length-here\r\n")
}
raw.WriteString("X-Canary: " + wantVal + "\r\n\r\n")
var h Header
h.Reset(make([]byte, 0, 8192), numHeaderCapacity) // Room for the block with plenty to spare.
err := h.ParseBytes(false, []byte(raw.String()))
if err != nil {
t.Fatalf("parsing a 42 field request into an 8kB buffer: %s", err)
}
if got := string(h.Get("X-Canary")); got != wantVal {
t.Fatalf("want X-Canary %q, got %q", wantVal, got)
}
}
// A buffer too small for the fields it is handed must be refused with an error
// the caller can act on, so a server answers 431 instead of dropping the peer.
func TestHeader_FieldTableFullIsReported(t *testing.T) {
var raw strings.Builder
raw.WriteString("GET / HTTP/1.1\r\n")
for i := range 64 {
raw.WriteString("H" + strconv.Itoa(i) + ":v\r\n") // As short as a field gets.
}
raw.WriteString("\r\n")
var h Header
h.Reset(make([]byte, 0, 512), numHeaderCapacity)
h.ConfigBufferGrowth(false)
err := h.ParseBytes(false, []byte(raw.String()))
if !errors.Is(err, ErrHeaderTooMany) {
t.Fatalf("want ErrHeaderFieldsTooLarge, got %v", err)
}
}
+272
View File
@@ -0,0 +1,272 @@
package httpraw
import (
"bytes"
"io"
)
// Multipart splits a "multipart/form-data" body into its parts. Such bodies
// frame their fields with a delimiter instead of escaping them, so a part's
// value has no length: it ends where the next delimiter begins.
//
// Multipart stores none of the body, leaving the caller to decide what to keep,
// what to skip and when a part has grown too large. Both methods report how much
// of buf they consumed, which the caller compacts away before reading more in:
//
// var m httpraw.Multipart
// m.SetContentType(contentType)
// var hdr httpraw.MultipartHeader
// for {
// parsed, err := m.NextHeader(&hdr, buf[:buflen])
// if err != nil {
// break // io.EOF at the closing delimiter, body done.
// } else if parsed == 0 {
// // Header block incomplete: read more into buf[buflen:] and retry.
// continue
// }
// buflen = copy(buf, buf[parsed:buflen])
// for {
// bodyLen, restOff, done := m.NextBody(buf[:buflen])
// // Consume buf[:bodyLen] for hdr.Name, then compact and read more.
// buflen = copy(buf, buf[restOff:buflen])
// if done {
// break
// }
// }
// }
type Multipart struct {
// Boundary is the delimiter parameter of the body's Content-Type field,
// without the leading "--" the delimiter carries on the wire.
Boundary []byte
}
// MultipartHeader is a part's header block and the Content-Disposition
// parameters that identify it.
type MultipartHeader struct {
// PartView is the part's raw header block, ending in its final CRLF. It
// aliases the buffer it was parsed from, so it is only valid until that
// buffer is compacted or read into again.
PartView []byte
// Name is the name parameter of a part's Content-Disposition field,
// i.e: "photo" for `form-data; name="photo"; filename="beach.png"`.
// Copied out of the buffer, so it outlives it, and reused between parts.
Name []byte
// Filename is the filename parameter of a part's Content-Disposition
// field, empty when the part is not a file upload. Copied like Name.
Filename []byte
}
// Reset clears the header for the next part, keeping the buffers Name and
// Filename were copied into so a reused header stops allocating.
func (hdr *MultipartHeader) Reset() {
hdr.PartView = nil
hdr.Name = hdr.Name[:0]
hdr.Filename = hdr.Filename[:0]
}
// SetContentType sets [Multipart.Boundary] from the boundary parameter of a
// Content-Type field value, i.e: "abc123" for
// "multipart/form-data; boundary=abc123". The leading "--" the delimiter carries
// on the wire is not included. Fails when the parameter is absent or is not
// 1 to 70 characters long, RFC 2046 5.1.1; a zero length boundary would match
// every "--" in the body.
func (m *Multipart) SetContentType(contentType []byte) error {
m.Boundary = ContentParam(contentType, "boundary")
if len(m.Boundary) == 0 || len(m.Boundary) > 70 {
return errNoBoundary // RFC 2046 5.1.1: 1..70 characters, required.
}
return nil
}
// NextHeader parses the leading part's header block off a multipart body into
// dst, returning how many bytes of data it consumed: the part's content begins
// at data[parsedLen]. A zero parsedLen and no error means data holds no complete
// delimiter and header block yet, so the caller reads more in and retries.
// Returns [io.EOF] once the closing delimiter is reached. dst is reset on error.
func (m *Multipart) NextHeader(dst *MultipartHeader, data []byte) (parsedLen int, err error) {
dst.Reset()
if len(m.Boundary) == 0 {
return 0, errNoBoundary
}
idx := m.indexDelimiter(data)
if idx < 0 {
return 0, nil
}
after := idx + len("--") + len(m.Boundary)
if after+2 > len(data) {
return 0, nil // Cannot tell a closing delimiter yet.
} else if data[after] == '-' && data[after+1] == '-' {
return 0, io.EOF
}
// Delimiter is followed by CRLF, then the part's header block.
if data[after] == '\r' {
after++
}
if after >= len(data) {
return 0, nil
} else if data[after] != '\n' {
return 0, errBadDelimiter
}
after++
end := bytes.Index(data[after:], []byte("\r\n\r\n"))
if end < 0 {
return 0, nil
}
dst.PartView = data[after : after+end+2]
disposition := partField(dst.PartView)
dst.Name = append(dst.Name[:0], ContentParam(disposition, "name")...)
dst.Filename = append(dst.Filename[:0], ContentParam(disposition, "filename")...)
return after + end + 4, nil
}
// NextBody reports how much of data is part content, data[:bodyLen], and where
// what is left begins, data[restOff:], which the caller compacts to the front of
// its buffer before reading more in. done reports the part ended, in which case
// data[restOff:] begins the next part's delimiter; otherwise the bytes past
// bodyLen are a tail held back because it could still turn into a delimiter.
func (m *Multipart) NextBody(data []byte) (bodyLen, restOff int, done bool) {
idx := m.indexPartEnd(data)
if idx >= 0 {
return idx, idx + len("\r\n"), true
}
// Longest prefix of "\r\n--"+boundary that could still be completed.
hold := min(len("\r\n--")+len(m.Boundary)-1, len(data))
return len(data) - hold, len(data) - hold, false
}
// indexDelimiter returns the offset of the leading "--"+Boundary in data.
func (m *Multipart) indexDelimiter(data []byte) int {
for i := 0; i+len("--")+len(m.Boundary) <= len(data); i++ {
dash := bytes.IndexByte(data[i:], '-')
if dash < 0 {
return -1
}
i += dash
if i+len("--")+len(m.Boundary) > len(data) {
return -1
}
if data[i+1] == '-' && b2s(data[i+2:i+2+len(m.Boundary)]) == b2s(m.Boundary) {
return i
}
}
return -1
}
// indexPartEnd returns the offset of the CRLF that closes a part, that is the
// CRLF preceding the next delimiter.
func (m *Multipart) indexPartEnd(data []byte) int {
for i := 0; i+len("\r\n--")+len(m.Boundary) <= len(data); i++ {
cr := bytes.IndexByte(data[i:], '\r')
if cr < 0 {
return -1
}
i += cr
if i+len("\r\n--")+len(m.Boundary) > len(data) {
return -1
}
if data[i+1] == '\n' && data[i+2] == '-' && data[i+3] == '-' &&
b2s(data[i+4:i+4+len(m.Boundary)]) == b2s(m.Boundary) {
return i
}
}
return -1
}
// MediaTypeIs reports whether a Content-Type field value carries the given
// media type, ignoring case and any parameters that follow it, i.e: true for
// "text/plain; charset=utf-8" and media type "text/plain". mediaType must be
// ASCII lowercase. RFC 9110 8.3.1.
func MediaTypeIs(value []byte, mediaType string) bool {
if semi := bytes.IndexByte(value, ';'); semi >= 0 {
value = value[:semi]
}
return equalFold(trimOWS(value), mediaType)
}
// ContentParam returns the value of a parameter of a header field value, i.e:
// "utf-8" for key "charset" of "text/plain; charset=utf-8". Quoted values are
// returned without their quotes and with escapes left as they appear on the
// wire. Key matching is case insensitive, RFC 9110 5.6.6.
func ContentParam(value []byte, key string) []byte {
for len(value) > 0 {
semi := bytes.IndexByte(value, ';')
if semi < 0 {
return nil // No parameters left.
}
value = trimOWS(value[semi+1:])
eq := bytes.IndexByte(value, '=')
if eq < 0 {
return nil
}
gotKey := trimOWS(value[:eq])
value = value[eq+1:]
param := value
if len(param) > 0 && param[0] == '"' {
end := bytes.IndexByte(param[1:], '"')
if end < 0 {
return nil // Unterminated quoted string.
}
param, value = param[1:end+1], param[end+2:]
} else {
end := bytes.IndexByte(param, ';')
if end >= 0 {
param, value = param[:end], param[end:]
} else {
value = nil
}
param = trimOWS(param)
}
if equalFold(gotKey, key) {
return param
}
}
return nil
}
// partField returns the Content-Disposition field value of a part header block.
func partField(partHdr []byte) []byte {
const key = "content-disposition"
for len(partHdr) > 0 {
eol := bytes.IndexByte(partHdr, '\n')
line := partHdr
if eol >= 0 {
line, partHdr = partHdr[:eol], partHdr[eol+1:]
} else {
partHdr = nil
}
colon := bytes.IndexByte(line, ':')
if colon > 0 && equalFold(line[:colon], key) {
return line[colon+1:]
}
}
return nil
}
// trimOWS trims optional whitespace off both ends of b, RFC 9110 5.6.3.
func trimOWS(b []byte) []byte {
for len(b) > 0 && (b[0] == ' ' || b[0] == '\t') {
b = b[1:]
}
for len(b) > 0 && (b[len(b)-1] == ' ' || b[len(b)-1] == '\t') {
b = b[:len(b)-1]
}
return b
}
// equalFold compares b to the ASCII lowercase key, case insensitively.
func equalFold(b []byte, key string) bool {
if len(b) != len(key) {
return false
}
const asciiCapDiff = 'a' - 'A'
for i := range b {
c := b[i]
if c >= 'A' && c <= 'Z' {
c += asciiCapDiff
}
if c != key[i] {
return false
}
}
return true
}
+334
View File
@@ -0,0 +1,334 @@
package httpraw
import (
"io"
"strconv"
"strings"
"testing"
)
// A two part body: a text field and a PNG upload whose bytes contain CRLFs and
// even the boundary text, which must not desync the parser.
const (
multiBoundary = "----abc123"
multiBody = "------abc123\r\n" +
"Content-Disposition: form-data; name=\"caption\"\r\n" +
"\r\n" +
"hi there\r\n" +
"------abc123\r\n" +
"Content-Disposition: form-data; name=\"photo\"; filename=\"beach.png\"\r\n" +
"Content-Type: image/png\r\n" +
"\r\n" +
"\x89PNG\r\n--not-the-boundary\r\n\x00\xff\r\n" +
"------abc123--\r\n"
)
func TestMultipartBoundary(t *testing.T) {
var mp Multipart
for _, test := range []struct {
contentType string
want string
wantErr bool
}{
{contentType: "multipart/form-data; boundary=abc123", want: "abc123"},
{contentType: "multipart/form-data; boundary=\"a b\"", want: "a b"},
{contentType: "multipart/form-data; charset=utf-8; boundary=xyz", want: "xyz"},
{contentType: "multipart/form-data; BOUNDARY=xyz", want: "xyz"}, // Keys are case insensitive.
{contentType: "multipart/form-data", wantErr: true}, // Absent, RFC 2046 5.1.1 requires it.
{contentType: "application/x-www-form-urlencoded", wantErr: true}, // Not multipart at all.
{contentType: "multipart/form-data; boundary=", wantErr: true}, // Empty matches every "--".
} {
err := mp.SetContentType([]byte(test.contentType))
if test.wantErr {
if err == nil {
t.Errorf("%q: want error, got boundary %q", test.contentType, mp.Boundary)
}
continue
} else if err != nil {
t.Errorf("%q: %s", test.contentType, err)
continue
}
got := string(mp.Boundary)
if got != test.want {
t.Errorf("%q: want %q, got %q", test.contentType, test.want, got)
}
}
}
func TestMediaTypeIs(t *testing.T) {
for _, test := range []struct {
value string
media string
want bool
}{
{value: "text/plain", media: "text/plain", want: true},
{value: "text/plain; charset=utf-8", media: "text/plain", want: true},
{value: "text/plain;charset=utf-8", media: "text/plain", want: true},
{value: "Text/Plain", media: "text/plain", want: true}, // Case insensitive, RFC 9110 8.3.1.
{value: " text/plain ; x=1", media: "text/plain", want: true},
{value: "text/plain", media: "text/html"},
{value: "text/plainish", media: "text/plain"}, // Prefix must not match.
{value: "", media: "text/plain"},
{value: "multipart/form-data; boundary=abc", media: "multipart/form-data", want: true},
} {
if got := MediaTypeIs([]byte(test.value), test.media); got != test.want {
t.Errorf("%q is %q: want %v, got %v", test.value, test.media, test.want, got)
}
}
}
func TestContentParam(t *testing.T) {
for _, test := range []struct {
value string
key string
want string
}{
{value: "text/plain; charset=utf-8", key: "charset", want: "utf-8"},
{value: "text/plain;charset=utf-8", key: "charset", want: "utf-8"}, // No space.
{value: "text/plain; charset=\"utf-8\"", key: "charset", want: "utf-8"},
{value: "form-data; name=\"photo\"; filename=\"a;b.png\"", key: "filename", want: "a;b.png"},
{value: "form-data; name=\"photo\"", key: "nope", want: ""},
{value: "form-data; names=x; name=y", key: "name", want: "y"}, // Prefix must not match.
{value: "text/plain", key: "charset", want: ""},
} {
got := ContentParam([]byte(test.value), test.key)
if string(got) != test.want {
t.Errorf("%q key %q: want %q, got %q", test.value, test.key, test.want, got)
}
}
}
func TestNextPartHeader(t *testing.T) {
m := Multipart{Boundary: []byte(multiBoundary)}
var hdr MultipartHeader
parsed, err := m.NextHeader(&hdr, []byte(multiBody))
if err != nil {
t.Fatal(err)
}
const wantHdr = "Content-Disposition: form-data; name=\"caption\"\r\n"
if string(hdr.PartView) != wantHdr {
t.Errorf("want header %q, got %q", wantHdr, hdr.PartView)
}
if string(hdr.Name) != "caption" {
t.Errorf("want name %q, got %q", "caption", hdr.Name)
}
if len(hdr.Filename) != 0 {
t.Errorf("want no filename for a non file part, got %q", hdr.Filename)
}
if !strings.HasPrefix(multiBody[parsed:], "hi there\r\n") {
t.Errorf("want rest at part body, got %q", multiBody[parsed:])
}
}
// Names and filenames must outlive the buffer they were parsed from, so a
// caller may compact it and read more without losing the part it is reading.
func TestNextPartHeaderOutlivesBuffer(t *testing.T) {
m := Multipart{Boundary: []byte(multiBoundary)}
data := []byte(multiBody)
var hdr MultipartHeader
if _, err := m.NextHeader(&hdr, data); err != nil {
t.Fatal(err)
}
for i := range data {
data[i] = 'x' // Buffer reused for the next read.
}
if string(hdr.Name) != "caption" {
t.Errorf("want name %q to survive the buffer, got %q", "caption", hdr.Name)
}
}
// Incomplete data must ask for more, never guess.
func TestNextPartHeaderNeedMore(t *testing.T) {
m := Multipart{Boundary: []byte(multiBoundary)}
for _, data := range []string{
"",
"------abc", // Delimiter cut short.
"------abc123\r\n", // No header block yet.
"------abc123\r\nContent-Disposition: form-", // Header block unterminated.
} {
var hdr MultipartHeader
parsed, err := m.NextHeader(&hdr, []byte(data))
if parsed != 0 || err != nil {
t.Errorf("%q: want (0, nil) asking for more data, got (%d, %v)", data, parsed, err)
}
}
}
// Junk between the delimiter and the part header is a multipart framing error,
// not a header field name error.
func TestNextPartHeaderJunk(t *testing.T) {
m := Multipart{Boundary: []byte("abc")}
var hdr MultipartHeader
if _, err := m.NextHeader(&hdr, []byte("--abcX\r\nA: b\r\n\r\n")); err != errBadDelimiter {
t.Errorf("want errBadDelimiter, got %v", err)
}
}
// The closing delimiter ends iteration.
func TestNextPartHeaderEnd(t *testing.T) {
m := Multipart{Boundary: []byte(multiBoundary)}
var hdr MultipartHeader
if _, err := m.NextHeader(&hdr, []byte("------abc123--\r\n")); err != io.EOF {
t.Errorf("want io.EOF, got %v", err)
}
}
func TestNextPartBody(t *testing.T) {
m := Multipart{Boundary: []byte(multiBoundary)}
var hdr MultipartHeader
parsed, err := m.NextHeader(&hdr, []byte(multiBody))
if err != nil {
t.Fatal(err)
}
rest := multiBody[parsed:]
bodyLen, restOff, done := m.NextBody([]byte(rest))
if !done {
t.Fatal("want the part to end within the buffer")
}
if rest[:bodyLen] != "hi there" {
t.Errorf("want body %q, got %q", "hi there", rest[:bodyLen])
}
if !strings.HasPrefix(rest[restOff:], "------abc123\r\n") {
t.Errorf("want rest at next delimiter, got %q", rest[restOff:])
}
}
// A part whose bytes contain CRLFs and boundary-like text must survive intact.
func TestNextPartBodyBinary(t *testing.T) {
m := Multipart{Boundary: []byte(multiBoundary)}
rest := []byte(multiBody)
var hdr MultipartHeader
parsed, err := m.NextHeader(&hdr, rest) // caption part.
if err != nil {
t.Fatal(err)
}
_, restOff, _ := m.NextBody(rest[parsed:])
rest = rest[parsed+restOff:]
parsed, err = m.NextHeader(&hdr, rest) // photo part.
if err != nil {
t.Fatal(err)
}
rest = rest[parsed:]
bodyLen, restOff, done := m.NextBody(rest)
if !done {
t.Fatal("want the part to end within the buffer")
}
const want = "\x89PNG\r\n--not-the-boundary\r\n\x00\xff"
if string(rest[:bodyLen]) != want {
t.Errorf("want body %q, got %q", want, rest[:bodyLen])
}
if _, err = m.NextHeader(&hdr, rest[restOff:]); err != io.EOF {
t.Errorf("want io.EOF after last part, got %v", err)
}
}
// A delimiter split across two reads must not be mistaken for part data: the
// tail is held back until proven not to be a delimiter.
func TestNextPartBodySplitDelimiter(t *testing.T) {
m := Multipart{Boundary: []byte(multiBoundary)}
const part = "hi there"
full := part + "\r\n------abc123\r\n"
for split := 1; split < len(full); split++ {
data := full[:split]
bodyLen, restOff, done := m.NextBody([]byte(data))
if done {
continue // Whole delimiter already present, nothing to prove.
}
if bodyLen > len(part) {
t.Fatalf("split %d: emitted %q, past the end of the part", split, data[:bodyLen])
}
if data[:bodyLen]+data[restOff:] != data {
t.Fatalf("split %d: body+rest %q%q does not reconstruct input", split, data[:bodyLen], data[restOff:])
}
}
}
// A file part carries both parameters, and the raw block stays available.
func TestNextHeaderFilePart(t *testing.T) {
m := Multipart{Boundary: []byte(multiBoundary)}
var hdr MultipartHeader
parsed, err := m.NextHeader(&hdr, []byte(multiBody)) // caption part.
if err != nil {
t.Fatal(err)
}
rest := []byte(multiBody[parsed:])
_, restOff, _ := m.NextBody(rest)
if _, err = m.NextHeader(&hdr, rest[restOff:]); err != nil { // photo part.
t.Fatal(err)
}
if got := string(hdr.Name); got != "photo" {
t.Errorf("want name %q, got %q", "photo", got)
}
if got := string(hdr.Filename); got != "beach.png" {
t.Errorf("want filename %q, got %q", "beach.png", got)
}
if !strings.Contains(string(hdr.PartView), "Content-Type: image/png") {
t.Errorf("want the raw block to hold every field, got %q", hdr.PartView)
}
}
// A failed call must not leave the previous part's fields behind.
func TestNextHeaderResetsOnError(t *testing.T) {
m := Multipart{Boundary: []byte(multiBoundary)}
var hdr MultipartHeader
if _, err := m.NextHeader(&hdr, []byte(multiBody)); err != nil {
t.Fatal(err)
}
if _, err := m.NextHeader(&hdr, []byte("------abc123--\r\n")); err != io.EOF {
t.Fatalf("want io.EOF, got %v", err)
}
if hdr.PartView != nil || len(hdr.Name) != 0 || len(hdr.Filename) != 0 {
t.Errorf("want cleared header on error, got %+v", hdr)
}
}
// A header reused across parts must stop allocating once its name and filename
// buffers are big enough.
func TestNextHeaderReuseNoAlloc(t *testing.T) {
m := Multipart{Boundary: []byte(multiBoundary)}
data := []byte(multiBody)
var hdr MultipartHeader
allocs := testing.AllocsPerRun(10, func() {
if _, err := m.NextHeader(&hdr, data); err != nil {
t.Fatal(err)
}
})
if allocs != 0 {
t.Errorf("want a reused header to allocate 0 times, got %v", allocs)
}
}
// The whole loop, as a caller writes it over a buffer it compacts.
func TestMultipartLoop(t *testing.T) {
m := Multipart{Boundary: []byte(multiBoundary)}
rest := []byte(multiBody)
var got []string
var hdr MultipartHeader
for {
parsed, err := m.NextHeader(&hdr, rest)
if err == io.EOF {
break
} else if err != nil {
t.Fatal(err)
} else if parsed == 0 {
t.Fatal("header must complete within the buffer")
}
name := string(hdr.Name)
total := 0
rest = rest[parsed:]
for {
bodyLen, restOff, done := m.NextBody(rest)
total += bodyLen
rest = rest[restOff:]
if done {
break
}
t.Fatal("part must complete within the buffer")
}
got = append(got, name+":"+strconv.Itoa(total))
}
want := "caption:8|photo:28"
if strings.Join(got, "|") != want {
t.Errorf("want %q, got %q", want, strings.Join(got, "|"))
}
}
+150 -48
View File
@@ -4,18 +4,27 @@ import (
"bytes"
"errors"
"slices"
"strconv"
"unsafe"
"github.com/soypat/lneto/internal"
)
var (
errNoProto = errors.New("missing protocol, HTTP/0.9 unsupported")
errNeedMore = errors.New("need more data: cannot find trailing lf")
errUnparsed = errors.New("need to finish parsing")
errInvalidName = errors.New("invalid header name")
errSmallBuffer = errors.New("small read buffer. Increase ReadBufferSize")
errOOM = errors.New("httpraw: buffer out of memory")
errNoProto = errors.New("missing protocol, HTTP/0.9 unsupported")
// ErrNeedMoreData signals a parser was handed an incomplete buffer: append
// more data to it and call again.
ErrNeedMoreData = errors.New("need more data: cannot find trailing lf/delimiter")
errNoBoundary = errors.New("httpraw: multipart boundary not set")
errUnparsed = errors.New("need to finish parsing")
errInvalidName = errors.New("invalid header name")
ErrSmallHeaderBuffer = errors.New("httpraw: Header buffer exhausted, increase size")
errOOM = errors.New("httpraw: Header incomplete due to OOM")
// ErrHeaderTooMany signals a header block carrying more fields than
// the buffer it is parsed into has room for, see [Header.Reset]. A server
// answers it with 431, RFC 6585 5: no larger buffer is coming, so reading
// the rest of the block would only spend memory on a request already lost.
ErrHeaderTooMany = errors.New("httpraw: more header fields than buffer holds")
// Header.Set and Header.Add mangles the buffer.
// Call them after retrieving the Body. Do not call them before parsing the header (why would you even do that?).
errMangledBuffer = errors.New("httpraw: mangled buffer")
@@ -27,8 +36,18 @@ var (
errNeedMethodURI = errors.New("need method/request URI to create request header")
errBadStatusCodeTxt = errors.New("invalid status code or text")
errCookiesParsed = errors.New("cookies already parsed, reset before parsing again")
errBufferTooLarge = errors.New("httpraw: buffer exceeds max size (offsets are uint16)")
errBadPercentEncode = errors.New("httpraw: invalid percent-encoding in URL")
errBadDelimiter = errors.New("httpraw: junk between multipart delimiter and part")
errNoContentLength = errors.New("httpraw: no Content-Length field")
errBadContentLength = errors.New("httpraw: invalid Content-Length value")
)
// maxBufLen bounds the header buffer. Offsets/lengths are stored as uint16
// (tokint); a buffer past this would truncate/overflow those, silently
// returning the wrong bytes or panicking on a wrapped slice bound.
const maxBufLen = 0xffff
type headerBuf struct {
// buf[:len] holds entire HTTP header data, which may be normalized by [flags]. buf[off:len] holds data not yet processed during parsing.
buf []byte
@@ -38,17 +57,21 @@ type headerBuf struct {
headers []argsKV
}
// reset sets the buffer data and discards all parsed data.
func (h *headerBuf) reset(buf []byte) {
// reset sets the buffer data and discards all parsed data. The field table is
// grown to match the new buffer's capacity and never shrinks, so a header
// reused across requests settles on its largest buffer and stops allocating.
func (h *headerBuf) reset(buf []byte, numHeaderCapacity int) {
if buf == nil {
buf = h.buf[:0] // Reuse buffer but discard raw data on nil input.
}
if cap(h.headers) == 0 {
h.headers = make([]argsKV, 16)
if numHeaderCapacity != 0 {
internal.SliceReuse(&h.headers, numHeaderCapacity)
} else {
h.headers = h.headers[:0]
}
*h = headerBuf{
buf: buf,
headers: h.headers[:0],
headers: h.headers,
}
}
@@ -86,23 +109,26 @@ func (h *Header) parse(asResponse bool) (err error) {
return err
}
debuglog("http:firstline:done")
err = h.parseNextHeaders()
err = h.parseNextHeaders(h.flags)
debuglog("http:headers:done")
return err
}
func (h *Header) parseFirstLine(asResponse bool) (err error) {
if len(h.hbuf.buf) > maxBufLen {
return errBufferTooLarge // Offsets would overflow uint16 tokint.
}
if asResponse {
h.statusCode, h.statusText, h.flags, err = h.hbuf.parseFirstLineResponse(h.flags)
} else {
h.method, h.requestURI, h.proto, h.flags, err = h.hbuf.parseFirstLineRequest(h.flags)
h.method, h.requestTarget, h.proto, h.flags, err = h.hbuf.parseFirstLineRequest(h.flags)
}
return err
}
func (h *Header) parseNextHeaders() error {
func (h *Header) parseNextHeaders(flags Flags) error {
var ss scannerState
h.hbuf.parseNextHeaders(&ss)
h.hbuf.parseNextHeaders(&ss, flags)
if ss.err != nil {
h.flags |= flagConnClose
return ss.err
@@ -117,10 +143,16 @@ func (hb *headerBuf) readFromBytes(b []byte) {
func (hb *headerBuf) free() int { return cap(hb.buf) - len(hb.buf) }
func (hb *headerBuf) parseNextHeaders(ss *scannerState) {
func (hb *headerBuf) parseNextHeaders(ss *scannerState, flags Flags) {
debuglog("http:nexthdr:loop")
for kv := hb.next(ss); kv.isValid(); kv = hb.next(ss) {
hb.headers = append(hb.headers, kv) // TODO(HEAP): inc=16B slice growth when capacity exceeded
if len(hb.headers) == cap(hb.headers) && flags.HasAny(flagNoBufferGrow) {
// Refuse to grow the headers slice: the caller granted this much
// memory and no more, see [Header.Reset].
ss.err = ErrHeaderTooMany
return
}
hb.headers = append(hb.headers, kv)
}
debuglog("http:nexthdr:done")
}
@@ -156,17 +188,17 @@ func (hb *headerBuf) scanUntilByte(c byte) []byte {
return buf
}
func (hb *headerBuf) parseFirstLineRequest(initFlags flags) (method, uri, proto headerSlice, flags flags, err error) {
func (hb *headerBuf) parseFirstLineRequest(initFlags Flags) (method, uri, proto headerSlice, flags Flags, err error) {
debuglog("http:req:scan")
hb.off = 0 // Parsing first line resets offset.
hb.skipLeadingCRLF()
flags = initFlags
if bytes.IndexByte(hb.offBuf(), '\n') < 0 {
return method, uri, proto, flags, errNeedMore // Incomplete line.
return method, uri, proto, flags, ErrNeedMoreData // Incomplete line.
}
b := hb.scanLine()
if len(b) < 5 {
return method, uri, proto, flags, errNeedMore
return method, uri, proto, flags, ErrNeedMoreData
}
debuglog("http:req:parse")
@@ -190,24 +222,24 @@ func (hb *headerBuf) parseFirstLineRequest(initFlags flags) (method, uri, proto
return method, uri, proto, flags, nil
}
func (hb *headerBuf) parseFirstLineResponse(initFlags flags) (statusCode, statusText headerSlice, flags flags, err error) {
func (hb *headerBuf) parseFirstLineResponse(initFlags Flags) (statusCode, statusText headerSlice, flags Flags, err error) {
debuglog("http:resp:scan")
hb.off = 0 // Parsing first line resets offset.
hb.skipLeadingCRLF()
flags = initFlags
if bytes.IndexByte(hb.offBuf(), '\n') < 0 {
return statusCode, statusText, flags, errNeedMore // Incomplete line.
return statusCode, statusText, flags, ErrNeedMoreData // Incomplete line.
}
b := hb.scanLine()
if len(b) < 5 {
return statusCode, statusText, flags, errNeedMore
return statusCode, statusText, flags, ErrNeedMoreData
}
debuglog("http:resp:parse")
// Parse protocol (e.g. "HTTP/1.1"), then status code, then status text.
protoEnd := bytes.IndexByte(b, ' ')
if protoEnd < 0 {
return statusCode, statusText, flags, errNeedMore
return statusCode, statusText, flags, ErrNeedMoreData
}
if b2s(b[:protoEnd]) != strHTTP11 {
flags |= flagNoHTTP11
@@ -296,32 +328,22 @@ func (h *Header) reuseOrAppend(tok headerSlice, value string) headerSlice {
func (h *Header) appendSlice(value string) headerSlice {
debuglog("http:appendslice:start")
free := h.hbuf.free()
if len(value) > free {
if h.flags.hasAny(flagNoBufferGrow) {
h.flags |= flagOOMReached
return headerSlice{}
}
debuglog("http:appendslice:grow-buf")
h.hbuf.buf = slices.Grow(h.hbuf.buf, len(value)+1) // Grow 1 beyond due to slice validity.
if !h.reserve(len(value)) {
return headerSlice{}
}
h.flags |= flagMangledBuffer
return h.hbuf.mustAppendSlice(value)
}
func (h *Header) appendHeader(key, value string) {
hb := &h.hbuf
free := hb.free()
buf := h.hbuf.buf
if len(key)+len(value) > free {
if h.flags.hasAny(flagNoBufferGrow) {
panic(errSmallBuffer)
}
debuglog("http:appendhdr:grow-buf")
hb.buf = slices.Grow(buf, len(key)+len(value))
// reserve accounts for the byte-0 reservation mustAppendSlice makes on an
// empty buffer, and drops (flagging OOM) rather than panicking when growth
// is disabled and space runs out.
if !h.reserve(len(key) + len(value)) {
return
}
h.flags |= flagMangledBuffer
hb := &h.hbuf
k := hb.mustAppendSlice(key)
v := hb.mustAppendSlice(value)
debuglog("http:appendhdr:grow-hdrs")
@@ -331,6 +353,83 @@ func (h *Header) appendHeader(key, value string) {
})
}
// appendHeaderInt is appendHeader's integer counterpart: it appends key and the
// formatted integer value as a new header field.
func (h *Header) appendHeaderInt(key string, value int64, base int) {
n := internal.IntLen(value, base)
if !h.reserve(len(key) + n) {
return // Drop and flag OOM; never panic.
}
h.flags |= flagMangledBuffer
hb := &h.hbuf
k := hb.mustAppendSlice(key)
v := hb.mustAppendInt(value, base)
hb.headers = append(hb.headers, argsKV{
key: k,
value: v,
})
}
// reserve ensures need free bytes are available in the buffer, growing it when
// permitted. It accounts for the byte-0 reservation on an empty buffer (see
// mustAppendSlice). It returns false and sets flagOOMReached when the space
// cannot be guaranteed: a tokint offset overflow, or a full buffer with
// flagNoBufferGrow set.
func (h *Header) reserve(need int) bool {
hb := &h.hbuf
if len(hb.buf) == 0 {
need++ // mustAppend* reserves byte 0 on an empty buffer.
}
if len(hb.buf)+need > maxBufLen {
h.flags |= flagOOMReached // Offsets would overflow uint16 tokint.
return false
}
if need > hb.free() {
if h.flags.HasAny(flagNoBufferGrow) {
h.flags |= flagOOMReached
return false
}
hb.buf = slices.Grow(hb.buf, need)
}
return true
}
// reuseOrAppendInt writes value into tok's slot in place when it fits, avoiding
// any buffer growth; otherwise it appends a fresh slot.
func (h *Header) reuseOrAppendInt(tok headerSlice, value int64, base int) headerSlice {
n := internal.IntLen(value, base)
if int(tok.len) >= n {
// Reuse: format directly over the existing slot. No free space needed
// since n <= tok.len and the slot already lives inside buf.
v := strconv.AppendInt(h.hbuf.buf[tok.start:tok.start], value, base)
tok.len = tokint(len(v))
h.flags |= flagMangledBuffer
return tok
}
return h.appendInt(value, base, n)
}
// appendInt reserves space (growing or flagging OOM) and appends value as a new slot.
func (h *Header) appendInt(value int64, base, n int) headerSlice {
if !h.reserve(n) {
return headerSlice{} // Drop and flag OOM; never panic.
}
h.flags |= flagMangledBuffer
return h.hbuf.mustAppendInt(value, base)
}
// mustAppendInt formats value into the buffer's free region and commits it.
// The caller must have reserved at least internal.IntLen(value, base) free bytes.
func (hb *headerBuf) mustAppendInt(value int64, base int) headerSlice {
L := len(hb.buf)
if L == 0 {
L++ // Valid key-values start after byte 0.
}
v := strconv.AppendInt(hb.buf[L:L], value, base)
hb.buf = hb.buf[:L+len(v)]
return hb.slice(hb.buf[L : L+len(v)])
}
func (hb *headerBuf) noKV() argsKV { return argsKV{} }
func (hb *headerBuf) next(ss *scannerState) argsKV {
@@ -360,15 +459,18 @@ func (hb *headerBuf) next(ss *scannerState) argsKV {
if x < 0 {
// A header name should always at some point be followed by a \n
// even if it's the one that terminates the header block.
ss.err = errNeedMore
ss.err = ErrNeedMoreData
return hb.noKV()
} else if x < n {
// There was a \n before the colon! This is invalid.
ss.err = errInvalidName
return hb.noKV()
} else if n < 0 {
// No colon found, probably missing data.
ss.err = errNeedMore
// A newline is present (x>=0 reached here) but the line has no
// colon: malformed, not incomplete. A split arriving before the
// colon has no newline yet and is caught by the x<0 branch above,
// so it still returns ErrNeedMoreData.
ss.err = errInvalidName
return hb.noKV()
}
}
@@ -395,7 +497,7 @@ func (hb *headerBuf) next(ss *scannerState) argsKV {
nl := bytes.IndexByte(buf[n:], '\n')
if nl < 0 || nl+n+1 == len(buf) {
// No newline or newline is last character and can't know if is multiline.
ss.err = errNeedMore
ss.err = ErrNeedMoreData
return hb.noKV()
}
n += nl + 1 // Index of the newly found newline.
@@ -416,9 +518,9 @@ func (hb *headerBuf) next(ss *scannerState) argsKV {
// ConnectionClose returns true if 'Connection: close' header is set or if a invalid header was found.
func (h *Header) ConnectionClose() bool {
closed := h.flags.hasAny(flagConnClose) ||
closed := h.flags.HasAny(flagConnClose) ||
h.hasHeaderValue(headerConnection, strClose) ||
(h.flags.hasAny(flagNoHTTP11) && !h.hasHeaderValue(headerConnection, "keep-alive"))
(h.flags.HasAny(flagNoHTTP11) && !h.hasHeaderValue(headerConnection, "keep-alive"))
if closed {
h.flags |= flagConnClose
}
+18 -22
View File
@@ -10,7 +10,7 @@ func TestTryParse_IncrementalRequest(t *testing.T) {
// Full HTTP request split across multiple ReadFromBytes calls.
full := "GET /index.html HTTP/1.1\r\nHost: example.com\r\nContent-Type: text/html\r\n\r\nbody here"
var hdr Header
hdr.Reset(make([]byte, 0, 256))
hdr.Reset(make([]byte, 0, 256), numHeaderCapacity)
// Feed data in small chunks to exercise incremental parsing.
chunks := splitInto(full, 10)
@@ -52,8 +52,8 @@ func TestTryParse_IncrementalRequest(t *testing.T) {
if string(hdr.Method()) != "GET" {
t.Errorf("method = %q; want GET", hdr.Method())
}
if string(hdr.RequestURI()) != "/index.html" {
t.Errorf("URI = %q; want /index.html", hdr.RequestURI())
if string(hdr.RequestTarget()) != "/index.html" {
t.Errorf("URI = %q; want /index.html", hdr.RequestTarget())
}
// Verify headers via ForEach.
@@ -85,7 +85,7 @@ func TestTryParse_IncrementalRequest(t *testing.T) {
func TestTryParse_IncrementalResponse(t *testing.T) {
full := "HTTP/1.1 200 OK\r\nContent-Length: 5\r\nServer: lneto\r\n\r\nhello"
var hdr Header
hdr.Reset(make([]byte, 0, 256))
hdr.Reset(make([]byte, 0, 256), numHeaderCapacity)
chunks := splitInto(full, 8)
var done bool
@@ -137,7 +137,7 @@ func TestReadFromLimited(t *testing.T) {
r := strings.NewReader(data)
var hdr Header
hdr.Reset(make([]byte, 0, 256))
hdr.Reset(make([]byte, 0, 256), numHeaderCapacity)
// Read in one shot.
n, err := hdr.ReadFromLimited(r, 256)
@@ -163,7 +163,7 @@ func TestReadFromLimited(t *testing.T) {
func TestReadFromLimited_MaxBytes(t *testing.T) {
var hdr Header
hdr.Reset(make([]byte, 0, 256))
hdr.Reset(make([]byte, 0, 256), numHeaderCapacity)
// Zero maxBytesToRead should error.
_, err := hdr.ReadFromLimited(strings.NewReader("data"), 0)
@@ -174,7 +174,7 @@ func TestReadFromLimited_MaxBytes(t *testing.T) {
func TestReadFromBytes_Empty(t *testing.T) {
var hdr Header
hdr.Reset(make([]byte, 0, 256))
hdr.Reset(make([]byte, 0, 256), numHeaderCapacity)
_, err := hdr.ReadFromBytes(nil)
if err == nil {
@@ -184,7 +184,7 @@ func TestReadFromBytes_Empty(t *testing.T) {
func TestBufferFreeAndCapacity(t *testing.T) {
var hdr Header
hdr.Reset(make([]byte, 0, 100))
hdr.Reset(make([]byte, 0, 100), numHeaderCapacity)
if hdr.BufferCapacity() != 100 {
t.Errorf("capacity = %d; want 100", hdr.BufferCapacity())
@@ -202,9 +202,8 @@ func TestBufferFreeAndCapacity(t *testing.T) {
func TestEnableBufferGrowth(t *testing.T) {
var hdr Header
buf := make([]byte, 0, 64)
hdr.Reset(buf)
hdr.EnableBufferGrowth(false)
hdr.Reset(buf, numHeaderCapacity)
hdr.ConfigBufferGrowth(false)
// With growth disabled, reading more than capacity should fail.
big := make([]byte, 128)
for i := range big {
@@ -389,7 +388,7 @@ func TestHeader_MultilineValue(t *testing.T) {
func TestHeader_ResponseRoundTrip(t *testing.T) {
var hdr Header
hdr.Reset(make([]byte, 0, 256))
hdr.Reset(make([]byte, 0, 256), numHeaderCapacity)
hdr.SetProtocol("HTTP/1.1")
hdr.SetStatus("404", "Not Found")
hdr.Add("Content-Type", "text/plain")
@@ -429,10 +428,10 @@ func TestHeader_ResponseRoundTrip(t *testing.T) {
func TestHeader_RequestRoundTrip(t *testing.T) {
var hdr Header
hdr.Reset(make([]byte, 0, 256))
hdr.Reset(make([]byte, 0, 256), numHeaderCapacity)
hdr.SetProtocol("HTTP/1.1")
hdr.SetMethod("POST")
hdr.SetRequestURI("/api/data")
hdr.SetRequestTarget("/api/data")
hdr.Add("Host", "example.com")
hdr.Add("Content-Type", "application/json")
@@ -454,8 +453,8 @@ func TestHeader_RequestRoundTrip(t *testing.T) {
if string(hdr2.Method()) != "POST" {
t.Errorf("re-parsed method = %q; want POST", hdr2.Method())
}
if string(hdr2.RequestURI()) != "/api/data" {
t.Errorf("re-parsed URI = %q; want /api/data", hdr2.RequestURI())
if string(hdr2.RequestTarget()) != "/api/data" {
t.Errorf("re-parsed URI = %q; want /api/data", hdr2.RequestTarget())
}
if string(hdr2.Get("Host")) != "example.com" {
t.Errorf("re-parsed Host = %q; want example.com", hdr2.Get("Host"))
@@ -504,8 +503,8 @@ func TestParseRequest_NoProtocol(t *testing.T) {
if string(hdr.Method()) != "GET" {
t.Errorf("method = %q; want GET", hdr.Method())
}
if string(hdr.RequestURI()) != "/simple" {
t.Errorf("URI = %q; want /simple", hdr.RequestURI())
if string(hdr.RequestTarget()) != "/simple" {
t.Errorf("URI = %q; want /simple", hdr.RequestTarget())
}
if hdr.Protocol() != nil {
t.Errorf("protocol should be nil for version-less request, got %q", hdr.Protocol())
@@ -533,10 +532,7 @@ func TestParseRequest_InvalidHeaderSpaceBeforeColon(t *testing.T) {
func splitInto(s string, n int) []string {
var chunks []string
for len(s) > 0 {
end := n
if end > len(s) {
end = len(s)
}
end := min(n, len(s))
chunks = append(chunks, s[:end])
s = s[end:]
}
-62
View File
@@ -1,62 +0,0 @@
package internal
import "time"
type BackoffFlags uint8
const (
BackoffHasPriority BackoffFlags = 1 << iota
BackoffCriticalPath
BackoffTCPConn
)
const backoffMinWait = time.Microsecond
func backoffMaxWait(priority BackoffFlags) time.Duration {
switch {
case priority&BackoffCriticalPath != 0:
return 1 * time.Millisecond
case priority&BackoffTCPConn != 0:
return 5 * time.Millisecond
default:
return time.Second >> (priority & BackoffHasPriority)
}
}
func NewBackoff(priority BackoffFlags) Backoff {
return Backoff{
wait: uint32(backoffMinWait),
maxWait: uint32(backoffMaxWait(priority)),
startWait: uint32(backoffMinWait),
}
}
// A Backoff with a non-zero MaxWait is ready for use.
type Backoff struct {
// wait defines the amount of time that Miss will wait on next call.
wait uint32
// Maximum allowable value for Wait.
maxWait uint32
// startWait is the intial Wait value, as well as the value that Wait takes after a call to Hit.
startWait uint32
}
// Hit sets eb.Wait to the StartWait value.
func (eb *Backoff) Hit() {
if eb.maxWait == 0 {
panic("MaxWait cannot be zero")
}
eb.wait = eb.startWait
}
// Miss sleeps for eb.Wait and increases eb.Wait exponentially.
func (eb *Backoff) Miss() {
if eb.maxWait == 0 {
panic("MaxWait cannot be zero")
}
time.Sleep(time.Duration(eb.wait))
eb.wait *= 2
if eb.wait > eb.maxWait {
eb.wait = eb.maxWait
}
}
+219
View File
@@ -0,0 +1,219 @@
// Command benchci parses `go test -bench` output and renders a Markdown
// report. It is repo-owned tooling so CI does not depend on third-party
// benchmark actions. With -count>1 it reports the median of each metric to
// reduce noise.
package main
import (
"bufio"
"flag"
"fmt"
"io"
"os"
"sort"
"strconv"
"strings"
)
// commentMarker is a stable HTML marker so a PR commenter can locate and
// update an existing report comment instead of posting duplicates.
const commentMarker = "<!-- lneto-bench -->"
func main() {
var (
currentPath = flag.String("current", "", "path to `go test -bench` output (default stdin)")
outPath = flag.String("out", "", "path to write Markdown report (default stdout)")
title = flag.String("title", "Benchmark results", "report heading")
)
flag.Parse()
in := io.Reader(os.Stdin)
if *currentPath != "" {
f, err := os.Open(*currentPath)
if err != nil {
fatal(err)
}
defer f.Close()
in = f
}
results, err := parse(in)
if err != nil {
fatal(err)
}
out := io.Writer(os.Stdout)
if *outPath != "" {
f, err := os.Create(*outPath)
if err != nil {
fatal(err)
}
defer f.Close()
out = f
}
if err := render(out, *title, results); err != nil {
fatal(err)
}
}
func fatal(err error) {
fmt.Fprintln(os.Stderr, "benchci:", err)
os.Exit(1)
}
// result is the aggregated metrics for a single benchmark.
type result struct {
pkg string
name string // benchmark name including the GOMAXPROCS suffix, e.g. BenchmarkFoo-12
nsPerOp []float64
bytesPerOp []float64
allocsPerOp []float64
}
// parse reads `go test -bench -benchmem` output and groups metric samples by
// package and benchmark name. Repeated lines (from -count) accumulate samples.
func parse(r io.Reader) ([]result, error) {
sc := bufio.NewScanner(r)
sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
byKey := make(map[string]*result)
var order []string
var pkg string
for sc.Scan() {
line := sc.Text()
fields := strings.Fields(line)
if len(fields) == 0 {
continue
}
if fields[0] == "pkg:" && len(fields) >= 2 {
pkg = fields[1]
continue
}
if !strings.HasPrefix(fields[0], "Benchmark") || len(fields) < 4 {
continue
}
// fields: name iters value unit [value unit]...
name := fields[0]
if _, err := strconv.Atoi(fields[1]); err != nil {
continue // second field must be the iteration count
}
key := pkg + "\x00" + name
res := byKey[key]
if res == nil {
res = &result{pkg: pkg, name: name}
byKey[key] = res
order = append(order, key)
}
parseMetrics(res, fields[2:])
}
if err := sc.Err(); err != nil {
return nil, err
}
out := make([]result, 0, len(order))
for _, k := range order {
out = append(out, *byKey[k])
}
sort.Slice(out, func(i, j int) bool {
if out[i].pkg != out[j].pkg {
return out[i].pkg < out[j].pkg
}
return out[i].name < out[j].name
})
return out, nil
}
// parseMetrics consumes (value, unit) pairs and appends the metrics benchci
// reports on. Unknown metrics are ignored.
func parseMetrics(res *result, tokens []string) {
for i := 0; i+1 < len(tokens); i += 2 {
v, err := strconv.ParseFloat(tokens[i], 64)
if err != nil {
continue
}
switch tokens[i+1] {
case "ns/op":
res.nsPerOp = append(res.nsPerOp, v)
case "B/op":
res.bytesPerOp = append(res.bytesPerOp, v)
case "allocs/op":
res.allocsPerOp = append(res.allocsPerOp, v)
}
}
}
// median returns the median of samples. ok is false when there are no samples.
func median(samples []float64) (value float64, ok bool) {
if len(samples) == 0 {
return 0, false
}
s := append([]float64(nil), samples...)
sort.Float64s(s)
n := len(s)
if n%2 == 1 {
return s[n/2], true
}
return (s[n/2-1] + s[n/2]) / 2, true
}
func render(w io.Writer, title string, results []result) error {
bw := bufio.NewWriter(w)
fmt.Fprintf(bw, "%s\n\n", commentMarker)
fmt.Fprintf(bw, "### %s\n\n", title)
if len(results) == 0 {
fmt.Fprintln(bw, "_No benchmarks found._")
return bw.Flush()
}
fmt.Fprintln(bw, "_Timing results (`ns/op`) depend on the host CPU and are only a rough guideline. Memory results (`B/op` and `allocs/op`) are not affected._")
fmt.Fprintln(bw)
fmt.Fprintln(bw, "| Package | Benchmark | ns/op | B/op | allocs/op |")
fmt.Fprintln(bw, "|---|---|---:|---:|---:|")
for _, r := range results {
fmt.Fprintf(bw, "| %s | %s | %s | %s | %s |\n",
shortPkg(r.pkg), r.name,
formatNs(median(r.nsPerOp)),
formatCount(median(r.bytesPerOp)),
formatCount(median(r.allocsPerOp)),
)
}
return bw.Flush()
}
// shortPkg trims the well-known module prefix for readability.
func shortPkg(pkg string) string {
const prefix = "github.com/soypat/lneto/"
if pkg == "" {
return "-"
}
return strings.TrimPrefix(pkg, prefix)
}
func formatCount(v float64, ok bool) string {
if !ok {
return "-"
}
return strconv.FormatFloat(v, 'f', -1, 64)
}
// formatNs renders a ns/op value using a human-friendly time unit.
func formatNs(v float64, ok bool) string {
if !ok {
return "-"
}
switch {
case v >= 1e9:
return fmt.Sprintf("%.3f s", v/1e9)
case v >= 1e6:
return fmt.Sprintf("%.3f ms", v/1e6)
case v >= 1e3:
return fmt.Sprintf("%.3f µs", v/1e3)
default:
return fmt.Sprintf("%.3f ns", v)
}
}
+29 -14
View File
@@ -2,16 +2,16 @@ package internal
import (
"log/slog"
"os"
"runtime"
"strconv"
"sync"
"unsafe"
)
const (
LevelTrace slog.Level = slog.LevelDebug - 2
usePrintLogAllocs = true
usePrintLogAllocs = HeapAllocDebugging
)
var (
@@ -19,9 +19,13 @@ var (
lastAllocs uint64
lastMallocs uint64
allocmu sync.Mutex
allocbuf [256]byte
allocbuf [128]byte
)
func LogAttrsAndAllocs(allocmsg string, l *slog.Logger, level slog.Level, msg string, attrs ...slog.Attr) {
logAttrsAndAllocs(allocmsg, l, level, msg, attrs...)
}
func LogAllocs(msg string) {
allocmu.Lock()
runtime.ReadMemStats(&memstats)
@@ -29,23 +33,34 @@ func LogAllocs(msg string) {
allocmu.Unlock()
return
}
inc := int64(memstats.TotalAlloc) - int64(lastAllocs)
numAlloc := int64(memstats.Mallocs) - int64(lastMallocs)
free := memstats.HeapSys - memstats.HeapInuse
if usePrintLogAllocs {
// Branch used when debugheaplog enabled
print("[ALLOC] ", msg)
print(" inc=", int64(memstats.TotalAlloc)-int64(lastAllocs))
print(" n=", int64(memstats.Mallocs)-int64(lastMallocs))
print(" inc=", inc)
print(" n=", numAlloc)
print(" heap=", memstats.HeapAlloc)
print(" free=", memstats.HeapSys-memstats.HeapInuse)
print(" free=", free)
print(" tot=", memstats.TotalAlloc)
println()
} else {
n := copy(allocbuf[:], "[ALLOC] ")
n += copy(allocbuf[n:], msg)
n += copyValueInt(allocbuf[n:], "inc", int64(memstats.TotalAlloc)-int64(lastAllocs))
n += copyValueInt(allocbuf[n:], "n", int64(memstats.Mallocs)-int64(lastMallocs))
n += copyValueUint(allocbuf[n:], "heap", memstats.HeapAlloc)
n += copyValueUint(allocbuf[n:], "free", memstats.HeapSys-memstats.HeapInuse)
n += copyValueUint(allocbuf[n:], "tot", memstats.TotalAlloc)
println(unsafe.String(&allocbuf[0], n))
buf := allocbuf[:len(allocbuf)-1] // keep one character for newline.
n := copy(buf[:], "[ALLOC] ")
n += copy(buf[n:], msg)
n += copyValueInt(buf[n:], "inc", inc)
n += copyValueInt(buf[n:], "n", numAlloc)
n += copyValueUint(buf[n:], "heap", memstats.HeapAlloc)
n += copyValueUint(buf[n:], "free", free)
lastN := copyValueUint(buf[n:], "tot", memstats.TotalAlloc)
n += lastN
allocbuf[n] = '\n' // n will never be larger than len(allocbuf)-1
os.Stdout.Write(allocbuf[:n+1])
if lastN == 0 {
n2 := copy(allocbuf[:], "[WARN] ALLOC BUF OVERRUN\n")
os.Stdout.Write(allocbuf[:n2])
}
}
lastAllocs = memstats.TotalAlloc
lastMallocs = memstats.Mallocs
+5 -11
View File
@@ -4,7 +4,6 @@ package internal
import (
"log/slog"
"runtime"
"time"
"unsafe"
)
@@ -22,10 +21,13 @@ func LogEnabled(l *slog.Logger, lvl slog.Level) bool {
return true
}
func logAttrsAndAllocs(allocmsg string, l *slog.Logger, level slog.Level, msg string, attrs ...slog.Attr) {
LogAttrs(nil, level, msg, attrs...) // already logs attributes
}
func LogAttrs(_ *slog.Logger, level slog.Level, msg string, attrs ...slog.Attr) {
now := time.Now()
n := len(now.AppendFormat(timebuf[:0], timefmt))
LogAllocs(msg)
print("time=", unsafe.String(&timebuf[0], n), " ")
if level == LevelTrace {
print("TRACE ")
@@ -35,7 +37,6 @@ func LogAttrs(_ *slog.Logger, level slog.Level, msg string, attrs ...slog.Attr)
print(level.String(), " ")
}
print(msg)
for _, a := range attrs {
switch a.Value.Kind() {
case slog.KindString:
@@ -49,12 +50,5 @@ func LogAttrs(_ *slog.Logger, level slog.Level, msg string, attrs ...slog.Attr)
}
}
println()
allocmu.Lock()
runtime.ReadMemStats(&memstats)
if lastAllocs != memstats.TotalAlloc {
print("alloc increase in heaplog")
}
lastAllocs = memstats.TotalAlloc
lastMallocs = memstats.Mallocs
allocmu.Unlock()
LogAllocs(msg)
}
+6
View File
@@ -21,3 +21,9 @@ func LogAttrs(l *slog.Logger, level slog.Level, msg string, attrs ...slog.Attr)
l.LogAttrs(context.Background(), level, msg, attrs...)
}
}
func logAttrsAndAllocs(allocmsg string, l *slog.Logger, level slog.Level, msg string, attrs ...slog.Attr) {
LogAllocs(allocmsg)
LogAttrs(l, level, msg, attrs...)
LogAllocs(msg) // Should not log again unless LogAttrs allocated.
}
+12
View File
@@ -26,6 +26,18 @@ func GetIPAddr(buf []byte) (src, dst []byte, id, ipEndOff uint16, err error) {
return src, dst, id, ipEndOff, err
}
// IsMulticastIPAddr reports whether addr is an IPv4 or IPv6 multicast address.
func IsMulticastIPAddr(addr []byte) bool {
switch len(addr) {
case 4:
return addr[0]&0xf0 == 0xe0
case 16:
return addr[0] == 0xff
default:
return false
}
}
func SetIPAddrs(buf []byte, id uint16, src, dst []byte) (err error) {
var dstaddr, srcaddr []byte
version := buf[0] >> 4
+26
View File
@@ -0,0 +1,26 @@
package internal
import "testing"
func TestIsMulticastIPAddr(t *testing.T) {
tests := []struct {
name string
addr []byte
want bool
}{
{"ipv4 multicast", []byte{224, 0, 0, 1}, true},
{"ipv4 multicast upper", []byte{239, 255, 255, 255}, true},
{"ipv4 unicast", []byte{192, 0, 2, 1}, false},
{"ipv6 multicast", []byte{0xff, 0x02, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}, true},
{"ipv6 unicast", []byte{0x20, 0x01, 0x0d, 0xb8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}, false},
{"invalid length", []byte{224}, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := IsMulticastIPAddr(tt.addr); got != tt.want {
t.Fatalf("got %t; want %t", got, tt.want)
}
})
}
}
+2 -5
View File
@@ -489,11 +489,8 @@ func mutateTCPOptions(opts []byte, seed int64, variant int64) int64 {
}
case 4: // SACK with garbage block data.
if len(opts) >= 10 {
opts[0] = byte(tcp.OptSACK) // kind=5
sackLen := len(opts)
if sackLen > 34 {
sackLen = 34 // max 4 SACK blocks
}
opts[0] = byte(tcp.OptSACK) // kind=5
sackLen := min(len(opts), 34) // max 4 SACK blocks
opts[1] = byte(sackLen)
for i := 2; i < sackLen; i++ {
opts[i] = byte(seed)
+131
View File
@@ -0,0 +1,131 @@
package ltesto
import (
"sync/atomic"
"testing"
"time"
"github.com/soypat/lneto"
)
// NewSched creates a cooperative two-goroutine scheduler modelling a
// coroutine handoff: the scheduled (stack) goroutine drives the [SchedGoro] handle
// while the controlling test thread drives the [SchedDriver] handle. Splitting the
// API across two handles makes it impossible to call a goroutine-side method
// from the test thread, or vice versa.
func NewSched(t testing.TB) *Sched {
return &Sched{
t: t,
goroYieldSignal: make(chan struct{}),
goroContinueSignal: make(chan struct{}),
finishChan: make(chan error, 1),
timeout: time.Second,
}
}
// Sched is the shared state behind a [SchedGoro]/[SchedDriver] pair. It exposes no
// handoff methods directly; obtain a handle with [Sched.Goro] (for the
// scheduled goroutine) or [Sched.Driver] (for the test thread).
type Sched struct {
t testing.TB
// when stack backs off it signals here and waits until channel read or timeout.
goroYieldSignal chan struct{}
// when main goroutine is ready for more information this channel is written to to signal waiting on stack activity.
goroContinueSignal chan struct{}
finishChan chan error
finishcalled atomic.Bool
coroCalls atomic.Int32
timeout time.Duration
}
// AwaitGoroYield blocks until the coroutine suspends itself via [SchedGoro.Yield].
func (ss *Sched) AwaitGoroYield() {
select {
case <-ss.goroYieldSignal:
case <-time.After(ss.timeout):
ss.t.Fatal("timeout waiting for stack to backoff")
}
}
// AwaitGoroYieldOrDone blocks until the coroutine either parks itself via
// [SchedGoro.Yield] (returning done=false) or terminates via [SchedGoro.FinishWithErr]
// /[SchedGoro.Finish] (returning done=true and the terminal error). It lets a driver
// loop service an a-priori-unknown number of yields and still observe completion in
// the same select, avoiding the deadlock of guessing whether the goroutine will yield
// again. Do not mix with [Sched.Done] on the same scheduler.
func (ss *Sched) AwaitGoroYieldOrDone() (done bool, err error) {
select {
case <-ss.goroYieldSignal:
return false, nil
case err = <-ss.finishChan:
return true, err
case <-time.After(ss.timeout):
ss.t.Fatal("timeout waiting for stack to yield or finish")
return true, nil
}
}
// YieldToGoro wakes a coroutine parked in [SchedGoro.Yield], letting the goroutine run on.
func (ss *Sched) YieldToGoro() {
select {
case ss.goroContinueSignal <- struct{}{}:
case <-time.After(ss.timeout):
ss.t.Fatal("timeout while trying to yield to stack")
}
}
// Done returns the channel that receives the coroutine's terminal error from
// [SchedGoro.FinishWithErr]. It may only be called once.
func (ss *Sched) Done() <-chan error {
if ss.finishcalled.CompareAndSwap(false, true) {
return ss.finishChan
}
panic("Done called twice")
}
// Goro returns the handle whose methods must be called from inside the
// scheduled (stack) goroutine.
func (ss *Sched) Goro() SchedGoro {
if !ss.coroCalls.CompareAndSwap(0, 1) {
panic("only one goroutine supported for now")
}
return SchedGoro{ss: ss}
}
// SchedGoro is the coroutine-side handle of a [Sched]. Every method MUST be
// called from inside the scheduled goroutine and never from the test thread.
type SchedGoro struct{ ss *Sched }
// Yield suspends the goroutine at a backoff point and parks until the driver
// calls [SchedDriver.YieldToGoro]. Its signature satisfies [lneto.BackoffStrategy] so it
// can be passed directly as the stack's backoff strategy.
func (c SchedGoro) Yield(consecutiveBackoffs uint) time.Duration {
ss := c.ss
timeout := time.After(ss.timeout)
select {
case ss.goroYieldSignal <- struct{}{}:
case <-timeout:
ss.t.Fatal("timeout backing off, possible race condition? Multiple stacks using same backoff is unexpected pattern")
}
select {
case <-ss.goroContinueSignal:
case <-timeout:
ss.t.Fatal("timeout waiting for continue")
}
return lneto.BackoffFlagNop // backoff yield implemented on our side.
}
// FinishWithErr terminates the coroutine, handing err to the driver's [SchedDriver.Done]
// channel. It must be called at most once.
func (c SchedGoro) FinishWithErr(err error) {
ss := c.ss
if len(ss.finishChan) != 0 {
ss.t.Fatal("Coro.FinishWithErr can be called once only")
}
ss.finishChan <- err
}
// Finish is just shorthand for c.FinishWithErr(nil).
func (c SchedGoro) Finish() {
c.FinishWithErr(nil)
}
+55 -16
View File
@@ -13,6 +13,7 @@ import (
var (
ErrRingBufferFull = lneto.ErrBufferFull
errRingNoData = errors.New("lneto/ring: empty write")
errInvalidCommit = errors.New("lneto/ring: invalid commit amount")
errInvalidDiscard = errors.New("lneto/ring: invalid discard amount")
errDiscardExceeds = errors.New("lneto/ring: discard exceeds length")
errOffsetOverflow = errors.New("lneto/ring: offset too large (32 bit overflow)")
@@ -69,7 +70,7 @@ func (r *Ring) Write(b []byte) (int, error) {
n := copy(r.Buf[r.End:r.Off], b)
r.End += n
if r.End <= 0 {
panic("zero end after write") // TODO: remove panics after validation.
panic("zero end after write") // invariant: End must be >0 after writing into midFree region
}
return n, nil
} else if r.End == 0 {
@@ -87,11 +88,63 @@ func (r *Ring) Write(b []byte) (int, error) {
n += n2
}
if r.End <= 0 {
panic("zero end after write")
panic("zero end after write") // invariant: End must be >0 after appending to the tail region
}
return n, nil
}
// writeStart returns the buffer index where the next [Ring.Write] or
// [Ring.Commit] begins, matching [Ring.Write]'s placement (including wrap).
func (r *Ring) writeStart() int {
if r.End == 0 {
return r.Off // Empty: writing begins at Off.
}
if r.End == len(r.Buf) {
return 0 // Tail full: next byte wraps to the start.
}
return r.End
}
// PeekWrite stages b offset bytes past the write position (see
// [Ring.writeStart]) without advancing it, so the bytes are not yet readable; a
// later [Ring.Commit] reveals them. It reports false, writing nothing, when
// offset is negative or offset+len(b) exceeds [Ring.Free]. Used to place
// out-of-order data ahead of a gap that a normal Write later fills.
func (r *Ring) PeekWrite(b []byte, offset int) bool {
if offset < 0 || offset+len(b) > r.Free() {
return false
}
off := r.writeStart() + offset
if off >= len(r.Buf) {
off -= len(r.Buf)
}
n := copy(r.Buf[off:], b)
if n < len(b) {
copy(r.Buf, b[n:])
}
return true
}
// Commit advances the write pointer by n bytes, making readable any bytes
// previously staged with [Ring.PeekWrite]. It copies nothing and errors if n is
// not positive or exceeds [Ring.Free].
func (r *Ring) Commit(n int) error {
if n <= 0 {
return errInvalidCommit
} else if n > r.Free() {
return ErrRingBufferFull
}
if r.End == 0 {
r.End = r.Off // Match Write: commit begins at Off when empty.
}
end := r.End + n
if end > len(r.Buf) {
end -= len(r.Buf)
}
r.End = end // Never 0 here: end==len(Buf) is kept.
return nil
}
// ReadDiscard is a performance auxiliary method that performs a dummy read or no-op read
// for advancing the read pointer n bytes without actually copying data.
// This method panics if amount of bytes is more than buffered (see [Ring.Buffered]).
@@ -271,20 +324,6 @@ func (r *Ring) addOff(a, b int) int {
return result
}
func max(a, b int) int {
if a > b {
return a
}
return b
}
func min(a, b int) int {
if a < b {
return a
}
return b
}
func (r *Ring) string() string {
var b bytes.Buffer
r2 := *r
+94 -13
View File
@@ -20,7 +20,7 @@ func TestRing(t *testing.T) {
}
const data = "hello"
// Set random data and write some more and read it back.
for i := 0; i < 32; i++ {
for i := range 32 {
nfirst := max(1, rng.Intn(bufSize)/2)
nsecond := max(1, rng.Intn(bufSize)/2)
if nfirst+nsecond > bufSize {
@@ -59,7 +59,7 @@ func TestRing(t *testing.T) {
var zeros [bufSize]byte
// Set random data and write some more and read it back with ReadAt and ReadPeek and ReadDiscard.
for i := 0; i < 32; i++ {
for range 32 {
nfirst := rng.Intn(len(data))/2 + 1 // write garbage data first.
nsecond := rng.Intn(len(data))/2 + 1
if nfirst+nsecond > bufSize {
@@ -72,7 +72,7 @@ func TestRing(t *testing.T) {
content = append(content, data[:nsecond]...)
setRingData(t, r, randOff, content)
// Two-tap ReadPeek to make sure pointer not advanced.
for i := 0; i < 2; i++ {
for range 2 {
n, err = r.ReadPeek(readback[:])
if err != nil && err != io.EOF {
t.Fatal("read failed", err)
@@ -87,7 +87,7 @@ func TestRing(t *testing.T) {
}
// Two-tap ReadAt to make sure pointer not advanced.
for i := 0; i < 2; i++ {
for range 2 {
off := rng.Intn(nfirst + nsecond)
n, err = r.ReadAt(readback[:nfirst+nsecond-off], int64(off))
@@ -135,7 +135,7 @@ func TestRing2(t *testing.T) {
ringbuf := make([]byte, maxsize)
auxbuf := make([]byte, maxsize)
rng.Read(data)
for i := 0; i < ntests; i++ {
for i := range ntests {
dsize := max(rng.Intn(len(data)), 1)
if !testRing1_loopback(t, rng, ringbuf, data[:dsize], auxbuf) {
t.Fatalf("failed test %d", i)
@@ -156,7 +156,7 @@ func TestRingEmpty(t *testing.T) {
for _, isonReadEndCalled := range []bool{false, true} {
name := fmt.Sprintf("reset=%v readend=%v", isResetCalled, isonReadEndCalled)
t.Run(name, func(t *testing.T) {
for off := 0; off < bufSize+1; off++ {
for off := range bufSize + 1 {
r.End = 0
r.Off = off
if isResetCalled {
@@ -201,7 +201,7 @@ func TestRingNonEmpty(t *testing.T) {
name := fmt.Sprintf("readend=%v checkWrite=%v checkRead=%v", isonReadEndCalled, checkWrite, checkRead)
t.Run(name, func(t *testing.T) {
for end := 1; end < bufSize+1; end++ {
for off := 0; off < bufSize+1; off++ {
for off := range bufSize + 1 {
r.End = end
r.Off = off
buf := r.Buffered()
@@ -251,7 +251,7 @@ func TestRing_OffWrite(t *testing.T) {
var rawbuf, auxbuf, readback [bufSize]byte
r := &Ring{Buf: rawbuf[:]}
for n := 1; n < bufSize+1; n++ {
for off := 0; off < bufSize+1; off++ {
for off := range bufSize + 1 {
r.Off = off // Start write at off.
r.End = 0 // Reset to use no data.
for i := 0; i < n; i++ {
@@ -288,7 +288,7 @@ func TestRing_TwoWrite(t *testing.T) {
var rawbuf, auxbuf, readback [bufSize]byte
r := &Ring{Buf: rawbuf[:]}
for i := 0; i < 1024; i++ {
for range 1024 {
n1 := rng.Intn(bufSize-1) + 1 // leave space for one more write
n2 := rng.Intn(bufSize-n1) + 1
off := rng.Intn(bufSize + 1)
@@ -319,8 +319,8 @@ func TestRingOverwrite(t *testing.T) {
const bufSize = 8
var rawbuf, auxbuf [bufSize]byte
r := &Ring{Buf: rawbuf[:]}
for off := 0; off < bufSize+1; off++ {
for buf := 0; buf < bufSize+1; buf++ {
for off := range bufSize + 1 {
for buf := range bufSize + 1 {
setRingData(t, r, off, rawbuf[:buf])
// Select write size overwriting data.
for osz := bufSize - buf + 1; osz < bufSize+1; osz++ {
@@ -403,7 +403,7 @@ func TestRing_findcrash(t *testing.T) {
rng := rand.New(rand.NewSource(0))
data := make([]byte, maxsize)
for i := 0; i < ntests; i++ {
for i := range ntests {
free := r.Free()
if free < 0 {
t.Fatal("free < 0")
@@ -448,7 +448,7 @@ func TestRingFreeLimited(t *testing.T) {
rng := rand.New(rand.NewSource(1))
buffer := make([]byte, n)
r := Ring{Buf: buffer}
for itest := 0; itest < 100000; itest++ {
for itest := range 100000 {
r.Off = rng.Intn(n)
r.End = rng.Intn(n + 1)
limit := rng.Intn(n)
@@ -580,3 +580,84 @@ func canonRing(r *Ring) {
r.onReadEnd(1)
}
}
// TestRingPeekWriteCommit verifies that bytes staged ahead of a gap with
// PeekWrite become readable in order once the gap is filled and committed.
func TestRingPeekWriteCommit(t *testing.T) {
r := &Ring{Buf: make([]byte, 16)}
// Stage "BBBB" 4 bytes ahead of the write position (the gap).
if !r.PeekWrite([]byte("BBBB"), 4) {
t.Fatal("PeekWrite should fit")
}
// Staged bytes are not yet readable.
if r.Buffered() != 0 {
t.Fatalf("staged bytes must not be readable, buffered=%d", r.Buffered())
}
// Fill the gap with a normal write, then commit the staged tail.
if _, err := r.Write([]byte("AAAA")); err != nil {
t.Fatal("gap write:", err)
}
if err := r.Commit(4); err != nil {
t.Fatal("commit:", err)
}
got := make([]byte, 8)
n, err := r.Read(got)
if err != nil {
t.Fatal("read:", err)
}
if string(got[:n]) != "AAAABBBB" {
t.Fatalf("read %q, want AAAABBBB", got[:n])
}
testRingSanity(t, r)
}
// TestRingPeekWriteWrap exercises PeekWrite/Commit when the staged region wraps
// across the end of the backing buffer. Existing data at Off=2,End=6 puts the
// write position at index 6, so a 2-byte gap fills indices 6,7 and the staged
// tail wraps to indices 0,1.
func TestRingPeekWriteWrap(t *testing.T) {
r := &Ring{Buf: make([]byte, 8)}
setRingData(t, r, 2, []byte("WXYZ")) // Off=2, End=6, 4 bytes buffered.
if r.writeStart() != 6 {
t.Fatalf("writeStart=%d, want 6", r.writeStart())
}
// Stage "CD" 2 bytes ahead of the write position (6) → wraps to indices 0,1.
if !r.PeekWrite([]byte("CD"), 2) {
t.Fatal("PeekWrite (wrap) should fit")
}
// Fill the 2-byte gap at indices 6,7, then commit the wrapped tail.
if _, err := r.Write([]byte("AB")); err != nil {
t.Fatal("gap write:", err)
}
if err := r.Commit(2); err != nil {
t.Fatal("commit:", err)
}
got := make([]byte, 8)
n, err := r.Read(got)
if err != nil {
t.Fatal("read:", err)
}
if string(got[:n]) != "WXYZABCD" {
t.Fatalf("read %q, want WXYZABCD", got[:n])
}
testRingSanity(t, r)
}
func TestRingPeekWriteRejects(t *testing.T) {
r := &Ring{Buf: make([]byte, 8)}
if r.PeekWrite([]byte("toolong!!"), 0) {
t.Error("PeekWrite must reject data larger than the buffer")
}
if r.PeekWrite([]byte("data"), 5) { // 5+4 > 8 free.
t.Error("PeekWrite must reject offset+len beyond free space")
}
if r.PeekWrite([]byte("x"), -1) {
t.Error("PeekWrite must reject negative offset")
}
if err := r.Commit(0); err == nil {
t.Error("Commit(0) must error")
}
if err := r.Commit(9); err == nil {
t.Error("Commit beyond free must error")
}
}
+1 -1
View File
@@ -32,7 +32,7 @@ func DeleteZeroed[T comparable](a []T) []T {
var z T
off := 0
deleted := false
for i := 0; i < len(a); i++ {
for i := range a {
if a[i] != z {
if deleted {
a[off] = a[i]
+19
View File
@@ -0,0 +1,19 @@
package internal
// IntLen returns the number of bytes [strconv.AppendInt] emits for value in the
// given base, including a leading minus sign for negatives. Lets callers size a
// buffer, or test whether a value fits an existing slot, before writing a byte.
// base must be in the range 2..36, as accepted by [strconv.AppendInt].
func IntLen(value int64, base int) int {
n := 1
u := uint64(value)
if value < 0 {
n++ // Leading minus sign.
u = -u // Two's-complement magnitude; correct even for math.MinInt64.
}
for u >= uint64(base) {
u /= uint64(base)
n++
}
return n
}
+214
View File
@@ -0,0 +1,214 @@
package internet
import (
"bytes"
"errors"
"io"
"math/rand"
"runtime"
"sync/atomic"
"testing"
"time"
"github.com/soypat/lneto/tcp"
)
// TestConn_ConcurrentCloseDoesNotDropWrite is a regression test for issue #82:
// when one goroutine has a Write in progress (blocked because the TX buffer is
// full) and another goroutine calls Close, the in-flight write data must not be
// silently dropped. With the atomic write-lock, Close waits for the in-progress
// write to finish queueing all of its data before tearing the connection down.
//
// The scenario is made deterministic by filling the TX buffer before any packets
// are exchanged: the writer blocks holding the write lock, Close is then issued
// (and must wait), and only afterwards is the packet pump started so the buffer
// can drain and the writer can run to completion.
func TestConn_ConcurrentCloseDoesNotDropWrite(t *testing.T) {
rng := rand.New(rand.NewSource(1))
var sbCl, sbSv StackIPv4
var connCl, connSv tcp.Conn
setupClientServerEstablished(t, rng, &sbCl, &sbSv, &connCl, &connSv)
// Payload several times larger than the 2048-byte TX buffer so the writer
// must block waiting for the buffer to drain across many segments.
payload := make([]byte, 4*2048)
for i := range payload {
payload[i] = byte(i)
}
// Watchdog: break any deadlock so a regression fails fast instead of hanging.
watchdog := time.AfterFunc(10*time.Second, func() {
t.Error("test timed out: Close likely blocked waiting on a stuck write")
connCl.Abort()
connSv.Abort()
})
defer watchdog.Stop()
// Writer (G1): writes the whole payload. Blocks once the TX buffer fills.
type writeResult struct {
n int
err error
}
writeDone := make(chan writeResult, 1)
go func() {
n, err := connCl.Write(payload)
writeDone <- writeResult{n, err}
}()
// Wait until the writer has filled the TX buffer and is blocked. At this
// point it owns the write lock, reproducing "Write in progress".
for connCl.FreeOutput() != 0 {
runtime.Gosched()
}
// Close (G2): issued while the write is in progress. Must wait for the
// writer to drain instead of dropping its data.
closeDone := make(chan error, 1)
go func() {
closeDone <- connCl.Close()
}()
// Reader: drains the server side until EOF, accumulating everything received.
readDone := make(chan []byte, 1)
go func() {
got := make([]byte, 0, len(payload))
rbuf := make([]byte, 1024)
for {
n, err := connSv.Read(rbuf)
got = append(got, rbuf[:n]...)
if err != nil {
break
}
}
readDone <- got
}()
// Packet pump: only now do we move packets between the two stacks, letting
// the buffer drain so the blocked writer can finish.
var stop atomic.Bool
pumpStopped := make(chan struct{})
go func() {
defer close(pumpStopped)
var buf [2048]byte
for !stop.Load() {
progressed := false
if n, err := sbCl.Encapsulate(buf[:], 0, 0); err == nil && n > 0 {
_ = sbSv.Demux(buf[:n], 0)
progressed = true
}
if n, err := sbSv.Encapsulate(buf[:], 0, 0); err == nil && n > 0 {
_ = sbCl.Demux(buf[:n], 0)
progressed = true
}
if !progressed {
runtime.Gosched()
}
}
}()
wr := <-writeDone
if wr.err != nil {
t.Errorf("issue #82: Write returned error %v; concurrent Close dropped in-flight data", wr.err)
}
if wr.n != len(payload) {
t.Errorf("issue #82: Write wrote %d of %d bytes; concurrent Close truncated the write", wr.n, len(payload))
}
if err := <-closeDone; err != nil {
t.Errorf("Close returned error: %v", err)
}
got := <-readDone
stop.Store(true)
<-pumpStopped
if len(got) != len(payload) {
t.Fatalf("server received %d of %d bytes; data was dropped by concurrent Close", len(got), len(payload))
}
if !bytes.Equal(got, payload) {
t.Fatal("server received corrupted data")
}
}
// TestConn_ConcurrentWritesSerialize verifies that the atomic write-lock
// serializes concurrent Write calls on the same connection so their payloads are
// not interleaved on the wire, and that all bytes from both writers are delivered.
func TestConn_ConcurrentWritesSerialize(t *testing.T) {
rng := rand.New(rand.NewSource(2))
var sbCl, sbSv StackIPv4
var connCl, connSv tcp.Conn
setupClientServerEstablished(t, rng, &sbCl, &sbSv, &connCl, &connSv)
const chunk = 1500
a := bytes.Repeat([]byte{0xAA}, chunk)
b := bytes.Repeat([]byte{0xBB}, chunk)
watchdog := time.AfterFunc(10*time.Second, func() {
t.Error("test timed out")
connCl.Abort()
connSv.Abort()
})
defer watchdog.Stop()
var stop atomic.Bool
pumpStopped := make(chan struct{})
go func() {
defer close(pumpStopped)
var buf [2048]byte
for !stop.Load() {
progressed := false
if n, err := sbCl.Encapsulate(buf[:], 0, 0); err == nil && n > 0 {
_ = sbSv.Demux(buf[:n], 0)
progressed = true
}
if n, err := sbSv.Encapsulate(buf[:], 0, 0); err == nil && n > 0 {
_ = sbCl.Demux(buf[:n], 0)
progressed = true
}
if !progressed {
runtime.Gosched()
}
}
}()
writeErr := make(chan error, 2)
writer := func(b []byte) {
n, err := connCl.Write(b)
if err == nil && n != len(b) {
err = io.ErrShortWrite
}
writeErr <- err
}
go writer(a)
go writer(b)
got := make([]byte, 0, 2*chunk)
rbuf := make([]byte, 1024)
for len(got) < 2*chunk {
n, err := connSv.Read(rbuf)
got = append(got, rbuf[:n]...)
if err != nil && !errors.Is(err, io.EOF) {
break
}
}
for range 2 {
if err := <-writeErr; err != nil {
t.Errorf("concurrent write failed: %v", err)
}
}
stop.Store(true)
<-pumpStopped
if len(got) != 2*chunk {
t.Fatalf("received %d bytes, want %d", len(got), 2*chunk)
}
// Serialized writes mean one chunk's bytes appear fully before the other's;
// the boundary is a single transition, never interleaved.
transitions := 0
for i := 1; i < len(got); i++ {
if got[i] != got[i-1] {
transitions++
}
}
if transitions != 1 {
t.Fatalf("expected exactly one A/B boundary (serialized writes), got %d transitions", transitions)
}
}
+31
View File
@@ -7,6 +7,7 @@ import (
"slices"
"github.com/soypat/lneto"
"github.com/soypat/lneto/internal"
)
// node is a concrete StackNode as stored in Stacks. Methods are devirtualized for performance benefits, especially on TinyGo.
@@ -159,6 +160,8 @@ func (h *handlers) encapsulateNode(node *node, buf []byte, offsetIP, offsetThisF
err = nil // CLOSE error handled gracefully by deleting node.
node = nil // Node is destroyed in tryHandleError and invalidated.
}
// TODO(soypat): We have fuzz tests in place, maybe we can start returning the error up the chain to catch invalid settings at application level so that users don't have to have logs in place to understand invalid config/buffer size.
// Encapsulate should only fail with error on programmer errors.
if n > 0 {
return n, err
} else if err != nil {
@@ -229,3 +232,31 @@ func incLim(v, max int) int {
}
return v
}
type logger struct {
log *slog.Logger
}
func (l logger) error(msg string, attrs ...slog.Attr) {
internal.LogAttrs(l.log, slog.LevelError, msg, attrs...)
}
func (l logger) info(msg string, attrs ...slog.Attr) {
internal.LogAttrs(l.log, slog.LevelInfo, msg, attrs...)
}
func (l logger) warn(msg string, attrs ...slog.Attr) {
internal.LogAttrs(l.log, slog.LevelWarn, msg, attrs...)
}
func (l logger) debug(msg string, attrs ...slog.Attr) {
internal.LogAttrs(l.log, slog.LevelDebug, msg, attrs...)
}
func (l logger) trace(msg string, attrs ...slog.Attr) {
internal.LogAttrs(l.log, internal.LevelTrace, msg, attrs...)
}
const enableAllocLog = internal.HeapAllocDebugging
func debugLog(msg string) {
if enableAllocLog {
internal.LogAllocs(msg)
}
}
+215 -24
View File
@@ -4,22 +4,22 @@ package pcap
import (
"encoding/binary"
"errors"
"fmt"
"math"
"strconv"
"strings"
"unsafe"
"github.com/soypat/lneto"
"github.com/soypat/lneto/arp"
"github.com/soypat/lneto/dhcpv4"
"github.com/soypat/lneto/dhcp/dhcpv4"
"github.com/soypat/lneto/dns"
"github.com/soypat/lneto/dns/mdns"
"github.com/soypat/lneto/ethernet"
"github.com/soypat/lneto/http/httpraw"
"github.com/soypat/lneto/internal"
"github.com/soypat/lneto/ipv4"
"github.com/soypat/lneto/ipv4/icmpv4"
"github.com/soypat/lneto/ipv6"
"github.com/soypat/lneto/mdns"
"github.com/soypat/lneto/ntp"
"github.com/soypat/lneto/tcp"
"github.com/soypat/lneto/udp"
@@ -384,14 +384,14 @@ func (pc *PacketBreakdown) CaptureICMPv4(dst []Frame, pkt []byte, bitOffset int)
}
finfo := reclaimFrame(&dst, "ICMP", bitOffset, baseICMPv4Fields[:])
tp := ifrm.Type()
// Add type-specific fields.
switch ifrm.Type() {
switch tp {
case icmpv4.TypeEcho, icmpv4.TypeEchoReply:
finfo.Fields = append(finfo.Fields, icmpv4EchoFields[:]...)
if len(icmpData) > 8 {
finfo.Fields = append(finfo.Fields, FrameField{
Name: "Data",
Name: tp.String(),
Class: FieldClassPayload,
FrameBitOffset: 8 * octet,
BitLength: (len(icmpData) - 8) * octet,
@@ -439,26 +439,202 @@ func (pc *PacketBreakdown) CaptureDNS(dst []Frame, pkt []byte, bitOffset int) ([
}
dnsData := pkt[bitOffset/8:]
pc.dmsg.LimitResourceDecoding(4, 4, 4, 4)
off, incomplete, err := pc.dmsg.Decode(dnsData)
_, incomplete, err := pc.dmsg.Decode(dnsData)
if err != nil && !incomplete {
return dst, err
}
debuglog("pcap:dns-decode")
finfo := reclaimFrame(&dst, "DNS", bitOffset, nil)
hdr, _ := dns.NewFrame(dnsData)
finfo := reclaimFrame(&dst, "DNS", bitOffset, baseDNSFields[:])
if incomplete {
finfo.Errors = append(finfo.Errors, ErrLimitExceeded)
}
field := internal.SliceReclaim(&finfo.Fields)
*field = FrameField{
Name: "Data",
FrameBitOffset: 0,
BitLength: int(off) * octet,
SubFields: field.SubFields[:0], // Reuse subfields.
if pc.SubfieldLimit <= 0 {
debuglog("pcap:dns-done")
return dst, nil
}
wireOff := dns.SizeHeader
// Questions section: walk all QDCount wire records to keep wireOff correct,
// but only emit SubFields for the decoded ones.
nq := int(hdr.QDCount())
if nq > 0 {
sectionStart := wireOff
qfield := internal.SliceReclaim(&finfo.Fields)
*qfield = FrameField{Name: "Questions", Class: fieldClassDNSResource, SubFields: qfield.SubFields[:0], Flags: FlagContainer}
decoded := pc.dmsg.Questions
for i := range nq {
nameStart := wireOff
wireOff, err = dnsSkipName(dnsData, wireOff)
if err != nil {
break
}
nameEnd := wireOff
wireOff += 4 // Type(2) + Class(2)
if i < len(decoded) && len(qfield.SubFields)+3 <= pc.SubfieldLimit {
qfield.SubFields = append(qfield.SubFields, FrameField{
Name: "Name",
Class: FieldClassDNSName,
FrameBitOffset: nameStart * octet,
BitLength: (nameEnd - nameStart) * octet,
}, FrameField{
Name: "Type",
Class: FieldClassOperation,
FrameBitOffset: nameEnd * octet,
BitLength: 2 * octet,
}, FrameField{
Name: "Class",
Class: FieldClassOperation,
FrameBitOffset: (nameEnd + 2) * octet,
BitLength: 2 * octet,
})
}
}
qfield.FrameBitOffset = sectionStart * octet
qfield.BitLength = (wireOff - sectionStart) * octet
}
wireOff = pc.appendDNSResources(finfo, "Answers", dnsData, pc.dmsg.Answers, int(hdr.ANCount()), wireOff)
wireOff = pc.appendDNSResources(finfo, "Authorities", dnsData, pc.dmsg.Authorities, int(hdr.NSCount()), wireOff)
wireOff = pc.appendDNSResources(finfo, "Additionals", dnsData, pc.dmsg.Additionals, int(hdr.ARCount()), wireOff)
_ = wireOff
debuglog("pcap:dns-done")
return dst, nil
}
// appendDNSResources adds a section FrameField with per-record SubFields to finfo.
// It walks all `total` wire records to keep wireOff accurate, but only emits SubFields
// for the decoded slice entries while nFields < pc.SubfieldLimit.
func (pc *PacketBreakdown) appendDNSResources(finfo *Frame, name string, dnsData []byte, decoded []dns.Resource, total, wireOff int) int {
if total == 0 {
return wireOff
}
var err error
sectionStart := wireOff
rfield := internal.SliceReclaim(&finfo.Fields)
*rfield = FrameField{Name: name, Class: fieldClassDNSResource, SubFields: rfield.SubFields[:0], Flags: FlagContainer}
for i := range total {
nameStart := wireOff
wireOff, err = dnsSkipName(dnsData, wireOff)
if err != nil || wireOff+10 > len(dnsData) {
break
}
nameEnd := wireOff
dataLen := int(binary.BigEndian.Uint16(dnsData[nameEnd+8:]))
wireOff += 10 + dataLen // Type(2)+Class(2)+TTL(4)+Length(2)+Data
if wireOff > len(dnsData) {
break
}
if i < len(decoded) && len(rfield.SubFields)+6 <= pc.SubfieldLimit {
rfield.SubFields = append(rfield.SubFields, FrameField{
Name: "Name",
Class: FieldClassDNSName,
FrameBitOffset: nameStart * octet,
BitLength: (nameEnd - nameStart) * octet,
}, FrameField{
Name: "Type",
Class: FieldClassOperation,
FrameBitOffset: nameEnd * octet,
BitLength: 2 * octet,
}, FrameField{
Name: "Class",
Class: FieldClassOperation,
FrameBitOffset: (nameEnd + 2) * octet,
BitLength: 2 * octet,
}, FrameField{
Name: "TTL",
Class: FieldClassID,
FrameBitOffset: (nameEnd + 4) * octet,
BitLength: 4 * octet,
}, FrameField{
Name: "Length",
Class: FieldClassSize,
FrameBitOffset: (nameEnd + 8) * octet,
BitLength: 2 * octet,
}, FrameField{
Name: "Data",
Class: FieldClassAddress,
FrameBitOffset: (nameEnd + 10) * octet,
BitLength: dataLen * octet,
})
}
}
rfield.FrameBitOffset = sectionStart * octet
rfield.BitLength = (wireOff - sectionStart) * octet
if err != nil {
finfo.Errors = append(finfo.Errors, err)
}
return wireOff
}
// dnsAppendDottedName walks a DNS name in wire format starting at off within
// dnsMsg, follows compression pointers, and appends the dotted representation
// (e.g. "example.com") to dst. Returns dst unchanged on error.
func dnsAppendDottedName(dst, dnsMsg []byte, off int) []byte {
// TODO: somehow move this to dns package. dns.Name.Append ? dns.Name is the wire format...
first := true
for ptr := 0; off < len(dnsMsg) && ptr <= 10; {
c := dnsMsg[off]
off++
switch c & 0xc0 {
case 0x00:
if c == 0 {
return dst // null terminator: done
}
end := off + int(c)
if end > len(dnsMsg) {
return dst
}
if !first {
dst = append(dst, '.')
}
dst = append(dst, dnsMsg[off:end]...)
off = end
first = false
case 0xc0:
if off >= len(dnsMsg) {
return dst
}
off = int(c&0x3f)<<8 | int(dnsMsg[off])
ptr++ // guard against pointer loops
default:
return dst // reserved label type
}
}
return dst
}
// dnsSkipName advances off past a DNS name in wire format without allocating.
// Compression pointers are followed and consume 2 bytes, terminating the name.
func dnsSkipName(b []byte, off int) (int, error) {
for {
if off >= len(b) {
return off, lneto.ErrTruncatedFrame
}
c := b[off]
off++
switch c & 0xc0 {
case 0x00:
if c == 0 {
return off, nil // null terminator
}
off += int(c) // skip label bytes
if off > len(b) {
return off, lneto.ErrTruncatedFrame
}
case 0xc0:
if off >= len(b) {
return off, lneto.ErrTruncatedFrame
}
return off + 1, nil // compression pointer: 2 bytes total, always terminal
default:
return off, lneto.ErrInvalidField // reserved label type
}
}
}
func (pc *PacketBreakdown) CaptureNTP(dst []Frame, pkt []byte, bitOffset int) ([]Frame, error) {
if bitOffset%8 != 0 {
return dst, errNotByteAligned
@@ -592,6 +768,11 @@ func httpBodyClass(contentType, body []byte) FieldClass {
return FieldClassText
}
// maxCapturedHeaderFields bounds the field table a captured HTTP header is
// parsed into. Captures are read once and discarded, so the table is sized for
// a realistic request rather than for whatever the capture happens to hold.
const maxCapturedHeaderFields = 64
func (pc *PacketBreakdown) CaptureHTTP(dst []Frame, pkt []byte, bitOffset int) ([]Frame, error) {
debuglog("pcap:http:start")
const httpProtocol = "HTTP"
@@ -601,10 +782,10 @@ func (pc *PacketBreakdown) CaptureHTTP(dst []Frame, pkt []byte, bitOffset int) (
const asResponse = true
const asRequest = false
httpData := pkt[bitOffset/8:]
pc.hdr.Reset(httpData)
pc.hdr.Reset(httpData, maxCapturedHeaderFields)
err := pc.hdr.Parse(asResponse)
if err != nil {
pc.hdr.Reset(httpData)
pc.hdr.Reset(httpData, 0) // Field table already sized, reuse it.
err = pc.hdr.Parse(asRequest) // try as request.
}
if err != nil {
@@ -656,6 +837,9 @@ type Flags uint32
const (
FlagRightAligned Flags = 1 << iota
FlagLegacy
// FlagContainer is used for [FrameField]s whose SubFields represent
// the entirety of the FrameField's data. i.e: DNS Questions/Answers.
FlagContainer
)
func (ff Flags) IsLegacy() bool { return ff&FlagLegacy != 0 }
@@ -771,7 +955,7 @@ func appendField(dst, pkt []byte, fieldBitStart, bitlen int, rightAligned bool)
if octets+octetsStart+1 > len(pkt) {
return dst, lneto.ErrShortBuffer
}
for i := 0; i < octets; i++ {
for i := range octets {
b := (pkt[octetsStart+i] & mask) << (8 - firstBitOffset)
b |= pkt[octetsStart+i+1] >> firstBitOffset
dst = append(dst, b)
@@ -801,18 +985,22 @@ func (frm Frame) String() string {
func (frm Frame) AppendString(b []byte) []byte {
bitlen := frm.LenBits()
b = fmt.Appendf(b, "%s", frm.Protocol)
b = append(b, frm.Protocol...)
if bitlen%8 == 0 {
b = fmt.Appendf(b, " len=%d", bitlen/8)
b = append(b, " len="...)
b = strconv.AppendInt(b, int64(bitlen/8), 10)
} else {
b = fmt.Appendf(b, " bits=%d", bitlen)
b = append(b, " bits="...)
b = strconv.AppendInt(b, int64(bitlen), 10)
}
iopt, err := frm.FieldByClass(FieldClassOptions)
if err == nil {
b = fmt.Appendf(b, " optlen=%d", (frm.Fields[iopt].BitLength+7)/8)
b = append(b, " optlen="...)
b = strconv.AppendInt(b, int64((frm.Fields[iopt].BitLength+7)/8), 10)
}
for _, err := range frm.Errors {
b = fmt.Appendf(b, " %s", err.Error())
b = append(b, ' ')
b = append(b, err.Error()...)
}
return b
}
@@ -826,10 +1014,10 @@ func (frm Frame) LenBits() (totalBitlen int) {
func (ff FrameField) String() string {
if ff.Class == FieldClassPayload {
return fmt.Sprintf("Payload len=%d", ff.BitLength/8)
return "Payload len=" + strconv.Itoa(ff.BitLength/8)
}
if ff.Name != "" {
return fmt.Sprintf("%s (%s)", ff.Name, ff.Class.String())
return ff.Name + " (" + ff.Class.String() + ")"
}
return ff.Class.String()
}
@@ -855,10 +1043,13 @@ const (
FieldClassBinaryText // binary-text
FieldClassOperation // op
FieldClassTimestamp // timestamp
FieldClassDNSName // dns name
)
const octet = 8
const fieldClassDNSResource = fieldClassUndefined
var baseEthernetFields = [...]FrameField{
{
Class: FieldClassDst,
+209
View File
@@ -0,0 +1,209 @@
package pcap
import (
"testing"
"time"
"github.com/soypat/lneto"
"github.com/soypat/lneto/dhcp/dhcpv4"
"github.com/soypat/lneto/dns"
"github.com/soypat/lneto/ethernet"
"github.com/soypat/lneto/ipv4"
"github.com/soypat/lneto/udp"
)
const benchSubfieldLimit = 32
// buildDHCPPacket builds an Ethernet+IPv4+UDP+DHCPv4 Discover packet, exercising
// the option-heavy DHCP path (hostname, client id, requested address, param list).
func buildDHCPPacket(b testing.TB) []byte {
const (
ethSize = 14
ipv4Size = 20
udpSize = 8
)
pkt := make([]byte, 600)
efrm, _ := ethernet.NewFrame(pkt)
*efrm.DestinationHardwareAddr() = [6]byte{0xff, 0xff, 0xff, 0xff, 0xff, 0xff}
*efrm.SourceHardwareAddr() = [6]byte{0xde, 0xad, 0xbe, 0xef, 0xca, 0xfe}
efrm.SetEtherType(ethernet.TypeIPv4)
ifrm, _ := ipv4.NewFrame(pkt[ethSize:])
ifrm.SetVersionAndIHL(4, 5)
ifrm.SetID(0x1234)
ifrm.SetFlags(0x4000)
ifrm.SetTTL(64)
ifrm.SetProtocol(lneto.IPProtoUDP)
ufrm, _ := udp.NewFrame(pkt[ethSize+ipv4Size:])
ufrm.SetSourcePort(dhcpv4.DefaultClientPort)
ufrm.SetDestinationPort(dhcpv4.DefaultServerPort)
var cl dhcpv4.Client
err := cl.BeginRequest(0xdeadbeef, dhcpv4.RequestConfig{
RequestedAddr: [4]byte{192, 168, 1, 100},
ClientHardwareAddr: [6]byte{0xde, 0xad, 0xbe, 0xef, 0xca, 0xfe},
Hostname: "myhost",
ClientID: "lneto-test",
})
if err != nil {
b.Fatal("begin request:", err)
}
dhcpLen, err := cl.Encapsulate(pkt, ethSize, ethSize+ipv4Size+udpSize)
if err != nil {
b.Fatal("encapsulate:", err)
}
totalLen := ipv4Size + udpSize + dhcpLen
ifrm.SetTotalLength(uint16(totalLen))
ufrm.SetLength(uint16(udpSize + dhcpLen))
ifrm.SetCRC(ifrm.CalculateHeaderCRC())
return pkt[:ethSize+totalLen]
}
// buildDNSPacket builds an Ethernet+IPv4+UDP+DNS message with multiple questions
// and answers, exercising name encoding and resource record rendering.
func buildDNSPacket(b testing.TB) []byte {
const (
ethSize = 14
ipv4Size = 20
udpSize = 8
)
var msg dns.Message
msg.Questions = []dns.Question{
{Name: dns.MustNewName("example.com"), Type: dns.TypeA, Class: dns.ClassINET},
{Name: dns.MustNewName("temu.com"), Type: dns.TypeAAAA, Class: dns.ClassANY},
}
msg.Answers = []dns.Resource{
dns.NewResource(dns.MustNewName("abc.com"), dns.TypeALL, dns.ClassANY, 64, []byte{10, 0, 11, 1}),
dns.NewResource(dns.MustNewName("123.com"), dns.TypeA, dns.ClassINET, 64, []byte{20, 0, 22, 2}),
}
dnsPayload, err := msg.AppendTo(nil, 0x1234, dns.NewClientHeaderFlags(dns.OpCodeQuery, true))
if err != nil {
b.Fatal("dns encode:", err)
}
pkt := make([]byte, ethSize+ipv4Size+udpSize+len(dnsPayload))
efrm, _ := ethernet.NewFrame(pkt)
*efrm.DestinationHardwareAddr() = [6]byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x01}
*efrm.SourceHardwareAddr() = [6]byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x02}
efrm.SetEtherType(ethernet.TypeIPv4)
ifrm, _ := ipv4.NewFrame(pkt[ethSize:])
ifrm.SetVersionAndIHL(4, 5)
ifrm.SetTTL(64)
ifrm.SetProtocol(lneto.IPProtoUDP)
ifrm.SetTotalLength(uint16(ipv4Size + udpSize + len(dnsPayload)))
ifrm.SetCRC(ifrm.CalculateHeaderCRC())
ufrm, _ := udp.NewFrame(pkt[ethSize+ipv4Size:])
ufrm.SetSourcePort(58200)
ufrm.SetDestinationPort(dns.ServerPort)
ufrm.SetLength(uint16(udpSize + len(dnsPayload)))
copy(pkt[ethSize+ipv4Size+udpSize:], dnsPayload)
return pkt
}
func configureBenchFormatter(f *Formatter) {
f.SubfieldLimit = benchSubfieldLimit
f.FrameSep = "\n"
f.FieldSep = "; "
f.SubfieldSep = "\n\t"
}
// BenchmarkPcap measures the decode, format, and decode+format (roundtrip) phases
// separately for the string-heavy DHCP and DNS frames. Run with -benchmem for
// per-phase allocs/op.
func BenchmarkPcap(b *testing.B) {
cases := []struct {
name string
pkt []byte
}{
{"DHCP", buildDHCPPacket(b)},
{"DNS", buildDNSPacket(b)},
}
for _, tc := range cases {
b.Run(tc.name, func(b *testing.B) {
b.Run("decode", func(b *testing.B) {
var pb PacketBreakdown
pb.SubfieldLimit = benchSubfieldLimit
var frames []Frame
b.ReportAllocs()
b.ResetTimer()
for b.Loop() {
frames, _ = pb.CaptureEthernet(frames[:0], tc.pkt, 0)
}
})
b.Run("format", func(b *testing.B) {
var pb PacketBreakdown
pb.SubfieldLimit = benchSubfieldLimit
frames, err := pb.CaptureEthernet(nil, tc.pkt, 0)
if err != nil {
b.Fatal(err)
}
var f Formatter
configureBenchFormatter(&f)
var buf []byte
b.ReportAllocs()
b.ResetTimer()
for b.Loop() {
buf, _ = f.FormatFrames(buf[:0], frames, tc.pkt)
}
})
b.Run("roundtrip", func(b *testing.B) {
var pb PacketBreakdown
pb.SubfieldLimit = benchSubfieldLimit
var f Formatter
configureBenchFormatter(&f)
var frames []Frame
var buf []byte
b.ReportAllocs()
b.ResetTimer()
for b.Loop() {
frames, _ = pb.CaptureEthernet(frames[:0], tc.pkt, 0)
buf, _ = f.FormatFrames(buf[:0], frames, tc.pkt)
}
})
})
}
}
// BenchmarkPcapPhases runs decode+format in a single benchmark loop while
// reporting per-phase wall time via custom metrics (decode-ns/op, format-ns/op).
// decode-ns/op + format-ns/op approximates ns/op minus time.Now overhead.
// Per-phase allocs are not split here (ReadMemStats is STW and skews timing);
// use BenchmarkPcap's decode/format sub-benchmarks with -benchmem for that.
func BenchmarkPcapPhases(b *testing.B) {
cases := []struct {
name string
pkt []byte
}{
{"DHCP", buildDHCPPacket(b)},
{"DNS", buildDNSPacket(b)},
}
for _, tc := range cases {
b.Run(tc.name, func(b *testing.B) {
var pb PacketBreakdown
pb.SubfieldLimit = benchSubfieldLimit
var f Formatter
configureBenchFormatter(&f)
var frames []Frame
var buf []byte
var decNs, fmtNs int64
b.ResetTimer()
for b.Loop() {
t0 := time.Now()
frames, _ = pb.CaptureEthernet(frames[:0], tc.pkt, 0)
t1 := time.Now()
buf, _ = f.FormatFrames(buf[:0], frames, tc.pkt)
t2 := time.Now()
decNs += t1.Sub(t0).Nanoseconds()
fmtNs += t2.Sub(t1).Nanoseconds()
}
b.ReportMetric(float64(decNs)/float64(b.N), "decode-ns/op")
b.ReportMetric(float64(fmtNs)/float64(b.N), "format-ns/op")
})
}
}
+90 -7
View File
@@ -9,7 +9,8 @@ import (
"testing"
"github.com/soypat/lneto"
"github.com/soypat/lneto/dhcpv4"
"github.com/soypat/lneto/dhcp/dhcpv4"
"github.com/soypat/lneto/dns"
"github.com/soypat/lneto/ethernet"
"github.com/soypat/lneto/http/httpraw"
"github.com/soypat/lneto/internal/ltesto"
@@ -35,7 +36,7 @@ func makeHttpPayload(body string) ([]byte, error) {
}
func TestCap(t *testing.T) {
const mtu = 1500
const mtu = ethernet.MaxMTU
const httpBody = "{200,ok}"
var buf [mtu]byte
var gen ltesto.PacketGen
@@ -486,9 +487,91 @@ func ExampleFormatter_dhcp() {
// (Hostname string)="myhost"
}
func writeOpt(dst []byte, opt dhcpv4.OptNum, data ...byte) int {
dst[0] = byte(opt)
dst[1] = byte(len(data))
copy(dst[2:], data)
return 2 + len(data)
func ExampleFormatter_dns() {
const (
ethSize = 14
ipv4Size = 20
udpSize = 8
)
// Build a DNS query for "example.com" A record.
var msg dns.Message
msg.Questions = []dns.Question{
{Name: dns.MustNewName("example.com"), Type: dns.TypeA, Class: dns.ClassINET},
{Name: dns.MustNewName("temu.com"), Type: dns.TypeAAAA, Class: dns.ClassANY},
}
msg.Answers = []dns.Resource{
dns.NewResource(dns.MustNewName("abc.com"), dns.TypeALL, dns.ClassANY, 64, []byte{10, 0, 11, 1}),
dns.NewResource(dns.MustNewName("123.com"), dns.TypeA, dns.ClassINET, 64, []byte{20, 0, 22, 2}),
}
dnsPayload, err := msg.AppendTo(nil, 0x1234, dns.NewClientHeaderFlags(dns.OpCodeQuery, true))
if err != nil {
fmt.Println("dns encode error:", err)
return
}
pkt := make([]byte, ethSize+ipv4Size+udpSize+len(dnsPayload))
efrm, _ := ethernet.NewFrame(pkt)
*efrm.DestinationHardwareAddr() = [6]byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x01}
*efrm.SourceHardwareAddr() = [6]byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x02}
efrm.SetEtherType(ethernet.TypeIPv4)
ifrm, _ := ipv4.NewFrame(pkt[ethSize:])
ifrm.SetVersionAndIHL(4, 5)
ifrm.SetTTL(64)
ifrm.SetProtocol(lneto.IPProtoUDP)
ifrm.SetTotalLength(uint16(ipv4Size + udpSize + len(dnsPayload)))
ifrm.SetCRC(ifrm.CalculateHeaderCRC())
ufrm, _ := udp.NewFrame(pkt[ethSize+ipv4Size:])
ufrm.SetSourcePort(58200)
ufrm.SetDestinationPort(dns.ServerPort)
ufrm.SetLength(uint16(udpSize + len(dnsPayload)))
copy(pkt[ethSize+ipv4Size+udpSize:], dnsPayload)
var cap PacketBreakdown
cap.SubfieldLimit = 20
frames, err := cap.CaptureEthernet(nil, pkt, 0)
if err != nil {
fmt.Println("capture error:", err)
return
}
var fmtr Formatter
fmtr.SubfieldLimit = cap.SubfieldLimit
fmtr.FrameSep = "\n"
fmtr.FieldSep = "; "
fmtr.SubfieldSep = "\n\t"
out, err := fmtr.FormatFrames(nil, frames, pkt)
if err != nil {
fmt.Println("format error:", err)
return
}
fmt.Println(string(out))
// Output:
// Ethernet len=14; destination=00:00:00:00:00:01; source=00:00:00:00:00:02; protocol=0x0800
// IPv4 len=20; version=0x04; (Header Length)=5; (Type of Service)=0x00; (Total Length)=117; identification=0x0000; flags=0x0000; (Time to live)=0x40; protocol=0x11; checksum=0x7a79; source=0.0.0.0; destination=0.0.0.0
// UDP len=8; (Source port)=58200; (Destination port)=53; size=97; checksum=0x0000
// DNS len=89; identification=0x1234; flags=0x0100; Questions=2; Answers=2; Authorities=0; Additionals=0; Questions
// Name=example.com
// Type=1
// Class=1
// Name=temu.com
// Type=28
// Class=255; Answers
// Name=abc.com
// Type=255
// Class=255
// TTL=0x00000040
// Length=4
// Data=10.0.11.1
// Name=123.com
// Type=1
// Class=1
// TTL=0x00000040
// Length=4
// Data=20.0.22.2
}
+17 -4
View File
@@ -105,12 +105,12 @@ func (f *Formatter) FormatFrame(dst []byte, frm Frame, pkt []byte) (_ []byte, er
}
func (f *Formatter) filterField(field FrameField) bool {
return f.FilterClasses != nil && !slices.Contains(f.FilterClasses, field.Class) ||
return f.FilterClasses != nil && field.Flags&FlagContainer == 0 && !slices.Contains(f.FilterClasses, field.Class) ||
(field.Flags.IsLegacy() && !f.DisableLegacyFilter)
}
func (f *Formatter) FormatField(dst []byte, pktStartOff int, field FrameField, pkt []byte) (_ []byte, err error) {
printOnlySubfields := field.Class == FieldClassOptions && len(field.SubFields) > 0
printOnlySubfields := (field.Flags&FlagContainer != 0 || field.Class == FieldClassOptions) && len(field.SubFields) > 0
if !printOnlySubfields {
dst, err = f.formatField(dst, pktStartOff, field, pkt)
} else {
@@ -119,7 +119,10 @@ func (f *Formatter) FormatField(dst []byte, pktStartOff int, field FrameField, p
if f.SubfieldLimit > 0 && len(field.SubFields) > 0 {
sep := f.subfieldSep()
lim := min(len(field.SubFields), f.SubfieldLimit)
for i := 0; err == nil && i < lim && !f.filterField(field.SubFields[i]); i++ {
for i := 0; err == nil && i < lim; i++ {
if f.filterField(field.SubFields[i]) {
continue
}
dst = append(dst, sep...)
// Notice we only format subfields one level low
dst, err = f.formatField(dst, pktStartOff, field.SubFields[i], pkt)
@@ -143,9 +146,13 @@ func (f *Formatter) formatField(dst []byte, pktStartOff int, field FrameField, p
dst = append(dst, ')')
}
if field.BitLength == 0 {
// We do not print empty nor container fields.
return dst, nil
}
dst = append(dst, '=')
if field.Flags&FlagContainer != 0 {
return dst, nil
}
f.mubuf.Lock()
defer f.mubuf.Unlock()
f.buf, err = appendField(f.buf[:0], pkt, field.FrameBitOffset+pktStartOff, field.BitLength, field.Flags.IsRightAligned())
@@ -177,6 +184,12 @@ func (f *Formatter) formatField(dst []byte, pktStartOff int, field FrameField, p
dst = strconv.AppendQuote(dst, unsafe.String(&f.buf[0], len(f.buf)))
}
debuglog("pcap:fmtfield:text-done")
case FieldClassDNSName:
// Walk the DNS message from pktStartOff to resolve the name, following
// compression pointers that reference earlier bytes in the DNS message.
dnsMsg := pkt[pktStartOff/8:]
nameOff := field.FrameBitOffset / 8
dst = dnsAppendDottedName(dst, dnsMsg, nameOff)
case FieldClassDst, FieldClassSrc, FieldClassSize, FieldClassAddress, FieldClassOperation:
// IP, MAC addresses and ports.
if field.BitLength <= 16 {
@@ -221,7 +234,7 @@ func (f *Formatter) fieldSep() string {
func (f *Formatter) subfieldSep() string {
sep := f.SubfieldSep
if sep == "" {
sep = "_" // default sub-field separator
sep = "," // default sub-field separator
}
return sep
}
+6 -4
View File
@@ -25,15 +25,17 @@ func _() {
_ = x[FieldClassBinaryText-14]
_ = x[FieldClassOperation-15]
_ = x[FieldClassTimestamp-16]
_ = x[FieldClassDNSName-17]
}
const _FieldClass_name = "undefinedsourcedestinationprotocolversiontypesizeflagsidentificationchecksumoptionspayloadtextaddressbinary-textoptimestamp"
const _FieldClass_name = "undefinedsourcedestinationprotocolversiontypesizeflagsidentificationchecksumoptionspayloadtextaddressbinary-textoptimestampdns name"
var _FieldClass_index = [...]uint8{0, 9, 15, 26, 34, 41, 45, 49, 54, 68, 76, 83, 90, 94, 101, 112, 114, 123}
var _FieldClass_index = [...]uint8{0, 9, 15, 26, 34, 41, 45, 49, 54, 68, 76, 83, 90, 94, 101, 112, 114, 123, 131}
func (i FieldClass) String() string {
if i >= FieldClass(len(_FieldClass_index)-1) {
idx := int(i) - 0
if i < 0 || idx >= len(_FieldClass_index)-1 {
return "FieldClass(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _FieldClass_name[_FieldClass_index[i]:_FieldClass_index[i+1]]
return _FieldClass_name[_FieldClass_index[idx]:_FieldClass_index[idx+1]]
}
+17 -3
View File
@@ -39,6 +39,8 @@ type StackEthernet struct {
gwmac [6]byte
mtu uint16
acceptMulticast bool
onSend func(p []byte)
// crcupdate set when crc32 has been configured to be appended.
crcupdate func(crc uint32, p []byte) uint32
}
@@ -63,6 +65,10 @@ func (ls *StackEthernet) HardwareAddr6() [6]byte {
return ls.mac
}
func (ls *StackEthernet) OnEncapsulate(cb func([]byte)) {
ls.onSend = cb
}
// Reset6 resets the stack with the given parameters.
//
// Deprecated: Use [StackEthernet.Configure] instead.
@@ -79,7 +85,7 @@ func (ls *StackEthernet) Reset6(mac, gateway [6]byte, mtu, maxNodes int) error {
// It validates the configuration parameters and resets internal state.
// The connection ID is incremented on each call to invalidate existing connections.
func (ls *StackEthernet) Configure(cfg StackEthernetConfig) error {
if cfg.MTU > (math.MaxUint16-ethernet.MaxOverheadSize) || cfg.MTU < 256 {
if cfg.MTU > ethernet.MaxMTU || cfg.MTU < ethernet.MinimumMTU {
return lneto.ErrInvalidConfig
} else if cfg.MaxNodes <= 0 {
return lneto.ErrInvalidConfig
@@ -122,7 +128,7 @@ func (ls *StackEthernet) LocalPort() uint16 { return 0 }
func (ls *StackEthernet) Protocol() uint64 { return 1 }
func (ls *StackEthernet) Register(h lneto.StackNode) error {
func (ls *StackEthernet) RegisterEthernet(h lneto.StackNode) error {
proto := h.Protocol()
if proto > math.MaxUint16 || proto <= 1500 {
return lneto.ErrInvalidConfig
@@ -152,7 +158,12 @@ func (ls *StackEthernet) Demux(carrierData []byte, frameOffset int) (err error)
return err
}
DROP:
ls.handlers.info("LinkStack:drop-packet", internal.SlogAddr6("dsthw", dstaddr), slog.String("ethertype", efrm.EtherTypeOrSize().String()))
// Frames not addressed to us are routine noise on a shared LAN (multicast/
// broadcast spam). Log at debug so a production logger at info level gates it
// out instead of allocating slog attrs per dropped packet.
if internal.LogEnabled(ls.handlers.logger.log, slog.LevelDebug) {
ls.handlers.debug("LinkStack:drop-packet", internal.SlogAddr6("dsthw", dstaddr), slog.String("ethertype", efrm.EtherTypeOrSize().String()))
}
return lneto.ErrPacketDrop
}
@@ -194,6 +205,9 @@ func (ls *StackEthernet) Encapsulate(carrierData []byte, offsetToIP, offsetToFra
dst[n] = 0
n++
}
if ls.onSend != nil {
ls.onSend(dst[:n])
}
if ls.crcupdate != nil {
crc := ls.crcupdate(0, carrierData[offsetToFrame:offsetToFrame+n])
binary.LittleEndian.PutUint32(carrierData[offsetToFrame+n:], crc)
-251
View File
@@ -1,251 +0,0 @@
package internet
import (
"io"
"log/slog"
"net/netip"
"github.com/soypat/lneto"
"github.com/soypat/lneto/ethernet"
"github.com/soypat/lneto/internal"
"github.com/soypat/lneto/ipv4"
"github.com/soypat/lneto/tcp"
"github.com/soypat/lneto/udp"
)
var _ lneto.StackNode = (*StackIP)(nil)
type StackIP struct {
connID uint64
ipID uint16
ip [4]byte
acceptMulticast bool
validator lneto.Validator
handlers handlers
}
func (sb *StackIP) Reset(addr netip.Addr, maxNodes int) error {
if maxNodes <= 0 {
return lneto.ErrInvalidConfig
}
err := sb.SetAddr(addr)
if err != nil {
return err
}
sb.handlers.reset("StackIP", maxNodes)
*sb = StackIP{
connID: sb.connID + 1,
validator: sb.validator,
handlers: sb.handlers,
ip: sb.ip,
acceptMulticast: sb.acceptMulticast,
}
return nil
}
func (sb *StackIP) SetAddr(addr netip.Addr) error {
if !addr.IsValid() {
return lneto.ErrInvalidAddr
} else if !addr.Is4() {
return lneto.ErrUnsupported
}
sb.ip = addr.As4()
return nil
}
func (sb *StackIP) ConnectionID() *uint64 {
return &sb.connID
}
func (sb *StackIP) Protocol() uint64 {
return uint64(ethernet.TypeIPv4) // Only support ipv4 for now.
}
func (sb *StackIP) LocalPort() uint16 { return 0 }
func (sb *StackIP) Addr() netip.Addr {
return netip.AddrFrom4(sb.ip)
}
func (sb *StackIP) SetAcceptMulticast(accept bool) {
sb.acceptMulticast = accept
}
func (sb *StackIP) SetLogger(logger *slog.Logger) {
sb.handlers.log = logger
}
func (sb *StackIP) Demux(carrierData []byte, offset int) error {
debugLog("ip:demux")
sb.handlers.info("StackIP.Demux:start")
frame := carrierData[offset:] // we don't care about carrier data in IP.
ifrm, err := ipv4.NewFrame(frame)
if err != nil {
return err
}
dst := ifrm.DestinationAddr()
if sb.ip != ([4]byte{}) && *dst != sb.ip {
if !sb.acceptMulticast || dst[0]&0xF0 != 0xE0 {
sb.handlers.debug("ip:not-for-us")
return lneto.ErrPacketDrop // Not meant for us.
}
}
sb.validator.ResetErr()
ifrm.ValidateExceptCRC(&sb.validator)
if err = sb.validator.ErrPop(); err != nil {
sb.handlers.error("ip:Demux.validate")
return err
}
if ifrm.CalculateHeaderCRC() != 0 {
sb.handlers.error("ip:demux.crc")
return lneto.ErrBadCRC
}
off := ifrm.HeaderLength()
totalLen := ifrm.TotalLength()
proto := ifrm.Protocol()
node := sb.handlers.nodeByProto(uint16(proto))
// nodeIdx := getNodeByProto(sb.handlers, uint16(proto))
if node == nil {
// Drop packet.
sb.handlers.info("ip:demux.drop", internal.SlogAddr4("dstaddr", ifrm.DestinationAddr()), slog.String("proto", ifrm.Protocol().String()))
return lneto.ErrPacketDrop
}
// Incoming CRC Validation of common IP Protocols.
var crc lneto.CRC791
switch proto {
case lneto.IPProtoTCP:
ifrm.CRCWriteTCPPseudo(&crc)
if crc.PayloadSum16(ifrm.Payload()) != 0 {
sb.handlers.error("ip:demux.tcpcrc")
return lneto.ErrBadCRC
}
case lneto.IPProtoUDP:
ufrm, err := udp.NewFrame(ifrm.Payload())
if err != nil {
return err
}
ufrm.ValidateSize(&sb.validator)
if err = sb.validator.ErrPop(); err != nil {
sb.handlers.error("ip:demux.udpvalidatesize")
return err
}
frameLen := ufrm.Length()
ifrm.CRCWriteUDPPseudo(&crc, frameLen)
if crc.PayloadSum16(ufrm.RawData()[:frameLen]) != 0 {
sb.handlers.error("ip:demux.udpcrc")
return lneto.ErrBadCRC
}
}
sb.handlers.info("ipDemux", slog.String("ipproto", proto.String()), slog.Int("plen", int(totalLen)))
err = node.callbacks.Demux(frame[:totalLen], off)
if sb.handlers.tryHandleError(node, err) {
sb.handlers.info("ipclose", slog.String("proto", proto.String()))
err = nil
}
return err
}
func (sb *StackIP) Encapsulate(carrierData []byte, offsetToIP, offsetToFrame int) (int, error) {
frame := carrierData[offsetToFrame:]
if len(frame) < 256 {
return 0, io.ErrShortBuffer
}
ifrm, _ := ipv4.NewFrame(frame)
const ihl = 5
const headerlen = ihl * 4
const dontFrag = 0x4000
ifrm.SetVersionAndIHL(4, ihl)
ifrm.SetToS(0)
seed := sb.ipID + uint16(sb.connID)
id := internal.Prand16(seed)
ifrm.SetID(id)
ifrm.SetFlags(dontFrag)
ifrm.SetTTL(64)
*ifrm.SourceAddr() = sb.ip
sb.ipID = id
// Children (TCP/UDP) start at offset headerlen (20 bytes after IP header start).
// offsetToIP is 0 relative to this slice (frame), children's frame starts at headerlen.
node, n, err := sb.handlers.encapsulateAny(carrierData, offsetToFrame, offsetToFrame+headerlen)
if n == 0 {
return n, err
}
proto := lneto.IPProto(node.proto)
totalLen := n + headerlen
ifrm.SetTotalLength(uint16(totalLen))
ifrm.SetProtocol(proto)
// Zero the CRC field so its value does not add to the final result.
ifrm.SetCRC(0)
crcValue := ifrm.CalculateHeaderCRC()
ifrm.SetCRC(crcValue)
// Calculate CRC for our newly generated packet.
var crc lneto.CRC791
payload := ifrm.Payload()
switch proto {
case lneto.IPProtoTCP:
ifrm.CRCWriteTCPPseudo(&crc)
tfrm, _ := tcp.NewFrame(payload)
// Zero the CRC field so its value does not add to the final result.
tfrm.SetCRC(0)
crcValue = crc.PayloadSum16(payload)
tfrm.SetCRC(crcValue)
case lneto.IPProtoUDP:
ufrm, _ := udp.NewFrame(payload)
ifrm.CRCWriteUDPPseudo(&crc, uint16(n))
ufrm.SetLength(uint16(n))
// Zero the CRC field so its value does not add to the final result.
ufrm.SetCRC(0)
crcValue = lneto.NeverZeroSum(crc.PayloadSum16(payload))
ufrm.SetCRC(crcValue)
}
return totalLen, err
}
func (sb *StackIP) Register(h lneto.StackNode) error {
proto := h.Protocol()
if proto > 255 {
return lneto.ErrInvalidConfig
}
return sb.handlers.registerByPortProto(nodeFromStackNode(h, h.LocalPort(), proto, nil))
}
func (sb *StackIP) IsRegistered(proto lneto.IPProto) bool {
return sb.handlers.nodeByProto(uint16(proto)) != nil
}
func (sb *StackIP) recvicmp(icmpData []byte) error {
var crc lneto.CRC791
if crc.PayloadSum16(icmpData) != 0 {
return lneto.ErrBadCRC
}
return nil
}
type logger struct {
log *slog.Logger
}
func (l logger) error(msg string, attrs ...slog.Attr) {
internal.LogAttrs(l.log, slog.LevelError, msg, attrs...)
}
func (l logger) info(msg string, attrs ...slog.Attr) {
internal.LogAttrs(l.log, slog.LevelInfo, msg, attrs...)
}
func (l logger) warn(msg string, attrs ...slog.Attr) {
internal.LogAttrs(l.log, slog.LevelWarn, msg, attrs...)
}
func (l logger) debug(msg string, attrs ...slog.Attr) {
internal.LogAttrs(l.log, slog.LevelDebug, msg, attrs...)
}
func (l logger) trace(msg string, attrs ...slog.Attr) {
internal.LogAttrs(l.log, internal.LevelTrace, msg, attrs...)
}
const enableAllocLog = internal.HeapAllocDebugging
func debugLog(msg string) {
if enableAllocLog {
internal.LogAllocs(msg)
}
}
+231
View File
@@ -0,0 +1,231 @@
package internet
import (
"io"
"log/slog"
"github.com/soypat/lneto"
"github.com/soypat/lneto/ethernet"
"github.com/soypat/lneto/internal"
"github.com/soypat/lneto/ipv4"
"github.com/soypat/lneto/tcp"
"github.com/soypat/lneto/udp"
)
// stackip4 is NOT a StackNode implementation.
// It is meant to be embedded within StackNodes.
// var _ lneto.StackNode = (*stackip4)(nil)
type StackIPv4 struct {
connID uint64
stackip4
}
func (stackip4 *StackIPv4) Reset(vld *lneto.Validator, maxNodes int) error {
stackip4.connID++
stackip4.reset4(vld, maxNodes)
return nil
}
func (stackip4 *StackIPv4) ConnectionID() *uint64 {
return &stackip4.connID
}
func (stackip4 *StackIPv4) Protocol() uint64 {
return uint64(ethernet.TypeIPv4)
}
func (stackip4 *StackIPv4) LocalPort() uint16 { return 0 }
func (stackip4 *StackIPv4) SetLogger(logger *slog.Logger) {
stackip4.stackip4.handlers.log = logger
}
func (stackip4 *StackIPv4) Demux(carrierData []byte, offset int) error {
debugLog("ip:demux")
return stackip4.stackip4.demux4(carrierData, offset)
}
func (stackip4 *StackIPv4) Encapsulate(carrierData []byte, offsetToIP, offsetToFrame int) (n int, err error) {
if offsetToFrame != offsetToIP {
return 0, lneto.ErrBug
}
return stackip4.stackip4.encapsulate4(carrierData, offsetToIP)
}
type stackip4 struct {
handlers handlers
vld *lneto.Validator
ipID uint16
ip4 [4]byte
acceptMulticast bool
acceptBroadcast bool
}
func (si4 *stackip4) reset4(vld *lneto.Validator, maxNodes int) {
*si4 = stackip4{
ip4: [4]byte{},
ipID: 1,
acceptMulticast: false,
handlers: si4.handlers,
vld: vld,
}
si4.handlers.reset("stackip4", maxNodes)
}
func (si4 *stackip4) Register4(h lneto.StackNode) error {
proto := h.Protocol()
if proto > 255 {
return lneto.ErrInvalidConfig
}
return si4.handlers.registerByPortProto(nodeFromStackNode(h, h.LocalPort(), proto, nil))
}
func (si4 *stackip4) IsRegistered4(proto lneto.IPProto) bool {
return si4.handlers.nodeByProto(uint16(proto)) != nil
}
func (si4 *stackip4) SetAcceptMulticast4(accept bool) {
si4.acceptMulticast = accept
}
func (si4 *stackip4) SetAcceptBroadcast4(accept bool) {
si4.acceptBroadcast = accept
}
func (si4 *stackip4) Addr4() [4]byte { return si4.ip4 }
func (si4 *stackip4) SetAddr4(ip4 [4]byte) {
si4.ip4 = ip4
}
func (si4 *stackip4) demux4(carrierData []byte, offset int) error {
debugLog("ip4:demux")
si4.handlers.info("demux:start")
frame := carrierData[offset:] // we don't care about carrier data in IP.
ifrm, err := ipv4.NewFrame(frame)
if err != nil {
return err
}
dst := ifrm.DestinationAddr()
if si4.ip4 != ([4]byte{}) && *dst != si4.ip4 { // Accept all packets when IP zeroed.
switch {
case si4.acceptMulticast && ipv4.IsMulticast(*dst):
// accept multicast.
case si4.acceptBroadcast && ipv4.IsBroadcast(*dst):
// accept broadcast.
default:
si4.handlers.debug("ip:not-for-us")
return lneto.ErrPacketDrop // Not meant for us.
}
}
si4.vld.ResetErr()
ifrm.ValidateExceptCRC(si4.vld)
if err = si4.vld.ErrPop(); err != nil {
si4.handlers.error("ip:Demux.validate")
return err
}
if ifrm.CalculateHeaderCRC() != 0 {
si4.handlers.error("ip:demux.crc")
return lneto.ErrBadCRC
}
off := ifrm.HeaderLength()
proto := ifrm.Protocol()
node := si4.handlers.nodeByProto(uint16(proto))
// nodeIdx := getNodeByProto(sb.handlers, uint16(proto))
if node == nil {
// Drop packet.
si4.handlers.info("ip:demux.drop", internal.SlogAddr4("dstaddr", ifrm.DestinationAddr()), slog.String("proto", ifrm.Protocol().String()))
return lneto.ErrPacketDrop
}
// Incoming CRC Validation of common IP Protocols.
var crc lneto.CRC791
switch proto {
case lneto.IPProtoTCP:
ifrm.CRCWriteTCPPseudo(&crc)
if crc.PayloadSum16(ifrm.Payload()) != 0 {
si4.handlers.error("ip:demux.tcpcrc")
return lneto.ErrBadCRC
}
case lneto.IPProtoUDP:
ufrm, err := udp.NewFrame(ifrm.Payload())
if err != nil {
return err
}
ufrm.ValidateSize(si4.vld)
if err = si4.vld.ErrPop(); err != nil {
si4.handlers.error("ip:demux.udpvalidatesize")
return err
}
frameLen := ufrm.Length()
ifrm.CRCWriteUDPPseudo(&crc, frameLen)
if crc.PayloadSum16(ufrm.RawData()[:frameLen]) != 0 {
si4.handlers.error("ip:demux.udpcrc")
return lneto.ErrBadCRC
}
}
totalLen := ifrm.TotalLength()
si4.handlers.info("ipDemux", slog.String("ipproto", proto.String()), slog.Int("tlen", int(totalLen)))
err = node.callbacks.Demux(frame[:totalLen], off)
if si4.handlers.tryHandleError(node, err) {
si4.handlers.info("ipclose", slog.String("proto", proto.String()))
err = nil
}
return err
}
func (si4 *stackip4) encapsulate4(carrierData []byte, offsetToIP int) (int, error) {
frame := carrierData[offsetToIP:]
if len(frame) < ipv4.MinimumMTU {
return 0, io.ErrShortBuffer
}
ifrm, _ := ipv4.NewFrame(frame)
const ihl = 5
const headerlen = ihl * 4
const dontFrag = 0x4000
ifrm.SetVersionAndIHL(4, ihl)
ifrm.SetToS(0)
seed := (si4.ipID + 1) ^ uint16(si4.ip4[0])
id := internal.Prand16(seed)
ifrm.SetID(id)
ifrm.SetFlags(dontFrag)
ifrm.SetTTL(64)
*ifrm.SourceAddr() = si4.ip4
si4.ipID = id
// Children (TCP/UDP) start at offset headerlen (20 bytes after IP header start).
// offsetToIP is 0 relative to this slice (frame), children's frame starts at headerlen.
node, n, err := si4.handlers.encapsulateAny(carrierData, offsetToIP, offsetToIP+headerlen)
if n == 0 {
return n, err
}
proto := lneto.IPProto(node.proto)
totalLen := n + headerlen
ifrm.SetTotalLength(uint16(totalLen))
ifrm.SetProtocol(proto)
// Zero the CRC field so its value does not add to the final result.
ifrm.SetCRC(0)
crcValue := ifrm.CalculateHeaderCRC()
ifrm.SetCRC(crcValue)
// Calculate CRC for our newly generated packet.
var crc lneto.CRC791
payload := ifrm.Payload()
switch proto {
case lneto.IPProtoTCP:
ifrm.CRCWriteTCPPseudo(&crc)
tfrm, _ := tcp.NewFrame(payload)
// Zero the CRC field so its value does not add to the final result.
tfrm.SetCRC(0)
crcValue = crc.PayloadSum16(payload)
tfrm.SetCRC(crcValue)
case lneto.IPProtoUDP:
ufrm, _ := udp.NewFrame(payload)
ifrm.CRCWriteUDPPseudo(&crc, uint16(n))
ufrm.SetLength(uint16(n))
// Zero the CRC field so its value does not add to the final result.
ufrm.SetCRC(0)
crcValue = lneto.NeverZeroSum(crc.PayloadSum16(payload))
ufrm.SetCRC(crcValue)
}
return totalLen, err
}
+183
View File
@@ -0,0 +1,183 @@
package internet
import (
"log/slog"
"github.com/soypat/lneto"
"github.com/soypat/lneto/ethernet"
"github.com/soypat/lneto/internal"
"github.com/soypat/lneto/ipv6"
"github.com/soypat/lneto/tcp"
"github.com/soypat/lneto/udp"
)
// stackip6 is NOT a StackNode implementation.
// It is meant to be embedded within StackNodes.
// var _ lneto.StackNode = (*stackip6)(nil)
type StackIPv6 struct {
connID uint64
stackip6
}
func (stackip6 *StackIPv6) Reset(vld *lneto.Validator, maxNodes int) error {
stackip6.connID++
stackip6.reset6(vld, maxNodes)
return nil
}
func (stackip6 *StackIPv6) ConnectionID() *uint64 {
return &stackip6.connID
}
func (stackip6 *StackIPv6) Protocol() uint64 {
return uint64(ethernet.TypeIPv6)
}
func (stackip6 *StackIPv6) LocalPort() uint16 { return 0 }
func (stackip6 *StackIPv6) SetLogger(logger *slog.Logger) {
stackip6.stackip6.handlers.log = logger
}
func (stackip6 *StackIPv6) Demux(carrierData []byte, offset int) error {
debugLog("ip:demux")
return stackip6.stackip6.demux6(carrierData, offset)
}
func (stackip6 *StackIPv6) Encapsulate(carrierData []byte, offsetToIP, offsetToFrame int) (n int, err error) {
if offsetToFrame != offsetToIP {
return 0, lneto.ErrBug
}
return stackip6.stackip6.encapsulate6(carrierData, offsetToIP)
}
type stackip6 struct {
handlers handlers
vld *lneto.Validator
ip6 [16]byte
acceptMulticast bool
acceptBroadcast bool
}
func (si6 *stackip6) Register6(h lneto.StackNode) error {
proto := h.Protocol()
if proto > 255 {
return lneto.ErrInvalidConfig
}
return si6.handlers.registerByPortProto(nodeFromStackNode(h, h.LocalPort(), proto, nil))
}
func (si6 *stackip6) IsRegistered6(proto lneto.IPProto) bool {
return si6.handlers.nodeByProto(uint16(proto)) != nil
}
func (si6 *stackip6) SetAcceptMulticast6(accept bool) { si6.acceptMulticast = accept }
func (si6 *stackip6) Addr6() [16]byte { return si6.ip6 }
func (si6 *stackip6) SetAddr6(ip6 [16]byte) { si6.ip6 = ip6 }
func (si6 *stackip6) reset6(vld *lneto.Validator, maxNodes int) {
*si6 = stackip6{
handlers: si6.handlers,
vld: vld,
}
si6.handlers.reset("stackip6", maxNodes)
}
func (si6 *stackip6) demux6(carrierData []byte, offset int) error {
debugLog("ip6:demux")
si6.handlers.info("StackIP6.Demux:start")
ifrm, err := ipv6.NewFrame(carrierData[offset:])
if err != nil {
return err
}
dst := ifrm.DestinationAddr()
if si6.ip6 != ([16]byte{}) && *dst != si6.ip6 {
if !si6.acceptMulticast || !internal.IsMulticastIPAddr(dst[:]) {
si6.handlers.debug("ip6:not-for-us")
return lneto.ErrPacketDrop
}
}
si6.vld.ResetErr()
ifrm.ValidateSize(si6.vld)
if err = si6.vld.ErrPop(); err != nil {
si6.handlers.error("ip6:Demux.validate")
return err
}
proto := ifrm.NextHeader()
node := si6.handlers.nodeByProto(uint16(proto))
if node == nil {
si6.handlers.info("ip6:demux.drop", slog.String("proto", proto.String()))
return lneto.ErrPacketDrop
}
payload := ifrm.Payload()
var crc lneto.CRC791
switch proto {
case lneto.IPProtoTCP:
ifrm.CRCWritePseudo(&crc)
if crc.PayloadSum16(payload) != 0 {
si6.handlers.error("ip6:demux.tcpcrc")
return lneto.ErrBadCRC
}
case lneto.IPProtoUDP:
ufrm, err := udp.NewFrame(payload)
if err != nil {
return err
}
ufrm.ValidateSize(si6.vld)
if err = si6.vld.ErrPop(); err != nil {
si6.handlers.error("ip6:demux.udpvalidatesize")
return err
}
ifrm.CRCWritePseudo(&crc)
if crc.PayloadSum16(payload) != 0 {
si6.handlers.error("ip6:demux.udpcrc")
return lneto.ErrBadCRC
}
}
const headerlen = 40
plen := ifrm.PayloadLength()
si6.handlers.info("ip6Demux", slog.String("ipproto", proto.String()), slog.Int("plen", int(plen)))
err = node.callbacks.Demux(carrierData[offset:offset+headerlen+int(plen)], headerlen)
if si6.handlers.tryHandleError(node, err) {
si6.handlers.info("ip6close", slog.String("proto", proto.String()))
err = nil
}
return err
}
func (si6 *stackip6) encapsulate6(carrierData []byte, offsetToIP int) (int, error) {
ifrm, err := ipv6.NewFrame(carrierData[offsetToIP:])
if err != nil {
return 0, err
}
// Set default parameters which node is free to change.
ifrm.SetVersionTrafficAndFlow(6, 0, 0)
ifrm.SetHopLimit(64)
*ifrm.SourceAddr() = si6.ip6
const headerlen = 40
node, n, err := si6.handlers.encapsulateAny(carrierData, offsetToIP, offsetToIP+headerlen)
if n == 0 {
return n, err
}
proto := lneto.IPProto(node.proto)
ifrm.SetNextHeader(proto)
ifrm.SetPayloadLength(uint16(n))
var crc lneto.CRC791
payload := ifrm.Payload()
switch proto {
case lneto.IPProtoTCP:
ifrm.CRCWritePseudo(&crc)
tfrm, _ := tcp.NewFrame(payload)
tfrm.SetCRC(0)
tfrm.SetCRC(crc.PayloadSum16(payload))
case lneto.IPProtoUDP:
ufrm, _ := udp.NewFrame(payload)
ufrm.SetLength(uint16(n))
ifrm.CRCWritePseudo(&crc)
ufrm.SetCRC(0)
ufrm.SetCRC(lneto.NeverZeroSum(crc.PayloadSum16(payload)))
}
return headerlen + n, err
}
+11 -9
View File
@@ -33,8 +33,6 @@ func (ps *StackPorts) ResetTCP(maxNodes uint16) error {
func (ps *StackPorts) Reset(protocol uint64, dstPortOffset, maxNodes uint16) error {
if protocol > math.MaxUint16 {
return lneto.ErrInvalidConfig
} else if maxNodes <= 0 {
return lneto.ErrInvalidConfig
}
ps.handlers.reset("StackPorts(proto="+strconv.Itoa(int(protocol))+")", int(maxNodes))
*ps = StackPorts{
@@ -105,25 +103,29 @@ type StackPortsMACFiltered struct {
sp StackPorts
}
func (mfsp *StackPortsMACFiltered) Register(h lneto.StackNode, addr []byte) error {
func (mfsp *StackPortsMACFiltered) RegisterMACFiltered(h lneto.StackNode, macAddr []byte) error {
// TODO(soypat): We can likely constrain memory and the slice lifetime if StackPortsMACFiltered owns it
// or better yet, if the handlers node slice owns the memory. We need to think carefully of who has write access (the ARP and NDP handlers)
// and make sure that they never write after the connection has been terminated. Idea:
// RegisterMACFiltered(h lneto.StackNode, filterMAC bool) (macAddr *[6]byte, connIDthing *uint8, err error)
port := h.LocalPort()
proto := h.Protocol()
if port <= 0 {
return lneto.ErrZeroSource
} else if proto != uint64(mfsp.sp.protocol) {
return lneto.ErrInvalidConfig
} else if addr != nil && len(addr) != 6 {
} else if macAddr != nil && len(macAddr) != 6 {
return lneto.ErrInvalidAddr
}
return mfsp.sp.handlers.registerByPortProto(nodeFromStackNode(h, port, proto, addr))
return mfsp.sp.handlers.registerByPortProto(nodeFromStackNode(h, port, proto, macAddr))
}
func (ps *StackPortsMACFiltered) ResetUDP(maxNodes uint16) error {
return ps.sp.ResetUDP(maxNodes)
func (ps *StackPortsMACFiltered) ResetUDP(maxNodes uint16) {
ps.sp.ResetUDP(maxNodes) // Can't error.
}
func (ps *StackPortsMACFiltered) ResetTCP(maxNodes uint16) error {
return ps.sp.ResetTCP(maxNodes)
func (ps *StackPortsMACFiltered) ResetTCP(maxNodes uint16) {
ps.sp.ResetTCP(maxNodes) // Can't error.
}
func (ps *StackPortsMACFiltered) Reset(protocol uint64, dstPortOffset, maxNodes uint16) error {
+9 -1
View File
@@ -45,7 +45,15 @@ func (sudp *StackUDPPort) Demux(carrierData []byte, frameOffset int) error {
if dst != sudp.h.lport {
return lneto.ErrPacketDrop // Not meant for us.
}
// TODO remote ip address handling.
if len(sudp.raddr) > 0 && !internal.IsMulticastIPAddr(sudp.raddr) {
srcIP, _, _, _, err := internal.GetIPAddr(carrierData[:frameOffset])
if err != nil {
return err
}
if !internal.BytesEqual(srcIP, sudp.raddr) {
return lneto.ErrPacketDrop
}
}
src := ufrm.SourcePort()
if sudp.rmport != 0 && src != sudp.rmport {

Some files were not shown because too many files have changed in this diff Show More