tinygo-org/net
1
0
Fork 0
mirror of https://github.com/tinygo-org/net.git synced 2026-08-03 11:37:46 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add network device driver model, netdev

Browse Source 
This PR adds a network device driver model called netdev. There will be a companion PR for TinyGo drivers to update the netdev drivers and network examples. This PR covers the core "net" package.

An RFC for the work is here: #tinygo-org/drivers#487. Some things have changed from the RFC, but nothing major.

The "net" package is a partial port of Go's "net" package, version 1.19.3. The src/net/README file has details on what is modified from Go's "net" package.

Most "net" features are working as they would in normal Go. TCP/UDP/TLS protocol support is there. As well as HTTP client and server support. Standard Go network packages such as golang.org/x/net/websockets and Paho MQTT client work as-is. Other packages are likely to work as-is.

Testing results are here (https://docs.google.com/spreadsheets/d/e/2PACX-1vT0cCjBvwXf9HJf6aJV2Sw198F2ief02gmbMV0sQocKT4y4RpfKv3dh6Jyew8lQW64FouZ8GwA2yjxI/pubhtml?gid=1013173032&single=true).
This commit is contained in:
Scott Feldman
2023-03-27 14:15:21 -07:00
committed by deadprogram
parent b46e2ec2ac
commit 693edae782
39 changed files with 11271 additions and 1092 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+108 -1
View File
@@ -1 +1,108 @@
# net
# net
This is a port of Go's "net" package. The port offers a subset of Go's "net"
package. The subset maintains Go 1 compatiblity guarantee.
The "net" package is modified to use netdev, TinyGo's network device driver interface.
Netdev replaces the OS syscall interface for I/O access to the networking
device.
#### Table of Contents
- ["net" Package](#net-package)
- [Netdev and Netlink](#netdev-and-netlink)
- [Using "net" and "net/http" Packages](#using-net-and-nethttp-packages)
## "net" Package
The "net" package is ported from Go 1.19.3. The tree listings below shows the
files copied. If the file is marked with an '\*', it is copied _and_ modified
to work with netdev. If the file is marked with an '+', the file is new. If
there is no mark, it is a straight copy.
```
src/net
├── dial.go *
├── http
│   ├── client.go *
│   ├── clone.go
│   ├── cookie.go
│   ├── fs.go
│   ├── header.go *
│   ├── http.go
│   ├── internal
│   │   ├── ascii
│   │   │   ├── print.go
│   │   │   └── print_test.go
│   │   ├── chunked.go
│   │   └── chunked_test.go
│   ├── jar.go
│   ├── method.go
│   ├── request.go *
│   ├── response.go *
│   ├── server.go *
│   ├── sniff.go
│   ├── status.go
│   ├── transfer.go *
│   └── transport.go *
├── ip.go
├── iprawsock.go *
├── ipsock.go *
├── mac.go
├── mac_test.go
├── netdev.go +
├── net.go *
├── parse.go
├── pipe.go
├── README.md
├── tcpsock.go *
├── tlssock.go +
└── udpsock.go *
src/crypto/tls/
├── common.go *
└── tls.go *
```
The modifications to "net" are to basically wrap TCPConn, UDPConn, and TLSConn
around netdev socket calls. In Go, these net.Conns call out to OS syscalls for
the socket operations. In TinyGo, the OS syscalls aren't available, so netdev
socket calls are substituted.
The modifications to "net/http" are on the client and the server side. On the
client side, the TinyGo code changes remove the back-end round-tripper code and
replaces it with direct calls to TCPConns/TLSConns. All of Go's http
request/response handling code is intact and operational in TinyGo. Same holds
true for the server side. The server side supports the normal server features
like ServeMux and Hijacker (for websockets).
### Maintaining "net"
As Go progresses, changes to the "net" package need to be periodically
back-ported to TinyGo's "net" package. This is to pick up any upstream bug
fixes or security fixes.
Changes "net" package files are marked with // TINYGO comments.
The files that are marked modified * may contain only a subset of the original
file. Basically only the parts necessary to compile and run the example/net
examples are copied (and maybe modified).
## Netdev and Netlink
Netdev is TinyGo's network device driver model. Network drivers implement the
netdever interface, providing a common network I/O interface to TinyGo's "net"
package. The interface is modeled after the BSD socket interface. net.Conn
implementations (TCPConn, UDPConn, and TLSConn) use the netdev interface for
device I/O access.
Network drivers also (optionally) implement the Netlinker interface. This
interface is not used by TinyGo's "net" package, but rather provides the TinyGo
application direct access to the network device for common settings and control
that fall outside of netdev's socket interface.
See the README-net.md in drivers repo for more details on netdev and netlink.
## Using "net" and "net/http" Packages
See README-net.md in drivers repo to more details on using "net" and "net/http"
packages in a TinyGo application.
conn_test.go
-478
View File
@@ -1,478 +0,0 @@
// The following is copied from x/net official implementation.
// Source: https://cs.opensource.google/go/x/net/+/f15817d1:nettest/conntest.go
// Changes from original the file:
// - Some variables are pulled in from nettest/nettest.go file.
// - The implementation of checkForTimeoutError() function is changed in
// accordance with error returned by the Pipe implementation.
// Copyright 2016 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package net
import (
"bytes"
"encoding/binary"
"io"
"io/ioutil"
"math/rand"
"os"
"runtime"
"sync"
"testing"
"time"
)
// The following variables are copied from nettest/nettest.go file
var (
aLongTimeAgo = time.Unix(233431200, 0)
neverTimeout = time.Time{}
)
// MakePipe creates a connection between two endpoints and returns the pair
// as c1 and c2, such that anything written to c1 is read by c2 and vice-versa.
// The stop function closes all resources, including c1, c2, and the underlying
// Listener (if there is one), and should not be nil.
type MakePipe func() (c1, c2 Conn, stop func(), err error)
// testConn tests that a Conn implementation properly satisfies the interface.
// The tests should not produce any false positives, but may experience
// false negatives. Thus, some issues may only be detected when the test is
// run multiple times. For maximal effectiveness, run the tests under the
// race detector.
func testConn(t *testing.T, mp MakePipe) {
t.Run("BasicIO", func(t *testing.T) { timeoutWrapper(t, mp, testBasicIO) })
t.Run("PingPong", func(t *testing.T) { timeoutWrapper(t, mp, testPingPong) })
t.Run("RacyRead", func(t *testing.T) { timeoutWrapper(t, mp, testRacyRead) })
t.Run("RacyWrite", func(t *testing.T) { timeoutWrapper(t, mp, testRacyWrite) })
t.Run("ReadTimeout", func(t *testing.T) { timeoutWrapper(t, mp, testReadTimeout) })
t.Run("WriteTimeout", func(t *testing.T) { timeoutWrapper(t, mp, testWriteTimeout) })
t.Run("PastTimeout", func(t *testing.T) { timeoutWrapper(t, mp, testPastTimeout) })
t.Run("PresentTimeout", func(t *testing.T) { timeoutWrapper(t, mp, testPresentTimeout) })
t.Run("FutureTimeout", func(t *testing.T) { timeoutWrapper(t, mp, testFutureTimeout) })
t.Run("CloseTimeout", func(t *testing.T) { timeoutWrapper(t, mp, testCloseTimeout) })
t.Run("ConcurrentMethods", func(t *testing.T) { timeoutWrapper(t, mp, testConcurrentMethods) })
}
type connTester func(t *testing.T, c1, c2 Conn)
func timeoutWrapper(t *testing.T, mp MakePipe, f connTester) {
t.Helper()
c1, c2, stop, err := mp()
if err != nil {
t.Fatalf("unable to make pipe: %v", err)
}
var once sync.Once
defer once.Do(func() { stop() })
timer := time.AfterFunc(time.Minute, func() {
once.Do(func() {
t.Error("test timed out; terminating pipe")
stop()
})
})
defer timer.Stop()
f(t, c1, c2)
}
// testBasicIO tests that the data sent on c1 is properly received on c2.
func testBasicIO(t *testing.T, c1, c2 Conn) {
want := make([]byte, 1<<20)
rand.New(rand.NewSource(0)).Read(want)
dataCh := make(chan []byte)
go func() {
rd := bytes.NewReader(want)
if err := chunkedCopy(c1, rd); err != nil {
t.Errorf("unexpected c1.Write error: %v", err)
}
if err := c1.Close(); err != nil {
t.Errorf("unexpected c1.Close error: %v", err)
}
}()
go func() {
wr := new(bytes.Buffer)
if err := chunkedCopy(wr, c2); err != nil {
t.Errorf("unexpected c2.Read error: %v", err)
}
if err := c2.Close(); err != nil {
t.Errorf("unexpected c2.Close error: %v", err)
}
dataCh <- wr.Bytes()
}()
if got := <-dataCh; !bytes.Equal(got, want) {
t.Error("transmitted data differs")
}
}
// testPingPong tests that the two endpoints can synchronously send data to
// each other in a typical request-response pattern.
func testPingPong(t *testing.T, c1, c2 Conn) {
var wg sync.WaitGroup
defer wg.Wait()
pingPonger := func(c Conn) {
defer wg.Done()
buf := make([]byte, 8)
var prev uint64
for {
if _, err := io.ReadFull(c, buf); err != nil {
if err == io.EOF {
break
}
t.Errorf("unexpected Read error: %v", err)
}
v := binary.LittleEndian.Uint64(buf)
binary.LittleEndian.PutUint64(buf, v+1)
if prev != 0 && prev+2 != v {
t.Errorf("mismatching value: got %d, want %d", v, prev+2)
}
prev = v
if v == 1000 {
break
}
if _, err := c.Write(buf); err != nil {
t.Errorf("unexpected Write error: %v", err)
break
}
}
if err := c.Close(); err != nil {
t.Errorf("unexpected Close error: %v", err)
}
}
wg.Add(2)
go pingPonger(c1)
go pingPonger(c2)
// Start off the chain reaction.
if _, err := c1.Write(make([]byte, 8)); err != nil {
t.Errorf("unexpected c1.Write error: %v", err)
}
}
// testRacyRead tests that it is safe to mutate the input Read buffer
// immediately after cancelation has occurred.
func testRacyRead(t *testing.T, c1, c2 Conn) {
go chunkedCopy(c2, rand.New(rand.NewSource(0)))
var wg sync.WaitGroup
defer wg.Wait()
c1.SetReadDeadline(time.Now().Add(time.Millisecond))
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
defer wg.Done()
b1 := make([]byte, 1024)
b2 := make([]byte, 1024)
for j := 0; j < 100; j++ {
_, err := c1.Read(b1)
copy(b1, b2) // Mutate b1 to trigger potential race
if err != nil {
checkForTimeoutError(t, err)
c1.SetReadDeadline(time.Now().Add(time.Millisecond))
}
}
}()
}
}
// testRacyWrite tests that it is safe to mutate the input Write buffer
// immediately after cancelation has occurred.
func testRacyWrite(t *testing.T, c1, c2 Conn) {
go chunkedCopy(ioutil.Discard, c2)
var wg sync.WaitGroup
defer wg.Wait()
c1.SetWriteDeadline(time.Now().Add(time.Millisecond))
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
defer wg.Done()
b1 := make([]byte, 1024)
b2 := make([]byte, 1024)
for j := 0; j < 100; j++ {
_, err := c1.Write(b1)
copy(b1, b2) // Mutate b1 to trigger potential race
if err != nil {
checkForTimeoutError(t, err)
c1.SetWriteDeadline(time.Now().Add(time.Millisecond))
}
}
}()
}
}
// testReadTimeout tests that Read timeouts do not affect Write.
func testReadTimeout(t *testing.T, c1, c2 Conn) {
go chunkedCopy(ioutil.Discard, c2)
c1.SetReadDeadline(aLongTimeAgo)
_, err := c1.Read(make([]byte, 1024))
checkForTimeoutError(t, err)
if _, err := c1.Write(make([]byte, 1024)); err != nil {
t.Errorf("unexpected Write error: %v", err)
}
}
// testWriteTimeout tests that Write timeouts do not affect Read.
func testWriteTimeout(t *testing.T, c1, c2 Conn) {
go chunkedCopy(c2, rand.New(rand.NewSource(0)))
c1.SetWriteDeadline(aLongTimeAgo)
_, err := c1.Write(make([]byte, 1024))
checkForTimeoutError(t, err)
if _, err := c1.Read(make([]byte, 1024)); err != nil {
t.Errorf("unexpected Read error: %v", err)
}
}
// testPastTimeout tests that a deadline set in the past immediately times out
// Read and Write requests.
func testPastTimeout(t *testing.T, c1, c2 Conn) {
go chunkedCopy(c2, c2)
testRoundtrip(t, c1)
c1.SetDeadline(aLongTimeAgo)
n, err := c1.Write(make([]byte, 1024))
if n != 0 {
t.Errorf("unexpected Write count: got %d, want 0", n)
}
checkForTimeoutError(t, err)
n, err = c1.Read(make([]byte, 1024))
if n != 0 {
t.Errorf("unexpected Read count: got %d, want 0", n)
}
checkForTimeoutError(t, err)
testRoundtrip(t, c1)
}
// testPresentTimeout tests that a past deadline set while there are pending
// Read and Write operations immediately times out those operations.
func testPresentTimeout(t *testing.T, c1, c2 Conn) {
var wg sync.WaitGroup
defer wg.Wait()
wg.Add(3)
deadlineSet := make(chan bool, 1)
go func() {
defer wg.Done()
time.Sleep(100 * time.Millisecond)
deadlineSet <- true
c1.SetReadDeadline(aLongTimeAgo)
c1.SetWriteDeadline(aLongTimeAgo)
}()
go func() {
defer wg.Done()
n, err := c1.Read(make([]byte, 1024))
if n != 0 {
t.Errorf("unexpected Read count: got %d, want 0", n)
}
checkForTimeoutError(t, err)
if len(deadlineSet) == 0 {
t.Error("Read timed out before deadline is set")
}
}()
go func() {
defer wg.Done()
var err error
for err == nil {
_, err = c1.Write(make([]byte, 1024))
}
checkForTimeoutError(t, err)
if len(deadlineSet) == 0 {
t.Error("Write timed out before deadline is set")
}
}()
}
// testFutureTimeout tests that a future deadline will eventually time out
// Read and Write operations.
func testFutureTimeout(t *testing.T, c1, c2 Conn) {
var wg sync.WaitGroup
wg.Add(2)
c1.SetDeadline(time.Now().Add(100 * time.Millisecond))
go func() {
defer wg.Done()
_, err := c1.Read(make([]byte, 1024))
checkForTimeoutError(t, err)
}()
go func() {
defer wg.Done()
var err error
for err == nil {
_, err = c1.Write(make([]byte, 1024))
}
checkForTimeoutError(t, err)
}()
wg.Wait()
go chunkedCopy(c2, c2)
resyncConn(t, c1)
testRoundtrip(t, c1)
}
// testCloseTimeout tests that calling Close immediately times out pending
// Read and Write operations.
func testCloseTimeout(t *testing.T, c1, c2 Conn) {
go chunkedCopy(c2, c2)
var wg sync.WaitGroup
defer wg.Wait()
wg.Add(3)
// Test for cancelation upon connection closure.
c1.SetDeadline(neverTimeout)
go func() {
defer wg.Done()
time.Sleep(100 * time.Millisecond)
c1.Close()
}()
go func() {
defer wg.Done()
var err error
buf := make([]byte, 1024)
for err == nil {
_, err = c1.Read(buf)
}
}()
go func() {
defer wg.Done()
var err error
buf := make([]byte, 1024)
for err == nil {
_, err = c1.Write(buf)
}
}()
}
// testConcurrentMethods tests that the methods of Conn can safely
// be called concurrently.
func testConcurrentMethods(t *testing.T, c1, c2 Conn) {
if runtime.GOOS == "plan9" {
t.Skip("skipping on plan9; see https://golang.org/issue/20489")
}
go chunkedCopy(c2, c2)
// The results of the calls may be nonsensical, but this should
// not trigger a race detector warning.
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(7)
go func() {
defer wg.Done()
c1.Read(make([]byte, 1024))
}()
go func() {
defer wg.Done()
c1.Write(make([]byte, 1024))
}()
go func() {
defer wg.Done()
c1.SetDeadline(time.Now().Add(10 * time.Millisecond))
}()
go func() {
defer wg.Done()
c1.SetReadDeadline(aLongTimeAgo)
}()
go func() {
defer wg.Done()
c1.SetWriteDeadline(aLongTimeAgo)
}()
go func() {
defer wg.Done()
c1.LocalAddr()
}()
go func() {
defer wg.Done()
c1.RemoteAddr()
}()
}
wg.Wait() // At worst, the deadline is set 10ms into the future
resyncConn(t, c1)
testRoundtrip(t, c1)
}
// checkForTimeoutError checks that the error satisfies the OpError interface
// and that underlying Err is os.ErrDeadlineExceeded
func checkForTimeoutError(t *testing.T, err error) {
t.Helper()
operr, ok := err.(*OpError)
if !ok {
t.Errorf("got %T: %v, want OpError", err, err)
return
}
if operr.Err != os.ErrDeadlineExceeded {
t.Errorf("got %T: %v, want os.ErrDeadlineExceeded", err, err)
}
}
// testRoundtrip writes something into c and reads it back.
// It assumes that everything written into c is echoed back to itself.
func testRoundtrip(t *testing.T, c Conn) {
t.Helper()
if err := c.SetDeadline(neverTimeout); err != nil {
t.Errorf("roundtrip SetDeadline error: %v", err)
}
const s = "Hello, world!"
buf := []byte(s)
if _, err := c.Write(buf); err != nil {
t.Errorf("roundtrip Write error: %v", err)
}
if _, err := io.ReadFull(c, buf); err != nil {
t.Errorf("roundtrip Read error: %v", err)
}
if string(buf) != s {
t.Errorf("roundtrip data mismatch: got %q, want %q", buf, s)
}
}
// resyncConn resynchronizes the connection into a sane state.
// It assumes that everything written into c is echoed back to itself.
// It assumes that 0xff is not currently on the wire or in the read buffer.
func resyncConn(t *testing.T, c Conn) {
t.Helper()
c.SetDeadline(neverTimeout)
errCh := make(chan error)
go func() {
_, err := c.Write([]byte{0xff})
errCh <- err
}()
buf := make([]byte, 1024)
for {
n, err := c.Read(buf)
if n > 0 && bytes.IndexByte(buf[:n], 0xff) == n-1 {
break
}
if err != nil {
t.Errorf("unexpected Read error: %v", err)
break
}
}
if err := <-errCh; err != nil {
t.Errorf("unexpected Write error: %v", err)
}
}
// chunkedCopy copies from r to w in fixed-width chunks to avoid
// causing a Write that exceeds the maximum packet size for packet-based
// connections like "unixpacket".
// We assume that the maximum packet size is at least 1024.
func chunkedCopy(w io.Writer, r io.Reader) error {
b := make([]byte, 1024)
_, err := io.CopyBuffer(struct{ io.Writer }{w}, struct{ io.Reader }{r}, b)
return err
}
dial.go
+151 -7
View File
@@ -1,25 +1,169 @@
// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
// TINYGO: Omit DualStack support
// TINYGO: Omit Fast Fallback support
// TINYGO: Don't allow alternate resolver
// TINYGO: Omit DialTimeout
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package net
import (
"context"
"fmt"
"time"
)
// defaultTCPKeepAlive is a default constant value for TCPKeepAlive times
// See golang.org/issue/31510
const (
defaultTCPKeepAlive = 15 * time.Second
)
// A Dialer contains options for connecting to an address.
//
// The zero value for each field is equivalent to dialing
// without that option. Dialing with the zero value of Dialer
// is therefore equivalent to just calling the Dial function.
//
// It is safe to call Dialer's methods concurrently.
type Dialer struct {
Timeout time.Duration
Deadline time.Time
DualStack bool
// Timeout is the maximum amount of time a dial will wait for
// a connect to complete. If Deadline is also set, it may fail
// earlier.
//
// The default is no timeout.
//
// When using TCP and dialing a host name with multiple IP
// addresses, the timeout may be divided between them.
//
// With or without a timeout, the operating system may impose
// its own earlier timeout. For instance, TCP timeouts are
// often around 3 minutes.
Timeout time.Duration
// Deadline is the absolute point in time after which dials
// will fail. If Timeout is set, it may fail earlier.
// Zero means no deadline, or dependent on the operating system
// as with the Timeout option.
Deadline time.Time
// LocalAddr is the local address to use when dialing an
// address. The address must be of a compatible type for the
// network being dialed.
// If nil, a local address is automatically chosen.
LocalAddr Addr
// KeepAlive specifies the interval between keep-alive
// probes for an active network connection.
// If zero, keep-alive probes are sent with a default value
// (currently 15 seconds), if supported by the protocol and operating
// system. Network protocols or operating systems that do
// not support keep-alives ignore this field.
// If negative, keep-alive probes are disabled.
KeepAlive time.Duration
}
// Dial connects to the address on the named network.
//
// See Go "net" package Dial() for more information.
//
// Note: Tinygo Dial supports a subset of networks supported by Go Dial,
// specifically: "tcp", "tcp4", "udp", and "udp4". IP and unix networks are
// not supported.
func Dial(network, address string) (Conn, error) {
return nil, ErrNotImplemented
var d Dialer
return d.Dial(network, address)
}
func Listen(network, address string) (Listener, error) {
return nil, ErrNotImplemented
// DialTimeout acts like Dial but takes a timeout.
//
// The timeout includes name resolution, if required.
// When using TCP, and the host in the address parameter resolves to
// multiple IP addresses, the timeout is spread over each consecutive
// dial, such that each is given an appropriate fraction of the time
// to connect.
//
// See func Dial for a description of the network and address
// parameters.
func DialTimeout(network, address string, timeout time.Duration) (Conn, error) {
d := Dialer{Timeout: timeout}
return d.Dial(network, address)
}
// Dial connects to the address on the named network.
//
// See func Dial for a description of the network and address
// parameters.
//
// Dial uses context.Background internally; to specify the context, use
// DialContext.
func (d *Dialer) Dial(network, address string) (Conn, error) {
return d.DialContext(context.Background(), network, address)
}
// DialContext connects to the address on the named network using
// the provided context.
//
// The provided Context must be non-nil. If the context expires before
// the connection is complete, an error is returned. Once successfully
// connected, any expiration of the context will not affect the
// connection.
//
// When using TCP, and the host in the address parameter resolves to multiple
// network addresses, any dial timeout (from d.Timeout or ctx) is spread
// over each consecutive dial, such that each is given an appropriate
// fraction of the time to connect.
// For example, if a host has 4 IP addresses and the timeout is 1 minute,
// the connect to each single address will be given 15 seconds to complete
// before trying the next one.
//
// See func Dial for a description of the network and address
// parameters.
func (d *Dialer) DialContext(ctx context.Context, network, address string) (Conn, error) {
return nil, ErrNotImplemented
// TINYGO: Ignoring context
switch network {
case "tcp", "tcp4":
raddr, err := ResolveTCPAddr(network, address)
if err != nil {
return nil, err
}
return DialTCP(network, nil, raddr)
case "udp", "udp4":
raddr, err := ResolveUDPAddr(network, address)
if err != nil {
return nil, err
}
return DialUDP(network, nil, raddr)
}
return nil, fmt.Errorf("Network %s not supported", network)
}
// Listen announces on the local network address.
//
// See Go "net" package Listen() for more information.
//
// Note: Tinygo Listen supports a subset of networks supported by Go Listen,
// specifically: "tcp", "tcp4". "tcp6" and unix networks are not supported.
func Listen(network, address string) (Listener, error) {
// println("Listen", address)
switch network {
case "tcp", "tcp4":
default:
return nil, fmt.Errorf("Network %s not supported", network)
}
laddr, err := ResolveTCPAddr(network, address)
if err != nil {
return nil, err
}
return listenTCP(laddr)
}
errors.go
-10
View File
@@ -1,10 +0,0 @@
package net
import "errors"
var (
// copied from poll.ErrNetClosing
errClosed = errors.New("use of closed network connection")
ErrNotImplemented = errors.New("operation not implemented")
)
http/client.go
+523
View File
@@ -0,0 +1,523 @@
// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// HTTP client. See RFC 7230 through 7235.
//
// This is the high-level Client interface.
// The low-level implementation is in transport.go.
package http
import (
"bufio"
"crypto/tls"
"encoding/base64"
"errors"
"fmt"
"io"
"net"
"net/http/internal/ascii"
"net/url"
"strings"
"time"
"golang.org/x/net/http/httpguts"
)
// A Client is an HTTP client. Its zero value (DefaultClient) is a
// usable client that uses DefaultTransport.
//
// The Client's Transport typically has internal state (cached TCP
// connections), so Clients should be reused instead of created as
// needed. Clients are safe for concurrent use by multiple goroutines.
//
// A Client is higher-level than a RoundTripper (such as Transport)
// and additionally handles HTTP details such as cookies and
// redirects.
//
// When following redirects, the Client will forward all headers set on the
// initial Request except:
//
// • when forwarding sensitive headers like "Authorization",
// "WWW-Authenticate", and "Cookie" to untrusted targets.
// These headers will be ignored when following a redirect to a domain
// that is not a subdomain match or exact match of the initial domain.
// For example, a redirect from "foo.com" to either "foo.com" or "sub.foo.com"
// will forward the sensitive headers, but a redirect to "bar.com" will not.
//
// • when forwarding the "Cookie" header with a non-nil cookie Jar.
// Since each redirect may mutate the state of the cookie jar,
// a redirect may possibly alter a cookie set in the initial request.
// When forwarding the "Cookie" header, any mutated cookies will be omitted,
// with the expectation that the Jar will insert those mutated cookies
// with the updated values (assuming the origin matches).
// If Jar is nil, the initial cookies are forwarded without change.
type Client struct {
// Jar specifies the cookie jar.
//
// The Jar is used to insert relevant cookies into every
// outbound Request and is updated with the cookie values
// of every inbound Response. The Jar is consulted for every
// redirect that the Client follows.
//
// If Jar is nil, cookies are only sent if they are explicitly
// set on the Request.
Jar CookieJar
// Timeout specifies a time limit for requests made by this
// Client. The timeout includes connection time, any
// redirects, and reading the response body. The timer remains
// running after Get, Head, Post, or Do return and will
// interrupt reading of the Response.Body.
//
// A Timeout of zero means no timeout.
//
// The Client cancels requests to the underlying Transport
// as if the Request's Context ended.
//
// For compatibility, the Client will also use the deprecated
// CancelRequest method on Transport if found. New
// RoundTripper implementations should use the Request's Context
// for cancellation instead of implementing CancelRequest.
Timeout time.Duration
}
// DefaultClient is the default Client and is used by Get, Head, and Post.
var DefaultClient = &Client{}
// didTimeout is non-nil only if err != nil.
func (c *Client) send(req *Request, deadline time.Time) (resp *Response, didTimeout func() bool, err error) {
if c.Jar != nil {
for _, cookie := range c.Jar.Cookies(req.URL) {
req.AddCookie(cookie)
}
}
resp, didTimeout, err = send(req, deadline)
if err != nil {
return nil, didTimeout, err
}
if c.Jar != nil {
if rc := resp.Cookies(); len(rc) > 0 {
c.Jar.SetCookies(req.URL, rc)
}
}
return resp, nil, nil
}
func (c *Client) deadline() time.Time {
if c.Timeout > 0 {
return time.Now().Add(c.Timeout)
}
return time.Time{}
}
// send issues an HTTP request.
// Caller should close resp.Body when done reading from it.
func send(req *Request, deadline time.Time) (resp *Response, didTimeout func() bool, err error) {
// TINYGO: Removed round tripper
if req.URL == nil {
req.closeBody()
return nil, alwaysFalse, errors.New("http: nil Request.URL")
}
if req.RequestURI != "" {
req.closeBody()
return nil, alwaysFalse, errors.New("http: Request.RequestURI can't be set in client requests")
}
// TINYGO: Removed forkReq stuff
// Most the callers of send (Get, Post, et al) don't need
// Headers, leaving it uninitialized. We guarantee to the
// Transport that this has been initialized, though.
if req.Header == nil {
req.Header = make(Header)
}
if u := req.URL.User; u != nil && req.Header.Get("Authorization") == "" {
username := u.Username()
password, _ := u.Password()
req.Header.Set("Authorization", "Basic "+basicAuth(username, password))
}
resp, err = roundTrip(req)
if err != nil {
// TINYGO: Remove TLS error check
return nil, didTimeout, err
}
if resp == nil {
return nil, didTimeout, fmt.Errorf("http: sendit returned a nil *Response with a nil error")
}
// TINYGO: Skip check for resp.Body == nil since we'll set it in roundTrip
return resp, nil, nil
}
func roundTrip(req *Request) (*Response, error) {
// TINYGO: This is an approximation of Transport.roudTrip()
if req.URL == nil {
req.closeBody()
return nil, errors.New("http: nil Request.URL")
}
if req.Header == nil {
req.closeBody()
return nil, errors.New("http: nil Request.Header")
}
scheme := req.URL.Scheme
isHTTP := scheme == "http" || scheme == "https"
if isHTTP {
for k, vv := range req.Header {
if !httpguts.ValidHeaderFieldName(k) {
req.closeBody()
return nil, fmt.Errorf("net/http: invalid header field name %q", k)
}
for _, v := range vv {
if !httpguts.ValidHeaderFieldValue(v) {
req.closeBody()
// Don't include the value in the error, because it may be sensitive.
return nil, fmt.Errorf("net/http: invalid header field value for %q", k)
}
}
}
}
// TINYGO: Skipping alternate round tripper
if !isHTTP {
req.closeBody()
return nil, badStringError("unsupported protocol scheme", scheme)
}
if req.Method != "" && !validMethod(req.Method) {
req.closeBody()
return nil, fmt.Errorf("net/http: invalid method %q", req.Method)
}
if req.URL.Host == "" {
req.closeBody()
return nil, errors.New("http: no Host in request URL")
}
// TINYGO: From here on just brute force dial a connection,
// TINYGO: send the request, read and return the response.
// TINYGO: The connection is closed when resp body is closed.
var conn net.Conn
var err error
host := req.Host
missingPort := !strings.Contains(host, ":")
switch scheme {
case "http":
if missingPort {
host = host + ":80"
}
conn, err = net.Dial("tcp", host)
case "https":
if missingPort {
host = host + ":443"
}
conn, err = tls.Dial("tcp", host, nil)
}
if err != nil {
req.closeBody()
return nil, err
}
// TINYGO: TODO handle timeouts
writer := bufio.NewWriter(conn)
if err = req.Write(writer); err != nil {
req.closeBody()
return nil, err
}
req.closeBody()
if err = writer.Flush(); err != nil {
return nil, err
}
req.onEOF = func() { conn.Close() }
reader := bufio.NewReader(conn)
return ReadResponse(reader, req)
}
// See 2 (end of page 4) https://www.ietf.org/rfc/rfc2617.txt
// "To receive authorization, the client sends the userid and password,
// separated by a single colon (":") character, within a base64
// encoded string in the credentials."
// It is not meant to be urlencoded.
func basicAuth(username, password string) string {
auth := username + ":" + password
return base64.StdEncoding.EncodeToString([]byte(auth))
}
// Get issues a GET to the specified URL. If the response is one of
// the following redirect codes, Get follows the redirect, up to a
// maximum of 10 redirects:
//
// 301 (Moved Permanently)
// 302 (Found)
// 303 (See Other)
// 307 (Temporary Redirect)
// 308 (Permanent Redirect)
//
// An error is returned if there were too many redirects or if there
// was an HTTP protocol error. A non-2xx response doesn't cause an
// error. Any returned error will be of type *url.Error. The url.Error
// value's Timeout method will report true if the request timed out.
//
// When err is nil, resp always contains a non-nil resp.Body.
// Caller should close resp.Body when done reading from it.
//
// Get is a wrapper around DefaultClient.Get.
//
// To make a request with custom headers, use NewRequest and
// DefaultClient.Do.
//
// To make a request with a specified context.Context, use NewRequestWithContext
// and DefaultClient.Do.
func Get(url string) (resp *Response, err error) {
return DefaultClient.Get(url)
}
// Get issues a GET to the specified URL. If the response is one of the
// following redirect codes, Get follows the redirect after calling the
// Client's CheckRedirect function:
//
// 301 (Moved Permanently)
// 302 (Found)
// 303 (See Other)
// 307 (Temporary Redirect)
// 308 (Permanent Redirect)
//
// An error is returned if the Client's CheckRedirect function fails
// or if there was an HTTP protocol error. A non-2xx response doesn't
// cause an error. Any returned error will be of type *url.Error. The
// url.Error value's Timeout method will report true if the request
// timed out.
//
// When err is nil, resp always contains a non-nil resp.Body.
// Caller should close resp.Body when done reading from it.
//
// To make a request with custom headers, use NewRequest and Client.Do.
//
// To make a request with a specified context.Context, use NewRequestWithContext
// and Client.Do.
func (c *Client) Get(url string) (resp *Response, err error) {
req, err := NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
return c.Do(req)
}
func alwaysFalse() bool { return false }
// urlErrorOp returns the (*url.Error).Op value to use for the
// provided (*Request).Method value.
func urlErrorOp(method string) string {
if method == "" {
return "Get"
}
if lowerMethod, ok := ascii.ToLower(method); ok {
return method[:1] + lowerMethod[1:]
}
return method
}
// Do sends an HTTP request and returns an HTTP response, following
// policy (such as redirects, cookies, auth) as configured on the
// client.
//
// An error is returned if caused by client policy (such as
// CheckRedirect), or failure to speak HTTP (such as a network
// connectivity problem). A non-2xx status code doesn't cause an
// error.
//
// If the returned error is nil, the Response will contain a non-nil
// Body which the user is expected to close. If the Body is not both
// read to EOF and closed, the Client's underlying RoundTripper
// (typically Transport) may not be able to re-use a persistent TCP
// connection to the server for a subsequent "keep-alive" request.
//
// The request Body, if non-nil, will be closed by the underlying
// Transport, even on errors.
//
// On error, any Response can be ignored. A non-nil Response with a
// non-nil error only occurs when CheckRedirect fails, and even then
// the returned Response.Body is already closed.
//
// Generally Get, Post, or PostForm will be used instead of Do.
//
// If the server replies with a redirect, the Client first uses the
// CheckRedirect function to determine whether the redirect should be
// followed. If permitted, a 301, 302, or 303 redirect causes
// subsequent requests to use HTTP method GET
// (or HEAD if the original request was HEAD), with no body.
// A 307 or 308 redirect preserves the original HTTP method and body,
// provided that the Request.GetBody function is defined.
// The NewRequest function automatically sets GetBody for common
// standard library body types.
//
// Any returned error will be of type *url.Error. The url.Error
// value's Timeout method will report true if the request timed out.
func (c *Client) Do(req *Request) (*Response, error) {
return c.do(req)
}
func (c *Client) do(req *Request) (retres *Response, reterr error) {
if req.URL == nil {
req.closeBody()
return nil, &url.Error{
Op: urlErrorOp(req.Method),
Err: errors.New("http: nil Request.URL"),
}
}
var err error
var didTimeout func() bool
var resp *Response
var deadline = c.deadline()
// TINYGO: lots removed here, mostly handling multiple requests.
// TINYGO: we just want simple GET, POST, etc. In and out.
if resp, didTimeout, err = c.send(req, deadline); err != nil {
// c.send() always closes req.Body
if !deadline.IsZero() && didTimeout() {
return nil, fmt.Errorf("%s (Client.Timeout exceeded while awaiting headers)", err.Error())
}
return nil, err
}
return resp, nil
}
// Post issues a POST to the specified URL.
//
// Caller should close resp.Body when done reading from it.
//
// If the provided body is an io.Closer, it is closed after the
// request.
//
// Post is a wrapper around DefaultClient.Post.
//
// To set custom headers, use NewRequest and DefaultClient.Do.
//
// See the Client.Do method documentation for details on how redirects
// are handled.
//
// To make a request with a specified context.Context, use NewRequestWithContext
// and DefaultClient.Do.
func Post(url, contentType string, body io.Reader) (resp *Response, err error) {
return DefaultClient.Post(url, contentType, body)
}
// Post issues a POST to the specified URL.
//
// Caller should close resp.Body when done reading from it.
//
// If the provided body is an io.Closer, it is closed after the
// request.
//
// To set custom headers, use NewRequest and Client.Do.
//
// To make a request with a specified context.Context, use NewRequestWithContext
// and Client.Do.
//
// See the Client.Do method documentation for details on how redirects
// are handled.
func (c *Client) Post(url, contentType string, body io.Reader) (resp *Response, err error) {
req, err := NewRequest("POST", url, body)
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", contentType)
return c.Do(req)
}
// PostForm issues a POST to the specified URL, with data's keys and
// values URL-encoded as the request body.
//
// The Content-Type header is set to application/x-www-form-urlencoded.
// To set other headers, use NewRequest and DefaultClient.Do.
//
// When err is nil, resp always contains a non-nil resp.Body.
// Caller should close resp.Body when done reading from it.
//
// PostForm is a wrapper around DefaultClient.PostForm.
//
// See the Client.Do method documentation for details on how redirects
// are handled.
//
// To make a request with a specified context.Context, use NewRequestWithContext
// and DefaultClient.Do.
func PostForm(url string, data url.Values) (resp *Response, err error) {
return DefaultClient.PostForm(url, data)
}
// PostForm issues a POST to the specified URL,
// with data's keys and values URL-encoded as the request body.
//
// The Content-Type header is set to application/x-www-form-urlencoded.
// To set other headers, use NewRequest and Client.Do.
//
// When err is nil, resp always contains a non-nil resp.Body.
// Caller should close resp.Body when done reading from it.
//
// See the Client.Do method documentation for details on how redirects
// are handled.
//
// To make a request with a specified context.Context, use NewRequestWithContext
// and Client.Do.
func (c *Client) PostForm(url string, data url.Values) (resp *Response, err error) {
return c.Post(url, "application/x-www-form-urlencoded", strings.NewReader(data.Encode()))
}
// Head issues a HEAD to the specified URL. If the response is one of
// the following redirect codes, Head follows the redirect, up to a
// maximum of 10 redirects:
//
// 301 (Moved Permanently)
// 302 (Found)
// 303 (See Other)
// 307 (Temporary Redirect)
// 308 (Permanent Redirect)
//
// Head is a wrapper around DefaultClient.Head.
//
// To make a request with a specified context.Context, use NewRequestWithContext
// and DefaultClient.Do.
func Head(url string) (resp *Response, err error) {
return DefaultClient.Head(url)
}
// Head issues a HEAD to the specified URL. If the response is one of the
// following redirect codes, Head follows the redirect after calling the
// Client's CheckRedirect function:
//
// 301 (Moved Permanently)
// 302 (Found)
// 303 (See Other)
// 307 (Temporary Redirect)
// 308 (Permanent Redirect)
//
// To make a request with a specified context.Context, use NewRequestWithContext
// and Client.Do.
func (c *Client) Head(url string) (resp *Response, err error) {
req, err := NewRequest("HEAD", url, nil)
if err != nil {
return nil, err
}
return c.Do(req)
}
http/clone.go
+76
View File
@@ -0,0 +1,76 @@
// TINYGO: The following is copied from Go 1.19.3 official implementation.
// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package http
import (
"mime/multipart"
"net/textproto"
"net/url"
)
func cloneURLValues(v url.Values) url.Values {
if v == nil {
return nil
}
// http.Header and url.Values have the same representation, so temporarily
// treat it like http.Header, which does have a clone:
return url.Values(Header(v).Clone())
}
func cloneURL(u *url.URL) *url.URL {
if u == nil {
return nil
}
u2 := new(url.URL)
*u2 = *u
if u.User != nil {
u2.User = new(url.Userinfo)
*u2.User = *u.User
}
return u2
}
func cloneMultipartForm(f *multipart.Form) *multipart.Form {
if f == nil {
return nil
}
f2 := &multipart.Form{
Value: (map[string][]string)(Header(f.Value).Clone()),
}
if f.File != nil {
m := make(map[string][]*multipart.FileHeader)
for k, vv := range f.File {
vv2 := make([]*multipart.FileHeader, len(vv))
for i, v := range vv {
vv2[i] = cloneMultipartFileHeader(v)
}
m[k] = vv2
}
f2.File = m
}
return f2
}
func cloneMultipartFileHeader(fh *multipart.FileHeader) *multipart.FileHeader {
if fh == nil {
return nil
}
fh2 := new(multipart.FileHeader)
*fh2 = *fh
fh2.Header = textproto.MIMEHeader(Header(fh.Header).Clone())
return fh2
}
// cloneOrMakeHeader invokes Header.Clone but if the
// result is nil, it'll instead make and return a non-nil Header.
func cloneOrMakeHeader(hdr Header) Header {
clone := hdr.Clone()
if clone == nil {
clone = make(Header)
}
return clone
}
http/cookie.go
+470
View File
@@ -0,0 +1,470 @@
// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package http
import (
"errors"
"fmt"
"log"
"net"
"net/http/internal/ascii"
"net/textproto"
"strconv"
"strings"
"time"
)
// A Cookie represents an HTTP cookie as sent in the Set-Cookie header of an
// HTTP response or the Cookie header of an HTTP request.
//
// See https://tools.ietf.org/html/rfc6265 for details.
type Cookie struct {
Name string
Value string
Path string // optional
Domain string // optional
Expires time.Time // optional
RawExpires string // for reading cookies only
// MaxAge=0 means no 'Max-Age' attribute specified.
// MaxAge<0 means delete cookie now, equivalently 'Max-Age: 0'
// MaxAge>0 means Max-Age attribute present and given in seconds
MaxAge int
Secure bool
HttpOnly bool
SameSite SameSite
Raw string
Unparsed []string // Raw text of unparsed attribute-value pairs
}
// SameSite allows a server to define a cookie attribute making it impossible for
// the browser to send this cookie along with cross-site requests. The main
// goal is to mitigate the risk of cross-origin information leakage, and provide
// some protection against cross-site request forgery attacks.
//
// See https://tools.ietf.org/html/draft-ietf-httpbis-cookie-same-site-00 for details.
type SameSite int
const (
SameSiteDefaultMode SameSite = iota + 1
SameSiteLaxMode
SameSiteStrictMode
SameSiteNoneMode
)
// readSetCookies parses all "Set-Cookie" values from
// the header h and returns the successfully parsed Cookies.
func readSetCookies(h Header) []*Cookie {
cookieCount := len(h["Set-Cookie"])
if cookieCount == 0 {
return []*Cookie{}
}
cookies := make([]*Cookie, 0, cookieCount)
for _, line := range h["Set-Cookie"] {
parts := strings.Split(textproto.TrimString(line), ";")
if len(parts) == 1 && parts[0] == "" {
continue
}
parts[0] = textproto.TrimString(parts[0])
name, value, ok := strings.Cut(parts[0], "=")
if !ok {
continue
}
name = textproto.TrimString(name)
if !isCookieNameValid(name) {
continue
}
value, ok = parseCookieValue(value, true)
if !ok {
continue
}
c := &Cookie{
Name: name,
Value: value,
Raw: line,
}
for i := 1; i < len(parts); i++ {
parts[i] = textproto.TrimString(parts[i])
if len(parts[i]) == 0 {
continue
}
attr, val, _ := strings.Cut(parts[i], "=")
lowerAttr, isASCII := ascii.ToLower(attr)
if !isASCII {
continue
}
val, ok = parseCookieValue(val, false)
if !ok {
c.Unparsed = append(c.Unparsed, parts[i])
continue
}
switch lowerAttr {
case "samesite":
lowerVal, ascii := ascii.ToLower(val)
if !ascii {
c.SameSite = SameSiteDefaultMode
continue
}
switch lowerVal {
case "lax":
c.SameSite = SameSiteLaxMode
case "strict":
c.SameSite = SameSiteStrictMode
case "none":
c.SameSite = SameSiteNoneMode
default:
c.SameSite = SameSiteDefaultMode
}
continue
case "secure":
c.Secure = true
continue
case "httponly":
c.HttpOnly = true
continue
case "domain":
c.Domain = val
continue
case "max-age":
secs, err := strconv.Atoi(val)
if err != nil || secs != 0 && val[0] == '0' {
break
}
if secs <= 0 {
secs = -1
}
c.MaxAge = secs
continue
case "expires":
c.RawExpires = val
exptime, err := time.Parse(time.RFC1123, val)
if err != nil {
exptime, err = time.Parse("Mon, 02-Jan-2006 15:04:05 MST", val)
if err != nil {
c.Expires = time.Time{}
break
}
}
c.Expires = exptime.UTC()
continue
case "path":
c.Path = val
continue
}
c.Unparsed = append(c.Unparsed, parts[i])
}
cookies = append(cookies, c)
}
return cookies
}
// SetCookie adds a Set-Cookie header to the provided ResponseWriter's headers.
// The provided cookie must have a valid Name. Invalid cookies may be
// silently dropped.
func SetCookie(w ResponseWriter, cookie *Cookie) {
if v := cookie.String(); v != "" {
w.Header().Add("Set-Cookie", v)
}
}
// String returns the serialization of the cookie for use in a Cookie
// header (if only Name and Value are set) or a Set-Cookie response
// header (if other fields are set).
// If c is nil or c.Name is invalid, the empty string is returned.
func (c *Cookie) String() string {
if c == nil || !isCookieNameValid(c.Name) {
return ""
}
// extraCookieLength derived from typical length of cookie attributes
// see RFC 6265 Sec 4.1.
const extraCookieLength = 110
var b strings.Builder
b.Grow(len(c.Name) + len(c.Value) + len(c.Domain) + len(c.Path) + extraCookieLength)
b.WriteString(c.Name)
b.WriteRune('=')
b.WriteString(sanitizeCookieValue(c.Value))
if len(c.Path) > 0 {
b.WriteString("; Path=")
b.WriteString(sanitizeCookiePath(c.Path))
}
if len(c.Domain) > 0 {
if validCookieDomain(c.Domain) {
// A c.Domain containing illegal characters is not
// sanitized but simply dropped which turns the cookie
// into a host-only cookie. A leading dot is okay
// but won't be sent.
d := c.Domain
if d[0] == '.' {
d = d[1:]
}
b.WriteString("; Domain=")
b.WriteString(d)
} else {
log.Printf("net/http: invalid Cookie.Domain %q; dropping domain attribute", c.Domain)
}
}
var buf [len(TimeFormat)]byte
if validCookieExpires(c.Expires) {
b.WriteString("; Expires=")
b.Write(c.Expires.UTC().AppendFormat(buf[:0], TimeFormat))
}
if c.MaxAge > 0 {
b.WriteString("; Max-Age=")
b.Write(strconv.AppendInt(buf[:0], int64(c.MaxAge), 10))
} else if c.MaxAge < 0 {
b.WriteString("; Max-Age=0")
}
if c.HttpOnly {
b.WriteString("; HttpOnly")
}
if c.Secure {
b.WriteString("; Secure")
}
switch c.SameSite {
case SameSiteDefaultMode:
// Skip, default mode is obtained by not emitting the attribute.
case SameSiteNoneMode:
b.WriteString("; SameSite=None")
case SameSiteLaxMode:
b.WriteString("; SameSite=Lax")
case SameSiteStrictMode:
b.WriteString("; SameSite=Strict")
}
return b.String()
}
// Valid reports whether the cookie is valid.
func (c *Cookie) Valid() error {
if c == nil {
return errors.New("http: nil Cookie")
}
if !isCookieNameValid(c.Name) {
return errors.New("http: invalid Cookie.Name")
}
if !c.Expires.IsZero() && !validCookieExpires(c.Expires) {
return errors.New("http: invalid Cookie.Expires")
}
for i := 0; i < len(c.Value); i++ {
if !validCookieValueByte(c.Value[i]) {
return fmt.Errorf("http: invalid byte %q in Cookie.Value", c.Value[i])
}
}
if len(c.Path) > 0 {
for i := 0; i < len(c.Path); i++ {
if !validCookiePathByte(c.Path[i]) {
return fmt.Errorf("http: invalid byte %q in Cookie.Path", c.Path[i])
}
}
}
if len(c.Domain) > 0 {
if !validCookieDomain(c.Domain) {
return errors.New("http: invalid Cookie.Domain")
}
}
return nil
}
// readCookies parses all "Cookie" values from the header h and
// returns the successfully parsed Cookies.
//
// if filter isn't empty, only cookies of that name are returned.
func readCookies(h Header, filter string) []*Cookie {
lines := h["Cookie"]
if len(lines) == 0 {
return []*Cookie{}
}
cookies := make([]*Cookie, 0, len(lines)+strings.Count(lines[0], ";"))
for _, line := range lines {
line = textproto.TrimString(line)
var part string
for len(line) > 0 { // continue since we have rest
part, line, _ = strings.Cut(line, ";")
part = textproto.TrimString(part)
if part == "" {
continue
}
name, val, _ := strings.Cut(part, "=")
name = textproto.TrimString(name)
if !isCookieNameValid(name) {
continue
}
if filter != "" && filter != name {
continue
}
val, ok := parseCookieValue(val, true)
if !ok {
continue
}
cookies = append(cookies, &Cookie{Name: name, Value: val})
}
}
return cookies
}
// validCookieDomain reports whether v is a valid cookie domain-value.
func validCookieDomain(v string) bool {
if isCookieDomainName(v) {
return true
}
if net.ParseIP(v) != nil && !strings.Contains(v, ":") {
return true
}
return false
}
// validCookieExpires reports whether v is a valid cookie expires-value.
func validCookieExpires(t time.Time) bool {
// IETF RFC 6265 Section 5.1.1.5, the year must not be less than 1601
return t.Year() >= 1601
}
// isCookieDomainName reports whether s is a valid domain name or a valid
// domain name with a leading dot '.'. It is almost a direct copy of
// package net's isDomainName.
func isCookieDomainName(s string) bool {
if len(s) == 0 {
return false
}
if len(s) > 255 {
return false
}
if s[0] == '.' {
// A cookie a domain attribute may start with a leading dot.
s = s[1:]
}
last := byte('.')
ok := false // Ok once we've seen a letter.
partlen := 0
for i := 0; i < len(s); i++ {
c := s[i]
switch {
default:
return false
case 'a' <= c && c <= 'z' || 'A' <= c && c <= 'Z':
// No '_' allowed here (in contrast to package net).
ok = true
partlen++
case '0' <= c && c <= '9':
// fine
partlen++
case c == '-':
// Byte before dash cannot be dot.
if last == '.' {
return false
}
partlen++
case c == '.':
// Byte before dot cannot be dot, dash.
if last == '.' || last == '-' {
return false
}
if partlen > 63 || partlen == 0 {
return false
}
partlen = 0
}
last = c
}
if last == '-' || partlen > 63 {
return false
}
return ok
}
var cookieNameSanitizer = strings.NewReplacer("\n", "-", "\r", "-")
func sanitizeCookieName(n string) string {
return cookieNameSanitizer.Replace(n)
}
// sanitizeCookieValue produces a suitable cookie-value from v.
// https://tools.ietf.org/html/rfc6265#section-4.1.1
//
// cookie-value = *cookie-octet / ( DQUOTE *cookie-octet DQUOTE )
// cookie-octet = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E
// ; US-ASCII characters excluding CTLs,
// ; whitespace DQUOTE, comma, semicolon,
// ; and backslash
//
// We loosen this as spaces and commas are common in cookie values
// but we produce a quoted cookie-value if and only if v contains
// commas or spaces.
// See https://golang.org/issue/7243 for the discussion.
func sanitizeCookieValue(v string) string {
v = sanitizeOrWarn("Cookie.Value", validCookieValueByte, v)
if len(v) == 0 {
return v
}
if strings.ContainsAny(v, " ,") {
return `"` + v + `"`
}
return v
}
func validCookieValueByte(b byte) bool {
return 0x20 <= b && b < 0x7f && b != '"' && b != ';' && b != '\\'
}
// path-av = "Path=" path-value
// path-value = <any CHAR except CTLs or ";">
func sanitizeCookiePath(v string) string {
return sanitizeOrWarn("Cookie.Path", validCookiePathByte, v)
}
func validCookiePathByte(b byte) bool {
return 0x20 <= b && b < 0x7f && b != ';'
}
func sanitizeOrWarn(fieldName string, valid func(byte) bool, v string) string {
ok := true
for i := 0; i < len(v); i++ {
if valid(v[i]) {
continue
}
log.Printf("net/http: invalid byte %q in %s; dropping invalid bytes", v[i], fieldName)
ok = false
break
}
if ok {
return v
}
buf := make([]byte, 0, len(v))
for i := 0; i < len(v); i++ {
if b := v[i]; valid(b) {
buf = append(buf, b)
}
}
return string(buf)
}
func parseCookieValue(raw string, allowDoubleQuote bool) (string, bool) {
// Strip the quotes, if present.
if allowDoubleQuote && len(raw) > 1 && raw[0] == '"' && raw[len(raw)-1] == '"' {
raw = raw[1 : len(raw)-1]
}
for i := 0; i < len(raw); i++ {
if !validCookieValueByte(raw[i]) {
return "", false
}
}
return raw, true
}
func isCookieNameValid(raw string) bool {
if raw == "" {
return false
}
return strings.IndexFunc(raw, isNotToken) < 0
}
http/fs.go
+974
View File
@@ -0,0 +1,974 @@
// TINYGO: The following is copied from Go 1.19.3 official implementation.
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// HTTP file system request handler
package http
import (
"errors"
"fmt"
"io"
"io/fs"
"mime"
"mime/multipart"
"net/textproto"
"net/url"
"os"
"path"
"path/filepath"
"sort"
"strconv"
"strings"
"time"
)
// A Dir implements FileSystem using the native file system restricted to a
// specific directory tree.
//
// While the FileSystem.Open method takes '/'-separated paths, a Dir's string
// value is a filename on the native file system, not a URL, so it is separated
// by filepath.Separator, which isn't necessarily '/'.
//
// Note that Dir could expose sensitive files and directories. Dir will follow
// symlinks pointing out of the directory tree, which can be especially dangerous
// if serving from a directory in which users are able to create arbitrary symlinks.
// Dir will also allow access to files and directories starting with a period,
// which could expose sensitive directories like .git or sensitive files like
// .htpasswd. To exclude files with a leading period, remove the files/directories
// from the server or create a custom FileSystem implementation.
//
// An empty Dir is treated as ".".
type Dir string
// mapOpenError maps the provided non-nil error from opening name
// to a possibly better non-nil error. In particular, it turns OS-specific errors
// about opening files in non-directories into fs.ErrNotExist. See Issues 18984 and 49552.
func mapOpenError(originalErr error, name string, sep rune, stat func(string) (fs.FileInfo, error)) error {
if errors.Is(originalErr, fs.ErrNotExist) || errors.Is(originalErr, fs.ErrPermission) {
return originalErr
}
parts := strings.Split(name, string(sep))
for i := range parts {
if parts[i] == "" {
continue
}
fi, err := stat(strings.Join(parts[:i+1], string(sep)))
if err != nil {
return originalErr
}
if !fi.IsDir() {
return fs.ErrNotExist
}
}
return originalErr
}
// Open implements FileSystem using os.Open, opening files for reading rooted
// and relative to the directory d.
func (d Dir) Open(name string) (File, error) {
if filepath.Separator != '/' && strings.ContainsRune(name, filepath.Separator) {
return nil, errors.New("http: invalid character in file path")
}
dir := string(d)
if dir == "" {
dir = "."
}
fullName := filepath.Join(dir, filepath.FromSlash(path.Clean("/"+name)))
f, err := os.Open(fullName)
if err != nil {
return nil, mapOpenError(err, fullName, filepath.Separator, os.Stat)
}
return f, nil
}
// A FileSystem implements access to a collection of named files.
// The elements in a file path are separated by slash ('/', U+002F)
// characters, regardless of host operating system convention.
// See the FileServer function to convert a FileSystem to a Handler.
//
// This interface predates the fs.FS interface, which can be used instead:
// the FS adapter function converts an fs.FS to a FileSystem.
type FileSystem interface {
Open(name string) (File, error)
}
// A File is returned by a FileSystem's Open method and can be
// served by the FileServer implementation.
//
// The methods should behave the same as those on an *os.File.
type File interface {
io.Closer
io.Reader
io.Seeker
Readdir(count int) ([]fs.FileInfo, error)
Stat() (fs.FileInfo, error)
}
type anyDirs interface {
len() int
name(i int) string
isDir(i int) bool
}
type fileInfoDirs []fs.FileInfo
func (d fileInfoDirs) len() int { return len(d) }
func (d fileInfoDirs) isDir(i int) bool { return d[i].IsDir() }
func (d fileInfoDirs) name(i int) string { return d[i].Name() }
type dirEntryDirs []fs.DirEntry
func (d dirEntryDirs) len() int { return len(d) }
func (d dirEntryDirs) isDir(i int) bool { return d[i].IsDir() }
func (d dirEntryDirs) name(i int) string { return d[i].Name() }
func dirList(w ResponseWriter, r *Request, f File) {
// Prefer to use ReadDir instead of Readdir,
// because the former doesn't require calling
// Stat on every entry of a directory on Unix.
var dirs anyDirs
var err error
if d, ok := f.(fs.ReadDirFile); ok {
var list dirEntryDirs
list, err = d.ReadDir(-1)
dirs = list
} else {
var list fileInfoDirs
list, err = f.Readdir(-1)
dirs = list
}
if err != nil {
logf(r, "http: error reading directory: %v", err)
Error(w, "Error reading directory", StatusInternalServerError)
return
}
sort.Slice(dirs, func(i, j int) bool { return dirs.name(i) < dirs.name(j) })
w.Header().Set("Content-Type", "text/html; charset=utf-8")
fmt.Fprintf(w, "<pre>\n")
for i, n := 0, dirs.len(); i < n; i++ {
name := dirs.name(i)
if dirs.isDir(i) {
name += "/"
}
// name may contain '?' or '#', which must be escaped to remain
// part of the URL path, and not indicate the start of a query
// string or fragment.
url := url.URL{Path: name}
fmt.Fprintf(w, "<a href=\"%s\">%s</a>\n", url.String(), htmlReplacer.Replace(name))
}
fmt.Fprintf(w, "</pre>\n")
}
// ServeContent replies to the request using the content in the
// provided ReadSeeker. The main benefit of ServeContent over io.Copy
// is that it handles Range requests properly, sets the MIME type, and
// handles If-Match, If-Unmodified-Since, If-None-Match, If-Modified-Since,
// and If-Range requests.
//
// If the response's Content-Type header is not set, ServeContent
// first tries to deduce the type from name's file extension and,
// if that fails, falls back to reading the first block of the content
// and passing it to DetectContentType.
// The name is otherwise unused; in particular it can be empty and is
// never sent in the response.
//
// If modtime is not the zero time or Unix epoch, ServeContent
// includes it in a Last-Modified header in the response. If the
// request includes an If-Modified-Since header, ServeContent uses
// modtime to decide whether the content needs to be sent at all.
//
// The content's Seek method must work: ServeContent uses
// a seek to the end of the content to determine its size.
//
// If the caller has set w's ETag header formatted per RFC 7232, section 2.3,
// ServeContent uses it to handle requests using If-Match, If-None-Match, or If-Range.
//
// Note that *os.File implements the io.ReadSeeker interface.
func ServeContent(w ResponseWriter, req *Request, name string, modtime time.Time, content io.ReadSeeker) {
sizeFunc := func() (int64, error) {
size, err := content.Seek(0, io.SeekEnd)
if err != nil {
return 0, errSeeker
}
_, err = content.Seek(0, io.SeekStart)
if err != nil {
return 0, errSeeker
}
return size, nil
}
serveContent(w, req, name, modtime, sizeFunc, content)
}
// errSeeker is returned by ServeContent's sizeFunc when the content
// doesn't seek properly. The underlying Seeker's error text isn't
// included in the sizeFunc reply so it's not sent over HTTP to end
// users.
var errSeeker = errors.New("seeker can't seek")
// errNoOverlap is returned by serveContent's parseRange if first-byte-pos of
// all of the byte-range-spec values is greater than the content size.
var errNoOverlap = errors.New("invalid range: failed to overlap")
// if name is empty, filename is unknown. (used for mime type, before sniffing)
// if modtime.IsZero(), modtime is unknown.
// content must be seeked to the beginning of the file.
// The sizeFunc is called at most once. Its error, if any, is sent in the HTTP response.
func serveContent(w ResponseWriter, r *Request, name string, modtime time.Time, sizeFunc func() (int64, error), content io.ReadSeeker) {
setLastModified(w, modtime)
done, rangeReq := checkPreconditions(w, r, modtime)
if done {
return
}
code := StatusOK
// If Content-Type isn't set, use the file's extension to find it, but
// if the Content-Type is unset explicitly, do not sniff the type.
ctypes, haveType := w.Header()["Content-Type"]
var ctype string
if !haveType {
ctype = mime.TypeByExtension(filepath.Ext(name))
if ctype == "" {
// read a chunk to decide between utf-8 text and binary
var buf [sniffLen]byte
n, _ := io.ReadFull(content, buf[:])
ctype = DetectContentType(buf[:n])
_, err := content.Seek(0, io.SeekStart) // rewind to output whole file
if err != nil {
Error(w, "seeker can't seek", StatusInternalServerError)
return
}
}
w.Header().Set("Content-Type", ctype)
} else if len(ctypes) > 0 {
ctype = ctypes[0]
}
size, err := sizeFunc()
if err != nil {
Error(w, err.Error(), StatusInternalServerError)
return
}
// handle Content-Range header.
sendSize := size
var sendContent io.Reader = content
if size >= 0 {
ranges, err := parseRange(rangeReq, size)
if err != nil {
if err == errNoOverlap {
w.Header().Set("Content-Range", fmt.Sprintf("bytes */%d", size))
}
Error(w, err.Error(), StatusRequestedRangeNotSatisfiable)
return
}
if sumRangesSize(ranges) > size {
// The total number of bytes in all the ranges
// is larger than the size of the file by
// itself, so this is probably an attack, or a
// dumb client. Ignore the range request.
ranges = nil
}
switch {
case len(ranges) == 1:
// RFC 7233, Section 4.1:
// "If a single part is being transferred, the server
// generating the 206 response MUST generate a
// Content-Range header field, describing what range
// of the selected representation is enclosed, and a
// payload consisting of the range.
// ...
// A server MUST NOT generate a multipart response to
// a request for a single range, since a client that
// does not request multiple parts might not support
// multipart responses."
ra := ranges[0]
if _, err := content.Seek(ra.start, io.SeekStart); err != nil {
Error(w, err.Error(), StatusRequestedRangeNotSatisfiable)
return
}
sendSize = ra.length
code = StatusPartialContent
w.Header().Set("Content-Range", ra.contentRange(size))
case len(ranges) > 1:
sendSize = rangesMIMESize(ranges, ctype, size)
code = StatusPartialContent
pr, pw := io.Pipe()
mw := multipart.NewWriter(pw)
w.Header().Set("Content-Type", "multipart/byteranges; boundary="+mw.Boundary())
sendContent = pr
defer pr.Close() // cause writing goroutine to fail and exit if CopyN doesn't finish.
go func() {
for _, ra := range ranges {
part, err := mw.CreatePart(ra.mimeHeader(ctype, size))
if err != nil {
pw.CloseWithError(err)
return
}
if _, err := content.Seek(ra.start, io.SeekStart); err != nil {
pw.CloseWithError(err)
return
}
if _, err := io.CopyN(part, content, ra.length); err != nil {
pw.CloseWithError(err)
return
}
}
mw.Close()
pw.Close()
}()
}
w.Header().Set("Accept-Ranges", "bytes")
if w.Header().Get("Content-Encoding") == "" {
w.Header().Set("Content-Length", strconv.FormatInt(sendSize, 10))
}
}
w.WriteHeader(code)
if r.Method != "HEAD" {
io.CopyN(w, sendContent, sendSize)
}
}
// scanETag determines if a syntactically valid ETag is present at s. If so,
// the ETag and remaining text after consuming ETag is returned. Otherwise,
// it returns "", "".
func scanETag(s string) (etag string, remain string) {
s = textproto.TrimString(s)
start := 0
if strings.HasPrefix(s, "W/") {
start = 2
}
if len(s[start:]) < 2 || s[start] != '"' {
return "", ""
}
// ETag is either W/"text" or "text".
// See RFC 7232 2.3.
for i := start + 1; i < len(s); i++ {
c := s[i]
switch {
// Character values allowed in ETags.
case c == 0x21 || c >= 0x23 && c <= 0x7E || c >= 0x80:
case c == '"':
return s[:i+1], s[i+1:]
default:
return "", ""
}
}
return "", ""
}
// etagStrongMatch reports whether a and b match using strong ETag comparison.
// Assumes a and b are valid ETags.
func etagStrongMatch(a, b string) bool {
return a == b && a != "" && a[0] == '"'
}
// etagWeakMatch reports whether a and b match using weak ETag comparison.
// Assumes a and b are valid ETags.
func etagWeakMatch(a, b string) bool {
return strings.TrimPrefix(a, "W/") == strings.TrimPrefix(b, "W/")
}
// condResult is the result of an HTTP request precondition check.
// See https://tools.ietf.org/html/rfc7232 section 3.
type condResult int
const (
condNone condResult = iota
condTrue
condFalse
)
func checkIfMatch(w ResponseWriter, r *Request) condResult {
im := r.Header.Get("If-Match")
if im == "" {
return condNone
}
for {
im = textproto.TrimString(im)
if len(im) == 0 {
break
}
if im[0] == ',' {
im = im[1:]
continue
}
if im[0] == '*' {
return condTrue
}
etag, remain := scanETag(im)
if etag == "" {
break
}
if etagStrongMatch(etag, w.Header().get("Etag")) {
return condTrue
}
im = remain
}
return condFalse
}
func checkIfUnmodifiedSince(r *Request, modtime time.Time) condResult {
ius := r.Header.Get("If-Unmodified-Since")
if ius == "" || isZeroTime(modtime) {
return condNone
}
t, err := ParseTime(ius)
if err != nil {
return condNone
}
// The Last-Modified header truncates sub-second precision so
// the modtime needs to be truncated too.
modtime = modtime.Truncate(time.Second)
if modtime.Before(t) || modtime.Equal(t) {
return condTrue
}
return condFalse
}
func checkIfNoneMatch(w ResponseWriter, r *Request) condResult {
inm := r.Header.get("If-None-Match")
if inm == "" {
return condNone
}
buf := inm
for {
buf = textproto.TrimString(buf)
if len(buf) == 0 {
break
}
if buf[0] == ',' {
buf = buf[1:]
continue
}
if buf[0] == '*' {
return condFalse
}
etag, remain := scanETag(buf)
if etag == "" {
break
}
if etagWeakMatch(etag, w.Header().get("Etag")) {
return condFalse
}
buf = remain
}
return condTrue
}
func checkIfModifiedSince(r *Request, modtime time.Time) condResult {
if r.Method != "GET" && r.Method != "HEAD" {
return condNone
}
ims := r.Header.Get("If-Modified-Since")
if ims == "" || isZeroTime(modtime) {
return condNone
}
t, err := ParseTime(ims)
if err != nil {
return condNone
}
// The Last-Modified header truncates sub-second precision so
// the modtime needs to be truncated too.
modtime = modtime.Truncate(time.Second)
if modtime.Before(t) || modtime.Equal(t) {
return condFalse
}
return condTrue
}
func checkIfRange(w ResponseWriter, r *Request, modtime time.Time) condResult {
if r.Method != "GET" && r.Method != "HEAD" {
return condNone
}
ir := r.Header.get("If-Range")
if ir == "" {
return condNone
}
etag, _ := scanETag(ir)
if etag != "" {
if etagStrongMatch(etag, w.Header().Get("Etag")) {
return condTrue
} else {
return condFalse
}
}
// The If-Range value is typically the ETag value, but it may also be
// the modtime date. See golang.org/issue/8367.
if modtime.IsZero() {
return condFalse
}
t, err := ParseTime(ir)
if err != nil {
return condFalse
}
if t.Unix() == modtime.Unix() {
return condTrue
}
return condFalse
}
var unixEpochTime = time.Unix(0, 0)
// isZeroTime reports whether t is obviously unspecified (either zero or Unix()=0).
func isZeroTime(t time.Time) bool {
return t.IsZero() || t.Equal(unixEpochTime)
}
func setLastModified(w ResponseWriter, modtime time.Time) {
if !isZeroTime(modtime) {
w.Header().Set("Last-Modified", modtime.UTC().Format(TimeFormat))
}
}
func writeNotModified(w ResponseWriter) {
// RFC 7232 section 4.1:
// a sender SHOULD NOT generate representation metadata other than the
// above listed fields unless said metadata exists for the purpose of
// guiding cache updates (e.g., Last-Modified might be useful if the
// response does not have an ETag field).
h := w.Header()
delete(h, "Content-Type")
delete(h, "Content-Length")
delete(h, "Content-Encoding")
if h.Get("Etag") != "" {
delete(h, "Last-Modified")
}
w.WriteHeader(StatusNotModified)
}
// checkPreconditions evaluates request preconditions and reports whether a precondition
// resulted in sending StatusNotModified or StatusPreconditionFailed.
func checkPreconditions(w ResponseWriter, r *Request, modtime time.Time) (done bool, rangeHeader string) {
// This function carefully follows RFC 7232 section 6.
ch := checkIfMatch(w, r)
if ch == condNone {
ch = checkIfUnmodifiedSince(r, modtime)
}
if ch == condFalse {
w.WriteHeader(StatusPreconditionFailed)
return true, ""
}
switch checkIfNoneMatch(w, r) {
case condFalse:
if r.Method == "GET" || r.Method == "HEAD" {
writeNotModified(w)
return true, ""
} else {
w.WriteHeader(StatusPreconditionFailed)
return true, ""
}
case condNone:
if checkIfModifiedSince(r, modtime) == condFalse {
writeNotModified(w)
return true, ""
}
}
rangeHeader = r.Header.get("Range")
if rangeHeader != "" && checkIfRange(w, r, modtime) == condFalse {
rangeHeader = ""
}
return false, rangeHeader
}
// name is '/'-separated, not filepath.Separator.
func serveFile(w ResponseWriter, r *Request, fs FileSystem, name string, redirect bool) {
const indexPage = "/index.html"
// redirect .../index.html to .../
// can't use Redirect() because that would make the path absolute,
// which would be a problem running under StripPrefix
if strings.HasSuffix(r.URL.Path, indexPage) {
localRedirect(w, r, "./")
return
}
f, err := fs.Open(name)
if err != nil {
msg, code := toHTTPError(err)
Error(w, msg, code)
return
}
defer f.Close()
d, err := f.Stat()
if err != nil {
msg, code := toHTTPError(err)
Error(w, msg, code)
return
}
if redirect {
// redirect to canonical path: / at end of directory url
// r.URL.Path always begins with /
url := r.URL.Path
if d.IsDir() {
if url[len(url)-1] != '/' {
localRedirect(w, r, path.Base(url)+"/")
return
}
} else {
if url[len(url)-1] == '/' {
localRedirect(w, r, "../"+path.Base(url))
return
}
}
}
if d.IsDir() {
url := r.URL.Path
// redirect if the directory name doesn't end in a slash
if url == "" || url[len(url)-1] != '/' {
localRedirect(w, r, path.Base(url)+"/")
return
}
// use contents of index.html for directory, if present
index := strings.TrimSuffix(name, "/") + indexPage
ff, err := fs.Open(index)
if err == nil {
defer ff.Close()
dd, err := ff.Stat()
if err == nil {
name = index
d = dd
f = ff
}
}
}
// Still a directory? (we didn't find an index.html file)
if d.IsDir() {
if checkIfModifiedSince(r, d.ModTime()) == condFalse {
writeNotModified(w)
return
}
setLastModified(w, d.ModTime())
dirList(w, r, f)
return
}
// serveContent will check modification time
sizeFunc := func() (int64, error) { return d.Size(), nil }
serveContent(w, r, d.Name(), d.ModTime(), sizeFunc, f)
}
// toHTTPError returns a non-specific HTTP error message and status code
// for a given non-nil error value. It's important that toHTTPError does not
// actually return err.Error(), since msg and httpStatus are returned to users,
// and historically Go's ServeContent always returned just "404 Not Found" for
// all errors. We don't want to start leaking information in error messages.
func toHTTPError(err error) (msg string, httpStatus int) {
if errors.Is(err, fs.ErrNotExist) {
return "404 page not found", StatusNotFound
}
if errors.Is(err, fs.ErrPermission) {
return "403 Forbidden", StatusForbidden
}
// Default:
return "500 Internal Server Error", StatusInternalServerError
}
// localRedirect gives a Moved Permanently response.
// It does not convert relative paths to absolute paths like Redirect does.
func localRedirect(w ResponseWriter, r *Request, newPath string) {
if q := r.URL.RawQuery; q != "" {
newPath += "?" + q
}
w.Header().Set("Location", newPath)
w.WriteHeader(StatusMovedPermanently)
}
// ServeFile replies to the request with the contents of the named
// file or directory.
//
// If the provided file or directory name is a relative path, it is
// interpreted relative to the current directory and may ascend to
// parent directories. If the provided name is constructed from user
// input, it should be sanitized before calling ServeFile.
//
// As a precaution, ServeFile will reject requests where r.URL.Path
// contains a ".." path element; this protects against callers who
// might unsafely use filepath.Join on r.URL.Path without sanitizing
// it and then use that filepath.Join result as the name argument.
//
// As another special case, ServeFile redirects any request where r.URL.Path
// ends in "/index.html" to the same path, without the final
// "index.html". To avoid such redirects either modify the path or
// use ServeContent.
//
// Outside of those two special cases, ServeFile does not use
// r.URL.Path for selecting the file or directory to serve; only the
// file or directory provided in the name argument is used.
func ServeFile(w ResponseWriter, r *Request, name string) {
if containsDotDot(r.URL.Path) {
// Too many programs use r.URL.Path to construct the argument to
// serveFile. Reject the request under the assumption that happened
// here and ".." may not be wanted.
// Note that name might not contain "..", for example if code (still
// incorrectly) used filepath.Join(myDir, r.URL.Path).
Error(w, "invalid URL path", StatusBadRequest)
return
}
dir, file := filepath.Split(name)
serveFile(w, r, Dir(dir), file, false)
}
func containsDotDot(v string) bool {
if !strings.Contains(v, "..") {
return false
}
for _, ent := range strings.FieldsFunc(v, isSlashRune) {
if ent == ".." {
return true
}
}
return false
}
func isSlashRune(r rune) bool { return r == '/' || r == '\\' }
type fileHandler struct {
root FileSystem
}
type ioFS struct {
fsys fs.FS
}
type ioFile struct {
file fs.File
}
func (f ioFS) Open(name string) (File, error) {
if name == "/" {
name = "."
} else {
name = strings.TrimPrefix(name, "/")
}
file, err := f.fsys.Open(name)
if err != nil {
return nil, mapOpenError(err, name, '/', func(path string) (fs.FileInfo, error) {
return fs.Stat(f.fsys, path)
})
}
return ioFile{file}, nil
}
func (f ioFile) Close() error { return f.file.Close() }
func (f ioFile) Read(b []byte) (int, error) { return f.file.Read(b) }
func (f ioFile) Stat() (fs.FileInfo, error) { return f.file.Stat() }
var errMissingSeek = errors.New("io.File missing Seek method")
var errMissingReadDir = errors.New("io.File directory missing ReadDir method")
func (f ioFile) Seek(offset int64, whence int) (int64, error) {
s, ok := f.file.(io.Seeker)
if !ok {
return 0, errMissingSeek
}
return s.Seek(offset, whence)
}
func (f ioFile) ReadDir(count int) ([]fs.DirEntry, error) {
d, ok := f.file.(fs.ReadDirFile)
if !ok {
return nil, errMissingReadDir
}
return d.ReadDir(count)
}
func (f ioFile) Readdir(count int) ([]fs.FileInfo, error) {
d, ok := f.file.(fs.ReadDirFile)
if !ok {
return nil, errMissingReadDir
}
var list []fs.FileInfo
for {
dirs, err := d.ReadDir(count - len(list))
for _, dir := range dirs {
info, err := dir.Info()
if err != nil {
// Pretend it doesn't exist, like (*os.File).Readdir does.
continue
}
list = append(list, info)
}
if err != nil {
return list, err
}
if count < 0 || len(list) >= count {
break
}
}
return list, nil
}
// FS converts fsys to a FileSystem implementation,
// for use with FileServer and NewFileTransport.
func FS(fsys fs.FS) FileSystem {
return ioFS{fsys}
}
// FileServer returns a handler that serves HTTP requests
// with the contents of the file system rooted at root.
//
// As a special case, the returned file server redirects any request
// ending in "/index.html" to the same path, without the final
// "index.html".
//
// To use the operating system's file system implementation,
// use http.Dir:
//
// http.Handle("/", http.FileServer(http.Dir("/tmp")))
//
// To use an fs.FS implementation, use http.FS to convert it:
//
// http.Handle("/", http.FileServer(http.FS(fsys)))
func FileServer(root FileSystem) Handler {
return &fileHandler{root}
}
func (f *fileHandler) ServeHTTP(w ResponseWriter, r *Request) {
upath := r.URL.Path
if !strings.HasPrefix(upath, "/") {
upath = "/" + upath
r.URL.Path = upath
}
serveFile(w, r, f.root, path.Clean(upath), true)
}
// httpRange specifies the byte range to be sent to the client.
type httpRange struct {
start, length int64
}
func (r httpRange) contentRange(size int64) string {
return fmt.Sprintf("bytes %d-%d/%d", r.start, r.start+r.length-1, size)
}
func (r httpRange) mimeHeader(contentType string, size int64) textproto.MIMEHeader {
return textproto.MIMEHeader{
"Content-Range": {r.contentRange(size)},
"Content-Type": {contentType},
}
}
// parseRange parses a Range header string as per RFC 7233.
// errNoOverlap is returned if none of the ranges overlap.
func parseRange(s string, size int64) ([]httpRange, error) {
if s == "" {
return nil, nil // header not present
}
const b = "bytes="
if !strings.HasPrefix(s, b) {
return nil, errors.New("invalid range")
}
var ranges []httpRange
noOverlap := false
for _, ra := range strings.Split(s[len(b):], ",") {
ra = textproto.TrimString(ra)
if ra == "" {
continue
}
start, end, ok := strings.Cut(ra, "-")
if !ok {
return nil, errors.New("invalid range")
}
start, end = textproto.TrimString(start), textproto.TrimString(end)
var r httpRange
if start == "" {
// If no start is specified, end specifies the
// range start relative to the end of the file,
// and we are dealing with <suffix-length>
// which has to be a non-negative integer as per
// RFC 7233 Section 2.1 "Byte-Ranges".
if end == "" || end[0] == '-' {
return nil, errors.New("invalid range")
}
i, err := strconv.ParseInt(end, 10, 64)
if i < 0 || err != nil {
return nil, errors.New("invalid range")
}
if i > size {
i = size
}
r.start = size - i
r.length = size - r.start
} else {
i, err := strconv.ParseInt(start, 10, 64)
if err != nil || i < 0 {
return nil, errors.New("invalid range")
}
if i >= size {
// If the range begins after the size of the content,
// then it does not overlap.
noOverlap = true
continue
}
r.start = i
if end == "" {
// If no end is specified, range extends to end of the file.
r.length = size - r.start
} else {
i, err := strconv.ParseInt(end, 10, 64)
if err != nil || r.start > i {
return nil, errors.New("invalid range")
}
if i >= size {
i = size - 1
}
r.length = i - r.start + 1
}
}
ranges = append(ranges, r)
}
if noOverlap && len(ranges) == 0 {
// The specified ranges did not overlap with the content.
return nil, errNoOverlap
}
return ranges, nil
}
// countingWriter counts how many bytes have been written to it.
type countingWriter int64
func (w *countingWriter) Write(p []byte) (n int, err error) {
*w += countingWriter(len(p))
return len(p), nil
}
// rangesMIMESize returns the number of bytes it takes to encode the
// provided ranges as a multipart response.
func rangesMIMESize(ranges []httpRange, contentType string, contentSize int64) (encSize int64) {
var w countingWriter
mw := multipart.NewWriter(&w)
for _, ra := range ranges {
mw.CreatePart(ra.mimeHeader(contentType, contentSize))
encSize += ra.length
}
mw.Close()
encSize += int64(w)
return
}
func sumRangesSize(ranges []httpRange) (size int64) {
for _, ra := range ranges {
size += ra.length
}
return
}
http/header.go
+275
View File
@@ -0,0 +1,275 @@
// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
// TINYGO: Removed trace stuff
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package http
import (
"io"
"net/http/internal/ascii"
"net/textproto"
"sort"
"strings"
"sync"
"time"
"golang.org/x/net/http/httpguts"
)
// A Header represents the key-value pairs in an HTTP header.
//
// The keys should be in canonical form, as returned by
// CanonicalHeaderKey.
type Header map[string][]string
// Add adds the key, value pair to the header.
// It appends to any existing values associated with key.
// The key is case insensitive; it is canonicalized by
// CanonicalHeaderKey.
func (h Header) Add(key, value string) {
textproto.MIMEHeader(h).Add(key, value)
}
// Set sets the header entries associated with key to the
// single element value. It replaces any existing values
// associated with key. The key is case insensitive; it is
// canonicalized by textproto.CanonicalMIMEHeaderKey.
// To use non-canonical keys, assign to the map directly.
func (h Header) Set(key, value string) {
textproto.MIMEHeader(h).Set(key, value)
}
// Get gets the first value associated with the given key. If
// there are no values associated with the key, Get returns "".
// It is case insensitive; textproto.CanonicalMIMEHeaderKey is
// used to canonicalize the provided key. Get assumes that all
// keys are stored in canonical form. To use non-canonical keys,
// access the map directly.
func (h Header) Get(key string) string {
return textproto.MIMEHeader(h).Get(key)
}
// Values returns all values associated with the given key.
// It is case insensitive; textproto.CanonicalMIMEHeaderKey is
// used to canonicalize the provided key. To use non-canonical
// keys, access the map directly.
// The returned slice is not a copy.
func (h Header) Values(key string) []string {
return textproto.MIMEHeader(h).Values(key)
}
// get is like Get, but key must already be in CanonicalHeaderKey form.
func (h Header) get(key string) string {
if v := h[key]; len(v) > 0 {
return v[0]
}
return ""
}
// has reports whether h has the provided key defined, even if it's
// set to 0-length slice.
func (h Header) has(key string) bool {
_, ok := h[key]
return ok
}
// Del deletes the values associated with key.
// The key is case insensitive; it is canonicalized by
// CanonicalHeaderKey.
func (h Header) Del(key string) {
textproto.MIMEHeader(h).Del(key)
}
// Write writes a header in wire format.
func (h Header) Write(w io.Writer) error {
return h.write(w)
}
func (h Header) write(w io.Writer) error {
return h.writeSubset(w, nil)
}
// Clone returns a copy of h or nil if h is nil.
func (h Header) Clone() Header {
if h == nil {
return nil
}
// Find total number of values.
nv := 0
for _, vv := range h {
nv += len(vv)
}
sv := make([]string, nv) // shared backing array for headers' values
h2 := make(Header, len(h))
for k, vv := range h {
if vv == nil {
// Preserve nil values. ReverseProxy distinguishes
// between nil and zero-length header values.
h2[k] = nil
continue
}
n := copy(sv, vv)
h2[k] = sv[:n:n]
sv = sv[n:]
}
return h2
}
var timeFormats = []string{
TimeFormat,
time.RFC850,
time.ANSIC,
}
// ParseTime parses a time header (such as the Date: header),
// trying each of the three formats allowed by HTTP/1.1:
// TimeFormat, time.RFC850, and time.ANSIC.
func ParseTime(text string) (t time.Time, err error) {
for _, layout := range timeFormats {
t, err = time.Parse(layout, text)
if err == nil {
return
}
}
return
}
var headerNewlineToSpace = strings.NewReplacer("\n", " ", "\r", " ")
// stringWriter implements WriteString on a Writer.
type stringWriter struct {
w io.Writer
}
func (w stringWriter) WriteString(s string) (n int, err error) {
return w.w.Write([]byte(s))
}
type keyValues struct {
key string
values []string
}
// A headerSorter implements sort.Interface by sorting a []keyValues
// by key. It's used as a pointer, so it can fit in a sort.Interface
// interface value without allocation.
type headerSorter struct {
kvs []keyValues
}
func (s *headerSorter) Len() int { return len(s.kvs) }
func (s *headerSorter) Swap(i, j int) { s.kvs[i], s.kvs[j] = s.kvs[j], s.kvs[i] }
func (s *headerSorter) Less(i, j int) bool { return s.kvs[i].key < s.kvs[j].key }
var headerSorterPool = sync.Pool{
New: func() any { return new(headerSorter) },
}
// sortedKeyValues returns h's keys sorted in the returned kvs
// slice. The headerSorter used to sort is also returned, for possible
// return to headerSorterCache.
func (h Header) sortedKeyValues(exclude map[string]bool) (kvs []keyValues, hs *headerSorter) {
hs = headerSorterPool.Get().(*headerSorter)
if cap(hs.kvs) < len(h) {
hs.kvs = make([]keyValues, 0, len(h))
}
kvs = hs.kvs[:0]
for k, vv := range h {
if !exclude[k] {
kvs = append(kvs, keyValues{k, vv})
}
}
hs.kvs = kvs
sort.Sort(hs)
return kvs, hs
}
// WriteSubset writes a header in wire format.
// If exclude is not nil, keys where exclude[key] == true are not written.
// Keys are not canonicalized before checking the exclude map.
func (h Header) WriteSubset(w io.Writer, exclude map[string]bool) error {
return h.writeSubset(w, exclude)
}
func (h Header) writeSubset(w io.Writer, exclude map[string]bool) error {
ws, ok := w.(io.StringWriter)
if !ok {
ws = stringWriter{w}
}
kvs, sorter := h.sortedKeyValues(exclude)
for _, kv := range kvs {
if !httpguts.ValidHeaderFieldName(kv.key) {
// This could be an error. In the common case of
// writing response headers, however, we have no good
// way to provide the error back to the server
// handler, so just drop invalid headers instead.
continue
}
for _, v := range kv.values {
v = headerNewlineToSpace.Replace(v)
v = textproto.TrimString(v)
for _, s := range []string{kv.key, ": ", v, "\r\n"} {
if _, err := ws.WriteString(s); err != nil {
headerSorterPool.Put(sorter)
return err
}
}
}
}
headerSorterPool.Put(sorter)
return nil
}
// CanonicalHeaderKey returns the canonical format of the
// header key s. The canonicalization converts the first
// letter and any letter following a hyphen to upper case;
// the rest are converted to lowercase. For example, the
// canonical key for "accept-encoding" is "Accept-Encoding".
// If s contains a space or invalid header field bytes, it is
// returned without modifications.
func CanonicalHeaderKey(s string) string { return textproto.CanonicalMIMEHeaderKey(s) }
// hasToken reports whether token appears with v, ASCII
// case-insensitive, with space or comma boundaries.
// token must be all lowercase.
// v may contain mixed cased.
func hasToken(v, token string) bool {
if len(token) > len(v) || token == "" {
return false
}
if v == token {
return true
}
for sp := 0; sp <= len(v)-len(token); sp++ {
// Check that first character is good.
// The token is ASCII, so checking only a single byte
// is sufficient. We skip this potential starting
// position if both the first byte and its potential
// ASCII uppercase equivalent (b|0x20) don't match.
// False positives ('^' => '~') are caught by EqualFold.
if b := v[sp]; b != token[0] && b|0x20 != token[0] {
continue
}
// Check that start pos is on a valid token boundary.
if sp > 0 && !isTokenBoundary(v[sp-1]) {
continue
}
// Check that end pos is on a valid token boundary.
if endPos := sp + len(token); endPos != len(v) && !isTokenBoundary(v[endPos]) {
continue
}
if ascii.EqualFold(v[sp:sp+len(token)], token) {
return true
}
}
return false
}
func isTokenBoundary(b byte) bool {
return b == ' ' || b == ',' || b == '\t'
}
http/http.go
+161
View File
@@ -0,0 +1,161 @@
// TINYGO: The following is copied from Go 1.19.3 official implementation.
// Copyright 2016 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:generate bundle -o=h2_bundle.go -prefix=http2 -tags=!nethttpomithttp2 golang.org/x/net/http2
package http
import (
"io"
"strconv"
"strings"
"time"
"unicode/utf8"
"golang.org/x/net/http/httpguts"
)
// incomparable is a zero-width, non-comparable type. Adding it to a struct
// makes that struct also non-comparable, and generally doesn't add
// any size (as long as it's first).
type incomparable [0]func()
// maxInt64 is the effective "infinite" value for the Server and
// Transport's byte-limiting readers.
const maxInt64 = 1<<63 - 1
// aLongTimeAgo is a non-zero time, far in the past, used for
// immediate cancellation of network operations.
var aLongTimeAgo = time.Unix(1, 0)
// omitBundledHTTP2 is set by omithttp2.go when the nethttpomithttp2
// build tag is set. That means h2_bundle.go isn't compiled in and we
// shouldn't try to use it.
var omitBundledHTTP2 bool
// TODO(bradfitz): move common stuff here. The other files have accumulated
// generic http stuff in random places.
// contextKey is a value for use with context.WithValue. It's used as
// a pointer so it fits in an interface{} without allocation.
type contextKey struct {
name string
}
func (k *contextKey) String() string { return "net/http context value " + k.name }
// Given a string of the form "host", "host:port", or "[ipv6::address]:port",
// return true if the string includes a port.
func hasPort(s string) bool { return strings.LastIndex(s, ":") > strings.LastIndex(s, "]") }
// removeEmptyPort strips the empty port in ":port" to ""
// as mandated by RFC 3986 Section 6.2.3.
func removeEmptyPort(host string) string {
if hasPort(host) {
return strings.TrimSuffix(host, ":")
}
return host
}
func isNotToken(r rune) bool {
return !httpguts.IsTokenRune(r)
}
// stringContainsCTLByte reports whether s contains any ASCII control character.
func stringContainsCTLByte(s string) bool {
for i := 0; i < len(s); i++ {
b := s[i]
if b < ' ' || b == 0x7f {
return true
}
}
return false
}
func hexEscapeNonASCII(s string) string {
newLen := 0
for i := 0; i < len(s); i++ {
if s[i] >= utf8.RuneSelf {
newLen += 3
} else {
newLen++
}
}
if newLen == len(s) {
return s
}
b := make([]byte, 0, newLen)
for i := 0; i < len(s); i++ {
if s[i] >= utf8.RuneSelf {
b = append(b, '%')
b = strconv.AppendInt(b, int64(s[i]), 16)
} else {
b = append(b, s[i])
}
}
return string(b)
}
// NoBody is an io.ReadCloser with no bytes. Read always returns EOF
// and Close always returns nil. It can be used in an outgoing client
// request to explicitly signal that a request has zero bytes.
// An alternative, however, is to simply set Request.Body to nil.
var NoBody = noBody{}
type noBody struct{}
func (noBody) Read([]byte) (int, error) { return 0, io.EOF }
func (noBody) Close() error { return nil }
func (noBody) WriteTo(io.Writer) (int64, error) { return 0, nil }
var (
// verify that an io.Copy from NoBody won't require a buffer:
_ io.WriterTo = NoBody
_ io.ReadCloser = NoBody
)
// PushOptions describes options for Pusher.Push.
type PushOptions struct {
// Method specifies the HTTP method for the promised request.
// If set, it must be "GET" or "HEAD". Empty means "GET".
Method string
// Header specifies additional promised request headers. This cannot
// include HTTP/2 pseudo header fields like ":path" and ":scheme",
// which will be added automatically.
Header Header
}
// Pusher is the interface implemented by ResponseWriters that support
// HTTP/2 server push. For more background, see
// https://tools.ietf.org/html/rfc7540#section-8.2.
type Pusher interface {
// Push initiates an HTTP/2 server push. This constructs a synthetic
// request using the given target and options, serializes that request
// into a PUSH_PROMISE frame, then dispatches that request using the
// server's request handler. If opts is nil, default options are used.
//
// The target must either be an absolute path (like "/path") or an absolute
// URL that contains a valid host and the same scheme as the parent request.
// If the target is a path, it will inherit the scheme and host of the
// parent request.
//
// The HTTP/2 spec disallows recursive pushes and cross-authority pushes.
// Push may or may not detect these invalid pushes; however, invalid
// pushes will be detected and canceled by conforming clients.
//
// Handlers that wish to push URL X should call Push before sending any
// data that may trigger a request for URL X. This avoids a race where the
// client issues requests for X before receiving the PUSH_PROMISE for X.
//
// Push will run in a separate goroutine making the order of arrival
// non-deterministic. Any required synchronization needs to be implemented
// by the caller.
//
// Push returns ErrNotSupported if the client has disabled push or if push
// is not supported on the underlying connection.
Push(target string, opts *PushOptions) error
}
http/internal/ascii/print.go
+63
View File
@@ -0,0 +1,63 @@
// TINYGO: The following is copied from Go 1.19.3 official implementation.
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ascii
import (
"strings"
"unicode"
)
// EqualFold is strings.EqualFold, ASCII only. It reports whether s and t
// are equal, ASCII-case-insensitively.
func EqualFold(s, t string) bool {
if len(s) != len(t) {
return false
}
for i := 0; i < len(s); i++ {
if lower(s[i]) != lower(t[i]) {
return false
}
}
return true
}
// lower returns the ASCII lowercase version of b.
func lower(b byte) byte {
if 'A' <= b && b <= 'Z' {
return b + ('a' - 'A')
}
return b
}
// IsPrint returns whether s is ASCII and printable according to
// https://tools.ietf.org/html/rfc20#section-4.2.
func IsPrint(s string) bool {
for i := 0; i < len(s); i++ {
if s[i] < ' ' || s[i] > '~' {
return false
}
}
return true
}
// Is returns whether s is ASCII.
func Is(s string) bool {
for i := 0; i < len(s); i++ {
if s[i] > unicode.MaxASCII {
return false
}
}
return true
}
// ToLower returns the lowercase version of s if s is ASCII and printable.
func ToLower(s string) (lower string, ok bool) {
if !IsPrint(s) {
return "", false
}
return strings.ToLower(s), true
}
http/internal/ascii/print_test.go
+95
View File
@@ -0,0 +1,95 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ascii
import "testing"
func TestEqualFold(t *testing.T) {
var tests = []struct {
name string
a, b string
want bool
}{
{
name: "empty",
want: true,
},
{
name: "simple match",
a: "CHUNKED",
b: "chunked",
want: true,
},
{
name: "same string",
a: "chunked",
b: "chunked",
want: true,
},
{
name: "Unicode Kelvin symbol",
a: "chunKed", // This "K" is 'KELVIN SIGN' (\u212A)
b: "chunked",
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := EqualFold(tt.a, tt.b); got != tt.want {
t.Errorf("AsciiEqualFold(%q,%q): got %v want %v", tt.a, tt.b, got, tt.want)
}
})
}
}
func TestIsPrint(t *testing.T) {
var tests = []struct {
name string
in string
want bool
}{
{
name: "empty",
want: true,
},
{
name: "ASCII low",
in: "This is a space: ' '",
want: true,
},
{
name: "ASCII high",
in: "This is a tilde: '~'",
want: true,
},
{
name: "ASCII low non-print",
in: "This is a unit separator: \x1F",
want: false,
},
{
name: "Ascii high non-print",
in: "This is a Delete: \x7F",
want: false,
},
{
name: "Unicode letter",
in: "Today it's 280K outside: it's freezing!", // This "K" is 'KELVIN SIGN' (\u212A)
want: false,
},
{
name: "Unicode emoji",
in: "Gophers like 🧀",
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := IsPrint(tt.in); got != tt.want {
t.Errorf("IsASCIIPrint(%q): got %v want %v", tt.in, got, tt.want)
}
})
}
}
http/internal/chunked.go
+264
View File
@@ -0,0 +1,264 @@
// TINYGO: The following is copied from Go 1.19.3 official implementation.
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// The wire protocol for HTTP's "chunked" Transfer-Encoding.
// Package internal contains HTTP internals shared by net/http and
// net/http/httputil.
package internal
import (
"bufio"
"bytes"
"errors"
"fmt"
"io"
)
const maxLineLength = 4096 // assumed <= bufio.defaultBufSize
var ErrLineTooLong = errors.New("header line too long")
// NewChunkedReader returns a new chunkedReader that translates the data read from r
// out of HTTP "chunked" format before returning it.
// The chunkedReader returns io.EOF when the final 0-length chunk is read.
//
// NewChunkedReader is not needed by normal applications. The http package
// automatically decodes chunking when reading response bodies.
func NewChunkedReader(r io.Reader) io.Reader {
br, ok := r.(*bufio.Reader)
if !ok {
br = bufio.NewReader(r)
}
return &chunkedReader{r: br}
}
type chunkedReader struct {
r *bufio.Reader
n uint64 // unread bytes in chunk
err error
buf [2]byte
checkEnd bool // whether need to check for \r\n chunk footer
}
func (cr *chunkedReader) beginChunk() {
// chunk-size CRLF
var line []byte
line, cr.err = readChunkLine(cr.r)
if cr.err != nil {
return
}
cr.n, cr.err = parseHexUint(line)
if cr.err != nil {
return
}
if cr.n == 0 {
cr.err = io.EOF
}
}
func (cr *chunkedReader) chunkHeaderAvailable() bool {
n := cr.r.Buffered()
if n > 0 {
peek, _ := cr.r.Peek(n)
return bytes.IndexByte(peek, '\n') >= 0
}
return false
}
func (cr *chunkedReader) Read(b []uint8) (n int, err error) {
for cr.err == nil {
if cr.checkEnd {
if n > 0 && cr.r.Buffered() < 2 {
// We have some data. Return early (per the io.Reader
// contract) instead of potentially blocking while
// reading more.
break
}
if _, cr.err = io.ReadFull(cr.r, cr.buf[:2]); cr.err == nil {
if string(cr.buf[:]) != "\r\n" {
cr.err = errors.New("malformed chunked encoding")
break
}
} else {
if cr.err == io.EOF {
cr.err = io.ErrUnexpectedEOF
}
break
}
cr.checkEnd = false
}
if cr.n == 0 {
if n > 0 && !cr.chunkHeaderAvailable() {
// We've read enough. Don't potentially block
// reading a new chunk header.
break
}
cr.beginChunk()
continue
}
if len(b) == 0 {
break
}
rbuf := b
if uint64(len(rbuf)) > cr.n {
rbuf = rbuf[:cr.n]
}
var n0 int
n0, cr.err = cr.r.Read(rbuf)
n += n0
b = b[n0:]
cr.n -= uint64(n0)
// If we're at the end of a chunk, read the next two
// bytes to verify they are "\r\n".
if cr.n == 0 && cr.err == nil {
cr.checkEnd = true
} else if cr.err == io.EOF {
cr.err = io.ErrUnexpectedEOF
}
}
return n, cr.err
}
// Read a line of bytes (up to \n) from b.
// Give up if the line exceeds maxLineLength.
// The returned bytes are owned by the bufio.Reader
// so they are only valid until the next bufio read.
func readChunkLine(b *bufio.Reader) ([]byte, error) {
p, err := b.ReadSlice('\n')
if err != nil {
// We always know when EOF is coming.
// If the caller asked for a line, there should be a line.
if err == io.EOF {
err = io.ErrUnexpectedEOF
} else if err == bufio.ErrBufferFull {
err = ErrLineTooLong
}
return nil, err
}
if len(p) >= maxLineLength {
return nil, ErrLineTooLong
}
p = trimTrailingWhitespace(p)
p, err = removeChunkExtension(p)
if err != nil {
return nil, err
}
return p, nil
}
func trimTrailingWhitespace(b []byte) []byte {
for len(b) > 0 && isASCIISpace(b[len(b)-1]) {
b = b[:len(b)-1]
}
return b
}
func isASCIISpace(b byte) bool {
return b == ' ' || b == '\t' || b == '\n' || b == '\r'
}
var semi = []byte(";")
// removeChunkExtension removes any chunk-extension from p.
// For example,
//
// "0" => "0"
// "0;token" => "0"
// "0;token=val" => "0"
// `0;token="quoted string"` => "0"
func removeChunkExtension(p []byte) ([]byte, error) {
p, _, _ = bytes.Cut(p, semi)
// TODO: care about exact syntax of chunk extensions? We're
// ignoring and stripping them anyway. For now just never
// return an error.
return p, nil
}
// NewChunkedWriter returns a new chunkedWriter that translates writes into HTTP
// "chunked" format before writing them to w. Closing the returned chunkedWriter
// sends the final 0-length chunk that marks the end of the stream but does
// not send the final CRLF that appears after trailers; trailers and the last
// CRLF must be written separately.
//
// NewChunkedWriter is not needed by normal applications. The http
// package adds chunking automatically if handlers don't set a
// Content-Length header. Using newChunkedWriter inside a handler
// would result in double chunking or chunking with a Content-Length
// length, both of which are wrong.
func NewChunkedWriter(w io.Writer) io.WriteCloser {
return &chunkedWriter{w}
}
// Writing to chunkedWriter translates to writing in HTTP chunked Transfer
// Encoding wire format to the underlying Wire chunkedWriter.
type chunkedWriter struct {
Wire io.Writer
}
// Write the contents of data as one chunk to Wire.
// NOTE: Note that the corresponding chunk-writing procedure in Conn.Write has
// a bug since it does not check for success of io.WriteString
func (cw *chunkedWriter) Write(data []byte) (n int, err error) {
// Don't send 0-length data. It looks like EOF for chunked encoding.
if len(data) == 0 {
return 0, nil
}
if _, err = fmt.Fprintf(cw.Wire, "%x\r\n", len(data)); err != nil {
return 0, err
}
if n, err = cw.Wire.Write(data); err != nil {
return
}
if n != len(data) {
err = io.ErrShortWrite
return
}
if _, err = io.WriteString(cw.Wire, "\r\n"); err != nil {
return
}
if bw, ok := cw.Wire.(*FlushAfterChunkWriter); ok {
err = bw.Flush()
}
return
}
func (cw *chunkedWriter) Close() error {
_, err := io.WriteString(cw.Wire, "0\r\n")
return err
}
// FlushAfterChunkWriter signals from the caller of NewChunkedWriter
// that each chunk should be followed by a flush. It is used by the
// http.Transport code to keep the buffering behavior for headers and
// trailers, but flush out chunks aggressively in the middle for
// request bodies which may be generated slowly. See Issue 6574.
type FlushAfterChunkWriter struct {
*bufio.Writer
}
func parseHexUint(v []byte) (n uint64, err error) {
for i, b := range v {
switch {
case '0' <= b && b <= '9':
b = b - '0'
case 'a' <= b && b <= 'f':
b = b - 'a' + 10
case 'A' <= b && b <= 'F':
b = b - 'A' + 10
default:
return 0, errors.New("invalid byte in chunk length")
}
if i == 16 {
return 0, errors.New("http chunk length too large")
}
n <<= 4
n |= uint64(b)
}
return
}
http/internal/chunked_test.go
+241
View File
@@ -0,0 +1,241 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package internal
import (
"bufio"
"bytes"
"fmt"
"io"
"strings"
"testing"
"testing/iotest"
)
func TestChunk(t *testing.T) {
var b bytes.Buffer
w := NewChunkedWriter(&b)
const chunk1 = "hello, "
const chunk2 = "world! 0123456789abcdef"
w.Write([]byte(chunk1))
w.Write([]byte(chunk2))
w.Close()
if g, e := b.String(), "7\r\nhello, \r\n17\r\nworld! 0123456789abcdef\r\n0\r\n"; g != e {
t.Fatalf("chunk writer wrote %q; want %q", g, e)
}
r := NewChunkedReader(&b)
data, err := io.ReadAll(r)
if err != nil {
t.Logf(`data: "%s"`, data)
t.Fatalf("ReadAll from reader: %v", err)
}
if g, e := string(data), chunk1+chunk2; g != e {
t.Errorf("chunk reader read %q; want %q", g, e)
}
}
func TestChunkReadMultiple(t *testing.T) {
// Bunch of small chunks, all read together.
{
var b bytes.Buffer
w := NewChunkedWriter(&b)
w.Write([]byte("foo"))
w.Write([]byte("bar"))
w.Close()
r := NewChunkedReader(&b)
buf := make([]byte, 10)
n, err := r.Read(buf)
if n != 6 || err != io.EOF {
t.Errorf("Read = %d, %v; want 6, EOF", n, err)
}
buf = buf[:n]
if string(buf) != "foobar" {
t.Errorf("Read = %q; want %q", buf, "foobar")
}
}
// One big chunk followed by a little chunk, but the small bufio.Reader size
// should prevent the second chunk header from being read.
{
var b bytes.Buffer
w := NewChunkedWriter(&b)
// fillBufChunk is 11 bytes + 3 bytes header + 2 bytes footer = 16 bytes,
// the same as the bufio ReaderSize below (the minimum), so even
// though we're going to try to Read with a buffer larger enough to also
// receive "foo", the second chunk header won't be read yet.
const fillBufChunk = "0123456789a"
const shortChunk = "foo"
w.Write([]byte(fillBufChunk))
w.Write([]byte(shortChunk))
w.Close()
r := NewChunkedReader(bufio.NewReaderSize(&b, 16))
buf := make([]byte, len(fillBufChunk)+len(shortChunk))
n, err := r.Read(buf)
if n != len(fillBufChunk) || err != nil {
t.Errorf("Read = %d, %v; want %d, nil", n, err, len(fillBufChunk))
}
buf = buf[:n]
if string(buf) != fillBufChunk {
t.Errorf("Read = %q; want %q", buf, fillBufChunk)
}
n, err = r.Read(buf)
if n != len(shortChunk) || err != io.EOF {
t.Errorf("Read = %d, %v; want %d, EOF", n, err, len(shortChunk))
}
}
// And test that we see an EOF chunk, even though our buffer is already full:
{
r := NewChunkedReader(bufio.NewReader(strings.NewReader("3\r\nfoo\r\n0\r\n")))
buf := make([]byte, 3)
n, err := r.Read(buf)
if n != 3 || err != io.EOF {
t.Errorf("Read = %d, %v; want 3, EOF", n, err)
}
if string(buf) != "foo" {
t.Errorf("buf = %q; want foo", buf)
}
}
}
func TestChunkReaderAllocs(t *testing.T) {
if testing.Short() {
t.Skip("skipping in short mode")
}
var buf bytes.Buffer
w := NewChunkedWriter(&buf)
a, b, c := []byte("aaaaaa"), []byte("bbbbbbbbbbbb"), []byte("cccccccccccccccccccccccc")
w.Write(a)
w.Write(b)
w.Write(c)
w.Close()
readBuf := make([]byte, len(a)+len(b)+len(c)+1)
byter := bytes.NewReader(buf.Bytes())
bufr := bufio.NewReader(byter)
mallocs := testing.AllocsPerRun(100, func() {
byter.Seek(0, io.SeekStart)
bufr.Reset(byter)
r := NewChunkedReader(bufr)
n, err := io.ReadFull(r, readBuf)
if n != len(readBuf)-1 {
t.Fatalf("read %d bytes; want %d", n, len(readBuf)-1)
}
if err != io.ErrUnexpectedEOF {
t.Fatalf("read error = %v; want ErrUnexpectedEOF", err)
}
})
if mallocs > 1.5 {
t.Errorf("mallocs = %v; want 1", mallocs)
}
}
func TestParseHexUint(t *testing.T) {
type testCase struct {
in string
want uint64
wantErr string
}
tests := []testCase{
{"x", 0, "invalid byte in chunk length"},
{"0000000000000000", 0, ""},
{"0000000000000001", 1, ""},
{"ffffffffffffffff", 1<<64 - 1, ""},
{"000000000000bogus", 0, "invalid byte in chunk length"},
{"00000000000000000", 0, "http chunk length too large"}, // could accept if we wanted
{"10000000000000000", 0, "http chunk length too large"},
{"00000000000000001", 0, "http chunk length too large"}, // could accept if we wanted
}
for i := uint64(0); i <= 1234; i++ {
tests = append(tests, testCase{in: fmt.Sprintf("%x", i), want: i})
}
for _, tt := range tests {
got, err := parseHexUint([]byte(tt.in))
if tt.wantErr != "" {
if !strings.Contains(fmt.Sprint(err), tt.wantErr) {
t.Errorf("parseHexUint(%q) = %v, %v; want error %q", tt.in, got, err, tt.wantErr)
}
} else {
if err != nil || got != tt.want {
t.Errorf("parseHexUint(%q) = %v, %v; want %v", tt.in, got, err, tt.want)
}
}
}
}
func TestChunkReadingIgnoresExtensions(t *testing.T) {
in := "7;ext=\"some quoted string\"\r\n" + // token=quoted string
"hello, \r\n" +
"17;someext\r\n" + // token without value
"world! 0123456789abcdef\r\n" +
"0;someextension=sometoken\r\n" // token=token
data, err := io.ReadAll(NewChunkedReader(strings.NewReader(in)))
if err != nil {
t.Fatalf("ReadAll = %q, %v", data, err)
}
if g, e := string(data), "hello, world! 0123456789abcdef"; g != e {
t.Errorf("read %q; want %q", g, e)
}
}
// Issue 17355: ChunkedReader shouldn't block waiting for more data
// if it can return something.
func TestChunkReadPartial(t *testing.T) {
pr, pw := io.Pipe()
go func() {
pw.Write([]byte("7\r\n1234567"))
}()
cr := NewChunkedReader(pr)
readBuf := make([]byte, 7)
n, err := cr.Read(readBuf)
if err != nil {
t.Fatal(err)
}
want := "1234567"
if n != 7 || string(readBuf) != want {
t.Fatalf("Read: %v %q; want %d, %q", n, readBuf[:n], len(want), want)
}
go func() {
pw.Write([]byte("xx"))
}()
_, err = cr.Read(readBuf)
if got := fmt.Sprint(err); !strings.Contains(got, "malformed") {
t.Fatalf("second read = %v; want malformed error", err)
}
}
// Issue 48861: ChunkedReader should report incomplete chunks
func TestIncompleteChunk(t *testing.T) {
const valid = "4\r\nabcd\r\n" + "5\r\nabc\r\n\r\n" + "0\r\n"
for i := 0; i < len(valid); i++ {
incomplete := valid[:i]
r := NewChunkedReader(strings.NewReader(incomplete))
if _, err := io.ReadAll(r); err != io.ErrUnexpectedEOF {
t.Errorf("expected io.ErrUnexpectedEOF for %q, got %v", incomplete, err)
}
}
r := NewChunkedReader(strings.NewReader(valid))
if _, err := io.ReadAll(r); err != nil {
t.Errorf("unexpected error for %q: %v", valid, err)
}
}
func TestChunkEndReadError(t *testing.T) {
readErr := fmt.Errorf("chunk end read error")
r := NewChunkedReader(io.MultiReader(strings.NewReader("4\r\nabcd"), iotest.ErrReader(readErr)))
if _, err := io.ReadAll(r); err != readErr {
t.Errorf("expected %v, got %v", readErr, err)
}
}
http/jar.go
+29
View File
@@ -0,0 +1,29 @@
// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package http
import (
"net/url"
)
// A CookieJar manages storage and use of cookies in HTTP requests.
//
// Implementations of CookieJar must be safe for concurrent use by multiple
// goroutines.
//
// The net/http/cookiejar package provides a CookieJar implementation.
type CookieJar interface {
// SetCookies handles the receipt of the cookies in a reply for the
// given URL. It may or may not choose to save the cookies, depending
// on the jar's policy and implementation.
SetCookies(u *url.URL, cookies []*Cookie)
// Cookies returns the cookies to send in a request for the given URL.
// It is up to the implementation to honor the standard cookie use
// restrictions such as in RFC 6265.
Cookies(u *url.URL) []*Cookie
}
http/method.go
+22
View File
@@ -0,0 +1,22 @@
// TINYGO: The following is copied from Go 1.19.3 official implementation.
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package http
// Common HTTP methods.
//
// Unless otherwise noted, these are defined in RFC 7231 section 4.3.
const (
MethodGet = "GET"
MethodHead = "HEAD"
MethodPost = "POST"
MethodPut = "PUT"
MethodPatch = "PATCH" // RFC 5789
MethodDelete = "DELETE"
MethodConnect = "CONNECT"
MethodOptions = "OPTIONS"
MethodTrace = "TRACE"
)
http/request.go
+1447
View File
File diff suppressed because it is too large Load Diff
http/response.go
+373
View File
@@ -0,0 +1,373 @@
// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
// TINYGO: Removed TLS connection state
// TINYGO: Added onEOF hook to get callback when response has been read
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// HTTP Response reading and parsing.
package http
import (
"bufio"
"bytes"
"errors"
"fmt"
"io"
"net/textproto"
"net/url"
"strconv"
"strings"
"golang.org/x/net/http/httpguts"
)
var respExcludeHeader = map[string]bool{
"Content-Length": true,
"Transfer-Encoding": true,
"Trailer": true,
}
// Response represents the response from an HTTP request.
//
// The Client and Transport return Responses from servers once
// the response headers have been received. The response body
// is streamed on demand as the Body field is read.
type Response struct {
Status string // e.g. "200 OK"
StatusCode int // e.g. 200
Proto string // e.g. "HTTP/1.0"
ProtoMajor int // e.g. 1
ProtoMinor int // e.g. 0
// Header maps header keys to values. If the response had multiple
// headers with the same key, they may be concatenated, with comma
// delimiters. (RFC 7230, section 3.2.2 requires that multiple headers
// be semantically equivalent to a comma-delimited sequence.) When
// Header values are duplicated by other fields in this struct (e.g.,
// ContentLength, TransferEncoding, Trailer), the field values are
// authoritative.
//
// Keys in the map are canonicalized (see CanonicalHeaderKey).
Header Header
// Body represents the response body.
//
// The response body is streamed on demand as the Body field
// is read. If the network connection fails or the server
// terminates the response, Body.Read calls return an error.
//
// The http Client and Transport guarantee that Body is always
// non-nil, even on responses without a body or responses with
// a zero-length body. It is the caller's responsibility to
// close Body. The default HTTP client's Transport may not
// reuse HTTP/1.x "keep-alive" TCP connections if the Body is
// not read to completion and closed.
//
// The Body is automatically dechunked if the server replied
// with a "chunked" Transfer-Encoding.
//
// As of Go 1.12, the Body will also implement io.Writer
// on a successful "101 Switching Protocols" response,
// as used by WebSockets and HTTP/2's "h2c" mode.
Body io.ReadCloser
// ContentLength records the length of the associated content. The
// value -1 indicates that the length is unknown. Unless Request.Method
// is "HEAD", values >= 0 indicate that the given number of bytes may
// be read from Body.
ContentLength int64
// Contains transfer encodings from outer-most to inner-most. Value is
// nil, means that "identity" encoding is used.
TransferEncoding []string
// Close records whether the header directed that the connection be
// closed after reading Body. The value is advice for clients: neither
// ReadResponse nor Response.Write ever closes a connection.
Close bool
// Uncompressed reports whether the response was sent compressed but
// was decompressed by the http package. When true, reading from
// Body yields the uncompressed content instead of the compressed
// content actually set from the server, ContentLength is set to -1,
// and the "Content-Length" and "Content-Encoding" fields are deleted
// from the responseHeader. To get the original response from
// the server, set Transport.DisableCompression to true.
Uncompressed bool
// Trailer maps trailer keys to values in the same
// format as Header.
//
// The Trailer initially contains only nil values, one for
// each key specified in the server's "Trailer" header
// value. Those values are not added to Header.
//
// Trailer must not be accessed concurrently with Read calls
// on the Body.
//
// After Body.Read has returned io.EOF, Trailer will contain
// any trailer values sent by the server.
Trailer Header
// Request is the request that was sent to obtain this Response.
// Request's Body is nil (having already been consumed).
// This is only populated for Client requests.
Request *Request
}
// Cookies parses and returns the cookies set in the Set-Cookie headers.
func (r *Response) Cookies() []*Cookie {
return readSetCookies(r.Header)
}
// ErrNoLocation is returned by Response's Location method
// when no Location header is present.
var ErrNoLocation = errors.New("http: no Location header in response")
// Location returns the URL of the response's "Location" header,
// if present. Relative redirects are resolved relative to
// the Response's Request. ErrNoLocation is returned if no
// Location header is present.
func (r *Response) Location() (*url.URL, error) {
lv := r.Header.Get("Location")
if lv == "" {
return nil, ErrNoLocation
}
if r.Request != nil && r.Request.URL != nil {
return r.Request.URL.Parse(lv)
}
return url.Parse(lv)
}
// ReadResponse reads and returns an HTTP response from r.
// The req parameter optionally specifies the Request that corresponds
// to this Response. If nil, a GET request is assumed.
// Clients must call resp.Body.Close when finished reading resp.Body.
// After that call, clients can inspect resp.Trailer to find key/value
// pairs included in the response trailer.
// TINYGO: Added onEOF func to be called when response body is closed
// TINYGO: so we can clean up the connection (r)
func ReadResponse(r *bufio.Reader, req *Request) (*Response, error) {
tp := textproto.NewReader(r)
resp := &Response{
Request: req,
}
// Parse the first line of the response.
line, err := tp.ReadLine()
if err != nil {
if err == io.EOF {
err = io.ErrUnexpectedEOF
}
return nil, err
}
proto, status, ok := strings.Cut(line, " ")
if !ok {
return nil, badStringError("malformed HTTP response", line)
}
resp.Proto = proto
resp.Status = strings.TrimLeft(status, " ")
statusCode, _, _ := strings.Cut(resp.Status, " ")
if len(statusCode) != 3 {
return nil, badStringError("malformed HTTP status code", statusCode)
}
resp.StatusCode, err = strconv.Atoi(statusCode)
if err != nil || resp.StatusCode < 0 {
return nil, badStringError("malformed HTTP status code", statusCode)
}
if resp.ProtoMajor, resp.ProtoMinor, ok = ParseHTTPVersion(resp.Proto); !ok {
return nil, badStringError("malformed HTTP version", resp.Proto)
}
// Parse the response headers.
mimeHeader, err := tp.ReadMIMEHeader()
if err != nil {
if err == io.EOF {
err = io.ErrUnexpectedEOF
}
return nil, err
}
resp.Header = Header(mimeHeader)
fixPragmaCacheControl(resp.Header)
err = readTransfer(resp, r, req.onEOF)
if err != nil {
return nil, err
}
return resp, nil
}
// RFC 7234, section 5.4: Should treat
//
// Pragma: no-cache
//
// like
//
// Cache-Control: no-cache
func fixPragmaCacheControl(header Header) {
if hp, ok := header["Pragma"]; ok && len(hp) > 0 && hp[0] == "no-cache" {
if _, presentcc := header["Cache-Control"]; !presentcc {
header["Cache-Control"] = []string{"no-cache"}
}
}
}
// ProtoAtLeast reports whether the HTTP protocol used
// in the response is at least major.minor.
func (r *Response) ProtoAtLeast(major, minor int) bool {
return r.ProtoMajor > major ||
r.ProtoMajor == major && r.ProtoMinor >= minor
}
// Write writes r to w in the HTTP/1.x server response format,
// including the status line, headers, body, and optional trailer.
//
// This method consults the following fields of the response r:
//
// StatusCode
// ProtoMajor
// ProtoMinor
// Request.Method
// TransferEncoding
// Trailer
// Body
// ContentLength
// Header, values for non-canonical keys will have unpredictable behavior
//
// The Response Body is closed after it is sent.
func (r *Response) Write(w io.Writer) error {
// Status line
text := r.Status
if text == "" {
text = StatusText(r.StatusCode)
if text == "" {
text = "status code " + strconv.Itoa(r.StatusCode)
}
} else {
// Just to reduce stutter, if user set r.Status to "200 OK" and StatusCode to 200.
// Not important.
text = strings.TrimPrefix(text, strconv.Itoa(r.StatusCode)+" ")
}
if _, err := fmt.Fprintf(w, "HTTP/%d.%d %03d %s\r\n", r.ProtoMajor, r.ProtoMinor, r.StatusCode, text); err != nil {
return err
}
// Clone it, so we can modify r1 as needed.
r1 := new(Response)
*r1 = *r
if r1.ContentLength == 0 && r1.Body != nil {
// Is it actually 0 length? Or just unknown?
var buf [1]byte
n, err := r1.Body.Read(buf[:])
if err != nil && err != io.EOF {
return err
}
if n == 0 {
// Reset it to a known zero reader, in case underlying one
// is unhappy being read repeatedly.
r1.Body = NoBody
} else {
r1.ContentLength = -1
r1.Body = struct {
io.Reader
io.Closer
}{
io.MultiReader(bytes.NewReader(buf[:1]), r.Body),
r.Body,
}
}
}
// If we're sending a non-chunked HTTP/1.1 response without a
// content-length, the only way to do that is the old HTTP/1.0
// way, by noting the EOF with a connection close, so we need
// to set Close.
if r1.ContentLength == -1 && !r1.Close && r1.ProtoAtLeast(1, 1) && !chunked(r1.TransferEncoding) && !r1.Uncompressed {
r1.Close = true
}
// Process Body,ContentLength,Close,Trailer
tw, err := newTransferWriter(r1)
if err != nil {
return err
}
err = tw.writeHeader(w)
if err != nil {
return err
}
// Rest of header
err = r.Header.WriteSubset(w, respExcludeHeader)
if err != nil {
return err
}
// contentLengthAlreadySent may have been already sent for
// POST/PUT requests, even if zero length. See Issue 8180.
contentLengthAlreadySent := tw.shouldSendContentLength()
if r1.ContentLength == 0 && !chunked(r1.TransferEncoding) && !contentLengthAlreadySent && bodyAllowedForStatus(r.StatusCode) {
if _, err := io.WriteString(w, "Content-Length: 0\r\n"); err != nil {
return err
}
}
// End-of-header
if _, err := io.WriteString(w, "\r\n"); err != nil {
return err
}
// Write body and trailer
err = tw.writeBody(w)
if err != nil {
return err
}
// Success
return nil
}
func (r *Response) closeBody() {
if r.Body != nil {
r.Body.Close()
}
}
// bodyIsWritable reports whether the Body supports writing. The
// Transport returns Writable bodies for 101 Switching Protocols
// responses.
// The Transport uses this method to determine whether a persistent
// connection is done being managed from its perspective. Once we
// return a writable response body to a user, the net/http package is
// done managing that connection.
func (r *Response) bodyIsWritable() bool {
_, ok := r.Body.(io.Writer)
return ok
}
// isProtocolSwitch reports whether the response code and header
// indicate a successful protocol upgrade response.
func (r *Response) isProtocolSwitch() bool {
return isProtocolSwitchResponse(r.StatusCode, r.Header)
}
// isProtocolSwitchResponse reports whether the response code and
// response header indicate a successful protocol upgrade response.
func isProtocolSwitchResponse(code int, h Header) bool {
return code == StatusSwitchingProtocols && isProtocolSwitchHeader(h)
}
// isProtocolSwitchHeader reports whether the request or response header
// is for a protocol switch.
func isProtocolSwitchHeader(h Header) bool {
return h.Get("Upgrade") != "" &&
httpguts.HeaderValuesContainsToken(h["Connection"], "Upgrade")
}
http/server.go
+3261
View File
File diff suppressed because it is too large Load Diff
http/sniff.go
+306
View File
@@ -0,0 +1,306 @@
// TINYGO: The following is copied from Go 1.19.3 official implementation.
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package http
import (
"bytes"
"encoding/binary"
)
// The algorithm uses at most sniffLen bytes to make its decision.
const sniffLen = 512
// DetectContentType implements the algorithm described
// at https://mimesniff.spec.whatwg.org/ to determine the
// Content-Type of the given data. It considers at most the
// first 512 bytes of data. DetectContentType always returns
// a valid MIME type: if it cannot determine a more specific one, it
// returns "application/octet-stream".
func DetectContentType(data []byte) string {
if len(data) > sniffLen {
data = data[:sniffLen]
}
// Index of the first non-whitespace byte in data.
firstNonWS := 0
for ; firstNonWS < len(data) && isWS(data[firstNonWS]); firstNonWS++ {
}
for _, sig := range sniffSignatures {
if ct := sig.match(data, firstNonWS); ct != "" {
return ct
}
}
return "application/octet-stream" // fallback
}
// isWS reports whether the provided byte is a whitespace byte (0xWS)
// as defined in https://mimesniff.spec.whatwg.org/#terminology.
func isWS(b byte) bool {
switch b {
case '\t', '\n', '\x0c', '\r', ' ':
return true
}
return false
}
// isTT reports whether the provided byte is a tag-terminating byte (0xTT)
// as defined in https://mimesniff.spec.whatwg.org/#terminology.
func isTT(b byte) bool {
switch b {
case ' ', '>':
return true
}
return false
}
type sniffSig interface {
// match returns the MIME type of the data, or "" if unknown.
match(data []byte, firstNonWS int) string
}
// Data matching the table in section 6.
var sniffSignatures = []sniffSig{
htmlSig("<!DOCTYPE HTML"),
htmlSig("<HTML"),
htmlSig("<HEAD"),
htmlSig("<SCRIPT"),
htmlSig("<IFRAME"),
htmlSig("<H1"),
htmlSig("<DIV"),
htmlSig("<FONT"),
htmlSig("<TABLE"),
htmlSig("<A"),
htmlSig("<STYLE"),
htmlSig("<TITLE"),
htmlSig("<B"),
htmlSig("<BODY"),
htmlSig("<BR"),
htmlSig("<P"),
htmlSig("<!--"),
&maskedSig{
mask: []byte("\xFF\xFF\xFF\xFF\xFF"),
pat: []byte("<?xml"),
skipWS: true,
ct: "text/xml; charset=utf-8"},
&exactSig{[]byte("%PDF-"), "application/pdf"},
&exactSig{[]byte("%!PS-Adobe-"), "application/postscript"},
// UTF BOMs.
&maskedSig{
mask: []byte("\xFF\xFF\x00\x00"),
pat: []byte("\xFE\xFF\x00\x00"),
ct: "text/plain; charset=utf-16be",
},
&maskedSig{
mask: []byte("\xFF\xFF\x00\x00"),
pat: []byte("\xFF\xFE\x00\x00"),
ct: "text/plain; charset=utf-16le",
},
&maskedSig{
mask: []byte("\xFF\xFF\xFF\x00"),
pat: []byte("\xEF\xBB\xBF\x00"),
ct: "text/plain; charset=utf-8",
},
// Image types
// For posterity, we originally returned "image/vnd.microsoft.icon" from
// https://tools.ietf.org/html/draft-ietf-websec-mime-sniff-03#section-7
// https://codereview.appspot.com/4746042
// but that has since been replaced with "image/x-icon" in Section 6.2
// of https://mimesniff.spec.whatwg.org/#matching-an-image-type-pattern
&exactSig{[]byte("\x00\x00\x01\x00"), "image/x-icon"},
&exactSig{[]byte("\x00\x00\x02\x00"), "image/x-icon"},
&exactSig{[]byte("BM"), "image/bmp"},
&exactSig{[]byte("GIF87a"), "image/gif"},
&exactSig{[]byte("GIF89a"), "image/gif"},
&maskedSig{
mask: []byte("\xFF\xFF\xFF\xFF\x00\x00\x00\x00\xFF\xFF\xFF\xFF\xFF\xFF"),
pat: []byte("RIFF\x00\x00\x00\x00WEBPVP"),
ct: "image/webp",
},
&exactSig{[]byte("\x89PNG\x0D\x0A\x1A\x0A"), "image/png"},
&exactSig{[]byte("\xFF\xD8\xFF"), "image/jpeg"},
// Audio and Video types
// Enforce the pattern match ordering as prescribed in
// https://mimesniff.spec.whatwg.org/#matching-an-audio-or-video-type-pattern
&maskedSig{
mask: []byte("\xFF\xFF\xFF\xFF\x00\x00\x00\x00\xFF\xFF\xFF\xFF"),
pat: []byte("FORM\x00\x00\x00\x00AIFF"),
ct: "audio/aiff",
},
&maskedSig{
mask: []byte("\xFF\xFF\xFF"),
pat: []byte("ID3"),
ct: "audio/mpeg",
},
&maskedSig{
mask: []byte("\xFF\xFF\xFF\xFF\xFF"),
pat: []byte("OggS\x00"),
ct: "application/ogg",
},
&maskedSig{
mask: []byte("\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF"),
pat: []byte("MThd\x00\x00\x00\x06"),
ct: "audio/midi",
},
&maskedSig{
mask: []byte("\xFF\xFF\xFF\xFF\x00\x00\x00\x00\xFF\xFF\xFF\xFF"),
pat: []byte("RIFF\x00\x00\x00\x00AVI "),
ct: "video/avi",
},
&maskedSig{
mask: []byte("\xFF\xFF\xFF\xFF\x00\x00\x00\x00\xFF\xFF\xFF\xFF"),
pat: []byte("RIFF\x00\x00\x00\x00WAVE"),
ct: "audio/wave",
},
// 6.2.0.2. video/mp4
mp4Sig{},
// 6.2.0.3. video/webm
&exactSig{[]byte("\x1A\x45\xDF\xA3"), "video/webm"},
// Font types
&maskedSig{
// 34 NULL bytes followed by the string "LP"
pat: []byte("\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00LP"),
// 34 NULL bytes followed by \xF\xF
mask: []byte("\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xFF\xFF"),
ct: "application/vnd.ms-fontobject",
},
&exactSig{[]byte("\x00\x01\x00\x00"), "font/ttf"},
&exactSig{[]byte("OTTO"), "font/otf"},
&exactSig{[]byte("ttcf"), "font/collection"},
&exactSig{[]byte("wOFF"), "font/woff"},
&exactSig{[]byte("wOF2"), "font/woff2"},
// Archive types
&exactSig{[]byte("\x1F\x8B\x08"), "application/x-gzip"},
&exactSig{[]byte("PK\x03\x04"), "application/zip"},
// RAR's signatures are incorrectly defined by the MIME spec as per
// https://github.com/whatwg/mimesniff/issues/63
// However, RAR Labs correctly defines it at:
// https://www.rarlab.com/technote.htm#rarsign
// so we use the definition from RAR Labs.
// TODO: do whatever the spec ends up doing.
&exactSig{[]byte("Rar!\x1A\x07\x00"), "application/x-rar-compressed"}, // RAR v1.5-v4.0
&exactSig{[]byte("Rar!\x1A\x07\x01\x00"), "application/x-rar-compressed"}, // RAR v5+
&exactSig{[]byte("\x00\x61\x73\x6D"), "application/wasm"},
textSig{}, // should be last
}
type exactSig struct {
sig []byte
ct string
}
func (e *exactSig) match(data []byte, firstNonWS int) string {
if bytes.HasPrefix(data, e.sig) {
return e.ct
}
return ""
}
type maskedSig struct {
mask, pat []byte
skipWS bool
ct string
}
func (m *maskedSig) match(data []byte, firstNonWS int) string {
// pattern matching algorithm section 6
// https://mimesniff.spec.whatwg.org/#pattern-matching-algorithm
if m.skipWS {
data = data[firstNonWS:]
}
if len(m.pat) != len(m.mask) {
return ""
}
if len(data) < len(m.pat) {
return ""
}
for i, pb := range m.pat {
maskedData := data[i] & m.mask[i]
if maskedData != pb {
return ""
}
}
return m.ct
}
type htmlSig []byte
func (h htmlSig) match(data []byte, firstNonWS int) string {
data = data[firstNonWS:]
if len(data) < len(h)+1 {
return ""
}
for i, b := range h {
db := data[i]
if 'A' <= b && b <= 'Z' {
db &= 0xDF
}
if b != db {
return ""
}
}
// Next byte must be a tag-terminating byte(0xTT).
if !isTT(data[len(h)]) {
return ""
}
return "text/html; charset=utf-8"
}
var mp4ftype = []byte("ftyp")
var mp4 = []byte("mp4")
type mp4Sig struct{}
func (mp4Sig) match(data []byte, firstNonWS int) string {
// https://mimesniff.spec.whatwg.org/#signature-for-mp4
// c.f. section 6.2.1
if len(data) < 12 {
return ""
}
boxSize := int(binary.BigEndian.Uint32(data[:4]))
if len(data) < boxSize || boxSize%4 != 0 {
return ""
}
if !bytes.Equal(data[4:8], mp4ftype) {
return ""
}
for st := 8; st < boxSize; st += 4 {
if st == 12 {
// Ignores the four bytes that correspond to the version number of the "major brand".
continue
}
if bytes.Equal(data[st:st+3], mp4) {
return "video/mp4"
}
}
return ""
}
type textSig struct{}
func (textSig) match(data []byte, firstNonWS int) string {
// c.f. section 5, step 4.
for _, b := range data[firstNonWS:] {
switch {
case b <= 0x08,
b == 0x0B,
0x0E <= b && b <= 0x1A,
0x1C <= b && b <= 0x1F:
return ""
}
}
return "text/plain; charset=utf-8"
}
http/status.go
+212
View File
@@ -0,0 +1,212 @@
// TINYGO: The following is copied from Go 1.19.3 official implementation.
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package http
// HTTP status codes as registered with IANA.
// See: https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml
const (
StatusContinue = 100 // RFC 9110, 15.2.1
StatusSwitchingProtocols = 101 // RFC 9110, 15.2.2
StatusProcessing = 102 // RFC 2518, 10.1
StatusEarlyHints = 103 // RFC 8297
StatusOK = 200 // RFC 9110, 15.3.1
StatusCreated = 201 // RFC 9110, 15.3.2
StatusAccepted = 202 // RFC 9110, 15.3.3
StatusNonAuthoritativeInfo = 203 // RFC 9110, 15.3.4
StatusNoContent = 204 // RFC 9110, 15.3.5
StatusResetContent = 205 // RFC 9110, 15.3.6
StatusPartialContent = 206 // RFC 9110, 15.3.7
StatusMultiStatus = 207 // RFC 4918, 11.1
StatusAlreadyReported = 208 // RFC 5842, 7.1
StatusIMUsed = 226 // RFC 3229, 10.4.1
StatusMultipleChoices = 300 // RFC 9110, 15.4.1
StatusMovedPermanently = 301 // RFC 9110, 15.4.2
StatusFound = 302 // RFC 9110, 15.4.3
StatusSeeOther = 303 // RFC 9110, 15.4.4
StatusNotModified = 304 // RFC 9110, 15.4.5
StatusUseProxy = 305 // RFC 9110, 15.4.6
_ = 306 // RFC 9110, 15.4.7 (Unused)
StatusTemporaryRedirect = 307 // RFC 9110, 15.4.8
StatusPermanentRedirect = 308 // RFC 9110, 15.4.9
StatusBadRequest = 400 // RFC 9110, 15.5.1
StatusUnauthorized = 401 // RFC 9110, 15.5.2
StatusPaymentRequired = 402 // RFC 9110, 15.5.3
StatusForbidden = 403 // RFC 9110, 15.5.4
StatusNotFound = 404 // RFC 9110, 15.5.5
StatusMethodNotAllowed = 405 // RFC 9110, 15.5.6
StatusNotAcceptable = 406 // RFC 9110, 15.5.7
StatusProxyAuthRequired = 407 // RFC 9110, 15.5.8
StatusRequestTimeout = 408 // RFC 9110, 15.5.9
StatusConflict = 409 // RFC 9110, 15.5.10
StatusGone = 410 // RFC 9110, 15.5.11
StatusLengthRequired = 411 // RFC 9110, 15.5.12
StatusPreconditionFailed = 412 // RFC 9110, 15.5.13
StatusRequestEntityTooLarge = 413 // RFC 9110, 15.5.14
StatusRequestURITooLong = 414 // RFC 9110, 15.5.15
StatusUnsupportedMediaType = 415 // RFC 9110, 15.5.16
StatusRequestedRangeNotSatisfiable = 416 // RFC 9110, 15.5.17
StatusExpectationFailed = 417 // RFC 9110, 15.5.18
StatusTeapot = 418 // RFC 9110, 15.5.19 (Unused)
StatusMisdirectedRequest = 421 // RFC 9110, 15.5.20
StatusUnprocessableEntity = 422 // RFC 9110, 15.5.21
StatusLocked = 423 // RFC 4918, 11.3
StatusFailedDependency = 424 // RFC 4918, 11.4
StatusTooEarly = 425 // RFC 8470, 5.2.
StatusUpgradeRequired = 426 // RFC 9110, 15.5.22
StatusPreconditionRequired = 428 // RFC 6585, 3
StatusTooManyRequests = 429 // RFC 6585, 4
StatusRequestHeaderFieldsTooLarge = 431 // RFC 6585, 5
StatusUnavailableForLegalReasons = 451 // RFC 7725, 3
StatusInternalServerError = 500 // RFC 9110, 15.6.1
StatusNotImplemented = 501 // RFC 9110, 15.6.2
StatusBadGateway = 502 // RFC 9110, 15.6.3
StatusServiceUnavailable = 503 // RFC 9110, 15.6.4
StatusGatewayTimeout = 504 // RFC 9110, 15.6.5
StatusHTTPVersionNotSupported = 505 // RFC 9110, 15.6.6
StatusVariantAlsoNegotiates = 506 // RFC 2295, 8.1
StatusInsufficientStorage = 507 // RFC 4918, 11.5
StatusLoopDetected = 508 // RFC 5842, 7.2
StatusNotExtended = 510 // RFC 2774, 7
StatusNetworkAuthenticationRequired = 511 // RFC 6585, 6
)
// StatusText returns a text for the HTTP status code. It returns the empty
// string if the code is unknown.
func StatusText(code int) string {
switch code {
case StatusContinue:
return "Continue"
case StatusSwitchingProtocols:
return "Switching Protocols"
case StatusProcessing:
return "Processing"
case StatusEarlyHints:
return "Early Hints"
case StatusOK:
return "OK"
case StatusCreated:
return "Created"
case StatusAccepted:
return "Accepted"
case StatusNonAuthoritativeInfo:
return "Non-Authoritative Information"
case StatusNoContent:
return "No Content"
case StatusResetContent:
return "Reset Content"
case StatusPartialContent:
return "Partial Content"
case StatusMultiStatus:
return "Multi-Status"
case StatusAlreadyReported:
return "Already Reported"
case StatusIMUsed:
return "IM Used"
case StatusMultipleChoices:
return "Multiple Choices"
case StatusMovedPermanently:
return "Moved Permanently"
case StatusFound:
return "Found"
case StatusSeeOther:
return "See Other"
case StatusNotModified:
return "Not Modified"
case StatusUseProxy:
return "Use Proxy"
case StatusTemporaryRedirect:
return "Temporary Redirect"
case StatusPermanentRedirect:
return "Permanent Redirect"
case StatusBadRequest:
return "Bad Request"
case StatusUnauthorized:
return "Unauthorized"
case StatusPaymentRequired:
return "Payment Required"
case StatusForbidden:
return "Forbidden"
case StatusNotFound:
return "Not Found"
case StatusMethodNotAllowed:
return "Method Not Allowed"
case StatusNotAcceptable:
return "Not Acceptable"
case StatusProxyAuthRequired:
return "Proxy Authentication Required"
case StatusRequestTimeout:
return "Request Timeout"
case StatusConflict:
return "Conflict"
case StatusGone:
return "Gone"
case StatusLengthRequired:
return "Length Required"
case StatusPreconditionFailed:
return "Precondition Failed"
case StatusRequestEntityTooLarge:
return "Request Entity Too Large"
case StatusRequestURITooLong:
return "Request URI Too Long"
case StatusUnsupportedMediaType:
return "Unsupported Media Type"
case StatusRequestedRangeNotSatisfiable:
return "Requested Range Not Satisfiable"
case StatusExpectationFailed:
return "Expectation Failed"
case StatusTeapot:
return "I'm a teapot"
case StatusMisdirectedRequest:
return "Misdirected Request"
case StatusUnprocessableEntity:
return "Unprocessable Entity"
case StatusLocked:
return "Locked"
case StatusFailedDependency:
return "Failed Dependency"
case StatusTooEarly:
return "Too Early"
case StatusUpgradeRequired:
return "Upgrade Required"
case StatusPreconditionRequired:
return "Precondition Required"
case StatusTooManyRequests:
return "Too Many Requests"
case StatusRequestHeaderFieldsTooLarge:
return "Request Header Fields Too Large"
case StatusUnavailableForLegalReasons:
return "Unavailable For Legal Reasons"
case StatusInternalServerError:
return "Internal Server Error"
case StatusNotImplemented:
return "Not Implemented"
case StatusBadGateway:
return "Bad Gateway"
case StatusServiceUnavailable:
return "Service Unavailable"
case StatusGatewayTimeout:
return "Gateway Timeout"
case StatusHTTPVersionNotSupported:
return "HTTP Version Not Supported"
case StatusVariantAlsoNegotiates:
return "Variant Also Negotiates"
case StatusInsufficientStorage:
return "Insufficient Storage"
case StatusLoopDetected:
return "Loop Detected"
case StatusNotExtended:
return "Not Extended"
case StatusNetworkAuthenticationRequired:
return "Network Authentication Required"
default:
return ""
}
}
http/transfer.go
+1127
View File
File diff suppressed because it is too large Load Diff
http/transport.go
+22
View File
@@ -0,0 +1,22 @@
// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// HTTP client implementation. See RFC 7230 through 7235.
//
// This is the low-level Transport implementation of RoundTripper.
// The high-level interface is in client.go.
package http
import (
"io"
)
type readTrackingBody struct {
io.ReadCloser
didRead bool
didClose bool
}
interface.go
-253
View File
@@ -1,253 +0,0 @@
// The following is copied from Go 1.16 official implementation.
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package net
import (
"errors"
"internal/itoa"
"sync"
"time"
)
var (
errInvalidInterface = errors.New("invalid network interface")
errInvalidInterfaceIndex = errors.New("invalid network interface index")
errInvalidInterfaceName = errors.New("invalid network interface name")
errNoSuchInterface = errors.New("no such network interface")
errNoSuchMulticastInterface = errors.New("no such multicast network interface")
)
// Interface represents a mapping between network interface name
// and index. It also represents network interface facility
// information.
type Interface struct {
Index int // positive integer that starts at one, zero is never used
MTU int // maximum transmission unit
Name string // e.g., "en0", "lo0", "eth0.100"
HardwareAddr HardwareAddr // IEEE MAC-48, EUI-48 and EUI-64 form
Flags Flags // e.g., FlagUp, FlagLoopback, FlagMulticast
}
type Flags uint
const (
FlagUp Flags = 1 << iota // interface is up
FlagBroadcast // interface supports broadcast access capability
FlagLoopback // interface is a loopback interface
FlagPointToPoint // interface belongs to a point-to-point link
FlagMulticast // interface supports multicast access capability
)
var flagNames = []string{
"up",
"broadcast",
"loopback",
"pointtopoint",
"multicast",
}
func (f Flags) String() string {
s := ""
for i, name := range flagNames {
if f&(1<<uint(i)) != 0 {
if s != "" {
s += "|"
}
s += name
}
}
if s == "" {
s = "0"
}
return s
}
// Addrs returns a list of unicast interface addresses for a specific
// interface.
func (ifi *Interface) Addrs() ([]Addr, error) {
if ifi == nil {
return nil, &OpError{Op: "route", Net: "ip+net", Source: nil, Addr: nil, Err: errInvalidInterface}
}
ifat, err := interfaceAddrTable(ifi)
if err != nil {
err = &OpError{Op: "route", Net: "ip+net", Source: nil, Addr: nil, Err: err}
}
return ifat, err
}
// MulticastAddrs returns a list of multicast, joined group addresses
// for a specific interface.
func (ifi *Interface) MulticastAddrs() ([]Addr, error) {
if ifi == nil {
return nil, &OpError{Op: "route", Net: "ip+net", Source: nil, Addr: nil, Err: errInvalidInterface}
}
ifat, err := interfaceMulticastAddrTable(ifi)
if err != nil {
err = &OpError{Op: "route", Net: "ip+net", Source: nil, Addr: nil, Err: err}
}
return ifat, err
}
// Interfaces returns a list of the system's network interfaces.
func Interfaces() ([]Interface, error) {
ift, err := interfaceTable(0)
if err != nil {
return nil, &OpError{Op: "route", Net: "ip+net", Source: nil, Addr: nil, Err: err}
}
if len(ift) != 0 {
zoneCache.update(ift, false)
}
return ift, nil
}
// InterfaceAddrs returns a list of the system's unicast interface
// addresses.
//
// The returned list does not identify the associated interface; use
// Interfaces and Interface.Addrs for more detail.
func InterfaceAddrs() ([]Addr, error) {
ifat, err := interfaceAddrTable(nil)
if err != nil {
err = &OpError{Op: "route", Net: "ip+net", Source: nil, Addr: nil, Err: err}
}
return ifat, err
}
// InterfaceByIndex returns the interface specified by index.
//
// On Solaris, it returns one of the logical network interfaces
// sharing the logical data link; for more precision use
// InterfaceByName.
func InterfaceByIndex(index int) (*Interface, error) {
if index <= 0 {
return nil, &OpError{Op: "route", Net: "ip+net", Source: nil, Addr: nil, Err: errInvalidInterfaceIndex}
}
ift, err := interfaceTable(index)
if err != nil {
return nil, &OpError{Op: "route", Net: "ip+net", Source: nil, Addr: nil, Err: err}
}
ifi, err := interfaceByIndex(ift, index)
if err != nil {
err = &OpError{Op: "route", Net: "ip+net", Source: nil, Addr: nil, Err: err}
}
return ifi, err
}
func interfaceByIndex(ift []Interface, index int) (*Interface, error) {
for _, ifi := range ift {
if index == ifi.Index {
return &ifi, nil
}
}
return nil, errNoSuchInterface
}
// InterfaceByName returns the interface specified by name.
func InterfaceByName(name string) (*Interface, error) {
if name == "" {
return nil, &OpError{Op: "route", Net: "ip+net", Source: nil, Addr: nil, Err: errInvalidInterfaceName}
}
ift, err := interfaceTable(0)
if err != nil {
return nil, &OpError{Op: "route", Net: "ip+net", Source: nil, Addr: nil, Err: err}
}
if len(ift) != 0 {
zoneCache.update(ift, false)
}
for _, ifi := range ift {
if name == ifi.Name {
return &ifi, nil
}
}
return nil, &OpError{Op: "route", Net: "ip+net", Source: nil, Addr: nil, Err: errNoSuchInterface}
}
// An ipv6ZoneCache represents a cache holding partial network
// interface information. It is used for reducing the cost of IPv6
// addressing scope zone resolution.
//
// Multiple names sharing the index are managed by first-come
// first-served basis for consistency.
type ipv6ZoneCache struct {
sync.RWMutex // guard the following
lastFetched time.Time // last time routing information was fetched
toIndex map[string]int // interface name to its index
toName map[int]string // interface index to its name
}
var zoneCache = ipv6ZoneCache{
toIndex: make(map[string]int),
toName: make(map[int]string),
}
// update refreshes the network interface information if the cache was last
// updated more than 1 minute ago, or if force is set. It reports whether the
// cache was updated.
func (zc *ipv6ZoneCache) update(ift []Interface, force bool) (updated bool) {
zc.Lock()
defer zc.Unlock()
now := time.Now()
if !force && zc.lastFetched.After(now.Add(-60*time.Second)) {
return false
}
zc.lastFetched = now
if len(ift) == 0 {
var err error
if ift, err = interfaceTable(0); err != nil {
return false
}
}
zc.toIndex = make(map[string]int, len(ift))
zc.toName = make(map[int]string, len(ift))
for _, ifi := range ift {
zc.toIndex[ifi.Name] = ifi.Index
if _, ok := zc.toName[ifi.Index]; !ok {
zc.toName[ifi.Index] = ifi.Name
}
}
return true
}
func (zc *ipv6ZoneCache) name(index int) string {
if index == 0 {
return ""
}
updated := zoneCache.update(nil, false)
zoneCache.RLock()
name, ok := zoneCache.toName[index]
zoneCache.RUnlock()
if !ok && !updated {
zoneCache.update(nil, true)
zoneCache.RLock()
name, ok = zoneCache.toName[index]
zoneCache.RUnlock()
}
if !ok { // last resort
name = itoa.Uitoa(uint(index))
}
return name
}
func (zc *ipv6ZoneCache) index(name string) int {
if name == "" {
return 0
}
updated := zoneCache.update(nil, false)
zoneCache.RLock()
index, ok := zoneCache.toIndex[name]
zoneCache.RUnlock()
if !ok && !updated {
zoneCache.update(nil, true)
zoneCache.RLock()
index, ok = zoneCache.toIndex[name]
zoneCache.RUnlock()
}
if !ok { // last resort
index, _, _ = dtoi(name)
}
return index
}
interface_tinygo.go
-53
View File
@@ -1,53 +0,0 @@
//go:build tinygo
package net
const (
tinyGoInterfaceName = "tinygo-undefined"
maxTransmissionUnit = 1500 // Ethernet?
)
// DE:AD:BE:EF:FE:FF
var defaultMAC = HardwareAddr{0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xFF}
// If the ifindex is zero, interfaceTable returns mappings of all
// network interfaces. Otherwise it returns a mapping of a specific
// interface.
func interfaceTable(ifindex int) ([]Interface, error) {
i, err := readInterface(0)
if err != nil {
return nil, err
}
return []Interface{*i}, nil
}
func readInterface(i int) (*Interface, error) {
if i != 0 {
return nil, errInvalidInterfaceIndex
}
ifc := &Interface{
Index: i + 1, // Offset the index by one to suit the contract
Name: tinyGoInterfaceName,
MTU: maxTransmissionUnit,
HardwareAddr: defaultMAC,
Flags: 0, // No flags since interface is not implemented.
}
return ifc, nil
}
func interfaceCount() (int, error) {
return 1, nil
}
// If the ifi is nil, interfaceAddrTable returns addresses for all
// network interfaces. Otherwise it returns addresses for a specific
// interface.
func interfaceAddrTable(ifi *Interface) ([]Addr, error) {
return nil, nil
}
// interfaceMulticastAddrTable returns addresses for a specific
// interface.
func interfaceMulticastAddrTable(ifi *Interface) ([]Addr, error) {
return nil, nil
}
ip.go
+28 -2
View File
@@ -1,4 +1,4 @@
// The following is copied from Go 1.16 official implementation.
// TINYGO: The following is copied from Go 1.19.3 official implementation.
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
@@ -130,6 +130,25 @@ func (ip IP) IsLoopback() bool {
return ip.Equal(IPv6loopback)
}
// IsPrivate reports whether ip is a private address, according to
// RFC 1918 (IPv4 addresses) and RFC 4193 (IPv6 addresses).
func (ip IP) IsPrivate() bool {
if ip4 := ip.To4(); ip4 != nil {
// Following RFC 1918, Section 3. Private Address Space which says:
// The Internet Assigned Numbers Authority (IANA) has reserved the
// following three blocks of the IP address space for private internets:
// 10.0.0.0 - 10.255.255.255 (10/8 prefix)
// 172.16.0.0 - 172.31.255.255 (172.16/12 prefix)
// 192.168.0.0 - 192.168.255.255 (192.168/16 prefix)
return ip4[0] == 10 ||
(ip4[0] == 172 && ip4[1]&0xf0 == 16) ||
(ip4[0] == 192 && ip4[1] == 168)
}
// Following RFC 4193, Section 8. IANA Considerations which says:
// The IANA has assigned the FC00::/7 prefix to "Unique Local Unicast".
return len(ip) == IPv6len && ip[0]&0xfe == 0xfc
}
// IsMulticast reports whether ip is a multicast address.
func (ip IP) IsMulticast() bool {
if ip4 := ip.To4(); ip4 != nil {
@@ -291,7 +310,7 @@ func ubtoa(dst []byte, start int, v byte) int {
// It returns one of 4 forms:
// - "<nil>", if ip has length 0
// - dotted decimal ("192.0.2.1"), if ip is an IPv4 or IP4-mapped IPv6 address
// - IPv6 ("2001:db8::1"), if ip is a valid IPv6 address
// - IPv6 conforming to RFC 5952 ("2001:db8::1"), if ip is a valid IPv6 address
// - the hexadecimal form of ip, without punctuation, if no other cases apply
func (ip IP) String() string {
p := ip
@@ -528,6 +547,9 @@ func (n *IPNet) Network() string { return "ip+net" }
// character and a mask expressed as hexadecimal form with no
// punctuation like "198.51.100.0/c000ff00".
func (n *IPNet) String() string {
if n == nil {
return "<nil>"
}
nn, m := networkNumberAndMask(n)
if nn == nil || m == nil {
return "<nil>"
@@ -557,6 +579,10 @@ func parseIPv4(s string) IP {
if !ok || n > 0xFF {
return nil
}
if c > 1 && s[0] == '0' {
// Reject non-zero components with leading zeroes.
return nil
}
s = s[c:]
p[i] = byte(n)
}
iprawsock.go
+19 -1
View File
@@ -1,4 +1,4 @@
// The following is copied from Go 1.16 official implementation.
// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
@@ -6,6 +6,24 @@
package net
// BUG(mikio): On every POSIX platform, reads from the "ip4" network
// using the ReadFrom or ReadFromIP method might not return a complete
// IPv4 packet, including its header, even if there is space
// available. This can occur even in cases where Read or ReadMsgIP
// could return a complete packet. For this reason, it is recommended
// that you do not use these methods if it is important to receive a
// full packet.
//
// The Go 1 compatibility guidelines make it impossible for us to
// change the behavior of these methods; use Read or ReadMsgIP
// instead.
// BUG(mikio): On JS and Plan 9, methods and functions related
// to IPConn are not implemented.
// BUG(mikio): On Windows, the File method of IPConn is not
// implemented.
// IPAddr represents the address of an IP end point.
type IPAddr struct {
IP IP
ipsock.go
+4 -2
View File
@@ -1,4 +1,4 @@
// The following is copied from Go 1.16 official implementation.
// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
@@ -6,7 +6,9 @@
package net
import "internal/bytealg"
import (
"internal/bytealg"
)
// SplitHostPort splits a network address of the form "host:port",
// "host%zone:port", "[host]:port" or "[host%zone]:port" into host or
mac.go
+11 -11
View File
@@ -1,4 +1,4 @@
// The following is copied from Go 1.16 official implementation.
// TINYGO: The following is copied from Go 1.19.3 official implementation.
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
@@ -40,49 +40,49 @@ func (a HardwareAddr) String() string {
// 0000.0000.fe80.0000.0000.0000.0200.5e10.0000.0001
func ParseMAC(s string) (hw HardwareAddr, err error) {
if len(s) < 14 {
goto err
goto error
}
if s[2] == ':' || s[2] == '-' {
if (len(s)+1)%3 != 0 {
goto err
goto error
}
n := (len(s) + 1) / 3
if n != 6 && n != 8 && n != 20 {
goto err
goto error
}
hw = make(HardwareAddr, n)
for x, i := 0, 0; i < n; i++ {
var ok bool
if hw[i], ok = xtoi2(s[x:], s[2]); !ok {
goto err
goto error
}
x += 3
}
} else if s[4] == '.' {
if (len(s)+1)%5 != 0 {
goto err
goto error
}
n := 2 * (len(s) + 1) / 5
if n != 6 && n != 8 && n != 20 {
goto err
goto error
}
hw = make(HardwareAddr, n)
for x, i := 0, 0; i < n; i += 2 {
var ok bool
if hw[i], ok = xtoi2(s[x:x+2], 0); !ok {
goto err
goto error
}
if hw[i+1], ok = xtoi2(s[x+2:], s[4]); !ok {
goto err
goto error
}
x += 5
}
} else {
goto err
goto error
}
return hw, nil
err:
error:
return nil, &AddrError{Err: "invalid MAC address", Addr: s}
}
mac_test.go
+109
View File
@@ -0,0 +1,109 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package net
import (
"reflect"
"strings"
"testing"
)
var parseMACTests = []struct {
in string
out HardwareAddr
err string
}{
// See RFC 7042, Section 2.1.1.
{"00:00:5e:00:53:01", HardwareAddr{0x00, 0x00, 0x5e, 0x00, 0x53, 0x01}, ""},
{"00-00-5e-00-53-01", HardwareAddr{0x00, 0x00, 0x5e, 0x00, 0x53, 0x01}, ""},
{"0000.5e00.5301", HardwareAddr{0x00, 0x00, 0x5e, 0x00, 0x53, 0x01}, ""},
// See RFC 7042, Section 2.2.2.
{"02:00:5e:10:00:00:00:01", HardwareAddr{0x02, 0x00, 0x5e, 0x10, 0x00, 0x00, 0x00, 0x01}, ""},
{"02-00-5e-10-00-00-00-01", HardwareAddr{0x02, 0x00, 0x5e, 0x10, 0x00, 0x00, 0x00, 0x01}, ""},
{"0200.5e10.0000.0001", HardwareAddr{0x02, 0x00, 0x5e, 0x10, 0x00, 0x00, 0x00, 0x01}, ""},
// See RFC 4391, Section 9.1.1.
{
"00:00:00:00:fe:80:00:00:00:00:00:00:02:00:5e:10:00:00:00:01",
HardwareAddr{
0x00, 0x00, 0x00, 0x00,
0xfe, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x02, 0x00, 0x5e, 0x10, 0x00, 0x00, 0x00, 0x01,
},
"",
},
{
"00-00-00-00-fe-80-00-00-00-00-00-00-02-00-5e-10-00-00-00-01",
HardwareAddr{
0x00, 0x00, 0x00, 0x00,
0xfe, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x02, 0x00, 0x5e, 0x10, 0x00, 0x00, 0x00, 0x01,
},
"",
},
{
"0000.0000.fe80.0000.0000.0000.0200.5e10.0000.0001",
HardwareAddr{
0x00, 0x00, 0x00, 0x00,
0xfe, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x02, 0x00, 0x5e, 0x10, 0x00, 0x00, 0x00, 0x01,
},
"",
},
{"ab:cd:ef:AB:CD:EF", HardwareAddr{0xab, 0xcd, 0xef, 0xab, 0xcd, 0xef}, ""},
{"ab:cd:ef:AB:CD:EF:ab:cd", HardwareAddr{0xab, 0xcd, 0xef, 0xab, 0xcd, 0xef, 0xab, 0xcd}, ""},
{
"ab:cd:ef:AB:CD:EF:ab:cd:ef:AB:CD:EF:ab:cd:ef:AB:CD:EF:ab:cd",
HardwareAddr{
0xab, 0xcd, 0xef, 0xab,
0xcd, 0xef, 0xab, 0xcd, 0xef, 0xab, 0xcd, 0xef,
0xab, 0xcd, 0xef, 0xab, 0xcd, 0xef, 0xab, 0xcd,
},
"",
},
{"01.02.03.04.05.06", nil, "invalid MAC address"},
{"01:02:03:04:05:06:", nil, "invalid MAC address"},
{"x1:02:03:04:05:06", nil, "invalid MAC address"},
{"01002:03:04:05:06", nil, "invalid MAC address"},
{"01:02003:04:05:06", nil, "invalid MAC address"},
{"01:02:03004:05:06", nil, "invalid MAC address"},
{"01:02:03:04005:06", nil, "invalid MAC address"},
{"01:02:03:04:05006", nil, "invalid MAC address"},
{"01-02:03:04:05:06", nil, "invalid MAC address"},
{"01:02-03-04-05-06", nil, "invalid MAC address"},
{"0123:4567:89AF", nil, "invalid MAC address"},
{"0123-4567-89AF", nil, "invalid MAC address"},
}
func TestParseMAC(t *testing.T) {
match := func(err error, s string) bool {
if s == "" {
return err == nil
}
return err != nil && strings.Contains(err.Error(), s)
}
for i, tt := range parseMACTests {
out, err := ParseMAC(tt.in)
if !reflect.DeepEqual(out, tt.out) || !match(err, tt.err) {
t.Errorf("ParseMAC(%q) = %v, %v, want %v, %v", tt.in, out, err, tt.out, tt.err)
}
if tt.err == "" {
// Verify that serialization works too, and that it round-trips.
s := out.String()
out2, err := ParseMAC(s)
if err != nil {
t.Errorf("%d. ParseMAC(%q) = %v", i, s, err)
continue
}
if !reflect.DeepEqual(out2, out) {
t.Errorf("%d. ParseMAC(%q) = %v, want %v", i, s, out2, out)
}
}
}
}
net.go
+1 -90
View File
@@ -1,4 +1,4 @@
// The following is copied from Go 1.18 official implementation.
// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
@@ -7,7 +7,6 @@
package net
import (
"io"
"time"
)
@@ -81,10 +80,6 @@ type Conn interface {
SetWriteDeadline(t time.Time) error
}
type conn struct {
//
}
// A Listener is a generic network listener for stream-oriented protocols.
//
// Multiple goroutines may invoke methods on a Listener simultaneously.
@@ -193,87 +188,3 @@ func (e *AddrError) Error() string {
}
return s
}
func (e *AddrError) Timeout() bool { return false }
func (e *AddrError) Temporary() bool { return false }
// ErrClosed is the error returned by an I/O call on a network
// connection that has already been closed, or that is closed by
// another goroutine before the I/O is completed. This may be wrapped
// in another error, and should normally be tested using
// errors.Is(err, net.ErrClosed).
var ErrClosed = errClosed
// buffersWriter is the interface implemented by Conns that support a
// "writev"-like batch write optimization.
// writeBuffers should fully consume and write all chunks from the
// provided Buffers, else it should report a non-nil error.
type buffersWriter interface {
writeBuffers(*Buffers) (int64, error)
}
// Buffers contains zero or more runs of bytes to write.
//
// On certain machines, for certain types of connections, this is
// optimized into an OS-specific batch write operation (such as
// "writev").
type Buffers [][]byte
var (
_ io.WriterTo = (*Buffers)(nil)
_ io.Reader = (*Buffers)(nil)
)
// WriteTo writes contents of the buffers to w.
//
// WriteTo implements io.WriterTo for Buffers.
//
// WriteTo modifies the slice v as well as v[i] for 0 <= i < len(v),
// but does not modify v[i][j] for any i, j.
func (v *Buffers) WriteTo(w io.Writer) (n int64, err error) {
if wv, ok := w.(buffersWriter); ok {
return wv.writeBuffers(v)
}
for _, b := range *v {
nb, err := w.Write(b)
n += int64(nb)
if err != nil {
v.consume(n)
return n, err
}
}
v.consume(n)
return n, nil
}
// Read from the buffers.
//
// Read implements io.Reader for Buffers.
//
// Read modifies the slice v as well as v[i] for 0 <= i < len(v),
// but does not modify v[i][j] for any i, j.
func (v *Buffers) Read(p []byte) (n int, err error) {
for len(p) > 0 && len(*v) > 0 {
n0 := copy(p, (*v)[0])
v.consume(int64(n0))
p = p[n0:]
n += n0
}
if len(*v) == 0 {
err = io.EOF
}
return
}
func (v *Buffers) consume(n int64) {
for len(*v) > 0 {
ln0 := int64(len((*v)[0]))
if ln0 > n {
(*v)[0] = (*v)[0][n:]
return
}
n -= ln0
(*v)[0] = nil
*v = (*v)[1:]
}
}
netdev.go
+46
View File
@@ -0,0 +1,46 @@
package net
import (
"time"
)
// netdev is the current netdev, set by the application with useNetdev()
var netdev netdever
// (useNetdev is go:linkname'd from tinygo/drivers package)
func useNetdev(dev netdever) {
netdev = dev
}
// Netdev is TinyGo's network device driver model. Network drivers implement
// the netdever interface, providing a common network I/O interface to TinyGo's
// "net" package. The interface is modeled after the BSD socket interface.
// net.Conn implementations (TCPConn, UDPConn, and TLSConn) use the netdev
// interface for device I/O access.
//
// A netdever is passed to the "net" package using net.useNetdev().
//
// Just like a net.Conn, multiple goroutines may invoke methods on a netdever
// simultaneously.
//
// NOTE: The netdever interface is mirrored in drivers/netdev.go.
// NOTE: If making changes to this interface, mirror the changes in
// NOTE: drivers/netdev.go, and visa-versa.
type netdever interface {
// GetHostByName returns the IP address of either a hostname or IPv4
// address in standard dot notation
GetHostByName(name string) (IP, error)
// Berkely Sockets-like interface, Go-ified. See man page for socket(2), etc.
Socket(domain int, stype int, protocol int) (int, error)
Bind(sockfd int, ip IP, port int) error
Connect(sockfd int, host string, ip IP, port int) error
Listen(sockfd int, backlog int) error
Accept(sockfd int, ip IP, port int) (int, error)
Send(sockfd int, buf []byte, flags int, timeout time.Duration) (int, error)
Recv(sockfd int, buf []byte, flags int, timeout time.Duration) (int, error)
Close(sockfd int) error
SetSockOpt(sockfd int, level int, opt int, value interface{}) error
}
parse.go
+256 -1
View File
@@ -1,11 +1,121 @@
// The following is copied from Go 1.16 official implementation.
// TINYGO: The following is copied from Go 1.19.3 official implementation.
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Simple file i/o and string manipulation, to avoid
// depending on strconv and bufio and strings.
package net
import (
"internal/bytealg"
"io"
"os"
"time"
)
type file struct {
file *os.File
data []byte
atEOF bool
}
func (f *file) close() { f.file.Close() }
func (f *file) getLineFromData() (s string, ok bool) {
data := f.data
i := 0
for i = 0; i < len(data); i++ {
if data[i] == '\n' {
s = string(data[0:i])
ok = true
// move data
i++
n := len(data) - i
copy(data[0:], data[i:])
f.data = data[0:n]
return
}
}
if f.atEOF && len(f.data) > 0 {
// EOF, return all we have
s = string(data)
f.data = f.data[0:0]
ok = true
}
return
}
func (f *file) readLine() (s string, ok bool) {
if s, ok = f.getLineFromData(); ok {
return
}
if len(f.data) < cap(f.data) {
ln := len(f.data)
n, err := io.ReadFull(f.file, f.data[ln:cap(f.data)])
if n >= 0 {
f.data = f.data[0 : ln+n]
}
if err == io.EOF || err == io.ErrUnexpectedEOF {
f.atEOF = true
}
}
s, ok = f.getLineFromData()
return
}
func open(name string) (*file, error) {
fd, err := os.Open(name)
if err != nil {
return nil, err
}
return &file{fd, make([]byte, 0, 64*1024), false}, nil
}
func stat(name string) (mtime time.Time, size int64, err error) {
st, err := os.Stat(name)
if err != nil {
return time.Time{}, 0, err
}
return st.ModTime(), st.Size(), nil
}
// Count occurrences in s of any bytes in t.
func countAnyByte(s string, t string) int {
n := 0
for i := 0; i < len(s); i++ {
if bytealg.IndexByteString(t, s[i]) >= 0 {
n++
}
}
return n
}
// Split s at any bytes in t.
func splitAtBytes(s string, t string) []string {
a := make([]string, 1+countAnyByte(s, t))
n := 0
last := 0
for i := 0; i < len(s); i++ {
if bytealg.IndexByteString(t, s[i]) >= 0 {
if last < i {
a[n] = s[last:i]
n++
}
last = i + 1
}
}
if last < len(s) {
a[n] = s[last:]
n++
}
return a[0:n]
}
func getFields(s string) []string { return splitAtBytes(s, " \r\t\n") }
// Bigger than we need, not too big to worry about overflow
const big = 0xFFFFFF
@@ -78,6 +188,17 @@ func appendHex(dst []byte, i uint32) []byte {
return dst
}
// Number of occurrences of b in s.
func count(s string, b byte) int {
n := 0
for i := 0; i < len(s); i++ {
if s[i] == b {
n++
}
}
return n
}
// Index of rightmost occurrence of b in s.
func last(s string, b byte) int {
i := len(s)
@@ -88,3 +209,137 @@ func last(s string, b byte) int {
}
return i
}
// hasUpperCase tells whether the given string contains at least one upper-case.
func hasUpperCase(s string) bool {
for i := range s {
if 'A' <= s[i] && s[i] <= 'Z' {
return true
}
}
return false
}
// lowerASCIIBytes makes x ASCII lowercase in-place.
func lowerASCIIBytes(x []byte) {
for i, b := range x {
if 'A' <= b && b <= 'Z' {
x[i] += 'a' - 'A'
}
}
}
// lowerASCII returns the ASCII lowercase version of b.
func lowerASCII(b byte) byte {
if 'A' <= b && b <= 'Z' {
return b + ('a' - 'A')
}
return b
}
// trimSpace returns x without any leading or trailing ASCII whitespace.
func trimSpace(x []byte) []byte {
for len(x) > 0 && isSpace(x[0]) {
x = x[1:]
}
for len(x) > 0 && isSpace(x[len(x)-1]) {
x = x[:len(x)-1]
}
return x
}
// isSpace reports whether b is an ASCII space character.
func isSpace(b byte) bool {
return b == ' ' || b == '\t' || b == '\n' || b == '\r'
}
// removeComment returns line, removing any '#' byte and any following
// bytes.
func removeComment(line []byte) []byte {
if i := bytealg.IndexByte(line, '#'); i != -1 {
return line[:i]
}
return line
}
// foreachLine runs fn on each line of x.
// Each line (except for possibly the last) ends in '\n'.
// It returns the first non-nil error returned by fn.
func foreachLine(x []byte, fn func(line []byte) error) error {
for len(x) > 0 {
nl := bytealg.IndexByte(x, '\n')
if nl == -1 {
return fn(x)
}
line := x[:nl+1]
x = x[nl+1:]
if err := fn(line); err != nil {
return err
}
}
return nil
}
// foreachField runs fn on each non-empty run of non-space bytes in x.
// It returns the first non-nil error returned by fn.
func foreachField(x []byte, fn func(field []byte) error) error {
x = trimSpace(x)
for len(x) > 0 {
sp := bytealg.IndexByte(x, ' ')
if sp == -1 {
return fn(x)
}
if field := trimSpace(x[:sp]); len(field) > 0 {
if err := fn(field); err != nil {
return err
}
}
x = trimSpace(x[sp+1:])
}
return nil
}
// stringsHasSuffix is strings.HasSuffix. It reports whether s ends in
// suffix.
func stringsHasSuffix(s, suffix string) bool {
return len(s) >= len(suffix) && s[len(s)-len(suffix):] == suffix
}
// stringsHasSuffixFold reports whether s ends in suffix,
// ASCII-case-insensitively.
func stringsHasSuffixFold(s, suffix string) bool {
return len(s) >= len(suffix) && stringsEqualFold(s[len(s)-len(suffix):], suffix)
}
// stringsHasPrefix is strings.HasPrefix. It reports whether s begins with prefix.
func stringsHasPrefix(s, prefix string) bool {
return len(s) >= len(prefix) && s[:len(prefix)] == prefix
}
// stringsEqualFold is strings.EqualFold, ASCII only. It reports whether s and t
// are equal, ASCII-case-insensitively.
func stringsEqualFold(s, t string) bool {
if len(s) != len(t) {
return false
}
for i := 0; i < len(s); i++ {
if lowerASCII(s[i]) != lowerASCII(t[i]) {
return false
}
}
return true
}
func readFull(r io.Reader) (all []byte, err error) {
buf := make([]byte, 1024)
for {
n, err := r.Read(buf)
all = append(all, buf[:n]...)
if err == io.EOF {
return all, nil
}
if err != nil {
return nil, err
}
}
}
pipe.go
+1 -1
View File
@@ -1,4 +1,4 @@
// The following is copied from Go 1.19.2 official implementation.
// The following is copied from Go 1.19.3 official implementation.
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
pipe_test.go
-48
View File
@@ -1,48 +0,0 @@
// The following is copied from Go 1.19.2 official implementation.
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package net
import (
"io"
"testing"
"time"
)
func TestPipe(t *testing.T) {
testConn(t, func() (c1, c2 Conn, stop func(), err error) {
c1, c2 = Pipe()
stop = func() {
c1.Close()
c2.Close()
}
return
})
}
func TestPipeCloseError(t *testing.T) {
c1, c2 := Pipe()
c1.Close()
if _, err := c1.Read(nil); err != io.ErrClosedPipe {
t.Errorf("c1.Read() = %v, want io.ErrClosedPipe", err)
}
if _, err := c1.Write(nil); err != io.ErrClosedPipe {
t.Errorf("c1.Write() = %v, want io.ErrClosedPipe", err)
}
if err := c1.SetDeadline(time.Time{}); err != io.ErrClosedPipe {
t.Errorf("c1.SetDeadline() = %v, want io.ErrClosedPipe", err)
}
if _, err := c2.Read(nil); err != io.EOF {
t.Errorf("c2.Read() = %v, want io.EOF", err)
}
if _, err := c2.Write(nil); err != io.ErrClosedPipe {
t.Errorf("c2.Write() = %v, want io.ErrClosedPipe", err)
}
if err := c2.SetDeadline(time.Time{}); err != io.ErrClosedPipe {
t.Errorf("c2.SetDeadline() = %v, want io.ErrClosedPipe", err)
}
}
tcpsock.go
+231 -2
View File
@@ -1,8 +1,18 @@
// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package net
import (
"fmt"
"internal/itoa"
"net/netip"
"strconv"
"syscall"
"time"
)
// TCPAddr represents the address of a TCP end point.
@@ -54,12 +64,231 @@ func (a *TCPAddr) opAddr() Addr {
return a
}
// ResolveTCPAddr returns an address of TCP end point.
//
// The network must be a TCP network name.
//
// If the host in the address parameter is not a literal IP address or
// the port is not a literal port number, ResolveTCPAddr resolves the
// address to an address of TCP end point.
// Otherwise, it parses the address as a pair of literal IP address
// and port number.
// The address parameter can use a host name, but this is not
// recommended, because it will return at most one of the host name's
// IP addresses.
//
// See func Dial for a description of the network and address
// parameters.
func ResolveTCPAddr(network, address string) (*TCPAddr, error) {
switch network {
case "tcp", "tcp4":
default:
return nil, fmt.Errorf("Network '%s' not supported", network)
}
// TINYGO: Use netdev resolver
host, sport, err := SplitHostPort(address)
if err != nil {
return nil, err
}
port, err := strconv.Atoi(sport)
if err != nil {
return nil, fmt.Errorf("Error parsing port '%s' in address: %s",
sport, err)
}
if host == "" {
return &TCPAddr{Port: port}, nil
}
ip, err := netdev.GetHostByName(host)
if err != nil {
return nil, fmt.Errorf("Lookup of host name '%s' failed: %s", host, err)
}
return &TCPAddr{IP: ip, Port: port}, nil
}
// TCPConn is an implementation of the Conn interface for TCP network
// connections.
type TCPConn struct {
conn
fd int
laddr *TCPAddr
raddr *TCPAddr
readDeadline time.Time
writeDeadline time.Time
}
// DialTCP acts like Dial for TCP networks.
//
// The network must be a TCP network name; see func Dial for details.
//
// If laddr is nil, a local address is automatically chosen.
// If the IP field of raddr is nil or an unspecified IP address, the
// local system is assumed.
func DialTCP(network string, laddr, raddr *TCPAddr) (*TCPConn, error) {
switch network {
case "tcp", "tcp4":
default:
return nil, fmt.Errorf("Network '%s' not supported", network)
}
// TINYGO: Use netdev to create TCP socket and connect
if raddr == nil {
raddr = &TCPAddr{}
}
if raddr.IP.IsUnspecified() {
return nil, fmt.Errorf("Sorry, localhost isn't available on Tinygo")
}
fd, err := netdev.Socket(syscall.AF_INET, syscall.SOCK_STREAM, syscall.IPPROTO_TCP)
if err != nil {
return nil, err
}
if err = netdev.Connect(fd, "", raddr.IP, raddr.Port); err != nil {
netdev.Close(fd)
return nil, err
}
return &TCPConn{
fd: fd,
laddr: laddr,
raddr: raddr,
}, nil
}
// TINYGO: Use netdev for Conn methods: Read = Recv, Write = Send, etc.
func (c *TCPConn) Read(b []byte) (int, error) {
var timeout time.Duration
now := time.Now()
if !c.readDeadline.IsZero() {
if c.readDeadline.Before(now) {
return 0, fmt.Errorf("Read deadline expired")
} else {
timeout = c.readDeadline.Sub(now)
}
}
n, err := netdev.Recv(c.fd, b, 0, timeout)
// Turn the -1 socket error into 0 and let err speak for error
if n < 0 {
n = 0
}
return n, err
}
func (c *TCPConn) Write(b []byte) (int, error) {
var timeout time.Duration
now := time.Now()
if !c.writeDeadline.IsZero() {
if c.writeDeadline.Before(now) {
return 0, fmt.Errorf("Write deadline expired")
} else {
timeout = c.writeDeadline.Sub(now)
}
}
n, err := netdev.Send(c.fd, b, 0, timeout)
// Turn the -1 socket error into 0 and let err speak for error
if n < 0 {
n = 0
}
return n, err
}
func (c *TCPConn) Close() error {
return netdev.Close(c.fd)
}
func (c *TCPConn) LocalAddr() Addr {
return c.laddr
}
func (c *TCPConn) RemoteAddr() Addr {
return c.raddr
}
func (c *TCPConn) SetDeadline(t time.Time) error {
c.readDeadline = t
c.writeDeadline = t
return nil
}
func (c *TCPConn) SetKeepAlive(keepalive bool) error {
return netdev.SetSockOpt(c.fd, syscall.SOL_SOCKET, syscall.SO_KEEPALIVE, keepalive)
}
func (c *TCPConn) SetKeepAlivePeriod(d time.Duration) error {
// Units are 1/2 seconds
return netdev.SetSockOpt(c.fd, syscall.SOL_TCP, syscall.TCP_KEEPINTVL, 2*d.Seconds())
}
func (c *TCPConn) SetReadDeadline(t time.Time) error {
c.readDeadline = t
return nil
}
func (c *TCPConn) SetWriteDeadline(t time.Time) error {
c.writeDeadline = t
return nil
}
func (c *TCPConn) CloseWrite() error {
return &OpError{"close", "", nil, nil, ErrNotImplemented}
return fmt.Errorf("CloseWrite not implemented")
}
type listener struct {
fd int
laddr *TCPAddr
}
func (l *listener) Accept() (Conn, error) {
fd, err := netdev.Accept(l.fd, IP{}, 0)
if err != nil {
return nil, err
}
return &TCPConn{
fd: fd,
laddr: l.laddr,
}, nil
}
func (l *listener) Close() error {
return netdev.Close(l.fd)
}
func (l *listener) Addr() Addr {
return l.laddr
}
func listenTCP(laddr *TCPAddr) (Listener, error) {
fd, err := netdev.Socket(syscall.AF_INET, syscall.SOCK_STREAM, syscall.IPPROTO_TCP)
if err != nil {
return nil, err
}
err = netdev.Bind(fd, laddr.IP, laddr.Port)
if err != nil {
return nil, err
}
err = netdev.Listen(fd, 5)
if err != nil {
return nil, err
}
return &listener{fd: fd, laddr: laddr}, nil
}
tlssock.go
+154
View File
@@ -0,0 +1,154 @@
// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// TLS low level connection and record layer
package net
import (
"fmt"
"strconv"
"syscall"
"time"
)
func DialTLS(addr string) (*TLSConn, error) {
host, sport, err := SplitHostPort(addr)
if err != nil {
return nil, err
}
port, err := strconv.Atoi(sport)
if err != nil {
return nil, err
}
if port == 0 {
port = 443
}
fd, err := netdev.Socket(syscall.AF_INET, syscall.SOCK_STREAM, syscall.IPPROTO_TLS)
if err != nil {
return nil, err
}
if err = netdev.Connect(fd, host, IP{}, port); err != nil {
netdev.Close(fd)
return nil, err
}
return &TLSConn{
fd: fd,
}, nil
}
// A TLSConn represents a secured connection.
// It implements the net.Conn interface.
type TLSConn struct {
fd int
readDeadline time.Time
writeDeadline time.Time
}
// Access to net.Conn methods.
// Cannot just embed net.Conn because that would
// export the struct field too.
// LocalAddr returns the local network address.
func (c *TLSConn) LocalAddr() Addr {
// TODO
return nil
}
// RemoteAddr returns the remote network address.
func (c *TLSConn) RemoteAddr() Addr {
// TODO
return nil
}
// SetDeadline sets the read and write deadlines associated with the connection.
// A zero value for t means Read and Write will not time out.
// After a Write has timed out, the TLS state is corrupt and all future writes will return the same error.
func (c *TLSConn) SetDeadline(t time.Time) error {
c.readDeadline = t
c.writeDeadline = t
return nil
}
// SetReadDeadline sets the read deadline on the underlying connection.
// A zero value for t means Read will not time out.
func (c *TLSConn) SetReadDeadline(t time.Time) error {
c.readDeadline = t
return nil
}
// SetWriteDeadline sets the write deadline on the underlying connection.
// A zero value for t means Write will not time out.
// After a Write has timed out, the TLS state is corrupt and all future writes will return the same error.
func (c *TLSConn) SetWriteDeadline(t time.Time) error {
c.writeDeadline = t
return nil
}
func (c *TLSConn) Read(b []byte) (int, error) {
var timeout time.Duration
now := time.Now()
if !c.readDeadline.IsZero() {
if c.readDeadline.Before(now) {
return 0, fmt.Errorf("Read deadline expired")
} else {
timeout = c.readDeadline.Sub(now)
}
}
n, err := netdev.Recv(c.fd, b, 0, timeout)
// Turn the -1 socket error into 0 and let err speak for error
if n < 0 {
n = 0
}
return n, err
}
func (c *TLSConn) Write(b []byte) (int, error) {
var timeout time.Duration
now := time.Now()
if !c.writeDeadline.IsZero() {
if c.writeDeadline.Before(now) {
return 0, fmt.Errorf("Write deadline expired")
} else {
timeout = c.writeDeadline.Sub(now)
}
}
n, err := netdev.Send(c.fd, b, 0, timeout)
// Turn the -1 socket error into 0 and let err speak for error
if n < 0 {
n = 0
}
return n, err
}
func (c *TLSConn) Close() error {
return netdev.Close(c.fd)
}
// Handshake runs the client or server handshake
// protocol if it has not yet been run.
//
// Most uses of this package need not call Handshake explicitly: the
// first Read or Write will call it automatically.
//
// For control over canceling or setting a timeout on a handshake, use
// HandshakeContext or the Dialer's DialContext method instead.
func (c *TLSConn) Handshake() error {
panic("TLSConn.Handshake() not implemented")
return nil
}
udpsock.go
+211
View File
@@ -1,8 +1,18 @@
// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package net
import (
"fmt"
"internal/itoa"
"net/netip"
"strconv"
"syscall"
"time"
)
// UDPAddr represents the address of a UDP end point.
@@ -53,3 +63,204 @@ func (a *UDPAddr) opAddr() Addr {
}
return a
}
// ResolveUDPAddr returns an address of UDP end point.
//
// The network must be a UDP network name.
//
// If the host in the address parameter is not a literal IP address or
// the port is not a literal port number, ResolveUDPAddr resolves the
// address to an address of UDP end point.
// Otherwise, it parses the address as a pair of literal IP address
// and port number.
// The address parameter can use a host name, but this is not
// recommended, because it will return at most one of the host name's
// IP addresses.
//
// See func Dial for a description of the network and address
// parameters.
func ResolveUDPAddr(network, address string) (*UDPAddr, error) {
switch network {
case "udp", "udp4":
default:
return nil, fmt.Errorf("Network '%s' not supported", network)
}
// TINYGO: Use netdev resolver
host, sport, err := SplitHostPort(address)
if err != nil {
return nil, err
}
port, err := strconv.Atoi(sport)
if err != nil {
return nil, fmt.Errorf("Error parsing port '%s' in address: %s",
sport, err)
}
if host == "" {
return &UDPAddr{Port: port}, nil
}
ip, err := netdev.GetHostByName(host)
if err != nil {
return nil, fmt.Errorf("Lookup of host name '%s' failed: %s", host, err)
}
return &UDPAddr{IP: ip, Port: port}, nil
}
// UDPConn is the implementation of the Conn and PacketConn interfaces
// for UDP network connections.
type UDPConn struct {
fd int
laddr *UDPAddr
raddr *UDPAddr
readDeadline time.Time
writeDeadline time.Time
}
// Use IANA RFC 6335 port range 49152–65535 for ephemeral (dynamic) ports
var eport = int32(49151)
func ephemeralPort() int {
// TODO: this is racy, if concurrent DialUDPs; use atomic?
if eport == int32(65535) {
eport = int32(49151)
} else {
eport++
}
return int(eport)
}
// DialUDP acts like Dial for UDP networks.
//
// The network must be a UDP network name; see func Dial for details.
//
// If laddr is nil, a local address is automatically chosen.
// If the IP field of raddr is nil or an unspecified IP address, the
// local system is assumed.
func DialUDP(network string, laddr, raddr *UDPAddr) (*UDPConn, error) {
switch network {
case "udp", "udp4":
default:
return nil, fmt.Errorf("Network '%s' not supported", network)
}
// TINYGO: Use netdev to create UDP socket and connect
if laddr == nil {
laddr = &UDPAddr{}
}
if raddr == nil {
raddr = &UDPAddr{}
}
if raddr.IP.IsUnspecified() {
return nil, fmt.Errorf("Sorry, localhost isn't available on Tinygo")
}
// If no port was given, grab an ephemeral port
if laddr.Port == 0 {
laddr.Port = ephemeralPort()
}
fd, err := netdev.Socket(syscall.AF_INET, syscall.SOCK_DGRAM, syscall.IPPROTO_UDP)
if err != nil {
return nil, err
}
// Local bind
err = netdev.Bind(fd, laddr.IP, laddr.Port)
if err != nil {
netdev.Close(fd)
return nil, err
}
// Remote connect
if err = netdev.Connect(fd, "", raddr.IP, raddr.Port); err != nil {
netdev.Close(fd)
return nil, err
}
return &UDPConn{
fd: fd,
laddr: laddr,
raddr: raddr,
}, nil
}
// TINYGO: Use netdev for Conn methods: Read = Recv, Write = Send, etc.
func (c *UDPConn) Read(b []byte) (int, error) {
var timeout time.Duration
now := time.Now()
if !c.readDeadline.IsZero() {
if c.readDeadline.Before(now) {
return 0, fmt.Errorf("Read deadline expired")
} else {
timeout = c.readDeadline.Sub(now)
}
}
n, err := netdev.Recv(c.fd, b, 0, timeout)
// Turn the -1 socket error into 0 and let err speak for error
if n < 0 {
n = 0
}
return n, err
}
func (c *UDPConn) Write(b []byte) (int, error) {
var timeout time.Duration
now := time.Now()
if !c.writeDeadline.IsZero() {
if c.writeDeadline.Before(now) {
return 0, fmt.Errorf("Write deadline expired")
} else {
timeout = c.writeDeadline.Sub(now)
}
}
n, err := netdev.Send(c.fd, b, 0, timeout)
// Turn the -1 socket error into 0 and let err speak for error
if n < 0 {
n = 0
}
return n, err
}
func (c *UDPConn) Close() error {
return netdev.Close(c.fd)
}
func (c *UDPConn) LocalAddr() Addr {
return c.laddr
}
func (c *UDPConn) RemoteAddr() Addr {
return c.raddr
}
func (c *UDPConn) SetDeadline(t time.Time) error {
c.readDeadline = t
c.writeDeadline = t
return nil
}
func (c *UDPConn) SetReadDeadline(t time.Time) error {
c.readDeadline = t
return nil
}
func (c *UDPConn) SetWriteDeadline(t time.Time) error {
c.writeDeadline = t
return nil
}
writev_test.go
-132
View File
@@ -1,132 +0,0 @@
// The following is copied from Go 1.17 official implementation and
// modified to accommodate TinyGo.
// Copyright 2016 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package net
import (
"bytes"
"fmt"
"io"
"reflect"
"testing"
)
func TestBuffers_read(t *testing.T) {
const story = "once upon a time in Gopherland ... "
buffers := Buffers{
[]byte("once "),
[]byte("upon "),
[]byte("a "),
[]byte("time "),
[]byte("in "),
[]byte("Gopherland ... "),
}
got, err := io.ReadAll(&buffers)
if err != nil {
t.Fatal(err)
}
if string(got) != story {
t.Errorf("read %q; want %q", got, story)
}
if len(buffers) != 0 {
t.Errorf("len(buffers) = %d; want 0", len(buffers))
}
}
func TestBuffers_consume(t *testing.T) {
tests := []struct {
in Buffers
consume int64
want Buffers
}{
{
in: Buffers{[]byte("foo"), []byte("bar")},
consume: 0,
want: Buffers{[]byte("foo"), []byte("bar")},
},
{
in: Buffers{[]byte("foo"), []byte("bar")},
consume: 2,
want: Buffers{[]byte("o"), []byte("bar")},
},
{
in: Buffers{[]byte("foo"), []byte("bar")},
consume: 3,
want: Buffers{[]byte("bar")},
},
{
in: Buffers{[]byte("foo"), []byte("bar")},
consume: 4,
want: Buffers{[]byte("ar")},
},
{
in: Buffers{nil, nil, nil, []byte("bar")},
consume: 1,
want: Buffers{[]byte("ar")},
},
{
in: Buffers{nil, nil, nil, []byte("foo")},
consume: 0,
want: Buffers{[]byte("foo")},
},
{
in: Buffers{nil, nil, nil},
consume: 0,
want: Buffers{},
},
}
for i, tt := range tests {
in := tt.in
in.consume(tt.consume)
if !reflect.DeepEqual(in, tt.want) {
t.Errorf("%d. after consume(%d) = %+v, want %+v", i, tt.consume, in, tt.want)
}
}
}
func TestBuffers_WriteTo(t *testing.T) {
for _, name := range []string{"WriteTo", "Copy"} {
for _, size := range []int{0, 10, 1023, 1024, 1025} {
t.Run(fmt.Sprintf("%s/%d", name, size), func(t *testing.T) {
testBuffer_writeTo(t, size, name == "Copy")
})
}
}
}
func testBuffer_writeTo(t *testing.T, chunks int, useCopy bool) {
var want bytes.Buffer
for i := 0; i < chunks; i++ {
want.WriteByte(byte(i))
}
var b bytes.Buffer
buffers := make(Buffers, chunks)
for i := range buffers {
buffers[i] = want.Bytes()[i : i+1]
}
var n int64
var err error
if useCopy {
n, err = io.Copy(&b, &buffers)
} else {
n, err = buffers.WriteTo(&b)
}
if err != nil {
t.Fatal(err)
}
if len(buffers) != 0 {
t.Fatal(fmt.Errorf("len(buffers) = %d; want 0", len(buffers)))
}
if n != int64(want.Len()) {
t.Fatal(fmt.Errorf("Buffers.WriteTo returned %d; want %d", n, want.Len()))
}
all, err := io.ReadAll(&b)
if !bytes.Equal(all, want.Bytes()) || err != nil {
t.Fatal(fmt.Errorf("read %q, %v; want %q, nil", all, err, want.Bytes()))
}
}