Rez Moss for AWS Community Builders

Posted on May 13

Inside AWS S3 API Calls: Creating a Go-Based HTTPS Traffic Inspector

#aws #security #go

Ever wondered what HTTP requests your command-line tools are actually making? When you run aws s3 ls or curl https://api.example.com, what's really happening under the hood? In this article, we'll build a tool that reveals all HTTP and HTTPS traffic by creating our own intercepting proxy in Go.

We're going to create a command-line tool that:

Intercepts HTTP and HTTPS requests from any command
Decrypts HTTPS traffic to show the actual content
Displays formatted request and response details
Works transparently with tools like curl, AWS CLI, and others

The final usage will be simple:

./httpmon curl https://api.github.com
./httpmon aws s3 ls my-bucket

Understanding HTTP Proxies

Before we start coding, let's understand how HTTP proxies work. When a client (like curl) wants to make an HTTP request through a proxy:

The client connects to the proxy instead of the target server
For HTTP: The client sends the full request to the proxy, which forwards it
For HTTPS: The client sends a CONNECT request to establish a tunnel
The proxy forwards requests and responses between client and server

The challenge with HTTPS is that it's encrypted end-to-end. To see the actual content, we need to perform what's called a "Man-in-the-Middle" (MITM) attack - but in this case, it's intentional and for debugging purposes.

Let's begin with a minimal HTTP proxy that can handle unencrypted traffic:

package main

import (
    "fmt"
    "io"
    "net/http"
    "os"
    "os/exec"
    "strings"
    "time"
)

func proxyHandler(w http.ResponseWriter, r *http.Request) {
    fmt.Printf("\n=== REQUEST ===\n")
    fmt.Printf("%s %s\n", r.Method, r.URL.String())

    http.Error(w, "Not implemented", http.StatusNotImplemented)
}

func main() {
    if len(os.Args) < 2 {
        fmt.Println("Usage: httpmon <command> [args...]")
        os.Exit(1)
    }

    proxyPort := "8080"
    go func() {
        fmt.Printf("Starting proxy on :%s\n", proxyPort)
        http.ListenAndServe(":"+proxyPort, http.HandlerFunc(proxyHandler))
    }()

    time.Sleep(100 * time.Millisecond)

    cmdArgs := os.Args[1:]
    cmd := exec.Command(cmdArgs[0], cmdArgs[1:]...)

    proxyURL := "http://localhost:" + proxyPort
    cmd.Env = append(os.Environ(),
        "HTTP_PROXY="+proxyURL,
        "HTTPS_PROXY="+proxyURL,
    )

    cmd.Stdout = os.Stdout
    cmd.Stderr = os.Stderr
    cmd.Stdin = os.Stdin

    fmt.Printf("Running: %s\n", strings.Join(cmdArgs, " "))
    cmd.Run()
}

This basic version:

Starts a proxy server on port 8080
Sets environment variables to route traffic through our proxy
Runs the specified command
Currently just logs requests and returns an error

If you build and run this with ./httpmon curl http://example.com, you'll see it captures the request but doesn't forward it yet.

Forwarding HTTP Requests

Now let's make our proxy actually forward HTTP requests and log both requests and responses. We'll use Go's httputil.ReverseProxy to handle the heavy lifting:

package main

import (
    "bytes"
    "fmt"
    "io"
    "net/http"
    "net/http/httputil"
    "os"
    "os/exec"
    "strings"
    "time"
)

type LoggingTransport struct{}

func (t *LoggingTransport) RoundTrip(req *http.Request) (*http.Response, error) {
    fmt.Printf("\n=== REQUEST ===\n")
    fmt.Printf("%s %s %s\n", req.Method, req.URL.String(), req.Proto)
    fmt.Printf("Host: %s\n", req.Host)
    fmt.Println("\nHeaders:")
    for k, v := range req.Header {
        fmt.Printf("  %s: %s\n", k, strings.Join(v, ", "))
    }

    var bodyBytes []byte
    if req.Body != nil {
        bodyBytes, _ = io.ReadAll(req.Body)
        req.Body = io.NopCloser(bytes.NewReader(bodyBytes))
        if len(bodyBytes) > 0 {
            fmt.Printf("\nBody:\n%s\n", string(bodyBytes))
        }
    }

    transport := &http.Transport{}
    resp, err := transport.RoundTrip(req)
    if err != nil {
        fmt.Printf("\nERROR: %v\n", err)
        return nil, err
    }

    fmt.Printf("\n=== RESPONSE ===\n")
    fmt.Printf("%s %s\n", resp.Proto, resp.Status)
    fmt.Println("\nHeaders:")
    for k, v := range resp.Header {
        fmt.Printf("  %s: %s\n", k, strings.Join(v, ", "))
    }

    respBody, _ := io.ReadAll(resp.Body)
    resp.Body = io.NopCloser(bytes.NewReader(respBody))

    if len(respBody) > 0 {
        fmt.Printf("\nBody:\n%s\n", string(respBody))
    }

    fmt.Println("\n" + strings.Repeat("-", 60))

    return resp, nil
}

func proxyHandler(w http.ResponseWriter, r *http.Request) {
    proxy := &httputil.ReverseProxy{
        Director: func(req *http.Request) {
            if req.URL.Scheme == "" {
                req.URL.Scheme = "http"
            }
            if req.URL.Host == "" {
                req.URL.Host = req.Host
            }
        },
        Transport: &LoggingTransport{},
    }
    proxy.ServeHTTP(w, r)
}

This code:

LoggingTransport: A custom transport that intercepts and logs HTTP transactions
ReverseProxy: Handles the actual proxying of requests
Request/Response logging: Shows headers and bodies for both directions

The RoundTrip method is where the magic happens - it's called for every HTTP request, giving us a chance to log everything before and after the actual network call.

Now if you run ./httpmon curl http://httpbin.org/get, you'll see both the request curl makes and the response it receives, with all headers and body content visible.

Handling HTTPS CONNECT Requests

HTTPS requests work differently. When a client wants to make an HTTPS request through a proxy, it first sends a CONNECT request to establish a tunnel. Let's add support for this:

func handleConnect(w http.ResponseWriter, r *http.Request) {
    fmt.Printf("\n=== CONNECT %s ===\n\n", r.Host)

    targetConn, err := net.Dial("tcp", r.Host)
    if err != nil {
        http.Error(w, err.Error(), http.StatusServiceUnavailable)
        return
    }
    defer targetConn.Close()

    w.WriteHeader(http.StatusOK)

    hijacker, ok := w.(http.Hijacker)
    if !ok {
        http.Error(w, "Hijacking not supported", http.StatusInternalServerError)
        return
    }

    clientConn, _, err := hijacker.Hijack()
    if err != nil {
        http.Error(w, err.Error(), http.StatusServiceUnavailable)
        return
    }
    defer clientConn.Close()

    go io.Copy(targetConn, clientConn)
    go io.Copy(clientConn, targetConn)
}

func proxyHandler(w http.ResponseWriter, r *http.Request) {
    if r.Method == http.MethodConnect {
        handleConnect(w, r)
    } else {

    }
}

This code:

Detects CONNECT requests (used for HTTPS)
Establishes a TCP connection to the target server
"Hijacks" the HTTP connection to get raw TCP access
Creates a bidirectional tunnel between client and server

However, with this approach, we only see the CONNECT request - the actual HTTPS content remains encrypted. To decrypt HTTPS traffic, we need to perform TLS termination.

Setting Up MITM for HTTPS

To decrypt HTTPS traffic, we need to:

Generate our own Certificate Authority (CA)
Create certificates on-the-fly for each HTTPS host
Perform TLS termination and re-encryption

First, let's add certificate generation:

import (
    "crypto/rand"
    "crypto/rsa"
    "crypto/tls"
    "crypto/x509"
    "crypto/x509/pkix"
    "math/big"
)

var (
    caCert *x509.Certificate
    caKey  *rsa.PrivateKey
)

func init() {
    var err error
    caCert, caKey, err = generateCA()
    if err != nil {
        log.Fatal("Failed to generate CA:", err)
    }
}

func generateCA() (*x509.Certificate, *rsa.PrivateKey, error) {
    key, err := rsa.GenerateKey(rand.Reader, 2048)
    if err != nil {
        return nil, nil, err
    }

    template := x509.Certificate{
        SerialNumber: big.NewInt(1),
        Subject: pkix.Name{
            Organization: []string{"HTTP Monitor CA"},
        },
        NotBefore:             time.Now(),
        NotAfter:              time.Now().AddDate(10, 0, 0),
        KeyUsage:              x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
        ExtKeyUsage:           []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
        BasicConstraintsValid: true,
        IsCA:                  true,
    }

    certDER, err := x509.CreateCertificate(rand.Reader, &template, &template, &key.PublicKey, key)
    if err != nil {
        return nil, nil, err
    }

    cert, err := x509.ParseCertificate(certDER)
    if err != nil {
        return nil, nil, err
    }

    return cert, key, nil
}

This creates a self-signed CA that we'll use to sign certificates for each HTTPS domain. The CA is generated once when the program starts and stored in memory.

Intercepting and Decrypting HTTPS Traffic

Now let's implement the actual HTTPS interception. We'll modify our CONNECT handler to perform TLS termination:

func generateCert(host string) (*tls.Certificate, error) {
    key, err := rsa.GenerateKey(rand.Reader, 2048)
    if err != nil {
        return nil, err
    }

    template := x509.Certificate{
        SerialNumber: big.NewInt(1),
        Subject: pkix.Name{
            Organization: []string{"HTTP Monitor"},
        },
        NotBefore:    time.Now(),
        NotAfter:     time.Now().AddDate(1, 0, 0),
        KeyUsage:     x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
        ExtKeyUsage:  []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
        DNSNames:     []string{host},
    }

    certDER, err := x509.CreateCertificate(rand.Reader, &template, caCert, &key.PublicKey, caKey)
    if err != nil {
        return nil, err
    }

    cert := &tls.Certificate{
        Certificate: [][]byte{certDER},
        PrivateKey:  key,
    }

    return cert, nil
}

func handleConnect(w http.ResponseWriter, r *http.Request) {
    fmt.Printf("\n=== CONNECT %s ===\n\n", r.Host)

    host, _, err := net.SplitHostPort(r.Host)
    if err != nil {
        host = r.Host
    }

    cert, err := generateCert(host)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    w.WriteHeader(http.StatusOK)

    hijacker, ok := w.(http.Hijacker)
    if !ok {
        http.Error(w, "Hijacking not supported", http.StatusInternalServerError)
        return
    }

    clientConn, _, err := hijacker.Hijack()
    if err != nil {
        http.Error(w, err.Error(), http.StatusServiceUnavailable)
        return
    }
    defer clientConn.Close()

    tlsConfig := &tls.Config{
        Certificates: []tls.Certificate{*cert},
    }
    tlsConn := tls.Server(clientConn, tlsConfig)
    defer tlsConn.Close()

    reader := bufio.NewReader(tlsConn)

    for {
        req, err := http.ReadRequest(reader)
        if err != nil {
            if err != io.EOF {
                fmt.Printf("Error reading request: %v\n", err)
            }
            break
        }

        req.URL.Scheme = "https"
        req.URL.Host = r.Host
        req.RequestURI = ""

        logRequest(req)

        client := &http.Client{
            Transport: &http.Transport{
                TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
            },
        }

        resp, err := client.Do(req)
        if err != nil {
            fmt.Printf("Error making request: %v\n", err)
            continue
        }

        logResponse(resp)

        resp.Write(tlsConn)
        resp.Body.Close()
    }
}

Key concepts here:

Certificate Generation: We create a certificate for each domain, signed by our CA
TLS Server: We establish a TLS connection with the client using our certificate
Request Loop: We read HTTP requests from the TLS connection, make real HTTPS requests, and forward responses
Transparent Proxying: The client thinks it's talking to the real server

The flow is:

Client → [TLS with our cert] → Our Proxy → [TLS with real cert] → Server

This allows us to see decrypted HTTPS traffic in both directions.

Adding Curl and AWS CLI Support

Now we need to configure the tools to use our proxy and trust our CA certificate. Different tools require different approaches:

func main() {
    if len(os.Args) < 2 {
        fmt.Println("Usage: httpmon <command> [args...]")
        os.Exit(1)
    }

    caCertPEM := &bytes.Buffer{}
    pem.Encode(caCertPEM, &pem.Block{
        Type:  "CERTIFICATE",
        Bytes: caCert.Raw,
    })

    caCertFile := "/tmp/httpmon-ca.crt"
    err := os.WriteFile(caCertFile, caCertPEM.Bytes(), 0644)
    if err != nil {
        log.Fatal("Failed to write CA cert:", err)
    }

    go func() {
        fmt.Printf("Starting MITM proxy on :%s\n", proxyPort)
        fmt.Printf("CA certificate written to: %s\n", caCertFile)
        http.ListenAndServe(":"+proxyPort, http.HandlerFunc(proxyHandler))
    }()

    time.Sleep(100 * time.Millisecond)

    cmdArgs := os.Args[1:]

    if strings.Contains(cmdArgs[0], "curl") {
        hasProxy := false
        hasCACert := false

        for _, arg := range cmdArgs {
            if arg == "-x" || arg == "--proxy" {
                hasProxy = true
            }
            if arg == "--cacert" {
                hasCACert = true
            }
        }

        newArgs := []string{cmdArgs[0]}
        if !hasProxy {
            newArgs = append(newArgs, "-x", "http://localhost:"+proxyPort)
        }
        if !hasCACert {
            newArgs = append(newArgs, "--cacert", caCertFile)
        }
        newArgs = append(newArgs, cmdArgs[1:]...)
        cmdArgs = newArgs
    }

    cmd := exec.Command(cmdArgs[0], cmdArgs[1:]...)

    proxyURL := "http://localhost:" + proxyPort
    cmd.Env = append(os.Environ(),
        "HTTP_PROXY="+proxyURL,
        "HTTPS_PROXY="+proxyURL,
        "http_proxy="+proxyURL,
        "https_proxy="+proxyURL,
    )

    if strings.Contains(cmdArgs[0], "aws") {
        cmd.Env = append(cmd.Env, "AWS_CA_BUNDLE="+caCertFile)
    }

    cmd.Stdout = os.Stdout
    cmd.Stderr = os.Stderr
    cmd.Stdin = os.Stdin

    fmt.Printf("\nRunning: %s\n", strings.Join(cmdArgs, " "))
    fmt.Println(strings.Repeat("=", 60))

    cmd.Run()
}

This handles:

CA Certificate Export: Saves our CA cert to /tmp/httpmon-ca.crt
Curl Configuration: Automatically adds -x (proxy) and --cacert flags
AWS CLI Configuration: Sets AWS_CA_BUNDLE environment variable
Generic Proxy Setup: Sets standard proxy environment variables

Understanding AWS S3 Requests

When you run aws s3 ls bucket-name, here's what actually happens:

CONNECT Request: AWS CLI establishes HTTPS tunnel to s3.us-east-1.amazonaws.com
ListObjectsV2 API Call: Makes a GET request with specific parameters
Authentication: Uses AWS Signature Version 4 with your credentials

The actual request looks like:

GET /bucket-name?list-type=2&prefix=&delimiter=%2F&encoding-type=url
Host: s3.us-east-1.amazonaws.com
Authorization: AWS4-HMAC-SHA256 Credential=...
X-Amz-Date: 20250513T184908Z
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

example

./httpmon aws s3 ls mybucket1
Starting MITM proxy on :8080
CA certificate written to: /tmp/httpmon-ca.crt

Running: aws s3 ls mybucket1
============================================================

=== CONNECT s3.us-east-1.amazonaws.com:443 ===


=== REQUEST #1 ===
Time: 15:06:42
GET https://s3.us-east-1.amazonaws.com:443/mybucket1?list-type=2&prefix=&delimiter=%2F&encoding-type=url HTTP/1.1
Host: s3.us-east-1.amazonaws.com
S3 Bucket: mybucket1

Query Parameters:
  delimiter: /
  encoding-type: url
  list-type: 2
  prefix: 

Headers:
  Accept-Encoding: identity
  User-Agent: aws-cli/2.9.14 Python/3.9.11 Darwin/24.5.0 exe/x86_64 prompt/off command/s3.ls
  X-Amz-Date: 20250513T190642Z
  X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
  Authorization: AWS4-HMAC-SHA256 Credential=XADSFDSFGDSPZRJWH/20250513/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=0cfd83a5c8dc7b2fe56de3408d23f65ba81a1ad106fd620b6425a7e21c283a7d


=== RESPONSE ===
HTTP/1.1 200 OK

Headers:
  X-Amz-Id-2: w48pC+YLNpgK3Lv9nVP3KDzneks+XXdcwLzQW6SEinz3ggPJCvs7SPdgXV5cfDx9QPGjzUvVfO8=
  X-Amz-Request-Id: CX6A2DCCY1701S6Q
  Date: Tue, 13 May 2025 19:06:43 GMT
  X-Amz-Bucket-Region: us-east-1
  Content-Type: application/xml
  Server: AmazonS3

Body:
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>mybucket1</Name><Prefix></Prefix><KeyCount>3</KeyCount><MaxKeys>1000</MaxKeys><Delimiter>/</Delimiter><EncodingType>url</EncodingType><IsTruncated>false</IsTruncated><Contents><Key>index.html</Key><LastModified>2022-12-23T00:36:07.000Z</LastModified><ETag>&quot;5e88427e5bf9dc71b4e1a947ef1c70b3&quot;</ETag><Size>30</Size><StorageClass>STANDARD</StorageClass></Contents><CommonPrefixes><Prefix>assetes/</Prefix></CommonPrefixes><CommonPrefixes><Prefix>images/</Prefix></CommonPrefixes></ListBucketResult>

------------------------------------------------------------
                           PRE assetes/
                           PRE images/
2022-12-22 19:36:07         30 index.html

Key points:

The bucket name is in the URL path, not the hostname
list-type=2 specifies the ListObjectsV2 API
delimiter=/ groups results by "folders"
The X-Amz-Content-Sha256 header contains a hash of the (empty) request body
Authorization header contains the AWS signature

The response is XML containing the bucket contents, which AWS CLI then formats into the familiar listing.

curl example

./httpmon curl https://api.x.com/2/users                                             
Starting MITM proxy on :8080
CA certificate written to: /tmp/httpmon-ca.crt

Running: curl -x http://localhost:8080 --cacert /tmp/httpmon-ca.crt https://api.x.com/2/users
============================================================

=== CONNECT api.x.com:443 ===


=== REQUEST #1 ===
Time: 15:30:11
GET https://api.x.com:443/2/users HTTP/1.1
Host: api.x.com

Headers:
  User-Agent: curl/8.7.1
  Accept: */*


=== RESPONSE ===
HTTP/1.1 401 Unauthorized

Headers:
  Content-Length: 99
  X-Response-Time: 1
  X-Connection-Hash: 2880da004170bfc7797bc51e1d8b94bb27fb00798f03346f480dd37d4a260c41
  Set-Cookie: __cf_bm=kdW6u1gST38Rz8j76Ic.oYGItlWU_3neZwgf8ucpTrU-1747164611-1.0.1.1-k1eAKakMJcMnzW3byJJ9olylAnQXVZv5IfQTvHjmXRnwqsKBcEMr6bcv1cdSoBIW_kgsJrthIaWS5JG_SPPyFBnFYLj.BHQUo9kQA3WOmHU; path=/; expires=Tue, 13-May-25 20:00:11 GMT; domain=.x.com; HttpOnly; Secure; SameSite=None
  Date: Tue, 13 May 2025 19:30:11 GMT
  Perf: 7402827104
  X-Transaction-Id: 8bc4130c588fa097
  Server: cloudflare tsa_b
  Content-Type: application/problem+json
  Strict-Transport-Security: max-age=631138519
  Cf-Cache-Status: DYNAMIC
  Connection: keep-alive
  Cache-Control: no-cache, no-store, max-age=0
  Cf-Ray: 93f491a3bdbfa223-YYZ

Body:
{
  "title": "Unauthorized",
  "type": "about:blank",
  "status": 401,
  "detail": "Unauthorized"
}

A bit more

Let's add the finishing touches to make our tool better:

var (
    requestCounter int
    mutex          sync.Mutex
    certCache      = make(map[string]*tls.Certificate)
    certMutex      sync.Mutex
)

func logRequest(req *http.Request) {
    mutex.Lock()
    requestCounter++
    reqID := requestCounter
    mutex.Unlock()

    fmt.Printf("\n\033[36m=== REQUEST #%d ===\033[0m\n", reqID)
    fmt.Printf("Time: %s\n", time.Now().Format("15:04:05"))
    fmt.Printf("%s %s %s\n", req.Method, req.URL.String(), req.Proto)
    fmt.Printf("Host: %s\n", req.Host)

    if strings.Contains(req.Host, ".amazonaws.com") && strings.Contains(req.URL.Path, "/") {
        pathParts := strings.SplitN(req.URL.Path, "/", 3)
        if len(pathParts) >= 2 && pathParts[1] != "" {
            fmt.Printf("\033[93mS3 Bucket: %s\033[0m\n", pathParts[1])
            if len(pathParts) > 2 && pathParts[2] != "" {
                fmt.Printf("\033[93mS3 Key/Prefix: %s\033[0m\n", pathParts[2])
            }
        }
    }

    if req.URL.RawQuery != "" {
        fmt.Println("\nQuery Parameters:")
        params, _ := url.ParseQuery(req.URL.RawQuery)
        for k, v := range params {
            fmt.Printf("  %s: %s\n", k, strings.Join(v, ", "))
        }
    }

    fmt.Println("\nHeaders:")
    for k, v := range req.Header {
        fmt.Printf("  %s: %s\n", k, strings.Join(v, ", "))
    }

    if req.Body != nil {
        body, _ := io.ReadAll(req.Body)
        req.Body = io.NopCloser(bytes.NewReader(body))
        if len(body) > 0 {
            fmt.Printf("\nBody:\n%s\n", string(body))
        }
    }
    fmt.Println()
}

func logResponse(resp *http.Response) {
    fmt.Printf("\n\033[32m=== RESPONSE ===\033[0m\n")
    fmt.Printf("%s %s\n", resp.Proto, resp.Status)

    fmt.Println("\nHeaders:")
    for k, v := range resp.Header {
        fmt.Printf("  %s: %s\n", k, strings.Join(v, ", "))
    }

    if resp.Body != nil {
        body, _ := io.ReadAll(resp.Body)
        resp.Body = io.NopCloser(bytes.NewReader(body))
        if len(body) > 0 {
            fmt.Printf("\nBody:\n")
            if len(body) > 1000 {
                fmt.Printf("%s\n[... %d more bytes ...]\n", string(body[:1000]), len(body)-1000)
            } else {
                fmt.Printf("%s\n", string(body))
            }
        }
    }
    fmt.Println("\n" + strings.Repeat("-", 60))
}

func generateCert(host string) (*tls.Certificate, error) {
    certMutex.Lock()
    if cert, ok := certCache[host]; ok {
        certMutex.Unlock()
        return cert, nil
    }
    certMutex.Unlock()

    key, err := rsa.GenerateKey(rand.Reader, 2048)
    if err != nil {
        return nil, err
    }

    template := x509.Certificate{
        SerialNumber: big.NewInt(1),
        Subject: pkix.Name{
            Organization: []string{"HTTP Monitor"},
        },
        NotBefore:    time.Now(),
        NotAfter:     time.Now().AddDate(1, 0, 0),
        KeyUsage:     x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
        ExtKeyUsage:  []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
        DNSNames:     []string{host, "*." + host},
        IPAddresses:  []net.IP{net.IPv4(127, 0, 0, 1)},
    }

    certDER, err := x509.CreateCertificate(rand.Reader, &template, caCert, &key.PublicKey, caKey)
    if err != nil {
        return nil, err
    }

    cert := &tls.Certificate{
        Certificate: [][]byte{certDER},
        PrivateKey:  key,
    }

    certMutex.Lock()
    certCache[host] = cert
    certMutex.Unlock()

    return cert, nil
}

We've built a powerful HTTP/HTTPS interceptor that:

Captures all HTTP traffic transparently
Decrypts HTTPS traffic using MITM techniques
Works seamlessly with command-line tools
Provides detailed, formatted output for debugging

This tool is invaluable for understanding API interactions, debugging issues, and learning how various tools communicate over HTTP. The complete source code demonstrates Go's powerful networking capabilities and how to build sophisticated proxy servers.

Remember to use this responsibly - only intercept traffic on your own machine for debugging purposes. Never use MITM techniques on networks or systems you don't own.

You can find the complete working implementation of this HTTP/HTTPS interceptor at https://github.com/rezmoss/https-traffic-inspector

Make it make sense

Only get the information you need to fix your code that’s broken with Sentry.

Start debugging →

Top comments (4)

Dotallio • May 14

This is a fantastic deep dive into building a practical tool for inspecting HTTP and HTTPS trafficsuper useful for debugging and understanding what's really happening under the hood! The way you broke down the proxy and MITM concepts, and tailored the tool for both curl and AWS CLI, makes this a go-to resource. Thanks for sharing the code and walking through each step with clear explanations!

Rez Moss • May 14

Thanks!

Big Will • May 15

Awesome writeup. Shows the depth of the Golang language. I live for low level stuff like this

Rez Moss • May 16

Golang gets deep, for sure