Resolving intermittent 404 errors in Chrome on a home server (kubeadm) + SK Broadband environment


Home Server Infrastructure#

1.1 Overall Network Architecture#

flowchart TB
    subgraph Internet["Internet"]
        CF["Cloudflare Edge<br/>(Seoul PoP)"]
        ISP["SK Broadband<br/>Public IP: Dynamic<br/>(39.119.192.15)"]
    end

    subgraph HomeNetwork["Home Network (192.168.45.0/24)"]
        subgraph Router["SK Broadband Router"]
            NAT["NAT/DHCP"]
            FW_R["Firewall"]
            PF["Port Forwarding<br/>:80 → .154:80<br/>:443 → .154:443"]
            DHCP_RES["DHCP Reservation<br/>mini-gmk: .123<br/>mini-might: .154"]
        end

        subgraph ControlPlane["mini-gmk (Control Plane)<br/>192.168.45.123"]
            UFW_C["ufw firewall<br/>22, 6443, 10250-10252<br/>2379-2380, 179"]
            K8S_CP["Kubernetes Control Plane<br/>kube-apiserver<br/>etcd, scheduler<br/>controller-manager"]
            ISTIOD["istiod (pilot)"]
            CERTM["cert-manager"]
            DDNS["ddns-cloudflare<br/>CronJob"]
            ESO["external-secrets-operator"]
            CALICO_CP["calico-typha<br/>calico-kube-controllers"]
        end

        subgraph Worker["mini-might (Worker)<br/>192.168.45.154"]
            UFW_W["ufw firewall<br/>22, 80, 443<br/>10250, 30000-32767"]
            INGRESS["istio-ingressgateway<br/>externalIPs: .154"]
            GW["java-cloud-gateway<br/>:8085"]
            AUTH["auth-guard (x2)<br/>:8080"]
            APPS["order-core :8083<br/>seat :8082<br/>queue :8081"]
            CALICO_W["calico-node"]
        end

        subgraph VPN_Access["Team Member Access"]
            VPN["Tailscale/WireGuard<br/>VPN"]
            SSH["SSH Tunnel<br/>:22"]
        end
    end

    subgraph Clients["Clients"]
        Chrome["Chrome<br/>(QUIC/HTTP3)"]
        Safari["Safari<br/>(HTTP/2)"]
        Firefox["Firefox<br/>(HTTP/2)"]
        Mobile["Mobile LTE"]
        TeamMember["Team Member (VPN)"]
    end

    Chrome -->|"QUIC/UDP:443<br/>❌ intermittent failure"| ISP
    Safari -->|"HTTP/2/TCP:443"| ISP
    Firefox -->|"HTTP/2/TCP:443"| ISP
    Mobile -->|"HTTP/2"| ISP

    Chrome -->|"QUIC/UDP:443<br/>✅ stable"| CF
    CF -->|"HTTP/2/TCP:443"| ISP

    ISP --> NAT
    NAT --> FW_R
    FW_R --> PF
    PF -->|":443"| UFW_W
    UFW_W --> INGRESS

    INGRESS --> GW
    GW --> AUTH
    AUTH --> APPS

    TeamMember -->|"VPN"| VPN
    VPN --> SSH
    SSH --> UFW_C
    SSH --> UFW_W

    DDNS -->|"API call"| CF

    ISTIOD -.->|"xDS"| INGRESS
    CERTM -.->|"TLS cert"| INGRESS
    ESO -.->|"Secret sync"| DDNS

    classDef problem fill:#ff6b6b,stroke:#c0392b,color:#fff
    classDef solution fill:#2ecc71,stroke:#27ae60,color:#fff
    classDef infrastructure fill:#3498db,stroke:#2980b9,color:#fff

    class Chrome problem
    class CF solution
    class Router,ControlPlane,Worker infrastructure

1.2 SK Broadband Router Settings#

flowchart LR
    subgraph SKRouter["SK Broadband Router Settings"]
        direction TB

        subgraph DHCP["DHCP Settings"]
            DHCP_RANGE["DHCP range: 192.168.45.100 ~ .200"]
            DHCP_RES1["Reservation 1: mini-gmk<br/>MAC: XX:XX:XX:XX:XX:XX<br/>IP: 192.168.45.123"]
            DHCP_RES2["Reservation 2: mini-might<br/>MAC: YY:YY:YY:YY:YY:YY<br/>IP: 192.168.45.154"]
        end

        subgraph PortForward["Port Forwarding"]
            PF_HTTP["External :80 → 192.168.45.154:80"]
            PF_HTTPS["External :443 → 192.168.45.154:443"]
            PF_VPN["External :51820 → 192.168.45.123:51820<br/>(WireGuard, optional)"]
        end

        subgraph Firewall["Firewall Settings"]
            FW_IN["Inbound: allow 80, 443"]
            FW_OUT["Outbound: allow all"]
            FW_ICMP["ICMP: allow (ping)"]
        end

        subgraph NAT_Config["NAT Settings"]
            NAT_TYPE["NAT type: Symmetric"]
            NAT_UDP["UDP timeout: 30s (root cause)"]
            NAT_TCP["TCP timeout: 3600s"]
        end
    end

    style NAT_UDP fill:#ff6b6b,stroke:#c0392b,color:#fff

Router settings detail:

SettingValueNote
Router modelSK Broadband defaultipTIME or SK OEM model
Internal IP192.168.45.1Gateway
DHCP range192.168.45.100 ~ .200Static IPs assigned outside range
DNSAuto (SK DNS)168.126.63.1, 168.126.63.2
UDP NAT timeout30s (estimated)Root cause of QUIC issue
TCP NAT timeout3600sHTTP/2 is stable

1.3 DHCP Reservation (Fixed IPs)#

# Configure DHCP reservations in the router admin panel
# Or configure static IPs on each node

# mini-gmk (Control Plane)
# /etc/netplan/00-installer-config.yaml (Ubuntu 22.04)
network:
  version: 2
  ethernets:
    enp2s0:
      addresses:
        - 192.168.45.123/24
      routes:
        - to: default
          via: 192.168.45.1
      nameservers:
        addresses:
          - 168.126.63.1
          - 8.8.8.8

# mini-might (Worker)
network:
  version: 2
  ethernets:
    enp3s0:
      addresses:
        - 192.168.45.154/24
      routes:
        - to: default
          via: 192.168.45.1
      nameservers:
        addresses:
          - 168.126.63.1
          - 8.8.8.8

1.4 Linux Firewall (ufw) Settings#

flowchart TB
    subgraph UFW_Rules["ufw Firewall Rules"]
        direction LR

        subgraph MiniGMK["mini-gmk (Control Plane)"]
            direction TB
            UFW_G1["22/tcp - SSH"]
            UFW_G2["6443/tcp - K8s API Server"]
            UFW_G3["2379-2380/tcp - etcd"]
            UFW_G4["10250-10252/tcp - kubelet, scheduler, controller"]
            UFW_G5["179/tcp - Calico BGP"]
            UFW_G6["4789/udp - Calico VXLAN"]
            UFW_G7["51820/udp - WireGuard VPN"]
        end

        subgraph MiniMight["mini-might (Worker)"]
            direction TB
            UFW_M1["22/tcp - SSH"]
            UFW_M2["80/tcp - HTTP"]
            UFW_M3["443/tcp - HTTPS"]
            UFW_M4["10250/tcp - kubelet"]
            UFW_M5["30000-32767/tcp - NodePort"]
            UFW_M6["4789/udp - Calico VXLAN"]
        end
    end

    style UFW_M2 fill:#2ecc71,stroke:#27ae60,color:#fff
    style UFW_M3 fill:#2ecc71,stroke:#27ae60,color:#fff

mini-gmk (Control Plane) ufw settings:

# Default policies
sudo ufw default deny incoming
sudo ufw default allow outgoing

# SSH
sudo ufw allow 22/tcp comment 'SSH'

# Kubernetes Control Plane
sudo ufw allow 6443/tcp comment 'K8s API Server'
sudo ufw allow 2379:2380/tcp comment 'etcd'
sudo ufw allow 10250/tcp comment 'kubelet'
sudo ufw allow 10251/tcp comment 'kube-scheduler'
sudo ufw allow 10252/tcp comment 'kube-controller-manager'

# Calico CNI
sudo ufw allow 179/tcp comment 'Calico BGP'
sudo ufw allow 4789/udp comment 'Calico VXLAN'
sudo ufw allow from 192.168.45.0/24 comment 'Internal network'

# VPN (optional)
sudo ufw allow 51820/udp comment 'WireGuard'

# Enable
sudo ufw enable

mini-might (Worker) ufw settings:

# Default policies
sudo ufw default deny incoming
sudo ufw default allow outgoing

# SSH
sudo ufw allow 22/tcp comment 'SSH'

# Web Traffic (Istio IngressGateway)
sudo ufw allow 80/tcp comment 'HTTP'
sudo ufw allow 443/tcp comment 'HTTPS'

# Kubernetes Worker
sudo ufw allow 10250/tcp comment 'kubelet'
sudo ufw allow 30000:32767/tcp comment 'NodePort Services'

# Calico CNI
sudo ufw allow 4789/udp comment 'Calico VXLAN'
sudo ufw allow from 192.168.45.0/24 comment 'Internal network'

# Enable
sudo ufw enable

1.5 Team Member VPN/SSH Access#

sequenceDiagram
    participant TM as Team Member
    participant TS as Tailscale/WireGuard<br/>VPN Server
    participant GMK as mini-gmk<br/>192.168.45.123
    participant MIGHT as mini-might<br/>192.168.45.154
    participant K8S as Kubernetes<br/>API Server

    Note over TM,K8S: VPN connection and SSH access flow

    TM->>TS: VPN connection request
    TS-->>TM: VPN tunnel established<br/>(10.x.x.x range)

    TM->>GMK: SSH access (via VPN tunnel)<br/>ssh admin@192.168.45.123
    GMK-->>TM: SSH session established

    TM->>GMK: kubectl commands
    GMK->>K8S: API request (:6443)
    K8S-->>GMK: API response
    GMK-->>TM: Result returned

    Note over TM,K8S: Using k9s, helm, argocd CLI, etc.

    TM->>MIGHT: SSH access (if needed)<br/>ssh admin@192.168.45.154
    MIGHT-->>TM: SSH session established

Tailscale setup (recommended):

# Install Tailscale on each node
curl -fsSL https://tailscale.com/install.sh | sh

# Login (on each node)
sudo tailscale up --ssh

# Team member from their PC
tailscale up
ssh admin@mini-gmk  # using Tailscale MagicDNS

Problem Description#

2.1 Symptoms#

flowchart LR
    subgraph TestResults["Test Results"]
        direction TB

        subgraph Success["✅ Working"]
            CURL["curl (CLI)"]
            SAFARI["Safari"]
            FIREFOX["Firefox"]
            MOBILE["Mobile LTE"]
            INCOGNITO["Chrome Incognito"]
        end

        subgraph Fail["❌ Intermittent 404"]
            CHROME["Chrome (normal)"]
            CHROME_TEAM["Team members' Chrome"]
        end
    end

    CURL -->|"HTTP/2"| OK1["200 OK"]
    SAFARI -->|"HTTP/2"| OK2["200 OK"]
    FIREFOX -->|"HTTP/2"| OK3["200 OK"]
    MOBILE -->|"HTTP/2"| OK4["200 OK"]
    INCOGNITO -->|"QUIC attempt"| RANDOM1["Random 404"]

    CHROME -->|"QUIC/HTTP3"| RANDOM2["Random 404"]
    CHROME_TEAM -->|"QUIC/HTTP3"| RANDOM3["Random 404"]

    style CHROME fill:#ff6b6b,stroke:#c0392b,color:#fff
    style CHROME_TEAM fill:#ff6b6b,stroke:#c0392b,color:#fff
    style RANDOM2 fill:#ff6b6b,stroke:#c0392b,color:#fff
    style RANDOM3 fill:#ff6b6b,stroke:#c0392b,color:#fff
TestResultProtocol
curl -I https://api.dev.goormgb.space/swagger-ui/index.html200 OKHTTP/2
SafariOKHTTP/2
FirefoxOKHTTP/2
Mobile LTE networkOKHTTP/2
Chrome (same network)Intermittent 404QUIC/HTTP3
Team members’ ChromeSame symptomQUIC/HTTP3

2.2 Affected Scope#

# Affected domains (all pointing to home server)
- api.dev.goormgb.space    # Backend API (Swagger UI)
- argocd.goormgb.space     # GitOps
- grafana.goormgb.space    # Monitoring
- kiali.goormgb.space      # Service Mesh visualization
- cloudbeaver.goormgb.space # DB Admin

2.3 Detailed Symptom#

sequenceDiagram
    participant User as User
    participant Chrome as Chrome
    participant Router as SK Router
    participant GW as IngressGateway

    Note over User,GW: Random 404 when pressing F5 repeatedly

    User->>Chrome: F5 (refresh 1)
    Chrome->>Router: QUIC/UDP:443
    Router->>GW: Forward
    GW-->>Router: 200 OK
    Router-->>Chrome: 200 OK
    Chrome-->>User: ✅ OK

    User->>Chrome: F5 (refresh 2)
    Chrome->>Router: QUIC/UDP:443
    Note over Router: UDP NAT state unstable
    Router--xChrome: Timeout/drop
    Chrome->>Chrome: HTTP/2 fallback attempt
    Note over Chrome: Connection state confused
    Chrome-->>User: ❌ 404 Not Found

    User->>Chrome: F5 (refresh 3)
    Chrome->>Router: HTTP/2/TCP:443
    Router->>GW: Forward
    GW-->>Router: 200 OK
    Router-->>Chrome: 200 OK
    Chrome-->>User: ✅ OK

    User->>Chrome: F5 (refresh 4)
    Chrome->>Router: QUIC/UDP:443 (retry)
    Router--xChrome: Fails again
    Chrome-->>User: ❌ 404 Not Found

Root Cause Analysis#

3.1 How Chrome QUIC/HTTP3 Works#

flowchart TB
    subgraph Chrome["Chrome Browser"]
        direction TB
        REQ["Request start"]
        ALT_SVC["Check Alt-Svc header"]
        QUIC_TRY["Attempt QUIC connection<br/>(UDP:443)"]
        H2_FALLBACK["HTTP/2 Fallback<br/>(TCP:443)"]
        POOL["Connection Pool"]
    end

    subgraph HomeRouter["SK Broadband Router"]
        direction TB
        NAT_UDP["UDP NAT table"]
        NAT_TCP["TCP NAT table"]
        UDP_TIMEOUT["UDP timeout: 30s"]
        TCP_TIMEOUT["TCP timeout: 3600s"]
    end

    REQ --> ALT_SVC
    ALT_SVC -->|"h3=':443' found"| QUIC_TRY
    QUIC_TRY -->|"UDP:443"| NAT_UDP
    NAT_UDP --> UDP_TIMEOUT
    UDP_TIMEOUT -->|"expires after 30s"| QUIC_TRY
    QUIC_TRY -->|"failure"| H2_FALLBACK
    H2_FALLBACK --> NAT_TCP
    NAT_TCP --> TCP_TIMEOUT

    QUIC_TRY -->|"on success"| POOL
    H2_FALLBACK -->|"on success"| POOL
    POOL -->|"reuse"| REQ

    style UDP_TIMEOUT fill:#ff6b6b,stroke:#c0392b,color:#fff
    style QUIC_TRY fill:#f39c12,stroke:#d68910,color:#fff

3.2 The Core Problem: UDP NAT Timeout#

flowchart LR
    subgraph Problem["Where the Problem Occurs"]
        direction TB

        subgraph QUIC_Conn["QUIC Connection"]
            Q1["1. Chrome establishes QUIC connection"]
            Q2["2. Registered in SK router UDP NAT table"]
            Q3["3. No traffic for 30 seconds"]
            Q4["4. Entry removed from NAT table"]
            Q5["5. Next QUIC packet dropped"]
            Q6["6. Chrome confused → 404"]
        end

        Q1 --> Q2 --> Q3 --> Q4 --> Q5 --> Q6
    end

    subgraph Compare["TCP vs UDP Comparison"]
        direction TB
        TCP["TCP (HTTP/2)<br/>Timeout: 3600s<br/>Keep-Alive supported<br/>Stable state tracking"]
        UDP["UDP (QUIC)<br/>Timeout: 30s<br/>No connection state<br/>Unstable NAT traversal"]
    end

    style Q4 fill:#ff6b6b,stroke:#c0392b,color:#fff
    style Q5 fill:#ff6b6b,stroke:#c0392b,color:#fff
    style Q6 fill:#ff6b6b,stroke:#c0392b,color:#fff
    style UDP fill:#ff6b6b,stroke:#c0392b,color:#fff
    style TCP fill:#2ecc71,stroke:#27ae60,color:#fff

3.3 SK Broadband Environment Specifics#

flowchart TB
    subgraph ISP_Issues["SK Broadband Environment Issues"]
        direction TB

        ISP1["1. Symmetric NAT<br/>NAT type unfavorable for QUIC"]
        ISP2["2. UDP port filtering<br/>Some ISPs restrict UDP:443"]
        ISP3["3. Short UDP session timeout<br/>Home router default: 30s"]
        ISP4["4. Dynamic public IP<br/>Connection drops on IP change"]
    end

    subgraph HomeLab_Issues["Home Lab Issues"]
        direction TB

        HL1["1. Possible double NAT<br/>Router + modem"]
        HL2["2. Port forwarding limitations<br/>Difficult to track UDP state"]
        HL3["3. Firewall settings<br/>ufw UDP handling"]
        HL4["4. Using externalIPs<br/>No LoadBalancer"]
    end

    ISP1 --> ISP2 --> ISP3 --> ISP4
    HL1 --> HL2 --> HL3 --> HL4

    ISP3 -.->|"combined"| PROBLEM["Unstable QUIC connection"]
    HL2 -.->|"combined"| PROBLEM

    style PROBLEM fill:#ff6b6b,stroke:#c0392b,color:#fff

3.4 Istio IngressGateway and QUIC#

flowchart TB
    subgraph Istio["Istio IngressGateway"]
        direction TB

        SVC["Service: istio-ingressgateway<br/>type: ClusterIP<br/>externalIPs: [192.168.45.154]"]
        ENVOY["Envoy Proxy<br/>HTTP/2, QUIC supported"]
        GW_RES["Gateway Resource<br/>TLS Termination"]
        VS["VirtualService<br/>Routing rules"]
    end

    subgraph TLS_Config["TLS Configuration"]
        CERT["Let's Encrypt<br/>Wildcard certificate"]
        SECRET["Secret: goormgb-tls<br/>*.goormgb.space<br/>*.dev.goormgb.space"]
    end

    subgraph Problem["Problem Points"]
        EXT_IP["externalIPs binding<br/>also receives UDP traffic"]
        UDP_FW["443/udp open in ufw?<br/>→ NAT issue even if open"]
        QUIC_HANDSHAKE["QUIC handshake<br/>must complete before NAT timeout"]
    end

    SVC --> ENVOY --> GW_RES --> VS
    CERT --> SECRET --> GW_RES
    EXT_IP -.-> UDP_FW -.-> QUIC_HANDSHAKE

    style QUIC_HANDSHAKE fill:#f39c12,stroke:#d68910,color:#fff

Solution#

4.1 Available Options#

flowchart TB
    subgraph Solutions["Solutions"]
        direction TB

        subgraph Client["Client-side (not recommended)"]
            C1["Disable Chrome QUIC<br/>chrome://flags/#enable-quic"]
            C2["Disable DoH<br/>chrome://flags/#dns-over-https"]
        end

        subgraph Network["Network-side (difficult)"]
            N1["Replace router<br/>Increase UDP NAT timeout"]
            N2["Change ISP<br/>Not realistic"]
        end

        subgraph Server["Server-side (adopted)"]
            S1["Cloudflare Proxy<br/>QUIC → HTTP/2 conversion"]
            S2["Disable QUIC<br/>Remove Alt-Svc on server"]
        end
    end

    C1 -->|"requires config on all team members' machines"| BAD1["❌ Not recommended"]
    C2 -->|"reset on browser update"| BAD2["❌ Not recommended"]
    N1 -->|"cost + complexity"| BAD3["△ Fallback"]
    N2 -->|"not realistic"| BAD4["❌ Not feasible"]
    S1 -->|"free + additional benefits"| GOOD1["✅ Adopted"]
    S2 -->|"gives up QUIC benefits"| BAD5["△ Fallback"]

    style GOOD1 fill:#2ecc71,stroke:#27ae60,color:#fff
    style BAD1 fill:#e74c3c,stroke:#c0392b,color:#fff
    style BAD2 fill:#e74c3c,stroke:#c0392b,color:#fff
    style BAD4 fill:#e74c3c,stroke:#c0392b,color:#fff

4.2 Why Cloudflare Proxy Was Chosen#

flowchart LR
    subgraph Before["Before: Direct Connection"]
        B_Chrome["Chrome"]
        B_ISP["SK Broadband"]
        B_Router["Router<br/>UDP NAT unstable"]
        B_Ingress["IngressGateway"]

        B_Chrome -->|"QUIC/UDP"| B_ISP
        B_ISP -->|"UDP"| B_Router
        B_Router -->|"❌ intermittent drop"| B_Ingress
    end

    subgraph After["After: Cloudflare Proxy"]
        A_Chrome["Chrome"]
        A_CF["Cloudflare Edge<br/>(Seoul PoP)"]
        A_ISP["SK Broadband"]
        A_Router["Router<br/>TCP stable"]
        A_Ingress["IngressGateway"]

        A_Chrome -->|"QUIC/UDP"| A_CF
        A_CF -->|"HTTP/2/TCP"| A_ISP
        A_ISP -->|"TCP"| A_Router
        A_Router -->|"✅ stable"| A_Ingress
    end

    Before -.->|"migration"| After

    style B_Router fill:#ff6b6b,stroke:#c0392b,color:#fff
    style A_CF fill:#2ecc71,stroke:#27ae60,color:#fff
    style A_Router fill:#2ecc71,stroke:#27ae60,color:#fff

Cloudflare benefits:

ItemDescription
QUIC handlingConverts QUIC ↔ HTTP/2 at the edge
Connection stabilityTCP connection to origin (NAT-friendly)
Client configurationNo changes needed
DDoS protectionIncluded by default
Global CDNLeverages Seoul PoP
SSL certificateUniversal SSL auto-managed
CostFree plan is sufficient

Cloudflare Migration#

5.1 Full Migration Flow#

flowchart TB
    subgraph Step1["Step 1: Cloudflare Setup"]
        CF_SIGNUP["Sign up for Cloudflare"]
        CF_DOMAIN["Add domain<br/>goormgb.space"]
        CF_RECORDS["Configure DNS records"]
        CF_SSL["SSL/TLS settings<br/>Full (strict)"]
    end

    subgraph Step2["Step 2: Nameserver Change"]
        PORKBUN["Porkbun (registrar)"]
        NS_CHANGE["Change nameservers<br/>Route53 → Cloudflare"]
        NS_VERIFY["Confirm propagation<br/>(up to 48 hours)"]
    end

    subgraph Step3["Step 3: DDNS Migration"]
        DDNS_OLD["ddns-route53<br/>(delete)"]
        DDNS_NEW["ddns-cloudflare<br/>(new)"]
        SECRET_NEW["ExternalSecret<br/>Cloudflare API Token"]
    end

    subgraph Step4["Step 4: Verification"]
        DNS_CHECK["Confirm DNS propagation"]
        CF_HEADER["Check Cloudflare headers"]
        BROWSER_TEST["Test in Chrome"]
    end

    CF_SIGNUP --> CF_DOMAIN --> CF_RECORDS --> CF_SSL
    CF_SSL --> PORKBUN --> NS_CHANGE --> NS_VERIFY
    NS_VERIFY --> DDNS_OLD --> DDNS_NEW --> SECRET_NEW
    SECRET_NEW --> DNS_CHECK --> CF_HEADER --> BROWSER_TEST

    style DDNS_OLD fill:#e74c3c,stroke:#c0392b,color:#fff
    style DDNS_NEW fill:#2ecc71,stroke:#27ae60,color:#fff

5.2 Cloudflare DNS Record Configuration#

flowchart LR
    subgraph CloudflareDNS["Cloudflare DNS Records"]
        direction TB

        subgraph Proxied["Proxied (orange cloud)"]
            A1["A | @ | 39.119.192.15"]
            A2["A | api.dev | 39.119.192.15"]
            A3["A | argocd | 39.119.192.15"]
            A4["A | grafana | 39.119.192.15"]
            A5["A | kiali | 39.119.192.15"]
            A6["A | cloudbeaver | 39.119.192.15"]
        end

        subgraph DNSOnly["DNS Only (grey cloud)"]
            CNAME1["CNAME | dev | cname.vercel-dns.com"]
        end
    end

    A1 --> CF_EDGE["Cloudflare Edge<br/>QUIC → HTTP/2"]
    A2 --> CF_EDGE
    A3 --> CF_EDGE
    A4 --> CF_EDGE
    A5 --> CF_EDGE
    A6 --> CF_EDGE

    CF_EDGE --> HOME["Home server<br/>39.119.192.15"]

    CNAME1 --> VERCEL["Vercel<br/>(frontend)"]

    style Proxied fill:#f39c12,stroke:#d68910,color:#fff
    style DNSOnly fill:#95a5a6,stroke:#7f8c8d,color:#fff
TypeNameContentProxyTTLPurpose
A@39.119.192.15ProxiedAutoRoot domain
Aapi.dev39.119.192.15ProxiedAutoBackend API
Aargocd39.119.192.15ProxiedAutoGitOps
Agrafana39.119.192.15ProxiedAutoMonitoring
Akiali39.119.192.15ProxiedAutoService Mesh
Acloudbeaver39.119.192.15ProxiedAutoDB Admin
CNAMEdevcname.vercel-dns.comDNS onlyAutoFrontend

Important: The dev subdomain is hosted on Vercel, so it must be DNS only (grey cloud)

5.3 SSL/TLS Settings#

flowchart LR
    subgraph CloudflareSSL["Cloudflare SSL/TLS"]
        direction TB

        MODE["Encryption Mode:<br/>Full (strict)"]
        EDGE_CERT["Edge Certificate:<br/>Cloudflare Universal SSL"]
        ORIGIN_CERT["Origin Certificate:<br/>Let's Encrypt (existing)"]

        SETTINGS["Edge Certificate Settings:"]
        S1["Always Use HTTPS: On"]
        S2["Minimum TLS: 1.2"]
        S3["TLS 1.3: Enabled"]
        S4["Automatic HTTPS Rewrites: On"]
    end

    CLIENT["Client"]
    CF_EDGE["Cloudflare Edge"]
    ORIGIN["Home server (Origin)"]

    CLIENT -->|"HTTPS<br/>Cloudflare SSL"| CF_EDGE
    CF_EDGE -->|"HTTPS<br/>Let's Encrypt"| ORIGIN

    style MODE fill:#2ecc71,stroke:#27ae60,color:#fff

Full (strict) mode requirements:

  • Origin server needs a valid SSL certificate
  • We use Let’s Encrypt wildcard certificate → requirement satisfied

5.4 Nameserver Change#

sequenceDiagram
    participant User as Admin
    participant Porkbun as Porkbun<br/>(registrar)
    participant R53 as Route53<br/>(old)
    participant CF as Cloudflare<br/>(new)

    Note over User,CF: Nameserver migration

    User->>CF: 1. Request to add domain
    CF-->>User: 2. Provide nameserver info<br/>jacob.ns.cloudflare.com<br/>rita.ns.cloudflare.com

    User->>Porkbun: 3. Change nameservers
    Note over Porkbun: Route53 NS → Cloudflare NS

    Porkbun->>CF: 4. NS propagation begins
    Note over CF: Up to 48 hours<br/>(usually 1-2 hours)

    CF-->>User: 5. Domain activation complete

    Note over R53: Route53 Hosted Zone<br/>no longer in use<br/>(kept as cert-manager backup)

Changing nameservers in Porkbun:

# Old (Route53)
ns-xxx.awsdns-xx.com
ns-xxx.awsdns-xx.net
ns-xxx.awsdns-xx.org
ns-xxx.awsdns-xx.co.uk

# New (Cloudflare)
jacob.ns.cloudflare.com
rita.ns.cloudflare.com

5.5 DDNS Helm Chart Migration#

flowchart TB
    subgraph Before["Before: Route53 DDNS"]
        direction TB
        CHART_OLD["dev/charts/core/ddns-route53/"]
        IMAGE_OLD["amazon/aws-cli:2.15.0"]
        SECRET_OLD["route53-ddns-credentials<br/>(AWS IAM)"]
        API_OLD["aws route53 change-resource-record-sets"]
    end

    subgraph After["After: Cloudflare DDNS"]
        direction TB
        CHART_NEW["dev/charts/core/ddns/"]
        IMAGE_NEW["curlimages/curl:8.5.0"]
        SECRET_NEW["cloudflare-ddns-credentials<br/>(ExternalSecret)"]
        API_NEW["Cloudflare API<br/>PATCH /zones/{zone_id}/dns_records"]
    end

    CHART_OLD -->|"deleted"| REMOVED["Removed"]
    CHART_NEW -->|"newly created"| ACTIVE["Active"]

    style CHART_OLD fill:#e74c3c,stroke:#c0392b,color:#fff
    style REMOVED fill:#e74c3c,stroke:#c0392b,color:#fff
    style CHART_NEW fill:#2ecc71,stroke:#27ae60,color:#fff
    style ACTIVE fill:#2ecc71,stroke:#27ae60,color:#fff

values-ddns.yaml#

# 303-goormgb-k8s-helm/dev/values/core/values-ddns.yaml

domain: "goormgb.space"
zoneID: "f9fdc81712963738bc9d5ee628849e53"

records:
  - ""          # goormgb.space (root)
  - "api.dev"   # api.dev.goormgb.space
  - "argocd"    # argocd.goormgb.space
  - "grafana"   # grafana.goormgb.space
  - "kiali"     # kiali.goormgb.space
  - "cloudbeaver"  # cloudbeaver.goormgb.space

ttl: 60         # fast IP change propagation
schedule: "*/5 * * * *"  # check every 5 minutes

secret:
  name: cloudflare-ddns-credentials
  awsSecretKey: dev/infra/cloudflare  # AWS Secrets Manager path

image:
  repository: curlimages/curl
  tag: "8.5.0"

CronJob Template#

# 303-goormgb-k8s-helm/dev/charts/core/ddns/templates/cronjob.yaml

apiVersion: batch/v1
kind: CronJob
metadata:
  name: ddns-cloudflare-updater
  namespace: {{ .Release.Namespace }}
  labels:
    app.kubernetes.io/name: ddns-cloudflare
spec:
  schedule: {{ .Values.schedule | quote }}
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 1
  failedJobsHistoryLimit: 3
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          serviceAccountName: default
          containers:
            - name: ddns-updater
              image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
              env:
                - name: CF_API_TOKEN
                  valueFrom:
                    secretKeyRef:
                      name: {{ .Values.secret.name }}
                      key: api-token
                - name: CF_ZONE_ID
                  value: {{ .Values.zoneID | quote }}
                - name: DOMAIN
                  value: {{ .Values.domain | quote }}
                - name: TTL
                  value: {{ .Values.ttl | quote }}
              command:
                - /bin/sh
                - -c
                - |
                  set -e

                  echo "=========================================="
                  echo "DDNS Cloudflare Updater"
                  echo "Started at: $(date -Iseconds)"
                  echo "=========================================="

                  # 1. Get current public IP (with fallback)
                  echo "[Step 1] Getting current public IP..."
                  CURRENT_IP=$(curl -sf --max-time 10 https://api.ipify.org || \
                               curl -sf --max-time 10 https://checkip.amazonaws.com || \
                               curl -sf --max-time 10 https://ifconfig.me)

                  if [ -z "$CURRENT_IP" ]; then
                    echo "ERROR: Failed to get public IP from all sources"
                    exit 1
                  fi
                  echo "Current Public IP: ${CURRENT_IP}"
                  echo ""

                  # 2. Update each record
                  echo "[Step 2] Processing DNS records..."
                  {{- range .Values.records }}
                  RECORD="{{ . }}"
                  if [ -z "$RECORD" ]; then
                    FULL_NAME="${DOMAIN}"
                  else
                    FULL_NAME="${RECORD}.${DOMAIN}"
                  fi

                  echo "----------------------------------------"
                  echo "Processing: ${FULL_NAME}"

                  # 2-1. Look up current record in Cloudflare
                  RESPONSE=$(curl -sf -X GET \
                    "https://api.cloudflare.com/client/v4/zones/${CF_ZONE_ID}/dns_records?type=A&name=${FULL_NAME}" \
                    -H "Authorization: Bearer ${CF_API_TOKEN}" \
                    -H "Content-Type: application/json")

                  RECORD_ID=$(echo "$RESPONSE" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
                  CURRENT_CONTENT=$(echo "$RESPONSE" | grep -o '"content":"[^"]*"' | head -1 | cut -d'"' -f4)
                  PROXIED=$(echo "$RESPONSE" | grep -o '"proxied":[^,]*' | head -1 | cut -d':' -f2)

                  if [ -z "$RECORD_ID" ]; then
                    echo "  WARNING: Record not found for ${FULL_NAME}, skipping..."
                    continue
                  fi

                  echo "  Record ID: ${RECORD_ID}"
                  echo "  Current IP in Cloudflare: ${CURRENT_CONTENT}"
                  echo "  Proxied: ${PROXIED}"

                  # 2-2. Check if IP has changed
                  if [ "$CURRENT_CONTENT" = "$CURRENT_IP" ]; then
                    echo "  Status: UNCHANGED (no update needed)"
                    continue
                  fi

                  # 2-3. Update IP
                  echo "  Updating: ${CURRENT_CONTENT} → ${CURRENT_IP}"
                  UPDATE_RESULT=$(curl -sf -X PATCH \
                    "https://api.cloudflare.com/client/v4/zones/${CF_ZONE_ID}/dns_records/${RECORD_ID}" \
                    -H "Authorization: Bearer ${CF_API_TOKEN}" \
                    -H "Content-Type: application/json" \
                    --data "{\"content\":\"${CURRENT_IP}\",\"ttl\":${TTL},\"proxied\":true}")

                  if echo "$UPDATE_RESULT" | grep -q '"success":true'; then
                    echo "  Status: UPDATED successfully"
                  else
                    echo "  Status: FAILED"
                    echo "  Response: $UPDATE_RESULT"
                  fi
                  {{- end }}

                  echo ""
                  echo "=========================================="
                  echo "DDNS update completed at $(date -Iseconds)"
                  echo "=========================================="

ExternalSecret Template#

# 303-goormgb-k8s-helm/dev/charts/core/ddns/templates/external-secret.yaml

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: cloudflare-ddns-credentials
  namespace: {{ .Release.Namespace }}
  labels:
    app.kubernetes.io/name: ddns-cloudflare
spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore
    name: aws-secrets-manager
  target:
    name: {{ .Values.secret.name }}
    creationPolicy: Owner
  data:
    - secretKey: api-token
      remoteRef:
        key: {{ .Values.secret.awsSecretKey }}
        property: api-token

5.6 AWS Secrets Manager Configuration#

flowchart LR
    subgraph AWS["AWS Secrets Manager"]
        SECRET["dev/infra/cloudflare"]
        JSON["{<br/>  'api-token': 'cf_xxx...'<br/>}"]
    end

    subgraph K8s["Kubernetes"]
        ESO["External Secrets Operator"]
        K8S_SECRET["Secret:<br/>cloudflare-ddns-credentials"]
        CRONJOB["CronJob:<br/>ddns-cloudflare-updater"]
    end

    SECRET --> JSON
    ESO -->|"sync every 1h"| SECRET
    ESO --> K8S_SECRET
    CRONJOB -->|"env: CF_API_TOKEN"| K8S_SECRET

    style SECRET fill:#f39c12,stroke:#d68910,color:#fff
# Save Cloudflare API Token in AWS Secrets Manager
aws secretsmanager create-secret \
  --name dev/infra/cloudflare \
  --secret-string '{"api-token":"YOUR_CLOUDFLARE_API_TOKEN"}' \
  --region ap-northeast-2

# Cloudflare API Token permissions:
# - Zone > DNS > Edit
# - Zone > Zone > Read
# - Scope: goormgb.space domain only

6. Full Traffic Flow (After Migration)#

6.1 Detailed Request Flow#

sequenceDiagram
    participant Chrome as Chrome<br/>(Client)
    participant CF_Edge as Cloudflare Edge<br/>(Seoul PoP)
    participant ISP as SK Broadband<br/>(Public IP)
    participant Router as Home Router<br/>(NAT)
    participant UFW as ufw Firewall<br/>(mini-might)
    participant Ingress as IngressGateway<br/>(Istio)
    participant GW as java-cloud-gateway<br/>(:8085)
    participant Backend as Backend Service

    Note over Chrome,Backend: https://api.dev.goormgb.space/swagger-ui/index.html

    Chrome->>CF_Edge: 1. QUIC/HTTP3 request<br/>(UDP:443)
    Note over CF_Edge: QUIC connection terminated<br/>Converted to HTTP/2

    CF_Edge->>ISP: 2. HTTP/2 request<br/>(TCP:443)
    Note over ISP: TCP connection<br/>(stable)

    ISP->>Router: 3. Port forwarding<br/>:443 → 192.168.45.154:443
    Note over Router: TCP NAT table<br/>(timeout 3600s)

    Router->>UFW: 4. Pass through firewall<br/>(443/tcp allowed)
    UFW->>Ingress: 5. Forward to Envoy Proxy

    Note over Ingress: TLS Termination<br/>(Let's Encrypt)

    Ingress->>GW: 6. VirtualService routing<br/>→ java-cloud-gateway:8085
    GW->>Backend: 7. Internal routing<br/>→ swagger-ui service

    Backend-->>GW: 8. Response (200 OK)
    GW-->>Ingress: 9. Forward response
    Ingress-->>Router: 10. HTTPS response
    Router-->>ISP: 11. NAT reverse translation
    ISP-->>CF_Edge: 12. HTTP/2 response
    CF_Edge-->>Chrome: 13. QUIC/HTTP3 response<br/>(200 OK)

    Note over Chrome,Backend: ✅ All requests handled reliably

6.2 DDNS Update Flow#

sequenceDiagram
    participant Cron as CronJob<br/>(mini-gmk)
    participant IPify as api.ipify.org
    participant CF_API as Cloudflare API
    participant CF_DNS as Cloudflare DNS

    Note over Cron,CF_DNS: Runs every 5 minutes (*/5 * * * *)

    Cron->>IPify: 1. GET / (get public IP)
    IPify-->>Cron: 39.119.192.15

    loop For each record
        Cron->>CF_API: 2. GET /zones/{zone}/dns_records<br/>?name=api.dev.goormgb.space
        CF_API-->>Cron: record_id, current_ip

        alt IP changed
            Cron->>CF_API: 3. PATCH /dns_records/{id}<br/>{"content": "39.119.192.15", "proxied": true}
            CF_API-->>Cron: {"success": true}
            CF_API->>CF_DNS: Update DNS record
            Note over Cron: "api.dev: updated 1.2.3.4 → 39.119.192.15"
        else IP unchanged
            Note over Cron: "api.dev: unchanged (39.119.192.15)"
        end
    end

    Note over Cron,CF_DNS: TTL: 60s for fast propagation

7. Verification#

7.1 DNS Propagation Check#

# Check Cloudflare nameservers
dig NS goormgb.space +short
# Expected result:
# jacob.ns.cloudflare.com.
# rita.ns.cloudflare.com.

# Check A record - returns Cloudflare Anycast IP
dig api.dev.goormgb.space +short
# Expected result (Cloudflare IP, not origin IP):
# 104.21.x.x
# 172.67.x.x

7.2 Check Cloudflare Headers#

curl -I https://api.dev.goormgb.space/swagger-ui/index.html

# Expected response:
HTTP/2 200
date: Mon, 17 Mar 2026 10:00:00 GMT
content-type: text/html
server: cloudflare
cf-ray: 8xxxxx-ICN              # ICN = Seoul Edge
cf-cache-status: DYNAMIC
alt-svc: h3=":443"; ma=86400    # HTTP/3 support (CF ↔ client)

7.3 Verify DDNS CronJob#

# CronJob status
kubectl get cronjob -n core
# NAME                      SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE
# ddns-cloudflare-updater   */5 * * * *   False     0        2m

# Recent job logs
kubectl logs -n core -l app.kubernetes.io/name=ddns-cloudflare --tail=50

# Expected output:
# ==========================================
# DDNS Cloudflare Updater
# Started at: 2026-03-17T10:05:00+09:00
# ==========================================
# [Step 1] Getting current public IP...
# Current Public IP: 39.119.192.15
#
# [Step 2] Processing DNS records...
# ----------------------------------------
# Processing: goormgb.space
#   Record ID: abc123...
#   Current IP in Cloudflare: 39.119.192.15
#   Proxied: true
#   Status: UNCHANGED (no update needed)
# ----------------------------------------
# Processing: api.dev.goormgb.space
#   Record ID: def456...
#   Current IP in Cloudflare: 39.119.192.15
#   Proxied: true
#   Status: UNCHANGED (no update needed)
# ...
# ==========================================
# DDNS update completed at 2026-03-17T10:05:03+09:00
# ==========================================

# Manual run test
kubectl create job ddns-test --from=cronjob/ddns-cloudflare-updater -n core
kubectl logs -n core job/ddns-test -f
kubectl delete job ddns-test -n core

7.4 Chrome Test#

flowchart TB
    subgraph TestSteps["Chrome Test Steps"]
        direction TB

        STEP1["1. Clear cache<br/>chrome://net-internals/#dns<br/>→ Clear host cache"]
        STEP2["2. Clear sockets<br/>chrome://net-internals/#sockets<br/>→ Close idle sockets"]
        STEP3["3. Access test<br/>https://api.dev.goormgb.space/swagger-ui/index.html"]
        STEP4["4. Rapid F5 test<br/>(10+ times)"]
        STEP5["5. Check protocol<br/>DevTools → Network → Protocol"]
    end

    STEP1 --> STEP2 --> STEP3 --> STEP4 --> STEP5

    STEP5 --> RESULT["Check result"]
    RESULT --> H2["h2 (HTTP/2): CF↔Origin"]
    RESULT --> H3["h3 (HTTP/3): Client↔CF"]

    style RESULT fill:#2ecc71,stroke:#27ae60,color:#fff

Expected results:

  • All requests return 200 OK
  • Protocol column shows h2 or h3
  • cf-ray header shows ICN (Seoul)

8. Removed Resources#

flowchart LR
    subgraph Deleted["Removed Resources"]
        D1["ddns-route53 Helm Chart"]
        D2["ddns-route53 ArgoCD App"]
        D3["route53-ddns-credentials Secret"]
    end

    subgraph Kept["Retained Resources"]
        K1["Route53 Hosted Zone<br/>(kept as cert-manager backup)"]
    end

    D1 -->|"rm -rf"| REMOVED["Removed"]
    D2 -->|"kubectl delete app"| REMOVED
    D3 -->|"replaced by ESO"| REMOVED

    K1 --> BACKUP["Kept as backup"]

    style Deleted fill:#e74c3c,stroke:#c0392b,color:#fff
    style REMOVED fill:#e74c3c,stroke:#c0392b,color:#fff
    style Kept fill:#f39c12,stroke:#d68910,color:#fff
ResourceStatusNote
ddns-route53 Helm ChartDeleteddev/charts/core/ddns-route53/
ddns-route53 ArgoCD AppDeletedkubectl delete app ddns-route53 -n argocd
route53-ddns-credentialsDeletedReplaced by ESO
Route53 Hosted ZoneRetainedBackup for cert-manager DNS-01

9. Key Takeaways#

mindmap
  root((Chrome QUIC<br/>404 Troubleshooting))
    Root Cause
      Chrome QUIC/HTTP3
        Uses UDP:443
        Activated via Alt-Svc header
      SK Broadband Router
        UDP NAT timeout 30s
        Symmetric NAT
      Home server environment
        Port forwarding
        externalIPs
    Solution
      Cloudflare Proxy
        QUIC → HTTP/2 conversion
        Connection terminated at edge
        Only TCP to origin
      DDNS migration
        Route53 → Cloudflare
        CronJob reimplemented
    Additional benefits
      DDoS protection
      Global CDN
      Auto SSL
      Free plan
    Changed files
      values-ddns.yaml
      cronjob.yaml
      external-secret.yaml

10. References#


Appendix A: Troubleshooting Command Reference#

# === DNS checks ===
dig NS goormgb.space +short
dig api.dev.goormgb.space +short
dig api.dev.goormgb.space +trace

# === Check Cloudflare headers ===
curl -I https://api.dev.goormgb.space/swagger-ui/index.html
curl -svo /dev/null https://api.dev.goormgb.space/ 2>&1 | grep -i "cf-"

# === DDNS CronJob ===
kubectl get cronjob -n core
kubectl logs -n core -l app.kubernetes.io/name=ddns-cloudflare --tail=50
kubectl create job ddns-test --from=cronjob/ddns-cloudflare-updater -n core

# === Istio IngressGateway ===
kubectl get svc -n istio-system istio-ingressgateway -o yaml
kubectl logs -n istio-system -l app=istio-ingressgateway --tail=100

# === Gateway/VirtualService ===
kubectl get gw,vs -n istio-system
kubectl describe gw api-gateway -n istio-system

# === Certificate check ===
kubectl get certificate -n istio-system
kubectl describe certificate goormgb-tls -n istio-system

# === ufw status ===
# mini-gmk
sudo ufw status verbose
# mini-might
sudo ufw status verbose

# === Network connectivity test ===
# External access from inside
curl -I https://api.dev.goormgb.space/swagger-ui/index.html
# Direct access to service inside cluster
kubectl exec -n istio-system deploy/istio-ingressgateway -- \
  curl -I http://java-cloud-gateway.dev-webs.svc.cluster.local:8085/swagger-ui/index.html

Appendix B: How to Check Router UDP NAT Timeout#

# 1. Listen for UDP on an external server (if available)
nc -lu 12345

# 2. Send UDP packet from the home server
echo "test" | nc -u <external-server-ip> 12345

# 3. Send again after 30 seconds → confirm connection is dropped

# Or check NAT type using a STUN server
# https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/

Written: 2026-03-17 Environment: kubeadm cluster + SK Broadband + Cloudflare Related repos: 302-goormgb-k8s-bootstrap, 303-goormgb-k8s-helm