How Linux Processes Network Traffic on Localhost

Wide Area Network Routing & Gateway Mechanics

#

When an application sends a packet to a remote host outside the local subnet (e.g., 8.8.8.8), the transmit and receive pipelines interact through Layer 2 / Layer 3 split-identity routing.

1. Off-Network Transmit (TX)

#

When the destination IP does not belong to the local network range:

1. Routing Table Lookup: The IP layer (ip_queue_xmit()) evaluates the target IP against the local routing table. Instead of delivering directly to the target IP, it selects the Default Gateway (e.g., 192.168.1.1) as the next-hop address.
2. Neighbor Subsystem Resolution: The Neighbor Subsystem queries its ARP cache (arp_tbl) for the Default Gateway’s MAC address, rather than the remote target’s MAC address. If unmapped, it queues the sk_buff and broadcasts an ARP request across the local segment.
3. Split-Identity Frame Framing: The Neighbor Subsystem constructs an Ethernet frame with two distinct addresses:
   • Layer 3 (IP Header): Destination IP = Remote Target IP (8.8.8.8)
   • Layer 2 (Ethernet Header): Destination MAC = Local Gateway MAC (00:11:22:33:44:55)
4. Physical Send: The frame is written to the driver TX ring buffer via DMA and transmitted across the wire to the local router.

flowchart LR
    subgraph ClientHost["User / Client Host (TX Path)"]
        direction LR
        App1["User Process (send)"] --> IP1["IP Layer / Route Lookup"]
        IP1 --> Neigh1["Neighbor Subsystem"]
        Neigh1 --> Ring1["TX Ring Buffer"]
    end

2. Intermediate WAN Traversal

#

Once the local router receives the physical frame:

1. L2 Decapsulation: The router sees its own MAC address in the Ethernet header, strips the L2 framing, and inspects the inner L3 packet header.
2. Hop-by-Hop Forwarding: The router reads the destination IP (8.8.8.8), checks its WAN routing tables, decrements the IP TTL, and rewrites new L2 headers for each intermediate hop across the internet.

   flowchart LR
   subgraph WAN["Internet / Routers"]
           direction LR
           GW1["Local Default Gateway"] --> Routers["Intermediate Hops"]
           Routers --> GW2["Remote Edge Router"]
       end
   

3. Remote Ingress (RX) Arrival

#

When the frame reaches the remote network segment containing 8.8.8.8:

1. Last-Hop Resolution: The final edge router uses its Neighbor Subsystem (ARP/NDP) to resolve the MAC address of 8.8.8.8 and delivers the frame onto the remote host’s physical network segment.
2. Hardware Ingress: The remote host’s NIC receives the electrical/optical signal and writes raw bytes to its RX Ring Buffer via DMA.
3. HardIRQ to SoftIRQ Switch: The remote NIC triggers a hardware interrupt, scheduling NET_RX_SOFTIRQ and ksoftirqd to strip L2/L3 headers, perform GRO reassembly, evaluate Netfilter rules, and queue the payload inside the target socket’s sk_receive_queue.

   flowchart LR
    subgraph ServerHost["Remote Server Host (RX Path)"]
           direction LR
           Ring2["RX Ring Buffer (DMA)"] --> IRQ["HardIRQ / SoftIRQ"]
           IRQ --> IP2["Netfilter / IP Layer"]
           IP2 --> App2["Server Process (recv)"]
       end
   

Layer Identity Across the Wire

#

Localhost & Loopback (lo) Traffic Mechanics

#

When an application communicates with another process on the same machine—whether using 127.0.0.1 or the host’s assigned physical interface IP (e.g., 192.168.1.50)—the kernel bypasses physical network hardware entirely while retaining protocol stack guarantees.

1. Local Route Lookup & Address Identity

#

During interface initialization or DHCP address assignment, the kernel populates the local routing table:

# View local routing table entries managed by the kernel
ip route list table local

• Local Address Normalization: When the kernel evaluates an IP packet’s destination, it checks the local route table first. Any IP bound to a local interface (e.g., 127.0.0.1, 192.168.1.50) routes directly to the virtual loopback driver (lo).

• Performance Myth (127.0.0.1 vs. Local IP): Using 127.0.0.1 offers no performance advantage over connecting to the machine’s assigned IP (192.168.1.50). Both resolve to the same local table rules and traverse the exact same loopback execution path.

• Troubleshooting with tcpdump: Because all local IP traffic routes through the virtual loopback device, capturing local traffic requires listening explicitly on lo:

Capture local traffic directed to a local port

#

tcpdump -i lo port 8080

2. High Loopback MTU & Large Frames

#

While physical Ethernet interfaces typically enforce a Maximum Transmission Unit (MTU) of 1500 bytes, the virtual loopback interface defaults to 65,536 bytes (64KB).

• Reduced Fragmentation Overhead: Because the MTU is vastly larger, the IP layer rarely needs to fragment packets on lo.

• Lower CPU Overhead: Transport protocols (TCP/UDP) can process significantly larger segment sizes in a single pass, drastically reducing per-packet allocation churn (sk_buff slicing) and softirq overhead.

3. Loopback Transmit Optimization: Skipping Driver Queues & DMA

#

Unlike network flows directed across physical hardware, loopback execution eliminates hardware queues and DMA operations entirely:

    flowchart TD
    subgraph ClientHost["Local Host (Loopback Path)"]
    direction TD
    App1["App (send)"] --> TCP1["TCP / IP Layer"]
    TCP1 --> LoDriver["Loopback Driver (lo)"]
    LoDriver -->|Enqueue to per-CPU backlog| SoftIRQ["NET_RX_SOFTIRQ"]
    SoftIRQ --> TCP2["TCP / IP Layer"]
    TCP2 --> App2["App (recv)"]
    end

• Hardware Queue & Ring Buffer Bypass: The packet bypasses the software egress queue (qdisc), the hardware doorbell write, and the TX/RX Ring Buffers.
• Direct Enqueue to input_pkt_queue: Instead of triggering a physical DMA transfer, the loopback driver (loopback_xmit) takes the sk_buff directly and places it into the current CPU’s input_pkt_queue (process backlog).
• No Hardware Interrupts: Physical NIC interrupts (HardIRQ) are completely skipped. The loopback driver directly schedules a deferred software interrupt (NET_RX_SOFTIRQ) to initiate immediate receive processing.

4. Memory Allocations & Stack Overhead Still Apply

#

Although loopback I/O skips physical hardware, it is not completely overhead-free:

• Socket Buffer Churn: Transport layers still allocate memory for socket buffers (sk_buff), copy user data from user space to kernel space, and free buffers upon completion.

• Full Protocol Processing: Packets must still traverse TCP state machines, windowing checks, IP header generation, and Netfilter firewall hooks (iptables).