Kubernetes Networking for Beginners

Kubernetes Networking for Beginners

Published on
Authors

1. Pod-to-Pod Communication

  • What it is: Every Pod gets its own unique IP address in the cluster.
  • How it works:
    • Same node → direct via local bridge (cbr0 or similar).
    • Different nodes → traffic routed via the cluster network (CNI plugin).
  • Common Confusion:
    • “Why can’t I ping one Pod from another?” → Check if CNI is working or if NetworkPolicy is blocking.
    • Pods on same node share a network namespace? → No, each Pod has its own unless using hostNetwork or shareProcessNamespace.

2. Cluster Networking (CNI Plugins)

  • What it is: Kubernetes doesn’t provide networking — it delegates to CNI plugins.
  • Popular CNI plugins:
    • Flannel → simple overlay (VXLAN)
    • Calico → BGP + optional NetworkPolicy
    • Cilium → eBPF-powered, fast & secure
  • Common Confusion:
    • “I deployed Kubernetes, why no Pod IPs?” → CNI plugin not installed or misconfigured.
    • Thinking Kubernetes manages IPs → No, CNI does.

3. Service Abstraction (The Magic Glue)

  • What it is: A stable IP + port (ClusterIP) that points to dynamic Pods.
  • How it works: kube-proxy maintains iptables/nftables or uses IPVS to forward traffic.
  • Common Confusion:
    • “My Pod died, why is the app still reachable?” → Because Service routes to healthy Pods.
    • “I see traffic going to wrong Pod” → Check selector labels match.

4. ClusterIP vs NodePort vs LoadBalancer

Type Scope Access Use Case
ClusterIP Internal only 10.x.x.x Backend APIs, DB
NodePort External via node NodeIP:3xxxx Testing, no cloud LB
LoadBalancer External via cloud Public IP Production apps
  • Common Confusion:
    • NodePort opens port on all nodes → even if no Pod is there.
    • LoadBalancer not working? → Only works in supported cloud environments.

5. DNS in Kubernetes (CoreDNS)

  • What it is: Built-in DNS server that resolves:
    • my-svc.my-namespace.svc.cluster.local → Service IP
    • pod-ip-with-dashes.my-namespace.pod.cluster.local → Pod IP (if enabled)
  • Common Confusion:
    • “Why can’t I use just my-svc?” → Need full FQDN or be in same namespace.
    • DNS not resolving? → Check CoreDNS pods are running and Pod’s dnsPolicy.

6. NetworkPolicies (Firewall for Pods)

  • What it is: Rules to allow/deny traffic between Pods (like security groups).
  • Default: All Pods can talk to each other.
  • Example: 
    allow frontend → backend on port 8080
  • Common Confusion:
    • “I applied NetworkPolicy, still open!” → Need a CNI that supports it (Calico, Cilium, etc.).
    • Forgetting to allow DNS → Pods can’t resolve names.

7. Ingress & Ingress Controller

  • Ingress: YAML that defines HTTP routing rules (paths, hosts, TLS).
  • Ingress Controller: Actual reverse proxy (NGINX, Traefik, HAProxy) that reads Ingress and serves traffic.
  • Common Confusion:
    • “I created Ingress, but 404!” → No Ingress Controller running.
    • Thinking Ingress = LoadBalancer → No, it needs a Service (usually LoadBalancer) behind it.

8. Egress Traffic Control

  • What it is: Controlling what Pods can access outside the cluster.
  • Methods:
    • NetworkPolicy (egress rules)
    • Egress Gateway (NAT + routing)
    • Firewall appliances
  • Common Confusion:
    • “Why can’t my Pod curl google.com?” → Egress blocked or no NAT gateway.
    • Thinking all traffic goes through node → Usually via cluster NAT.

9. Service Mesh (Istio, Linkerd, etc.)

  • What it adds:
    • mTLS encryption
    • Retries, timeouts, circuit breaking
    • Observability (traces, metrics)
  • How: Sidecar proxy (Envoy) injected into Pods.
  • Common Confusion:
    • “Too slow after enabling mesh!” → Sidecar adds latency (~1–5ms).
    • Thinking it replaces CNI → No, runs on top of CNI.

10. Host Networking Mode (hostNetwork: true)

  • What it is: Pod uses node’s network stack (no isolation).
  • Use case: DaemonSets (CNI agents, monitoring), high-performance needs.
  • Common Confusion:
    • “Why can’t I use Pod IP?” → It uses node IP.
    • Port conflicts → Pod can clash with node services.

11. Pod IP vs Cluster IP vs External IP

  • Pod IP: Ephemeral, per Pod, internal.
  • Cluster IP: Virtual, stable, internal (Service).
  • External IP: Cloud LB or NodePort public IP.
  • Confusion: Mixing them in configs → connection refused.

12. kube-proxy Modes (iptables vs IPVS vs eBPF)

  • iptables: Default, simple, can slow with 1000+ services.
  • IPVS: Faster, better for large clusters.
  • eBPF (Cilium): Fastest, programmable.
  • Confusion: “Why is load balancing slow?” → Still on iptables with 500 services.

13. Pod Readiness & Liveness vs Network Readiness

  • ReadinessGate (with PodReadyToStartContainers) lets CNI signal when networking is ready.
  • Why it matters: Pod marked “Ready” before IP is assigned → traffic drops.
  • Confusion: “Service sends traffic to Pod with no IP!” → Use readiness gates with Cilium/Calico.

Cheers,

Sim