If you've spent any time working with Kubernetes, you know that the API server is the heart of the cluster. When you start writing your own Go binaries to interact with it—maybe a custom controller or a cleanup script—it's tempting to treat the Kubernetes API like a standard REST API. I've seen plenty of engineers start by writing a loop that calls List() on a set of Pods every few seconds to see if anything has changed. It feels intuitive, but in a production cluster, this is a recipe for a very angry SRE message in your Slack DMs.

Polling the API and the 'Cluster-Slayer' approach

Let's look at the naive way. Imagine we're building a service that monitors Pods with the label app=payment-gateway to ensure we have a minimum number of healthy replicas across different nodes. The "wrong" way looks something like this:

for {
    pods, err := clientset.CoreV1().Pods("default").List(context.TODO(), metav1.ListOptions{
        LabelSelector: "app=payment-gateway",
    })
    if err != nil {
        log.Printf("Error listing pods: %v", err)
    }
    processPods(pods.Items)
    time.Sleep(10 * time.Second)
}

On the surface, this works. But consider what's happening under the hood. Every ten seconds, your binary is hitting the API server, which in turn queries etcd. If you have a few hundred pods, it's fine. If you scale to thousands, or if you have ten different micro-services all polling the API this way, you're effectively DDoS-ing your own control plane. You're wasting bandwidth sending the same giant JSON blob over the wire every few seconds, and you're introducing a latency gap—if a pod crashes one second after your loop runs, you won't know about it for another nine seconds.

Trading Memory for API Sanity with Informers

The professional way to handle this in client-go is through an Informer. Instead of asking the API "What is the state of the world right now?" every few seconds, an Informer asks the API "Tell me everything once, and then tell me only when something changes."

An Informer maintains a local cache of the resources it's watching. When you want to "list" pods, you aren't hitting the API server; you're reading from a local Go map. This shifts the burden from the network and the API server to your application's memory. In 99% of cases, this is a trade-off you want to make. Here is how we actually structure this using a SharedInformerFactory:

factory := informers.NewSharedInformerFactory(clientset, time.Minute*30)
podInformer := factory.Core().V1().Pods().Informer()

podInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{
    AddFunc: func(obj interface{}) {
        pod := obj.(*v1.Pod)
        fmt.Printf("New Pod added: %s\n", pod.Name)
    },
    UpdateFunc: func(oldObj, newObj interface{}) {
        newPod := newObj.(*v1.Pod)
        fmt.Printf("Pod updated: %s\n", newPod.Name)
    },
    DeleteFunc: func(obj interface{}) {
        pod := obj.(*v1.Pod)
        fmt.Printf("Pod deleted: %s\n", pod.Name)
    },
})

stopCh := make(chan struct{})
defer close(stopCh)
factory.Start(stopCh)

The magic here is the AddEventHandler. You're no longer polling; you're reacting. Your code becomes event-driven. The API server pushes a "Watch" event to you, and the Informer updates the local cache and triggers your callback. It's faster, leaner, and won't crash your cluster.

The Nuance of the 'Shared' Factory

You'll notice I used NewSharedInformerFactory rather than a standalone informer. This is a detail that often trips people up. In a complex Go program, you might have five different components that all need to know about Pods. If each component created its own Informer, you'd have five separate watch connections and five duplicate caches of the same Pods in memory.

A SharedInformerFactory ensures that only one watch connection is maintained for a specific resource type, regardless of how many different event handlers are listening. I always recommend using the shared factory by default. It keeps your memory footprint predictable and reduces the load on the API server even further. Just remember that the factory doesn't actually start the watches until you call factory.Start(stopCh). If you forget that line, your handlers will never fire, and you'll spend an hour wondering why your code is "silently failing."

📋 Practical Task

Exercise: Building a Label-Filtered Pod Event Logger

Your goal is to create a Go program using client-go that monitors the cluster and logs specific events, but only for Pods that carry the label environment=production.

Requirements:

• Initialize a SharedInformerFactory with a resync period of 10 minutes.
• Implement a ResourceEventHandler that prints a message to the console whenever a Pod with the environment=production label is Added or Deleted.
• If a Pod is added that does not have that label, the program should ignore it silently.
• Ensure the program stays running (doesn't exit immediately) until you manually terminate it (Ctrl+C).
• Use clientcmd to load your local ~/.kube/config for authentication.

Expected Output Example:
[EVENT] Production Pod Created: payment-api-7f8d9
[EVENT] Production Pod Deleted: auth-service-1a2b3