kubeconfig, contexts, and multiple clusters

We've talked about kubeconfig before, and you've seen commands such as

kubectl config set-credentials oidc \
  --exec-api-version=client.authentication.k8s.io/v1beta1 \
  --exec-command=kubectl \
  --exec-arg=oidc-login \
  --exec-arg=get-token \
  --exec-arg=--oidc-issuer-url=https://auth.mydomain \
  --exec-arg=--oidc-client-id=402acfd0-fad4-42fd-bfe7-69e7827ec5b3 \
  --exec-arg=--oidc-client-secret=kYWetlqktCObArnE.13a9MYgpRSylAnXxIwyvPpsj0Jj_V~rK3XMVuf.AtlJ98kBBOK4s2gY

kubectl config set-context --current --user=oidc

Remember that a kubeconfig context consists of cluster hostnames, ports, certificates, namespaces and users. So the above command creates a user in kubeconfig called oidc with the supplied parameters. It then sets the current context to use oidc as the default user when connecting to a cluster.

Our current kubeconfig looks like this:

kubectl config get-contexts
CURRENT   NAME          CLUSTER       AUTHINFO   NAMESPACE
*         k0s-cluster   k0s-cluster   oidc

Pretty simple! In reality, you're likely to be using multiple clusters throughout the day, or even at the same time and you'll have a kubeconfig that looks more like this:

CURRENT   NAME                    CLUSTER                 AUTHINFO           NAMESPACE
          amazon-eks              amazon-eks              aws-iam            default
*         devcluster              devcluster              admin-devcluster
          prod-cluster-direct     prod-cluster-direct     oidc-k0s           default
          prod-cluster-fallback   prod-cluster-fallback   admin              default
          prod-cluster-ts         prod-cluster-ts         oidc-k0s           default
          uat-cluster             uat-cluster             oidc               default

If you want to add --context to every kubectl command, then you're good to go like that. That gets quite tedius and there's a better way, kubectx and kubens.

They are plugins we install once again via krew,

kubectl crew install ctx
kubectl crew install ns

Now you can use kubectl ctx to set the current context to operate on, and kubectl ns to set the namespace, like so:

kubectl ctx devcluster
kubectl ns kube-system
kubectl get pods

See the asciicast below, demonstrating how ctx, ns and commandline flags work together.