BGP

BGP is how we'll clients external to kubernetes will access our workloads, and for that we will need a pool of ip addresses to announce into the internal routing mesh.

BGP Policy

Cilium natively support BGP to announce service addresses, either iBGP or eBGP.

This is a sample policy that creates one BGP sessions per node Multiple sessions are possible, but at this point an unnecessary complication.

Adjust the ASN and peer addresses below. There is currently no BFD support, so also adjust timers as appropriate for your network.

Click the for more information.

cat <<EOF | kubectl apply -f -
apiVersion: "cilium.io/v2alpha1"
kind: CiliumBGPPeeringPolicy
metadata:
  name: bgp-peering-policy
  namespace: kube-system
spec:
  virtualRouters:
  - localASN: 65552
    exportPodCIDR: false (1)
    serviceSelector: (2)
      matchExpressions:
      - {key: somekey, operator: NotIn, values: ['never-used-value']}
    podIPPoolSelector: (3)
      matchExpressions:
      - {key: somekey, operator: NotIn, values: ['never-used-value']}
    neighbors:
    - peerAddress: '192.168.1.11/32'
      peerASN: 65552
      eBGPMultihopTTL: 10
      connectRetryTimeSeconds: 120
      holdTimeSeconds: 90
      keepAliveTimeSeconds: 30
      families:
        - afi: ipv4 (4)
          safi: unicast
      gracefulRestart:
        enabled: true
        restartTimeSeconds: 120
EOF

Advertise the entire pod CIDR range, or not
By default, Cilium will not announce any services, specify a match statement to advertise, in this case a non-existant values means every service
By default, Cilium will not announce any ippools, specify a match statement to advertise, in this case a non-existant values means every ippool
Restrict to IPv4, multi-protocol is possible but messy. To keep simple we will create one session limited to IPv4. Create a second peer with afi: IPv6 for IPv6 announcements.

Examine our BGP state

cilium bgp peers

See the asciicast below: