Kubernetes

Implement the IPmode Proxy in the cloud-controller-manager to solve the problem of flow to a load balancer from a pod in the case of using the proxy protocol, see https://github.com/digitalocean/digitalocean-cloud-controller-manager/issues/811

DOKS installs the Gateway API CRDs as part of the Cilium addon and its reconciler owns them. They have been pinned at bundle v1.2.1 since our clusters were created. Upstream Gateway API is at v1.6.0. Support has confirmed there is no way to opt out, and that overriding the version is unsupported and may break the cluster on a patch or minor upgrade. The problem is that Gateway API is not a DigitalOcean resource. It is a standard upstream extension API, and it is how you are supposed to run ingress on Kubernetes now. Any Gateway API controller other than the one DOKS ships needs CRDs newer than v1.2.1. Envoy Gateway 1.9 watches TLSRoute, TCPRoute, UDPRoute and ListenerSet at v1. Three of those four do not exist in the DOKS set at any version. So in practice DOKS today supports exactly one Gateway API implementation, at one version, chosen by DigitalOcean. We hit this migrating our ingress from Traefik to Envoy Gateway. We can install newer CRDs, and it works, but we have no guarantee it survives the next cluster upgrade, and the resources behind those CRDs are our whole routing table. Support could not tell us what happens to a cluster when their reconciler fails to revert them, beyond "it doesn't function properly". What we would like is simple: a label or annotation on the cluster, or a flag at creation, that turns off DOKS management of the Gateway API CRDs and hands ownership to us. Same idea as opting out of any other addon. If we take ownership we accept responsibility for keeping them compatible with Cilium. Alternative, if opting out is not on the table: track upstream. v1.2.1 shipped in late 2024. Sitting four minor versions behind on a core networking API, with no override path, is a hard blocker for anyone who needs a second Gateway controller. Either would work. Right now there is no supported path at all, and that is the part that needs fixing.

Currently DOKS comes with a Cilium gateway implementation out of the box. However, Cilium doesn't support proxy protocol to preserve the client IP address. There are plenty of other gateway implementations that do support proxy protocol, but since DOKS comes with the outdated Gateway API CRD's bundle 1.2.1, these alternatives can't be installed (or you need to install a version that is end-of-life). When updating the Gateway API CRD's manually, DOKS cluster updates won't work anymore. So this isn't an option either. An alternative would be to use a regional network load balancer instead, but these load balancers don't support IPv6. Any solution would do but preferably DOKS comes with up-to-date Gateway API CRD's.

Even though Kubernetes does not support IPv6, pods running with host networking enabled can make use of the IPv6 address. IPv6 should be enabled by default or there can be an option to enable it for a Kubernetes node pool.

Currently DigitalOcean Managed Kubernetes doesn't provide any methods to track down control plane logs for auditing. Audit logs are highly necessary and useful for security-related issues and situations especially on production environments. This should be one of the first priority feature to be added on DOKS to let user keep their services safe and fully trust DOKS as a production-ready platform.

Since DOKS already uses Cilium as its CNI, it would be helpful to join multiple DOKS clusters together with Cilium Cluster Mesh. This is especially nice to have a super cluster spread across multiple data centers or regions, with telemetry that encompasses everything.

Was super excited to play around with cilium and hubble when the dashboard feature was expired. Meanwhile, I thought of: is there possibility of exporting the metrics to prometheus from the hubble! It would be nice to have that feature - which enable monitoring various aspect of the clusters.

We would love to see support for a Kubernetes Private Cluster where the following configurations are applied: • No Public IPs for Nodes: Ensure that nodes in the cluster don’t receive any public IPs, improving security by keeping the internal infrastructure isolated from external networks. • Service Exposure via LoadBalancer: Allow services to be exposed securely using a LoadBalancer, providing controlled access without the need for public-facing nodes. This setup would enable more secure, private clusters while still allowing external traffic to reach specific services via managed, secure entry points (LoadBalancers). It’s the perfect blend of security and usability, ensuring that sensitive infrastructure remains isolated while still making applications accessible as needed. Would love to see this feature added for better flexibility and security in Kubernetes deployments!

Cilium includes built-in WireGuard transparent encryption for securing pod-to-pod and node-to-node traffic, but this feature isn’t available on DOKS today. Adding an option to enable it at the cluster level would make it much easier to improve in-cluster security without extra components or custom setups.

Add ability to scale Kubernetes across multiple regions.