I get the following message when doing a cluster init if I use the garden nginx ingress controller: "container controller specifies host ports [80 443], which are disallowed in Autopilot." I have tried following the kubernettes.io instructions for using nginx ingress, setting it up as a kubernetes module, but that requires adding custom args to the deployment - which I can't find a way to do. I can create the service this way, but it appears to be missing a link to the exposed services config map as its not actually forwarding requests and in the gke console it shows no mapped (https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/) Do I need to manage the load balancer/ingress directly with kubectl? Is there some other magic at play here?

Hi @dry-cartoon-14056 I didn't encounter this the last time I set up an Autopilot cluster with Garden. I will create a new one tomorrow (it's the end of my work day now) and let you know my findings

Thanks for looking into it. I have to admit I'm really getting frustrated with AWS and GCP and what feels like an illusion of service portability between them. If I "own" the infra, its easy enough to spin up anything I want in the infra I manage, but trying to use the PaaS offerings so far has been very very painful. Spinning up my own cluster on cloud compute VM's just doesn't seem like ti should be the right answer to multi-cloud portability. I really like the appeal of a mostly auto-provisioning cluster which i theory we should get from GCP Autopilot or AWS Fargate, but maybe its not meant to be.

I feel you @dry-cartoon-14056 . You can get a bit more portability when you use standart GKE or EKS clusters, but even there are some lock-ins. Garden works out of the box with standard GKE clusters, if you want to take the easy route here (you can still enable autoscaling). However feels like we should really create an example with autopilot and Garden that documents all the needed config.

I was trying to switch to an Autopilot cluster to keep costs down and I'm running into some different issues. I'm running

garden plugins kubernetes cluster-init --env=dev

and I get this error: https://gist.github.com/salotz/51e63dabc5bc8a5a1b6e7d450e51c505

@orange-ability-1812 I think it's safe to say Garden just isn't a good fit for Autopilot clusters. We could think about tracking aggregate issues with Autopilot in a "Support GKE Autopilot clusters" issue request. If you're looking to save you may be interested in this "Run a Google Kubernetes Engine Cluster for Under $25/month" article https://thenewstack.io/run-a-google-kubernetes-engine-cluster-for-under-25-month/

I ended up going back to a standard GKE. It is what is I guess. Thanks for the guide a few good tips in there. I ended up just enabling autoscaling, making it zonal, and taking a little more care in setting my cluster up which helped.

That makes sense. Yeah sadly ATM our ingress doesn't work in Autopilot. If you are interested in seeing this feature feel free to add a Feature Request to Github so our team can work on that one! ❤️