https://garden.io logo
#🌱|help-and-getting-started
Does anyone have a recipe for getting ingress to work on GKE with an autopilot cluster?
# 🌱|help-and-getting-started
d

dry-cartoon-14056

03/27/2023, 3:10 PM
I get the following message when doing a cluster init if I use the garden nginx ingress controller: "container controller specifies host ports [80 443], which are disallowed in Autopilot." I have tried following the kubernettes.io instructions for using nginx ingress, setting it up as a kubernetes module, but that requires adding custom args to the deployment - which I can't find a way to do. I can create the service this way, but it appears to be missing a link to the exposed services config map as its not actually forwarding requests and in the gke console it shows no mapped (https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/) Do I need to manage the load balancer/ingress directly with kubectl? Is there some other magic at play here?
q

quaint-dress-831

03/27/2023, 4:03 PM
Hi @dry-cartoon-14056 I didn't encounter this the last time I set up an Autopilot cluster with Garden. I will create a new one tomorrow (it's the end of my work day now) and let you know my findings
@dry-cartoon-14056 I tested with my own Autopilot cluster and get the same errors as you. Between this https://github.com/garden-io/garden/issues/3470, https://github.com/garden-io/garden/issues/3469, and https://github.com/garden-io/garden/issues/3072 it's safe to say that the friction to using Autopilot with Garden is high. You can use GKE's own ingress controller with the workaround in 3072 but please only do so armed with the knowledge of @freezing-pharmacist-34446's advice contained in the same issue.
d

dry-cartoon-14056

03/28/2023, 2:47 PM
Thanks for looking into it. I have to admit I'm really getting frustrated with AWS and GCP and what feels like an illusion of service portability between them. If I "own" the infra, its easy enough to spin up anything I want in the infra I manage, but trying to use the PaaS offerings so far has been very very painful. Spinning up my own cluster on cloud compute VM's just doesn't seem like ti should be the right answer to multi-cloud portability. I really like the appeal of a mostly auto-provisioning cluster which i theory we should get from GCP Autopilot or AWS Fargate, but maybe its not meant to be.
f

freezing-pharmacist-34446

03/28/2023, 3:45 PM
I feel you @dry-cartoon-14056 . You can get a bit more portability when you use standart GKE or EKS clusters, but even there are some lock-ins. Garden works out of the box with standard GKE clusters, if you want to take the easy route here (you can still enable autoscaling). However feels like we should really create an example with autopilot and Garden that documents all the needed config.
o

orange-ability-1812

04/28/2023, 10:10 PM
I was trying to switch to an Autopilot cluster to keep costs down and I'm running into some different issues. I'm running
garden plugins kubernetes cluster-init --env=dev
and I get this error: https://gist.github.com/salotz/51e63dabc5bc8a5a1b6e7d450e51c505
q

quaint-dress-831

05/02/2023, 10:31 AM
@orange-ability-1812 I think it's safe to say Garden just isn't a good fit for Autopilot clusters. We could think about tracking aggregate issues with Autopilot in a "Support GKE Autopilot clusters" issue request. If you're looking to save you may be interested in this "Run a Google Kubernetes Engine Cluster for Under $25/month" article https://thenewstack.io/run-a-google-kubernetes-engine-cluster-for-under-25-month/
o

orange-ability-1812

05/02/2023, 4:46 PM
I ended up going back to a standard GKE. It is what is I guess. Thanks for the guide a few good tips in there. I ended up just enabling autoscaling, making it zonal, and taking a little more care in setting my cluster up which helped.
b

bright-policeman-43626

05/02/2023, 7:41 PM
That makes sense. Yeah sadly ATM our ingress doesn't work in Autopilot. If you are interested in seeing this feature feel free to add a Feature Request to Github so our team can work on that one! ❤️
80 Views