https://garden.io logo
Join Discord
Powered by
Remote kubernetes garden-docker-registry unbound p...
# 🌱｜help-and-getting-started
a
Hi all, I have a remote kubernetes cluster setup, and am trying to deploy my stack against it with garden, as I've been doing locally with minikube, but am having some issues. I'm using this guide: https://docs.garden.io/v/acorn-0.12/kubernetes-plugins/remote-k8s (but I skipped step 2 and 3, as I want to build on cluster without an external docker registry, and don't think I need a hostname as I'll just use portforwards to access the services I deploy). There will only be one garden environment on this cluster. Issue I'm having is when I run 
garden --env=remote plugins kubernetes cluster-init
, I see it creates the 
garden-system
resources on the cluster, but the 
garden-docker-registry
pod is pending. Looking into it, it appears a PVC its using (
garden-docker-registry
) is also pending, with the following event:
Copy code
0/4 nodes are available: 4 pod has unbound immediate PersistentVolumeClaims. preemption: 0/4 nodes are available: 4 Preemption is not helpful for scheduling.
I don't see anything in the guide about setting up persistent volumes. Do I need to do something get this pvc to work?
q
Hi @average-painter-87026 what service provider have you chosen for your Kubernetes cluster? I'd 
get
your 
pv
pvc
and 
storageclass
, and make sure they're available.
Have you set up an in-cluster build mode? If so, I'd check https://docs.garden.io/v/acorn-0.12/reference/providers/local-kubernetes#providers-.storage.registry and make sure your storage is configured correctly to your provider.
a
@quaint-dress-831 Sorry, got side tracked with work but finally back around to this. I was able to get the in-cluster build pods to run by creating a persistent volume on the cluster (attached) and the 
garden-system/garden-docker-registry
PVC bound itself to it https://cdn.discordapp.com/attachments/1118609692751564860/1123649968100606063/remote-garden-build-pv.yaml
I'm using the 
cluster-buildkit
, and the build seems to work but there are issues now when it tries to push the built image to it's local registry. The 
garden build <service>
logs are showing 
500 Internal Server Error
when pushing the image, and I see similar error in 
garden-buildkit
pod. I've attached both. The 
garden-docker-registry
pod (in 
default
namespace) is logging this error: 
filesystem: mkdir /var/lib/registry/docker: permission denied
. I've attached those logs as well. It seems like the garden-docker-registry service isn't able to write to that hostPath persistentVolume (in previous post), even though it has RWO access modes? I've tried using 
/tmp/garden-build
for the hostPath of the PV, but still getting the permission issue so I'm not totally sure. Do you have any ideas what I should be doing here? Thank you https://cdn.discordapp.com/attachments/1118609692751564860/1123651419216556122/garden-build-service.log https://cdn.discordapp.com/attachments/1118609692751564860/1123651419522736279/garden-docker-registry.log https://cdn.discordapp.com/attachments/1118609692751564860/1123651419866681454/garden-buildkit.log
q
@average-painter-87026 can you please just dump exactly what you've written into a GitHub Issue? This looks like a bug we need to look into.
In the meantime, if you're able to switch to another local K8s provider we support like MicroK8s or kind, that should unblock you for now.
More on supported local flavors and instructions for each at https://docs.garden.io/kubernetes-plugins/local-k8s/install
@average-painter-87026 I am setting this to resolved since it's been some time since I've last heard from you. If this is still an issue, just ping me here 😄
14 Views
PreviousNext
Powered by