Local docker registry
b
blue-kite-93685
03/03/2023, 10:48 AMI'm having some trouble figuring out how garden builds interact with docker registries.
I'm trying to develop entirely locally for the moment. Whether I use local-docker or cluster-buildkit, my deploys fail with "Error response from daemon: pull access denied for image, repository does not exist or may require 'docker login'", which doesn't seem right. The images in question do visibly exist as far as my host docker daemon is concerned. What do I need to do for garden to be able to deploy them?
I tried setting
deploymentRegistry:
hostname: localhost
port: 5000
insecure: true
in my provider but this doesn't seem to have helped, it's still using an unqualified image name in the deployment and at a guess it's trying to pull from docker.io. How can I confirm that? How can I change that?
I am realising garden may be trying to use an internal cluster registry instead, except it doesn't exist. I've ran garden plugins local-kubernetes cluster-init in my local environment, but that hasn't made it exist. It put a configmap and a serviceaccount in my garden-system namespace, but nothing else. What am I missing? I do not have a deploymentRegistry defined, and I have tried this with a completely blank environment several times.
b
bright-policeman-43626
03/03/2023, 3:33 PMHey @blue-kite-93685 I’m so sorry you are getting this issues, let’s try to find the root cause for this one!
Would you mind sharing the project configuration you have for this deployment?
Thank you!🌸
b
blue-kite-93685
03/03/2023, 3:35 PMb
bright-policeman-43626
03/03/2023, 3:44 PMplease let me give it a quick test if you don't mind!
I'm going to add a quick dockerfile that bootstrap a react application to test this one! Will update in a couple of minutes.
b
blue-kite-93685
03/03/2023, 3:50 PMthe cluster-init output doesn't really seem to suggest it's trying to create an in-cluster registry
b
bright-policeman-43626
03/03/2023, 4:08 PMSorry one more question, to run docker are you currently using
docker-desktopb
blue-kite-93685
03/03/2023, 4:08 PMYep
Rancher-desktop is providing a k3s cluster with docker desktop as the container engine
I can't work out what the intended behaviour is meant to be to be honest. What does local-docker use by default to get images to the cluster? I'm slightly surprised it needs to pull anything per se
b
bright-policeman-43626
03/03/2023, 4:09 PMDo you mind checking if you are using dockerd for rancher k3s?
b
blue-kite-93685
03/03/2023, 4:10 PMb
bright-policeman-43626
03/03/2023, 4:10 PMdo you mind doing a
garden delete environmentrm -fr .gardenb
blue-kite-93685
03/03/2023, 4:10 PMI've been doing that all day ;p suresure.
b
bright-policeman-43626
03/03/2023, 4:10 PMAnd remove the
buildMode: local-dockerb
blue-kite-93685
03/03/2023, 4:11 PMoh. just no buildmode? haven't tried that
b
bright-policeman-43626
03/03/2023, 4:12 PMYes please try that, usually when I'm local I don't have to set that up and it works, maybe try doing the cleanup and then deploy without that!
b
blue-kite-93685
03/03/2023, 4:14 PMnm, didn't do that quite right
(taking a lil while to take the env down, bear with me.)
b
bright-policeman-43626
03/03/2023, 4:20 PMno worries! I'm here!
b
blue-kite-93685
03/03/2023, 4:22 PMmmyeah. much the same.
b
bright-policeman-43626
03/03/2023, 4:23 PMcan you please do
cat ~/.docker/config.jsonb
blue-kite-93685
03/03/2023, 4:25 PMnot much of interest set up - what're you looking for? I'm on windows, so it's using the windows crential store
there's no entry for e.g. an in-cluster registry if that's the idea
b
bright-policeman-43626
03/03/2023, 4:34 PMI was trying to see if maybe there is a misconfiguration of the registries, it seems like yours is trying to go directly to docker hub instead of locally.
Have you tried
kanikob
blue-kite-93685
03/03/2023, 4:36 PMPulling unqualified names from docker.io is default behaviour afaik
b
bright-policeman-43626
03/03/2023, 4:36 PMI was able to reproduce the issue you are having but to be fair I have a really conflicted installation of Docker atm as I installed docker-desktop but also have docker running at the system level in Linux
b
blue-kite-93685
03/03/2023, 4:36 PMheh
I have not played with kaniko yet, I'll give that a try. I did try cluster buildkit, which didn't seem to change much, but I'm not sure I gave it an adequate reset
b
bright-policeman-43626
03/03/2023, 4:38 PMyeah when resetting the stuff try to always delete the metadata, I'm currently testing kaniko, It seems like it installed the registry in the cluster locally successfully.
b
blue-kite-93685
03/03/2023, 4:47 PMI ran the self-update along the way and now it's just giving me
Path C:\Users\<user>\.garden\bin\static is not in a git repository root.b
bright-policeman-43626
03/03/2023, 4:49 PMI'm so sorry that this is being to painful, I don't have a lot of experience with Windows, but maybe the self-update command did not properly installed the new binary?
b
blue-kite-93685
03/03/2023, 4:51 PMreplaced with the backup. crisis averted..
b
bright-policeman-43626
03/03/2023, 4:51 PMHey good news, Kaniko ended up working for me. I just followed the steps:
1.
garden delete environmentrm -fr .gardenkanikogarden deploydo you mind trying the above?
b
blue-kite-93685
03/03/2023, 4:51 PMyepyep
b
bright-policeman-43626
03/03/2023, 4:54 PMI can upload my whole project if you want to take a look?
b
blue-kite-93685
03/03/2023, 4:58 PM🎉 All green
Thanks a bunch for your help 😛
b
bright-policeman-43626
03/03/2023, 4:59 PMNo worries! I hope the best for your project and thanks for using Garden! 🌸