TL;DR: How do I pass an id_rsa secret to a Dockerfile being built in a

cluster-buildkit

remote environment so that I can pull from a private repo during a build? I've been researching using Garden for building our enterprise applications which use Node's NPM and PHP Composer with lots of packages pulled from private repositories during the build. Ideally I'd like to use

cluster-buildkit

to build inside our remote k8s cluster, however I haven't been able to find a way to pass secrets into a remote build. We currently build in Jenkins using buildkit's experiental SSH mounts in the Dockerfile (eg

RUN --mount=type=ssh composer install

) which avoids pushing ssh keys into images during the build, and we'd like to retain that method. Am I missing something obvious or is this method not currently supported?

Hi @stocky-hair-84003 There's not really a proper way to use SSH mounts like that in the remote builder. For this particular use case you'd have to use the local Docker build mode (i.e.

buildMode: local-docker

) You can however mount secrets as you normally would using the

extraFlags

field. So if you have a Docker command like:

docker build --secret id=my-token,src=./my-token.txt .

then with Garden you'd do:

kind: Build
name: my-build
type: container
spec:
  extraFlags:
    - --secret=id=my-token,src=./my-token.txt

And from the Dockerfile you could now reference it like so:

RUN --mount=type=secret,id=my-token \
    set "//<registry>/:_authToken=$(cat /run/secrets/my-token)"

Hey @brief-restaurant-63679 thanks for the detailed response, that's in line with what I could find elsewhere and looking at the underlying code. This is a bit of a showstopper for us at scale; if this is just a missing feature in Garden I'd be happy to take a swing at implementing it? Looking at buildctl docs, it should be possible to pass

--ssh default=<keyfile|socket|etc>

(https://github.com/AkihiroSuda/buildkit_poc/blob/b5003d53eb522f629c41ad8c48b31c2d6c340afc/frontend/dockerfile/docs/experimental.md#example-access-to-gitlab) which we can do under

extraFlags

in garden, for example:

spec:
  extraFlags:
    - --ssh=/keys/id_rsa

The problem with this approach is Garden doesn't expose a method of mounting the key inside the buildctl container on the remote; the k8s deployment which runs the

buildctl

command only mounts secrets for Docker, and doesn't provide an avenue for other secrets: https://github.com/garden-io/garden/blob/c4e6982cb45640ae51f0515d6cc5e7a514562d5d/core/src/plugins/kubernetes/container/build/buildkit.ts#L493 Citing the way Docker Registry secrets are added to k8s and mounted for

buildctl

, I can't see an obvious reason why an SSH Key couldn't be mounted in the same manner, using a separate secret and volume? Securing the k8s Secret and limiting the scope of the SSH Key is entirely my responsibility as an end-user.

Hey @stocky-hair-84003 You're right that it's basically just missing at the moment. I guess this could be handled the same way as we currently handle image pull secrets. In that case, the user is responsible for creating the image pull secrets in some namespace and then Garden copies it to the namespace where the build happens and adds it to the

buildkit

Deployment. Another approach could be to use the "cloud builder" which is an alternative to in-cluster building that we're working on and should solve this issue, but may or may not fit your use case. I hope it's all right that I reach out over DM to discuss further. I'd love to learn more about your use case and try to figure out what the best approach would be since every stack is different and there tend to be unexpected gotchas.

@brief-restaurant-63679 Sure, I'm happy to chat. I'm not sure we can leverage the cloud builder (security teams are fun things to have) and we're only at the proof-of-concept phase with Garden, but it's looking promising besides this one thing.