Including files, yet not hashing them for version
# 🌱|help-and-getting-started
l
We normally build binaries using garden in the buildkit docker images. However, the remote machines are not particularly quick, when compared to the CI machines we are able to use. I have set it up such, that on CI, the binaries are build in the CI workers (which is around 3x faster than in-cluster), and those binaries are then passed into the docker context. In the dockerfile, if the binaries are found and the ARG flag is set, it skips over the expensive steps and just uses the CI-built binaries. So this in principle works, with one exception - the pre-built binaries are of course taken into account when hashing and thus calculating the version of the module. And so it results in two different versions the same code is built in CI (with the binaries) and outside the CI (without the binaries). This is of course wasted work I'd like to avoid (i.e. re-building the module / recompiling binaries from identical code) Is there a way to make this case work? If not currently (and I admit we're yet to migrate to 0.13.x), this might be a feature request, I guess
c
Hi @lively-actor-89932, sorry for the late reply, this thread somehow fell through the cracks. That's an interesting problem you have here. It would be very helpful to understand what kind of binaries we are talking about. E.g. Could they be stored somewhere after being built in CI and accessed locally? Have you tried in-cluster building? Is it as slow as in local Docker? Are the binaries invalidating the cache even in subsequential local builds? I think if you could provide us with more info we can figure out if this is a feature request or just a matter of setting up your project in a particular way. 🙏
Also, just a reminder that we announced the EOL for 0.12, so now is the time to upgrade. 🙂 See: https://github.com/garden-io/garden/issues/6119
l
Yup, am aware of the EOL, will migrate before then 🙂 We are actually talking about in-cluster building. And the binaries are indeed taken into account when present, and so my current workaround is to include them (once compiled) into all the following CI stages as an artifact. Yet that still misses potential cache hits (version matches) from local garden commands that devs would call - because they don't necessarily have the binaries locally.
Another possibility would be to always compile the binaries locally, and work that into garden DAG, but I'd rather allow opting into that, as it's a heavy operation (we're building rust binaries), and might not work well if built on macs
c
I think I understand the problem a bit better and I agree it might be a feature request. Currently the version for an action/module is going to be affected by its dependencies and there's not so much we can do about it, although I admit that it can be frustrating. We are planning to tackle this and we started experimenting with different approaches, e.g. allowing users to specify include/exclude files for version calculation or providing facilities to cache more easily docker layers (and speed up builds in general). Unfortunately I can't give you a precise timeline, but it's definitely a "limitation" that we are aware of. What I'd suggest is to open an issue to express interest, that's the best way to make the rest of the team more aware that's something like this is needed by the community. 🙏