Recommended Way To Build Dev Cluster
# 🌱|help-and-getting-started
h
Hello! I'm currently building a dev cluster of sorts where I want myself a few others to be able to deploy their code to run in isolated namespaces. I'm pretty much done with everything, but one problem I have is that some resources, like
ExternalDNS
, are not specific to a particular developer's namespace. What would be the recommended way to manage deployments that are done at the cluster level vs deployments done for a particular namespace. (I know my wording is confusing - i'm not really sure the best way to phrase the question 😛 )
In short, I have two "deployments": * A deployment devs will run when they want to build + deploy their code into their dev-cluster * A deployment the dev-cluster admin wants to run to manage resources like ExternalDNS How can I split these two, ergonomically, with garden.io? I currently have ExternalDNS deployed as part of the normal
garden deploy
step which has the unfortunate problem that if one dev runs
garden deploy
(which deploys their code into their own namespace, and ExternalDNS into the
default
namespace), a second dev also deploys, and then the first dev runs
garden delete environment ...
, then ExternalDNS will be taken down while the second dev's services are still up
q
Hi @hallowed-apartment-22484! The Garden team has been on Easter vacation until today. I'm taking a looksee at this now 👀
What I'd do is split your Garden projects out whether by repo or folder within a monorepo. You could have one
project.garden.yml
file declaring resources like ExternalDNS and cert-manager (should you use it) in the
default
namespace. Your devs would have their own
project.garden.yml
that creates per-developer namespaces based on e.g. their local username https://docs.garden.io/reference/template-strings/modules#usd-local.username I'm a big fan of our opinionated guide on namespaces if you want a reference for environments https://docs.garden.io/guides/namespaces#an-opinionated-guide-on-using-namespaces Using it as a template, your
project.garden.yml
for your globally installed services might look like this
Copy code
yaml
kind: Project
name: my-project

variables:
  global-env-name: default

environments:
  - name: global
    defaultNamespace: ${var.global-env-name}

providers:
  - name: kubernetes
    namespace: ${environment.namespace} # <--- Ensure the K8s namespace matches the Garden namespace
    defaultHostname: ${var.hostname}
    # ...
And your dev
project.garden.yml
might look like:
Copy code
yaml
kind: Project
name: my-project

variables:
  dev-env-name: my-project-${local.username}

environments:
  - name: dev
    defaultNamespace: ${var.dev-env-name}

providers:
  - name: kubernetes
    namespace: ${environment.namespace} # <--- Ensure the K8s namespace matches the Garden namespace
    defaultHostname: ${var.hostname}
    # ...
h
So, basically, have two project.garden.yml files?
q
Yep
b
Hey @hallowed-apartment-22484 where you able o solve this issue? I would also like to recommend you to have different folders for critical components, for example. I don't think your service deployment should be affecting critical components like externalDNS, so perhaps you can move all the Infrastructure Resources to a different Garden folder and deploy from there. This will give you the ability that users only can affect the code/services instead of the infrastructure components that are kind of delicate.
h
Thanks! It seems to work 🙂 I ended up creating a separate repository for this which is not the solution I'd have preferred, but it works well enough
b
Amazing, you could also have separated them on a folder basis, for example your repo structure could be: - externaldns/garden.project.yml - frontend/garden.project.yml But your solution works as well, and also allows you to have more granular rules in those repos, which is great!
7 Views