Standard strategy for delays between deploys? garden.io #🌱

Standard strategy for delays between deploys?

fresh-yak-35965

06/07/2024, 1:29 PM

Hello again team! I love your product so much I have another question. Is there a standard way to introduce a delay between two kubernetes deploys? Example:

Copy code

yaml
---
kind: Deploy
type: kubernetes
name: cockroach-setup
spec:
  files:
    - crd.yml
    - operator.yml
    - dev/configmap.yml
---
kind: Deploy
type: kubernetes
name: cockroach-db
dependencies:
  - deploy.cockroach-setup
spec:
  namespace: kraken-${environment.namespace}
  files:
    - database.yml
  portForwards:
    - name: webui
      resource: Service/cockroachdb
      targetPort: 8080
      localPort: 8999

The

cockroach-db

deplay (second deploy) requires a webhook to be provisioned from

cockroach-setup

, the first deploy. The reason I split them out is because I thought doing so would cause cockroach-db to wait for the first tasks resources to be fully provisioned, but alas it does not. What is the standard way to solve this issue in Garden (or maybe Kubernetes?). I see that one can introduce a "delay job" which is just sleeping busybox in between for 10 seconds. This sounds hacky to me and I'd prefer not to do that if there's some way to tell

deploy.cockroach-db

to confirm the webhook is up first, or perhaps give

cockroach-setup

a livenesscheck that depends on the webhook being available? Thanks

fresh-yak-35965

06/07/2024, 1:57 PM

I think I got this. I put

tcpSocket

liveness and readiness probes on the pod with a delay.

fresh-yak-35965

06/07/2024, 7:11 PM

Sorry to piggy back on this, but I still sort of have an issue here:

Copy code

kind: Deploy
type: kubernetes
name: cockroach-db
dependencies:
  - deploy.cockroach-setup
spec:
  namespace: kraken-${environment.namespace}
  files:
    - database.yml
  portForwards:
    - name: webui
      resource: Service/cockroachdb
      targetPort: 8080
      localPort: 8999

This deploy attempts to create the portForwards before the crd resource it's deploying (a StatefulSet) is deployed. Thus it spams my console with errors until the service is up. Where would make Garden aware of the service it's looking for? If I specify it as the default target, does garden wait to establish the port forwards?

freezing-pharmacist-34446

06/11/2024, 12:14 PM

Hi @fresh-yak-35965 Thank you for your kind words about Garden! Regarding your questions: 1. Creating a delay between deploys: a depending service like cockroach-db waits for the resources in cockroach-setup to be ready. There is one caveat though: we check readiness on all kinds of replicasets but not all resources are implemented there. I would generally choose the same approach as you and implement some readiness probes with delays - one nice way to do this is with a startupProbe, which if available could even check sth specific from cockroach-setup like the availability of an endpoint. 2. PortForwards: Again here it is possible that Garden is not aware of the resource you are deploying if it is using a CRD. Specifying a

defaultTarget

could potentially alleviate that so i would definitely give it a spin. Let me know how it goes

Previous Next