https://garden.io logo
Join Discord
Powered by
Inconsistent concurrency error during deployment
# 🌱｜help-and-getting-started
m
I've been getting this error inconsistently but very often. I think that it's because garden is trying to deploy an image while it's still being built, or is trying to deploy more things at once than is allowed. I'm really not sure though. My environment is k3s with docker as the containerization runtime (not containerd). I'm using garden 0.13.57 for now while I work on migrating to 0.14 I upgraded to 0.14.3 and am seeing the same thing. Error:
Copy code
SYNCHRONOUS TERMINATION NOTICE: When explicitly exiting the process via process.exit or via a parent process, asynchronous tasks in your exitHooks will not run. Either remove these tasks, use gracefulExit() instead of process.exit(), or ensure your parent process sends a SIGINT to the process running this code.
node:events:497
      throw er; // Unhandled 'error' event
      ^

Error: write EPIPE
    at afterWriteDispatched (node:internal/stream_base_commons:159:15)
    at writeGeneric (node:internal/stream_base_commons:150:3)
    at Socket._writeGeneric (node:net:958:11)
    at Socket._write (node:net:970:8)
    at writeOrBuffer (node:internal/streams/writable:572:12)
    at _write (node:internal/streams/writable:501:10)
    at Writable.write (node:internal/streams/writable:510:10)
    at Transform.<anonymous> (file:///home/ryan/.local/share/garden/1746479452-NIEIYT4.r/rollup/container-CIK3SmVD.mjs:565:37)
    at Transform.emit (node:events:519:28)
    at Transform.emit (node:domain:488:12)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (node:internal/streams/destroy:170:8)
    at emitErrorCloseNT (node:internal/streams/destroy:129:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:90:21) {
  errno: -32,
  code: 'EPIPE',
  syscall: 'write'
}

Node.js v22.9.0
This is essentially my garden file, just modified to obscure the names for privacy
Copy code
kind: Build
name: build-service
description:  service container image
type: container
spec:
  dockerfile: garden/dockerfiles/Dockerfile

---
kind: Deploy
name: -mongodb-helm-deploy
description: Deploys the MongoDB community operator using its helm chart
type: helm
varfiles:
  - garden/config/action_vars.yml

spec:
  releaseName: "${var.ssasm_staging ? var.global_mongo_release_name : var._service_mongo_release_name}"
  chart:
    repo: https://mongodb.github.io/helm-charts
    name: community-operator

  namespace: ${var._service_namespace}${var.namespace_suffix}
  values:
    # Disable crds so multiple instances of mongo can be deployed at once
    community-operator-crds:
      enabled: false

---
kind: Deploy
name: -mongodb-deploy
description: Deploys the  Service's MongoDB replica set to kubernetes
type: kubernetes
varfiles:
  - garden/config/action_vars.yml
variables:
  temp_ssasm_staging: "${var.ssasm_staging || ''}"
dependencies:
  - deploy.-mongodb-helm-deploy
spec:
  manifestTemplates:
    - garden/config/mongo_config.yml
  namespace: ${var._service_namespace}${var.namespace_suffix}
  waitForJobs: true
  portForwards:
    - name: mongo
      resource: service/service-mongodb-svc
      targetPort: 27017
      localPort: ${var._service_db_local_port}

---
kind: Deploy
name: service
type: kubernetes
dependencies:
  - run.-initialize-mongodb
build:
  build-service
varfiles:
  - garden/config/action_vars.yml
spec:
  manifestTemplates:
    - garden/manifests/deployment.yml
  namespace: ${var.service_namespace}

---
kind: Test
name: service-code-coverage
type: exec
varfiles:
  - garden/config/action_vars.yml

spec:
  command: ["bash", "-c", "cd code && ./coverage.sh"]
  env:
    MONGO_CON_STR: "mongodb://localhost:27017"
If I run 
garden build
and then 
garden deploy
the deployment always works. It's only if I delete the docker image and run 
garden deploy
without manually building first that I sometimes get this error - it will work probably about 80% of the time and fail 20% of the time with this error.
bump
I just upgraded garden to 0.14.3 and I'm seeing the same behavior
b
@most-tent-39133 thank you for the detailed crash report! Can you try again if this also happens in the latest edge-cedar version? You can update by running garden self-update edge-cedar, and then attempting to reproduce. We've made 2 changes in main that might affect this. If it doesn't help, we'll have a closer look at this!
Let's continue the conversation in the GitHub issue comments: https://github.com/garden-io/garden/issues/7290
2 Views
PreviousNext
Powered by