Benchmarks

Kestra is an orchestration platform: you define a flow, and Kestra orchestrates it. Flows can range from lightweight tasks running in milliseconds to complex scripts in containers that run for tens of minutes.

This benchmark focuses on orchestration performance, including dispatching to the Kestra Worker, rather than workload execution, which varies by use case. To isolate orchestration performance, we use workflows with fast tasks, such as:

• io.kestra.plugin.core.log.Log — logs a single message.
• io.kestra.plugin.core.output.OutputValues — produces a single output (simulating a data-oriented workflow).

Test Environment

Benchmarks were run on a Google Cloud e2-standard-4 VM (4 vCPUs, 16 GB RAM) with two setups:

1. Kestra Open Source (OSS) — Postgres 16 backend (4 vCPUs, 16 GB RAM). Database runs remotely to simulate production.
2. Kestra Enterprise Edition (EE) — Kafka backend (4 vCPUs, 16 GB RAM). Kafka and Elasticsearch run on separate VMs.

Benchmark 1 — Simple flow

Description Triggered by a Webhook. Contains two tasks:

1. Outputs a variable.
2. Logs that variable.

id: benchmark01
namespace: benchmarks

triggers:
  - id: webhook
    type: io.kestra.plugin.core.trigger.Webhook
    key: benchmark

inputs:
  - id: name
    type: STRING
    defaults: World

tasks:
  - id: concatenate
    type: io.kestra.plugin.core.output.OutputValues
    values:
      message: Hello {{ inputs.name }}
  - id: hello
    type: io.kestra.plugin.core.log.Log
    message: "{{ outputs.concatenate.values.message }}"

Results for Kestra OSS

Results for Kestra EE

Key takeaways

• At 250 executions/min (500 tasks/min), execution latency is approximately 200ms — similar to single execution time.
• Kestra OSS (JDBC backend) sustains up to 1500 executions/min (3000 tasks/min) with an execution duration of less than 1s, which is what we could realistically target for such a workflow.
• Kestra EE (Kafka backend) sustains up to 4000 executions/min (8000 tasks/min).
• Kestra EE has a slightly higher latency due to low throughput, but supports way higher throughput than Kestra OSS.

Benchmark 2 — Complex flow

Description Triggered by a Webhook. Contains 5 If tasks with 2 subtasks each (only one executes per run). This creates 10 task runs per execution and stresses the Executor.

id: benchmark02
namespace: benchmarks

inputs:
  - id: condition
    type: BOOL
    defaults: true

triggers:
  - id: webhook
    type: io.kestra.plugin.core.trigger.Webhook
    key: benchmark

tasks:
  - id: if1
    type: io.kestra.plugin.core.flow.If
    condition: "{{inputs.condition}}"
    then:
      - id: hello-true-1
        type: io.kestra.plugin.core.log.Log
        message: Hello True 1
    else:
      - id: hello-false-1
        type: io.kestra.plugin.core.log.Log
        message: Hello False 1
  - id: if2
    type: io.kestra.plugin.core.flow.If
    condition: "{{inputs.condition}}"
    then:
      - id: hello-true-2
        type: io.kestra.plugin.core.log.Log
        message: Hello True 2
    else:
      - id: hello-false-2
        type: io.kestra.plugin.core.log.Log
        message: Hello False 2
  - id: if1-3
    type: io.kestra.plugin.core.flow.If
    condition: "{{inputs.condition}}"
    then:
      - id: hello-true-3
        type: io.kestra.plugin.core.log.Log
        message: Hello True 3
    else:
      - id: hello-false-3
        type: io.kestra.plugin.core.log.Log
        message: Hello False 3
  - id: if4
    type: io.kestra.plugin.core.flow.If
    condition: "{{inputs.condition}}"
    then:
      - id: hello-true-4
        type: io.kestra.plugin.core.log.Log
        message: Hello True 4
    else:
      - id: hello-false-4
        type: io.kestra.plugin.core.log.Log
        message: Hello False 4
  - id: if5
    type: io.kestra.plugin.core.flow.If
    condition: "{{inputs.condition}}"
    then:
      - id: hello-true-5
        type: io.kestra.plugin.core.log.Log
        message: Hello True 5
    else:
      - id: hello-false-5
        type: io.kestra.plugin.core.log.Log
        message: Hello False 5

Results for Kestra OSS

Results for Kestra EE

Key takeaways

• At 250 executions/min (500 tasks/min), execution latency is approximately 1s — similar to single execution time.
• Kestra OSS (JDBC backend) sustains up to 400 executions/min (4000 tasks/min) with an execution duration of less than 3s, which is what we could realistically target for such a workflow.
• Kestra EE (Kafka backend) sustains up to 700 executions/min (7000 tasks/min).
• The Kestra Executor processing capability is independent of the type of tasks to process; the number of tasks per minute sustained in this benchmark is the same as in the first benchmark.

Benchmark 3 — Large ForEach loop

Description Executes 100 iterations of a ForEach loop with unbounded concurrency.

id: benchmark03
namespace: benchmarks

tasks:
  - id: foreach
    type: io.kestra.plugin.core.flow.ForEach
    values: "{{range(1, 100)}}"
    concurrencyLimit: 0
    tasks:
      - id: output
        type: io.kestra.plugin.core.output.OutputValues
        values:
          some: value

Observations On average, the execution times for the OSS JDBC backend and EE Kafka are approximately 8s and 9s respectively. That is about 25 tasks/s, as the ForEach task is executed on each iteration, so we will end up with 200 task executions.

This is lower than the throughput in the previous benchmarks because a single flow with many task runs creates a large execution context, which is costly to orchestrate.

Benchmark 4 — Realtime Trigger with JSON Transformation

Description Consumes messages from a Kafka topic in real time, transforms them with JSONata TransformValue task, and outputs new data in the OutputValues task. This triples the size of the data in the execution context.

id: benchmark04
namespace: benchmarks

triggers:
  - id: kafka-logs
    type: io.kestra.plugin.kafka.RealtimeTrigger
    topic: test_kestra
    properties:
      bootstrap.servers: localhost:9092
    groupId: myGroup

tasks:
  - id: transform
    type: io.kestra.plugin.transform.jsonata.TransformValue
    from:  "{{trigger.value}}"
    expression: |
      $.{
        "order_id": order_id,
        "customer_name": first_name & ' ' & last_name,
        "address": address.city & ', ' & address.country,
        "total_price": $sum(items.(quantity * price_per_unit))
      }
  - id: hello
    type: io.kestra.plugin.core.output.OutputValues
    values:
      log: "{{outputs.transform.value}}"

Benchmarked with:

• Small messages (~1.6 KB)
• Medium messages (~16 KB)
• Large messages (~160 KB)

Results for Kestra OSS

With 1.6 KB small-sized messages:

With 16 KB medium-sized messages:

With 160 KB large-sized messages:

Results for Kestra EE

With 1.6 KB small-sized messages:

With 16 KB medium-sized messages:

With 160 KB large-sized messages:

Key takeaways

• Small messages: Similar performance to Benchmark 1, which is expected.
• Medium messages: Kestra sustains up to 1250 executions/min (2500 tasks/min) with an execution duration of less than 1s, which is what we could realistically target for such a workflow.
• Large messages: Performance starts to degrade significantly, which is expected due to the increased Worker workload and Executor sensitivity to execution size.
• EE sustains higher throughput than Kestra OSS (7000 tasks/min vs. 2500 tasks/min) in real-time scenarios for small messages.

Conclusion

Kestra is a platform, not just a framework. It provides orchestration plus logging, metrics, retries, SLAs, error handling, governance, and observability. While this adds overhead compared to lightweight tools, performance is balanced with feature richness.

Kestra is designed for high performance in workflow orchestration and task dispatching, ensuring minimal time spent in the orchestrator and more time in actual task execution. Thanks to continuous performance tuning by the engineering team, Kestra remains among the fastest, most high-performing workflow orchestrators in every release.

To further note, Kestra is built to scale horizontally. When a use case demands it, seamlessly add Executor/Worker nodes to further increase throughput.