Variables

Variables are key–value pairs that let you reuse values across tasks.

You can also store variables at the namespace level to reuse them across multiple flows in that namespace.

How to configure variables

The example below shows how you can configure variables in your flow:

id: hello_world
namespace: company.team

variables:
  myvar: hello
  numeric_variable: 42

tasks:
  - id: log
    type: io.kestra.plugin.core.debug.Return
    format: "{{ vars.myvar }} world {{ vars.numeric_variable }}"

Use variables with the syntax {{ vars.variable_name }}.

How variables are rendered

You can use variables in any task property documented as dynamic.

Dynamic variables are rendered by the Pebble templating engine, which processes expressions with filters and functions. More information on variable processing can be found under Expressions.

As of 0.14, variables are no longer rendered recursively. Learn more about this change — and how to adjust behavior — in the 0.14 migration guide.

Dynamic variables

If a variable contains an expression, wrap it with render when using it in a task.

For example, the variable below displays the current time only when wrapped with render; otherwise, the log prints the expression as a string:

id: dynamic_variable
namespace: company.team

variables:
  time: "{{ now() }}"

tasks:
  - id: log
    type: io.kestra.plugin.core.log.Log
    message: "{{ render(vars.time) }}"

Wrap the variable expression with render every time you use it in a task.

Set or modify execution variables

The SetVariables and UnsetVariables tasks can modify or delete variables within the execution context. For example, take the following flow:

id: variables_demo
namespace: company.team

variables:
  state: FAILED
  ansibleTicket: myticket
  nested:
    child: property
    unchanged: stay the same

tasks:
  - id: request
    type: io.kestra.plugin.core.output.OutputValues
    values:
      ansibleTicket: new ticket value
      state: SUCCESS

  - id: updateVariables
    type: io.kestra.plugin.core.execution.SetVariables
    overwrite: true # true by default
    variables:
      state: "{{ outputs.request.values.state }}"
      ansibleTicket: "{{ outputs.request.values.ansibleTicket }}"
      nested:
        child: new value

  - id: confirmUpdate
    type: io.kestra.plugin.core.log.Log
    message: Hello "{{ vars }}"

Initially, state is FAILED and ansibleTicket is myticket. Within the flow, the updateVariables task uses io.kestra.plugin.core.execution.SetVariables to modify state to SUCCESS and ansibleTicket to new ticket value per the request task, as well as change one of the nested variables, nested.child to new value (nested.unchanged is unmodified so it’ll remain the same).

After the flow runs, state, ansibleTicket, and nested.child have their new values, and nested.unchanged remains unchanged.

Set Variables

Delete or unset execution variables

To unset variables, use io.kestra.plugin.core.execution.UnsetVariables. Building on the example above, add the following task:

  - id: deleteVariables
    type: io.kestra.plugin.core.execution.UnsetVariables
    variables:
      - state
      - ansibleTicket
      - nested.child # remove only this key from the nested map

After executing the flow, the only remaining variable is nested.unchanged with the value stay the same. In the unset task, state, ansibleTicket, and nested.child were deleted.

Unset Variables

FAQ

How do I escape a block in Pebble syntax to ensure that it won’t be parsed?

To ensure that a block of code won’t be parsed by Pebble, you can use the {% raw %} and {% endraw %} Pebble tags. For example, the following returns the string {{ myvar }} instead of the value of myvar:

{% raw %}{{ myvar }}{% endraw %}

In which order are inputs and variables resolved?

Inputs are resolved first, before the execution starts. If a flow has an invalid input value, the execution will not be created.

Therefore, you can use inputs within variables, but you cannot use variables or Pebble expressions within inputs.

[Expressions are rendered recursively: if a variable references another variable, the inner one is resolved first.

Triggers are handled similarly to inputs because they are known before the execution starts (they create the execution). his means you cannot use inputs (unless they have defaults) or variables within triggers, but you can use trigger variables inside variables.

Examples

This flow uses inputs, trigger, and execution variables which are resolved before variables:

id: upload_to_s3
namespace: company.team

inputs:
  - id: bucket
    type: STRING
    defaults: declarative-data-orchestration

tasks:
  - id: get_zip_file
    type: io.kestra.plugin.core.http.Download
    uri: https://wri-dataportal-prod.s3.amazonaws.com/manual/global_power_plant_database_v_1_3.zip

  - id: unzip
    type: io.kestra.plugin.compress.ArchiveDecompress
    algorithm: ZIP
    from: "{{outputs.get_zip_file.uri}}"

  - id: csv_upload
    type: io.kestra.plugin.aws.s3.Upload
    from: "{{ outputs.unzip.files['global_power_plant_database.csv'] }}"
    bucket: "{{ inputs.bucket }}"
    key: "powerplant/{{ trigger.date ?? execution.startDate | date('yyyy_MM_dd__HH_mm_ss') }}.csv"

triggers:
  - id: hourly
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "@hourly"

This flow starts a task conditionally based on whether the input is provided or not:

id: conditional_branching
namespace: company.team

inputs:
  - id: parameter
    type: STRING
    required: false

tasks:
  - id: if
    type: io.kestra.plugin.core.flow.If
    condition: "{{inputs.parameter ?? false }}"
    then:
      - id: if_not_null
        type: io.kestra.plugin.core.log.Log
        message: Received input {{inputs.parameter}}
    else:
      - id: if_null
        type: io.kestra.plugin.core.log.Log
        message: No input provided

Below is an example that uses a trigger variable within a trigger itself (that’s allowed!):

id: backfill_past_mondays
namespace: company.team

tasks:
  - id: log_trigger_or_execution_date
    type: io.kestra.plugin.core.log.Log
    message: "{{ trigger.date ?? execution.startDate }}"

triggers:
  - id: first_monday_of_the_month
    type: io.kestra.plugin.core.trigger.Schedule
    timezone: Europe/Berlin
    backfill:
      start: 2023-11-11T00:00:00Z
    cron: "0 11 * * MON" # at 11:00 every Monday
    conditions: # only first Monday of the month
      - type: io.kestra.plugin.core.condition.DayWeekInMonth
        date: "{{ trigger.date }}"
        dayOfWeek: "MONDAY"
        dayInMonth: "FIRST"

Can I transform variables with Pebble expressions?

Yes. Kestra uses Pebble templates along with the execution context to render dynamic properties. (such as filters, functions, and operators) to transform inputs and variables.

The example below illustrates how to use variables and Pebble expressions to transform string values in dynamic task properties:

id: variables_demo
namespace: company.team

variables:
  DATE_FORMAT: "yyyy-MM-dd"

tasks:
  - id: seconds_of_day
    type: io.kestra.plugin.core.debug.Return
    format: '{{ 60 * 60 * 24 }}'

  - id: start_date
    type: io.kestra.plugin.core.debug.Return
    format: "{{ execution.startDate | date(vars.DATE_FORMAT) }}"

  - id: curr_date_unix
    type: io.kestra.plugin.core.debug.Return
    format: "{{ now() | date(vars.DATE_FORMAT) | timestamp() }}"

  - id: next_date
    type: io.kestra.plugin.core.debug.Return
    format: "{{ now() | dateAdd(1, 'DAYS') | date(vars.DATE_FORMAT) }}"

  - id: next_date_unix
    type: io.kestra.plugin.core.debug.Return
    format: "{{ now() | dateAdd(1, 'DAYS') | date(vars.DATE_FORMAT) | timestamp() }}"

  - id: pass_downstream
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    commands:
      - echo "{{ outputs.next_date_unix.value }}"

Can I use nested variables?

Yes. Depending on the task, you may need to wrap the root variable with json() to access specific keys. Below is an example using a list of maps as a variable:

id: vars
namespace: company.myteam

variables:
  servers:
    - fqn: server01.mydomain.io
      user: root
    - fqn: server02.mydomain.io
      user: guest
    - fqn: server03.mydomain.io
      user: rick

tasks:
  - id: parallel
    type: io.kestra.plugin.core.flow.ForEach
    concurrencyLimit: 0
    values: "{{ vars.servers }}"
    tasks:
      - id: log
        type: io.kestra.plugin.core.log.Log
        message:
           - "{{ taskrun.value }}" # for each element, prints the full JSON object (e.g., {"fqn":"server01.mydomain.io","user":"root"})
           - "{{ json(taskrun.value).fqn }}" # prints the value for that key (e.g., server01.mydomain.io)
           - "{{ json(taskrun.value).user }}" # prints the value for that key (e.g., root)