Templates Guide

Complete guide to using templates for data transformation and dynamic configuration in Operion workflows

This guide covers how to use templates for data transformation and dynamic configuration in Operion workflows.

What are templates?

Templates are a built-in templating system from the Go programming language. Operion uses Go's text/template package for:

Extracting data from trigger payloads
Transforming data between workflow steps
Creating dynamic configuration values
Implementing conditional logic

Template syntax

Operion uses template expressions with {{}} delimiters in JSON configuration fields:

{
  "message": "Hello {{.vars.user_name}}!",
  "url": "https://api.example.com/users/{{.steps.get_user.id}}",
  "condition": "{{eq .steps.validation.valid true}}"
}

Data Sources

Template Context

All templates receive a context object with the following structure:

{
  "steps": {},      // Results from previous workflow steps
  "vars": {},       // Workflow variables
  "trigger": {},    // Data from the trigger that started the workflow
  "metadata": {},   // Workflow execution metadata
  "env": {},        // Environment variables
  "execution": {    // Execution context
    "id": "",
    "workflow_id": ""
  }
}

Trigger Data

Access data from workflow triggers:

// Schedule trigger data
"{{.trigger.schedule.timestamp}}"        // Execution timestamp
"{{.trigger.schedule.cron}}"             // Cron expression

// Kafka trigger data
"{{.trigger.kafka.topic}}"               // Kafka topic
"{{.trigger.kafka.partition}}"           // Partition number
"{{.trigger.kafka.offset}}"              // Message offset
"{{.trigger.kafka.key}}"                 // Message key
"{{.trigger.kafka.message}}"             // Message content
"{{.trigger.kafka.headers}}"             // Message headers

Step Results

Reference results from previous workflow steps:

// Step execution results
"{{.steps.step_id.result}}"              // Action result
"{{.steps.step_id.status_code}}"         // HTTP status code (for http_request)
"{{.steps.step_id.body}}"                // HTTP response body
"{{.steps.step_id.headers}}"             // HTTP response headers
"{{.steps.step_id.error}}"               // Error information if step failed

// Transform action results
"{{.steps.transform_step}}"              // Direct result of transform action

Variables and Environment

Access workflow variables and environment variables:

"{{.vars.api_key}}"                      // Simple variable
"{{.vars.config.timeout}}"               // Nested variable
"{{index .vars.endpoints 0 \"url\"}}"    // Array variable

"{{.env.DATABASE_URL}}"                  // Environment variable
"{{.env.API_KEY}}"                       // Environment variable

Basic template operations

Data Extraction

// Simple property access
"{{.user.name}}"                         // Extract name property
"{{.user.address.city}}"                 // Nested property access
"{{index .users 0 \"email\"}}"           // Array element access

// Array operations with range
"{{range .products}}{{.name}} {{end}}"   // Map over array, extract name
"{{range .orders}}{{.total}} {{end}}"    // Extract total from all orders

String Operations

// String concatenation
"{{.user.firstName}} {{.user.lastName}}"
"{{range $i, $tag := .tags}}{{if $i}} {{end}}{{$tag}}{{end}}"  // Join array with spaces

Number Operations

// Built-in comparisons
"{{if gt .price 100}}Expensive{{else}}Affordable{{end}}"

Conditional Logic

Templates use if/else statements for conditionals:

// Simple conditions
"{{if eq .status \"active\"}}enabled{{else}}disabled{{end}}"

// Multiple conditions
"{{if ge .score 90}}A{{else if ge .score 80}}B{{else if ge .score 70}}C{{else if ge .score 60}}D{{else}}F{{end}}"

// Conditional object creation (as JSON string)
"{{if eq .status \"error\"}}{\"level\": \"error\", \"message\": \"Operation failed\", \"retry\": true}{{else}}{\"level\": \"info\", \"message\": \"Operation succeeded\", \"retry\": false}{{end}}"

Object Construction

// Simple object creation (as JSON string)
"{\"id\": {{.user.id}}, \"fullName\": \"{{.user.firstName}} {{.user.lastName}}\", \"isActive\": {{eq .user.status \"active\"}}}"

// Using with condition for boolean values
"{\"userId\": {{.user.id}}, \"active\": {{if eq .user.status \"active\"}}true{{else}}false{{end}}}"

// Array construction
"[{{range $i, $user := .users}}{{if $i}}, {{end}}{\"id\": {{.id}}, \"name\": \"{{.name}}\"}{{end}}]"

Array Processing

// Array transformation with range
"[{{range $i, $user := .users}}{{if $i}}, {{end}}{\"id\": {{.id}}, \"name\": \"{{.firstName}} {{.lastName}}\", \"contact\": \"{{.email}}\"}{{end}}]"

// Filtering with conditionals (math operations would need to be calculated elsewhere)
"[{{$first := true}}{{range .products}}{{if gt .price 50}}{{if not $first}}, {{end}}{\"name\": \"{{.name}}\", \"originalPrice\": {{.price}}}{{$first = false}}{{end}}{{end}}]"

// Simple counting
"{{len .items}}"

Advanced template patterns

Complex JSON Construction

For complex object construction, it's better to use the transform action:

{
  "id": "transform-user-data",
  "action": "transform",
  "config": {
    "expression": "{\"id\": {{.id}}, \"profile\": {\"fullName\": \"{{.name}}\", \"email\": \"{{.email}}\", \"phone\": \"{{.phone}}\", \"company\": \"{{.company.name}}\", \"role\": \"{{.company.catchPhrase}}\"}, \"address\": {\"street\": \"{{.address.street}}\", \"city\": \"{{.address.city}}\", \"zipcode\": \"{{.address.zipcode}}\", \"coordinates\": {\"lat\": {{.address.geo.lat}}, \"lng\": {{.address.geo.lng}}}}, \"metadata\": {\"processedAt\": \"{{.execution.timestamp}}\", \"source\": \"api\"}}",
    "input": "steps.fetch_user.body"
  }
}

Template functions

Templates support these built-in functions:

// Comparison functions
"{{eq .a .b}}"                           // Equal
"{{ne .a .b}}"                           // Not equal
"{{lt .a .b}}"                           // Less than
"{{le .a .b}}"                           // Less than or equal
"{{gt .a .b}}"                           // Greater than
"{{ge .a .b}}"                           // Greater than or equal

// Logical functions
"{{and .a .b}}"                          // Logical AND
"{{or .a .b}}"                           // Logical OR
"{{not .a}}"                             // Logical NOT

// Utility functions
"{{len .array}}"                         // Length of array/string/map
"{{index .array 0}}"                     // Access array/map element
"{{printf \"%s: %d\" .name .age}}"       // Formatted string

Common Use Cases

1. API Response Transformation

Transform external API responses into your required format:

{
  "id": "transform-user-data",
  "action": "transform",
  "config": {
    "expression": "{\"id\": {{.id}}, \"profile\": {\"fullName\": \"{{.name}}\", \"email\": \"{{.email}}\", \"phone\": \"{{.phone}}\", \"company\": \"{{.company.name}}\", \"role\": \"{{.company.catchPhrase}}\"}, \"address\": {\"street\": \"{{.address.street}}\", \"city\": \"{{.address.city}}\", \"zipcode\": \"{{.address.zipcode}}\", \"coordinates\": {\"lat\": {{.address.geo.lat}}, \"lng\": {{.address.geo.lng}}}}, \"metadata\": {\"processedAt\": \"{{.execution.timestamp}}\", \"source\": \"api\"}}",
    "input": "steps.fetch_user.body"
  }
}

2. Dynamic URL Construction

Build URLs dynamically based on step results:

{
  "id": "fetch-user-posts",
  "action": "http_request",
  "config": {
    "url": "https://api.example.com/users/{{.steps.get_user.id}}/posts?limit={{if .vars.post_limit}}{{.vars.post_limit}}{{else}}10{{end}}",
    "method": "GET"
  }
}

3. Conditional Workflow Steps

Execute steps conditionally based on data:

{
  "id": "send-notification",
  "action": "http_request",
  "config": {
    "url": "https://api.notifications.com/send",
    "method": "POST",
    "body": "{\"recipient\": \"{{.user.email}}\", \"subject\": \"Welcome {{.user.name}}\", \"template\": \"{{if .user.vip}}vip-welcome{{else}}standard-welcome{{end}}\", \"data\": {\"name\": \"{{.user.name}}\", \"accountType\": \"{{if .user.vip}}VIP{{else}}Standard{{end}}\"}}"
  },
  "condition": "{{eq .steps.validate_user.valid true}}"
}

4. Error Handling with Templates

Create intelligent error responses:

{
  "id": "handle-api-error",
  "action": "transform",
  "config": {
    "expression": "{{$status := .steps.api_call.status_code}}{{if ge $status 400}}{\"error\": true, \"code\": {{$status}}, \"message\": \"{{if eq $status 404}}Resource not found{{else if eq $status 401}}Authentication failed{{else if eq $status 429}}Rate limit exceeded{{else if ge $status 500}}Server error{{else}}Client error{{end}}\", \"retryable\": {{or (ge $status 500) (eq $status 429)}}}{{else}}{\"error\": false, \"data\": {{.steps.api_call.body}}}{{end}}",
    "input": ""
  }
}

5. Data Validation

Validate input data with templates:

{
  "id": "validate-order",
  "action": "transform",
  "config": {
    "expression": "{{$input := .trigger.kafka.message}}{{$hasCustomer := ne $input.customer_id \"\"}}{{$hasItems := gt (len $input.items) 0}}{{$validTotal := gt $input.total 0}}{{if and $hasCustomer $hasItems $validTotal}}{\"valid\": true, \"data\": {{$input}}}{{else}}{\"valid\": false, \"errors\": [{{if not $hasCustomer}}\"customer_id is required\"{{end}}{{if not $hasItems}}{{if not $hasCustomer}}, {{end}}\"at least one item is required\"{{end}}{{if not $validTotal}}{{if or (not $hasCustomer) (not $hasItems)}}, {{end}}\"total must be greater than 0\"{{end}}]}{{end}}",
    "input": ""
  }
}

Testing templates

Local Testing

Create test files to validate your templates:

# test-data.json
{
  "user": {
    "id": 123,
    "name": "John Doe",
    "email": "[email protected]",
    "orders": [
      {"id": 1, "total": 25.99},
      {"id": 2, "total": 15.50}
    ]
  }
}

# Use Go's template testing
go run -c 'package main

import (
    "encoding/json"
    "text/template"
    "os"
)

func main() {
    tmpl := template.Must(template.New("test").Parse("{{.user.name}} has {{len .user.orders}} orders"))
    
    data := map[string]any{
        "user": map[string]any{
            "name": "John Doe",
            "orders": []any{
                map[string]any{"id": 1, "total": 25.99},
                map[string]any{"id": 2, "total": 15.50},
            },
        },
    }
    
    tmpl.Execute(os.Stdout, data)
}'

Debugging Templates in Operion

Enable debug logging to see template evaluation:

export LOG_LEVEL=debug
./bin/operion-worker

Look for log entries showing template evaluation:

DEBUG: Template evaluation: {{.steps.api_call.body.name}} -> "John Doe"
DEBUG: Condition evaluation: {{eq .steps.validation.valid true}} -> true

Performance Considerations

Efficient Templates

// Good: Direct property access
"{{.user.name}}"

// Avoid: Unnecessary complexity
"{{with .user}}{{.name}}{{end}}"

// Good: Simple conditionals
"{{if .active}}enabled{{else}}disabled{{end}}"

// Avoid: Complex nested operations
"{{range .items}}{{range .subitems}}{{.complex.operation}}{{end}}{{end}}"

Memory Usage

For large data processing, consider breaking into multiple transform steps rather than complex single templates.

Common Pitfalls and Solutions

1. Null/Undefined Handling

// Problem: Error when property doesn't exist
"{{.user.profile.bio}}"

// Solution: Use with statement and conditionals
"{{with .user.profile}}{{if .bio}}{{.bio}}{{else}}No bio available{{end}}{{else}}No profile available{{end}}"

// Or simpler with default
"{{if .user.profile.bio}}{{.user.profile.bio}}{{else}}No bio available{{end}}"

2. Type Handling

"{{if gt .user.age \"18\"}}Adult{{else}}Minor{{end}}"

3. JSON Escaping

// Problem: Special characters in JSON strings
"{{.user.bio}}"  // Fails if bio contains quotes

// Solution: Use printf for proper JSON escaping
"{{printf \"%q\" .user.bio}}"

Best Practices

Keep templates simple: Break complex transformations into multiple steps
Use meaningful data structure: Organize template context logically
Test with edge cases: Test with null, empty, and unexpected data
Document complex templates: Add comments in workflow description
Handle errors gracefully: Use conditionals to check for data existence
Use transform action for complex JSON: Don't try to build complex JSON in HTTP request bodies

Next Steps

Visual Workflow Viewer - View workflows visually
API Reference - Complete API documentation
Triggers - All available triggers
Go Template Documentation - Complete Go template reference

Templates Guide

On this page