Parallel

The Parallel block is a container that executes multiple instances concurrently for faster workflow processing. Process items simultaneously instead of sequentially.

Parallel blocks are container nodes that execute their contents multiple times simultaneously, unlike loops which execute sequentially.

Configuration Options

Parallel Type

Choose between two types of parallel execution:

Parallel Count (count-based) - Execute a fixed number of parallel instances:

Count-based parallel execution

Use this when you need to run the same operation multiple times concurrently.

Example: Run 5 parallel instances
- Instance 1 ┐
- Instance 2 ├─ All execute simultaneously
- Instance 3 │
- Instance 4 │
- Instance 5 ┘

How to Use Parallel Blocks

Creating a Parallel Block

  1. Drag a Parallel block from the toolbar onto your canvas
  2. Configure the parallel type and parameters
  3. Drag the blocks that should run in each instance inside the parallel container
  4. Connect the blocks as needed

Referencing Parallel Data

There's an important distinction between referencing parallel data from inside vs outside the parallel block:

Inside the parallel, use <parallel.> references to access the current instance context:

  • <parallel.index>: Current instance number (0-based)
  • <parallel.currentItem>: Item for this instance (collection-based only)
  • <parallel.items>: Full collection being distributed (collection-based only)
// Inside a Function block within the parallel
const idx = <parallel.index>;           // 0, 1, 2, ...
const item = <parallel.currentItem>;    // This instance's item

These references are only available for blocks inside the parallel container. They give you access to the current instance's context.

Example Use Cases

Batch API Processing - Process multiple API calls simultaneously

Parallel (Collection) → API (Call Endpoint) → Function (Aggregate)

Multi-Model AI Processing - Get responses from multiple AI models concurrently

Parallel (["gpt-4o", "claude-sonnet-4-5", "gemini-2.5-pro"]) → Agent → Evaluator (Select Best)

Advanced Features

Result Aggregation

SteelEngine collects the results from all parallel instances and makes them accessible via the block name. results[i] is the array of settled cells for instance i, so the output keeps the input order even though the instances ran concurrently:

// In a Function block after a parallel named "Process Tasks"
const instances = <processtasks.results>;
const succeeded = instances.flat().filter((cell) => cell.status === 'fulfilled');
const failed = instances.flat().filter((cell) => cell.status === 'rejected');
return { values: succeeded.map((cell) => cell.value), errors: failed.map((cell) => cell.error) };

Instance Isolation

Each parallel instance runs independently:

  • Its own <parallel.index>, <parallel.currentItem>, and block outputs. One instance cannot reference a block in a sibling instance
  • Workflow variables (<variable.name>) are shared across the whole run, not per instance, so avoid writing them from inside a parallel
  • Failures in one instance don't affect others. A failed block becomes a rejected cell in that instance's results. The rest of that instance is skipped, and the other instances finish normally

Limitations

Container blocks (Loops and Parallels) support nesting. You can place parallels inside parallels, loops inside parallels, and any combination of container blocks to build complex multi-dimensional workflows.

For a Loop inside each Parallel branch, overlapping work across branches depends on the deployment's TEMPORAL_NESTED_PARALLEL_COHORTS setting. When enabled, ready blocks from different branches can run together while each loop's iterations remain sequential. Self-hosted Workers default to false; operators must upgrade every Worker before enabling it. See Temporal environment variables.

While parallel execution is faster, be mindful of:

  • API rate limits when making concurrent requests
  • Memory usage with large datasets
  • The 20-instance limit: a count or collection above 20 fails the block with "Parallel branch count (N) exceeds maximum allowed (20). Execution blocked." For larger collections use the Parallel Window block. It processes up to 1,000 items with a bounded number active at a time.

Parallel vs Loop

Understanding when to use each:

FeatureParallelLoop
ExecutionConcurrentSequential
SpeedFaster for independent operationsSlower but ordered
OrderResults indexed by instance, so input order is preservedMaintains order
Use caseIndependent operationsDependent operations
Resource usageHigherLower

Inputs and Outputs

  • Parallel Type: Choose between 'count' or 'collection'

  • Count: Number of instances to run (count-based)

  • Collection: Array or object to distribute (collection-based)

Best Practices

  • Independent operations only: Ensure operations don't depend on each other
  • Handle rate limits: Add delays or throttling for API-heavy workflows
  • Error handling: Each instance should handle its own errors gracefully

Common Questions

20. A count or collection larger than 20 does not run partially. The Parallel block fails before starting with 'Parallel branch count (N) exceeds maximum allowed (20). Execution blocked.' For collections larger than that, use the Parallel Window block. It accepts up to 1,000 items and runs a bounded number of them at a time.
Not through block outputs. Each instance has its own <parallel.> context and its own copies of the body blocks. One instance cannot read another instance's block outputs during execution. Workflow variables are the exception. They belong to the whole run, so a Variables block inside a parallel writes a value every instance can see. Instead, compute per-instance values in the body and combine them from <blockname.results> after the parallel completes.
Failures in one instance do not affect other instances. The failed block becomes a { status: 'rejected', error } cell in that instance's entry of <blockname.results>. The remaining blocks in that instance are skipped, and the other instances continue to execute normally.
Yes. Container blocks (Parallels and Loops) support nesting. You can place parallels inside parallels, loops inside parallels, and any combination to build multi-dimensional workflows.
Use <blockname.results> where blockname is the normalized name of your Parallel block (lowercase, no spaces). This returns one array per instance. Each array holds a settled cell per block that ran in that instance, either { status: 'fulfilled', value } or { status: 'rejected', error }.
Yes. Instances run concurrently, but SteelEngine assembles the results array by instance index. results[0] belongs to the first item, results[1] to the second, and so on, regardless of which instance finished first.
Count-based (Parallel Count) runs a fixed number of identical instances (e.g., run 5 times). Collection-based (Parallel Each) distributes items from an array or object across instances, with each instance processing one item. Use count-based for repeated operations and collection-based for batch processing.

On this page