Running 100 AI Agents in Parallel

Sequential vs parallel: five real scenarios

The table below shows what teams actually measure when they move from a single-threaded agent loop to a parallel fleet on Sandbox Platform. Wall-clock time and throughput change dramatically; compute cost stays almost flat.

The math: sequential vs parallel

Parallelism is virtually free in compute (the same per-task work happens; only wall clock changes). The real cost is engineering: failure handling, queue management, result aggregation. This guide is the architecture for handling that cost cleanly.

The reference architecture

Five components:

1. Trigger — what kicks off the fan-out. A cron tick, a webhook, an operator clicking a button. Pushes N task messages onto the job queue.
2. Job queue (SQS or equivalent) — buffers tasks, handles retries, deduplicates, and supplies backpressure when the worker pool is saturated.
3. Sandbox pool — N sandboxes, each consuming one task at a time. Either pre-warmed (warm pool) or on-demand. Concurrency capped at the slowest downstream system.
4. Result queue — successful results go here.
5. Aggregator — reads results, writes to your destination (a report, a database, a Slack message). Optionally waits for all-N before writing, or streams as results arrive.
6. Dead-letter queue (DLQ) — failed-after-retries tasks land here for human triage.

Work distribution patterns

Three named patterns cover almost every parallel agent workload. Pick the one that fits; don't over-engineer.

Pattern 1 — Map: one task, one sandbox, collect all

When to use it: You have a fixed list of N independent tasks, each task takes similar time, and you want all N results before proceeding. Classic examples: reviewing N files, extracting data from N URLs, running N test suites. Goldman Sachs uses this exact pattern for parallel code-review at the PR level.

What Sandbox Platform provides: Isolated filesystem per agent (no cross-contamination), auto-stop billing (each sandbox stops the moment its task finishes), and a per-sandbox timeout so a runaway agent can't hold a sandbox indefinitely.

import asyncio, os, httpx
from anthropic import Anthropic

SANDBOX_URL = os.environ["SANDBOX_URL"]
API_KEY = os.environ["SANDBOX_API_KEY"]
client = Anthropic()

async def run_one_task(task: dict, http: httpx.AsyncClient) -> dict:
    """Create one sandbox, run one agent task, return result."""
    # 1. Spin up sandbox
    resp = await http.post("/api/sandboxes", json={
        "name": f"map-{task['id']}",
        "instance_type": "ab0t.micro",
        "auto_stop_minutes": 10,
    })
    sandbox_id = resp.json()["sandbox_id"]

    try:
        # 2. Run the agent (single Claude call with tool use)
        message = client.messages.create(
            model="claude-opus-4-5",
            max_tokens=2048,
            system="You are a precise data extractor. Return only valid JSON.",
            messages=[{"role": "user", "content": task["prompt"]}],
        )
        return {"task_id": task["id"], "result": message.content[0].text, "ok": True}
    except Exception as e:
        return {"task_id": task["id"], "error": str(e), "ok": False}
    finally:
        # 3. Always stop sandbox to end billing
        await http.post(f"/api/sandboxes/{sandbox_id}/stop")

async def map_fanout(tasks: list) -> list:
    """Fan out all tasks concurrently; collect all results."""
    async with httpx.AsyncClient(
        base_url=SANDBOX_URL,
        headers={"Authorization": f"Bearer {API_KEY}"},
        timeout=300.0,
    ) as http:
        # return_exceptions=True — one failure doesn't abort the rest
        results = await asyncio.gather(
            *[run_one_task(t, http) for t in tasks],
            return_exceptions=True,
        )

    successes = [r for r in results if isinstance(r, dict) and r["ok"]]
    failures  = [r for r in results if not (isinstance(r, dict) and r["ok"])]
    print(f"{len(successes)}/{len(tasks)} tasks succeeded")
    return successes

Pattern 2 — Pool: fixed workers drain a queue

When to use it: Task count is large or unpredictable, tasks vary in duration, or you need to cap concurrency below task count (e.g., a downstream API rate-limits you to 10 req/sec). Block/Goose deployments for developer workflows often use this pattern — a pool of 5-10 agents draining a queue of developer-submitted tasks.

What Sandbox Platform provides: Long-lived sandboxes that idle between tasks without billing (billing only runs while a command is executing), auto-restart if a sandbox crashes, and per-sandbox resource sizing.

import asyncio
from anthropic import Anthropic

client = Anthropic()

async def worker(worker_id: int, queue: asyncio.Queue, results: list):
    """One worker: drain the queue until empty."""
    while True:
        try:
            task = queue.get_nowait()
        except asyncio.QueueEmpty:
            break

        try:
            msg = client.messages.create(
                model="claude-opus-4-5",
                max_tokens=1024,
                messages=[{"role": "user", "content": task["prompt"]}],
            )
            results.append({
                "worker": worker_id,
                "task_id": task["id"],
                "result": msg.content[0].text,
                "ok": True,
            })
        except Exception as e:
            results.append({"task_id": task["id"], "error": str(e), "ok": False})
        finally:
            queue.task_done()

async def pool_fanout(tasks: list, pool_size: int = 10) -> list:
    """Run tasks with a fixed pool of pool_size concurrent workers."""
    queue: asyncio.Queue = asyncio.Queue()
    for task in tasks:
        queue.put_nowait(task)

    results: list = []
    workers = [
        asyncio.create_task(worker(i, queue, results))
        for i in range(pool_size)
    ]
    await asyncio.gather(*workers)
    return results

# Usage: 200 tasks, 10 workers
# asyncio.run(pool_fanout(my_tasks, pool_size=10))

Pattern 3 — Pipeline: staged agents, each transforms for the next

When to use it: Each stage depends on the previous stage's output and transforms it in a meaningful way. Examples: fetch raw HTML → extract structured data → generate report. Devin's PR pipeline is a 3-stage pipeline: read diff → generate review comments → post to GitHub. This pattern is inherently sequential between stages but can be parallelized within each stage.

What Sandbox Platform provides: Each stage gets a clean sandbox with no state from the previous stage (unless you explicitly pass files), so bugs in stage 2 can't contaminate stage 1's environment.

from anthropic import Anthropic
import asyncio

client = Anthropic()

async def stage_fetch(url: str) -> str:
    """Stage 1: fetch raw content. Runs in a browser sandbox."""
    msg = client.messages.create(
        model="claude-haiku-4-5",  # cheap model for simple fetch
        max_tokens=4096,
        messages=[{"role": "user", "content": f"Fetch and return the text content of: {url}"}],
    )
    return msg.content[0].text

async def stage_extract(raw_content: str) -> dict:
    """Stage 2: extract structured data. Runs in a terminal sandbox."""
    msg = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=2048,
        system="Extract key facts. Return valid JSON only.",
        messages=[{"role": "user", "content": raw_content}],
    )
    import json
    return json.loads(msg.content[0].text)

async def stage_report(facts: dict) -> str:
    """Stage 3: generate a human-readable report."""
    msg = client.messages.create(
        model="claude-opus-4-5",
        max_tokens=4096,
        system="Write a clear executive summary from the provided facts.",
        messages=[{"role": "user", "content": str(facts)}],
    )
    return msg.content[0].text

async def pipeline(url: str) -> str:
    """3-stage pipeline: fetch → extract → report."""
    raw     = await stage_fetch(url)
    facts   = await stage_extract(raw)
    report  = await stage_report(facts)
    return report

# Parallelize the pipeline across N URLs at the item level:
async def parallel_pipelines(urls: list) -> list:
    return await asyncio.gather(*[pipeline(u) for u in urls], return_exceptions=True)

Dispatching the fan-out

import boto3, json
sqs = boto3.client("sqs")

def dispatch(tasks):
    """Send N tasks to the job queue."""
    for i, task in enumerate(tasks):
        sqs.send_message(
            QueueUrl=JOB_QUEUE_URL,
            MessageBody=json.dumps(task),
            MessageGroupId="main",           # for FIFO; use "fanout-{i}" for ordered groups
            MessageDeduplicationId=task["id"],  # idempotency
        )

dispatch([
    {"id": "task-001", "url": "https://acme.com/pricing"},
    {"id": "task-002", "url": "https://blueco.com/pricing"},
    # ... 98 more
])

The worker side: one task per sandbox

import os, requests, json

SANDBOX_API = "https://sandbox.service.ab0t.com"
API_KEY = os.environ["SANDBOX_API_KEY"]

def spawn_for_task(task):
    """Spin up a sandbox, run the task, post result, terminate."""
    sandbox = requests.post(
        f"{SANDBOX_API}/api/sandboxes",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={"name": f"worker-{task['id']}",
              "instance_type": "ab0t.micro",
              "auto_stop_minutes": 5,
              "metadata": {"task": task}}
    ).json()
    # sandbox auto-terminates when work is done; result posted to result queue by the agent
    return sandbox["sandbox_id"]

Or use a warm pool — see Warm Pools: Sub-Second Agent Start.

Isolation guarantees: what each sandbox actually provides

Isolation is the foundational guarantee that makes parallel agents safe. Here is exactly what "isolated sandbox" means at the system level, and why each guarantee matters for concurrent work.

What each sandbox gets

Isolation vs threads vs shared containers

The guarantee is stronger than what you get from Python threading or a shared container namespace:

The isolation boundary also applies across organizations. Sandbox A from Org X and Sandbox B from Org Y running on the same physical host share no memory pages, network routes, or filesystem mounts. Credential leakage between tenants is architecturally impossible — there is no shared address space to leak into.

Capping concurrency at the slowest downstream

If your fan-out talks to a single API that rate-limits you to 10 req/sec, running 100 sandboxes in parallel just stuffs the rate limiter. Cap concurrency at the bottleneck:

• Per-domain cap. Max 5 concurrent sandboxes hitting publisher.com; queue rest.
• Token-bucket per downstream. Refill at the rate the downstream allows.
• Backpressure on the queue. SQS visibility timeout + concurrency cap; if you can't process, leave it on the queue.
• Stagger starts with jitter. Don't dispatch 100 at second-0; spread over the first minute.

Per-task failure isolation

The single most valuable property: one agent's failure doesn't cascade. Each task gets its own sandbox; a hang in one doesn't block others. Concrete safeguards:

• Per-task sandbox — isolation by default
• Per-task budget — kill the sandbox if a single task exceeds N seconds or N dollars
• Loop detector — kill if the agent makes the same tool call 5× in a row
• Visibility timeout — if the queue message isn't acked in T seconds, redeliver to a fresh sandbox
• Max retries — after 3 redeliveries, push to DLQ and move on

Result aggregation patterns

Dead-letter queue + human triage

For any production fan-out, DLQ is mandatory. The pattern:

1. Failed task (after retries) lands in DLQ with full context (input, last error, screenshot if browser-based)
2. Daily Slack digest of DLQ size and category breakdown
3. Manual triage: 5-10 min/day to look at DLQ; redeliver after fix, or accept the loss
4. Pattern-detect: 5+ failures from the same downstream = vendor issue, not agent issue

The parallel billing model: pay per sandbox-second

Understanding the billing model is essential before you architect a large fan-out. The model is simple: you pay per sandbox-second, not per agent or per request.

The core math

Running 100 agents in parallel for 30 seconds costs exactly the same as running 1 agent sequentially for 3,000 seconds (50 minutes). The compute is identical; only the wall-clock changes.

The ab0t.micro rate is approximately $0.011/hour ($0.0000031/second). A ab0t.medium (4 GB, 2 vCPU) is approximately $0.044/hour.

Auto-stop: billing ends when the task ends

When you create a sandbox with auto_stop_minutes=5, the sandbox automatically stops (and billing stops) 5 minutes after the last command completes. You are not billed for idle time. If your agent finishes in 47 seconds, you pay for 47 seconds, not 5 minutes.

This is the key difference from running 100 EC2 instances:

Real cost projections for common fleet sizes

When parallel is wrong

• Strict ordering required. If task N+1 depends on task N, parallel breaks it. Use a single agent or a state machine.
• Strong consistency in writes. 100 agents writing to the same row = race conditions. Either single-writer or per-shard.
• Downstream is single-tenant. If the downstream system can only process one client at a time (some legacy ERPs), parallel doesn't speed anything up.
• Cost per task is dominated by model. If 100 tasks cost $50 in model calls, the wall-clock saving is real but the total isn't free.

Worked example: parallel code review across a 200-file PR

This is a concrete end-to-end architecture for a real use case: Devin-style parallel code review. Goldman Sachs runs this pattern today for large refactoring PRs. Here is how to build it yourself.

The problem

A 200-file pull request is submitted. A single reviewer (human or AI) reading files sequentially takes hours. The files are mostly independent: changes in auth/ don't affect billing/. The right architecture is to split by logical module and review in parallel.

Architecture diagram

GitHub Webhook (PR opened)
        │
        ▼
  Orchestrator (Python)
  ┌─────────────────────────────────────────────┐
  │ 1. git diff --name-only HEAD~1              │
  │ 2. group_by_module(files)  → 20 groups     │
  │    auth/: 12 files                          │
  │    billing/: 8 files                        │
  │    api/: 15 files  ... (17 more groups)    │
  │ 3. asyncio.gather(review_group × 20)        │
  └─────────────────────────────────────────────┘
        │ (20 concurrent)
        ▼
┌─────────────┐ ┌─────────────┐     ┌─────────────┐
│ Sandbox #1  │ │ Sandbox #2  │ ... │ Sandbox #20 │
│ auth/       │ │ billing/    │     │ scripts/    │
│ Claude Code │ │ Claude Code │     │ Claude Code │
│ reviews 12  │ │ reviews 8   │     │ reviews 7   │
│ files       │ │ files       │     │ files       │
└──────┬──────┘ └──────┬──────┘     └──────┬──────┘
       │               │                   │
       └───────────────┴───────────────────┘
                        │ (fan-in)
                        ▼
              Aggregator (Python)
              ┌─────────────────────┐
              │ merge review JSONs  │
              │ deduplicate issues  │
              │ rank by severity    │
              └─────────────────────┘
                        │
                        ▼
              POST /pulls/123/reviews  (GitHub API)
              Structured review with per-file comments

Orchestrator code

import asyncio, json, httpx, os
from anthropic import Anthropic

client = Anthropic()
SANDBOX_URL = os.environ["SANDBOX_URL"]
API_KEY = os.environ["SANDBOX_API_KEY"]

def group_by_module(files: list[str], max_per_group: int = 10) -> list[list[str]]:
    """Group files by top-level directory; cap group size at max_per_group."""
    from collections import defaultdict
    by_module: dict = defaultdict(list)
    for f in files:
        module = f.split("/")[0] if "/" in f else "root"
        by_module[module].append(f)
    # split large modules into sub-groups
    groups = []
    for files_in_module in by_module.values():
        for i in range(0, len(files_in_module), max_per_group):
            groups.append(files_in_module[i : i + max_per_group])
    return groups

async def review_group(group_files: list[str], diff_content: str,
                        http: httpx.AsyncClient) -> dict:
    """Review one group of files in a dedicated sandbox."""
    resp = await http.post("/api/sandboxes", json={
        "name": f"review-{group_files[0].replace('/', '-')}",
        "instance_type": "ab0t.micro", "auto_stop_minutes": 8
    })
    sandbox_id = resp.json()["sandbox_id"]

    try:
        prompt = (f"Review these files from a pull request diff:\n"
                  f"Files: {', '.join(group_files)}\n\n"
                  f"Diff excerpt:\n{diff_content[:8000]}\n\n"
                  "Return JSON: {\"issues\": [{\"file\": ..., \"line\": ..., \"severity\": ..., \"comment\": ...}], \"summary\": ...}")
        msg = client.messages.create(
            model="claude-sonnet-4-5", max_tokens=2048,
            system="You are a senior code reviewer. Be precise and actionable.",
            messages=[{"role": "user", "content": prompt}],
        )
        return json.loads(msg.content[0].text)
    except Exception as e:
        return {"issues": [], "error": str(e), "files": group_files}
    finally:
        await http.post(f"/api/sandboxes/{sandbox_id}/stop")

async def parallel_review(changed_files: list[str], diff: str) -> dict:
    """Orchestrate parallel review: group → fan-out → aggregate."""
    groups = group_by_module(changed_files, max_per_group=10)
    print(f"Reviewing {len(changed_files)} files in {len(groups)} parallel groups")

    async with httpx.AsyncClient(
        base_url=SANDBOX_URL,
        headers={"Authorization": f"Bearer {API_KEY}"}, timeout=300.0
    ) as http:
        reviews = await asyncio.gather(
            *[review_group(g, diff, http) for g in groups],
            return_exceptions=True
        )

    # Aggregate: flatten issues, sort by severity
    all_issues = []
    for r in reviews:
        if isinstance(r, dict) and "issues" in r:
            all_issues.extend(r["issues"])
    severity_order = {"critical": 0, "high": 1, "medium": 2, "low": 3}
    all_issues.sort(key=lambda x: severity_order.get(x.get("severity", "low"), 3))
    return {"total_issues": len(all_issues), "issues": all_issues,
            "groups_reviewed": len(groups), "files_reviewed": len(changed_files)}

A 200-file PR split into 20 groups of 10 completes in roughly 60-90 seconds of wall-clock time, at a compute cost under $0.05. The same review done sequentially by one agent takes 20-30 minutes.

Claude Code's Agent Teams: parallel out of the box

If your fan-out is coding work, Claude Code Agent Teams already handles parallel workers across git worktrees. You spawn N workers, each on a separate branch / issue, results converge in a final review. Same architecture, packaged. Great for "10 PRs in parallel" workflows; less great for "100 vendor portals" because Agent Teams assumes a shared repo.

For non-code fan-outs (research, scraping, vendor portals), build the queue + sandbox pool yourself. The platform's primitives make this a 50-line Python script.

Using Claude Code CLI inside a parallel fleet

Claude Code is a TUI that can also run non-interactively via --print. This makes it ideal for parallel agent work: your orchestrator spawns N sandboxes, runs claude --print in each, and collects results. Each Claude Code instance gets a clean sandbox with the repo already cloned.

import asyncio, httpx, os

SANDBOX_URL = os.environ["SANDBOX_URL"]
API_KEY = os.environ["SANDBOX_API_KEY"]

async def claude_code_task(task: dict, http: httpx.AsyncClient) -> dict:
    """Run Claude Code non-interactively inside a sandbox."""
    # Create sandbox with repo pre-cloned
    resp = await http.post("/api/sandboxes", json={
        "name": f"cc-{task['id']}",
        "instance_type": "ab0t.small",
        "auto_stop_minutes": 15,
        "startup_script": (
            f"git clone {task['repo_url']} /workspace/repo && "
            "cd /workspace/repo && git checkout -b review-branch"
        ),
    })
    sandbox_id = resp.json()["sandbox_id"]
    await _wait_ready(sandbox_id, http)

    try:
        # Run Claude Code non-interactively
        exec_resp = await http.post(
            f"/api/sandboxes/{sandbox_id}/execute",
            json={
                "command": (
                    f"cd /workspace/repo && claude --print \"{task['prompt']}\" "
                    "--output-format json"
                ),
                "timeout": 600,
            },
            timeout=650.0,
        )
        result = exec_resp.json()
        return {
            "task_id": task["id"],
            "stdout": result.get("stdout", ""),
            "exit_code": result.get("exit_code", -1),
            "ok": result.get("exit_code") == 0,
        }
    except Exception as e:
        return {"task_id": task["id"], "error": str(e), "ok": False}
    finally:
        await http.post(f"/api/sandboxes/{sandbox_id}/stop")

async def _wait_ready(sandbox_id: str, http: httpx.AsyncClient, timeout: int = 120):
    for _ in range(timeout // 2):
        r = await http.get(f"/api/sandboxes/{sandbox_id}")
        if r.json().get("status") == "running":
            return
        await asyncio.sleep(2)
    raise TimeoutError(f"Sandbox {sandbox_id} did not start in time")

Choosing between Claude Code and direct Anthropic SDK

Real-world examples

These are production deployments, not hypotheticals. Numbers are from real implementations or publicly referenced deployments.

Goldman Sachs: parallel code review at the PR level

Goldman Sachs gave Devin a desk for parallel code review. The pattern: each PR in the review queue gets its own Devin agent (one sandbox per PR), running concurrently. A 50-PR queue that previously took a senior developer 8 hours to triage runs in under 20 minutes with 50 parallel agents. The aggregate review output is loaded into their internal review dashboard. Issues are ranked by severity; critical-severity issues auto-flag the PR for human review.

Block / Goose: developer workflow parallelism

Block open-sourced Goose after deploying it internally for developer productivity. A common pattern: when a developer submits a ticket, Goose spawns 3-5 parallel agents to explore different implementation approaches. The fastest, cleanest solution "wins" and is presented to the developer. This is the quorum aggregation pattern — not waiting for all N, but taking the best of N parallel attempts.

AP clerk fleet: 20 vendor portals in parallel

An accounts payable team manages 20 supplier relationships, each with a different web portal for invoice submission. Sequential processing (one agent, one portal at a time) takes 3-4 hours nightly. Parallel processing (20 browser sandboxes, one per vendor) completes in 12 minutes. Each sandbox has vendor-specific credentials injected at creation time; no credential sharing between sandboxes. The daily DLQ usually has 1-2 portals that required CAPTCHA handling or had downtime — these are reviewed manually and retried the next morning.

Research analyst: 50 concurrent literature searches

A pharmaceutical research team runs nightly literature searches across 50 drug targets simultaneously. Each agent uses a browser sandbox to query PubMed, extract abstracts, and classify findings by relevance tier. 50 agents × 15 minutes each = 12.5 hours of equivalent sequential work, completed in 15 minutes wall clock. Monthly compute cost: $6.30. The team's previous approach (a single research assistant running searches sequentially) took 3 full days per week.

Competitive intelligence: 30 competitor sites, daily

A SaaS company monitors 30 competitor pricing pages, changelog pages, and job boards daily. Each of 30 browser sandboxes visits its assigned competitor pages, extracts structured data, and posts a JSON summary to a results queue. An aggregator compares today's data to yesterday's, identifies changes (new pricing tier, new feature announced, new senior hire), and posts a digest to a Slack channel. The whole run costs approximately $1.50/day in compute ($45/month). The team's previous manual process (one person spending 2 hours per day) was $4,000/month in labor cost — an 89x cost reduction.

Test suite parallelism: 10 feature branches, 9 minutes

A CI/CD team runs the full test suite (8 minutes per branch) against 10 active feature branches simultaneously. Each terminal sandbox clones the repo, checks out the branch, installs dependencies, and runs pytest. Results are posted back to GitHub as check runs. Previously, 10 branches × 8 minutes = 80 minutes on a serial CI runner. With 10 parallel sandboxes: 8 minutes wall clock, plus ~1 minute for sandbox startup and result aggregation = 9 minutes total. 89% reduction in developer wait time for test feedback.

Choosing the right agent harness for parallel work

The harness you run inside each sandbox matters as much as the orchestration around it. Different harnesses have different strengths; the wrong choice adds complexity without adding capability.

The 2024 anti-pattern: LangChain/CrewAI for parallel work

If you are migrating from a 2024 LangChain or CrewAI setup, here is why it didn't scale to 100 agents and what to do instead.

What LangChain/CrewAI does: Runs multiple agents as Python threads or async tasks in a single process. "Parallelism" is cooperative multitasking — one agent yields when it waits for an LLM response, another runs. No isolation. Shared heap. One OOM can kill all agents.

Why it breaks at 100 agents:

• Python's GIL means CPU-bound work (image processing, local model inference) is not actually parallel.
• Shared filesystem — agents write to the same /tmp; naming collisions cause data corruption at scale.
• Shared network — rate limits on one agent affect all agents using the same HTTP client pool.
• One exception in the wrong place can tear down the entire process and all 99 other "agents."
• No per-agent billing — you pay for the whole process (usually an EC2 instance) regardless of how many agents are "active."

The migration path:

1. Extract the per-agent logic (what one agent does) from the orchestration logic (how agents are coordinated).
2. Wrap the per-agent logic in a function that accepts a task dict and returns a result dict.
3. Replace the CrewAI Crew instantiation with an asyncio.gather() call over your per-agent function, each in its own sandbox.
4. The system prompt and tool definitions from your LangChain/CrewAI agents become the system and tools parameters in your Anthropic SDK calls.

The migration is typically 1-2 days of engineering work for a production deployment that was previously taking 3-4 days to run 50 tasks sequentially.

Monitoring and observability for parallel fleets

A fan-out running 100 agents generates 100x the signals of a single agent. Without structured observability, debugging a partial failure across a large fleet is painful. Build these three instruments from day one.

Per-run structured logging

Every fan-out run should emit a structured summary: tasks dispatched, tasks completed, tasks failed, wall-clock duration, total sandbox-seconds, estimated cost. Log this to wherever your team watches metrics (Datadog, CloudWatch, a simple database table).

import time, json, logging
from dataclasses import dataclass, field, asdict
from typing import Optional

@dataclass
class RunSummary:
    run_id: str
    fan_out_name: str
    started_at: float = field(default_factory=time.time)
    ended_at: Optional[float] = None
    tasks_total: int = 0
    tasks_succeeded: int = 0
    tasks_failed: int = 0
    sandbox_seconds_total: float = 0.0
    errors: list = field(default_factory=list)

    def finish(self):
        self.ended_at = time.time()
        duration = self.ended_at - self.started_at
        compute_cost = (self.sandbox_seconds_total / 3600) * 0.0116  # ab0t.micro/hr rate
        summary = asdict(self)
        summary["wall_clock_seconds"] = duration
        summary["estimated_compute_cost_usd"] = round(compute_cost, 4)
        summary["success_rate"] = (
            self.tasks_succeeded / self.tasks_total
            if self.tasks_total > 0 else 0
        )
        logging.info("fan_out_complete", extra={"run_summary": summary})
        return summary

# Usage in your orchestrator:
# run = RunSummary(run_id="run-2026-05-10-001", fan_out_name="pr_review")
# run.tasks_total = len(tasks)
# ... after each task: run.tasks_succeeded += 1 or run.tasks_failed += 1
# run.finish()  →  logs the structured summary

Health probes for long-running fleets

For a fleet that runs continuously (pool pattern draining an indefinite queue), add a health probe that your monitoring system can ping to verify the fleet is alive and making progress:

from fastapi import FastAPI
import time

app = FastAPI()
_last_task_completed_at: float = time.time()
_tasks_in_last_minute: int = 0
_active_sandboxes: int = 0

@app.get("/health")
async def health():
    now = time.time()
    stale = now - _last_task_completed_at > 300  # no completions in 5 min = stale
    return {
        "status": "stale" if stale else "ok",
        "active_sandboxes": _active_sandboxes,
        "tasks_per_minute": _tasks_in_last_minute,
        "last_completion_seconds_ago": int(now - _last_task_completed_at),
    }

Cost guardrails: hard stops and soft alerts

Two levels of cost control are required for any production fan-out:

Result aggregation: beyond "wait for all"

Collecting results is where most fan-out implementations are under-specified. A robust aggregation layer handles the full range of outcomes: all succeed, partial succeed, all fail, timeout before completion. Here are the production patterns.

Streaming aggregation: process results as they arrive

For dashboards, incremental reports, or high-latency fan-outs where you want partial results while the fleet is still running, use an async generator that yields results as each task completes rather than waiting for all N.

import asyncio

async def streaming_fanout(tasks: list):
    """Yield results as each task completes, not when all are done."""
    pending = {
        asyncio.create_task(run_one_task(t), name=t["id"]): t
        for t in tasks
    }
    completed_count = 0

    while pending:
        done, pending = await asyncio.wait(
            pending, return_when=asyncio.FIRST_COMPLETED
        )
        for task in done:
            completed_count += 1
            try:
                result = task.result()
                yield result
            except Exception as e:
                yield {"task_id": task.get_name(), "error": str(e), "ok": False}

# Usage: process each result as soon as it's ready
# async for result in streaming_fanout(tasks):
#     if result["ok"]:
#         await write_to_dashboard(result)

Best-of-N: take the first M results, cancel the rest

When you need confirmation from M out of N agents (e.g., 3 of 5 agents agree on a classification, or 1 of 10 agents produces an acceptable answer and you want the fastest one), use the "quorum" or "first-N" pattern:

import asyncio

async def first_n_fanout(tasks: list, need: int = 1) -> list:
    """Run all tasks; return as soon as 'need' successes are collected.
    Cancel remaining tasks to stop sandbox billing.

    Use case: run 5 agents on the same research task, take the fastest
    good answer. 80% cost reduction vs waiting for all 5."""
    pending_tasks = {
        asyncio.create_task(run_one_task(t)): t for t in tasks
    }
    successes = []

    while pending_tasks and len(successes) < need:
        done, still_pending = await asyncio.wait(
            pending_tasks, return_when=asyncio.FIRST_COMPLETED
        )
        for done_task in done:
            del pending_tasks[done_task]
            try:
                r = done_task.result()
                if r.get("ok"):
                    successes.append(r)
            except Exception:
                pass

    # Cancel everything still running (stops their sandboxes and billing)
    for t in pending_tasks:
        t.cancel()
    await asyncio.gather(*pending_tasks, return_exceptions=True)

    return successes

Structured aggregation: merging JSON results from multiple agents

When each agent returns structured JSON (e.g., a list of issues, a set of extracted facts), the aggregation step needs to merge these without losing information and without creating duplicates. A simple approach: use a set of content hashes for deduplication, then sort by the most important field.

import hashlib, json
from typing import Any

def merge_results(reviews: list[dict], sort_key: str = "severity") -> dict:
    """Merge structured results from N agents into a deduplicated aggregate.

    Expects each review to have an 'issues' list of dicts.
    Deduplicates by content hash; sorts by sort_key."""
    seen: set = set()
    all_items: list = []
    error_count = 0

    severity_order = {"critical": 0, "high": 1, "medium": 2, "low": 3, "info": 4}

    for review in reviews:
        if not isinstance(review, dict) or "issues" not in review:
            error_count += 1
            continue
        for item in review.get("issues", []):
            # Deduplicate: hash (file, line, comment) to detect identical findings
            fingerprint = hashlib.md5(
                json.dumps({
                    "file": item.get("file"),
                    "line": item.get("line"),
                    "comment": item.get("comment", "")[:100],
                }, sort_keys=True).encode()
            ).hexdigest()
            if fingerprint not in seen:
                seen.add(fingerprint)
                all_items.append(item)

    all_items.sort(
        key=lambda x: severity_order.get(x.get(sort_key, "info"), 4)
    )
    return {
        "total_items": len(all_items),
        "agent_errors": error_count,
        "items": all_items,
        "critical_count": sum(1 for i in all_items if i.get("severity") == "critical"),
    }

Timeout-bounded aggregation

For SLA-bound fan-outs where you must return a result within T seconds regardless of how many agents finished, use asyncio.wait_for() with a timeout:

import asyncio

async def time_bounded_fanout(tasks: list, timeout_seconds: float = 300) -> dict:
    """Run all tasks; return whatever arrived within timeout_seconds.
    Tasks that didn't finish go to the retry queue, not silently dropped."""
    all_coroutines = [run_one_task(t) for t in tasks]
    task_objs = [asyncio.create_task(c) for c in all_coroutines]

    try:
        await asyncio.wait_for(
            asyncio.gather(*task_objs, return_exceptions=True),
            timeout=timeout_seconds,
        )
    except asyncio.TimeoutError:
        pass  # Expected — some tasks may not be done yet

    results, timed_out = [], []
    for task_obj, original_task in zip(task_objs, tasks):
        if task_obj.done():
            try:
                results.append(task_obj.result())
            except Exception as e:
                results.append({"task_id": original_task["id"], "error": str(e), "ok": False})
        else:
            task_obj.cancel()  # Stop the sandbox and billing
            timed_out.append(original_task["id"])

    return {
        "results": results,
        "timed_out_task_ids": timed_out,  # push these to retry queue
        "completed_in_time": len(results),
        "timed_out_count": len(timed_out),
    }

Production operations checklist

Before going to production with a parallel agent fleet, verify each item on this checklist:

Pre-flight

• Dry run with 3 tasks. Verify the full loop: dispatch → sandbox create → task execute → result collected → sandbox stopped. Check the DLQ is wired and a test failure lands there correctly.
• Test the partial failure path. Deliberately fail one task (e.g., pass an invalid input). Verify that the remaining tasks complete and the failed task lands in the DLQ, not silently dropped.
• Verify sandbox auto-stop. Create a sandbox with auto_stop_minutes=1, do nothing, wait 90 seconds, verify it stopped. This confirms billing will not run away on idle sandboxes.
• Set the workspace monthly budget. Set it to 2x your expected monthly spend. This gives headroom for growth while capping runaway costs.
• Test retry idempotency. Run the same task twice with the same task ID. Verify the result is the same both times and no duplicate records appear in your output store.

Day 2 operations

• Check the DLQ daily. Set a Slack alert if DLQ depth exceeds 5. Triage within 24 hours. A growing DLQ that nobody watches is a signal that part of your fleet is silently failing.
• Watch the success rate trend. If your success rate drops from 99% to 94% over a week, something changed — a vendor updated their login flow, an API added CAPTCHA, a model changed behavior. Catch it early.
• Review cost weekly. Plot compute cost per run. A sudden spike means either more tasks or longer-running tasks. Investigate before it compounds.
• Rotate credentials quarterly. Credentials injected into sandboxes (API keys, passwords) should be rotated. Update them in your secrets store; the next fan-out run picks them up automatically.

Scaling up

• Load test before scaling. Run 10x your normal task count in a staging environment. Verify the orchestrator doesn't bottleneck (it shouldn't — asyncio handles 1,000 concurrent coroutines cleanly), that your result store handles the write rate, and that your DLQ doesn't fill up with false positives under load.
• Request concurrency increase before you need it. If you're on Growth tier (100 concurrent sandboxes) and plan to scale to 300, request the increase a week before your planned scale date. Same-day increases are possible but not guaranteed.
• Consider warm pools at high volume. At 500+ runs/day, the 20-40 second startup latency per run adds up. Warm pools eliminate this. See Warm Pools: Sub-Second Agent Start.

Quickstart: your first parallel fleet in 50 lines

Copy this and run it. It fans out 10 tasks to 10 parallel sandboxes using the Anthropic SDK, collects results, handles partial failures, and logs a run summary. Requires SANDBOX_API_KEY, SANDBOX_URL, and ANTHROPIC_API_KEY set in the environment.

#!/usr/bin/env python3
"""
Parallel agent fleet quickstart — Sandbox Platform + Anthropic SDK.
Fans out N tasks to N sandboxes; prints a run summary.

Install:  pip install anthropic httpx
Set:      export SANDBOX_API_KEY=ab0t_sk_live_...
          export SANDBOX_URL=https://sandbox.dev.ab0t.com
          export ANTHROPIC_API_KEY=sk-ant-...
Run:      python parallel_quickstart.py
"""

import asyncio, time, os, httpx
from anthropic import Anthropic

client = Anthropic()
SANDBOX_URL = os.environ["SANDBOX_URL"]
API_KEY     = os.environ["SANDBOX_API_KEY"]

# 10 classification tasks — each runs in its own sandbox
TASKS = [
    {"id": f"t{i:03d}", "prompt": f"Classify this text as POSITIVE, NEUTRAL, or NEGATIVE. "
                              f"Return only the label. Text: '{SAMPLE_TEXTS[i]}'"}
    for i in range(10)
]
SAMPLE_TEXTS = [
    "The product exceeded all expectations, truly excellent.",
    "Delivery was a week late and the packaging was damaged.",
    "It arrived on time.",
    "Best purchase I've made this year, highly recommend.",
    "Average quality, nothing special.",
    "Complete waste of money, broke after two days.",
    "Decent for the price point, does what it says.",
    "Outstanding customer service when I had an issue.",
    "The color was wrong, had to return it.",
    "Works as advertised, no complaints.",
]

async def run_task(task: dict, http: httpx.AsyncClient) -> dict:
    # Create sandbox
    resp = await http.post("/api/sandboxes", json={
        "name": f"qs-{task['id']}", "instance_type": "ab0t.micro", "auto_stop_minutes": 5
    })
    sandbox_id = resp.json()["sandbox_id"]
    try:
        msg = client.messages.create(
            model="claude-haiku-4-5", max_tokens=16,
            messages=[{"role": "user", "content": task["prompt"]}],
        )
        return {"id": task["id"], "result": msg.content[0].text.strip(), "ok": True}
    except Exception as e:
        return {"id": task["id"], "error": str(e), "ok": False}
    finally:
        await http.post(f"/api/sandboxes/{sandbox_id}/stop")

async def main():
    t0 = time.time()
    print(f"Fanning out {len(TASKS)} tasks...")
    async with httpx.AsyncClient(
        base_url=SANDBOX_URL,
        headers={"Authorization": f"Bearer {API_KEY}"},
        timeout=120.0,
    ) as http:
        results = await asyncio.gather(
            *[run_task(t, http) for t in TASKS], return_exceptions=True
        )
    ok = [r for r in results if isinstance(r, dict) and r["ok"]]
    fail = [r for r in results if not (isinstance(r, dict) and r["ok"])]
    print(f"\nCompleted in {time.time()-t0:.1f}s")
    print(f"{len(ok)}/{len(TASKS)} succeeded, {len(fail)} failed")
    for r in ok:
        print(f"  {r['id']}: {r['result']}")

if __name__ == "__main__":
    asyncio.run(main())

# Expected output:
# Fanning out 10 tasks...
# Completed in 28.3s
# 10/10 succeeded, 0 failed
#   t000: POSITIVE
#   t001: NEGATIVE
#   t002: NEUTRAL
#   ...  (all 10 ran in parallel, wall clock ≈ single task time)

Scale this from 10 to 100 by updating TASKS — the orchestrator logic is unchanged. The asyncio.gather() handles the concurrency automatically.

Common mistakes from the field

These are the six mistakes teams make on their first real fan-out deployment. Each one is a real failure mode, not a theoretical concern.

1. Not handling partial failures — treating the result list as complete

You fan out 100 tasks. 97 succeed, 3 fail. Your aggregation code does for result in results: process(result) without checking whether each result is an error. Three corrupt entries flow into your database. Fix: use asyncio.gather(return_exceptions=True) and explicitly tally successes vs failures. Log failures to a DLQ. Never pass an exceptions list to your aggregation logic.

2. Hardcoding the agent count instead of making it configurable

Your orchestrator has CONCURRENCY = 100 in the source. The downstream vendor changes their rate limit from 50 req/min to 10 req/min. Your fleet is now hammering the vendor and getting 429s. Fix: make CONCURRENCY an environment variable, cap it with a asyncio.Semaphore, and tune per-environment without code changes.

3. Shared filesystem between agents

You mount an S3 bucket (or a host directory) into all sandboxes so agents can "share work." Agent 12 writes a temp file with the same name as Agent 47's temp file. One overwrites the other. Now you have a data corruption bug that only triggers at scale. Fix: each agent gets a path prefixed by its task ID. Or better: don't use a shared filesystem — use an external store with task-scoped keys.

4. Not setting per-agent timeouts

One agent hits a vendor page that never responds. The browser sandbox idles, the HTTP request never times out, and the sandbox keeps billing. Your aggregator is waiting for a result that will never arrive. Fix: always set auto_stop_minutes when creating sandboxes. Set a timeout on every HTTP call inside the sandbox. Set a max_execution_time in your orchestrator for each asyncio.Task.

5. Non-idempotent tasks

A sandbox crashes halfway through writing a record to a database. Your orchestrator retries the task. The second run inserts a duplicate. Fix: every task should be idempotent — running it twice produces the same result as running it once. Use task IDs as idempotency keys. Check for existence before inserting. This is especially important for billing-critical workflows.

6. Confusing asyncio concurrency with true parallelism

You see asyncio.gather() and assume 100 tasks are running "in parallel." They are concurrent — running interleaved on one thread — but not parallel. CPU-bound work (e.g., encoding a video, training a model) will not see a speedup from asyncio. For true CPU parallelism, you need separate processes or, better, separate sandbox VMs. Sandbox Platform solves this natively: each sandbox is a separate VM with separate CPUs. The asyncio.gather() in your orchestrator only handles the I/O-bound coordination; the real parallel work happens in the sandboxes.

Advanced concurrency control

Basic asyncio.gather() works for simple fan-outs. When you need finer control — rate limiting, progressive dispatch, or per-domain concurrency caps — here are the production-grade patterns.

Semaphore-based rate limiting

An asyncio.Semaphore caps how many tasks run concurrently. This is the right tool when you need to respect a downstream rate limit regardless of how many tasks are queued.

import asyncio

async def rate_limited_fanout(tasks: list, max_concurrent: int = 10) -> list:
    """Fan-out tasks with at most max_concurrent running at once."""
    sem = asyncio.Semaphore(max_concurrent)

    async def bounded_task(task):
        async with sem:
            return await run_one_task(task)

    results = await asyncio.gather(
        *[bounded_task(t) for t in tasks],
        return_exceptions=True,
    )
    return [r for r in results if isinstance(r, dict)]

# Example: 200 tasks, never more than 20 sandboxes active at once
# asyncio.run(rate_limited_fanout(my_200_tasks, max_concurrent=20))

Per-domain concurrency caps

When your fan-out hits multiple different downstream APIs, you may need different concurrency limits per domain. A vendor portal that allows 5 concurrent sessions should cap at 5; an internal API that handles 100 req/sec should allow 50 concurrent agents.

from collections import defaultdict
import asyncio

# Per-domain semaphores — configure based on each vendor's limits
DOMAIN_LIMITS = {
    "vendor-a.com": 3,   # strict: only 3 simultaneous sessions allowed
    "vendor-b.com": 10,  # moderate
    "internal-api": 50,  # generous: high-throughput internal API
}
DEFAULT_LIMIT = 5

_domain_sems: dict = defaultdict(lambda: None)

def get_sem(domain: str) -> asyncio.Semaphore:
    if _domain_sems[domain] is None:
        limit = DOMAIN_LIMITS.get(domain, DEFAULT_LIMIT)
        _domain_sems[domain] = asyncio.Semaphore(limit)
    return _domain_sems[domain]

async def run_task_with_domain_cap(task: dict) -> dict:
    domain = task["domain"]
    async with get_sem(domain):
        return await run_one_task(task)

Progressive dispatch with staggered starts

Launching 100 sandboxes simultaneously creates a thundering-herd effect on the sandbox API and on your downstream systems. Stagger the starts over the first 30-60 seconds to smooth out the spike:

import asyncio, random

async def staggered_fanout(tasks: list, spread_seconds: float = 30.0) -> list:
    """Dispatch tasks spread evenly over spread_seconds with jitter."""
    interval = spread_seconds / len(tasks)

    async def delayed_task(task, delay: float):
        jitter = random.uniform(0, interval * 0.5)  # ±25% jitter
        await asyncio.sleep(delay + jitter)
        return await run_one_task(task)

    results = await asyncio.gather(
        *[delayed_task(t, i * interval) for i, t in enumerate(tasks)],
        return_exceptions=True,
    )
    return results

# 100 tasks spread over 30 seconds = one new sandbox every 0.3s on average
# With jitter: no two sandboxes start at exactly the same millisecond

Retry with exponential backoff and jitter

Transient failures (network glitch, sandbox briefly unavailable, rate limit 429) should be retried. Hard failures (invalid input, auth error, DLQ-bound task) should not. The difference: transient errors are retryable on the same input; hard errors require human intervention.

import asyncio, random

# Errors that are safe to retry
RETRYABLE_ERRORS = {"sandbox_unavailable", "rate_limit", "network_timeout"}

async def run_with_retry(task: dict, max_retries: int = 3) -> dict:
    """Run task with exponential backoff on retryable errors."""
    last_error = None
    for attempt in range(max_retries + 1):
        if attempt > 0:
            # Exponential backoff: 2^attempt seconds ± 20% jitter
            base_delay = 2 ** attempt
            jitter = random.uniform(0.8, 1.2)
            await asyncio.sleep(base_delay * jitter)

        result = await run_one_task(task)

        if result["ok"]:
            if attempt > 0:
                result["retries"] = attempt
            return result

        error_type = result.get("error_type", "unknown")
        last_error = result

        if error_type not in RETRYABLE_ERRORS:
            # Hard failure — don't retry, send to DLQ immediately
            result["dlq"] = True
            return result

    # Exhausted retries — DLQ
    last_error["dlq"] = True
    last_error["retries"] = max_retries
    return last_error

MCP integration: exposing your fleet to any agent

Model Context Protocol (MCP) is the universal connector that lets Claude Code, Cursor, Gemini CLI, Codex CLI, and any MCP-compatible agent call your fan-out fleet as a tool. Instead of building a separate orchestrator for each agent harness, you expose your parallel fleet as an MCP tool and any agent can dispatch tasks to it.

from mcp.server import Server
from mcp.types import Tool, TextContent
import asyncio, json

server = Server("parallel-fleet")

@server.list_tools()
async def list_tools():
    return [
        Tool(
            name="fan_out_tasks",
            description="""Dispatch a list of tasks to a parallel agent fleet.
Each task runs in its own isolated sandbox. Returns results when all complete.
Use this for: code review across many files, data extraction from many URLs,
parallel research queries, test execution across branches.""",
            inputSchema={
                "type": "object",
                "properties": {
                    "tasks": {
                        "type": "array",
                        "description": "List of {id, prompt} objects to run in parallel",
                        "items": {"type": "object"}
                    },
                    "max_concurrent": {
                        "type": "integer",
                        "description": "Maximum parallel sandboxes (default: 20)",
                        "default": 20
                    }
                },
                "required": ["tasks"]
            }
        )
    ]

@server.call_tool()
async def call_tool(name: str, arguments: dict):
    if name == "fan_out_tasks":
        tasks = arguments["tasks"]
        max_concurrent = arguments.get("max_concurrent", 20)
        results = await rate_limited_fanout(tasks, max_concurrent)
        successes = [r for r in results if r.get("ok")]
        failures = [r for r in results if not r.get("ok")]
        return [TextContent(type="text", text=json.dumps({
            "total": len(tasks),
            "succeeded": len(successes),
            "failed": len(failures),
            "results": successes,
            "failures": failures,
        }, indent=2))]

Once this MCP server is running, any agent that speaks MCP — Claude Code, Cursor, Gemini CLI, Codex CLI — can call fan_out_tasks as a native tool. An operator prompt like "review all 200 files in this PR" triggers the fan-out automatically, without the operator knowing anything about sandboxes.

Scale limits, anti-patterns, and the right granularity

Practical concurrency limits

In practice, hitting the sandbox limit is rare. The real bottleneck for most teams is the LLM API rate limit (tokens per minute) or downstream API rate limits (requests per minute to a vendor). Design around those before worrying about sandbox count.

When NOT to parallelize

• High coordination overhead. If agents spend more time passing state to each other than doing actual work, the coordination cost swamps the parallelism benefit. A good rule: if >30% of your agent's runtime is reading/writing shared state, consider a single-agent design.
• Shared mutable state. If all N agents need to write to the same database row, parallel writes create contention and race conditions. Use a single writer or a dedicated aggregation step.
• Very short tasks where startup dominates. If each task takes 10 seconds of work but sandbox startup takes 25 seconds, you're paying more in startup overhead than you're saving in parallelism. Batch tasks until each agent runs for at least 60 seconds.
• Strict ordering. If task N+1 must read the result of task N before starting, parallelism doesn't help and adds complexity. Use a sequential pipeline instead.
• A downstream that is genuinely single-tenant. Some legacy ERPs process one API call at a time. Sending 100 parallel requests doesn't speed anything up — it just queues at the vendor. Match your concurrency to the downstream's actual throughput.

The right granularity heuristic

The startup cost for a sandbox is typically 20-40 seconds. To amortize this:

• Minimum task duration: 60 seconds of actual work per sandbox. Below 30 seconds, startup overhead dominates.
• Sweet spot: 2-15 minutes per sandbox. Long enough to amortize startup; short enough that a failure isn't catastrophic.
• Too long: Tasks longer than 60 minutes should be checkpointed. If a 90-minute task fails at minute 89, you want to retry from a checkpoint, not from scratch.

Anti-pattern: 100 agents for 100 two-second tasks. Startup takes 25 seconds, work takes 2 seconds, total elapsed is 27 seconds — but you paid for 27 seconds of sandbox time where 2 seconds was useful. Instead, batch: give each of 10 agents 10 tasks in sequence. Each agent runs for ~45 seconds, amortizing the startup cost across 10 tasks.

Frequently Asked Questions

How many sandboxes can I run concurrently?

Per workspace, the default limit is 100 concurrent sandboxes. Growth tier supports up to 500 (burst-requestable via dashboard) and Enterprise supports 1,000+. In practice, the bottleneck is almost always the downstream system (a rate-limited API, a vendor portal) or your LLM token budget, not sandbox count.

What does it cost to run 100 agents in parallel for 15 minutes?

100 ab0t.micro sandboxes running for 15 minutes costs approximately $0.83 in compute. LLM token costs depend on your model and task complexity and typically dominate — a single Claude claude-opus-4-5 call with 2K input + 1K output tokens costs ~$0.024. Your total per-run cost is almost always model tokens, not compute.

Is true parallelism possible with Python asyncio?

asyncio is concurrent but not parallel — it is single-threaded and cannot run CPU-bound work simultaneously. For true parallelism you need separate sandbox VMs (which is what Sandbox Platform provides). Each sandbox runs on a separate VM with its own CPUs, so you get real OS-level parallelism, not cooperative scheduling.

How do I prevent one failing agent from aborting the whole fan-out?

Use asyncio.gather(return_exceptions=True). This collects exceptions as values rather than raising them, so one sandbox failure doesn't cancel all the others. Each agent also runs in its own sandbox VM, so a crash at the OS level in one doesn't affect any other sandbox.

What is the map pattern and when should I use it?

The map pattern creates one sandbox per task, runs all tasks concurrently via asyncio.gather(), and collects all results when done. Use it when you have a fixed task list, tasks are independent, and you want all results before proceeding. It's the simplest pattern and covers most embarrassingly parallel workloads.

When should I use a pool pattern instead of a map pattern?

Use a pool when task count is dynamic or very large, tasks vary in duration, or you need to cap concurrency below task count (to respect downstream rate limits). A pool of 10 workers draining a 1,000-task queue is more efficient than 1,000 simultaneous sandboxes when tasks are short or downstream APIs are rate-limited.

What isolation does each sandbox provide?

Each sandbox is a separate VM with its own process tree, filesystem, network namespace, and environment variables. Agent A cannot read Agent B's files, access Agent B's env vars, or consume Agent B's outbound rate-limit quota. Credential leakage between tenants is architecturally impossible — there is no shared address space.

How do I pass data between agents in a pipeline pattern?

Download the output file from the upstream sandbox using the files API (GET /api/sandboxes/{id}/files?path=...) and upload it to the downstream sandbox (POST /api/sandboxes/{id}/files) before starting the next stage. For large files, write to S3 and pass the object key.

What task granularity is too fine for parallel agents?

If each task completes in under 30 seconds, sandbox startup latency (20-40 seconds) dominates wall-clock time. Batch sub-30-second tasks together until each sandbox runs for at least 60-120 seconds. The sweet spot is 2-15 minutes per sandbox — long enough to amortize startup, short enough that a failure isn't catastrophic.

How do Goldman Sachs and Devin use parallel agents?

Goldman Sachs deploys Devin agents in parallel for code review and refactoring — multiple PRs running concurrently, each in its own isolated environment. This is the map pattern: 1 PR = 1 agent = 1 sandbox, results aggregated into a review dashboard. The exact architecture from the worked example in this guide.

Can I use Claude Code CLI in a parallel agent fleet?

Yes. Claude Code runs in each sandbox's terminal environment. You can spawn N sandboxes, run claude --print "<prompt>" or use the Anthropic SDK's Messages API in each, and collect the outputs. For coding tasks that need a full agent loop (read files, run tests, make edits), Claude Code Agent Teams provides this fan-out natively across git worktrees.

What is backpressure and why does it matter?

Backpressure is the mechanism that slows task dispatch when workers are saturated. Without it, dispatching 1,000 tasks to 1,000 sandboxes can overwhelm a downstream API limited to 10 req/sec. Use an asyncio.Semaphore(N) to cap active concurrency, or leave tasks on an SQS queue with a visibility timeout so unprocessed messages stay in the queue until a worker is free.

Should I use the same LLM model across all parallel agents?

Generally yes for a given fan-out — same model, same system prompt, same tool set. Different models per task is a sign you're doing multiple job types in one batch; split those into separate fan-outs. The exception is pipeline stages: use a cheap model (Haiku) for simple fetch/parse stages and a capable model (claude-opus-4-5) only for the reasoning stage.

How do I debug when 5 of 100 agents fail?

Check the dead-letter queue. Each failed task carries its original input, the last error message, and (for browser-based agents) a screenshot of the state at failure. If the same error appears across all 5, it is likely a downstream issue (rate limit, site change, auth expiry). If errors differ, spot-check each individually by re-running that single task against the DLQ entry.

What happens to billing when a sandbox crashes mid-task?

Billing stops when the sandbox stops — whether it completed cleanly, timed out, or crashed. The auto_stop_minutes setting is a hard billing ceiling: a sandbox cannot be billed beyond that duration regardless of what happens inside. Crashed sandboxes are cleaned up automatically; you are not billed for idle time after a crash.

What's next