File I/O: Datasets In, Results Out

Why file I/O is the production bottleneck

The first AI agent workload most teams ship is "agent reads a thing, writes a thing, posts the result." It works fine in dev — the dataset is small, you scp it in, the agent runs, you scp the result out. Then it goes to production. Volumes scale. Files get bigger. Latency matters. The naive scp pattern breaks.

The four common production failures we see, in order of frequency:

1. Sandbox disk fills up. A multi-day research agent accumulates intermediate files; one day there's no space for the next download. Disk monitoring + cleanup is mandatory at scale.
2. Upload bottleneck blocks the agent. Operator uploads a 2 GB file; agent waits 8 minutes for upload to finish before starting. Asynchronous upload + event-trigger fixes this.
3. Result file too large for the destination. Agent writes a 200 MB PDF; tries to attach to Slack (25 MB limit); fails silently. Results-via-S3-pre-signed-URL fixes this universally.
4. Data leaves the residency boundary. EU-resident customer's data accidentally written to a us-east-1 S3 bucket. Region-pinning and bucket-policy enforcement matter.

This guide is the catalog of patterns that handle each of these failure modes. Read it before you ship a production data pipeline; refer back when an existing one starts hitting limits.

The four upload patterns

Pick by file size, network reliability, and trigger style. For most production AI agent fleets, S3 pre-signed URLs are the right default — they decouple upload from processing, scale to any volume, and handle the data-residency story cleanly via bucket placement.

Pattern 1: The Files API (under 1 GB, synchronous)

Simplest pattern. The operator (or an upstream service) uploads a file directly to the sandbox's filesystem via a POST to /api/sandboxes/{id}/files. The file lands in the sandbox's /workspace directory. The agent reads it from there.

# Upload a CSV to the agent's workspace
curl -X POST "$SANDBOX_URL/api/sandboxes/sandbox_abc/files" \
  -H "Authorization: Bearer $SANDBOX_API_KEY" \
  -F "path=/workspace/sales-2026-q1.csv" \
  -F "file=@./local-sales.csv"

# Response:
# { "path": "/workspace/sales-2026-q1.csv", "bytes": 4823914, "sha256": "..." }

When to use: dev workflows, operator-initiated one-shots, files small enough to upload in under a minute. Best for files up to 100-200 MB; works up to 1 GB but starts to feel slow.

When NOT to use: bulk pipelines (10+ files), large datasets, anything operator-facing where the operator shouldn't have to wait. The synchronous nature blocks both operator and sandbox.

Pattern 2: Multipart upload (1 GB - 5 TB, resumable)

For larger files, the multipart upload API splits the file into 5 MB - 5 GB chunks, uploads them in parallel, and assembles them on the receiver. Each chunk gets its own request; failed chunks retry without restarting the whole upload.

import requests, hashlib, os

API = os.environ["SANDBOX_URL"]
KEY = os.environ["SANDBOX_API_KEY"]
HDRS = {"Authorization": f"Bearer {KEY}"}
CHUNK_SIZE = 8 * 1024 * 1024   # 8 MB chunks

def upload_multipart(sandbox_id, local_path, remote_path):
    file_size = os.path.getsize(local_path)
    # 1. Initiate multipart upload
    init = requests.post(
        f"{API}/api/sandboxes/{sandbox_id}/files/multipart/init",
        headers=HDRS,
        json={"path": remote_path, "size": file_size},
    ).json()
    upload_id = init["upload_id"]

    # 2. Upload chunks (parallelize for speed)
    parts = []
    with open(local_path, "rb") as f:
        part_num = 0
        while chunk := f.read(CHUNK_SIZE):
            part_num += 1
            etag = requests.put(
                f"{API}/api/sandboxes/{sandbox_id}/files/multipart/{upload_id}/{part_num}",
                headers=HDRS,
                data=chunk,
            ).json()["etag"]
            parts.append({"part_num": part_num, "etag": etag})

    # 3. Complete (server assembles)
    requests.post(
        f"{API}/api/sandboxes/{sandbox_id}/files/multipart/{upload_id}/complete",
        headers=HDRS,
        json={"parts": parts},
    )

upload_multipart("sandbox_abc", "large-dataset.parquet", "/workspace/data.parquet")

When to use: files over 1 GB, slow or unreliable networks, anything where you can benefit from parallel chunk upload (5-10× speedup over single-stream).

Gotcha: if the upload fails partway through and you don't call abort, the chunks accumulate in storage and incur cost. The platform auto-cleans incomplete multipart uploads after 24 hours; configure shorter if you want.

Pattern 3: scp / rsync over SSH (developer workflow)

Once you've SSH'd into the sandbox (per the SSH guide), scp and rsync work as you'd expect. Best for developer workflows, debugging, and many-small-files transfers.

# Copy a local file to the sandbox
scp -i ~/Downloads/my.pem ./report.csv ec2-user@$SANDBOX_IP:/workspace/

# Copy a directory tree, preserving structure
scp -r -i ~/Downloads/my.pem ./datasets/ ec2-user@$SANDBOX_IP:/workspace/

# rsync — resumable, only transfers diffs, ideal for many small files
rsync -avz --progress \
  -e "ssh -i ~/Downloads/my.pem" \
  ./datasets/ ec2-user@$SANDBOX_IP:/workspace/datasets/

# Pull a result file back
scp -i ~/Downloads/my.pem ec2-user@$SANDBOX_IP:/workspace/output.pdf ./

# Or rsync the whole results directory back
rsync -avz \
  -e "ssh -i ~/Downloads/my.pem" \
  ec2-user@$SANDBOX_IP:/workspace/results/ ./local-results/

When to use: debugging, dev workflows, ad-hoc transfers, many small files (rsync's diff-only behavior shines here), interactive sessions where you're already SSH'd in.

When NOT to use: production pipelines (the human-in-the-loop is wrong), event-driven flows, anything you want to automate without SSH key management on the operator's machine.

Pattern 4: S3 pre-signed URLs (production default)

The pattern that scales. Operator uploads to a pre-signed S3 URL (no AWS credentials on the operator's side); S3 ObjectCreated event triggers the agent; agent reads from S3; agent writes results back to S3; agent posts a pre-signed download URL to Slack. End to end, no inline file transfer through the sandbox API.

import boto3, uuid

s3 = boto3.client("s3")
BUCKET = "agent-uploads"

def issue_upload_url(filename, content_type="application/octet-stream"):
    """Operator's app calls this; gets back a URL the operator can PUT to."""
    key = f"incoming/{uuid.uuid4()}/{filename}"
    url = s3.generate_presigned_url(
        ClientMethod="put_object",
        Params={"Bucket": BUCKET, "Key": key, "ContentType": content_type},
        ExpiresIn=3600,   # 1 hour to upload
    )
    return {"upload_url": url, "key": key}

# Operator (or their app) does a simple PUT — no AWS creds needed
curl -X PUT "$UPLOAD_URL" \
  -H "Content-Type: text/csv" \
  --upload-file ./sales-2026-q1.csv

# S3 receives the file. ObjectCreated event fires automatically.

import boto3, os
s3 = boto3.client("s3")
BUCKET = os.environ["AGENT_BUCKET"]

def handle_new_file(event):
    """Triggered by S3 ObjectCreated event via SQS."""
    for record in event["Records"]:
        key = record["s3"]["object"]["key"]
        local_path = f"/workspace/{key.split('/')[-1]}"

        # Stream the file from S3 to local disk (no full memory load)
        s3.download_file(BUCKET, key, local_path)

        # Agent does its work
        result_path = process_file(local_path)

        # Write result back to S3
        result_key = f"results/{key.split('/')[-2]}/output.pdf"
        s3.upload_file(result_path, BUCKET, result_key)

        # Generate a 24-hour download URL and post to Slack
        download_url = s3.generate_presigned_url(
            ClientMethod="get_object",
            Params={"Bucket": BUCKET, "Key": result_key},
            ExpiresIn=86400,
        )
        post_to_slack(f"Result ready: {download_url}")

Why this is the production default:

• Decoupled. Upload finishes; agent processes when ready; result lands; download URL goes out. No blocking.
• Scales to any volume. S3 absorbs unlimited parallel uploads; agent fleet scales independently.
• Data residency. Bucket region = data region. Hard guarantee.
• Encryption. SSE-KMS or SSE-S3 on the bucket; everything inherits.
• Audit trail. S3 access logs + CloudTrail give you "who uploaded, when, from where" without instrumentation.
• Time-bounded URLs. Leaked URL only stays valid for the expiry window.

This is what every mature AI agent fleet eventually converges on. Adopt it from the start if you can.

The five output patterns

Once the agent has produced a result, where does it go? Five common destinations:

For most workloads, "write to S3 and post a pre-signed URL to Slack" handles every case. The other patterns are special cases for specific destinations.

Slack attachment vs Slack link

Two ways to get a result file into Slack:

Most production fleets use the link pattern even for small files — keeps Slack storage clean, handles the residency story, gives a uniform API.

Size limits across the stack

Plan around the smallest constraint in your pipeline. Common ones:

The pattern: when in doubt, use S3 + a URL. Almost every layer of the stack passes URLs cheaply; few pass large bytes cheaply.

Worked example: CSV in, PDF out

The full pipeline. An operator uploads a sales CSV; the agent processes it, builds a quarterly variance PDF, posts to Slack with a download link. Walk through the whole thing.

The setup (one-time, by your IT person)

1. S3 bucket agent-uploads in your region with SSE-KMS enabled and 30-day lifecycle on incoming/, 90-day on results/.
2. S3 ObjectCreated event configured to push to SQS queue agent-events.
3. Sandbox Platform receiver pulls from agent-events and routes events to the agent's workspace.
4. Slack app installed; agent has chat:write scope.

The flow

Step 1: Operator uploads (UI)

// 1. Operator clicks Upload in the operator app
async function uploadCsv(file) {
  // 2. App requests a pre-signed URL from your backend
  const { upload_url, key } = await fetch("/api/issue-upload-url", {
    method: "POST",
    body: JSON.stringify({ filename: file.name, content_type: file.type }),
  }).then(r => r.json());

  // 3. App uploads directly to S3 — no AWS creds in the browser
  await fetch(upload_url, {
    method: "PUT",
    headers: { "Content-Type": file.type },
    body: file,
  });

  // 4. Show progress: "Upload complete; agent will start in <30 sec"
  showStatus("Upload complete. Result will be posted to #ap-summary.");
}

Step 2: S3 event triggers the agent

S3 ObjectCreated event fires automatically. Routed via SQS to the agent's event receiver. Agent's sandbox auto-resumes if it was idle (8-15 sec). Agent reads the event from its inbox.

Step 3: Agent processes

import boto3, pandas as pd, os
from reportlab.lib.pagesizes import letter
from reportlab.platypus import SimpleDocTemplate, Table, Paragraph
from slack_sdk import WebClient

s3 = boto3.client("s3")
slack = WebClient(token=os.environ["SLACK_TOKEN"])
BUCKET = os.environ["AGENT_BUCKET"]

def handle_event(event):
    key = event["s3"]["object"]["key"]
    run_id = key.split("/")[1]

    # Download the CSV
    local = f"/workspace/{run_id}/input.csv"
    os.makedirs(os.path.dirname(local), exist_ok=True)
    s3.download_file(BUCKET, key, local)

    # Process — pandas + your domain logic
    df = pd.read_csv(local)
    variance = df.groupby("region").agg({"actual": "sum", "budget": "sum"})
    variance["variance"] = variance["actual"] - variance["budget"]

    # Build PDF
    pdf_path = f"/workspace/{run_id}/q1-variance.pdf"
    doc = SimpleDocTemplate(pdf_path, pagesize=letter)
    elements = [
        Paragraph("Q1 2026 Sales Variance Report", styles["Title"]),
        Table([variance.columns.tolist()] + variance.values.tolist()),
    ]
    doc.build(elements)

    # Upload result to S3
    result_key = f"results/{run_id}/q1-variance.pdf"
    s3.upload_file(pdf_path, BUCKET, result_key,
                    ExtraArgs={"ContentType": "application/pdf",
                                "ServerSideEncryption": "aws:kms"})

    # Generate 24-hour pre-signed download URL
    download_url = s3.generate_presigned_url(
        "get_object",
        Params={"Bucket": BUCKET, "Key": result_key},
        ExpiresIn=86400,
    )

    # Post to Slack
    slack.chat_postMessage(
        channel="#ap-summary",
        text=f"Q1 variance report ready: {download_url}",
        unfurl_links=False,
    )

    # Clean up local files; sandbox returns to idle
    os.system(f"rm -rf /workspace/{run_id}")

Step 4: Operator clicks the link

Slack message posts in #ap-summary. Operator clicks the link; downloads the PDF directly from S3 (the pre-signed URL bypasses any auth). PDF lands; operator reviews; ready for the executive meeting.

Cost breakdown

Wait — the lead said $0.18. The $0.18 is the higher-end with Claude Opus for higher-stakes runs and a larger CSV (50 MB). The $0.05 is the typical case with mid-tier models. Both are dramatic compared to the human alternative (1-2 hours of analyst time at $80/hour fully loaded = $80-160).

When not to copy: mount S3 directly

For datasets larger than 5 GB or for read-mostly workloads, copying the whole file is wasteful — the agent reads 100 MB and you transferred 50 GB. Mount the S3 bucket as a filesystem instead.

# Install mountpoint-s3 (one-time, in your custom Docker image)
sudo yum install -y mount-s3

# Mount the bucket read-only at /mnt/datasets
mkdir -p /mnt/datasets
mount-s3 your-datasets-bucket /mnt/datasets --read-only --region us-west-2

# Agent reads files as if local — only the bytes accessed are pulled
ls /mnt/datasets/q1-2026/      # lists S3 objects
head /mnt/datasets/q1-2026/sales.csv   # downloads first ~64 KB

# Unmount when done
umount /mnt/datasets

mountpoint-s3 is AWS's official tool. s3fs is the older alternative (POSIX-compatible but slower). Pick mountpoint-s3 for read-mostly workloads; s3fs only when you need POSIX semantics (random write, file locks).

When to mount vs copy:

• Mount: dataset over 5 GB; agent reads 10-20% of it; sequential or pseudo-random access; read-only or append-only writes.
• Copy: dataset under 5 GB; agent reads all of it; random access with frequent rewrites; need POSIX semantics.

Encryption: three layers

For HIPAA / financial / GDPR-strict deployments, layers 1+2+3 with customer-managed KMS keys is the standard. Document in the BAA / DPA. The platform supports this configuration; ask sales for the compliance addendum.

Data residency

If your customers have data-residency requirements (EU GDPR, AU local-first, US-government-only), the rule is: data goes where the bucket is. Constrain everything to the right region:

• Sandbox region: set the workspace's allowed-region list to e.g. eu-west-1 only. Sandboxes can't be created elsewhere.
• S3 bucket region: match the sandbox region. Uploaded files never cross the boundary.
• SQS / EventBridge region: regional too. Events stay local.
• Model provider: Anthropic's EU endpoint, OpenAI's EU tier, Bedrock's EU region — pick the model endpoint that matches your residency requirement.
• Slack / email destinations: if these are within the residency boundary (most are global), use them as-is; if not, replace with regional alternatives.

Document the data-flow diagram for compliance audits. The platform's audit log captures the region of every operation — useful evidence for a regulator.

Cleanup: don't accumulate cruft

Most production AI agent fleets have one of two cleanup failures: disks fill up, or S3 buckets accumulate stale data. Both are preventable.

Sandbox disk cleanup

• Per-run cleanup. Each task uses /workspace/{run_id}/; delete the directory at end of run.
• Cron sweep. Daily cron deletes /workspace/* directories older than 7 days.
• Disk-pressure alert. Alert at 80% disk full; auto-cleanup at 90%; sandbox stop at 95%.
• Resize when persistent growth. If the agent's working set is genuinely growing, increase disk size (configurable in the dashboard).

S3 lifecycle policies

resource "aws_s3_bucket_lifecycle_configuration" "agent_uploads" {
  bucket = aws_s3_bucket.agent_uploads.id

  # Delete incoming/ files after 30 days (operator already got the result)
  rule {
    id     = "clean-incoming"
    status = "Enabled"
    filter { prefix = "incoming/" }
    expiration { days = 30 }
  }

  # Move results/ to cheaper storage after 90 days; delete at 365
  rule {
    id     = "archive-results"
    status = "Enabled"
    filter { prefix = "results/" }
    transition {
      days          = 90
      storage_class = "STANDARD_IA"
    }
    expiration { days = 365 }
  }

  # Auto-abort failed multipart uploads after 24 hours
  rule {
    id     = "abort-incomplete-multipart"
    status = "Enabled"
    abort_incomplete_multipart_upload {
      days_after_initiation = 1
    }
  }
}

Cost math at scale

The model dominates; the file plumbing is essentially free. Optimize the model first (cheaper model, smaller prompts, prompt caching) before optimizing the file plumbing.

Real-world patterns from the field

Pattern 1: Daily competitive intel snapshot

An agency. Cron fires at 6am. Agent fetches each competitor's pricing page, screenshots, runs visual diff. Each screenshot lands in S3 at s3://comp-intel/{date}/{competitor}/screenshot.png. Diffs land at s3://comp-intel/{date}/{competitor}/diff.png. Slack message at end-of-run links to a summary HTML page. Total daily storage: ~50 MB; lifecycle deletes after 90 days.

Pattern 2: Invoice-processing fleet

An accounting firm. Vendor invoices arrive via email at ap@firm.com. Email gateway extracts attachments to S3 at s3://invoices/{date}/{invoice-id}.pdf. S3 event triggers the AP-clerk agent. Agent reads the PDF, extracts line items, posts to NetSuite, writes processed metadata to S3 at s3://invoices/{date}/{invoice-id}.json. Originals retained 7 years (financial compliance); processed metadata 1 year.

Pattern 3: ML training data pipeline

An ML team. Operator drops training data (multi-GB Parquet files) to s3://training-data/raw/. ML preprocessing agent reads via mountpoint-s3, transforms in chunks, writes processed data to s3://training-data/processed/. GPU sandboxes (ab0t.gpu) read from processed/, train models, write checkpoints to s3://training-data/checkpoints/. The whole pipeline is event-driven; each stage triggers the next via S3 events.

Pattern 4: Customer-facing report generator

A SaaS startup. End user clicks "generate report" in the app. App writes the request to s3://app-requests/{user_id}/{request_id}/input.json. Agent processes, writes the PDF report to s3://app-results/{user_id}/{request_id}/report.pdf. App polls the result location; once present, generates a pre-signed download URL with the user's session as audience binding (only that user's session can access). User clicks; gets the report. Mean time end-to-end: 45 seconds.

Pattern 5: Audit log export

A compliance team. Quarterly cron fires. Agent queries audit log API, exports to a date-range tar.gz, signs it with the firm's GPG key, uploads to s3://audit-exports/{quarter}/audit-log.tar.gz.gpg. Posts download link + GPG signature to #compliance Slack. Compliance officer downloads, verifies signature, archives in long-term storage. The whole pipeline is one agent, one cron, one S3 bucket.

Anti-patterns: what not to do

• Putting datasets in EC2 user-data. 16 KB hard cap; user-data isn't for data. Use S3.
• scp from your laptop in production pipelines. Manual step; doesn't scale; key-management is painful. Use S3 pre-signed URLs.
• Loading a 5 GB file into memory to pass it to a model. The model's API handles 25-32 MB max. Stream chunks; summarize iteratively; or use the provider's Files API.
• No lifecycle policies on S3. Costs creep; old files accumulate forever. Lifecycle is essentially free; configure it from day one.
• Pre-signed URLs with multi-day expiry. 24 hours is the standard; longer increases the leaked-URL blast radius. If users need it longer, they can request fresh URLs.
• Reading from S3 thousands of times instead of mounting. Mount once; read at file-level granularity. The performance and cost difference is dramatic for large datasets.
• Storing result files in /tmp. /tmp is RAM-backed in some configurations; survives restart in others; never trust it for anything important. Use /data or /workspace.
• Cross-region S3 reads in hot paths. Cross-region transfer is slow ($0.02/GB) and adds latency. Keep buckets in the agent's region.
• Forgetting to abort failed multipart uploads. Incomplete multipart chunks accrue storage cost forever without the lifecycle abort rule. Always configure it.
• Returning result files from the agent's HTTP API. 6 MB sync limit; doesn't scale. Return a pre-signed URL; client downloads from S3 directly.

How this compares to other AI sandbox platforms

For AI agent workloads specifically, the patterns in this guide work across most of these platforms — the S3-pre-signed-URL pattern is platform-agnostic. Sandbox Platform's differentiation is the breadth of primitives (you can use any of the four upload patterns) and the auto-stop / persistence story (files survive stop without paying for compute).

Frequently asked questions

What's the largest file I can upload to a sandbox?

Via the multipart upload API: 5 TB per file in 5 GB chunks. Via the simple files API: 1 GB per request. Via scp/rsync over SSH: bounded only by sandbox disk size (default 30 GB; configurable). Via S3 pre-signed URL: 5 TB per object. For files over 1 GB, prefer S3 pre-signed URLs or multipart — they don't block sandbox memory and resume gracefully on network blips.

How do I avoid copying a 50 GB dataset into the sandbox just to read 100 MB?

Don't copy. Mount it. The sandbox supports S3 mount via mountpoint-s3 or s3fs — the agent reads from S3 paths as if they were local files; only the bytes the agent actually reads are pulled. For datasets over 5 GB or for read-mostly workloads, mounting beats copying every time. Reserve full copies for write-heavy or random-access workloads.

What's the right way to send results from the agent back to the operator?

Five patterns by destination. (1) Slack: agent posts a message with the file attached (under 25 MB) or a link to S3 (over). (2) Email: SES or Postmark with the file attached or a download link. (3) Linear / Jira / Notion: API write with file attachment. (4) Webhook back to your application: agent POSTs to your callback URL with the result payload. (5) Just write to S3 and notify: results land in s3://your-bucket/results/{date}/{run_id}/, agent posts the URL to Slack. Pattern 5 scales the best and is what most production fleets use.

Do files persist across sandbox stop/start?

Yes — anything on the sandbox's disk survives. Stopped sandboxes don't accrue compute charges; storage continues at $0.10/GB-month. Files in /workspace and /data and /home/ec2-user all survive. Files in /tmp do not — that's RAM-backed in some configurations. Use /data for state that should survive; /tmp for scratch. See Persistent Workspaces for the full survives-stop inventory.

How do I encrypt files at rest?

Three layers. (1) Sandbox disk: EBS volumes use AWS-managed KMS encryption by default; configurable to customer-managed KMS for stricter compliance. (2) S3 buckets the agent writes to: enable SSE-KMS or SSE-S3 on the bucket; the agent's writes inherit the encryption. (3) Application-level: the agent can encrypt with a per-file key before writing (rare; usually overkill if (1) and (2) are in place). HIPAA / financial deployments require all three layers documented in the BAA.

How does data residency work for cross-region requirements?

Sandboxes deploy to specific AWS regions (us-east-1, us-west-2, eu-west-1, ap-southeast-2). Data written to a sandbox's local disk stays in that region. Data written to S3 stays in the bucket's region. For EU-residency requirements, deploy sandboxes to eu-* regions and use eu-* S3 buckets — no data crosses an Atlantic boundary. The platform's workspace settings let you constrain new sandboxes to a region whitelist.

Can I stream large file uploads instead of buffering them?

Yes — the multipart upload API supports streaming. Each 5 MB chunk is uploaded as it's read; the receiver assembles the file. This is the right pattern for files over a few hundred MB or for slow-network uploads. The Python SDK wraps it; raw HTTP works too. For S3 destinations, boto3's TransferManager handles multipart automatically for files over 8 MB.

What if the agent's task takes longer than the upload?

Decouple. Don't upload-then-process inline. Pattern: (1) operator uploads file to S3 with a known prefix; (2) S3 ObjectCreated event fires; (3) event triggers the agent (via the platform's event integration); (4) agent processes the file from S3; (5) agent writes result. The agent never blocks on upload. For very large files (50+ GB), the agent might process while the upload continues — file is partially available; agent reads what's there.

How do I let an end user download a file the agent generated?

S3 pre-signed URL is the standard pattern. The agent writes the result to a private S3 bucket, then generates a pre-signed URL with a 24-hour expiry, then posts the URL to wherever the user can see it (Slack, email, your app's UI). The user clicks; downloads directly from S3 without authenticating. Pre-signed URLs are time-bounded so a leaked link only stays valid for the expiry window.

What size limits should I plan for?

Sandbox disk default: 30 GB (configurable up to 16 TB). EC2 user-data: 16 KB hard cap (this is why you don't put datasets in user-data). Multipart upload: 5 TB max file, 5 GB max chunk. Slack file attachment: 1 GB max via API, 25 MB practical for most channels. Email attachment: 25 MB at most providers; 10 MB safer. SQS message body: 256 KB (use a pointer to S3 for larger payloads). Lambda response: 6 MB sync, 256 KB async. Plan around the smallest constraint in your pipeline.

How do I handle compressed datasets?

Three options. (1) Decompress before upload — operator does it; the sandbox sees raw files. Best when the operator already has decompressed data on hand. (2) Upload compressed; decompress in the sandbox — faster upload, more sandbox CPU. Best for big text data (logs, CSVs) where compression ratios are 5-10×. (3) Stream-decompress while reading — the agent reads the compressed file directly using gzip or zstd modules. Best for read-once large datasets where you don't need the decompressed form on disk.

What about binary files like images, video, PDFs?

Treat them like any other file — upload via the same patterns. The agent's harness usually handles binary natively (Claude Code's tools include image and PDF reading). For very large media, use S3 mount to avoid copying full files; for batch processing, tar.gz the directory and upload once. Image inputs to vision models are typically encoded inline (base64) but the original file lives on disk for reuse.

Can multiple agents share a workspace's files?

Yes — within a workspace. Mount a shared EBS volume read-write across agents (with file locking discipline) or use a shared NFS mount. For coordination, agents can use sqlite with WAL mode or post lock notes to a shared file. For larger fleets, S3 + event-driven coordination is more reliable than filesystem-shared.

How long do pre-signed URLs stay valid?

Configurable up to 7 days (S3's hard limit). 24 hours is the typical default — long enough for human review, short enough to bound leaked-link exposure. For programmatic access (your app downloading on behalf of a user), shorter (15-60 minutes) is better. For "send a result to a customer they'll review later," 24-72 hours.

What happens to files when I delete a sandbox?

Files on the sandbox's disk are deleted with the sandbox. Files in S3 (and any other external stores) persist independently. To delete the S3 files too, configure lifecycle rules or use the platform's "purge" API which sweeps both sandbox-local and configured-S3-prefix files. For compliance regimes that require certified deletion, the platform supports cryptographic erasure (delete the KMS key the data was encrypted with).

What's next