dispatch for parallel agents

agent‑fleet / runs the list, not your machine into the ground

Run many Claude Code sessions in parallel against one shared task queue. How many actually start comes from the work and from what your computer can spare — a hundred tasks never means a hundred browser tabs.

Apache-2.0 python3 stdlib only no daemon · no broker 67 self-tests linux · x11 optional

The problem

Starting N agents is easy. Starting the right N is the whole job.

Hand a naive runner a list of a hundred pages to process and it will try to open a hundred sessions: a hundred browser tabs, a hundred model contexts, one host hit a hundred times at once. The machine swaps, the site rate-limits you, and half the work is duplicated because two workers grabbed the same item.

agent-fleet is the small amount of machinery that fixes exactly those failures — and nothing else. It is a queue, a planner, and a worker protocol. No daemon, no broker, no database.

Given
100
tasks queued — social posts, forms, pages to process
On this laptop
9
workers — 10.2 GB free ÷ the measured cost of a real worker
On a tired machine
1
worker — it degrades to serial instead of thrashing
Duplicated work
0
claiming is one atomic rename(2)

The mechanism

From a list to finished work, with two feedback loops

The planner reads the queue and the machine. Workers are a pool, not a partition: each one claims the next item as it finishes, so the queue length never decides how much is open at once.

fleet · dispatch flow claim → work → report → repeat
How agent-fleet turns a task list into finished work Input sources feed a queue. A planner combines the queue with a machine probe of cores, load, free memory and measured worker cost, takes the smallest of four ceilings, and dispatches that many workers across three lanes. Each worker claims, works, and reports done, blocked or failed, then claims again. Blocked tasks return to the queue after a human clears them. INPUT fleet add fleet from-note URLs · TASKS.md · plain text QUEUE · .agents/queue inbox/ active/ done/ blocked/ failed/ one JSON file per task MACHINE PROBE cores · load1 MemAvailable · cgroup measured worker RSS nothing assumed about your hardware PLANNER · fleet plan min( browser tasks, max_workers, distinct hosts, machine capacity ) the tightest ceiling wins LANES browser own tab + own session costs a terminal agent in-session subagent no terminal serial one at a time no terminal EACH WORKER, UNTIL THE LANE IS EMPTY claim do the task report once claim the next one — the pool never grows done = evidence observed · blocked = a human is needed done + confirmation seen blocked login · captcha · 2FA failed you clear the logins in batches → fleet unblock --all → back to the queue human, in batches
costs a browser + a terminal free — runs inside a session you already have needs a human, and says exactly why

Capacity

Four ceilings. The lowest one wins, every time.

Concurrency is never taken from the queue length. It is the minimum of what there is to do, what you asked for, what the hosts will tolerate, and what the machine can spare right now — re-read on every run, because the machine's answer changes.

workers = min(
  browser_tasks,  — never more workers than work
  max_workers,   — your configured preference
  distinct_hosts, — one worker per site, so nothing gets hammered
  machine_capacity — what this computer can actually take
)

machine_capacity = min(
  ⌊(cores × max_load_per_core − load1)⌋ capped at cores − reserve_cores,
  ⌊(MemAvailable − reserve_mem) ÷ mem_per_worker⌋
)

Memory is measured from /proc/meminfo, not guessed, and a container's memory.max is honoured when present — the host's RAM is not yours to spend. If nothing fits, capacity floors at one worker and says so, rather than refusing the run or thrashing the machine.

SituationQueueFree RAMWorkersWhy
Workstation, idle10010.2 GB9memory-bound: 900 MB per worker after the reserve
Same box, busy1002.1 GB1degrades to serial and prints “one at a time”
All on one host4010.2 GB1one worker per host — a rate-limit ban is not throughput
Small VM, 2 GB cap301.6 GB1cgroup limit read and respected
You insist1002.1 GB6--force — the only way past the ceiling
fleet plan · a hundred posts on a working laptop
machine: 20 cores, load 3.29, 10.1 GB free
         → room for 9 browser worker(s) and 23 subagent(s), bound by memory

pending: 100 task(s)

  browser  100 task(s)   own tab + own terminal (parallel, one per host)
                          → 4 terminal(s); hosts: site1.example, site2.example +98 more

terminals needed: 4
  limited by max_workers: max_workers is 4 (config or --max)
  the other 96 browser task(s) wait in the queue — each worker claims the next as it finishes

Lanes

Most work does not deserve a terminal

A terminal exists to hold a browser tab. Work that never opens a page runs as a subagent inside the session you are already in — no window, no tab, no memory cost worth planning around.

browser

Its own tab, its own session

Anything that acts on a live page: filling a form, posting, checking a listing. Pinned to one tab it created and never touches another worker's.

cost: 1 terminal + ~900 MB
agent

A subagent, right here

Research, drafting, summarising, code edits — everything with no page to click. Runs in parallel inside your current session.

cost: no terminal
serial

Strictly one at a time

Work that must not overlap: a shared file, an ordered migration, anything writing the same state.

cost: no terminal

Guarantees

What it will not do to you

Start

Three minutes, no dependencies

install and run
# python3 stdlib only — nothing to pip install
git clone https://github.com/MSKazemi/agent-fleet ~/agent-fleet
~/agent-fleet/bin/install
~/agent-fleet/bin/selftest        # 67 checks, no browser needed

cd ~/any/project
fleet init
fleet add "https://example.com/form/12" --title "vendor form"
fleet add "summarise the three RFCs into a table" --kind research
fleet plan                        # what will run, where, and why
fleet run                         # exactly that many, and no more

Everything else is fleet help. The full command reference, the worker protocol and the design notes live in the repository README.