# Microsoft Agent Framework fan-out on your InferOptimal endpoint **Status:** Recommended as the best overall harness we measured, with known failure modes where it fails completely: document retrieval, finding vulnerable files, a run that ends at its time cap with no answer, and very heavy token use. Every harness measured side by side, on the same model: https://www.inferoptimal.com/tools/results Microsoft's open-source Agent Framework runs this fan-out on your endpoint. Investigators work in parallel, 4 by default (the number we measured) or any number you choose, each following its own line of inquiry with tools over a folder (by default: search every document with a regular expression, read one in full), and an aggregator weighs their findings and writes the answer. `fanout.py` builds it with the framework's `ConcurrentBuilder`; it is the fan-out we measured. The InferOptimal subagent tool's script (`agents.mjs`) installs, configures and starts it. The agents inside are the framework's own: they call your endpoint directly, not through the subagent tool. Nothing installs, starts or calls it for you. Connecting an InferOptimal account or endpoint never installs it, and it runs nothing in the background: it works only while you run it. ## Good at, bad at, cost Each line is marked measured, with where it was measured, or designed for and not measured yet. The measured lines come from our generated legislation benchmark (hard questions whose obvious answer is overridden by something buried in about 2,830 documents, 1 to 4 runs per question) and from a first pass over public benchmarks that ended when the GLM box we measured on was torn down (1 to 4 items each). A cost per run prices GLM time at $20 an hour, that box's rate. They are not a general accuracy figure. Good at: - Hard research where the obvious answer is overridden: 9 of 14 test runs right on our legislation set, the most of any harness tried, narrowly ahead of OpenResearch's 7 of 14 and against 1 of 15 for one plain agent; with 2 or 3 tries a question, the lead rests mostly on one question. (Measured: Experiment 5, trap check and checklist control (5.10) and outside harnesses (5.6), on our generated legislation benchmark, 2026-10-18 and 19.) - Right on every short-task item it reached, in 4 to 18 minutes a run: tax law (DeonticBench) 2 of 2, NBA rule decisions (RuleArena) 1 of 1 and data analysis (KramaBench, with a command tool added) 1 of 1. It reached only 1 or 2 items of each before the GLM box was torn down. (Measured: Experiment 6, first pass on the public DeonticBench, RuleArena and KramaBench benchmarks, 2026-10-19.) Bad at: - Fails completely on retrieval and vulnerability finding: it scored 0 on its one BRIGHT run and its one VLoc run. (Measured: Experiment 6, first pass on the public BRIGHT and VLoc benchmarks, 2026-10-19.) - A run can fail completely: its first try on one hard legislation question hit the 40-minute cap with no answer, when an investigator kept generating after the 25-minute tool deadline; with a 15-minute deadline the next 3 tries on that question answered. (Measured: Experiment 5, outside harnesses (5.6), on our generated legislation benchmark, 2026-10-18 and 19.) - Very heavy on tokens: up to about 100 million input tokens a run (103 million on one VLoc run, a median 48 million on hard research), because each investigator resends its growing context with every tool call. (Measured: Experiment 5, outside harnesses (5.6), on our generated legislation benchmark, 2026-10-18 and 19, and Experiment 6, first pass on the public VLoc benchmark, 2026-10-19.) Cost: - $1.49 to $10.61 a run on average per benchmark, from tax law to hard research, with GLM time priced at the box's $20 an hour. (Measured: Experiment 5, outside harnesses (5.6), on our generated legislation benchmark, 2026-10-18 and 19, and Experiment 6, first pass on the public DeonticBench, KramaBench, RuleArena, BRIGHT and VLoc benchmarks, 2026-10-19.) ## How many agents to run Tested with 4 investigators plus a writer per task: about 2 to 3 model calls in flight on average and 4 at most. The measured strengths, failures and costs come from that shape. To use a whole endpoint, running many tasks in parallel as separate launches may work better than one very wide run: about the endpoint's provisioned agents divided by 4 launches at once, each with the default 4 investigators. --investigators takes any number, but a wider run on one task has not been measured. This is a suggestion; you decide. Every endpoint states how many concurrent agents its box is optimized for, and by default you keep all of them busy, through parallel launches or one wider run, unless the user explicitly chooses fewer; connected to an endpoint, guide and tool run say how many fan-outs fill it. What we measured: each task ran as one launch of 4 investigators and a writer. The investigators run at once, and each waits for its model reply before its next tool call, so a launch kept about 2 to 3 calls in flight on average and never more than 4. An endpoint provisioned for many agents was filled by running many such launches at the same time. For example, to answer several questions over the same folder at once, as separate launches: ```sh cd for question in "" "" ""; do node ~/.inferoptimal/agents.mjs tool run agent-framework --task "$question" --cwd . > "answer-$(date +%s%N).txt" & done wait ``` Or one wider launch on one question, which has not been measured: ```sh node ~/.inferoptimal/agents.mjs tool run agent-framework --task "" --cwd --investigators 12 ``` Nothing in the fan-out caps the number of agents: `--investigators` takes any whole number, each agent has its own connection to the endpoint, every tool call runs in its own thread, and no agent has a limit on tool round trips. Your endpoint's capacity is the only limit; requests beyond what it serves at once wait in its queue. ## License MIT, which allows any use, commercial included. agent-framework-core 1.22.0, agent-framework-openai 1.17.0 and agent-framework-orchestrations 1.3.2 (github.com/microsoft/agent-framework) are installed from PyPI by make install; we publish only the guide and the fan-out script. We checked the license in each package before listing it. ## Before you start - An InferOptimal endpoint that serves chat completions with tool calls. - The InferOptimal subagent tool set up for that endpoint (`get_codex_config` gives the one-paste install), for the one-call install below. Installing by hand instead needs the endpoint's base URL (ending in `/v1`), its model id and a file holding its key. - Python 3.11 or newer, `make` and `curl`. ## Install The subagent tool installs it and configures it for the endpoint it already uses, in one call: ```sh node ~/.inferoptimal/agents.mjs tool install agent-framework ``` It downloads this package into `~/.inferoptimal/agent-framework`, runs `make fetch install` there, then saves the endpoint's base URL and model in `endpoint.json` beside `fanout.py`, and prints what it configured. The key stays in the subagent tool's settings; the fan-out gets it only in its environment when it runs. `node ~/.inferoptimal/agents.mjs guide agent-framework` prints this guide. To install by hand instead: ```sh mkdir -p ~/.inferoptimal/agent-framework && cd ~/.inferoptimal/agent-framework && curl -fsSLO https://www.inferoptimal.com/agent-framework/Makefile && make fetch install ``` `make fetch` downloads this guide, `requirements.txt`, `constants.toml`, `lines.json` and `fanout.py` into the current folder. `make install` makes `.venv` and installs the pinned framework packages from PyPI. ## Start ```sh node ~/.inferoptimal/agents.mjs tool run agent-framework --task "" --cwd ``` It runs the fan-out on the question over that folder, with the endpoint key in its environment, and passes on every option below that you give. Add `--dry-run` to check the install, the configuration and those options without a model call: it runs `fanout.py --check` with them and prints the command it would run. | Option | Default | What it does | | --- | --- | --- | | `--investigators ` | one per line: 4 | How many investigators run at once, any whole number. | | `--lines override\|neutral\|` | `override` | The investigators' lines of inquiry (below). | | `--tools documents\|folder` | `documents` | `documents`: search every document and read one in full. `folder`: list files, search them and read numbered lines. | | `--run-command` | off | Adds a shell tool that runs commands in the folder (implies `--tools folder`). | | `--command-seconds ` | 300 | How long one command may run before it is stopped. | | `--max-tool-round-trips ` | no limit | Sets a limit on each agent's tool round trips. | | `--tool-minutes ` | 25 | After this, the tools answer "time is up" so the investigators report. | | `--run-minutes ` | none | A total run budget (below). | By hand, `fanout.py` reads the endpoint from `endpoint.json` when the one-call install saved it, and the key from `INFEROPTIMAL_API_KEY`; otherwise give them: ```sh ~/.inferoptimal/agent-framework/.venv/bin/python ~/.inferoptimal/agent-framework/fanout.py --question "" --folder --base-url --model --key-file ``` It prints the aggregator's reply; the answer is the text after its last `ANSWER:`. What it does: - Each investigator gets the question, the folder and its line, and calls the tools as often as it needs: there is no limit on tool round trips (the framework's default limit of 40 is lifted, as in our runs), and every call may write up to 32,000 tokens. - After `--tool-minutes` (25 by default) both tools answer "time is up", so the investigators report what they have. On one question we measured, 15 minutes ended runs that 25 did not. - The aggregator reads every finding, can check documents itself, and writes the answer with its quotations. - Only text files are searched and read; hidden folders, symbolic links and anything outside the folder are not. The 25-minute tool deadline is soft; optional --run-minutes sets a total run budget. It stops this runner, never the endpoint. The 40-minute benchmark cap is not a default. For example, append `--run-minutes 40` when you want to authorize at most 40 minutes for the whole run, including investigators and the aggregator. At that deadline the runner cancels its tasks and closes its HTTP clients; it allows five seconds for cleanup, then kills and reaps its own worker if a folder tool is blocked. It exits with status 124 rather than treating a timed-out workflow as a completed answer. The endpoint's own request cancellation, lease and billing rules still apply; this is not a command to stop its GPU or a guarantee that a server immediately abandons generation. Without `--run-minutes`, no total time limit is added, and the existing iteration and output settings remain unchanged. ## Lines of inquiry and other kinds of task Two sets of four lines ship, both measured: - `override` (the default, in `lines.json`): the lines behind the hard-research result above. They hunt for the rule that decides a question and what overrides it: the main rule and its facts, amendments and definitions, higher authority, and exceptions and validity. - `neutral` (`neutral-lines.json`): the general lines of our first pass on public benchmarks: requirements, evidence, alternatives, and checking the answer independently. Its writer checks the findings the answer depends on rather than which overriding documents apply. For another kind of task, write your own JSON list of `{"role", "focus"}` objects, one per investigator, and pass it with `--lines `. We have not measured other lines yet. When `--investigators` asks for more investigators than the lines given, an agent on your endpoint first writes the missing lines from the question, each different from the lines given and from the others it writes; it is asked again until there are enough, and the run stops if a reply adds no new line. Generated lines have not been measured. When it asks for fewer, the first lines are used. ## Folder and command tools `--tools folder` gives every agent the folder tools of our public-benchmark runs: `list_files` (files under a folder, with sizes), `search_files` (a regular expression over every text file) and `read_file` (numbered lines of one file). Only text files inside the folder are listed, searched and read; hidden folders and symbolic links are not. `--run-command` adds `run_command`, which runs a bash command in the folder and returns its exit code and the end of its output; our data-analysis run (KramaBench) used it. It is opt-in. Commands run with your own permissions and are not confined to the folder: an agent can change or delete files. Our measured runs confined each agent to its task folder with an operating-system sandbox; this tool does not, so give it a copy of the material or run it inside a sandbox or container.