FlowDrop AI BenchA living benchmark of AI content workflows

Overview

One task, one prompt, one owned corpus, ten workflow architectures from a fixed pipeline to an autonomous agent, compared on cost, speed and failure modes. Pick a study and filters once; every visual keeps them.

New here? Architectures draws what each of the ten cells does and which pairs differ by one decision; every cell label on this site reads input the model sees → who does the work → tools.

correct: every axis at threshold. degraded: all axes ≥ 0.75. silent: completed and the document is wrong. format: HTML came back. loud: nothing usable delivered. stale: the page changed after the run; ungraded. Axes and thresholds: scoring/score.py.

Visuals

Architectures

What each of the ten benchmark cells does, drawn with one shared vocabulary, and how the cells relate: which pairs differ by exactly one decision.

no filters

Scorecard

Outcome per cell (variant × model × page) and every run with its axes, output and trace.

filters: task, benchmark, model, page, outcome, tag, corpus

Matrix

One glance at the study: rows are workflow architectures, columns are models, a cell is correct runs over graded runs.

filters: task, benchmark, model, page, tag, corpus

Cost and time

What each architecture costs on each model, in dollars and seconds per run and in dollars per correct run, as figures rather than tables.

filters: task, benchmark, model, page, tag, corpus

Model focus

Pick one model and see how each benchmark (B0-B9) performs on it: correctness, axes, cost.

filters: task, benchmark, page, tag, corpus

Benchmark focus

Pick one benchmark (a workflow architecture) and compare the models that ran it: correctness, axes, cost.

filters: task, model, page, tag, corpus

Traces

One page per run with the full FlowDrop execution: pipelines, jobs on a timeline, checkpoints, sessions, metering.

filters: task, benchmark, model, page, outcome, tag, corpus

The task

The three pages, their gold bodies, the planted targets, homonym traps and protected names, and the prompt every cell runs with.

no filters

Contribute a run

Clone the repo, set up the runner, run a cell, open a pull request; CI scores it and merge publishes it here. The steps are in the README. Data: runs.csv, scores.csv, scores.json.