Fábrica: from request to verified system

The problem

The hackathon kit asked for something simple to say and hard to do: take a PEDIDO.md and deliver a system that someone else can verify. An agent's typical failure is not writing bad code. It is saying it is done when it is not, or writing tests shaped around what it implemented itself.

The decisions

1. A script approves. Every stage (Conception, Specification, Build, Review) ends in a gate run by fabrica.mjs, a dependency-free Node CLI. A failed gate sends the work back to the responsible role, and on the third retry round the stage is blocked. The agent has no way to "decide" it passed.

2. Tests sealed before the code. QA writes the acceptance tests from the spec and records their hash. If Dev changes a test, the gate fails. When a test contradicts the spec, there is a formal path: Dev disputes it, and QA decides and seals again, with the reason on record.

3. One role per context. There are nine roles, each in a clean-context subagent with its own instruction "card". The security reviewer does not carry the bias of whoever wrote the code.

4. Review through three lenses. Quality, adherence to the request and security, the last one with bandit, pip-audit, detect-secrets and semgrep. An open high- or medium-severity finding fails the final gate.

5. Evidence generated, not declared. The report links criterion → test → result, and the retro lists the run's errors. There is also an offline dashboard and screenshots of the screens with data, like the ones below.

How I measured

I measured through the exact path the judges would use: claude -p with the kit's prompt, in a clean folder containing only the PEDIDO.md and the skills, with no manual adjustment. The check was done afterwards, outside the agent.

| Request | Request criteria | Total (with derived) | Gates | Duration | |---|---|---|---|---| | Training: room booking | 10/10 | 19/19 | 4, no retries | 31 min | | Unseen: front-desk visitors | 10/10 | 22/22 | 4, no retries | 27 min | | Unseen: preventive maintenance (criteria in prose only) | 7 → 7 | 17/17 | 4, no retries | 28 min | | Unseen: on-call schedule (cross-cutting rules) | 10/10 | 23/23 | 4, no retries | 31 min | | Unseen: travel expenses (larger scope) | 12/12 | 31/31 | 4, no retries | 31 min |

The derived criteria are written by the Architect from the request's rules: edge cases, validations and the screen.

Where the AI got it wrong

I logged 31 errors. Four of them changed the package:

| Error | What changed | |---|---| | A subagent hung on an API failure and the orchestrator stopped to ask the user | Autonomous mode that never asks; the subagent is reopened up to 2 times, resuming from disk | | Two reviewers sat stuck for 13 min on a file lock, because in Node on Windows, rmSync did not delete a path with an accent and raised no error | Removal with a fallback across the whole package, and lock waits with a deadline | | A taskkill /IM python.exe took down every Python process on the machine | Killing a process by name is now forbidden; shutdown uses the recorded PID | | Dev changed code during review with no finding, and the new code was not scanned | The final gate fails code changed after the security scan |

How it compares

I compared the documentation of 19 similar public projects, including Spec Kit, BMAD, Kiro, superpowers and Taskmaster. Fábrica is the only one that combines a script-decided gate, immutable tests before the code, one role per isolated context, security with real tools, and request → system with no human. In return, they support multiple languages and existing projects; Fábrica validated one profile (Python web) for new systems.

Limitations

• Judges' path measured with 5 requests, on Windows only and with a single model. A sixth run stopped midway and is recorded together with the fix.
• One validated profile (FastAPI, SQLite and an HTML/JS screen).
• The synthetic requests were written by me.