Guglielmo Paesano AI engineer and founder of AIGROW

I build AI systems that do people's work, and I measure where they still need people.

contact@guglielmo.ioDiscord, 4K members

Instrument before you evaluate: write-path invariants for deployed human-agent systems

By Guglielmo Paesano and Romanus Berg. Accepted at the Human-AI Coevolution (HAIC) workshop, NeurIPS 2026.

Read it on OpenReview(opens in a new tab)The system it comes from

Authors
Guglielmo Paesano, Romanus Berg
Venue
Human-AI Coevolution (HAIC) workshop, NeurIPS 2026
Accepted
30 Sep 2026
Presentation
Atlanta, 13 December 2026
Full text
OpenReview

What the paper argues

Whether a deployed human-agent system can be evaluated is decided by its write path: what it records, with what attribution, and what it may overwrite. The title is the recommendation. Decide what the system writes down before deciding how to grade it.

It is a field report from a production system in which people and AI agents both act. The paper names four write-path invariants and closes with a checklist a team can answer before it designs an evaluation.

What a write path is

The write path is everything a system does when it records something: which events it stores, who or what each record is attributed to, and which records it is allowed to change later.

Those three choices are usually made early, for engineering reasons, long before anyone asks how well the system works. By the time the question is asked, the answer can only come from what was kept.

Where it comes from

The system is SocrateOS, the production system of 1o1.org, a nonprofit building a dialectic AI platform for reflective decision making. I designed, built and run it. Romanus Berg, the founder of 1o1.org, is the co-author.

The SocrateOS case study describes the system itself: the three depths of dialogue, the memory graph, retrieval over a person's own documents and the checks that run before any model call.

How it reaches client work

Every agent I deliver keeps a record of each run: what came in, what the model was asked, what it answered and what the agent did, attributed and append-only. That record is what lets a client ask whether the system works and get an answer from data.

LLM evaluation for a system that is already running starts from the same place: what the system records today, and what it would need to record for the question to be answerable.

Cite as

Paesano, G., and Berg, R. (2026). Instrument before you evaluate: write-path invariants for deployed human-agent systems. Human-AI Coevolution (HAIC) workshop, NeurIPS 2026. openreview.net/forum?id=pExskhNGlq

Running an AI system you cannot yet measure?

Tell me what it does and what it records today. I read every brief myself and answer in writing with a scope.

Or write to contact@guglielmo.io

The human check appears here.

No calls and no sales sequence. One written answer from me.