Prototypes Prototype docsP11 · Docs and assistant
Email answered by SI (P31)
DocsPage 31 of 39

Email answered by SI (P31)

Frontdesk drafts the replies to a clinic's customer email with SI, and a person approves each reply before it is sent.

The business is a physiotherapy clinic in Copenhagen, Klinik Nørrebro, made up for the prototype. Its emails ask for appointments, cancel them, ask about prices and insurance, and complain. Two of them try to give the SI instructions.

Reading an email

A model on Cloudflare Workers AI reads each email. When it gives no answer in 30 seconds, a second model from another family is asked instead. The model answers with JSON in a fixed shape. The shape holds the intent from a list of 12, the language, the urgency and the customer's name. It also holds the treatment, the physiotherapist whose work fits, and the days and hours the customer asks for. A value outside the shape is dropped: an unknown intent becomes "other", and a date outside the calendar is left out.

The clinic's own documents

The clinic has 10 short documents: prices, canceling, opening hours, insurance and referrals, treatments, booking, payment and refunds, complaints, finding the clinic, and health questions. Each is split into passages with an id such as prices-1.

The Worker finds the passages for an email with BM25 over their words. BM25 needs no model call and finds the same passages for the same email on every run. Its scores are shown in the audit log. A Danish email is matched through a list of Danish words and their English words.

Tools on the calendar

The calendar holds 14 days of hourly slots for 3 physiotherapists, in SQLite in a Durable Object. The agent has 4 tools: find free slots, find the sender's booking, hold a slot, and cancel a booking.

The tools for each intent are fixed in the code, and the model never calls a tool. A booking finds free slots and holds the first that fits. A cancellation finds the sender's own booking and cancels it. A move does both. The server refuses to cancel a booking that belongs to another customer.

A hold and a cancellation wait until a person approves the reply. They are undone when the person discards the draft or the email is read again.

The guards

The model drafts the reply from the passages and from the tool results, and lists the passages it used. The server then checks the draft:

  • Every cited passage exists.
  • Every amount in the draft is in a cited passage.
  • A price or a rule in the draft has a cited passage.
  • The draft contains no promise of a refund. Only the clinic manager decides on a refund, and the agent has no tool that pays money back.
  • The draft names the held time, and names no booking or cancellation the tools did not make.
  • The reply is in the language of the email.

A draft that fails is written once more with the reasons. If it fails again it is marked Check, and the server refuses to approve it until a person edits it.

Before the model reads an email, the server searches it for instructions to the agent, such as "ignore your instructions" or "system note to the assistant". When it finds one, the tools that change bookings are off for that email, and the audit log shows why.

The audit log

Every model step, tool call, guard, approval and discard is written to the audit log of its email, with its time and its duration. The inbox shows the log under the draft.

Evals

The Evals page runs the agent over the 20 emails of the inbox. Each email has a label: its intent, the exact tools that should run, the documents the reply must cite, and what must not happen.

Each case runs in a calendar of its own, seeded the same way, and the server scores it check by check. The page shows each email's checks, the audit log of a case, and the runs before.

Without a model

Without an AI binding the Worker answers 503 with a message. The inbox, the calendar and the replies a person writes still work.

Previous: Zibaldone workspace (P27). Next: Bookkeeping (P32).

How Prototype docs is built

The pages are written in Markdown and built into these pages and an index of 221 sections. The assistant answers from the index, and every answer names the sections it was taken from.