The quality test
Plain and true: every message should be true and worth reading even if the reader never buys. The quality test is how a change is checked against that as a number.
The quality test drafts fixed briefs for this account and scores them, so you can see whether a change to a prompt pack, a model or the Account Brief made writing better or worse.
- Articles: five briefs (an announcement, an explainer, an opinion piece, a how-to, a case story). Each is drafted, reviewed, and both versions are scored from 1 to 5 by one fixed rater.
- Email sequences: when the account has a template ready, three sequences (welcome, re-engagement, an offer) are drafted through the Composer and scored on the message rubric.
Nothing is saved as an article or a message. A run is about 32 AI calls and takes about fifteen minutes; it shows in the AI Queue. Run it before and after a change, and compare.
Testing draft writing rules
The global writing rules are the one pack every article, FAQ, SEO line and social post reads. When a draft of them exists, the test offers Writing rules for this run: Live, or the draft. Run once on Live and once on the draft, then compare the two runs. The rater reads the live rules both times, so the two runs face the same judge. Publish the draft only if its run scores at least as well.
What is measured, not scored
Under the scores, each run shows numbers counted in code rather than judged: tells per 1,000 words (phrases people almost never write, overused words, summing-up closes), concrete facts per 100 words (numbers, names, links), letters per word, commas per sentence and length. Fewer tells, more concrete facts and shorter words read more like a person.
Critic calibration
The Composer's three critics score every email on nine lines. Until people agree with them, those numbers are critic scores, not quality. Every email the critics scored in a quality test run joins a golden set. Under Critic calibration, Score rows shows one email at a time without the critics' findings; untick every line it fails and save. Each email needs two people.
The panel shows two agreement rates, as kappa (0 is chance, 1 is perfect): people with each other, and the critics with the lines both people agreed on. When 50 emails have two scores and the critics' agreement is above 0.7, the critics are calibrated. Until then every screen calls their numbers critic scores.
Without exemplars drafts the email sequences without the account's exemplars (see Baseline: Exemplars). Run once with and once without; earlier runs say "(no exemplars)" where they were off.