Task Testing
Part of AI writing tools compared
Checking whether a writing tool preserves the source meaning
Compare source and rewrite for changed actors, certainty, conditions, attribution and missing propositions before approving AI-written copy.
A rewrite preserves meaning when a careful reader takes the same material claims, conditions and actions from it as from the source. Check those elements directly.
Similar wording and a smoother tone cannot establish that the source has been represented faithfully.
Record what must survive
Before asking a tool to summarise or rewrite, mark the source’s essential propositions. For each, record who does what, to whom, when, under which conditions and with what degree of certainty. Note exact figures, units, dates, exceptions, attributions and quoted terms where they matter.
Consider a hypothetical service notice: “Customers may request a change before dispatch, subject to availability.” It contains a permission, a deadline and a condition. “Customers can change orders” loses two limits. “We will change orders before dispatch” also turns a possibility into a commitment.
Check the high-risk shifts
Read the source and rewrite side by side. Check for:
- Changed actor:A supplier’s obligation becomes the customer’s, or a named team becomes “we”.
- Changed strength:“May”, “usually” or “aims to” becomes “will” or “guarantees”.
- Lost boundary:A condition, exception, timeframe, population or geographic limit disappears.
- Changed relationship:A correlation becomes a cause, or a proposal becomes an approved decision.
- Misplaced attribution:A stakeholder’s view becomes the organisation’s established position.
- False precision:An estimate becomes an exact number, or a range becomes a single figure.
Mark each source proposition as preserved, altered, omitted or unclear. Then inspect claims added by the rewrite.
A statement can be true in isolation yet misrepresent the source if it changes emphasis or loses a condition.
Checklist: Ensure AI rewrite preserves source meaning
- Changed actor?Check if responsibility shifted (e.g., supplier → customer, named team → 'we')
- Changed strength?Ensure modal verbs like 'may', 'usually' or 'aims to' aren't replaced with 'will' or 'guarantees'
- Lost boundary?Verify conditions, exceptions, timeframes, populations or geographic limits are not omitted
- Changed relationship?Confirm correlation isn’t misrepresented as causation, or proposal as decision
- Misplaced attribution?Ensure stakeholder views aren’t presented as organisational positions
- False precision?Check estimates aren’t turned into exact figures or ranges into single values
Source vs. Rewrite: Key Meaning Shifts to Watch For
- Original Statement
- Customers may request a change before dispatch, subject to availability.
- Rewritten Version
- We will change orders before dispatch.
- Issue Detected
- Loss of condition (availability), change from permission ('may') to commitment ('will')
Decide what to do with a mismatch
For a minor wording issue, edit the rewrite and check the sentence again. Restore a missing qualification next to the claim it limits.
If the source is ambiguous, ask its owner what it means. If several material propositions need reconstruction, reject the rewrite and start again.
Repeat the check on the final version, after human edits and formatting. A late headline change or shortened excerpt can alter meaning even if the body was checked earlier.
Record the source version and the final wording approved by the responsible person.
This check asks whether the output faithfully carries the supplied source’s message. A separate factual check establishes whether the source and any added claims are correct against authoritative evidence.
Steps to Verify AI Rewrites Preserve Source Meaning
- Identify essential propositionsRecord who does what, when, under what conditions, and with what certainty
- Compare side by sideCheck for changed actors, strength, boundaries, relationships, attribution and precision
- Mark each propositionLabel as preserved, altered, omitted or unclear; inspect added claims
- Edit or rejectFix minor issues; reconstruct major mismatches and start again if needed
- Final reviewRecheck after human edits, formatting or headline changes
