An editor receives a precise finding: “the paints wash off easily after work” does not convey the brush-care instruction. The editor restores the brushes and “before the paint dries”, while also shortening the whole paragraph. The care instruction improves. The neighbouring sentences lose the requirement to prime the canvas and the drying temperature.

This result cannot be approved merely because the original finding was addressed. The new text must be checked. Let us return to the fictional Alfa set (Russian name: «Альфа») from the first article, now focusing on the history of the edit and the preservation of information that was already correct.

One finding, three versions of a paragraph

Alfa's fictional dossier specifies paper, primed canvas and wood. A thin layer dries in 30 minutes at +20 °C. Brushes must be rinsed with water immediately after work, before the paint dries. The set contains six tubes of 20 ml each, but the passage under review does not discuss the contents. These conditions do not describe real paint properties.

Original passage · incorrect care claim

For paper, primed canvas and wood. A thin layer dries in 30 minutes at +20 °C. The paints wash off easily after work.

The first two sentences agree with the dossier. The third loses the object being cleaned, omits the state of the paint and adds an assessment of ease. The requested correction concerns that claim.

After shortening · care corrected, conditions lost

For paper, canvas and wood. Dries in 30 minutes. Rinse brushes with water immediately after work, before the paint dries.

The care task has been completed. Yet the new version removes the priming condition stated in the dossier and leaves the drying time without its layer and temperature conditions. These are regressions: the edit has lost requirements that were previously preserved. A successful local correction does not compensate for them.

Revision with conditions preserved

For paper, primed canvas and wood. A thin layer dries in 30 minutes at +20 °C. Rinse brushes with water immediately after work, before the paint has dried.

The disputed claim is corrected here, and the conditions in both neighbouring sentences remain. This is a solution for this passage. Product-page completeness and consistency with other fields need their own checks.

Compare meaning, not the number of changed words

An ordinary version comparison shows deleted words. For acceptance, also record the meaning they preserved. “Primed” limits the surface; “thin” and “at +20 °C” qualify the time. “Easily”, by contrast, adds an assessment absent from the dossier.

For each edit, ask which claim appeared, disappeared, narrowed or broadened. Replacing the Russian «пока краска не высохла» (“while the paint has not yet dried”) with «после высыхания» (“after drying”) retains much of the vocabulary but changes the instruction. Replacing «6 туб по 20 мл» (“6 tubes of 20 ml each”) with «шесть туб по двадцать миллилитров» (“six tubes of twenty millilitres each”) can preserve the quantity even though many characters change.

Next, check the scope. If the temperature moves into a section heading, it may remain clear for all the lines below. If it moves to a footnote for another model, the mere presence of “+20 °C” on the page no longer helps. Check the link between the condition and the claim in the finished representation.

Semantic comparisons also need checking against the original passages: if the comparison misses the loss of “at +20 °C”, its conclusion that conditions were preserved cannot be accepted.

After care is corrected, check the primed-canvas condition and drying conditions separately: a thin layer, +20 degrees and 30 minutes.
Figure 1. A method for accepting the fictional edit. Resolving the finding and preserving other conditions are checked separately; this is not a result from testing an automated editor.

Different changes need different expectations

A repeatable process can be tested with short pairs. One pair should preserve the answer, another should require a changed decision. Otherwise, a system that always accepts everything or rejects everything will also appear stable.

CheckList distinguishes minimum functionality tests, invariance tests and directional expectation tests. For product-content checks, we can define a simple case with a known decision, then reword it without changing the meaning or alter a material condition with an expectation set in advance. This is our application of the research idea.

Fictional pairs for testing the editor and evaluator
ChangeExpected resultWhat is being checked
«6 туб по 20 мл» → «шесть туб по двадцать миллилитров» (“6 tubes of 20 ml each” → “six tubes of twenty millilitres each”)The quantity information remains supported.Semantic stability when numbers and units are written differently.
«До высыхания» → «пока краска не высохла» (“before drying” → “while the paint has not yet dried”) in an instruction to rinse brushes immediately after workThe condition is preserved.Rewording without losing the temporal boundary.
“For primed canvas” → “for canvas” in a description checked on its ownThe new phrase needs a finding about its broader scope.Sensitivity to the removal of a material condition.
T500 nominal capacity: “500 ml” → “450 ml”A supported parameter becomes incorrect.Distinguishing nominal capacity from the test fill.

A small change is not necessarily neutral. Gardner and colleagues' work on contrast sets uses small, meaningful changes to examine decision boundaries. In our example, deleting “primed” should change the assessment, while spelling out the numbers should preserve it.

Set the expectation before the run and check it against the task context. If a channel requires the literal Russian string «20 мл» (“20 ml”), spelling it out may preserve meaning but breach the format. The semantic check should then accept the quantity, while the literal check should report a separate violation. A single verdict for two requirements would hide the cause.

Such pairs expose selected weaknesses. A few successful examples do not establish performance across the catalogue. Expand the set with real error cases and independent examples so that testing does not become memorisation of familiar phrases.

When a local replacement is not enough

A targeted edit works well when the error is local and the surrounding text remains coherent. In our passage, replacing the care sentence is enough. If restrictions are scattered across several paragraphs, the same strategy may create a long chain of qualifications and contradictory references.

Sometimes the section needs restructuring: separate use from care, name the parameters and move comparable values into specifications. List the content that must survive and check it after restructuring. A small number of changed words is not in itself evidence of quality.

An edit request should state the objective and the requirements to preserve. “Shorten by a third” without priorities does not say what must remain. If one third cannot be removed without loss, the content or placement needs reconsideration. “Remove the repeated volume information; keep the model, set contents, drying conditions and care” gives a testable task. If it will not all fit, revisit the placement or length limit instead of quietly deleting a condition.

A finding about missing data should also change the workflow. When the dilution ratio is unknown, a new revision will not create a number. Asking the data owner and recording the unknown address another part of the task. Repeated instructions to “be more precise” would keep reprocessing the same gap.

Rechecking has its own scope

After editing, first check whether the finding was resolved, then whether preserved requirements were breached. For a short product page, reread the whole text. In a large library or catalogue, dependencies determine the scope of rechecking: which claims use the changed fact, where the condition is repeated and which channels receive the passage.

This selectivity depends on complete links. If nobody knows where a value is used, it is not safe to restrict rechecking to one paragraph. The history of data provenance and transformations helps locate affected places; the links themselves also need to remain correct.

Preserve both versions and the reason for the change. This distinguishes correcting an error from updating a fact. If a new document permits a different use, the dossier and related texts change. If the document is unchanged and the editor removes a restriction for rhythm, only the claim has changed.

The delivered version receives its final check after all shortening and formatting. A checked intermediate state cannot replace that point: export may truncate a condition, mix fields or insert another variant's description.

When the iteration ends

The absence of new findings from one evaluator does not establish complete review. It may have missed the same error again. Define acceptance conditions in advance: blocking violations resolved, required information preserved, disputed questions assigned their prescribed status and the final version inspected where it will appear. “Needs clarification” does not authorise release if the open question concerns a mandatory warning or another acceptance condition.

A limit on automatic attempts is useful too. If the editor and evaluator keep swapping the Russian near-synonyms «смыть» and «отмыть» (both ways of saying “wash off”) without restoring the object and time condition, another pass without new data is unlikely to resolve the dispute. Redirect the process to the evidence, rubric or task itself. Reaching the limit changes the workflow; it does not automatically approve the text.

For Alfa, the iteration ends with a revision that restores care and preserves the surface and drying conditions. Whether buyers prefer this version to another correct layout remains a separate question. It can now be formulated as a testable hypothesis, with data that would support it or require reconsideration.

Sources and supporting evidence

  1. Ribeiro et al. Beyond Accuracy: Behavioral Testing of NLP Models with CheckList, 2020Functional test types and expectations when inputs change. The product-content pairs and acceptance process are the author's adaptation.
  2. Gardner et al. Evaluating Models’ Local Decision Boundaries via Contrast Sets, 2020Small, meaningful changes for examining decision boundaries. Not evidence of the quality of our process.

Sources checked on 6 November 2026. Alfa, T500, the revisions and expected decisions are fictional teaching examples. No automated editor was tested on these pairs; no measured results are claimed.