Create gold_v2.json reference edits

Advanced, and the single highest-leverage file in the whole evaluation system. Every score RoughCut reports is agreement with this file. A careless gold does not produce a warning — it produces confident, wrong numbers. The safe default is not to build one at all unless you are going to build it properly. If you already have one you no longer trust, the way back is to delete or replace that clip's gold_v2.json; the clip is then skipped as missing_gold instead of poisoning your aggregates.

Prerequisites

• A clip you have already edited by hand, still open or still exportable from your editor.
• That clip's folder inside your golden set, containing its media. See Build a golden set.
• A plain-text editor. This is a small JSON file you write yourself.

Steps

1. Open your manual edit of the clip in your editor, on the timeline where you made the cuts.
2. Note the source media timebase. Every number you write is a time in the original clip, in seconds from its start — not a timeline position in a sequence that starts elsewhere, and not a timecode with a non-zero start.
3. Read off the ranges you kept: for each surviving segment, its start time and its end time in the source clip.
4. Create gold_v2.json in that clip's folder.
5. Write a keep_intervals array containing one [start, end] pair per kept range, in ascending order.
6. Check the arithmetic: every start is smaller than its end, no pair overlaps the next, and no end exceeds the clip's duration.
7. Save the file, then run Run smoke eval (5) from Evaluation and confirm the clip scores instead of being skipped.

The file

{
  "keep_intervals": [
    [0.0, 41.3],
    [55.8, 129.6],
    [140.2, 300.5]
  ]
}

That is a complete, valid reference edit. It says: keep the first 41.3 seconds, drop the next 14.5, keep from 55.8 to 129.6, drop again, keep from 140.2 to 300.5, and drop everything after that.

Everything outside the keep intervals counts as removed. You never list what you cut; you list what survived.

The rules the numbers have to follow

The scorer is defensive: it drops inverted spans, merges overlaps, and clips to the media duration rather than crashing. That is a safety net, not permission. A mistake there does not error — it changes the meaning of your reference edit without telling you.

Be consistent about silence

This is the mistake that quietly ruins a golden set.

If your manual edit trims pauses and breaths, then the pauses must be outside your keep intervals. If you leave pauses in, they must be inside them. Either policy is fine. Mixing them is not.

Where the gold keeps silence that you would in practice have cut, RoughCut is penalised for making the cut you actually wanted. Where the gold trims silence you would have kept, the app is penalised for leaving it. In both directions you end up measuring your own inconsistency and calling it accuracy.

Write the policy down alongside the set, and apply it to every clip.

Never build the gold from RoughCut's output

Exporting RoughCut's proposed cut and turning it into a gold_v2.json produces a near-perfect score that means nothing at all: it measures the app against itself.

The same applies in weaker form to "review RoughCut's cut, tweak a few decisions, save that as gold". The result is anchored to the app's behaviour, and it will flatter every future run. A reference edit has to come from a decision you made independently — ideally an edit you made before you ever saw what RoughCut proposed.

How a word becomes "removed"

The primary metric works at word level, so the interval list has to be projected onto words. The rule is simple and worth knowing:

A word counts as removed when less than 50% of its duration overlaps a keep interval. A word half-inside a boundary is kept; a word mostly outside it is removed.

The practical consequence is that boundaries do not have to be frame-perfect. What matters is that each boundary falls in roughly the right gap, not that it is accurate to the millisecond. Do not agonise over a tenth of a second; do care about putting a cut in the wrong pause.

What an inaccurate gold does to your numbers

An inaccurate reference edit does not produce an obviously broken result. It produces a plausible one.

• A gold that under-keeps — trimming more than you really would — makes RoughCut look too conservative, and makes "missed cuts" appear where nothing was missed.
• A gold that over-keeps makes RoughCut look reckless, and inflates the false-positive share of the fixup estimate.
• A gold built to a different silence policy than your real editing shifts every metric in one direction at once, which is exactly the pattern that looks like a genuine model difference.

Because the same gold is used for every run, these errors are consistent — and consistent error is invisible in a comparison. That is why the baselines exist. See Read evaluation metrics correctly.

Expected result

The clip appears in the results table with a Word-F1 value, a Δ remove-all delta, and a Fixup/30m estimate, and is not listed under skips. If it is flagged degenerate, the reference edit and the app's cut disagree so severely that the clip is excluded from the aggregates — check the gold before you blame the model.

Common mistakes

• Using sequence timecode instead of clip time. If your sequence starts at 01:00:00:00, subtract it. Every number is seconds from the start of the source clip.
• Listing what you removed. keep_intervals is what you kept.
• Writing timecode strings. "00:00:41:07" is not a number.
• Rounding to whole seconds. Decimals are supported and boundaries land inside pauses more often when you use them.
• Forgetting the last interval. If your edit keeps material to the end of the clip, the final interval must extend to it — otherwise you are claiming you cut the ending.
• One interval covering the whole clip. That is a valid file, but it claims you removed nothing, and the clip will be flagged as degenerate rather than scored into the aggregates.
• Editing the file after a run and expecting old runs to still compare. Changing gold changes the run's gold fingerprint, and comparisons across different gold are not honest comparisons.

Troubleshooting

For the meaning of each metric, continue to Read evaluation metrics correctly.