Skip to content

processed_volume_trend_meta — trend-shape artefacts

The third family inside processed_volume_trend_meta. Holds per-keyword artefacts of the trend-shape analysis and the bot-spike repair pass. All internal — none of these keys are surfaced to customers directly.

Keys

Key Type Meaning
trend_type string One of smooth, intermittent, intermittent_dying, lumpy, erratic, seasonal, isolated_event, dead. Routes forecast model selection.
adi float Average Demand Interval — periods per non-zero observation. Computed on trailing 24 months.
cv_sq float Squared coefficient of variation of non-zero demand sizes. Trailing 24 months.
bot_spike_months object {"YYYYMM": {"raw_gsc": N, "gt_implied": M, "estimated_bot": K}} — the months where bot-spike repair fired, with the raw and corrected values preserved
gsc_position_match bool Whether the keyword's GSC position data is internally consistent (anti-corruption signal)
prior_volume_trend object Frozen shadow: months ≤ max(last_gt_yyyymo, last_gkp_yyyymo) preserved from the prior iteration. Prevents bot-corrected values from flapping between runs when GT/GKP haven't been refreshed.
last_gt_yyyymo int Most recent GT month used in this iteration's blend
last_gkp_yyyymo int Most recent GKP month used in this iteration's blend
meta_written_at ISO8601 string Timestamp of this meta write (UTC)

How they're computed

Anchors driving each key:

  • trend_type / adi / cv_sq — processing.py:KB-ANCHOR:adi-cv2-trend-type-routing (and seasonal-route-acf, intermittent-dying-gate for the seasonal / dying sub-cases).
  • bot_spike_months — accumulated during the GT-blend loop when is_uncorroborated_gsc_spike() returns true. Constants at processing.py:KB-ANCHOR:gsc-spike-factor, spike-corroboration-thresholds. The bot-likeness check is at processing.py:KB-ANCHOR:bot-like-threshold; recovery-window detection at recovery-window-gate.
  • prior_volume_trend — captured before the new trend overwrites the old one in combined_volume_data_sorted. Indexed only above the GT/GKP boundary.
  • last_gt_yyyymo / last_gkp_yyyymo — set during the GT and GKP blend stages respectively.

The bot_spike_months shape

Critical for the /spikes demo page and the record_bot_spike_counts_snapshot.py monitor:

{
  "202403": {
    "raw_gsc": 8400,
    "gt_implied": 1200,
    "estimated_bot": 7200
  },
  "202404": {
    "raw_gsc": 9100,
    "gt_implied": 1350,
    "estimated_bot": 7750
  }
}

Empty {} means "we checked and found no bot spikes". Missing key means "we didn't check this iteration" (e.g. sub-gate keyword that never entered the spike-detection branch). Distinguishing these matters for monitoring queries.

The frozen shadow (prior_volume_trend)

For months ≤ max(last_gt_yyyymo, last_gkp_yyyymo), the prior published processed_volume_trend values are copied here. The reason: until GT or GKP is refreshed past a boundary, re-running processing.py should not change history. Without this shadow, bot-spike re-detection (which can shift slightly run-to-run) would make published months oscillate. The shadow keeps history stable across iterations.

When fresh GT/GKP lands and last_gt_yyyymo advances, the shadow is rebuilt with the new boundary.

Adding new artefacts (experimentation)

This family is the loosest of the three — it's the natural home for ad-hoc per-keyword signals you want to track without altering the table schema. Examples that could land here:

  • New trend-shape diagnostics (e.g. a Hurst exponent estimate)
  • A "needs-investigation" flag for keywords whose multi-source agreement is suspiciously bad
  • Provenance — which processing.py version wrote this meta

To add: emit the new key in the inline meta-write inside processing.py's main loop, then add a row to the Keys table above.

See also