SoccerDream AI 首页  ›  GEO 内容中心  ›  近期更新(自动生成)

The 8.9% you publish: auditing a visibility gate that is statistically indistinguishable from random

Every prediction platform grows a gate. Not the model — the filter after the model, the code deciding which predictions …

本页内容由人工智能生成,仅供参考

Every prediction platform grows a gate. Not the model — the filter after the model, the code deciding which predictions the public may see. Ours is one boolean, is_visible, on a table with 47,150 rows. 8.98% are visible; the other 91% are computed, stored, graded, and hidden.

A gate is a claim: the rows I publish are better than the rows I suppress. That claim is testable. Almost nobody tests it. This is the audit, run against production data — including the part where the gate fails.

The setup: six gate-related columns, three of them dead

ai_predictions holds one row per model per match. Columns that look like they participate in the publish decision:

is_visible          boolean   47,150 populated
rec_level           varchar   46,721 populated   (A/B/C/D)
bet_worthiness      smallint  21,002 populated   (1..8)
veto_reason         text      10,223 populated
gate_reason         text           0 populated
gate_want_visible   boolean        0 populated
direction_gate      varchar        0 populated

Three of six are NULL on every row. That is the first finding — gate_want_visible at 0% occupancy says the gate was refactored and the audit trail did not survive. veto_reason survives on 10,223 rows, which is a reason code we can work with.

Baseline: 47,150 rows, 46,996 settled, 46,877 graded, 26,773 hits (57.11%), 3,796 models, 16,898 matches.

Step 1 — the pooled comparison, and why it is nearly useless

Start with the obvious query — the one that will mislead you.

SELECT is_visible,
       count(*)                                              AS n,
       count(*) FILTER (WHERE hit)                            AS hits,
       round(100.0 * count(*) FILTER (WHERE hit)
             / nullif(count(*) FILTER (WHERE hit IS NOT NULL), 0), 2) AS hit_pct
FROM ai_predictions
WHERE hit IS NOT NULL
GROUP BY 1;
is_visible | n     | hits  | hit_pct
false      | 42684 | 24360 | 57.07
true       |  4231 |  2413 | 57.53

A lift of +0.48pp. With 4,231 published rows against 42,684 suppressed, the standard error of that difference is 0.80pp, so z = 0.60. Wilson intervals: visible 56.05–59.04, hidden 56.60–57.54 — overlapping across almost their entire width.

The gate's output is not measurably different from its own reject pile. Publish a random 8.9% sample instead and you would land within noise of what the gate actually publishes.

Two caveats. The gate need not be useless to produce this number — only diffuse: 4,231 rows from 543 models, selected by a mechanism that varies per model, pool flat even if each local gate works. And the audit was possible only because 42,684 of 42,918 suppressed rows (99.45%) were graded. Stop grading what you hide and the gate becomes unfalsifiable.

Step 2 — stratify by model, and watch the sign flip

The pooled query collapsed 3,796 models into one number. Redo it per model, keeping only models that publish some and suppress some — else it degrades to "always shows" vs "never shows".

WITH per_model AS (
  SELECT model_id,
         count(*) FILTER (WHERE hit IS NOT NULL AND is_visible)    AS vis_graded,
         count(*) FILTER (WHERE hit AND is_visible)                 AS vis_hits,
         count(*) FILTER (WHERE hit IS NOT NULL AND NOT is_visible) AS hid_graded,
         count(*) FILTER (WHERE hit AND NOT is_visible)             AS hid_hits
  FROM ai_predictions
  GROUP BY 1
)
SELECT count(*)                                                   AS models_compared,
       count(*) FILTER (WHERE hid_hits * 1.0 / hid_graded
                          > vis_hits * 1.0 / vis_graded)          AS gate_is_worse,
       round(avg((vis_hits * 1.0 / vis_graded
                - hid_hits * 1.0 / hid_graded) * 100), 2)         AS mean_lift_pp
FROM per_model
WHERE vis_graded >= 5 AND hid_graded >= 5;
models_compared | gate_is_worse | mean_lift_pp
218             | 146           | -7.92

On 146 of 218 models, the published rows are worse than the hidden ones. Mean per-model lift is −7.92pp. Across the 194 models publishing at least 5 graded rows, visible rows average 56.05% and their suppressed siblings average 63.15%.

This is Simpson's paradox, and it is why a pooled gate audit is worthless. The pooled number looks fine only because visible rows are unevenly distributed: 387 of the 675 models with ≥20 graded rows (57.3%) never publish a single row, contributing zero to the "visible" bucket. The pooled average is a weighted sum whose weights have nothing to do with the gate's competence.

Step 3 — the tier system does not rank what it claims to rank

rec_level is the human-facing grade: A strongest, D weakest. It is populated on 46,721 rows.

rec_levelgraded rowshit %95% Wilsonpublished share
A1,05252.9549.93–55.9515.97%
B2,02763.1561.06–65.1914.41%
C15,85957.6656.91–58.419.46%
D27,51456.5656.00–57.127.78%

A is the worst grade in the table — 3.61pp below D, z = −2.31, p ≈ 0.021. B is the best, 6.6pp over A. The ordering is B > C > D > A: rec_level does not encode "how likely this row is to be right", and 1,052 rows carry the strongest label while being the weakest cohort.

Now the gate's behaviour inside each tier:

tierpublished hit %suppressed hit %gate lift
A58.33 (n=168)51.92 (n=884)+6.41pp
B62.33 (n=292)63.29 (n=1,735)−0.96pp
C57.50 (n=1,501)57.68 (n=14,358)−0.18pp
D56.14 (n=2,141)56.60 (n=25,373)−0.46pp

Inside tier A the gate works. Inside B, C and D it does not. And the published mix is 168 A / 292 B / 1,501 C / 2,141 D — 86.86% of published rows come from the two lowest grades. The gate operates where it has no edge and stays quiet where it has one: an A row is barely more likely to surface than a D row (15.97% vs 7.78%), and that 2× ratio cannot rescue an inverted tier.

Step 4 — a confidence inversion visible in one table

If the gate were a confidence threshold, publication would rise with confidence. That column is an integer 1–10, not a percentage.

confidencegradedpublished %hit %
38,2124.4653.64
48,9787.0360.16
511,78012.5756.04
69,53412.4855.07
74,3298.4560.04
82,0414.2665.51
95491.0973.77
10580.0056.90

Publication peaks at 5–6, where accuracy sits below average, and collapses to 1.09% at 9 — the most accurate bucket in the table at 73.77%. The 58 rows at 10 are never published.

So it is not a confidence filter. rec_reason says what it actually is:

"结构未知(样本不足或无此组合)"    12,932 rows
"结构B但非赛事分析/赛事盘面(观察)"          7,490 rows
"低质结构 ROI-8.3% (n=222)"         3,043 rows
"低质结构 ROI-4.8% (n=31)"          2,399 rows
"低质结构 ROI-0.2% (n=493)"         2,075 rows

The gate keys off a structure bucket and labels it with a bucket-level return figure computed over n=31, n=222, n=493 prior rows. That is the failure mode in one line: a decision about a single match made by a statistic computed over a different set of matches. It also explains the negative per-model lift — bucket membership is a property of market shape, not of the evidence for that particular match.

Step 5 — two integrity failures that block any future audit

Published rows carrying a veto. veto_reason is populated on 10,223 rows; 803 of them sit where is_visible = true. 19.0% of the published set has a recorded reason for being rejected. Either the flag is written after publication, or the veto is advisory and not wired into the publish path. Both are unverifiable from outside, and both mean the gate has no single source of truth.

Partial slates. 4,232 published rows span 4,212 of 16,898 matches — 24.93% carry at least one published row, and exactly 4 matches are fully published. A reader is not seeing "this match has a published view" but "this match had one model whose row survived", with the model varying per match. Any comparison built on that surface compares different things under one label.

Step 6 — what an auditable gate looks like

None of this is fixable with a threshold. The gate has to become a decision with a recorded counterfactual and a published scorecard.

import math

def wilson(k, n, z=1.96):
    p = k / n
    d = 1 + z * z / n
    c = (p + z * z / (2 * n)) / d
    h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
    return c - h, c + h

def audit_gate(vis_hits, vis_n, hid_hits, hid_n, min_lift_pp=3.0, z_crit=1.96):
    """PASS only if the gate beats its own reject pile by a margin
    that this sample can actually resolve."""
    pv, ph = vis_hits / vis_n, hid_hits / hid_n
    pooled = (vis_hits + hid_hits) / (vis_n + hid_n)
    se = math.sqrt(pooled * (1 - pooled) * (1 / vis_n + 1 / hid_n))
    lift_pp = (pv - ph) * 100
    lo, hi = wilson(vis_hits, vis_n)
    need_n = 2 * pooled * (1 - pooled) / (min_lift_pp / 100) ** 2
    return {
        "lift_pp": round(lift_pp, 2),
        "z": round((pv - ph) / se, 3),
        "vis_ci": (round(lo * 100, 2), round(hi * 100, 2)),
        "n_needed_per_arm": round(need_n),
        "verdict": "PASS" if (lift_pp >= min_lift_pp
                              and abs((pv - ph) / se) >= z_crit
                              and min(vis_n, hid_n) >= need_n) else "FAIL",
    }

print(audit_gate(2413, 4231, 24360, 42684))
# {'lift_pp': 0.46, 'z': 0.578, 'vis_ci': (56.05, 59.01),
#  'n_needed_per_arm': 544, 'verdict': 'FAIL'}

The n_needed_per_arm term matters: at a 57% base rate, resolving a 3pp lift takes ~544 graded rows per arm. Both arms clear that bar, so the effect is genuinely absent — which blocks the usual answer ("we need more data").

Four changes follow: publish the gate's own scorecard, not only its output; keep grading suppressed rows, since 99.45% coverage is what made this audit possible; wire up the dead columns or delete them; and make the decision and the label one event, since 803 published rows with a veto reason mean the publish and skip paths can disagree.

The uncomfortable summary

A gate publishing 8.9% of 47,150 graded rows, with a pooled lift of +0.48pp (z = 0.60), a per-model lift of −7.92pp across 218 models where it fails on 146, an A-tier 3.6pp below its own D-tier, a confidence curve that publishes most where accuracy is lowest, and 19% of its output carrying a veto reason.

That is not a tuning failure. It is what happens when a publication decision is made by a bucket statistic and never scored against its own counterfactual. The fix is not a better threshold — it is making the gate answerable to the rows it threw away, which requires having graded them.

Analytics and research content. Not betting advice. 18+

本内容仅供数据分析与操作参考,不构成任何决策建议。

风险提示:足球比赛结果具有内在不确定性,任何数据分析工具都无法消除这种随机性。本页内容仅供数据操作与参考,不构成任何决策建议或收益承诺。请理性看待体育赛事,远离任何形式的非法决策活动。
看完方法论,直接看它怎么用在今天的比赛上

SoccerDream AI 每场给出胜平负概率分布、总进球区间与置信度, 赛后自动复盘、命中记录公开可查。注册后可查看全部 AI 模型推演与已结算战绩。

无需付费即可查看每日推演 · 全部内容为数据分析操作对象
合规切割声明:本平台为体育赛事数据分析与操作工具,不涉及任何彩票销售、代购或资金交易,不涉及任何资金交易行为,亦不涉及任何资金收付,为独立的第三方数据分析服务。全部内容仅供操作与参考。请遵守所在地法律法规,理性对待。
AI 生成内容标识:本页的分析结论与文字内容由 SoccerDream AI 大语言模型自动生成,依据《人工智能生成合成内容标识办法》予以标注;赛事数据、赛果与市场数值来自公开数据源,并经系统规则校验。内容仅供操作参考,不构成任何建议。