Adversarial Information-Time Testing for Financial Machine-Learning Pipelines
A Mutation-Based Benchmark for Temporal Leakage Detection
Edmen Wong · AQP TECH ENTERPRISE · Alpha Tick Lab Research / Alpha Quant Pro
Abstract
Temporal leakage in financial machine-learning pipelines is not limited to explicit future-row access. This paper develops Adversarial Information-Time Testing (AITT), a behavioral framework that preserves all information legitimately available by a historical cutoff while deliberately perturbing information outside that authorized set. If an artifact attributed to the cutoff is temporally non-anticipative, features, preprocessing state, model-selection state, training eligibility, signals, and execution state should remain invariant under such mutations.
In a controlled benchmark with six paired causal/leaky fixtures, each evaluated across 200 synthetic series and three cutoffs, no violations occur in 3,600 causal-control trials. Targeted mutations detect all 3,000 trials covering centered windows, full-sample normalization, global feature selection, immature labels, and open-stamped completed bars. A random execution mutation detects only 286 of 600 planted same-open execution violations, while a targeted sign-flip adversary detects all 600, showing that finite mutation testing depends on mutation adequacy and is a falsification tool rather than proof of universal causal correctness.
1. Research question and contribution
The research question is narrow: given an explicit information-time contract, can temporal violations be exposed by changing only information the simulated past was not authorized to use?
2. Formal test
Let F_T denote all information legitimately available by cutoff T. An artifact A_T is temporally non-anticipative when it can be written as a function of F_T alone.
Authorization-preserving mutation: M_T(D)|F_T = D|F_T Prefix invariance: A_T(M_T(D)) = A_T(D) Violation criterion: ||A_T(M_T(D)) - A_T(D)|| > epsilon
If the authorized information set is unchanged but a supposedly historical artifact changes, either the pipeline consumed unauthorized information, nondeterministic state was not controlled, or the declared temporal contract is incomplete.
3. Benchmark design
The reference benchmark uses five synthetic AR-style feature processes and a synthetic return process with known lagged dependencies. Each fixture is tested with 200 independent seeds at cutoffs 0.4N, 0.6N, and 0.8N, producing 600 trials per fixture.
The six paired classes test rolling-window causality, preprocessing ownership, feature-selection ownership, label maturity, completed-bar availability, and execution authority. Synthetic data are used so that the planted causal status is known by construction; the benchmark makes no claim about strategy profitability.
4. Results
| Fixture | Status | Trials | Violations | Rate |
|---|---|---|---|---|
| Trailing rolling mean | Causal | 600 | 0 | 0.000 |
| Centered rolling mean | Leaky | 600 | 600 | 1.000 |
| Expanding normalization | Causal | 600 | 0 | 0.000 |
| Full-sample normalization | Leaky | 600 | 600 | 1.000 |
| Fold-local feature selection | Causal | 600 | 0 | 0.000 |
| Global feature selection | Leaky | 600 | 600 | 1.000 |
| Maturity-aware training | Causal | 600 | 0 | 0.000 |
| Row-index-only label eligibility | Leaky | 600 | 600 | 1.000 |
| Availability-aware completed bar | Causal | 600 | 0 | 0.000 |
| Open-stamped completed bar | Leaky | 600 | 600 | 1.000 |
| Next-open execution | Causal | 600 | 0 | 0.000 |
| Same-open retro-execution · random mutation | Leaky | 600 | 286 | 0.477 |
5. Mutation adequacy matters
The execution fixture produces the paper's most important negative result. A random mutation of an unavailable close activates only 286 of 600 planted retroactive-execution faults. Replacing it with a targeted sign-flip adversary raises detection to 600 of 600.
Interpretation: passing one mutation test is not proof of causal correctness. Mutation families need coverage and adequacy checks, just as ordinary software test suites do.
6. Relationship to Technical Note 01
Technical Note 01 introduced the engineering contract: storage time is not automatically information time; labels have maturity times; selection must be fold-owned; and execution must respect signal availability. This paper does not re-claim those ideas as a new publication contribution.
Research Paper 01 focuses on the next problem: how those temporal claims can be falsified automatically. The new material is the formal mutation relation, paired causal/leaky fixtures, empirical fault-injection results, mutation-adequacy analysis, and public reproducibility package.
7. Limitations
The benchmark is synthetic and intentionally constructed. High mutant-kill rates are conditional on the specified fixtures and mutation distributions and must not be interpreted as universal leakage-detection accuracy.
Finite mutation testing can miss a dependency that the chosen adversary does not activate. Point-in-time source data, code review, walk-forward validation, purging or embargo where appropriate, transaction-cost modeling, and statistical robustness remain separate requirements.
AITT tests temporal admissibility. It does not establish predictive skill, economic value, capacity, robustness to regime change, or profitable live trading.
8. Reproducibility
The reference benchmark contains no proprietary market or client data. The public package includes the benchmark script, pinned reference environment, per-trial outputs, summary results, and targeted execution-adversary results.
Open reproducibility repository9. Public technical discussion
The benchmark's mutation-adequacy question is being discussed publicly. The external thread is deliberately framed as a technical question rather than a promotional link to the paper.