Random Forest Probability Calibration for Systematic Edge

How It Works The strategy trains a Random Forest classifier on features relevant to a market category (e.g., historical resolution rates, time-to-expiry, liquidity metrics, external data signals) and uses Platt scaling or isotonic regression to calibrate the raw model output into a true probability estimate. When the calibrated model probability differs from the current market price by more than a threshold, a directional position is taken.

Entry/Exit Conditions Entry is triggered when |model_prob - market_price| > edge_threshold, where the threshold is set to exceed transaction costs and provide a positive expected value. The position is sized proportionally to the edge. Exit occurs at a target price reversion, at market resolution, or via a time-based stop if the price fails to converge.

Verified Results The repo explicitly mentions backtesting as a component of the pipeline ('systematic trading pipeline... backtesting'), suggesting the model has been tested on historical Polymarket data. An automated paper-trading bot is also mentioned, implying forward-validation is in progress. No specific Sharpe ratio or accuracy metrics are disclosed.

Implementation Notes The pipeline architecture (data collection → feature engineering → Random Forest training → calibration → backtesting → paper trading) is well-structured for iterative improvement. Calibration is critical in prediction markets because raw classifier outputs are poorly suited to direct comparison with market prices. The author appears to be building toward live trading from a rigorous paper-trading baseline.

Risks Random Forests can overfit to the historical sample, especially with limited Polymarket resolution data. Calibration quality degrades for out-of-distribution events. Market prices may already incorporate the same publicly observable signals used as features, eliminating the edge. The model requires continuous retraining as market dynamics evolve.