Module 4 · Lesson 11 of 12 · Volume and indicators · About 5 minutes
Overfitting and Repainting: When Indicator Results Look Too Good
On this page
Before you read: Technical Indicators Explained: MA, RSI, MACD, Bollinger, ATR
What you will learn
- What overfitting is and how tuning many settings can produce results that fit only the past.
- How current-bar updates, higher-timeframe confirmation, backplotted pivots, future leaks and data revisions differ, and why historical placement does not establish earlier knowledge.
- How in-sample and out-of-sample data and walk-forward testing are used to check results.
What is overfitting?
If you tried a thousand combinations of settings, would the best one be skill or luck?
Overfitting happens when a set of rules is tuned so closely to past data that it captures noise, the random details of that period, rather than anything that repeats. Each extra setting, such as a moving average length, a threshold or a time filter, gives more chances to fit noise. Testing many combinations and keeping the best one is a form of multiple testing: some combination will look good by chance.
Warning signs include results that change sharply when one setting moves slightly, a very small number of trades, and rules that are hard to explain without pointing to specific past events.
What is repainting?
If a signal on yesterday’s chart can change today, what did a live user actually see?
TradingView’s Pine Script documentation describes repainting as script behavior that makes historical and realtime calculations or plots behave differently. It also notes that not all repainting is misleading. The word covers several different mechanisms, so record which one applies:
| Mechanism | What changed or was unavailable? | What a bar-close rule does NOT automatically fix |
|---|---|---|
| Fluid current bar | High/low/close and derived values change before that bar ends. | It can wait for that bar, but says nothing about another timeframe. |
| Unconfirmed higher timeframe | A closed five-minute bar can still use a developing hourly value. | Closing the five-minute bar does not close the hour. |
| Delayed pivot backplot | A pivot is confirmed using later bars, then drawn on its earlier event bar. | The marker’s plotted timestamp is not its availability timestamp. |
| Future leak/lookahead error | A historical calculation uses information unavailable at that earlier instant. | Simply requesting once per bar does not remove the future input. |
| Feed revision/history/seed change | Corrected observations or a changed history start alter recalculation. | Previously closed data are not guaranteed immutable; record version/input history. |
Ordinary fluid updates are not inherently fraud, nor is every changed number a profitable-looking historical trick. Future leakage and presenting later-confirmed markers as earlier knowledge are different problems.
Fictional availability record. All times are CST, and bar times are bar-close labels. The hourly interval is 09:00–10:00. At 09:05 a five-minute bar is closed, but the hourly close is still developing: its temporary last-price value is 102. At 10:00 the hourly close is 98. Reporting the provisional 102 as provisional at 09:05 is different from backfilling 98 as though known then. Waiting for the 09:05 bar alone would not have supplied the 10:00 value.
For a separate two-right-bar pivot illustration, five-minute bar highs at close labels 09:00, 09:05, 09:10, 09:15, 09:20 are 101, 103, 105, 104, 102. The strict local high 105 at 09:10 has two lower bars on each side only once 09:20 closes. A marker may then be plotted at 09:10, but its first confirmation is 09:20, a 10-minute delay under these rules. At 09:10 the two right-hand highs were unknown; no historical marker promises an earlier usable signal.
Keep an as-of input record: bar timeframe/close labels, selected higher-timeframe status, confirmation rule, feed/version and event-versus-availability timestamp. See Market Structure Explained: Swings, BOS and CHoCH for the existing delayed-pivot concept and Trading Plans, Journaling and Honest Backtesting for honest testing.
Overfitting diagram
Schematic, not measured performance.
PROVISIONAL
CONFIRMED AS OF THAT INPUT RECORD
Illustrative diagram constructed for teaching. Not real market data.
The curve is a schematic of overfitting. The inset separates when an event happened from when it could be known, and a closed five-minute bar from a still-open hourly bar. Neither panel shows live performance.
How are results checked?
How would you test rules on data they have never seen?
A common approach splits data into an in-sample period, used to build and tune rules, and an out-of-sample period, set aside and used once to check them. Walk-forward testing repeats this: tune on one window, test on the next, then move both forward. Results that hold up out of sample are less likely to be pure fitting, though no test removes the chance that conditions change.
What this does not tell you about testing
- Whether any rule set will hold up in the future.
- How many settings are too many. That depends on the data and the number of trades.
- Whether an indicator repaints. Check its documentation and watch it live.
- Nothing here recommends any strategy, setting or indicator.
Method in Practice: tune in-sample, then watch out-of-sample
Fictional example. John, Harbor Index futures, the prices, times, costs and results are all invented for teaching. This shows how a written plan can be built and tested. It is not a recommendation, and nothing here says how often this idea works.
Try it on a real chart. Test this idea yourself on past sessions with bar replay. Keep your own count, including the days with no trade.
The idea. Overfitting means tuning settings until they fit the past, including its noise (this lesson). John tries it on purpose with his moving-average filter from Moving Averages in Depth: SMA, EMA, Lag and Crossovers, so he can see it happen. He tunes on sessions he has already looked at, then checks the "best" setting once on sessions he held back.
John's test plan (written before testing)
- Rule
- The Moving Averages in Depth: SMA, EMA, Lag and Crossovers lesson's plan. Only the SMA length changes
- Lengths tried
- 20, 30, 40, 50, 60, 70, 80, 90, 100. All nine are logged
- In-sample
- sessions 1–60. Out-of-sample: sessions 61–100, opened once, after choosing
- Costs
- $5 per contract and 1 tick of slippage, from the start
Three outcomes (invented numbers: this shows how to count, not how the method performs)
-
In-sample: length 70 looked best, at +$1,240 over 30 trades
-
Out-of-sample: length 70 came out at −$310 over 18 trades
-
John's decision: no change. He keeps length 50, chosen before testing, and keeps testing it
Show the full replays
In-sample results, sessions 1–60 (invented numbers: this shows how to count, not how the method performs)
| SMA length | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 90 | 100 |
|---|---|---|---|---|---|---|---|---|---|
| Trades | 41 | 38 | 35 | 33 | 31 | 30 | 28 | 25 | 24 |
| Net after costs | −$420 | −$150 | +$60 | +$210 | −$180 | +$1,240 | +$90 | −$260 | −$330 |
What John notices: length 70 is far ahead, but its neighbors (60 and 80) are at −$180 and +$90. That's a lone peak.
Out-of-sample, sessions 61–100, opened once
| SMA length | Trades | Net after costs |
|---|---|---|
| 70 (in-sample "best") | 18 | −$310 |
| 50 (chosen before testing) | 20 | +$40 |
John's decision, journal: "Length 70 fit sessions 1–60, not the market. Nine tries made one of them look great by luck. I'm keeping 50, which I picked before testing, and I'm not calling +$40 on 20 trades a result either. Next test: 40 more held-back sessions."
Test it yourself
- Write every setting you'll try before you start, and log them all, not just the winner.
- Hold back sessions you won't look at until the end, and open them once.
- Look at the neighbors of the best setting. A lone peak with weak neighbors is a warning sign.
Where it breaks
- With enough settings, one always looks good by luck.
- A "best" setting surrounded by poor ones usually fit noise.
- Indicators that repaint make past signals look better than live ones. Check yours live (this lesson).
Calculators: expectancy
Reflection question: How many versions of your plan did you try before the one you use, and is that number written down?
Check yourself
- Why does testing many setting combinations raise the chance of a misleading best result? Hint: how many chances to fit noise?
- How can a repainting indicator look better on history than in live use? Hint: where is the signal drawn after it is confirmed?
- What is out-of-sample data, and why should it be used only once? Hint: what happens if you keep tuning after looking at it?
- At 09:05 is the five-minute bar closed? Is hourly 98 available then? When is the 09:10 pivot confirmed?
- Must every later change be a future leak?
Reflect on your own process
- How many setting combinations did you try before settling on your current rules, and is that number in your journal?
- Which part of your data, if any, have you set aside and not yet looked at?
- For each indicator in your plan, have you recorded when its value is final and whether it can redraw past bars?
- How many versions of your plan did you try before the one you use, and is that number written down?