051: Strategy evaluation techniques, flaws and solutions with Dave Walton

051: Strategy evaluation techniques, flaws and solutions with Dave Walton

Today we’re covering a topic which can really be a concern for traders of all levels, from beginner to pro, and that is the topic of strategy evaluation.

  • Have you ever found that real-life performance does not match expected results?
  • Or perhaps you have a strategy that is stuck in a drawdown and wondering if it’s actually broken?

I’m sure we’ve all heard of data mining bias, over-optimization and curve fitting and the impacts this can have on our trading accounts.

We may be even using techniques such as Out Of Sample testing, Walk Forward Analysis, Monte Carlo analysis and a number of other measures to identify or reduce the impact of these issues, but do these approaches actually work? Are there limitations or dangers with these techniques? Are there better ways?

In this episode we talk to someone who evaluates trading systems for a living, plus his research into system evaluation techniques has won awards. The guest is Dave Walton.

Dave was the winner of the Wagner award in 2014 for a paper titled ‘Know your system – turning data mining bias to benefit through System Parameter Permutation’.

In our chat today we talk about the technique in his paper and how it can be applied to trading strategy evaluation. We also discuss some of the assumptions and limitations of the approach, and he shares with us some valuable insights he’s made since publishing the paper which have resulted in an updated approach he now considers a better alternative, so make sure you listen out for that.

Topics discussed
  • How the typical approaches to system development can introduce datamining bias without you knowing
  • The types of systems that can increase the chance of data mining bias and what to look for
  • How the method of splitting your out of sample data could be causing you to throw away good strategies
  • Out of sample, walk forward analysis and Monte Carlo - do they actually reduce data mining bias?
  • The problems with using Monte Carlo analysis to assess strategy performance and why it doesn’t protect from overfitting
  • System Parameter Permutation - how to use it, why use the median, parameter range selection and new insights since the SPP paper was published
  • How System Parameter Randomization solves some of the issues of System Parameter Permutation
  • Stochastic modelling and how it can be used to determine if a rule is adding value to your strategy

Disclaimer:
Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

Jaksot(242)

185: How to avoid creating trading strategies that degrade quickly - Timothy Masters

185: How to avoid creating trading strategies that degrade quickly - Timothy Masters

Numerical Computing specialist and author Timothy Masters joins us (in his only ever interview) to discuss trading strategy development and validation techniques, including: Why trading strategies fall apart in live trading, How luck impacts trading strategy results and the impact of selecting the 'best' trading system, Why trades in an out of sample dataset are NOT representative of future trades, 3 ways to use Monte Carlo Permutation tests in trading strategy development to avoid overfitting, evaluate strategy robustness and assess the quality of your strategy development process, The danger of using out of sample trades to compute the probability of drawdowns, Entropy and information of indicators and why it's important to strategy development, How stationarity really impacts trading strategies and what strategy developers can do about it, Using Walk Forward to determine how robust a strategy is in the market, Incomplete Beta Distribution to track strategy deterioration, Bootstrapping, granular Profit Factor, the predictive ability of technical indicators, counter-trend trading and much more.   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

12 Huhti 20211h 16min

184: Increasing returns using "Performance Qualifiers" – Tomas Nesnidal

184: Increasing returns using "Performance Qualifiers" – Tomas Nesnidal

Breakout trading specialist and Hedge Fund manager Tomas Nesnidal joins us to discuss how to increase the returns of trading strategies using "Performance Qualifiers" while also managing risk, including: What is Dynamic Position Sizing (DPS) and how can it improve risk-adjusted returns, How Dynamic Position Sizing is different to classical position sizing (and why DPS is better), Why it’s important for traders to connect position size with what’s going on in the markets, instead of just their equity curve, The 5 groups of Performance Qualifiers and how to use them to identify the probabilities of future trades, Why using filters to improve trading strategies can be dangerous (and DPS is a much better option instead), Plus, Dynamic Position Sizing and risk management, Optimal-f and older position sizing techniques, Dynamic Position Sizing for stocks, forex and other markets, why it’s important for traders to try new concepts, and much more. ► Download the free DPS ebook at https://dpstradingtechnique.com  or discover more from Tomas at https://systemsontheroad.com   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

30 Maalis 202158min

183: Overcoming Broken Strategies – Marsten Parker

183: Overcoming Broken Strategies – Marsten Parker

Unknown market wizard Marsten Parker joins us to share how he overcame broken strategies and adapted his trading to deliver a 20-year track record most traders would envy, including: Being featured in the “Unknown Market Wizards” book by Jack Schwager, The impacts of a 45% drawdown and how Marsten got back into trading, How to use a system stop to protect your trading account from large losses, Why it’s important to understand why a strategy works, Mean Reversion short strategies, why they’re dangerous and how to mitigate some of the risk, How to detect and protect yourself from a strategy that’s broken, Uncertainty in the market and the future performance of strategies, Equity curve trading, market speed, diversified strategies, tracking old strategies, indicators, market regimes and much more.   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

24 Maalis 20211h 2min

182: How to identify high probability markets - David Steets

182: How to identify high probability markets - David Steets

David Steets from Systematic Individual Investor joins us to discuss how to map market probabilities and identify high probability markets using "edge stacking" techniques, including: The importance of mapping market probabilities, How to identify if something really contains an edge, Blending return probabilities across buckets to create a probability map, Classifying markets over multiple timeframes, How to confirm which edges are currently relevant, Using both technical and fundamental in models, The reliability of edges over time, Managing conflicting information and edges, Bear market signals, integrating discretion, weighting edges, VIX term structure, “anti-edges” and much more.   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

9 Maalis 20211h

181: Short-term Swing Trading Techniques – Ivanhoff

181: Short-term Swing Trading Techniques – Ivanhoff

On this BST live show, Ivan from Ivanhoff Capital joins us to discuss “Short-term swing trading techniques”. Here are some of the topics you’ll discover in this episode: The 2 most important factors in swing trading, Why short-term trading is better than longer-term trading, Measuring market environment and detecting changes, How to find the hot sectors in the market, Shorting stocks and adapting to market corrections and downtrends, Swing trading in choppy markets, Stacking the odds in your favour, High conviction setups, breakouts vs dips, protecting profits, trailing stops, time stops, swing trading psychology and much more.   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

3 Maalis 202142min

180: 3 ways traders kill trading strategies (and how to avoid them) – Rob Carver

180: 3 ways traders kill trading strategies (and how to avoid them) – Rob Carver

On this BST live show, Rob Carver from systematicmoney.org joins us to discuss “3 ways traders kill trading strategies”. Here are just some of the topics you’ll discover in this episode: What can go wrong with strategies in live trading, The impact of model complexity on prediction error, (and are simple models really better than more complicated ones - the answer may surprise you!), The dangers of overstated backtest performance, The impact of historic costs on backtest results, The 3 types of overfitting, including 1 that most traders don’t even realise they’re doing, and how to reduce their impact on your trading models, Strategy robustness and letting data define complexity, The overfitting "levels of sin", Correcting for the "multiple testing problem", Fitting metrics, sample size, data quality, prediction error and much more.   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

24 Helmi 20211h 13min

179: Effective techniques to identify trends – Doc Ahrens

179: Effective techniques to identify trends – Doc Ahrens

On this episode of BST Live, Doc Ahrens from Trendline Dynamics joins us to discuss “Effective techniques to identify trends”, including: The Macro-trend Analyzer indicator and 3 ways to use moving averages for macro trend analysis, Why it's important to have simple trading solutions and what failures in aerospace can teach us about trading strategies,  Studies that conclude technical indicators don't work, Using visual confirmation to understand indicator reaction to market scenarios, The 3 types of support and resistance,  How to draw trendlines properly, why it's impossible to “understand the market”, the importance of persistence in trading and loads more.   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

16 Helmi 20211h 1min

178: How to build Mean Reversion trading strategies – Stefan Friedrichowski

178: How to build Mean Reversion trading strategies – Stefan Friedrichowski

So, the verdict is in… Afrotechmods says: “I think this is the most informative guest you have ever had! Thank you both.” What am I talking about? On this episode of BST Live, Stefan Friedrichowski from JFD Bank joined us to discuss “how to build Mean Reversion trading strategies”, and it was an absolute cracker! Even if you’re not a mean reversion trader, you have to see this one.  Here’s a small sample of the tips and insights you’ll discover: The "alarm signal" that got Stefan interested in Mean Reversion trading, How to determine if a market is good for Mean Reversion or not, How to reduce noise and increase the information content in trading charts,  The danger of using a Mean which moves too fast, How the distribution of 2 bar combinations gives clues about potential trading edges, Adjusting the Mean length based on the timeframe of the chart,  Stop-loss vs take profit size, trading without a stop loss, degrees of freedom, statistical significance, sample size, entries, exits, and loads more.   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

10 Helmi 20211h 30min

Suosittua kategoriassa Liike-elämä ja talous

sijotuskasti
psykopodiaa-podcast
mimmit-sijoittaa
puheenaihe
rss-rahapodi
ostan-asuntoja-podcast
rss-rahamania
herrasmieshakkerit
hyva-paha-johtaminen
rss-lahtijat
rss-startup-ministerio
rss-paasipodi
taloudellinen-mielenrauha
pomojen-suusta
rss-bisnesta-bebeja
rss-seuraava-potilas
oppimisen-psykologia
rss-myyntipodi
rss-doulapodi
rss-markkinointitrippi