051: Strategy evaluation techniques, flaws and solutions with Dave Walton

051: Strategy evaluation techniques, flaws and solutions with Dave Walton

Today we’re covering a topic which can really be a concern for traders of all levels, from beginner to pro, and that is the topic of strategy evaluation.

  • Have you ever found that real-life performance does not match expected results?
  • Or perhaps you have a strategy that is stuck in a drawdown and wondering if it’s actually broken?

I’m sure we’ve all heard of data mining bias, over-optimization and curve fitting and the impacts this can have on our trading accounts.

We may be even using techniques such as Out Of Sample testing, Walk Forward Analysis, Monte Carlo analysis and a number of other measures to identify or reduce the impact of these issues, but do these approaches actually work? Are there limitations or dangers with these techniques? Are there better ways?

In this episode we talk to someone who evaluates trading systems for a living, plus his research into system evaluation techniques has won awards. The guest is Dave Walton.

Dave was the winner of the Wagner award in 2014 for a paper titled ‘Know your system – turning data mining bias to benefit through System Parameter Permutation’.

In our chat today we talk about the technique in his paper and how it can be applied to trading strategy evaluation. We also discuss some of the assumptions and limitations of the approach, and he shares with us some valuable insights he’s made since publishing the paper which have resulted in an updated approach he now considers a better alternative, so make sure you listen out for that.

Topics discussed
  • How the typical approaches to system development can introduce datamining bias without you knowing
  • The types of systems that can increase the chance of data mining bias and what to look for
  • How the method of splitting your out of sample data could be causing you to throw away good strategies
  • Out of sample, walk forward analysis and Monte Carlo - do they actually reduce data mining bias?
  • The problems with using Monte Carlo analysis to assess strategy performance and why it doesn’t protect from overfitting
  • System Parameter Permutation - how to use it, why use the median, parameter range selection and new insights since the SPP paper was published
  • How System Parameter Randomization solves some of the issues of System Parameter Permutation
  • Stochastic modelling and how it can be used to determine if a rule is adding value to your strategy

Disclaimer:
Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

Avsnitt(242)

010: Perry Kaufman discusses applications of market noise, mitigating price shocks, volatility and using the Information Ratio to monitor strategy performance.

010: Perry Kaufman discusses applications of market noise, mitigating price shocks, volatility and using the Information Ratio to monitor strategy performance.

Perry Kaufman discusses market noise, the impact it has on trading styles and how it can be used to determine which strategies suit a particular market. We also talk about price shocks and how to mitigate their effects, how to use volatility in your favour, volatility parity for position sizing, the information ratio for strategy performance and some strategy ideas you can test yourself. Topics discussed How market noise impacts trend following and mean reversion The effects of money flow during a crisis The types of strategies that work best in new markets and why How the efficiency ratio can be used to determine the best type of strategy for a market The best markets for trend following and mean reversion What strategy style to choose if you're just starting out Using Volatility Parity for position sizing Impacts and dangers of price shocks on backtesting and how to handle them Mitigating the risk of price shocks Using the Information Ratio to measure strategy performance and detect possible over-fitting The effects of volatility on strategies and how to use volatility in your favour Fractal Geometry High Frequency Trading And some strategy ideas you can test   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

7 Juni 20151h 13min

009: Gary Antonacci discusses the different types of momentum and how they can be used in a strategy to make profit and protect during a market downturn.

009: Gary Antonacci discusses the different types of momentum and how they can be used in a strategy to make profit and protect during a market downturn.

Gary Antonacci discuss all things momentum; the different types of momentum and their individual applications, how they can be combined, how momentum can be refined, the benefits and challenges of momentum, why momentum works and much more. Topics discussed The different types of momentum and their use How to combine them for better performance How Dual Momentum protects in a bear market The benefits of Dual Momentum The impact of lookback periods and rebalancing Using Dual Momentum with other asset classes Reducing the impact of mean reversion in stocks Why momentum works and is likely to continue working in the future   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

31 Maj 201534min

008: Gary Stone discusses market timing, why it works, the impacts it can have on performance and how to do it correctly to beat the majority of fund managers.

008: Gary Stone discusses market timing, why it works, the impacts it can have on performance and how to do it correctly to beat the majority of fund managers.

Gary Stone discusses market timing, why it works, the impact it can have on strategy performance and simple methods you can test for yourself. We also discuss the misinformation published by some fund managers and how you can outperform the majority of professional fund managers using very simple techniques. Gary also explains the importance of perspective and how it can improve your trading and we get some tips on one of the most important components to executing your trading plan correctly.   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

24 Maj 201554min

007: Rob Hanna discusses trading ideas, market behaviour, market timing, where to find the best edges and how a confluence of ideas can improve trading results.

007: Rob Hanna discusses trading ideas, market behaviour, market timing, where to find the best edges and how a confluence of ideas can improve trading results.

Rob Hanna tells us how he comes up with so many trading ideas, the type of market behaviours to look out for and his favourite piece of information in a backtest report (clue: it’s not a number). He also discusses how a confluence of ideas can improve trading results, how to look at market behaviour from different angles, some simple market timing techniques, adjusting strategies based on market conditions plus loads of ideas we can test ourselves. Topics discussed Where to find trading ideas The types of market behaviour to look for and what to ignore The most important piece of information in a backtest – it’s not a number! How to handle an edge that is performing poorly How a confluence of ideas can improve trading results Looking at market behaviour from different angles Simple market timing techniques Where to find the biggest edges Adjusting strategies based on market conditions How Fed meeting days impact the market and how it can be traded PLUS loads of ideas to test!   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

17 Maj 20151h 3min

006: Dr Howard Bandy talks about major changes in system development and trade management, handling systems that are out of sync with the markets and much more.

006: Dr Howard Bandy talks about major changes in system development and trade management, handling systems that are out of sync with the markets and much more.

In our chat with Dr Howard Bandy we talk about major changes occurring in the fields of trading system development, trade management and technical analysis. We also discuss why trading is becoming increasingly difficult, what causes trading systems to experience periods of poor performance, how to identify and manage a trading system when it is out of sync with the market and the 2 most important skills in system development. Topics discussed The major changes occurring in the fields of trading system development, trade management and technical analysis Why trading is becoming increasingly hard How to compete with the professionals What causes trading systems to experience periods of poor performance How to identify and manage a trading system when it is out of sync with the market The shift towards machine learning and pattern recognition The importance of data mining in the system development process The 2 most important skills in system development   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

10 Maj 201550min

005: Kevin Davey discusses important aspects of system development, the best systems to use, the correct backtesting process and how to be a successful trader

005: Kevin Davey discusses important aspects of system development, the best systems to use, the correct backtesting process and how to be a successful trader

Kevin Davey talks about how we went about winning the trading competition, the type of systems you want for the future, some very important aspects of system development that will cost you money if you ignore them, what you need to do before you start trading a strategy and what skills you need to be a good system developer. He offers some really valuable insights and provides an excellent process to system development so you do not want to miss it. Topics discussed:   Why goals and objectives are so important in trading How to use "the monkey test" to validate entries and exits Which type of systems adapt better to future conditions Why walk forward testing is so important and when you shouldn't use it How often should you re-optimise your systems Why ignoring Monte Carlo testing can cost you money Why you shouldn't start trading a strategy immediately after you've finished testing it How to eliminate bad systems before risking your money on them Adding or removing systems from your portfolio Why you need to determine when you will stop trading a system BEFORE you start trading it What skills are required to be a good system developer   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

3 Maj 20151h 4min

004: Nick Radge discusses system design, the best type of trading systems, handling a bear market, measuring performance and the difference between successful traders and everyone else.

004: Nick Radge discusses system design, the best type of trading systems, handling a bear market, measuring performance and the difference between successful traders and everyone else.

In our chat with Nick we discuss system design, the best type of systems to trade, how observations can lead to better strategies, the best way to handle a bear market, measuring system performance, survivorship bias and the difference between successful traders and everyone else. Topics discussed: The trading style most people should try first Systematic ways to determine a trend Why simple systems are the best Why broker consensus is a poor indicator The best type of stocks for trend following strategies What you need to know about a system to be able to handle the drawdown How observations can lead to better strategies Why buy and hold offers average returns and the best approach to handling a bear market How trading can take less than 5 minutes per day Measuring the performance of a system The difference between very successful traders and everyone else The types of survivorship bias that could be ruining your backtest results The warning signs of over-optimisation Coping with drawdown and personal doubts 2 basic strategies you can start testing today!   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

26 Apr 20151h 1min

003: Cesar Alvarez on trading ideas, backtesting, stops and market timing

003: Cesar Alvarez on trading ideas, backtesting, stops and market timing

In our chat with Cesar Alvarez we talk about trading ideas, key concepts and pitfalls of backtesting, the impact of stops on returns, market timing, over-optimisation, drawdown and using multiple strategies in a portfolio. Topics discussed: Where to find trading ideas Why you should test all of your ideas, even if you think they won't work Pitfalls of backtesting you need to look out for How to identify coding errors in backtesting Some clues that indicate you may have over-optimised a strategy Issues with splitting data into in-sample and out-of-sample portions The impact of stops on returns Handling a 50% overnight drop in a stock price Finding your personal drawdown tolerance How market timing can impact a strategy Why you need multiple strategies in a portfolio   Disclaimer: Trading in the financial markets involves a substantial risk of loss and is not suitable for everyone. All content produced by Better System Trader is for informational or educational purposes only and does not constitute trading or investment advice. Past performance is not necessarily indicative of future results.

19 Apr 201554min

Populärt inom Business & ekonomi

framgangspodden
varvet
badfluence
rss-borsens-finest
uppgang-och-fall
svd-ledarredaktionen
avanzapodden
lastbilspodden
rikatillsammans-om-privatekonomi-rikedom-i-livet
rss-kort-lang-analyspodden-fran-di
fill-or-kill
rss-dagen-med-di
affarsvarlden
kapitalet-en-podd-om-ekonomi
dynastin
borsmorgon
tabberaset
montrosepodden
rss-inga-dumma-fragor-om-pengar
borslunch-2