10. System Analysis

Your trading system review

Following the previous chapter youíve now tested your system at least once and have the results sitting in front of you. So how do you interpret those results and make informed system tweaks? Read on and Iíll tell you.

As Iíve mentioned before, trading, as a business, is unique in that you can test your business before you ever risk a cent.You can gain a complete and intricate understanding of how your system works.

Through trading system review, you will discover how even a slight change in your systemís variables can have dramatic effect. And the more you play around with these variables, the more you will come to understand the relationships between them.An example of this can be seen in the relationship between risk and reward. Systems that tend to have higher returns also tend to have larger drawdowns (risk).

In the past, I have designed systems that return up to 300% p.a. but they have drawdowns of over 100% Ė that is to say, youíre guaranteed to lose your entire float and some when trading this system. In short, the greater the reward, the greater the risk.

With this in mind the astute reader may have realised profitability isnít the only criterion by which you should be evaluating a trading system.

Here are a few other questions you should be asking:

  1. What percentage of wins are you achieving against the percentage of losses?
  2. What is the average value of your wins compared with the average value of your losses?
  3. How much money can your system make, on average, for every dollar that you risk?
  4. How many losses in a row does your system generate?
  5. What is your systemís maximum drawdown?
  6. How many trades does your system generate?
  7. And, of course, how profitable is your system?

To fully answer these, and other similar questions, you must analyse the results from your back testing. Conduct a trading system review. Unfortunately, with the overabundance of trading statistics which most back testing programs provide, this can be easier said than done.

Letís take a closer look at the key metrics you need to pay close attention to.

Win-to-loss ratio

When assessing the performance of a trading system, one of the first statistics that gives you a good indication of tradability is the win-to-loss ratio.Quite simply, this is the ratio of the average winning trades taken against the average losing trades taken.If this ratio indicates you are winning more often than you are losing, you are on the right track.

But donít get caught up in this statistic on its own, because it doesnít tell the whole story. It doesnít consider the size of your winning trades versus the size of your losing trades.

Remember the Turtles? Their win-to-loss ratio was 40:60, but they were still hugely profitable.

Average wins and losses

In addition to the win-to-loss ratio, you will want to make sure that the average value of your winning trades is greater than the average value of your losing ones.Say your back testing consisted of 200 trades. If 150 are losing trades and only 50 are winning trades, obviously your win-to-loss ratio is 25:75.

But that on its own isnít enough to determine if a system is good or bad.Understand that, if the average of your wins was, for example, $2000 and the average of your losses was $500, you are still coming out on top ((50×2000)Ė(150×500)=$25,000).

Expectancy

We talked about expectancy in chapter 6 on money management, but Iím going to take this concept a step further.A trading systemís expectancy is perhaps one of the most powerful statistics you can have because it is a way of quantifying the performance of a system that is independent of the size of the trading float.

Expectancy is different to the risk-to-reward ratio and average wins to losses that we described above, in that it defines a return in dollar terms for every dollar that you risk.If your system has an expectancy of +0.75, on average, you would expect to make 0.75 times the amount you risked in the trade.

If you risk $1, then you would expect to make, on average, $0.75 profit for every trade you take. As a guide, if you can achieve expectancy of $0.60, youíre heading in the right direction.

Please refer back to chapter 6 for details on how to calculate expectancy.

Maximum consecutive losses

Look back through your testing results to see, statistically, how many losses in a row your system sustained while still being profitable.This is important to know upfront, since this statistic will give you confidence during those low times when it feels like you should throw in the towel.

For example, imagine you have been hit with five or six losses in a row. Without knowing your maximum consecutive losses, you might think your system isnít working.This is where most naive traders go wrong. The truth be known, based on the historical data, your system may have actually sustained ten losses and still been profitable. This is why it is so important† to do a trading system review.

Maximum drawdown

The maximum drawdown is the worst period of Ďpeak to valleyí performance of your system, regardless of whether or not the drawdown consisted of consecutive months of negative performance.

This statistic is automatically calculated, so itís just a matter of asking yourself: am I comfortable with that size loss?If not, you will need to do more system tweaking to get it to a level that you can live with.

Again, it all comes back to the risk-to-reward ratio. Typically the more risk you take, the greater the reward.

I have traded a system in the past that returned 140% p.a. Now that sounds great, but that particular system had a maximum drawdown of 80%.Could you trade a system where itís likely youíd lose 80% of all your capital at least once while trading it? Could you stomach that?

Itís important you trade a system youíre comfortable with.

Number of trades

Then thereís the number of trades a system gives over the course of a year. I find this an invaluable, yet rarely talked about, statistic.Your trading system should not give too many or too few trades. The number of trades that a trading system gives should be approximately the same as that which can realistically be taken.

The two sides of the coin are equally dangerous. If a system gives too many trades, you will be forced to choose between signals, therefore adding ambiguity to the system.With ambiguity comes human discretion and this often has a detrimental effect on the performance of the trading system.

On the other hand, if a system gives too few trades, your trading capital will not be fully utilised and you may not be taking full advantage of the available trading opportunities.

So how do you calculate the optimal number of trades for a trading system?This is done with the calculation called Ďopportunityí. Opportunity helps determine your optimal opportunity for a trading system, as demonstrated below.

uts-fig-101

Profitability

Profitability is simply the return on investment over a yearly term.Letís be blunt. Weíre all in this to make money. At the end of the day, profitability really is the most important metric by which to measure our success. But, while itís important, it needs to be balanced with the other six measures Iíve just discussed.

Ongoing monitoring

Calculating all these metrics manually doesnít seem too appealing. Thankfully, most back testing software will calculate these measures for you.

As Iíve mentioned, the goal of back testing is to make sure you know and are comfortable with the metrics of your system. The metrics will give you rules by which to trade, but you must continue to monitor your system and compare real-time statistics with your back tested results.If it trades outside of those statistics, when trading real time, you will need to investigate why.

Remember, analysing your trading metrics is critical, however often overlooked. I cannot stress how important it is to conduct a trading system review.

Action

  1. To date Iím yet to find a course that teaches, in detail, how to correctly analyse a system Ė accordingly, Iím now looking to create my own. In the meantime, check out Tharpís Trade Your Way to Financial Freedom for more tips†† www.ultimate-trading-systems.com/tywtff

6 Responses to 10. System Analysis

  1. Jim B November 10, 2009 at 3:05 pm #

    David,

    You did not mention your excel program “Trading Summary” which answers all these questions.

    Again a great video

  2. David Jenyns November 10, 2009 at 4:52 pm #

    Hey Jim,

    Thanks for the prompt…

    Yes, the spreadsheet in http://www.trading-secrets-revealed.com helps with the on going maintenance of your system – it calculates all the key metrics you need to keep an eye on. Again, I very much appreciate your support.

    Your Trading Coach,

    David Jenyns

  3. Steven M December 17, 2009 at 6:19 pm #

    In chapter 6 you defined expectancy as:

    (% of wins x average win size) Ė (% of losses x average loss size)

    So in our example we can see that:

    (80% x $100) Ė (20% x $1000) or $80 Ė $200 = Ė$120

    In this chapter you express expectancy as a decimal instead of a dollar amount:

    “If your system has an expectancy of +0.75, on average, you would expect to make 0.75 times the amount you risked in the trade.”

    I’m a little confused, does this mean that in the first example
    that I would lose $120 for every dollar I risked?

  4. David Jenyns December 18, 2009 at 12:19 am #

    Hi Steve thanks for the question… you’re right I didn’t clearly explain this.

    The difference between the two examples is that one is referencing the expectancy per trade, while the other is the expectancy per $1 risked.

    Here’s what I mean:

    Expectancy = (% of wins x average win size) Ė (% of losses x average loss size)

    Per trade that would be:
    (80% x $100) Ė (20% x $1000) or $80 Ė $200 = Ė$120

    Per $ risked that would be:
    (0.8*1) – (0.2*10) or 0.8 – 2 = -1.2

    Hope that helps clear things up.

    Your Trading Coach,

    David Jenyns

  5. Andrew August 13, 2010 at 7:46 am #

    Hi David,

    I’ve been testing my system in real time and have kept all of the metrics you’ve mentioned. I’m interested in backtesting but my entry is based on historical landmark prices + Moving Averages etc.

    I’m not sure how this can be mechanically back-tested? Any ideas?

    Cheers,

    Andrew

  6. David Jenyns August 22, 2010 at 11:30 pm #

    Hi Andrew,

    You should be able to do it. To ensure I’m understanding you correctly, please define what you mean by “landmark prices” – are these subjective or discretionary?

    In short, if you can define your entry system to a set of rules, you should be able to back test it.

    Your Trading Coach,
    Dave

Leave a Reply