How to backtest trading strategies

Backtesting: Interpreting The Past.

Backtesting is a key component of effective trading-system development. It is accomplished by reconstructing, with historical data, trades that would have occurred in the past using rules defined by a given strategy. The result offers statistics that can be used to gauge the effectiveness of the strategy. Using this data, traders can optimize and improve their strategies, find any technical or theoretical flaws, and gain confidence in their strategy before applying it to the real markets. The underlying theory is that any strategy that worked well in the past is likely to work well in the future, and conversely, any strategy that performed poorly in the past is likely to perform poorly in the future. This article takes a look at what applications are used to backtest, what kind of data is obtained, and how to put it to use!

The Data and The Tools.

Net Profit or Loss - Net percentage gain or loss. Time frame - Past dates in which testing occurred. Universe - Stocks that were included in the backtest. Volatility measures - Maximum percentage upside and downside. Averages - Percentage average gain and average loss, average bars held. Exposure - Percentage of capital invested (or exposed to the market). Ratios - Wins-to-losses ratio. Annualized return - Percentage return over a year. Risk-adjusted return - Percentage return as a function of risk.

Typically, backtesting software will have two screens that are important. The first allows the trader to customize the settings for backtesting. These customizations include everything from time period to commission costs. Here is an example of such a screen in AmiBroker:

The second screen is the actual backtesting results report. This is where you can find all of the statistics mentioned above. Again, here is an example of this screen in AmiBroker:

In general, most trading software contains similar elements. Some high-end software programs also include additional functionality to perform automatic position sizing, optimization and other more-advanced features.

The 10 Commandments.

Take into account the broad market trends in the time frame in which a given strategy was tested. For example, if a strategy was only backtested from 1999-2000, it may not fare well in a bear market. It is often a good idea to backtest over a long time frame that encompasses several different types of market conditions . Take into account the universe in which backtesting occurred. For example, if a broad market system is tested with a universe consisting of tech stocks, it may fail to do well in different sectors. As a general rule, if a strategy is targeted towards a specific genre of stock, limit the universe to that genre; but, in all other cases, maintain a large universe for testing purposes . Volatility measures are extremely important to consider in developing a trading system. This is especially true for leveraged accounts, which are subjected to margin calls if their equity drops below a certain point. Traders should seek to keep volatility low in order to reduce risk and enable easier transition in and out of a given stock . The average number of bars held is also very important to watch when developing a trading system. Although most backtesting software includes commission costs in the final calculations, that does not mean you should ignore this statistic. If possible, raising your average number of bars held can reduce commission costs, and improve your overall return . Exposure is a double-edged sword. Increased exposure can lead to higher profits or higher losses, while decreased exposure means lower profits or lower losses. However, in general, it is a good idea to keep exposure below 70% in order to reduce risk and enable easier transition in and out of a given stock . The average-gain/loss statistic, combined with the wins-to-losses ratio, can be useful for determining optimal position sizing and money management using techniques like the Kelly Criterion. (See Money Management Using the Kelly Criterion .) Traders can take larger positions and reduce commission costs by increasing their average gains and increasing their wins-to-losses ratio . Annualized return is important because it is used as a tool to benchmark a system\'s returns against other investment venues. It is important not only to look at the overall annualized return, but also to take into account the increased or decreased risk. This can be done by looking at the risk-adjusted return, which accounts for various risk factors. Before a trading system is adopted, it must outperform all other investment venues at equal or less risk . Backtesting customization is extremely important. Many backtesting applications have input for commission amounts, round (or fractional) lot sizes, tick sizes, margin requirements, interest rates, slippage assumptions, position-sizing rules, same-bar exit rules, (trailing) stop settings and much more. T o get the most accurate backtesting results, i t is important to tune these settings to mimic the broker that will be used when the system goes live . Backtesting can sometimes lead to something known as over-optimization. This is a condition where performance results are tuned so highly to the past that they are no longer as accurate in the future. It is generally a good idea to implement rules that apply to all stocks, or a select set of targeted stocks, and are not optimized to the extent that the rules are no longer understandable by the creator . Backtesting is not always the most accurate way to gauge the effectiveness of a given trading system. Sometimes strategies that performed well in the past fail to do well in the present. Past performance is not indicative of future results. Be sure to paper trade a system that has been successfully backtested before going live to be sure that the strategy still applies in practice.

Backtesting is one of the most important aspects of developing a trading system. If created and interpreted properly, it can help traders optimize and improve their strategies, find any technical or theoretical flaws, as well as gain confidence in their strategy before applying it to the real world markets.

Smart Stock Charts.

Harnessing the power of Neural Networks to predict critical prices in the Stock Market!

Post navigation.

Ultimate Tools for Backtesting Trading Strategies.

Ultimate Tools for Backtesting Trading Strategies.

Backtesting is the art and science of appraising the performance of a trading or investing strategy by simulating its performance using historical data. You can get a sense of how it performed in the past and its stability and volatility. However, as you may have heard countless times, great backtested performance does not guarantee great future performance. Nevertheless, a not so desirable backtested performance is often a valid reason to drop a particular trading strategy and move on to the next.

Free Backtesting Tools for the Non-Programmer.

There really is no “one size fits all” backtesting tool out there that can backtest virtually any strategy under the sun without the user knowing some programming. If you’re serious about trading, then I urge you to learn enough programming to be able to backtest. But if you’d like to quickly get the results from backtesting some rather simple strategies involving passive investing or basic indicators like moving average crossovers, then a tool will definitely save you some time. Here are the tools I use if I need to run a quick backtest:

#1: ETF Replay.

ETFReplay is a Freemium service that allows you to backtest a variety of different ETF-based investment strategies. A lot of the advanced features and strategies they offer require a subscription but one of the most useful FREE features is the ability to backtest an ETF portfolio of up to 5 components. You can’t rebalance but if you need to quickly compute the equity curve of, say, the performance of a 60/40 portfolio of SPY and TLT between 2008 and 2011, you can do that easily here. This is an excellent tool for helping you backtest asset allocations for the passive portion of your trading portfolio.

#2: StockBackTest.

StockBackTest allows you to backtest strategies involving crossovers of Moving Averages and Bollinger Bands. This is one of the few services that allows you to backtest simple technical indicators like these but the catch is that you can only pick from their list of stocks (which consists of mostly S&P500 securities and the most liquid ETFs.)

#3: Portfolio Visualizer.

Portfolio Visualizer is one of the newer and more sophisticated free backtesters that doesn’t require you to be a programmer. It allows you to backtest passive asset allocations as well as predefined tactical strategies like Gary Antonacci’s Dual Momentum. They’ve also got one of the best Monte Carlo retirement simulators I’ve seen.

Free Backtesting Tools for the Programmer.

For quick backtests of custom strategies, I recommend just downloading some historical data and testing it in Excel or another spreadsheet first. More sophisticated trading strategies will call for GNU R or GNU Octave, both of which have specialized packages for backtesting. If these still don’t cut it for the complexity of your strategy, then you’ve got two robust yet free options available below:

#1: Quantopian (RECOMMENDED)

Quantopian has minute by minute data on all US stocks traded since 2002 which allows you to backtest intraday strategies without being subjected to survivorship bias. You’ll need knowledge of Python to their backtester. If you’re starting from scratch and are serious about learning to backtest your strategies, THIS is the platform I’d recommend concentrating on learning!

#2: MI Backtester.

Jamie Gritton’s MI Backtester is one of the older programmable backtesters available. One of the coolest features of this tool is the ability to backtest stock screens. You might be able to pull up a stock screen like the following: profitable companies with a P/E in the bottom 10% of the US market and a price momentum in the top 10% of the market and get the current picks but you may be wondering how such a screen would have performed historically. The MI Backtester, while a bit slow, will allow you to test the historical performance of such investment strategies based on a combination of fundamentals and technicals.

Did I miss any other Free Backtesters?

If you have any other free backtesters that you use on a regular basis that I’ve not mentioned, please let me know in the comments below!

How to Backtest Your Day Trading Strategy.

In backtesting , a day trader specifies the strategy that he or she would use and then runs that strategy through a database of historic securities prices to see whether it would have made money. The test includes assumptions about commissions, leverage, and position size. The results give information on returns, volatility, and win-loss ratios that you can use to refine a trading strategy and implement it well.

Start with a hypothesis.

You can lay out your strategy as a hypothesis, which may be something like this: “High-momentum, small-cap stocks tend to close up for the day, so I can buy them in the morning and make money selling them in the afternoon.” Or this: “News events take at least half an hour to affect pork belly prices, so I can buy or sell on the news and make a profit.” With this statement, you can move on to the test to see whether your hypothesis holds.

Run the test of your day trading strategy.

Say you start with something simple: Maybe you have reason to think that pharmaceutical companies that are moving down in price on decreasing volume will turn and close up for the day. The first thing you do is enter that into the software: the industry group and the buy pattern that you’re looking for. The results will show whether your hunch is correct and how often and for what time periods.

If you like what you see, you can add more variables. Most backtesting software allows for optimization, which means that it can come up with the leverage, position, holding period, and other parameters that will generate the best risk-adjusted return given the data on hand. You can then compare this result to your trading style and your capital position to see whether it works.

Backtesting is subject to something that traders call over-optimization , mathematicians call curve-fittin g, and analysts call data mining . Although over-optimization sounds great, what often happens is that the test generates a model that includes unnecessary variables and that makes no logical sense in practice.

If you find a strategy that works when the stock closes up one day, down two days, then up a third day, followed by four down days when it hits an intra-day high, you probably haven’t made an amazing discovery; you’ve just fit the curve.

Compare the results with market cycles.

When you backtest, be sure to do so over a long enough period of time so that you can see how your strategy would work over different market conditions. Here are some things to check:

How did the strategy do in periods of inflation? Economic growth? High interest rates? Low interest rates?

What was happening in the markets during the time that the strategy worked best? What was happening when it worked worst? How likely is either of those to happen again?

How does market volatility affect the strategy? Is the security more volatile than the market, less volatile, or does it seem to be removed from the market?

Have major changes occurred in the sector over the period of the test? Examples of these types of changes include new technologies that increase demand for certain commodities or changes in regulation that make industries obsolete. Does this mean that past performance still applies?

Have there been changes in the way that the security trades? For example, the bulk of trading in most commodities used to take place in open-outcry trading pits. Now, trading is almost entirely electronic. How do your test results look given current trading technologies?

How to Backtest Your Trading Strategy Correctly.

Many successful traders share one habit – they backtest their trading strategies. Backtesting your trading strategy will not alone guarantee that you will become profitable, but it is a giant step in the right direction. In this article we examine some potential biases that can creep into your backtesting, and we will look at how to minimize the impact of these biases.

There are many problems that can occur when you backtest your trading system, but most problems fall into one of three categories: postdictive errors, too many variables, or failing to anticipate drastic changes in the market. Each of these errors is explained, along with methods of avoiding errors.

Click here to learn how to utilize Bollinger Bands with a quantified, structured approach to increase your trading edges and secure greater gains with Trading with Bollinger Bands® – A Quantified Guide.

1. Postdictive Error.

The postdictive error is just a fancy way of saying that you have used information only available “after the fact” to test your system. Believe it or not, this is a very common error when testing trading systems.

This error is easy to make. Some software will allow you to use today’s data in testing a trading system, which is always a postdictive error (we do not know if today’s data is useful yet for predicting the future, but we certainly do know if it is useful in predicting the past). Wouldn’t you love to be able to use the closing price of the GBP/USD to predict what the market will do today? Of course you would, I definitely would, but unfortunately, this information is not available to us until the day is over. For example, you may have a system that incorporates the closing price, then this obviously means that the trade cannot be initiated until the day is over , otherwise this is a postdictive error. Another example may help illustrate the postdictive error, if you have a rule in your trading system about highest prices, then you will have a postdictive error. This is because highest prices are often defined by data that comes later, in the future.

The way to avoid the postdictive error is to make sure that when you backtest a system that only information that is available in the past at that point in time is used in backtesting. With manual backtesting or backtesting with forex tester you can accomplish this quite easily, but with automated backtesting the postdictive error can sneak into your trading system.

2. Too Many Variables.

This is also known as the “Degrees of Freedom” bias. This simply means that you have too many variables, or trading indicators in your trading system. It is very possible to come up with a trading system that can explain past price behavior of a currency pair. In fact, the more indicators you add, the easier it often becomes. The problem arrives when you want to apply this system to the future.

Often when a trading system has too many indicators it can predict the behavior of the market during a time period extremely well. But, that’s all the system is good for, because in the future the system falls apart.

The above statement is often difficult for traders to come to grips with, but it is true. Consider what William Eckhardt, of the New Market Wizards has to say about trading systems, In general, the delicate tests that statisticians use to squeeze significance out of marginal data have no place in trading. We need blunt statistical instruments, robust techniques.

Obviously, he is warning against the degrees of freedom error and suggesting that simple trading systems are more likely to stand test of time. This is absolutely true.

Some of the most powerful trading systems available are extremely simple.

Keep this in mind as you trade, and as you attempt to find a profitable trading system. Most traders will find that with experience, they become more likely to embrace the view that simpler trading is preferred over a complex approach.

3. Drastic Changes in the Market.

Many traders forget to anticipate unforeseen events that will occur in the future. It doesn’t really matter that you don’t know what is going to happen in the future – because you do know this: there will be times in the future when the markets will behave erratically. When this happens, you should have designed your trading system to remain functioning during these times.

Perhaps some examples may help with this: When Saddam Hussein was found (over the weekend), the currency markets reacted quite drastically on Monday’s opening. When the global financial crisis started unfolding in September 2008, most currency pairs traded with much more volatility than had been seen for years.

The fact is that there will be unexpected events in the future, and these events will affect the markets, so the best thing you can do is to be prepared. How do you prepare for the unexpected? Consider these simple solutions:

1) Exaggerate your expected losses. If your backtesting reveals a maximum loss of $5000, assume a maximum loss of $10,000. Will your trading systems still be profitable under these conditions?

2) Decide on an appropriate level of risk for each trade. Remember that even this level of risk is likely to be exceeded. If you have decided to risk 1% on each trade, you should assume that sometime in the future, you may be in a trade and an unexpected event will occur, and your trade will not lose 1%, but instead 5% will be lost.

3) You should have a contingency plan set up. That is, how will you exit a trade if something bad happens and you cannot access your account? For instance, what happens if your trading platform is inaccessible and you desperately want out of a trade? Most brokers offer a telephone line to traders for these instances. Do you have the phone number?

4) Do you have a maximum risk level set? This would be applicable if you have several trades open simultaneously. If you decide to risk 1% per trade and you have 7 trades open simultaneously, does this mean that you will be risking 7% of your account? Or have you decided on a maximum risk level of say, 3%? Keeping in mind that the unexpected will occur, you should probably have a maximum risk level for those times when you have several open trades.

5) What is the maximum drawdown (amount of money your trading system loses over an extended period of time) you are willing to tolerate? Keeping in mind that you (and you are not alone) are more likely to overestimate the severity of drawdowns that you can withstand, it is important to be realistic. If you lose 30% of your account will you stop trading? What about if you lose 50%? Or if you see 70% of your account disappear? Again, the best way to plan for drawdowns is to do extensive backtesting to find out what sort of historical drawdowns your trading system experiences and then plan for even worse drawdowns in the future.

Anticipating drastic changes in the markets is the single best way to preserve the equity in your account.

So, you know that successful traders share this habit – they backtest their trading strategies. You know that backtesting separates the wealthy traders from those who lose money. You also know several ways of incorporating backtesting into your trading regimen. And you know of the pitfalls – what to look out for – when you are backtesting, so that you can get the most out of the process. But, what exactly, will you get out of backtesting your trading system? In the next article I will explore the side effects of backtesting.

Walter Peters, PhD is a professional forex trader and money manager for a private forex fund. In addition, Walter is the co-founder of Fxjake, a resource for forex traders. Walter loves to hear from other traders, he can be reached by at walterfxjake.

Recent Articles on TradingMarkets.

Company Info.

The Connors Group, Inc.

10 Exchange Place, Suite 1800.

Jersey City, NJ 07302.

Company Resources.

Properties.

Connect with TradingMarkets.

© Copyright 2017 The Connors Group, Inc.

It should not be assumed that the methods, techniques, or indicators presented in these products will be profitable or that they will not result in losses. Past results of any individual trader or trading system published by Company are not indicative of future returns by that trader or system, and are not indicative of future returns which be realized by you. In addition, the indicators, strategies, columns, articles and all other features of Company's products (collectively, the "Information") are provided for informational and educational purposes only and should not be construed as investment advice. Examples presented on Company's website are for educational purposes only. Such set-ups are not solicitations of any order to buy or sell. Accordingly, you should not rely solely on the Information in making any investment. Rather, you should use the Information only as a starting point for doing additional independent research in order to allow you to form your own opinion regarding investments. You should always check with your licensed financial advisor and tax advisor to determine the suitability of any investment.

HYPOTHETICAL OR SIMULATED PERFORMANCE RESULTS HAVE CERTAIN INHERENT LIMITATIONS. UNLIKE AN ACTUAL PERFORMANCE RECORD, SIMULATED RESULTS DO NOT REPRESENT ACTUAL TRADING AND MAY NOT BE IMPACTED BY BROKERAGE AND OTHER SLIPPAGE FEES. ALSO, SINCE THE TRADES HAVE NOT ACTUALLY BEEN EXECUTED, THE RESULTS MAY HAVE UNDER - OR OVER-COMPENSATED FOR THE IMPACT, IF ANY, OF CERTAIN MARKET FACTORS, SUCH AS LACK OF LIQUIDITY. SIMULATED TRADING PROGRAMS IN GENERAL ARE ALSO SUBJECT TO THE FACT THAT THEY ARE DESIGNED WITH THE BENEFIT OF HINDSIGHT. NO REPRESENTATION IS BEING MADE THAT ANY ACCOUNT WILL OR IS LIKELY TO ACHIEVE PROFITS OR LOSSES SIMILAR TO THOSE SHOWN.