Download as PDF, PPTX
Uploaded byQuantopian
Beware of Low Frequency Data by Ernie Chan, Managing Member, QTS Capital Management, LLC.
The document discusses the importance of data integrity in backtesting trading strategies, highlighting that poor data can lead to inflated backtest performance. It provides examples of trading strategies, including mean reversion and intraday momentum, underscoring the difference in performance outcomes when using various price data sources. The conclusion stresses that the type of historical data required depends on both execution price determination and trading signal triggering.
In this document
Powered by AI
Introduction to Dr. Ernest Chan and his background in algorithmic trading and machine learning.
GIGO principle; emphasizes data integrity's crucial role in backtesting trading strategies.
Mean reversion trading strategy for Closed-End Funds (CEF); includes results from backtests.
Charts displaying cumulative returns and performance metrics of CEF Premium Reversion strategy.
Differing results from using closing prices vs. midpoints; need for accurate execution data.
Strategy based on rank of returns from previous close to today's open; backtest and live performance.
Intraday momentum strategy applied to futures; impact of data granularity on strategy performance.
Mean-reversion pair trading strategy using ETF pairs; issues with intraday live trading performance.
Strategies require high frequency historical data for accurate backtesting and execution.
Thank you note and resources for further information on trading strategies.
More Related Content
Similar to Beware of Low Frequency Data by Ernie Chan, Managing Member, QTS Capital Management, LLC.
![[Data Meetup] Data Science in Finance - Building a Quant ML pipeline](https://cdn.slidesharecdn.com/ss_thumbnails/buildingaquantmlpipeline-191009091209-thumbnail.jpg?width=640&height=640&fit=bounds)
Beware of Low Frequency Data by Ernie Chan, Managing Member, QTS Capital Management, LLC.
- 1. Beware of Low Frequency Data Ernest Chan, Ph.D. QTS Capital Management, LLC.
- 2. • Previously, researcher at IBM T. J. Watson Lab in machine learning, researcher/trader for Morgan Stanley, Credit Suisse, and various hedge funds. • Principal of QTS Capital Management, a commodity pool operator and trading advisor. • Author: – Quan%ta%ve Trading: How to Build Your Own Algorithmic Trading Business (Wiley 2009). – Algorithmic Trading: Winning Strategies and Their Ra%onale (Wiley 2013). • Blogger: epchan.blogspot.com About Me 2
- 3. GIGO • Garbage in, garbage out is well-‐known to programmers. • Data integrity is crucial to backtesVng trading strategies. – Common problem: Historical prices backtested weren’t the actual prices we could execute at. – Typical outcome: backtest performance is greatly inflated compared to realisVc historical performance.
- 4. Example 1: CEF Premium Reversion • Patro et al published a paper on trading the mean reversion of closed-‐end funds’ (CEF) premium. – ssrn.com/abstract=2468061 • CEFs with high premium (market cap-‐NAV) will have negaVve returns, while those with steep discount will have posiVve returns. • Rank CEFs based on % premium and buy the bobom quinVle and short the top quinVle with monthly rebalancing.
- 5. Example 1: CEF Premium Reversion • Authors obtained fund price and shares outstanding data from CRSP, and fund NAV data from Bloomberg. • Sharpe raVo is 1.5 from 1998-‐2011. • I repeated their backtest also using CRSP prices, and fund NAV from Computstat from 2007-‐2014.
- 6. CEF Premium Reversion: closes 2008/01 2010/01 2012/01 2014/01 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Date CumulativeReturns
- 7. CEF Premium Reversion: midpoints 2008/01 2010/01 2012/01 2014/01 -0.25 -0.2 -0.15 -0.1 -0.05 0 Date CumulativeReturns
- 8. Midpoints vs closes • The dramaVc differences in performance due to using closing prices vs midpoint between bid and ask prices at the close. – You wouldn’t think bid and ask prices maber for strategies that rebalance only monthly! • Actual execuVons will use MOC (Market-‐on-‐close) or LOC (Limit-‐on-‐close) orders. • Actual execuVon prices will be the close price (“closing cross”) at primary exchanges where aucVons take place, not consolidated closing prices which most backtests use. – Rf. Prof. Joel Hasbrouck “SecuriVes Trading” NYU Teaching Notes
- 9. Consolidated closes • Consolidated closing price represents the last execuVon price from any one of > 50 market centers at which a stock, ETF, or CEF can be executed. • ExecuVon can take place in a dark pool, ECN, or the primary exchange. • If we send a LMT/MKT order, no guarantee it will be routed to that parVcular market center and filled at the consolidated closing price.
- 10. Primary closes • Where can we get historical primary exchange (“aucVon”, “official”, “crossing”) close prices? – Buy from the primary exchanges. – Subscribe to Bloomberg. – EsVmate using midpoints from CRSP. • This is what I did. – Use Vck data and select the trades with the Cross flag*. *Hat-‐Vp: Chris at QuantGo.com
- 11. Example 2: Opening gap • Rank stocks based on their returns from previous close to today’s open: retGap. • Apply fundamental and technical filters e.g. eliminaVng stocks which just had earnings announcements. – See my book “Algorithmic Trading”. • Buy 10 stocks with the lowest retGap, and short 10 with the highest retGap at the open. • Exit at the same day’s close. • Backtest from 2012-‐2014. • Live trading from mid 2013-‐2014.
- 12. Opening Gap: Backtest vs Live 2012/01 2013/01 2014/01 -0.05 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 Date CumulativeReturns Backtest with 5 bps cost Live
- 13. What happens at the open? • Backtest has already used midpoints at close: very near the closing crosses. • Backtest also included 5 bps per trade transacVon cost. • Live trading sVll underperformed backtest substanVally. • Causes: – Open prices also need to use aucVon prices. • Unfortunately CRSP does not provide bid/ask at open. – Need quotes at 9:28 (Nasdaq deadline for LOO/MOO orders) to generate trading signals.
- 14. Example 3: Futures momentum • Intraday momentum strategy applied to various futures (E.g. RB or GC). • Rank all trades (or quotes) in previous day’s trading session. – Long if last price above 95th percenVle. • Exit long if last price below 60th percenVle. – Short if last price below 5th percenVle. • Exit short if last price above 40th percenVle.
- 15. Futures momentum • Compare backtests based on – 1-‐minute trades bars from eSignal, back-‐adjusted conVnuous contracts. – BBO quotes with 1-‐millisecond Vmestamps from QuantGo.com / Algoseek data, actual contracts. • 1-‐min data shows that strategy trades only 1 round-‐trip a day: low frequency!
- 16. Futures momentum • In all cases, 1-‐ms data produce much worse returns than 1-‐min data. • 1-‐ms data shows that strategy someVmes flip-‐ flops: rapid changes of last prices cause rapid succession of (losing) trades.
- 17. Example 4: Pair trading ETFs • E.g. ETFs EWA (Australian stock index) and EWC (Canadian stock index) are good candidates for mean-‐reversion pair trading. • Bollinger band strategy applied to spread. • Backtest on daily closes (aucVon or consolidated prices): good results. • Why not live trade intraday, using Bollinger bands to set limit prices? – Expect more trading opportuniVes and more profits!
- 18. Pair trading ETFs • Reality: Intraday live trading using InteracVve Brokers live Vck feed (250ms bars) osen suffers mysterious losses due to mysterious trades. • Culprit: Flip-‐flopping due to order book “mini-‐ flash crashes” – Small change in price on one leg leads to large % error in spread! • These flip-‐flopping and losses disappear if we use Yahoo RealTime (1s bars).
- 19. Pair trading ETFs • Moral of story: if you want to trade intraday, must use Vck data for backtest, even if holding period is long (e.g. hours). • What if we restrict live data to 1-‐sec or longer bars? – This would be arVficial and nonsensical: why should we only trade at … 10:01, 10:02, 10:03, … instead of … 10:01:01, 10:01:02, 10:01:03, …?
- 20. LF backtest requires HF historical data • CEF monthly rebalancing → need Vck data to find closing crosses (aucVon) prices (unless you have Bloomberg). • Opening gap stocks strategy → need Vck data to find NBBO at 9:28 am and opening crosses. • Intraday low-‐frequency futures momentum strategy → need Vck data to check for intra-‐1-‐ min-‐bar flip-‐flopping/mini-‐flash crashes. • Intraday low-‐frequency ETF mean reversion pair trading → need Vck data to check for intra-‐1-‐sec-‐ bar flip-‐flopping/mini-‐flash crashes.
- 21. Conclusion • Whether a trading strategy requires low or high frequency historical data depends not only on holding period, but also on: – How execuVon prices are determined. – How trading signals are triggered.
- 22. Thank you for your Vme! www.epchan.com Twiber: @chanep Blog: epchan.blogspot.com