As an author, hedge fund manager and educator Dr. Ernest P. Chan has been an expert in the quantitative trading world for over a decade. His focus has been on the development of statistical models and advanced computer algorithms to find patterns and trends in large quantities of data. He is the author of “Quantitative Trading: How to Build Your Own Algorithmic Trading Business” and “Algorithmic Trading: Winning Strategies and Their Rationale“, both published by Wiley. His new book “Machine Trading: Deploying Computer Algorithms to Conquer the Markets” was published this year. Ernie is also the Managing Member of QTS Capital Management, LLC., a commodity pool operator and trading advisor.
Ernie discussed a variety of topics with us including how to avoid overfitting, advice for those looking to enter the quantitative trading industry, and limitations of machine learning. Be sure to check out his twitter account at @chanep and his popular financial blog at http://epchan.blogspot.com. You can also find more information on his firm, QTS Capital Management at http://www.qtscm.com/
Kurt: Take us back to how you first got into trading and what that experience was like.
Ernie: When I first got into trading it was very much a research role so I was just doing analysis programs, building models, and running various tests to find arbitrage opportunities. We would then forward it to the trading desk at Morgan Stanley. At that point, I was a step removed from the actual action. After six months we found ourselves positioned on the trading desk in a different firm at Credit Suisse. All of the sudden we have P/L responsibilities on us and that was more of stressful situation. I had two colleagues that ran the proprietary trading group at the time and part of the stress was that it was the first time I was faced with helping to manage a large portfolio for a major international bank. Secondly, some of our strategies did not quite work as of that date. In the investment banking environment people have very short patience and most were used to a sell side not a buy side proprietary group. There was a tremendous amount of pressure from the higher ups to produce profits. After that I went to a number of hedge funds that had a similar environment except for Millennium Partners which I found the environment to be quite conducive to research. Other places were not ideal in my mind for research and idea generation and it was not until I started my own firm did I find a suitable environment that I can actually generate profit.
Kurt: What were some of the big hurdles that you had to overcome when first getting into machine learning with algorithmic trading?
Ernie: A major hurdle is, of course, the propensity to overfit using a machine trading model. Overfitting and also the fact that there are regime shifts that machine learning oftentimes cannot handle. There’s also the matter of feature selection, meaning what features you feed into the machine learning model. That is a manual thing. You don’t have to program the algorithm, you can get it free from any number of sources. Those algorithms are not going to help you to produce predictions unless you enter the right features. How to collect the right features and how to make sure that the data is clean and so forth is all quite manual. Machine learning only solves one part of the process. The big part of the problem, which is feature selection and preparation, is still a completely manual process. Machine learning doesn’t do anything to help with that very important part of the problem.
Algorithms are not going to help you to produce predictions unless you enter the right features.
Kurt: Kind of taking it back a little, you said one of the main issues would be overfitting. How do you go about avoiding that?
Ernie: There are many standard techniques for avoiding overfitting. There is cross validation, there is the notion of random forest, there is oversampling the original data with replacements. There’s also a way of simulating data that’s just based on a model of the data and how it should behave to train the model in order to generate a much bigger training dataset. In deep learning there is a technique that was invented by Dr. Hinton where you are essentially randomly removing certain nodes in the deep learning network so that the neighboring nodes cannot rely on the neighbors to generate a prediction. It has to really be able to predict on its own pretty well. The random leaving out of nodes is a famous technique and a major improvement in deep learning and there are many of these methods. When they are applied to finance they become all the more important but oftentimes even this technique is not sufficient. This is because not only is financial data subject to overfitting because of the limited amount of it but also because it has regime shift, so that is something that machine learning simply cannot handle. I’m not saying that any other model will work, maybe a discretionary trader will be able to adapt better in that case actually if they understand the market intuitively better. They may be able to sense that the current market condition is different from last year and different than five years ago. Where the machine is at this point in time it is unable to generate these high level insights. Now maybe in the future when you have another twenty years of advancements in machine learning they are able to form high level concepts like a trader can. Maybe at that point, we would be able to overcome the regime shift problem in machine learning.
Kurt: So it sounds like an advantage that humans have over a system could be predicting regime changes. Would you say there are any other advantages that humans have over a system?
Ernie: I think the human advantage is that the human is bombarded with extremely high dimensional data every day. From the economic arena, political arena, financial arena and so on and so forth and some humans are able to pick out what’s important very quickly and discard the noise. A machine can’t do that, oftentimes it doesn’t have the background knowledge to understand what’s important and what’s not. You have to let it learn explicitly that, for example, a political election may not be important to the crude oil market but other times it may be important. Expert knowledge oftentimes can substitute for a lot of machines. So yes, a machine can learn these things in theory, but in practice it doesn’t have a contextual understanding. A human can pick out the important features much faster among the sea of data than a machine and that’s where the discretionary trader has an edge.
Grace: How would you define the success of a trading system and do you take into consideration the time and effort spent to create it?
Ernie: A trading system is only really successful when it produces real profits. That takes a while to figure out. Research and testing is never enough. The only time you can call it a success is if you have traded it for six months and it produces a consistent profit. Now the timeframe is also strategy dependent so if you have a high frequency trading system that trades every second after two weeks you can judge success if it’s profitable. It should be profitable every day if it is a high frequency trading system. On the other hand, if it is a traditional factor model which rebalances monthly, you may not be able to tell whether it’s a success or failure until about one year. So the more frequent the trade, the shorter the time it takes to judge whether a system is successful. Typically, it’s a matter of statistical significance. If a system has generated one hundred trades and you see a consistently high winning ratio, you can consider it a success so one hundred is our minimum threshold for determining success.
Kurt: Can you take us through the process of creating and validating a trading system and how you do it?
Ernie: To create a system, of course, one starts with some intuition of where the inefficiency or arbitrage opportunity is. Given that, you build a trading strategy to exploit it and then backtest it with historical data. After we determined that the backtest is superior the person who initially backtested the system should always turn over the strategy to a colleague who will backtest it using a completely different system. Let’s say one is backtesting using Matlab then the colleague should backtest it using Python or C# or Java or whatnot so as to confirm that the backtest does not have bugs. If the backtest is successful, meaning that it has a Sharpe ratio of greater than two, then one should turn this into a paper trading system to see if it has similar results out of sample. The only real out of sample test is paper trading. You cannot really leave sample out in your old data set because one can use that out of sample data to improve your model. Paper trading is really the only way to determine if a model actually works and as I said it depends on how frequent the trading is. The more frequent the trades the less time it takes to paper trade. After we have obtained one hundred trades in the paper trading and we determine that it is suitable then we should turn it live with some amount of capital. We continue with that level of capital until we collect more statistical significance that this is working and gradually increase leverage as consistency is being demonstrated.
Kurt: Do you have any recommendations for where traders should get their data from?
Ernie: One supermarket for data for retail traders is Quandl.com. The founder of Quandl also has a blog on that website which talks a lot about exotic data and I find it very informative. Quandl is usually the first place I would look when I am reaching for data but they didn’t use to have a lot of nontraditional data. They do have data to cover a wide range of instruments and typically their data is not that expensive because it’s for retail traders. For nontraditional data one has to go to the specialty data provider one at a time. Typically Thomson Reuters will have a lot of that kind of data as they are in the business of selling it but there are also other specialty vendors too so there is no one place for specialty data.
I have written three books for this particular purpose.
Kurt: Are there any resources or books that you would recommend for people looking to get into quantitative trading?
Ernie: I have written three books for this particular purpose. Also on my blog I have referenced a number of books on my recommended book list and they are books written by academics such as Dr. Andrew Ang. He used to be an esteemed finance professor at Columbia but now he is working for BlackRock. His book is very very thick but very understandable with a very small amount of math in it. Of course there are books especially for options traders; an endless number of them. For example, one of them is written by Euan Sinclair who is one of my favorite authors in the options trading world. In terms of more recent knowledge there is always Twitter. I monitor the Twitter feed of a number of places. Some of them are article aggregators, some of them are people who just like to tweet about articles they read. I find the most efficient way to keep tabs on new research is to follow the Twitter feed of different reliable sources and read their articles. I make sure that I read almost every paper that I make note of. Typically every week I come across an article that was mentioned on Twitter and I will make note of that and I will always get to it at some point or another. So that’s how I keep up to date with information and research.
The risking of money is the only way to force your brain to focus on the important money making opportunity rather than theoretical opportunity.
Kurt: What advice do you have for traders who are looking to start developing a system and really get into quantitative trading?
Ernie: I would say the advice is to, on one hand, read a lot, specifically read what other traders have done. Read books and articles. And on the other hand risk your own money. It seems strange to say but I find that the risking of money is the only way to force your brain to focus on the important money making opportunity rather than theoretical opportunity. So really getting your hands dirty and risking your own money is the best way to learn trading.