Saturday, September 23, 2023
007 - Lessons from Artificial Intelligence (Part V)
In the fifth and penultimate episode of this sub-series, I discuss the 2013 Flash Crash, sentiment analysis, my inroads into business analysis, and the phenomenon of neural overfitting.
Duration: 00:06:10
Episode Transcript
Intro
Good afternoon. You’re listening to the Reflections in Beige Podcast, hosted by Michael LeSane. The date is Saturday, September twenty-third, 2023.
I hope that everyone’s enjoying their pumpkin spice lattes on this beautiful first day of autumn. Let’s begin.
2013 Twitter Flash Crash
On Tuesday, April 23rd, 2013, I was running a routine neural algorithmic trading simulation and periodically checking in on the health of the system and the performance of its trading. As usual. What was not so usual, however, was how things would play out that day in the market.
During one of my routine check-ins, I noticed that the simulation had perplexingly sold everything at the same time at a loss. How could this be? Checking the chart for the stock market index that day, I noticed that there had been a sharp crash, followed a total recovery shortly afterward. This was what is known as a flash crash.
Looking into it further, I learned that the Associated Press’s Twitter account had been hacked, and a tweet had been sent out falsely claiming that there had been an attack on the White House in which the president had been hurt.
Whether through automated sentiment analysis of social media or manual discretion based on human sentiment, this single tweet caused the nation’s high frequency trading systems and human traders to panic sell everything in unison, wiping out 140 billion dollars in value. My system, of course, reacted to this trend by following suit.
This, I suppose, was the price ultimately paid for following the herd of market sentiment, the herd itself following a corrupted source of truth.
Sentiment Analysis and Algorithmic Trading
And yet, this demonstrates how automated sentiment analysis – that is to say, anticipating investor sentiment and thus behavior by processing natural language and other data – can potentially be a powerful tool for preempting the herd.
Using a form of sentiment analysis, I began to explore ways to anticipate market volatility and other trends later in the year, which I could use to amplify or hamper the aggressiveness of trading activity encouraged during training.
The gameplan was to pull all news headlines and financial articles related to a stock and the sector it was a part of, crunch the news word by word, and then correlate it all with various metrics at the end of the day in order to create additional inputs to the neural network.
I never quite got around to working this into the simulation though, due to time demands as I began to transition into the industry after leaving academia. If you happened to give it a try, let me know how it went.
Business Analysis
Digging deeper into what drives the valuation of individual stocks and indeed principles guiding picking them, I’d come to learn about value investing: investing in securities that are underpriced relative to their intrinsic value, especially relative to similar assets, based on metrics like earnings relative to market capitalization, cash reserves, historical performance, and debt levels.
Even after I drifted away from the algorithmic trading project, I continued to independently study business analysis and administration, hoping to garner a deeper perspective into how to vet the health of operations. This would prove valuable to my career in the years that followed in the most unexpected ways, and I would encourage everyone with some time available to do so.
The most general and enduring lesson from this saga, however, is that when in doubt, buy and hold index funds. It’s the safest way to go – tried and true.
Overfitting
I’m going to cut this episode a little bit short, since the subject matter for the final segment of this series likely calls for its own dedicated episode. But before I conclude, I’d like to go on a tangent and take a little bit of time to discuss the concept of overfitting.
The Oxford English Dictionary defines overfitting as, quote:
End quote.
Carnegie Mellon University’s Machine Learning Department defines overfitting as when the empirical training risk of a model is relatively small compared to the true test risk.
Simply stated, you could think of it as when a neural model is so thoroughly exposed to past examples or patterns that it performs poorly when learning from or responding to new examples or patterns, especially those it may encounter in the wild, so to speak.
One way this can be avoided in training is by using a larger neural network. It can also be helpful to provide a neural network with a more diverse set of training data, which for lack of better words can offer a more nuanced grasp of the intricacies and possibilities associated with the data.
The state of deep learning has evolved a great deal over the years and I’m not up to date with the state of applied research, but as an academic working with dynamic neural networks at the time, I hypothesized that training with neutral inputs and neutral outputs could help reduce the intensity of of overfitting in neural networks of appropriate size, or at least the number of iterations required to reverse it. I saw this process as analogous to what meditation does for the human mind.
This is just my opinion, but I believe the human mind is just at risk of overfitting as statistical or neural models, but as it relates to lived experience, and with far greater capacity and dynamic potential, and thus that it can helped.
So in closing, try something new. Read a book. Leave your house. Travel if you can. Expose yourself to something you’ve never seen before, or do something you’ve never done before. Go outside your comfort zone.
Outro
Let that be today’s lesson.
Thank you for listening and sharing; live, laugh, love; and have a good evening.
Good afternoon. You’re listening to the Reflections in Beige Podcast, hosted by Michael LeSane. The date is Saturday, September twenty-third, 2023.
I hope that everyone’s enjoying their pumpkin spice lattes on this beautiful first day of autumn. Let’s begin.
2013 Twitter Flash Crash
On Tuesday, April 23rd, 2013, I was running a routine neural algorithmic trading simulation and periodically checking in on the health of the system and the performance of its trading. As usual. What was not so usual, however, was how things would play out that day in the market.
During one of my routine check-ins, I noticed that the simulation had perplexingly sold everything at the same time at a loss. How could this be? Checking the chart for the stock market index that day, I noticed that there had been a sharp crash, followed a total recovery shortly afterward. This was what is known as a flash crash.
Looking into it further, I learned that the Associated Press’s Twitter account had been hacked, and a tweet had been sent out falsely claiming that there had been an attack on the White House in which the president had been hurt.
Whether through automated sentiment analysis of social media or manual discretion based on human sentiment, this single tweet caused the nation’s high frequency trading systems and human traders to panic sell everything in unison, wiping out 140 billion dollars in value. My system, of course, reacted to this trend by following suit.
This, I suppose, was the price ultimately paid for following the herd of market sentiment, the herd itself following a corrupted source of truth.
Sentiment Analysis and Algorithmic Trading
And yet, this demonstrates how automated sentiment analysis – that is to say, anticipating investor sentiment and thus behavior by processing natural language and other data – can potentially be a powerful tool for preempting the herd.
Using a form of sentiment analysis, I began to explore ways to anticipate market volatility and other trends later in the year, which I could use to amplify or hamper the aggressiveness of trading activity encouraged during training.
The gameplan was to pull all news headlines and financial articles related to a stock and the sector it was a part of, crunch the news word by word, and then correlate it all with various metrics at the end of the day in order to create additional inputs to the neural network.
I never quite got around to working this into the simulation though, due to time demands as I began to transition into the industry after leaving academia. If you happened to give it a try, let me know how it went.
Business Analysis
Digging deeper into what drives the valuation of individual stocks and indeed principles guiding picking them, I’d come to learn about value investing: investing in securities that are underpriced relative to their intrinsic value, especially relative to similar assets, based on metrics like earnings relative to market capitalization, cash reserves, historical performance, and debt levels.
Even after I drifted away from the algorithmic trading project, I continued to independently study business analysis and administration, hoping to garner a deeper perspective into how to vet the health of operations. This would prove valuable to my career in the years that followed in the most unexpected ways, and I would encourage everyone with some time available to do so.
The most general and enduring lesson from this saga, however, is that when in doubt, buy and hold index funds. It’s the safest way to go – tried and true.
Overfitting
I’m going to cut this episode a little bit short, since the subject matter for the final segment of this series likely calls for its own dedicated episode. But before I conclude, I’d like to go on a tangent and take a little bit of time to discuss the concept of overfitting.
The Oxford English Dictionary defines overfitting as, quote:
The production of an analysis which corresponds too closely or exactly to a particular set of data, and may therefore fail to fit additional data or predict future observations reliably.
End quote.
Carnegie Mellon University’s Machine Learning Department defines overfitting as when the empirical training risk of a model is relatively small compared to the true test risk.
Simply stated, you could think of it as when a neural model is so thoroughly exposed to past examples or patterns that it performs poorly when learning from or responding to new examples or patterns, especially those it may encounter in the wild, so to speak.
One way this can be avoided in training is by using a larger neural network. It can also be helpful to provide a neural network with a more diverse set of training data, which for lack of better words can offer a more nuanced grasp of the intricacies and possibilities associated with the data.
The state of deep learning has evolved a great deal over the years and I’m not up to date with the state of applied research, but as an academic working with dynamic neural networks at the time, I hypothesized that training with neutral inputs and neutral outputs could help reduce the intensity of of overfitting in neural networks of appropriate size, or at least the number of iterations required to reverse it. I saw this process as analogous to what meditation does for the human mind.
This is just my opinion, but I believe the human mind is just at risk of overfitting as statistical or neural models, but as it relates to lived experience, and with far greater capacity and dynamic potential, and thus that it can helped.
So in closing, try something new. Read a book. Leave your house. Travel if you can. Expose yourself to something you’ve never seen before, or do something you’ve never done before. Go outside your comfort zone.
Outro
Let that be today’s lesson.
Thank you for listening and sharing; live, laugh, love; and have a good evening.