GPT4 Brings Moneyball Logic To Picking Stocks

(Bloomberg Opinion) -- University of Chicago researchers Alex Kim, Maximilian Muhn and Valeri V. Nikolaev fed OpenAI’s GPT4 large language model the anonymized financial statements of 15,401 public corporations from 1968 to 2021 and asked it to guess future earnings. What they found could eventually have profound implications for finance and the economy even if their immediate utility isn’t very impressive. 

The researchers found that GPT4 could predict with 52% accuracy whether next year’s earnings would be up or down — less than human analysts or statistical prediction methods. However, when they used a “chain of thought” technique that tells GPT4 to think like a human financial analyst, accuracy jumped to 60%, better than human analysts and about equal to statistical forecasts.

We have long known that if you ask experts what they do and tell computers to mimic the rules systematically, the computers beat the experts. Experts, for reasons of ego and professional authority, like to claim that they have finely honed intuitions about when to go against the simple rules, that their field is an art, not a science. But generally speaking, those intuitions are faulty, and you do better to stick to the simple rules. Experts do know stuff, but mostly simple stuff that can be easily encoded in simple rules.

We also know that it’s easy to beat human decision makers with simple rules. You don’t need state-of-the-art LLMs and five years of full financial statements to beat the market, just buy stocks with high book value to price ratios. You can predict defaults better than rating companies’ credit ratings with the Altman Z-score using five simple financial ratios — and that was developed over a half-century ago.

What’s impressive about the GPT4 model is not its performance, but how it was achieved.

This will be easier to see in the context of sports betting. Back in the 1960s, quants started to look into predicting sporting outcomes. Some of them tried to model individual contests in detail, while others focused on finding bets that had been mispriced by bookmakers. The first group looked at player statistics and past games to simulate likely outcomes of future contests. The second group looked for rules like, “bet against the Lakers at home,” because the Lakers were a glamorous basketball team and Los Angeles was a high betting city, so bookies would set the point spread in a manner that gave a positive expectation to bets against the team.

I’m going to name the first approach after Bill James. Although he was not the first to apply it, he became the most famous and is a kind of patron saint of sports analytics. I’ll call the second the Ed Thorp approach. Although Ed only tried some brief forays into sports betting, he is the best-known practitioner in finding mathematical edges in casinos and markets. GPT4 is Bill James, conventional quantitative hedge fund investing is Ed Thorp.

The Bill James crowd was not particularly successful in sports betting. One reason is it’s much harder than the Ed Thorp approach. But a bigger reason is bookmakers have no interest in consistently writing checks to winning bettors. It wasn’t hard to predict sporting outcomes with enough accuracy to make a profit; it was hard to find bookmakers who would take the bets and pay off when you won.

But bookmakers welcomed the Ed Thorp types betting against the Lakers at home, and happily paid them off. Suppose the fair spread was -2.5 on the Lakers, meaning there was a 50% chance the Lakers would win by three or more points. If Laker fans bet $100 million on the team, the bookmaker might only attract $30 million the other way with a -2.5 spread. If it set the spread at -4.5 it might get $50 million, in which case bets on the Lakers would have about a 40% chance of winning.

Setting the spread even higher wouldn’t attract a lot more money, there just wasn’t that much spread-sensitive sports-betting money around. Moreover, at the time, sports gambling was an organized crime monopoly. It regarded all $100 million Los Angeles bettors were putting up as its money, and it expected all retail bettors to eventually lose everything they bet. But they wanted to take it slowly, 5% per game. If they took it faster, bettors would find betting too unprofitable, and the golden goose would have been killed.

Las Vegas bookies welcomed quant bettors willing to put $10 million against the Lakers. Even though the bet had negative edge for the bookmakers, it lowered their risk. The mob wanted to have balanced books, $100 million on each side, so it made $10 million whatever happened (standard vigorish was to charge losers a 10% premium). That was zero risk and kept bettors happy. With unbalanced books, it made money on average over time but had considerable risk on individual games.

The Ed Thorp approach had far more practical use, but it did nothing to change the world. The Bill James approach, by contrast, slowly but surely changed the way sports were played. We see the effect of analytics in all sports — hockey teams pulling goalies earlier, basketball teams attempting more threes, baseball teams putting their best hitters first or second in the line up rather than third to fifth, football teams going for more fourth down plays and two-point conversions.

Similarly, six decades of quantitative trading has not eroded the anomalies quants noticed back in the 1960s. It has changed financial markets, but not business practice. Hedge fund geniuses have not been conspicuously successful in advising or running businesses. Private equity firms do not use much of the techniques favored by quant traders. Statistical patterns in observed market prices cannot be reverse-engineered to be much help in making actual business decisions.

GPT4 is different because it uses general knowledge to predict business outcomes. The research paper used a highly restricted set of data — five years of standardized financial statements with year and company redacted. More sophisticated models drawing on much broader datasets should do much better — and artificial intelligence research is progressing rapidly.

The research paper claimed a 10% annual alpha (return in excess of exposure to the market and major known factors) for buying the 10% of companies with the highest estimated probability of an earnings increase and shorting the 10% with the lowest estimated probabilities. While a realized alpha of 10% in actual trading would be phenomenal, finding this level in simulated backtests without transaction costs is not uncommon. The results could no doubt have practical use as part of professional hedge fund strategies.

What makes this 10% potentially earth-shaking is that it results from understanding financial numbers — such as the relation between revenue growth and profitability, or net income and balance sheet leverage — not just noticing patterns. It’s like a Bill James-style analysis about how a sport should be played, versus an Ed Thorp statistical pattern of how bookies misprice bets. It’s versus . A 10%-per-year increase in real gross domestic product growth is a far more compelling promise than 10% alpha in a hedge fund, making money lost by other investors.

Of course, this is just one working paper, not the magic potion to grow economies. It took many decades for sports analytics to have much effect on play, and there’s no reason to think economic policymakers and business executives are less conservative than sports coaches and managers. I don’t expect to see AI Federal Reserve chairs or Fortune 500 CEOs anytime soon. But this paper might be that rare one swallow that does make a summer.More From Bloomberg Opinion:

• Trust Ackman to Build $250 Billion Fund Manager?: Aaron Brown
• AI Will Transform Sex Work But Not Intimacy: Allison Schrager
• Dream of Replacing Humans in Finance May Come True: Aaron Brown

This column does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners.

Aaron Brown is a former head of financial market research at AQR Capital Management. He is also an active crypto investor, and has venture capital investments and advisory ties with crypto firms.

More stories like this are available on bloomberg.com/opinion

©2024 Bloomberg L.P.