Hedge Funds Battle To Turn ChatGPT From Intern To Analyst

(Bloomberg) -- Chris Pulman used to spend two days prepping previews for central bank meetings. Now it can take the chief economist for macro at Balyasny Asset Management as little as 30 minutes.

Thanks to the new amped-up generation of artificial intelligence, chatbots are now carrying out his time-consuming research chores. Everything from summing up the views of Wall Street economists and generating charts to extracting the latest pronouncements from monetary officials, and more. With Pulman’s input, the AI program then plugs all that market wisdom into a template to showcase his interest-rate call. 

“We found they’re actually substantially more powerful than you think at first,” said the Balyasny economist, referring to large language models. “But they don’t work straight out of the box.”

More than 20 months after OpenAI Inc. unveiled ChatGPT to great fanfare, hedge fund managers, including Two Sigma Investments and Man Group, are racing to exploit the disruptive potential of the technology by integrating chatbots in their day-to-day research and investing processes. Banks are also leveraging the tools, with JPMorgan Chase & Co. launching its own ChatGPT last month to employees in asset and wealth management while Goldman Sachs Group Inc. is building its own platform. 

For these early adopters, who are long in the business of deploying new technologies for an investing edge, chatbots can conduct the thankless tasks like any eager-to-please intern, including filtering through regulatory filings, summarizing research and writing basic code.

But a fully fledged analyst in chatbot form that can spit out savvy investment ideas, granular research and reliable predictions? That remains ways off, just as Wall Street frets that the new tech will struggle to justify this year’s frenetic stock rally. At its peak, the Nasdaq 100 was up more than 90% from its 2022 trough before retreating in recent weeks. 

Undaunted, fast-money proponents remain steadfast in their conviction that their investments will reap tangible wins now that there’s greater understanding about the practical limitations of chatbots straight out of the box.

With more time freed up, Pulman, for one, reckons he can take gen AI to a more advanced level, using it to come up with sophisticated code and economic forecasts on its own. He says it’s “plausible” AI could handle 70% to 80% of what a country economist in finance does within two to three years. 

To get there, the industry has to confront a couple of big issues. They include the fact generative AI can simply make things up, throwing up a phantom research paper here or an erroneous market factoid there. It also struggles with abstract or multi-layered questions — without intensive coaching from a human overlord.

In one instance, a Balyasny portfolio manager wanted to see if its chatbot could figure out the stock winners or losers from higher tariffs — a reasonable question that it couldn’t answer at the get-go. Engineers had to first train the model by breaking such scenarios down to a series of sub-questions. It took 99 minutes scanning 20,000 documents and going step-by-step before formulating a satisfactory answer. 

“We’re relying on the capabilities of a junior intern: you ask AI to do some simple analysis with internal data sources, it will do it, but either you have to give lots of very specific prompting or the analysis itself is quite rudimentary,” said Charlie Flanagan, head of applied AI at Balyasny, which runs about $22 billion. “So how do we move it from a junior intern to a senior intern to a junior analyst so that at the end of 2024, folks are able to ask quite robust questions?”

None of this comes cheap. Goldman Sachs estimated that building out AI infrastructure across the economy will cost more than $1 trillion over the next few years. Balyasny has a 12-person team for AI while Man is about to have six people devoted to gen AI specifically. Fully trained systems like ChatGPT or Anthropic’s Claude charge for every incremental use, while building one up from open-source models like Meta’s Llama requires a big splurge on talent and computing power. 

A robot that can interpret text for trading is nothing new on Wall Street. For years, computers have been scanning news articles and earnings transcripts for their market implications. But the very appeal of ChatGPT is that it takes all this up a notch: parsing context, answering questions coherently and drawing on a litany of sources to come up with sophisticated conclusions. 

“We are beyond the point of being impressed by their native capabilities,” said Tim Mace, head of data and machine learning at Man Group. “It has to be as good as or better than what a human could potentially do.”

At the $178 billion manager in London, the low-hanging fruit has been using AI to make humans more productive, by generating price charts or extracting information from bond prospectuses. For now, the firm thinks it’s too soon to plug LLMs directly into systematic trading models where there is less direct human oversight, Mace adds. 

Yet this realism belies grand ambitions, with Man talking up the possibility that AI will one day be able to search its research database, generate a hypothesis and create a code to test it. Or, that it will be able to spot subtle economic relationships from the massive amount of data it has ingested, informing trades that, say, buy one security and sell another.  

And even when the AI currently lags humans on cognitive prowess, it has the advantage of speed and scale, says Ben Wellington, deputy head of feature forecasting at Two Sigma. He points to the example of tracking corporate executive departures. While quants like him used to write a formula to identify the latter via particular keywords or expressions, he can now do that much faster by querying an LLM. 

“If I had a list of 50 ideas, maybe before I could study 10,” he said. “I can now go from 10 to 25 with relatively low cost because I don’t need to build a team or tool to study each idea.”

A lot of successful use-cases require much more than an off-the-shelf GPT. Balyasny’s Flanagan showed an example where its GPT read an academic paper about a trading strategy and computed how it would have performed historically. To do that, the model was, in fact, using a calculator coded by his team rather than the chatbot alone. 

Gen AI will “tell you a totally made-up story and be absolutely certain,” so human judgment will remain the ultimate gatekeeper, Claudia Perlich, head of strategic data science for investment management at Two Sigma, said at the Bloomberg Invest conference in June. 

To avoid the risk of fabricating facts, many firms use a technique called retrieval-augmented generation, where the AI is made to look through particular additional sources. Phrasing prompts well also makes a big difference. 

“It’s fascinating to watch it work because you can actually see it go, I’ve gotten this information, this is the next step I should take,” said Flanagan, a former Google engineer. “But we’re still forcing the model to behave like that.”

For some money managers, the bet is investing early puts them ahead of rivals if future breakthroughs manage to bring LLMs closer to human intelligence, as the likes of OpenAI are envisioning. 

That’s nothing certain yet. At Atalaya Capital Management, a roughly $10 billion private-credit firm, gen AI has greatly accelerated the process of searching for potential borrowers in its equipment-leasing business and drafting legal contracts. Yet humans are still firmly in charge of picking investments and negotiating terms, says head of data science Andy Halleran.

So while it may be the most hard-working intern on Wall Street, the bar for promotion to a full-blown analyst is high.

“They’re still not at the level where you can just give them a super broad task, so you can’t say: ‘Hey, is this a good investment?’” he said, referring to ChatGPT prompts. “If someone already has a workflow and you want them to do the new thing, it has to be not just like a little bit better, a little bit more convenient — it has to be dramatically better.” 

--With assistance from Sam Potter.

(Adds stocks update in sixth paragraph)

More stories like this are available on bloomberg.com

©2024 Bloomberg L.P.