Artificial Intelligence (AI) in Portfolio Management is a must, and no longer a 'nice-to-have'

By Irene Aldridge

This year has been a reasonable year for investors: the U.S. markets dealt well with inflation and the feared recession has not materialized, at least, not yet. Most investors and portfolio managers, however, have been on high alert from the innovations in Artificial Intelligence (AI) that have been broadly sweeping all industries. From efficient art copy ideas to textual outlines, the AI has already enabled millions to lead more productive lives. What about portfolio management? What kind of inferences can AI enable us to have there?

According to Bhaskar Krishnamachari, Professor of Electrical Engineering at University of Southern California, true AI is computer systems configured to perform traditionally human tasks, such as:

Perceiving and understanding the world around us (vision, speech, object recognition)

Communicating in natural language

Reasoning and logical deductions

Making plans and decisions and navigating the world in autonomous fashion

Coordinating actions with other machines and humans

In the context of portfolio management, we are most interested in #3, “reasoning and logical deductions” as applied to the science of portfolio management. Of course, #3 cannot work in isolation from #1: machines need to “understand” and incorporate the available data to make sound portfolio inferences.

And the data available for portfolio managers is enormous. In addition to the now-traditional daily data, an explosion in alternative data brings portfolio managers a variety of inferences to choose from. And, yes, AI can help choose the optimal and most effective alternative data source as well, more on this later. For now, let’s focus on the immediate improvements AI research in Finance can bring to portfolio managers using classic portfolio allocation strategies.

Take, for example, passive buy-and-hold funds. Vanguard has a few. Vanguard Total Stock Market ETF, now available as an Admiral Shares mutual fund, is a prime example. The fund passively invests into the CRSP Index equities in the stock market, no matter how large or how small. The fund charges 0.04% administration fee, or $4 on every $10,000 of assets under management per year and is widely held: it has a market capitalization of U.S. $ 173.03 Billion, as of September 27, 2023, per Google. On that capital, every $4 per $10,000 of assets adds up rather quickly to a nice US $69.212 million per annum. For comparison, SPDR S&P 500 ETF (NYSE:SPY) has $384.4 Billion in market capitalization as of September 27, 2023, and charges $9 on every $10,000 under management, or $345.960 million per year. And in addition, the SPY ETF beat the VTI by 0.8% year-to-date, 1% over the past year, 5% over the past 5 years, and by nearly 400% since 2001.

Figure 1. Performance of Vanguard Total Stock Index ETF (VTI) vis-a-vis the STDR S&P 500 ETF and Russell 3000 Index.

In its composition, Vanguard covers close to 4,000 stocks as defined by CRSP U.S. Total Market Index. The index is managed by the Center for Research in Securities Prices (CRSP), an affiliated entity of the University of Chicago Booth School of Business, who determine the weightings that each stock represents in the total market portfolio. CRSP is reallocated once a year, at the end of each calendar year, with the date known to all market participants. Vanguard Total Index ETF follows suit and reallocates its holdings annually as well, as prescribed by CRSP.

While the fees collected by Vanguard appear considerable and comparable to those of SPY, Vanguard Total Index ETF also faces much higher trading expenses than does the SPDR S&P 500. The transaction costs accrue. Reallocating every stock at the end of the year may not seem like much. (In comparison, the S&P 500 ETF needs to be reallocated four times per year.) However, the transaction costs for processing the 4,000 stocks, including very small issues, can be very expensive.

The transaction costs comprise several components. First, there is the obvious percentage of the value traded. Typically, an institution like Vanguard would encounter about 0.3% transaction fee on their reallocation. Next, however, there are less tangible fees. For example, reallocating 500,000 shares in a small stock can create severe market imbalances and not only disadvantage investors, but also the company behind the stock. When Vanguard sells a significant portion of its holdings in a small firm, it often accounts for as much as 25% of the company’s stock. This, inevitably, drives the corporate stock price down. Next, the company’s creditors observe the drop in the stock price, determine that something is wrong with the business and reduce their credit lines to already small firms. The actions of Vanguard stretch far beyond its immediate fund.

Furthermore, small stocks tend to be characterized by low supply and demand, a condition known as illiquidity. When a stock is illiquid, its best available trading price (best bid or best offer) can be quite far from the polite weighted average reported in the news and the Internet. For some issues, there is simply not enough market interest in trading them to take the side opposite to Vanguard’s large orders. In response, sophisticated investors like hedge funds step in to “help” Vanguard in their plight, but at a very high cost. In many cases, large positions in small stocks can only be bought (sold) at as much as a 30% premium (discount). Trading in illiquid issues, therefore, may not only obliterate all projected gain, it may also result in severe losses in the affected stocks that will reduce the gains on performing assets in the portfolio. Vanguard executives will just pass those losses on to its trusting investors.

How can portfolio managers tasked with a CRSP mandate not invest in illiquid issues? AI to the rescue! The new article, “The AI Revolution: From Linear Regression to ChatGPT and beyond and How It All Connects to Finance”, by Aldridge (2023), just published in the Journal of Portfolio Management covers this topic in great detail. Specifically, the article details how portfolio managers can use AI to distill from data the most important ingredients. In case of portfolio optimization, these ingredients can be the most important stocks in a large pool of names.

How does the technique work? In a nutshell, the technology is similar to that of restoring old photographs:

• Take all data
• “Squeeze the data” to remove noise and columns with little relevance
• Reconstruct the original dataset in a much more concise and powerful way.

The technique enabling the above process is known as Singular Value Decomposition (SVD), a close cousin of Principal Component Analysis (PCA). The SVD creates an efficient digital summary of the entire dataset, essentially separating the dataset into two components:

Original dataset = Data Summary + Nice-to-have noise (1)

The “Data Summary” can be as concise or verbose as the researcher likes. The more concise Data Summary produces a smaller set of the most important inferences. The more verbose Data Summary includes additional details, all ranked in the order of importance. When aiming for the big picture, the Data Summary delivers the goods.

The “Nice-to-have noise” is not a complete waste, either. It may describe small details that often make the pictures complete, but bear no relevance to the central message in the data. These details, however, may be very important in the analysis of differences or, in the academic financial speak, “idiosyncrasies” of individual data elements. The “Nice-to-have noise” can play a key role in identifying the differences among the data points in the datasets.

The SVD is a step up from Machine Learning. Machine Learning tends to be a tedious iterative process of finding a complicated model that fits the data at hand to a relatively good degree. Machine Learning models include Linear Regressions, Neural Networks and, yes, even ChatGPT. Due to their design of being tightly wound around a particular dataset, Machine Learning models tend to break down in the real-world applications, when taken out of the lab and exposed to the real-world conditions.

SVD, on the other hand, goes beyond that into identifying what exactly the data is trying to show us. The core drivers in the data tend to carry over, even when exposed to a fresh new set of related data. As a result, SVD is robust to small changes in the data and is substantially less likely to break down when applied to the future data, as compared to Machine Learning.

How can SVD help structure more efficient portfolios? Like in any real-life situation, portfolios tend to follow the usual 90%-10% rule: 90% of portfolio performance is driven by just 10% of the portfolio constituents. The larger the portfolio, the more true this thesis becomes. The “Nice-to-have'' 90% of the portfolio constituents just add a little noise, a color here and there, that may not at all be meaningful for the overall portfolio performance. In particular, after the transaction costs and other frictions are taken into account, the “Nice-to-have” 90% may cause more harm than good. Eliminating the noise of illiquid issues will not change the big picture, but will help save investors their retirement dreams.

What would SVD do to a CRSP portfolio like Vanguard’s? The answer may appear to be too simple: SVD can help identify the dominant portfolio dimensions, important this year. Would the SVD turn a CRSP dataset into the S&P 500? Not quite, mainly because we can show that the S&P 500 can use a fair bit of optimization itself. The optimal allocation would also change from one year to the next. However, fewer illiquid and hard-to-trade stocks will improve the fund’s operational performance and bring the cost structure in line with that of the S&P 500.

Most importantly, such portfolio optimization would benefit investors. Not paying for expensive reallocation would return the money where it belongs: in the investor’s savings and retirement accounts, and not the already plump broker-dealers, hedge funds and other third-party firms.

Irene Aldridge is CEO of AbleMarkets, an AI-based Financial Research platform, a co-author of Big Data Science in Finance (with Marco Avellaneda, Wiley, 2021), and a consultant. Aldridge teaches AI at Cornell University Financial Engineering Program in Manhattan and at Cambridge University in the U.K.