Broadly diversified, low-cost index funds tend to be strong core holdings that are hard to beat. But that doesn't mean they're always the best choice. Some areas of the market are likely more conducive to active management. Still, active management isn't right for everyone. It's a tough game to win, and one that requires patience.

The framework presented below can help guide decisions about where active strategies might play a role in your portfolio.

Start with fees

1 | How large is the fee hurdle?
Active managers must clear their fee hurdles to match the performance of their benchmarks. The lower that hurdle is, the easier the task.

Gauge the opportunity for active managers

1 | How representative is the index?
In aggregate, active management is a zero-sum game (before fees) because active investors define the composition of the market. For one investor to beat the market, another must be on the losing side.

While there's no way around the math of active management, index funds aren't always representative of the opportunity set available to active funds. The less representative the index, the less the arithmetic of active management pencils out. For example, many active managers in the U.S. large-blend Morningstar Category have some exposure to foreign stocks, which are not available to U.S.-focused benchmarks like the Russell 1000 Index. If foreign stocks outpace their U.S. counterparts, a disproportionate share of active managers could outperform the benchmark.

To assess an index's representativeness, it's helpful to compare its sector, regional, and factor characteristics to the category average. If those are similar, the index is probably representative of how active managers invest. In categories where indexes are less representative, active management may be more viable.

Most broad, market-cap-weighted U.S. stock index funds are fairly representative of the opportunity set available to their actively managed peers, despite their exclusion of foreign stocks. That's also true of most traditional foreign developed- and emerging-markets stock index funds.

However, the Bloomberg Barclays U.S. Aggregate Bond Index is less representative of how active investment-grade bond managers invest. It tends to take significantly less credit risk than most active funds in the intermediate core-bond category, owing to its heavy stake in Treasuries and light exposure to corporate bonds.

This is because active bond fund managers aren't the only investors in the market, and their objectives differ from other types of investors. For example, while active managers are generally concerned with benchmark-relative performance, banks often hold Treasuries to meet capital reserve requirements, not to maximize returns. Similarly, some foreign central banks use Treasuries as a foreign-exchange reserve to influence the value of their currencies.

Market-value-weighted bond indexes, like the Aggregate Index, are shaped by all these investors. Consequently, they can look and perform quite differently from their active fund peers. Here, there's probably greater opportunity for active managers to beat their benchmarks than there is in the stock market.

2 | How liquid are the underlying assets?
Index investing works best when the underlying holdings are regularly traded with decent volume. That mitigates transaction costs and makes the index easier to track, as it's easier to obtain the holdings. Index managers prize low tracking error above all else. So, when securities are added to or removed from an index, the managers who track it are forced to trade, regardless of price. Those forced trades can move prices away from the managers before they can fully enter or exit the positions.

This is the market impact cost of trading, and it hurts the performance of the index itself. This cost tends to be greater the more thinly traded the underlying securities are, as index managers must make larger price concessions to entice others to trade with them. Consequently, market segments with limited liquidity, like micro-cap stocks and high-yield bonds, generally aren't very conducive to index investing. Active managers in illiquid markets face these liquidity challenges as well, but they have the advantage of being able to trade patiently and walk away from a trade if the price isn't right.

3 | How susceptible is this area of the market to mispricing?
Skilled active managers should do better in markets where there is a larger supply of mispricing, though the cost of choosing the wrong manager is also greater here. If mispricing helps some, it hurts others. Such mispricing is more likely among securities that attract less scrutiny, like small-cap stocks, or that are subject to forced trades, like investment-grade bonds that are downgraded to junk status ("fallen angels").

This consideration is behind the conventional wisdom that it is best to use index funds in highly competitive markets (that are more likely to be efficient), like U.S. large-cap stocks, and active funds in less efficient markets, like foreign small-cap stocks.

There is some credence to this view, as mispricing is necessary for skill to pay off. But it doesn't guarantee success. Mispricing usually isn't obvious in real time, and there's almost never a risk-free way to profit from it. It's also tough to consistently be on the right side of these trades.

4 | What are the base rates of success?
To set realistic expectations of the odds of choosing a market-beating active manager, it is helpful to look at active managers' past success rates. The point isn't to anchor to these numbers (they change all the time) but rather to understand the magnitude of the challenge active managers face.

Morningstar's Active/Passive Barometer, which is published twice each year, tracks historical success rates for active managers in several categories over several horizons. To be counted as a success, a fund must survive and outperform the average of its index peers over a given period. Exhibit 1 shows active funds' success rates over the trailing 10 years through June 2019.

Exhibit 1

In most categories, most active managers failed to beat their index peers over the long term. Market prices tend to do a good job reflecting information that's available to the public, which makes it tough for most managers to recoup their fees.

Active management appears to be particularly tough among U.S. large-cap stocks, and somewhat easier--though still no walk in the park--in the smaller-cap, foreign-stock, and bond categories. That said, the success rates in the bond categories aren't necessarily representative of manager skill, as active managers often take greater risk that tends to pay off.

Is active investing suitable for you?

1 | How confident are you in your ability to identify skilled managers?
The success rates in Exhibit 1 reflect the odds of success that investors with no skill in manager selection would have had choosing from the menu of available options at the end of June 2009. Most investors probably wouldn't attempt manager selection if they didn't think they had some skill with it, but guard against overconfidence. It's hard to identify skilled managers, even for those who do it for a living. Those with no skill should just stick to index funds, even in categories with high base rates of success.

Investors with greater skill in manager selection have greater odds of success, but such skill is not always sufficient to overcome the odds. For example, suppose you could correctly identify whether a manager was skilled 80 per cent of the time (which is hard to do). What would your probability of picking a winning manager in the large-blend category be?

If the success rates presented in Exhibit 1 are representative of the percentage of skilled managers in each category, that probability would be less than 80 per cent because there are far more unskilled managers than skilled. In fact, it would be about 25.8 per cent. If there are 100 managers in the category, you might invest with 6.4 of the skilled managers (80%*8) and 18.4 of the unskilled managers (20%*92), leaving a success rate of 6.4/(6.4+18.2)=25.8%.[1] The last column in Exhibit 1 extends this illustration to the other categories.

2 | How willing are you to stick with a fund through periods of underperformance?
Successful active investing requires patience. All active strategies underperform at some point. At times, even the most skilled managers can underperform significantly for years, as a Morningstar study shows.[2] Investors who don't have the patience to stick with a fund through its rough patches may not be around when the strategy does outperform and should probably just invest in broad index funds.

Where to be active?
Investors with the necessary patience and insight can be successful with active management almost anywhere, though it's tougher in some categories than others.

The U.S. large-cap market has been especially difficult for active managers. That's because it is a highly competitive market where significant mispricing doesn't happen often, and the indexes are representative of the opportunity set. While foreign developed-markets stock managers have had higher success rates and can pull more levers to differentiate themselves from their benchmarks, those same conditions apply. It's not clear that they should have higher rates of success than their U.S.-focused counterparts.

There are likely greater opportunities for active managers in the bond, small-cap, and emerging-markets stock categories. Broad bond indexes often aren't representative of how active managers invest, while small-cap and emerging-markets stock managers might have an easier time finding mispriced securities. That said, index investing still works fairly well in most of these categories. There's nothing wrong with an all-index portfolio. That's probably the best approach for most investors.

[1] This is given by Bayes' Theorem.
[2] Kaplan, P., & Kowara, M. 2018. "How Long Can a Good Fund Underperform Its
Benchmark?" Morningstar. https://www.morningstar.com/lp/how-long-good-fund-underperform-benchmark?cid=RED_RES0002.