Separately Managed Accounts (SMAs): Are They Worth Your Money?

I received a question recently via email from a WCIer:

“I have a brokerage account and invest in low-cost index funds like the Fidelity 500 Index Fund (FXAIX). I have been speaking to a Fidelity ‘advisor' who is recommending that I transfer money from my brokerage account to a Separately Managed Account (SMA) for the tax savings and [a] better net return. The ‘advisor' claims that despite these low-cost index funds having low expense ratios, they have significant tax burden that an SMA account can mitigate. The problem for me is that the SMA account has a much higher expense of 0.4% vs. 0.01% or 40 times that of the low-cost index funds.

The advisors will obviously be making much more off of my money, but do you believe it is worth it for the lower tax burden in the brokerage account? They claim that the net return after considering tax over the past 10 years was 13% in the SMA account vs. 11% in a fund like FXAIX in a brokerage account. The other component of this is that I would need to sell all of my shares in my brokerage account which would incur a large capital gain tax burden, approximately $20,000. I'm also not sure about the tax-filing difficulties that may come with an SMA account. Are SMAs accounts worth it?”

TL; DR Answer

No, an SMA is not worth it. But there is so much more to unpack in this email that we can turn this into a full blog post.

The 500 Index Fund Craze

First, let's talk about the 500 index fund craze. What I mean by that is that for the last couple of years, many investors have talked about investing in a 500 index fund instead of the much better total stock market (TSM) funds. The 500 index funds own 500 stocks. TSM funds own close to 4,000 stocks. The 500 index funds don't own mid cap or small cap stocks. TSM funds own all the stocks. Savvy institutional quant investors “front-run” 500 index funds, depressing their returns. You can't really front-run a fund that already owns everything.

Why is there so much interest in 500 index funds lately? It's performance-chasing, pure and simple. Large stocks, particularly large growth tech stocks, have outperformed the stock market over the last decade or so. Consider this chart of QQQ (mostly large cap growth tech stocks) against VB (US small cap stocks):

Notice anything weird? Especially in the last four years? Exactly. Now you know why people are talking about tilting their portfolios to growth and tech—something that hasn't happened since the late 1990s—and using 500 index funds instead of TSM funds (even though correlation between the two has been 0.99 over the years).

Momentum (an outperforming stock continues to outperform for a while) exists but not indefinitely. So, recent outperformance is not a reason to buy one investment over another. It is at least as likely that it is a reason not to do so. And we're not even getting into the fact that not many people like bonds or international stocks these days either. Don't abandon your diversification. Use TSM over 500 index funds when possible. When not possible, 500 index funds are fine, so don't worry about it too much.

More information here:

My Favorite Mutual Fund

How Do You Evaluate and Compare Mutual Funds and Exchange Traded Funds?

‘Advisors'

The quotes around advisor didn't exist in the original email. I added them. I always add them when people who aren't real advisors refer to themselves as advisors. Real financial advisors earn fees, not commissions, for their advice. They do real financial planning before moving on to portfolio management. Their focus is on plans, not products. Maybe there are some real financial advisors who work at Fidelity, but I've never heard of one. Real financial advisors don't work for insurance companies or loaded mutual fund companies. We could make a huge, long list of the companies that pretend they offer financial advice but really just sell products.

If you don't know the companies you shouldn't be doing business with, you are not yet financially literate. Keep talking to financially literate people and asking questions, and you'll figure it out. I assure you that a large percentage of WCIers can make a list of “bad guy” companies that is 6-12 companies long, and the overlap on those lists will be close to 90%.

Ditch Vanguard for Fidelity?

I don't actually have a lot against Fidelity. In fact, I have multiple accounts custodied at Fidelity, including our HSA and the WCI 401(k). We have a 2% cash back credit card there, too. It is a fine company, and it wouldn't end up anywhere near my dirty dozen list. In reality, as mutually owned Vanguard has struggled over the years with information technology and customer service due to their rapid growth and their focus on low costs, many intelligent investors have abandoned Vanguard for Fidelity.

However, going to Fidelity has a few downsides. First, the company is a relative latecomer to index fund investing. It just doesn't have quite as robust a selection of index mutual funds as Vanguard. That's not a big deal, though. You can just buy the Vanguard ETF version of its funds at Fidelity if you don't like the Fidelity option for that asset class.

The bigger deal is that Fidelity has a much bigger incentive to sell you other stuff. Vanguard doesn't really have expensive, crummy mutual funds. Fidelity has a whole bunch of them. Vanguard does try to push its investment management service these days, but at 0.35% of assets under management, it's hard to complain. I wouldn't expect high-quality, physician-specific financial planning out of Vanguard advisory services, but the advisors are not going to put you into some bonkers portfolio. I'm not sure that would be the case if you open up an SMA at Fidelity.

What Is an SMA?

A separately managed account is just that. You're not buying a mutual fund. You're paying an investment manager to pick stocks for you. But if that person is really good at picking stocks, do you think they'd really be doing it for your $300,000 portfolio and that of 20 other people? No way. They'd be managing billions of dollars, and you'd be reading all about them in the financial press. It's really hard to pick stocks well enough to beat an index fund, especially long-term and after-tax. Especially if you have to sell everything you already own and pay capital gains taxes at the beginning of this new strategy.

More information here:

Should Doctors Consider Angel Investing? The Other 5% of Your Money

The Emotions Behind Short-Term Trading: The Other 5% of Your Money

Beware People Selling Lower Taxes

The word is out now that actively managed mutual funds really don't beat index funds long-term, after-tax. A couple of them might, but you can't identify those a priori. If you don't know that, you're not an informed investor. Rather than advertising market-beating returns, the salespeople now market lower taxes. There are all kinds of schemes for this. It might be whole life insurance. It might be an automated tax-loss harvesting program. It might be sovereign tribal tax credits or conservation easements. It might be direct indexing (more on that in a moment). Apparently, it might also be an SMA.

It isn't that most of these things don't work at all or don't have an element of truth to them. It's just that chasing these things is the definition of letting the tax tail wag the investment dog, and you definitely don't want to do that. Choose good investments. Then, keep your tax bill as low as is reasonable. In that order. The goal isn't to pay less in taxes. The goal is to have more money after paying any taxes due. Don't forget that.

How does an SMA lower taxes? If the average dividend yield on a total stock market fund is 1.15%, many stocks will have a lower yield than that. In fact, many, many stocks in the US don't pay dividends at all. About 80 of the stocks in the 500 index don't pay dividends. What's the problem with dividends? If you're not in a very low tax bracket, you have to pay tax on them—even if you didn't want to spend that money and just wanted to keep it all invested. So, ALL ELSE BEING EQUAL, lower dividends mean less taxes and a higher after-tax return, at least until you sell. All else, of course, is never equal, but an SMA is offering to lower your taxes by just buying low- or no-dividend stocks instead of all the stocks. Is that worth 0.40% a year? Not in my opinion. I'm not sure I'd take that offer for free.

Now, the tax situation with an SMA isn't going to be too bad. There will be a lot more investments to keep track of, and if you change your mind on your investing strategy, you're going to have to get out of all those or deal with a complex portfolio for the rest of your life. But for the most part, tax software can just import the 1099 from the brokerage account. You don't have to input all those transactions manually.

How Is This Different from Direct Indexing?

Direct indexing is buying all the stocks and running your own index fund. Why would anyone want to do that? The problem is that mutual funds (including ETFs) can't legally pass losses through to their investors. If you're investing directly in all the stocks in a taxable account, you could theoretically tax-loss harvest hundreds of losing stocks every year and book those losses to use on your taxes. So, you're going to get more losses than if you just wait for market downturns and do huge tax-loss harvesting transactions like I do. Whether MORE losses are helpful to you is specific to your tax situation, but I think for most white coat investors, the easy way of tax-loss harvesting will provide plenty of losses.

By way of example, we tax-loss harvested once in 2020 and a couple of times in 2022, and we are carrying forward seven figures of tax losses. Keep in mind, we can only use $3,000 a year of those against our ordinary income. The rest just gets carried forward, and they are used to offset occasional capital gains we're forced to realize. We don't realize all that many since we routinely flush out gains from our portfolio through charitable giving. It wouldn't be a BAD thing for us to have more losses, but I'm not going to pay all that much for it.

When people started trotting out direct indexing and wanting to charge 40, 50, or 100 basis points for it, I said, “No way.” These days, some folks out there are offering to do it for 10 basis points. That's not a lot of money, and maybe that's worth it to some people. Probably not us, but some people. But that's a whole lot different from an SMA. With an SMA, they're trying to pick stocks to beat the market AND lower taxes. I don't have much faith in the former and fear the latter effort will cause further underperformance in many situations.

Don't Pick Investments Based Solely on Past Performance

It's nice to see good past performance numbers, but choosing an SMA manager based only on them telling you they did 13% and your favorite index fund only did 11% is pretty much the definition of financially illiterate. I don't care about your past performance. I can't buy that. Explain to me why you're going to have superior future performance. And you'd better be convincing because I've heard a lot of dumb arguments about why someone is going to beat an index fund but seen precious few people actually do it long-term, especially after-tax.

More information here:

A Die-Hard White Coat Investor Buys an Individual Stock — An M&M Conference

The Bottom Line

I have yet to see someone offering SMAs that I thought were a good deal for the investor. I doubt this one the Fidelity rep is offering is going to be an exception.

What do you think? Have you used an SMA? Were you happy you did? Why or why not?