[Editor's Note: As you accumulate wealth, you need a way to protect your assets from potential malpractice lawsuits. WCI’s newest book is The White Coat Investor's Guide to Asset Protection, and it provides the techniques you can use to safeguard your money AND has the most comprehensive list of state-specific asset protection laws ever published. Pick up the Number 1 Amazon best-selling book today and make sure to protect your wealth!]
By Dr. James M. Dahle, WCI Founder
I've been teaching people to tax-loss harvest here at The White Coat Investor for more than a decade. Only once during that time period did I receive pushback from someone about doing it. They didn't have any sort of legal or ethical issue with it. They simply questioned whether it was a good use of a doctor's time. I do think it is something worth learning to do if you have enough interest to manage your own portfolio, but it's not hard to make a case against doing it.
I'm not wealthy because I tax-loss harvest. I'm wealthy because I made a lot of money, saved a big chunk of it, and invested it in a reasonable way. Tax-loss harvesting is, at best, icing on the cake. At worst, it's actually impeding the building of wealth.
What Is Tax-Loss Harvesting?
Before we get too far into this article, let me briefly explain what tax-loss harvesting is. If this isn't enough detail, see the big article about it at the link above. Basically, when you have a realized loss on an investment, you can use $3,000 of that loss against ordinary earned income each year and an unlimited amount of that loss against capital gains. So, it does not really ever make sense to hold a taxable investment with a loss. However, you do not want to “sell low” after a loss; that's a recipe for bad investment behavior.
To capture that loss to use on your taxes without selling low, you simply exchange the asset for one that has a high correlation with the original asset but is not, in the words of the IRS, “substantially identical.” In practical terms, substantially identical basically means the same investment (same CUSIP number). You can't exchange a Vanguard Total Stock Market Fund for the Vanguard Total Stock Market ETF, but you can exchange it for the Vanguard Large Cap Index Fund or the Vanguard 500 Fund, which have a correlation of 0.99 with the Total Stock Market Fund. There is also a “wash sale” rule that does not allow you to buy the fund you just sold for a loss within 30 days before or after the sale, which further complicates matters (although interestingly, it does not apply to cryptocurrencies under current law.)
10 Reasons Not to Tax-Loss Harvest
Now that you know what it is, let's talk about why you may not want to bother with it.
#1 The Tax Break Isn't That Big
Your deduction is limited to $3,000 a year, and it hasn't increased the entire time I've been investing. It's definitely not indexed to inflation. My marginal tax rate is 42% between federal and state, so it's $3,000 x 42% = $1,260. That represents 4-6 hours seeing patients in the ED (or maybe a full shift after tax). Only you can judge how much your time and hassle are worth.
If you are in a situation where you can save a lot more than $3,000 a year because you have other reasons to have capital gains, then it can continue to make sense to tax-loss harvest regularly. But if not, you could probably generate an entire lifetime of $3,000-per-year deductions with a single transaction in a bear market if you have any sort of a reasonably sized taxable account. Imagine you have a million dollars in stocks, and the market drops 40%. It wouldn't at all be unusual to book $200,000 in losses from an event like that.
But if you own a small business, a practice, or a bunch of property you expect to sell at a gain in the future, then it can make sense to try to keep acquiring losses. We may end up selling WCI at some point down the road, so I'm still tax-loss harvesting. But I don't even really bother any more unless it's a six-figure loss.
#2 It Might Be Just a Deferral of Tax
In some cases, you are simply deferring the payment of your tax, particularly if you are using your capital losses from tax-loss harvesting against capital gains on the same securities down the road when you spend the money. If you save $1,260 in ordinary income taxes and then later pay $3,000 x 28.8% = $864 in capital gains taxes, your saving is now down to just $396 (28.8% is my federal plus state marginal tax rate on long-term capital gains.) The longer you can defer taxes the better, but this tax break is significantly larger if you never sell those securities that you tax-loss harvested (instead you could donate them to charity after a year or leave them to your heirs to get a step-up in basis at death). This is particularly noteworthy in an Intentionally Defective Grantor Trust like ours that will not get a step-up in basis at death on those securities.
#3 Not Worth Hiring Someone to Do
Some people look into hiring a robo-advisor or even a full-service financial advisor just because they find tax-loss harvesting overwhelming. The robo-advisors certainly plug this as a big value-add for you. But the truth is that advisory fees cost a lot more than tax-loss harvesting is going to save you. Imagine you're paying 0.25% with Wealthfront or Betterment on a $2 million portfolio. That's $5,000 a year. You're paying $5,000 to save $1,000 (or maybe less). There better be something else you really value that they are offering. It could be even worse if you are overpaying for advice. Tax-loss harvesting can be a marketing tool. Know what it's really worth to you so you don't fall for the marketing pitch.
#4 More Complex Portfolio
When you tax-loss harvest, you had better be exchanging into something you are willing to hold long term because you may very well end up holding it long term (unless you give it away to charity). But even if you are willing, you will end up with a more complex portfolio. My parents have a relatively simple portfolio, but back at the beginning of the pandemic, I grabbed them a few tax losses in their little taxable account (I couldn't get them to spend their RMDs so we just reinvested the money in taxable.) But now they're stuck with TSM, a large cap index, and a 500 index in their taxable account until they die or decide to spend more than their RMD in a given year. The portfolio is not as simple as it could be.
Now, imagine you have a taxable account with four or five different asset classes in it and two or three different funds for each asset class. Tax-loss harvesting has made your life more complicated. And for what: $400 a year? One could very reasonably decide that isn't worth it, especially given that the $3,000-per-year limit is not indexed to inflation and that it will become less and less and less each year in real, after-inflation terms.
#5 Have to Watch the Markets More Closely
I pride myself on not looking at my investments for months or even years at a time. Aside from the hassle factor, I actually go off-grid for weeks from time to time where I couldn't look at my investments if I wanted to—at least not without a satellite device/phone. But when market drops are rapid and severe (like the Coronabear of March 2020), you might miss out on a big tax-loss harvesting opportunity.
Aside from that, tax-loss harvesting makes you look at the markets at precisely the time when, behaviorally speaking, maybe you shouldn't be looking at the markets. I wonder how many people have gone to tax-loss harvest and ended up panic-selling?
#6 Can Get Burned During the Transaction
I try to make as a few trips through the “Wall Street casino” as possible with every invested dollar. Commissions, bid-ask spreads, and impact costs can add up after a while. But a bigger problem, at least if you are using ETFs, is that the market can rise a bit between the time you sell the first ETF and the time you buy the second one, especially on a volatile day when tax-loss harvesting is most tempting. It doesn't take too much of a rise to eliminate the entire benefit of claiming the loss.
This isn't an issue if you are using traditional mutual funds and are just exchanging them at the end of the day. But I feel like, Murphy's Law being what it is, the prices of the ETFs I'm exchanging usually go up between the sale and the buy when you would think that half the time it should go down.
This is an even bigger deal if you have elected to wait 30 days to buy back the same security rather than exchange immediately to a similar one.
#7 Unqualify Dividends
A lot of people forget about this little rule. If you don't own a security for 60 days around the time a dividend is paid out, that dividend becomes unqualified even if it would otherwise qualify for the lower qualified dividend tax rate. So, frenetic tax-loss harvesting in March-April, June-July, September-October, and December-January can easily cost you more in additional taxes on the dividends than you will save with the losses. Another great reason to only tax-loss harvest occasionally.
#8 Can't Put Investments on Autopilot
This is one of the best arguments against tax-loss harvesting. I bet not a week goes by that I don't have someone ask me how they can tax-loss harvest when they are automatically buying mutual funds every month. Automation is a very powerful investing tool, but it does not play well with tax-loss harvesting. You pretty much have to choose one or the other.
#9 Can't Reinvest Dividends
I don't reinvest dividends in my taxable account, and the main reason is just to limit the number of tax lots I have (although it assists a bit with rebalancing too.) The truth, though, is that Vanguard, Fidelity, Schwab, etc. all do a fine job of keeping track of all those tax lots for you. The real issue is that those reinvested dividends can cause an unintended wash sale. Just like you can't leave your investments on autopilot and tax-loss harvest, you can't really reinvest dividends and tax-loss harvest. Would you benefit more from reinvesting dividends than tax-loss harvesting? It's entirely possible.
#10 Impact on IRA and 401(k) Investing
Wash sales occur when you buy the same thing you just sold, even if you did so in an IRA. The IRS is very clear that IRAs are included in the wash sale rule (although if your IRA and taxable account are at different institutions, I have no idea how they would ever find out). Whether 401(k)s are included is unclear. But now you have to watch what you are doing in all of your accounts just to be able to tax-loss harvest. You either have to use different securities in taxable or really be careful not to buy in a retirement account within 30 days of tax-loss harvesting. Now you may not be able to reinvest dividends in retirement accounts either!
The bottom line is that if you have been looking for an excuse not to tax-loss harvest, there are plenty of them in this post. I'm certainly tax-loss harvesting a lot less frequently than I used to—and after reading this, I bet you will too.
What do you think? Do you tax loss harvest? Why or why not? Comment below!