Podcast #126 Show Notes: How to Reduce your Investment Taxes with Phil DeMuth

Every doctor and other high income earner wants to reduce their investment taxes. Our podcast guest, Phil DeMuth, is just the man to help you do that. Phil is the managing director at Conservative Wealth Management, LLC,  where he enjoys working with “high net worth clients and saving them from the depredations of the financial services industry.” He is the author of 11 books, my favorites being The Affluent Investor and The Overtaxed Investor.  We really focus on reducing taxes in this episode, especially investment-related taxes. Phil says that doctors are ducks in a shooting gallery. We are the most vulnerable people in terms of taxes and we need to learn to defend ourselves as best we can.

In this episode we look at all the tax reducing tactics like maximizing the use of retirement accounts, using tax efficient investments in your taxable accounts, getting your asset location right, making sure your dividends are qualified and that your capital gains are long-term, depreciating real estate, tax loss harvesting, donating appreciated shares to charity to flush those capital gains out of your portfolio, getting a step up in basis at death, etc. Which of these tactics will make the most impact? Listen to this episode to figure it out.

Contract Diagnostics is a long-term advertiser with us here at WCI.  I love this company as they’ve helped hundreds of WCI get a ‘fair shake’ when it comes to reviewing and understanding their employment contracts – its 100% what they do there. This is a company that specializes in physician contract reviews – specialization is something we can all appreciate here. All contracts are reviewed by an in-house attorney and presented in a simplified way back to you.  Using custom documentation, compensation data, and times from 6am to 8pm they make it easy for you.  All packages are flat priced so you know what you will pay upfront – residents and fellows can even make interest free payments over time.   They have a new ‘Compensation Review’ offering coming soon for those of you happy in your positions as well.  So look them up today – contractdiagnostics.com or 888-574-5526. They’ve have helped many of your WCI colleagues.

Quote of the Day

Our quote of the day today comes from Peter Lynch, who said,

“I can’t recall ever once having seen the name of a market timer on Forbes annual list of the richest people in the world. If it were truly possible to predict corrections, you’d think somebody would have made billions by doing it.”

I think that’s absolutely true. That’s a great quote from Peter Lynch, reminding us of just how difficult it is to time the market.

How to Reduce your Investment Taxes with Phil DeMuth

Physicians as Investors

A lot of Phil’s clients are physicians. I asked him what he thinks is unique about the financial life of a doctor? He says in theory doctors should be fantastic investors. In theory, there should be no White Coat Investor website, because doctors should never have gotten into any trouble in the first place, because we are super smart and we know how to make cost-benefit decisions, how to do risk analysis, and how to live with consequences. We have all the right skills to be great investors. But then in practice, a lot of doctors are lousy investors.

“I suspect that it has to do just with the fact that doctors tend to be extremely smart, and that means that they can also be somewhat easily bored and restless, and then they look to, in the financial world, they look to Jim Cramer, they look to CNBC, as being a source of infotainment and investotainment, and then so all of a sudden they’ve entered this whole area through the wrong door. They’ve entered it through the casino door, and it’s a terrible approach, and you can lose a lot of money or spend a long time before you find a way to make a lot of money when you start out that way.”

Given those habits and inclinations does Phil think it is reasonable for any doctor to be a do-it-yourself investor? What percentage of them does he think can do it well enough that they should not hire a financial planner and/or an investment manager?

“Again, it’s a very tough call because, in theory, especially on the investing side, they can do it very easily. Every doctor could buy a Vanguard target date fund targeted for the year they plan to retire, and that could take care of their entire investment life, up to a point, anyway. They could do great doing it, and yet I’m aware that Dr. Bill Bernstein, who I think very highly of, he thinks that doctors, maybe only 1 or 2% could do it because there are temperamental issues with it all. But that’s not to say that I have a great endorsement for the profession of registered investment advisers or financial planners. I think most financial advisers, they charge too much, they don’t really deliver a lot for what they do charge. I think that you have to choose somebody very carefully in that regard, and then it’s also just a question of do I want to spend a lot of my time learning how to manage money when my alternative use, being a physician, is very highly paid? It’s a tough trade-off, and there’s a real opportunity cost involved. I don’t think there’s an easy answer there.”

I think even a doctor that’s relatively highly paid looks at what they’re paying for investment management, financial planning, et cetera, and realizes that that is the highest paid use of their time. It may be the equivalent of $400 or $500 an hour, or a thousand dollars an hour for them to learn how to do this stuff themselves, and I think that provides this pretty significant motivation. One of my readers said, it’s the best-paid hobby out there, and if you hate it, it’s the best-paid chore out there.  It’s just so expensive,  it’s almost like you feel like you have to learn it, but obviously some people still choose to outsource that part of their life.

“It could be a very highly paid hobby if you have the temperament to be disciplined about it. If you don’t have the temperament, the problem is your college buddy comes to visit with his wife, and she’s driving a Mercedes and your wife’s driving a Honda, and then they leave, and your wife says to you, “I hear that he’s been investing in marijuana stocks, and he’s really making a lot of money. Why aren’t we doing that?” You’ve got to be able to stand up to these kind of situations when you see other people who are idiots doing better than you are.”

Tax Avoidance vs Tax Evasion

I asked Phil does he have any advice to my listeners about how to make sure that the tax reduction tactics they are employing are tax avoidance and not tax evasion?

“For some reason, this seems to be an especial weakness of the physician class, because doctors seem to always be interested in the tax angle, probably because they’re so incredibly taxed to begin with, it’s a sensitive subject. Again, if it sounds too good to be true, it almost certainly is too good to be true, and if the tax picture sounds bad, if your accountant says, “Gosh, you’re paying a lot of taxes this year,” he’s probably right. If you’re going to be suffering, that’s a correct perception of reality.

On the other hand, the good news is, I tell my clients, you’re never going to be audited, because they’re going to look at how much you make, but then they’re going to look at how much taxes you pay and think this guy is fine. He’s suffered enough. Anything that promises to be a tax haven, anything that takes your money offshore, anything that’s … if you’re going to be trading bitcoin in a foreign account and not telling the IRS, these are going to get you into a lot of trouble, because the IRS, they don’t care about any of this stuff. They look at facts and circumstances. They are willing to undo any series of step transactions you’ve done to try to hide money from them, so don’t mess around.

If you’re ultra high net worth, you can fight the IRS. You can say, “Oh, yeah, I got your letter. Deal with my tax attorney. See you.” But if you don’t have $100 million, you’re not in that position. If you have $5 million, and you say to the IRS, “Oh, yeah, deal with my tax attorney,” well, pretty soon you’re not going to have $5 million. That money’s going to all go to your attorney and paying fees. You want to color within the lines, you want to give the IRS a wide berth, because you can’t afford to fight them. They’ve got more power than you.”

All in Tax Rate

In the first chapter of The Overtaxed Investor  Phil lists all the taxes we pay, which is overwhelming to read through. Income tax, sales tax, gas tax, hotel tax, excise tax, property tax, etc. What percentage of our income is going to taxes of some kind?

“This is a terrifying question. What I have done is I’ve just noticed that Greg Mankiw, the Harvard economist, who wrote the standard economics 101 tax book that everybody uses in college. He estimates, he’s a college professor, that his all-in tax rate is 90%. This is just what he pays on his salary, dividends, interest, estate taxes, money going to his kids. I was reading John Cochrane, an economist a Stanford. He says he pays 42% federal, 13% state, 10% sales tax, 6% property tax. It goes on and on, and he argued that if, as some presidential candidates have argued, we have a 70% marginal tax rate, he said if that was the all-in inclusive tax rate for everything, that would be a tax cut for many high-income Americans. It’s a horrifying figure, and of course, a lot of it’s disguised, especially now. Congress delegates authority to different agencies, and then the agencies set up their own tax rates and embed them into what they’re doing, so it’s a big mess.”

Phil argues that Star Wars is really all about taxes, quoting in the beginning scroll where it talks about the taxation of trade routes is in dispute.  I asked him does he think this country can actually implement a fair tax solution without six movies of intergalactic dispute?

“It looks like what’s happened is we’ve moved into a situation where every eight years, or four years, even, the tax code can change when a new administration comes into town. The democrats come in, they sweep, they get congress, and they suddenly raise taxes to where they should be, so to speak, and the republicans get in four, eight years later. They lower taxes, and this is a planning catastrophe. If you don’t have a stable set of rules that you’re planning under, it makes planning … you kind of just throw your number two pencil up in the air because it might be this way or it might be that way. If we could have a steady state tax code, even if a lot of people thought it was bad, a lot of people thought it was unfair and complained about it, if it was just the same, if it just stayed the same for three decades at a time, it would make everybody’s life a lot easier, and we just lose a lot of economic value through this needless jostling back and forth, having to do our wills over, our financial plans over. It just creates a lot of chaos. It’s completely unnecessary.

Economists feel that taxes should really be just based on consumption. A value added tax would do the least overall damage to the economy, and we could scrap everything else we’ve got going on, and then if you want to help people, at the end of the day, the congress simply says, okay, well, we’re going to give poor people money to make them less poor, and it’s just sort of a social policy. Instead of trying to use the tax code to solve 500 different problems. I don’t like it, and of course none of this will ever happen. We’re going to continue to have this kind of a mess because it’s very useful in terms of funding reelection campaigns for our representatives. That’s my cynical opinion, but I don’t think I’m wrong.”

Tax Reduction or Asset Allocation First?

I occasionally hear Phil argue for zero dividend individual stocks held in a taxable account. Sounds like it is letting the tax tail wag the investment dog.  In fact, in his book, he says your asset allocation is an illusion because most people don’t tax adjust their asset allocation, so first, you should make sure you’re not needlessly overpaying the taxes on your portfolio, and then worry about your asset allocation after that. Taxes come first. I asked him does tax reduction really matter more than asset allocation?

“For some people, I think it does. It’s not only the problem of the fact that your asset allocation is not really right, because you haven’t figured out what everything is worth after tax and made that adjustment, but here’s another big problem. Another illusion that we have is that I think I own the S&P 500 index fund, so I’m great. But the problem is when you get a downturn. It’s down 50%, and you think, oh, gosh, I’ve made a terrible mistake here. I sell it. So even though a Boglehead efficient portfolio seems like the answer for 99% of the population, if not 100% of the population, in practice, many people can’t stick to it because they are closet market timers, and they also way overestimate their risk tolerance, and so they panic at the worst possible time.

Now, the part about not owning inefficient tax assets, one of the people I was thinking about there is a group that I have a very soft spot for in my heart. The ultra high net worth. People with $50-$100 million, because what happens when you get into that group, I’m told, I’m certainly not in that group myself, what happens is that once the first generation of the family who actually earned all the money in the first place dies off, then the rest of the family is relatively clueless about money, and so the investment banks prey upon these people, and they say, “We’ll take care of you. We’re going to educate you.” They pack them off to these various seminars for ultra-rich people, and at the seminars, you’ll spend Monday morning learning about stocks. But then Tuesday through Friday is spent learning about hedge funds, private equity, venture capital, esoteric assets, collectibles, all this stuff that is extremely tax inefficient to get involved with, for the most part, and yet they give people a kind of pseudo sense that, oh, yeah, I know about how to invest in hedge funds, because I spent a couple of hours yesterday morning learning about them, where basically they have no clue what they’re doing, they’re going to be completely reliant on multiple layers of advisers to help them, and it’s just going to be a way of milking the family fortune, and they’re going to be steered into these snobby and inscrutable assets.”

He is concerned that a lot of people get funneled into this stuff that they don’t really understand. They’re scripted for dependence, and the taxes and the fees just go through the roof. He thinks even very rich people would be far better off invested in a Bogleheads portfolio and not getting side tracked with all the other nonsense.

There is obviously a lot of money that can be saved in keeping expenses and taxes down, but I think the traditional approach has always been asset allocation first, and then you pick your investments, and then you try to reduce your taxes as best you can with tax loss harvesting and using retirement accounts, et cetera. But Phil seems to think that taxes are really where it’s at. That’s really where the bang for your buck is at, and your asset allocation is almost superfluous. That’s a pretty unusual viewpoint.

“Well, basically what it means is you get rid of holding very tax inefficient assets. Alternative investments can be very tax inefficient. You can probably live without them. REITS can be very tax inefficient if you don’t have a good place to hold them. Otherwise, if you’re just investing in a global stock fund, you know, Vanguard’s Global, VT global stock exchange traded fund, I think that’s just dandy. I’m happy to have people invest in that, including rich people who pay high taxes, invest in a fund like that all day long. I’m not quite as controversial.

The one thing that I do that is more controversial is that I’ve taken an especial interest in zero dividend stocks and very low dividend stocks, and I’ve been punished for doing this because it’s worked extremely well over the last decade. Why? Well, because what are some of these low dividend stocks? Google, Netflix. They’ve taken off like a rocket, and meanwhile my small value funds are suffering. They’re lagging in the market. The client’s like, oh my gosh, this guy’s a genius. Look at how well he’s done. But in fact, it could just as easily have gone the other way. I’m simply banking that, in many cases, once you have a good basic asset allocation in place, you have the right kind of exposures, for the marginal dollar, instead of buying more mutual funds that have a lot of tax consequences, you might be better off buying stock like Berkshire Hathaway. It’s almost like a mutual fund in its own right, but doesn’t pay dividend.

Now, I love dividend stocks. I think they’re great to the extent to which you need to spend money or you can house them in an IRA. But if you get a dividend that you don’t need, that’s like a realized capital gain. It’s an accounting identity with a realized capital gain with zero cost basis, which is the worst kind you can have. The expense of accruing dividends in taxable accounts year after year for high income people can subtract a lot of value, so I’m very sensitive. My heart goes out to these people.

I’m trying to do everything I can to make that tax bill as low as possible, especially because, as I’ve seen, I can’t control investment returns. I can probably have a global asset allocation, and I can hope to get the rewards from that, but it’s out of my hands. I thought small value stocks were going to do great for the last 10 years and they haven’t done that great. They’ve lagged. Just a simple index fund. The extent to which I can, I really try to control the taxes where I know what I’m doing, and I know that, if I do this, I’m going to cut people’s taxes by this amount, where if I invest their account differently, different kinds of asset classes, maybe it’ll turn out great, maybe it won’t turn out great. The tax focus gives me an illusion of more and better control. It’s something that really pays off, especially for people who are tax sensitive.”

 

Now, admittedly, a portfolio of individual zero dividend stocks is going to be slightly more tax efficient than buying the Vanguard Total Stock Market Index Fund, which does pay out about a 2% yield each year. How many of these stocks do you have to buy before that uncompensated risk you’re taking on buying individual stocks can be lowered enough that it is lower than the cost of paying that additional tax on that 2% yield?

“Well, let’s just take the S&P 500. Let’s say there are 500 stocks in that group. Of those, you’ll probably find that, I don’t know, 300 to 325 of them pay a dividend, so those are out. Then you’re left with this other portfolio of maybe 150, 175 stocks, and that portfolio historically has done better after tax than if you owned all of them and paid taxes of 15% on your dividends and capital gains. That’s the good news. In practice, I don’t own all those stocks. I probably own 50 of them that are sort of a sampling that I think are good ones to focus on. That’s the goal, but historically, you’ve been better off right from the gate doing it. Then there are other things people can do with these zero tax or super low tax portfolios. The most obvious one of which, or not necessarily obvious, but the most conspicuous is you can take a portfolio loan against it.

f you’re a rich person, you’ve got $50 million, and you own a bunch of zero dividend stocks, you own Berkshire Hathaway, you own Google, you own Amazon, and you want spending money, well, instead of selling those and paying capital gains, and those capital gains could be pretty significant if you’ve owned them for a while, what you do is you just borrow against it. You take a portfolio loan, and the portfolio loan, which comes to you at a very low rate because it’s totally secured by the liquid assets underlying it, that’s not a taxable event, so you have to pay a little interest on the loan, but there are no taxes because the asset is offset by a liability on the other side. There’s no net income. It’s not an income producing event. You live off the income for 20 years, and then you die, and your heirs get a step up and basis, and then the next generation does the same thing. That’s one way that ultra high net worth live and pay less taxes than doctors do.”

You just have to weight the interest costs against the tax costs. Take whichever one’s lower.

While we were on the subject of asset allocation, one of my favorite quotes of Phil’s is, “Even if risk tolerance existed and could be measured accurately, why would it be an important factor to consult when considering how to invest? You should invest in the way that has the greatest prospect to fulfill your investment goals. That might mean taking more or less risk than you would prefer. If you are a sensitive soul who can brook no paper losses, the solution is to get a grip, not to invest safely if that locks in running out of money when you’re old.” What does he mean by that quote, and can people really buck up and tolerate a really volatile portfolio just like that?

“Probably not. It’s probably just technically correct but completely useless advice. On the other hand, when I was in high school and I broke my ankle when I was playing football, and I went to the emergency room, the doctor did not ask me about what is my cast-wearing tolerance. He didn’t say, “Well, Phil, how do you feel about wearing a cast for the next month? Would that be bad? Would that make you uncomfortable, make you feel bad?” He didn’t even ask that question. He just put a cast on my leg and basically said, “Deal with it, kid,” you know. I think that, in a way, that’s the correct approach. If there’s something that’s this important, it would be better if we could learn to deal with it.

Now, and part of the problem here, the SEC says that advisers need to analyze their clients’ risk tolerance before putting them into a portfolio, and this makes sense to some extent because you don’t want people to just sell everything the first time the market drops 3% in one day. It’s still a problem because there’s no way to actually judge a person’s risk tolerance in advance.”

Passive Investing vs Active Management and Taxes

Phil says in the book that everything you’ve heard about how passive investing beats active management goes double after tax. I asked him to explain what he means by that.

“It’s almost a miracle, it seems to me, because the best portfolio is what might be called the global market portfolio, the portfolio that owns everything. It owns every stock in the world, every bond in the world, every publicly traded asset class that you can invest in that have market cap waiting. This is the ideal portfolio, and it turns out you can buy that much more cheaply than you can by hiring active managers to try their wizardry to do better. This is what you might call the ideal Boglehead approach.

Well, it turns out, almost by coincidence, that that exact portfolio is also comparatively cheap to hold, at least on the stock side, because it has very low turnover. Whoever at Standard & Poor’s, when they get together every year to decide on what stocks are going to be in the index, it doesn’t change the index very much. They get rid of a couple of dogs, they hire on a couple of new stocks, so it’s very low turnover, very low, cheap. It’s a cheap portfolio to buy, it’s cheap to hold in terms of your expense ratio, and it also turns out to be very cheap in terms of your tax cost ratio. This is just a win, win, win, as far as I’m concerned.”

Mutual Funds and Taxes

I love mutual funds. They provide liquidity, they provide professional management, they provide ready diversification, but they have one kind of nasty quirk. They like to pass along any capital gains that they generate buying and selling securities in the fund to their investors. Yet, by law, they’re not allowed to pass along their internal losses to the shareholders. You never get a deductible loss you can use on your taxes from your mutual fund. I asked Phil does he think that’s fair and does he think it’s ever likely to change?

“No, it’s certainly not fair, and it’s also very unlikely to change. I remember when Eugene Fama, the Nobel Prize winner, wrote a editorial in the Wall Street Journal, along with Ken French, complaining about this 20 years ago, and nothing ever happened. I think the people that would benefit, just fund shareholders, are too dispersed, and they don’t have any political clout. There’s no constituency saying, yes, we must do it, and then now the problem is that this issue with the traditional open-ended mutual funds has been sort of bypassed by the exchange traded funds, which don’t have the same capital gains pass through problem. They’re able to diffuse their capital gains in the market. It’s the way they trade.

I was talking to an investor the other day. He says, “So what do we do? We’ve all got these huge holdings in mutual funds that we’ve built up over the last 20 years. We have enormous embedded gains, and we can’t get rid of them. They’re a monkey strapped to our back. Every year they get all these dividends, we get all these capital gains we don’t want. Help. What can we do? We’re all in this boat. We all wish we had exchange traded funds where we weren’t victim to this same process, and now what’s going to happen is that even these exchange traded funds are going to be bypassed by something that’s now called … It’s just coming along. They call it direct indexing.”

But I think the better way of thinking about it is what I wrote about in The Affluent Investor book, which is that, given computerization, it’s now comparatively easy to just take a look at a client, take a look at what’s their tax situation, what’s their personal human capital like, what industry do they work in, how secure is their career, and then to basically custom tailor a portfolio of individual stocks, and bonds and whatever, around that person, giving them, in effect, the holdings of the global market portfolio, but correcting for the tilts and the biases that their own happenstance circumstances, you know, I’m a doctor, and I live in Houston, and I own a million dollar house there.

I mean, they can correct for all of this stuff by counter-weighting the rest of the portfolio so it’s not distorted in the same way. All this can be done. Computers can easily handle the task, and I’ve been waiting for a decade or so for this to happen, and it’s only now just beginning to come on the horizon, but when this hits, and it’ll hit in a big way, suddenly we’re going to be saying, wait a minute, we’ve still got these mutual funds. I also want out of these exchange traded funds. Help get me out of here. Get me into this new, better system. It’s like the Apple CarPlay, and not like the old 8-track that I’ve got in my old car, you know. I think that’s what’s coming.”

Biggest Investment-Related Tax Mistakes Investors Make

I asked Phil what are some of the biggest investment related mistakes that he sees investors making.

“Probably the worst one is that investors have somehow got the idea that it’s important to realize their gains. If a stock is up, okay, I made 10% on Amazon, so it’s time for me to sell it and buy something else, and then, at the same time, they say, oh, I bought that marijuana stock, and it thought it was going to do well. I was certain it was, but now it’s down 20%. I know, at legalization, it’s going to all come back, so I’m just going to wait. I’m going to hold that, and, of course, this is just psychological manipulation. Basically, people should be doing the exact opposite. They should be holding on to anything they’ve got with a gain, not realizing it, just letting it run, but their losers, they should be selling almost immediately, and banking the tax loss from it, and using it against ordinary income or to offset other gains they’re getting. That’s just an obvious thing, but people invariably get it wrong, so keep your winners and sell your losers, not the other way around.”

The other mistakes he sees are people not funding their retirement accounts to the fullest extent and not tax loss harvesting. He thinks tax loss harvesting is oversold but under practiced.

“That’s because tax loss harvesting, if you’re going to really do it well, it requires searching through all of the individual tax lots of all of your holdings. You just look at the mutual fund. You know, I own the Fidelity Magellan Fund, and I’m up 10% at that, so it’s great, but in fact, if you’ve been reinvesting dividends and capital gains in that fund for 20 years, then you actually sort through all of the underlying lots that you own, some of those owns are going to have done great, and some of the lots you bought at the wrong time, and they’re going to have done badly, so you should always comb through, sell whatever is down, and keep whatever fits within your investment strategy.

You know, it’s nice. The $3,000 a year deduction from ordinary income is pleasant. It’s not exactly going to make you rich, but the real benefit is if you can use tax loss harvesting to be able to draw down from a taxable account, say, during retirement, for a number of years without realizing capital gains, and eventually shove all of your capital gains into an account that is left in your estate, where it gets a reset to fair market value, and that’s of course a great thing.”

Family Loans as an Advanced Tax Trick

Phil has discussed family loans before as an advanced tax trick. I asked him to explain how that works to reduce your taxes.

“Family loans are basically about wealth transfer. If I’m from a wealthy family, and I have some kids, and let’s say I’ve got an extra couple of million dollars just sitting in the bank, what might I do with it? Well, if I invest that money myself, then the money’s going to grow, but then all that growth is going to be part of my estate, so it’s all going to be taxed at some point, assuming I have an estate that’s big enough to be taxed. It’s going to be a problem in some way. A better way of doing that would be to loan the kids a couple of million dollars, and let the kids invest it in some way. Let them use it to buy a house, use it to buy an investment portfolio, use it to start a business of their own, and then have them pay me back just a rate of interest on the loan.

Now, the other good thing about this is that, when you do this, you can use what’s called the applicable federal rate, which is actually a sub market rate of interest. Today, I don’t know what it is. Probably a little over 2%, which means that the kids could pay you back. If your kids pay you back $15,000 a year on the loan, that would cover a pretty good amount, and plus, while you can’t forgive that amount, you could separately, completely unrelated to anything else, this is a totally different conversation now,  you and your bride can give the kids and their spouses, $15,000 a year just as a gift, and if they happen to pay you back a check that may be something that’s about that amount in relation to servicing the loan that you’ve given them, then that’s all.

The main thing, though, if you try this, is that it has to be formally set up. It’s got to be for real. The IRS, when they look at family loans, they say, “Oh, Dr. Jim, very nice of you to do this. I see you made a two and a half million dollar gift to your son. That’s very nice,” and you say, “Wait, wait, excuse me, that wasn’t a gift. I’m loaning him,” and they say, “Prove it.” At that point, you want to be able to say, “Well, here’s the documents. Here’s the loan document. Here’s the collateral. Here is the checks he’s been paying me every month on this. This is a legit transaction. This is not a gift. This is a loan.” You have to be able to document that it’s really there. But if so, it’s a nice technique. It’s a nice way to help your kids if you have the money to do it.”

It is far more than $15,000, too. If there’s two spouses, your kid, and your kid’s spouse, and you and your spouse, we’re now talking about $60,000 in gifts that wouldn’t be included in the gift tax. That covers an awful lot of interest on a pretty large loan at 2%.

Tax Tactics

We talk a lot about tax tactics on the blog and podcast but we have never ranked them. I asked Phil to look at all of these tactics like maximizing the use of retirement accounts, using tax efficient investments in your taxable accounts, getting your asset location right, making sure your dividends are qualified and that your capital gains are long-term, depreciating real estate, tax loss harvesting, donating appreciated shares to charity to flush those capital gains out of your portfolio, getting a step up in basis at death, etc.  If he had to rank these, which two or three would he put at the top of the list that gives you the biggest bang for your buck? If you’re one of these 80/20 kind of people, which of those do you need to get right in order to get most of the bang for your buck out of investing tax efficiently?

“Unfortunately, each of them make a small positive contribution, so they’re all worth doing, especially keeping the turnover of your accounts low is extremely important in getting rid of these gratuitous capital gains. If you can minimize dividends where possible, that’s a great thing to do. They’re all good. I think that there are a couple of things that are even beyond that list that are worth paying close attention to, and that has to do with the fact that there are a couple of periods in your life where you can make a big difference to the overall curve, your overall financial curve, if you take advantage of them when you’re there, and they’re both periods when you’re likely to be relatively low income, and so you’re going to have more options.

One of them would be when you’re starting out in your career. Very early stages. You’d have opportunities to invest to the extent to which you can save any money. That’s incredibly valuable, because it gives you the gift of long-term compounding, you might have a chance to open Roth IRAs and be able to do long-term compounding tax-free. That’s extremely important. In fact, I’ve come up with a phrase. It’s called live like a resident. I’m going to trademark this because I think it’s so cool. Actually, I got that from you. But if you can do that for just a little bit, and get your financial ducks in a row early, and put more money away in savings, and have that roll through your lifetime, that’s going to make a difference between night and day for you.

The other interesting time is after you retire but before you have to start taking required minimum distributions and social security. This is a time when you can perhaps do Roth conversions of your IRA, and really try to optimize your whole tax picture in retirement, which, again, could be tremendously beneficial, especially if you also look through to the next generation, and figure out, well, who am I going to ultimately leave these assets to? What’s their tax bracket? And try to arrange the whole thing so you’re paying as much of the money as possible, paying as much of the tax as possible, in as low a rate as possible.

I think that that’s very important to take advantage of if you can do it, and it’s worth getting some help. If you’ve never used financial planners at any other time in your life, it’s worth talking to them when you’re starting out, and I think it’s worth talking to them when you’re on the cusp of retirement, to figure out a game plan. You don’t have to stick with them forever, but at least try to get everything set up correctly then, because it could really pay off.”

Secure Act in Congress

In Congress right now is the Secure Act, and I’m hearing a lot of politicians saying they think it’s going to pass, at least in some form. Phil recently wrote a very widely-read article about that. I asked him to talk about the basic changes that will occur if it passes and how much they really matter?

“The Secure Act is a bunch of different provisions that affect our retirement plans, and it passed the house 417 to three. It was slated to pass the senate, and I looked at the provisions, and I sort of analyzed them, and I said this is horrible. This is a nightmare. So I wrote about it for The Wall Street Journal, and then The Wall Street Journal, bless their souls, on their editorial page the following week, they basically just said that, “Yeah, we think this is a bad idea too.” Simultaneously, I emailed all of my clients, everybody I knew, to please contact your senators immediately. I sent their senators email contacts. I said, “Tell congress to keep their hands off of your IRA.”

What this bill does is it pays for itself by taxing our retirement plans when we leave them to our kids. Inherited IRAs. As it stands now, your kids can stretch out the payments over their actuarial lifetimes, and this is great. But the new plan is going to be, oh no, the kids have to take the money out in 10 years, and once your kid inherits a substantial IRA, that’s going to be a lot of money they’re pulling out over those 10 years, which means about an extra third of it is going to be taxed. If your kid happens to have kids that are going to college of his own, and needs to apply for some kind of financial aid, it’s going to completely screw up the financial aid package, so their expected family contributions is going to be much higher. It’s going to either go to the government or some college administrators. It’s not going to go to your kid’s retirement.

The fact that this bill could have gotten so far, it was just sailing through, no discussion, no debate, I was just completely … and I am still baffled. I understand that the bill is being promoted by the insurance industry because one of the provisions is that all of these retirement plans, your 401K, your IRAs, they’re going to have to have an annuity as a potential offering, so the insurance companies love this. But the financial press is pretty much asleep at the switch. I’m just astonished that nobody has picked up on this and what a terrible consequence it is, and of course the people it hits the worst are not … it doesn’t hurt the 44% of people that don’t pay any federal income tax. They’re not bothered. Really rich people aren’t bothered by this either, interestingly, because they don’t have very much of their wealth in retirement plans. They’re all in these huge taxable accounts.

But who is hurt by it are the upper income, people that have big salaries and have big IRAs. There’s another word for those people. Those are physicians and professionals. Lawyers, doctors, CPAs. These people are all going to just be hosed by this, and everybody thinks it’s great. Yeah, let’s have them pay for it. But there’s no constituency that’s represented these people in saying no, let’s not do this. AARP. Well, they get half their funding from the insurance industry, so they don’t think it’s a problem.

My suggestion for the listeners is that they’re going to try again this fall. We got it derailed for the summer. They’re going to try again to get it passed. It’s going to probably be attached to a spending bill, and if they can succeed in doing that, those bills always pass. This is going to be the new order. If you want to have a third of your money from your IRAs going to the government, that’s fine. Do nothing. It’ll probably happen. If you’d like to have that money go to your families instead, I would encourage you to just Google “contact my senators”, send them an email saying please do not vote for the Secure Act in any form. Have the government keep their hands off my IRA, because I’ve been planning on this for the last 20 years, and now you want to change the goal posts in the last minutes of the game. It’s not fair.”

Now, a lot of people would criticize that argument and say hardly anybody’s stretching their IRAs longer than 10 years anyway, and most people don’t have much of an IRA to start with. If you look at the average IRA or 401K balance in this country, it’s pitifully low, and most of it is spent in the first half of retirement by the typical middle class folks, so that’s kind of the criticism, is that this is a way to tax the rich, if you’re taxing anybody, but that really nobody’s stretching these things out anyway because the heirs can’t seem to keep their paws off the money longer than 10 years. How many people is this really going to affect?

“If you have a million dollar IRA, or a multimillion dollar IRA, that’s going to be a problem. Your million dollar IRA, when you’re 65 could still be a million dollar IRA after withdrawals at age 85, and so that’s going to your kids, and where it’s going to be quickly dispersed and taxed heavily. I think The Wall Street Journal’s editorial suggested that maybe one in every four dollars in retirement plans would be affected by this, and in fact, I downloaded the government’s own paperwork, own financial analysis, and what I realized is the way they sold it was that the bill would pay for itself over the course of 10 years. But if you extrapolate the cost versus the income, the new revenues they’re going to be bringing in, every year after those 10 years, the gap widens hugely, so by 2050, the government is going to be taking in an extra $4 billion a year in tax revenue from our IRAs and using it for its own secure projects, and this has completely screwed up, again, our ability to make any kind of financial plans. This is not the deal that we were promised when we were told we could contribute to these tax-free accounts.”

He is obviously clearly against the limitation on the stretch IRA provision. It doesn’t sound like he is very happy about the provision that requires an annuity option to be in the retirement accounts, as it sounds like a big boon to the insurance industry. There’s one other provision that’s pretty widespread and well-known about it, and that’s increasing the RMD age to 72, maybe 72 and a half, I don’t recall. I asked if he sees that as good, bad, indifferent, doesn’t matter?

“I’m all in favor of it. I would be happy to extend the RMD age any chance I can get. What I found so bitterly humorous is that there’s no possible way of extending the IRA RMD age from 70 to 72, which, by the way, for some people that’s one year, for other people that’s two years. It depends on which month of the year you were born in whether or not you get to collect a one year or two year benefit. But that somehow compensates for getting the entire IRA ultimately taxed over a 10-year period when it’s inherited. It’s just a complete joke.

But to get back to the insurance companies for a minute, my favorite provision in the bill is there’s a safe harbor given to the plan administrators. They don’t need to worry about making sure that it’s the lowest cost annuity that they offer. They can offer the highest price annuity if they think that’s what’s going to be in the best interest of their plan holders. I can just imagine the lavish seminars that are going to be held in Hawaii, where the insurance industry invites the plan sponsors to come and spend the week, and will spend a couple of hours a day talking about investing, then they’ll play golf, then go to the beach. It’s going to be a great time. It’s a great time to be in a 401K plan sponsor. You’re going to have some real benefits.”

Ending

Phil is writing a tax book currently that he is trying to figure out how to put the Secure Act in. Then he plans to write about charity, the last big wealth management topic he hasn’t covered. He wants to write a book about how to be a more effective giver when you do give to charity.

I, of course, love Phil’s books but he also wanted to recommend a few others to you.

Marty’s book is about how to fix the healthcare system. Phil thinks that doctors are getting hurt by association with the healthcare system and we should get out in front of how to fix the healthcare system, rather than just be bystanders while it gets fixed elsewhere.

He also wants to point out that doctors live rich, but they don’t die rich. They mistake having a big salary for wealth, and this is the problem. “You want to think about, when you go to sleep at night, how can I transform this high salary into a form of wealth? How can I transform my practice into a business? That’s the way that your family is going to benefit most.”

Full Transcription

Intro: This is the White Coat Investor Podcast, where we help those who wear the white coat get a fair shake on Wall Street. We’ve been helping doctors and other high income professionals stop doing dumb things with their money since 2011. Here’s your host, Dr. Jim Dahle.

Dr. Jim Dahle: This is White Coat Investor Podcast number 126, reducing investment taxes with Phil DeMuth. Thanks so much for what you do. I think you’re probably on your way into work, or you’re out walking around the neighborhood with the dog, or maybe you’re on your way home after a difficult shift or a difficult day in clinic, and if nobody told you thanks today, let me be the first. Be sure you check out our YouTube channel. You can find that at youtube.com/thewhitecoatinvestor. If you like the video format, it’s a great one, and we’re trying to boost the amount of content we put on there every month. We’re going to try to put a little bit more short-form content that’s just three to five minutes long rather than some of the longer stuff we’ve put up in the past, but be sure to check that out.

Dr. Jim Dahle: You can also join the White Coat Investor private Facebook group. That’s found at www.facebook.com/group/whitecoatinvestors, because, remember, you are the white coat investors, not me. I suppose I’m one as well, but the whole point of calling this whole enterprise the White Coat Investor was not that I was the white coat investor, but that you are the white coat investors. Our quote of the day today comes from Peter Lynch, who said, “I can’t recall ever once having seen the name of a market timer on Forbes annual list of the richest people in the world. If it were truly possible to predict corrections, you’d think somebody would have made billions by doing it.” I think that’s absolutely true. That’s a great quote from Peter Lynch, reminding us of just how difficult it is to time the market.

Cindy: Cindy here, the White Coat Investor business manager, to introduce this episode’s sponsor. Contract Diagnostics is a long-term advertiser here with us at White Coat Investor. In fact, they’ve been partnering with White Coat Investor longer than I’ve been here working with Jim. Jim loves this company as they’ve helped hundreds of White Coat Investor readers and listeners get a fair shake when it comes to reviewing and understanding their employment contracts. It’s 100% what they do there. All contracts are reviewed by an in-house attorney and presented in a simplified way back to you. Using custom documentation, compensation data, and times from 6AM to 8PM, they make it easy for you. All packages are flat price, so you know what you will pay up front. Residents and fellows can even make interest free payments over time. They have a new compensation review offering coming soon for those of you happy at your positions as well, so look them up at contractdiagnostics.com, or 8885745526. That’s contractdiagnostics.com, or 8885745526.

Dr. Jim Dahle: We have a guest today, Phil DeMuth, who will also be speaking at the 2020 Physician Wellness and Financial Literacy Conference in March in Los Vegas. Phil’s the author of 11 books, I believe, although I had a hard time counting them, there were so many, primarily about investing, including The Overtaxed Investor and The Affluent Investor, which I’ve reviewed on the blog over the years. He is the managing director of Conservative Wealth Management, he has a master’s in communication and a PhD in clinical psychology. He has described himself ion emails to me as your slavish follower and devoted fan, and I admit I am a sucker for his highly skilled flattery, so here he is. Welcome to the White Coat Investor podcast, Phil.

Phil DeMuth: Dr. Jim, thanks so much for having me. It’s great to be here.

Dr. Jim Dahle: I really wanted to focus in today on taxes, especially investment-related taxes. In fact, that’s what I titled this podcast. But before we get into that, something I do with all my guests, I wanted listeners to get to know you a little bit. Can you tell us a bit about your background, what your upbringing was like, your education and your career?

Phil DeMuth: Sure. Well, my superhero origin story is … it’s not as good as batman or Superman, but for what it’s worth, I grew up in the Midwest, in the suburbs of Chicago. I ended up at the University of California at Santa Barbara, and finally, I took my PhD in clinical psychology. I started my career as a psychologist, and the most relevant thing here for your listeners is that, for about 10 years, I was with the division of behavioral medicine at the Mount Sinai Medical Center in Cleveland, Ohio. I was actually a wearer of a white coat, and not only that … I understand now that there’s a pecking order of white coats within a hospital, or at least there was back then, and I got to wear the same white coat, probably by mistake at the laundry department, that the actual physicians got to wear. I had the early experience of being a fake physician, but actually just a clinical psychologist, working in a hospital setting, and got to know a lot of doctors in that context.

Dr. Jim Dahle: You know, it’s interesting about the white coats. In some facilities, none of the doctors wear white coats and everybody else seems to, so it definitely gets confused, although traditionally there’s a hierarchy, and it’s all about the length of your coat. You get a really short one when you’re a medical student, then you become a resident and it’s a little longer, and the attendings have longer ones, but I think that’s much more common on the East Coast than it is out where I’m at.
Phil DeMuth: Fascinating. Well, eventually, I got out of the psychology gig because I wanted to write, and I ended up in Los Angeles. My good friend there, Ben Stein, we used to have lunch all the time and talk about the stock market ad nauseam, and finally Ben said, “We should write this stuff up because it’s good enough to be published.” he said. I said, “Great, but I don’t know anybody in the publishing world,” so we just sent off an email, and 10 minutes later we had a book contract, so we were off to the races with it.

Dr. Jim Dahle: It was that quick, huh?
Phil DeMuth: In this case, it went fast, which is I guess what happens when you’ve got a celebrity you’re working with. Then I thought, well, I’m writing this books. It’s really for my own benefit, but it would be just as easy, he said to himself, to just take other people along on the same investment journey that I’m on, so I’ll just hang out a shingle as an investment adviser, since I’m already doing everything, I thought, and so I started that as well, and I found there was actually a little more to it than just hanging out a shingle. It’s an SEC-regulated business, so there’s lots of hoops to jump through, but it’s gone well. It’s been a lot of fun.

Dr. Jim Dahle: Was your background kind of relatively impoverished, or middle class, or fairly well-to-do, and how did that affect your views on money as you grew up?
Phil DeMuth: Well, I grew up in the [inaudible 00:06:42] suburb of Chicago, which today would probably be considered well-to-do, and even back then, although, to tell you the truth, the suburb I grew up in is a lot nicer-looking today than when I grew up. I mean, when I grew up, people didn’t have color TV. They didn’t have air conditioning. It was sort of a log cabin, but I’m from a relatively upper middle class background, I’d say. My first summer job was working on south street in Chicago. I was a messenger.

Phil DeMuth: I was just carrying briefcases full of securities up and down the street from one firm to another, and it occurred to me recently that what I probably should have done was just taken those suitcases full of securities, got on a plane, and moved to Uruguay, and set up a mountain retreat there, but I wasn’t smart enough to be a financial criminal mastermind at the time, so I didn’t do that, and just ended up going along. But I think it’s given me a certain level of comfort, probably, dealing with upper-income individuals that I might not have had if I came from a different socioeconomic background.

Dr. Jim Dahle: Now, you’ve written a bunch of books. How many is it exactly? Is it nine, or 11 or how many?

Phil DeMuth: Well, I don’t actually know. I think it might be 11 by now. A couple of them are books on politics, from sort of a right-wing point of view. In my case, probably more a libertarian point of view, that Ben had suggested, that I was happy to piggyback on with, but the rest are really on different topics in finance that struck my fancy along the way.
Dr. Jim Dahle: Which one of these books is your favorite, which one sells the best, and which one do you think is the most important?

Phil DeMuth: Well, all of them, of course, but my favorite is what I’m working on at the time. Lately, I’ve just been obsessed with taxes. I pick some topic that’s an area that I feel I should know more about, and I use it as an excuse to do a deep dive and learn everything that I feel I ought to know as a financial adviser working with people. The other books are on the shelf, and I use them as references, but what I’m learning about now is always the most interesting. The ones that have sold the best … I think the political books have probably sold better. I don’t know why. Just I guess there’s more interest in that general area, although our first book we wrote together spent some time at number four on Amazon, which is a pretty high ranking in the Amazon world, until it got knocked off by a book about some boy wizard. That was the end of that.
Dr. Jim Dahle: That’s a bit of a claim to fame, isn’t it, you know? We got creamed by Harry Potter, you know?

Phil DeMuth: Yeah. That’s what happened, but it would have happened one way or another anyway.
Dr. Jim Dahle: If you look back at that body of work you produced, which one of those books do you think is the most important?

Phil DeMuth: Well, I think the one that summarizes everything the best is probably The Affluent Investor. That’s a more general, everything I know book in 150 pages. Looking back, I think that some of the other books have held up pretty well, even the book on retirement planning. It was the first book. It’s a book that I used to talk about sequence of return risks. The portfolio it suggested was a very successful portfolio, it turned out, for retirees to own, and it had different roles for drawing down the portfolio that also proved to be quite good. Even though it was written in 2005, I’m surprised at how well it’s all held up.

Dr. Jim Dahle: All right. Let’s talk a little bit about doctors, since most of our audience is doctors and similar high-income professionals. A lot of your clients are doctors. What do you think is unique about the financial life of a doctor?

Phil DeMuth: Well, it’s a tremendous puzzle, this whole thing with physicians, because, supposedly, in theory, doctors should be fantastic investors. In theory, there should be no White Coat Investor website, because they should never have gotten into any trouble in the first place, because they’re super smart, they know how to make cost-benefit decisions, they know how to do risk analysis, they know how to live with consequences. They’ve got all the right skills to be great investors, but then, in practice, what you find is that a lot of them are lousy investors. This is one of the great mysteries before us.
Phil DeMuth: I suspect that it has to do just with the fact that doctors tend to be extremely smart, and that means that they can also be somewhat easily bored and restless, and then they look to, in the financial world, they look to Jim Cramer, they look to CNBC, as being a source of infotainment and investotainment, and then so all of a sudden they’ve entered this whole area through the wrong door. They’ve entered it through the casino door, and it’s a terrible approach, and you can lose a lot of money or spend a long time before you find a way to make a lot of money when you start out that way.

Dr. Jim Dahle: Given those habits of doctors and those natural inclinations, do you think it’s reasonable for any doctor to be a do-it-yourself investor, and if so, what percentage of them do you think can do it well enough that they should not hire a financial planner and/or an investment manager?
Phil DeMuth: Again, it’s a very tough call because, in theory, especially on the investing side, they can do it very easily. Every doctor could buy a Vanguard target date fund targeted for the year they plan to retire, and that could take care of their entire investment life, up to a point, anyway. They could do great doing it, and yet I’m aware that Dr. Bill Bernstein, who I think very highly of, he thinks that doctors, maybe only one or 2% could do it because there are temperamental issues with it all. But that’s not to say that I have a great endorsement for the profession of registered investment advisers or financial planners. It took me years to find somebody to do a succession planning with my business. They said, oh, just go to a conference and have drinks with somebody, and your succession plan will be taken care of. It took me seven years to find somebody that I felt comfortable handing my clients’ affairs over to if and when I get hit by a bus or retire.

Phil DeMuth: I think most financial advisers, they charge too much, they don’t really deliver a lot for what they do charge. I think that you have to choose somebody very carefully in that regard, and then it’s also just a question of do I want to spend a lot of my time learning how to manage money when my alternative use, being a physician, is very highly paid? It’s a tough trade-off, and there’s a real opportunity cost involved. I don’t think there’s an easy answer there.
Dr. Jim Dahle: Yeah, there’s definitely a balance for sure. Some people do very well, but part of the difficultly, I think, is even a doctor that’s relatively highly paid looks at what they’re paying for investment management, financial planning, et cetera, and realizes that that is the highest paid use of their time, you know?
Phil DeMuth: Yeah.

Dr. Jim Dahle: It may be the equivalent of four or $500 an hour, or a thousand dollars an hour for them to learn how to do this stuff themselves, and I think that provides this pretty significant motivation. As I think one of my readers said, it’s the best-paid hobby out there, and if you hate it, it’s the best-paid chore out there, you know? It’s just so expensive to get that it’s almost like you feel like you have to learn it, but obviously some people still choose to outsource that part of their life.
Phil DeMuth: Right. That’s absolutely correct. It could be a very highly paid hobby if you have the temperament to be disciplined about it. If you don’t have the temperament, the problem is your college buddy comes to visit with his wife, and she’s driving a Mercedes and your wife’s driving a Honda, and then they leave, and your wife says to you, “I hear that he’s been investing in marijuana stocks, and he’s really making a lot of money. Why aren’t we doing that?” You’ve got to be able to stand up to these kind of situations when you see other people who are idiots doing better than you are.

Dr. Jim Dahle: All right, so you’re giving one of our keynote talks at the White Coat Investor conference next March. What should people expect to hear from you there?
Phil DeMuth: Well, you’re not going to like this, but, basically, what I’m going to be telling your audience is that they are ducks in a shooting gallery. These people are high income. Their only real tax shelter is their retirement plan, which is another golden goose, and they’re just moving along, and the IRS just sees people like this, congress sees people like this, and they just pick them off one by one. They’re the most vulnerable people in terms of taxes. You’re in a very tough spot. They’ve got to defend themselves as best they can. I see you also put me on a college funding panel. I’ve been doing some research on that, and I think I have a couple of ideas there that I haven’t seen commonly discussed, in terms of ways of optimizing paying for college. I think it’s going to be very, very interesting. I’m thrilled that you invited me.

Dr. Jim Dahle: Yeah. We’re excited to have you. It’s going to be a great conference. We got all kinds of people. For those who are just now hearing about this, I’m very sorry. It sold out in 22 hours back in July. There is a waiting list, and we have figured out a way to take about three quarters of the people off the waiting list, and then we’ve got a couple of people a week that are just having to … they’re not able to come, and so they’re canceling, and we’re replacing them off the waiting list. There’s still a possibility you could come if we go through enough of that waiting list. The registration is found at whitecoatinvestor.com, so feel free to get on the waiting list if you’d still like to come to it. It’s March 12th through 14th in Los Vegas at the Paris Las Vegas Hotel. We’ve got a lot of great people like Phil DeMuth speaking there. It’s going to be great.
Dr. Jim Dahle: Now, Phil, when you sent me your W9 for your conference speaking fee, you employed a rather interesting tax reduction tactic. You didn’t include your social security number on the form, and I’m going to get as much mileage out of this incident as I can. Now, obviously that was accidental, but do you have any advice to my listeners about how to make sure that the tax reduction tactics they are employing are tax avoidance and not tax evasion?

Phil DeMuth: Yeah. Well, and again, for some reason, this seems to be an especial weakness of the physician class, because doctors seem to always be interested in the tax angle, probably because they’re so incredibly taxed to begin with, it’s a sensitive subject. Again, if it sounds too good to be true, it almost certainly is too good to be true, and if the tax picture sounds bad, if your accountant says, “Gosh, you’re paying a lot of taxes this year,” he’s probably right. If you’re going to be suffering, that’s a correct perception of reality.

Phil DeMuth: On the other hand, the good news is, I tell my clients, you’re never going to be audited, because they’re going to look at how much you may, but then they’re going to look at how much taxes you pay and think this guy is fine. He’s suffered enough. Anything that promises to be a tax haven, anything that takes your money offshore, anything that’s … if you’re going to be trading bitcoin in a foreign account and not telling the IRS, these are going to get you into a lot of trouble, because the IRS, they don’t care about any of this stuff. They look at facts and circumstances. They are willing to undo any series of step transactions you’ve done to try to hide money from them, so don’t mess around.

Phil DeMuth: If you’re ultra high net worth, you can fight the IRS. You can say, “Oh, yeah, I got your letter. Deal with my tax attorney. See you.” But if you don’t have $100 million, you’re not in that position. If you have $5 million, and you say to the IRS, “Oh, yeah, deal with my tax attorney,” well, pretty soon you’re not going to have $5 million. That money’s going to all go to your attorney and paying fees. You want to color within the lines, you want to give the IRS a wide berth, because you can’t avoid to fight them. They’ve got more power than you.

Dr. Jim Dahle: Now, let’s talk about your newest book, which I love, The Overtaxed Investor. As I was thumbing through it again in preparation for this, I stopped at the opening anecdote, which has you talking to a client with $100,000 in capital gains, who paid zero dollars on those gains in tax. I found that intriguing, as that was not really my understanding of how the capital gains tax brackets worked, so I actually was worried. I was confused about it. I went through the instructions by hand, and it seemed there was no way to get out of paying taxes on $100,000 in capital gains. There was no way to get them completely tax free unless you had some other taxable loss out there to offset it, so in 2019, if you were married, filing jointly, and you had $78,700 in taxable ordinary income, and $100,000 in capital gains, you pay 15% on essentially all of those capital gains, despite being in the 12% ordinary income tax bracket. The anecdote seems to suggest that you pay 0% on those capital gains. Am I missing something there, or was that suggesting something that maybe shouldn’t be there?

Phil DeMuth: Well, it’s a good question. This emerged out of a conversation I had with a client. As far as I know, he’s not in jail at the present time for tax avoidance, and it’s the one conversation that got me intrigued in the whole area of, oh my gosh, what don’t I know here that I ought to know? But, in theory, as I think about it, if he’s got a $78,000 of overhead before he gets into a bracket where he has to start paying taxes, but he also has … he’s married. He’s got a $24,000 standard deduction, and then another $2,600 because he and his wife are over 65, then that puts you over the $100,000 mark. TurboTax and the TaxCaster, it doesn’t show that there’s any tax liability, but it’s not my tax return, so I haven’t analyzed it beyond that.

Dr. Jim Dahle: Okay, so basically, this person had other deductions that lowered their ordinary income tax.

Phil DeMuth: The standard deduction that every couple would have been enough to do it, as far as I can see.
Dr. Jim Dahle: Yeah. Now, in the first chapter of the book, you list all the taxes we pay, which is … it’s overwhelming to read through them all. There’s income taxes, and sales taxes, and gas tax, and hotel tax, and excise tax, and proper tax. What percentage of the income of a typical doctor do you think is going to taxes of some kind?
Phil DeMuth: Well, this is a terrifying question. What I have done is I’ve just noticed that Greg Mankiw, he’s the Harvard economist. He used to be head of the president’s Council of Economic Advisers, and he’s written the standard economics 101 tax book that everybody uses in college. He estimates, he’s a college professor, that his all-in tax rate is 90%.

Dr. Jim Dahle: Wow.
Phil DeMuth: This is just what he pays on his salary, dividends, interest, estate taxes, money going to his kids, and then I thought, God, that sounds high, and then I was reading John Cochrane, an economist a Stanford. I think he’s at The Hoover Institute there, and he says, well, he pays 42% federal, 13% state, 10% sales tax, 6% property tax. It goes on and on, and he argued that if, as some presidential candidates have argued, we have a 70% marginal tax rate, he said if that was the all-in inclusive tax rate for everything, that would be a tax cut for many high-income Americans.
Dr. Jim Dahle: Wow.

Phil DeMuth: It’s a horrifying figure, and of course, a lot of its disguised, especially now. Congress delegates authority to different agencies, and then the agencies set up their own tax rates and embed them into what they’re doing, so it’s a big mess.
Dr. Jim Dahle: Now, in the book, you also argued that the story of Star Wars is all about taxes. You quote the beginning scroll there in the first movie, where it talks about the taxation of trade routes is in dispute. Do you think this country can actually implement a fair tax solution without six movies of intergalactic dispute?
Phil DeMuth: It’s very important to look at Star Wars from the accounting point of view. This is often overlooked. People focus on the force and lightsabers, but actually the tax angle is a very fruitful one to follow as you watch the films. But anyway, it’s very, very problematic.

Phil DeMuth: It looks like what’s happened is we’ve moved into a situation where every eight years, or four years, even, the tax code can change when a new administration comes into town. The democrats come in, they sweep, they get congress, and they suddenly raise taxes to where they should be, so to speak, and the republicans get in four, eight years later. They lower taxes, and this is a planning catastrophe. If you don’t have a stable set of rules that you’re planning under, it makes planning … you kind of just throw your number two pencil up in the air because it might be this way or it might be that way. If we could have a steady state tax code, even if a lot of people thought it was bad, a lot of people thought it was unfair and complained about it, if it was just the same, if it just stayed the same for three decades at a time, it would make everybody’s life a lot easier, and we just lose a lot of economic value through this needless jostling back and forth, having to do our wills over, our financial plans over. It just creates a lot of chaos. It’s completely unnecessary.

Phil DeMuth: Economists feel that taxes should really be just based on consumption. A value added tax would do the least overall damage to the economy, and we could scrap everything else we’ve got going on, and then if you want to help people, at the end of the day, the congress simply says, okay, well, we’re going to give poor people money to make them less poor, and it’s just sort of a social policy. Instead of trying to use the tax code to solve 500 different problems. I don’t like it, and of course, none of this will ever happen. We’re going to continue to have this kind of a mess because it’s very useful in terms of funding reelection campaign for our representatives. That’s my cynical opinion, but I don’t think I’m wrong.

Dr. Jim Dahle: As we go into some of these other topics we’re going to cover today, I think it’s important to put this disclaimer on the beginning of it, and point out that you and I agree about 98, maybe 99% of the financial world, but in the effort to produce a more interesting and entertaining podcast, we’re going to focus a little bit more on some of the controversial topics, some of the controversial things you’ve said and written in the past, and get into some of these a little bit more gray topics where we don’t necessarily always agree, but I think it makes for an interesting conversation. I occasionally hear you argue for zero dividend individual stocks held in a taxable account. Isn’t that letting the tax tail wag the investment dog? In fact, in your book, you say your asset allocation is an illusion because most people don’t tax adjust their asset allocation, so first, you should make sure you’re not needlessly overpaying the taxes on your portfolio, and then worry about your asset allocation after that. Taxes come first. Does tax reduction really matter more than asset allocation?

Phil DeMuth: For some people, I think it does. It’s not only the problem of the fact that your asset allocation is not really right, because you haven’t figured out what everything is worth after tax and made that adjustment, but here’s another big problem. Another illusion that we have is that I think I own the S&P 500 index fund, so I’m great. But the problem is that in itself could be a miracle because then you get downturn. It’s down 50%, and you think, oh, gosh, I’ve made a terrible mistake here. I sell it. So even though a Boglehead efficient portfolio seems like the answer for 99% of the population, if not 100% of the population, in practice, many people can’t stick to it because they are closet market timers, and they also way overestimate their risk tolerance, and so they panic at the worst possible time.

Phil DeMuth: Now, the part about not owning inefficient tax assets, one of the people I was thinking about there is a group that I have a very soft spot for in my heart. The ultra high net worth. People with 50, $100 million, because what happens when you get into that group, I’m told, I’m certainly not in that group myself. I know that you are, Dr. Jim, but for the others of us who are not yet there, what happens is that once the first generation of the family who actually earned all the money in the first place dies off, then the rest of the family is relatively clueless about money, and so the investment banks prey upon these people, and they say, “We’ll take care of you. We’re going to educate you.” They pack them off to these various seminars for ultra-rich people, and at the seminars, you’ll spend Monday morning learning about stocks.

Phil DeMuth: But then Tuesday through Friday is spent learning about hedge funds, private equity, venture capital, esoteric assets, collectibles, all this stuff that is extremely tax inefficient to get involved with, for the most part, and yet they give people a kind of pseudo sense that, oh, yeah, I know about how to invest in hedge funds, because I spent a couple of hours yesterday morning learning about them, where basically they have no clue what they’re doing, they’re going to be completely reliant on multiple layers of advisers to help them, and it’s just going to be a way of milking the family fortune, and they’re going to be steered into these snobby and inscrutable assets.

Phil DeMuth: I remember one of these conferences that I went to, and I saw that Jack Bogle was speaking at the next version of this conference, so I immediately sent them an email and told them to give all these rich people hell about what they’re doing, and to just stick with a very simple portfolio. I’m just concerned that a lot of people get funneled into this stuff. They don’t really understand. They’re scripted for a dependence, and the taxes and the fees just go through the roof. Even very rich people, I think they’d be far better off if they invested in a Boglehead portfolio, and not get side tracked with all this other nonsense. That’s probably a minority view, but that’s what I think.

Dr. Jim Dahle: Now, I mean, there’s obviously a lot of money that can be saved in keeping expenses down and keeping taxes down, but I think the traditional approach has always been asset allocation first, and then you pick your investments, and then you try to reduce your taxes as best you can with tax loss harvesting and using retirement accounts, et cetera. But you seem to have that taxes are really where it’s at. That’s really where the bang for your buck is at, and your asset allocation is almost superfluous. That’s a pretty unusual viewpoint out there.

Phil DeMuth: Well, basically what it means is you get rid of holding very tax inefficient assets. Alternative investments can be very tax inefficient. You can probably live without them. REITS can be very tax inefficient if you don’t have a good place to hold them. Gold. Otherwise, if you’re just investing in a global stock fund, you know, Vanguard’s Global, VT global stock exchange traded fund, I think that’s just dandy. I’m happy to have people invest in that, including rich people who pay high taxes, invest in a fund like that all day long. I’m not quite as controversial.

Phil DeMuth: The one thing that I do that is more controversial is that I’ve taken an especial interest in zero dividend stocks and very low dividend stocks, and I’ve been punished for doing this because it’s worked extremely well over the last decade. Why? Well, because what are some of these low dividend stocks? Google, Netflix. They’ve taken off like a rocket, and meanwhile, my small value funds are suffering. They’re lagging in the market. The client’s like, oh my gosh, this guy’s a genius. Look at how well he’s done. But in fact, it could just as easily have gone the other way. I’m simply banking that, in many cases, once you have a good basic asset allocation in place, you have the right kind of exposures, for the marginal dollar, instead of buying more mutual funds that have a lot of tax consequences, you might be better off buying stock like Berkshire Hathaway. It’s almost like a mutual fund in its own right, but doesn’t pay dividend, and by the way, I’m starting to get some traction with this idea.

Phil DeMuth: has written an article that’s gotten some attention in financial planning that’s extremely critical of dividends. Now, I love dividend stocks. I wrote a book about dividend stocks. I think they’re great to the extent to which you need to spend money or you can house them in an IRA. But if you get a dividend that you don’t need, that’s like a realized capital gain. It’s an accounting identity with a realized capital gain with zero cost basis, which is the worst kind you can have. The expense of accruing dividends in taxable accounts year after year for high income people can subtract a lot of value, so I’m very sensitive. My heart goes out to these people.

Phil DeMuth: I’m trying to do everything I can to make that tax bill as low as possible, especially because, as I’ve seen, I can’t control investment returns. I can probably have a global asset allocation, and I can hope to get the rewards from that, but it’s out of my hands. I thought small value stocks were going to do great for the last 10 years and they haven’t done that great. They’ve lagged. Just a simple index fund. The extent to which I can, I really try to control the taxes where I know what I’m doing, and I know that, if I do this, I’m going to cut people’s taxes by this amount, where if I invest their account differently, different kinds of asset classes, maybe it’ll turn out great, maybe it won’t turn out great. The tax focus gives me an illusion of more and better control. It’s something that really pays off, especially for people who are tax sensitive.
Dr. Jim Dahle: Now, admittedly, a portfolio of individual zero dividend stocks is going to be slightly more tax efficient than buying the Vanguard Total Stock Market Index Fund, which does pay out about a 2% yield each year.
Phil DeMuth: Right.

Dr. Jim Dahle: How many of these stocks do you have to buy before that uncompensated risk you’re taking on buying individual stocks can be lowered enough that it is lower than the cost of paying that additional tax on that 2% yield?
Phil DeMuth: Fascinating. Well, let’s just take the S&P 500. Let’s say there are 500 stocks in that group. Of those, you’ll probably find that, I don’t know, 300 to 325 of them pay a dividend, so those are out. Then you’re left with this other portfolio of maybe 150, 175 stocks, and that portfolio historically has done better after tax than if you owned all of them and paid taxes of 15% on your dividends and capital gains. That’s the good news. In practice, I don’t own all those stocks. I probably own 50 of them that are sort of a sampling that I think are good ones to focus on. That’s the goal, but historically, you’ve been better off right from the gate doing it. Then there are other things people can do with these zero tax or super low tax portfolios. The most obvious one of which, or not necessarily obvious, but the most conspicuous is you can take a portfolio loan against it.

Phil DeMuth: If you’re a rich person, you’ve got a $50 million, and you own a bunch of zero dividend stocks, you own Berkshire Hathaway, you own Google, you own Amazon, and you want spending money, well, instead of selling those and paying capital gains, and those capital gains could be pretty significant if you’ve owned them for a while, what you do is you just borrow against it. You take a portfolio loan, and the portfolio loan, which comes to you at a very low rate because it’s totally secured by the liquid assets underlying it, that’s not a taxable event, so you have to pay a little interest on the loan, but there are no taxes because the asset is offset by a liability on the other side. There’s no net income. It’s not an income producing event. You live off the income for 20 years, and then you die, and your heirs get a step up and basis, and then the next generation does the same thing. That’s one way that ultra high net worth live and pay less taxes than doctors do.
Dr. Jim Dahle: Yeah, you just have to weight the interest costs against the tax costs. Take whichever one’s lower.
Phil DeMuth: Exactly. Exactly.

Dr. Jim Dahle: While we’re on the subject of asset allocation, one of my favorite quotes of yours is, “Even if risk tolerance existed and could be measured accurately, why would it be an important factor to consult when considering how to invest? You should invest in the way that has the greatest prospect to fulfill your investment goals. That might mean taking more or less risk than you would prefer. If you are a sensitive soul who can brook no paper losses, the solution is to get a grip, not to invest safely if that locks in running out of money when you’re old.” What do you mean by that quote, and can people really buck up and tolerate a really volatile portfolio just like that?
Phil DeMuth: Probably not. It’s probably just technically correct but completely useless advice. On the other hand, when I was in high school and I broke my ankle when I was playing football, and I went to the emergency room, the doctor did not ask me about what is my cast-wearing tolerance. He didn’t, “Well, Phil, how do you feel about wearing a cast for the next month? Would that be bad? Would that make you uncomfortable, make you feel bad?” He didn’t even ask that question. He just put a cast on my leg and basically said, “Deal with it, kid,” you know. I think that, in a way, that’s the correct approach. If there’s something that’s this important, it would be better if we could learn to deal with it.

Phil DeMuth: Now, and part of the problem here, the SEC says that advisers need to analyze their clients’ risk tolerance before putting them into a portfolio, and this makes sense to some extent because you don’t want people to just sell everything the first time the market drops 3% in one day. It’s still a problem because there’s no way to actually judge a person’s risk tolerance in advance. The industry is littered with these risk tolerance questionnaires, but these are questionnaires that have no predictive validity whatsoever. They have a kind of face validity. They’ll ask you do you like heli-skiing and mountain climbing, or do you prefer spending your afternoon in the library? You see questions like this, as if that has some relevance to your investing style.

Phil DeMuth: But the whole industry seems to be completely on the wrong foot here, and yet it would be easily possible to come up with a questionnaire that had some predictive validity because Schwab and Fidelity know all the people that capitulated during the last big market meltdown. It would be easily possible for some enterprising PhD student to get a hold of some of this data, see what questionnaires these clients might have been taking in the past, and see what if that had been a good predictor. But the whole risk tolerance game is a … it doesn’t really work, as far as I can tell.
Dr. Jim Dahle: Now, you say in the book that everything you’ve heard about how passive investing beats active management goes double after tax. Can you explain what you mean by that?

Phil DeMuth: It’s almost a miracle, it seems to me, because the best portfolio is what might be called the global market portfolio, the portfolio that owns everything. It owns every stock in the world, every bond in the world, every publicly traded asset class that you can invest in that have market cap waiting. This is the ideal portfolio, and it turns out you can buy that much more cheaply than you can by hiring active managers to try their wizardry to do better. This is what you might call the ideal Boglehead approach.
Phil DeMuth: Well, it turns out, almost by coincidence, that that exact portfolio is also comparatively cheap to hold, at least on the stock side, because it has very low turnover. Whoever at Standard & Poor’s, when they get together every year to decide on what stocks are going to be in the index, it doesn’t change the index very much. They get rid of a couple of dogs, they hire on a couple of new stocks, so it’s very low turnover, very low, cheap. It’s a cheap portfolio to buy, it’s cheap to hold in terms of your expense ratio, and it also turns out to be very cheap in terms of your tax cost ratio. This is just a win, win, win, as far as I’m concerned.

Dr. Jim Dahle: Let’s talk a little bit about mutual funds. I love mutual funds. They’re great. They provide liquidity, they provide professional management, they provide ready diversification, but they’ve got one kind of nasty quirk. They like to pass along any capital gains that they generate buying and selling securities in the fund to their investors. Yet, by law, they’re not allowed to pass along their internal losses to the shareholders. You never get a deductible loss you can use on your taxes from your mutual fund. Do you think that’s fair and do you think it’s ever likely to change?
Phil DeMuth: No, it’s certainly not fair, and it’s also very unlikely to change. I remember when Gene Fama, the Nobel Prize winner, wrote an editorial in the Wall Street Journal, along with Ken French, complaining about this 20 years ago, and nothing ever happened. I think the people that would benefit, just fund shareholders, are too dispersed, and they don’t have any political clout. There’s no constituency saying, yes, we must do it, and then now the problem is that this issue with the traditional open-ended mutual funds has been sort of bypassed by the exchange traded funds, which don’t have the same capital gains pass through problem. They’re able to diffuse their capital gains in the market. It’s the way they trade.

Phil DeMuth: I was talking to an investor the other day. He says, “So what do we do? We’ve all got these huge holdings in mutual funds that we’ve built up over the last 20 years. We have enormous embedded gains, and we can’t get rid of them. They’re a monkey strapped to our back. Every year they get all these dividends, we get all these capital gains we don’t want. Help. What can we do? We’re all in this boat. We all wish we had exchange traded funds where we weren’t victim to this same process, and now what’s going to happen is that even these exchange traded funds are going to be bypassed by something that’s now called … It’s just coming along. They call it direct indexing.”
Phil DeMuth: But I think the better way of thinking about it is what I wrote about in The Affluent Investor book, which is that, given computerization, it’s now comparatively easy to just take a look at a client, take a look at what’s their tax situation, what’s their personal human capital like, what industry do they work in, how secure is their career, and then to basically custom tailor a portfolio of individual stocks, and bonds and whatever, around that person, giving them, in effect, the holdings of the global market portfolio, but correcting for the tilts and the biases that their own happenstance circumstances, you know, I’m a doctor, and I live in Houston, and I own a million dollar house there.

Phil DeMuth: I mean, they can correct for all of this stuff by counter-weighting the rest of the portfolio so it’s not distorted in the same way. All this can be done. Computers can easily handle the task, and I’ve been waiting for a decade or so for this to happen, and it’s only now just beginning to come on the horizon, but when this hits, and it’ll hit in a big way, suddenly we’re going to be saying, wait a minute, we’ve still got these mutual funds. I also want out of these exchange traded funds. Help get me out of here. Get me into this new, better system. It’s like the Apple CarPlay, and not like the old 8-track that I’ve got in my old car, you know. I think that’s what’s coming.
Dr. Jim Dahle: Now, we’re talking about reducing investment taxes with Phil DeMuth here, if you’re just tuning in. Let’s talk about some of the biggest investment-related tax mistakes that you see investors making.

Phil DeMuth: Well, they make all of them. Probably the worst one is that investors have somehow got the idea that it’s important to realize their gains. If a stock is up, okay, I made 10% on Amazon, so it’s time for me to sell it and buy something else, and then, at the same time, they say, oh, I bought that marijuana stock, and it thought it was going to do well. I was certain it was, but now it’s down 20%. I know, at legalization, it’s going to all come back, so I’m just going to wait. I’m going to hold that, and, of course, this is just psychological manipulation. Basically, people should be doing the exact opposite. They should be holding on to anything they’ve got with a gain, not realizing it, just letting it run, but their losers, they should be selling almost immediately, and banking the tax loss from it, and using it against ordinary income or to offset other gains they’re getting. That’s just an obvious thing, but people invariably get it wrong, so keep your winners and sell your losers, not the other way around.
Dr. Jim Dahle: Any other mistakes you see people making frequently?
Phil DeMuth: Well, all kind of … They don’t fund their retirement plans to the fullest extent, they don’t tax loss harvest their accounts. You know, just the vanilla stuff that we run through in the book is all … it’s very commonsensical. They all should be doing all of these things, and they don’t, either out of ignorance, or laziness or something.
Dr. Jim Dahle: Do you think tax loss harvesting is oversold?

Phil DeMuth: It’s oversold but it’s under practiced, and that’s because tax loss harvesting, if you’re going to really do it well, it requires searching through all of the individual tax lots of all of your holdings. You just look at the mutual fund. You know, I own the Fidelity Magellan Fund, and I’m up 10% at that, so it’s great, but in fact, if you’ve been reinvesting dividends and capital gains in that fund for 20 years, then you actually sort through all of the underlying lots that you own, some of those owns are going to have done great, and some of the lots you bought at the wrong time, and they’re going to have done badly, so you should always comb through, sell whatever is down, and keep whatever is going otherwise fits within your investment strategy.

Phil DeMuth: You know, it’s nice. The $3,000 a year deduction from ordinary income is pleasant. It’s not exactly going to make you rich, but the real benefit is if you can use tax loss harvesting to be able to draw down from a taxable account, say, during retirement, for a number of years without realizing capital gains, and eventually shove all of your capital gains into an account that is left in your estate, where it gets a reset to fair market value, and that’s of course a great thing.
Dr. Jim Dahle: Now, you’ve discussed family loans before as an advanced tax trick. Can you explain how that works to reduce your taxes?

Phil DeMuth: Right. Family loans are basically about wealth transfer. If I’m from a wealthy family, and I have some kids, and let’s say I’ve got an extra couple of million dollars just sitting in the bank, what might I do with it? Well, if I invest that money myself, then the money’s going to grow, but then all that growth is going to be part of my estate, so it’s all going to be taxed at some point, assuming I have an estate that’s big enough to be taxed. It’s going to be a problem in some way. A better way of doing that would be to loan the kids a couple of million dollars, and let the kids invest it in some way. Let them use it to buy a house, use it to buy an investment portfolio, use it to start a business of their own, and then have them pay me back just a rate of interest on the loan.

Phil DeMuth: Now, the other good thing about this is that, when you do this, you can use what’s called the applicable federal rate, which is actually a sub market rate of interest. Today, I don’t know what it is. Probably a little over 2%, which means that the kids could pay you back. If your kids pay you back $15,000 a year on the loan, that would cover a pretty good amount, and plus, while you can’t forgive that amount, you could separately, completely unrelated to anything else, this is a totally different conversation now, you could give your kids and your spouses, you and your bride give the kids and their spouses, $15,000 a year just as a gift, and if they happen to pay you back a check that may be something that’s about that amount in relation to servicing the loan that you’ve given them, then that’s all.
Phil DeMuth: The main thing, though, if you try this, is that it has to be formally set up. It’s got to be for real. The IRS, when they look at family loans, they say, “Oh, Dr. Jim, very nice of you to do this. I see you made a two and a half million dollar gift to your son. That’s very nice,” and you say, “Wait, wait, excuse me, that wasn’t a gift. I’m loaning him,” and they say, “Prove it.” At that point, you want to be able to say, “Well, here’s the documents. Here’s the loan document. Here’s the collateral. Here is the checks he’s been paying me every month on this. This is a legit transaction. This is not a gift. This is a loan.” You have to be able to document that it’s really there. But if so, it’s a nice technique. It’s a nice way to help your kids if you have the money to do it.

Dr. Jim Dahle: Cool. That is a pretty slick trick, and you can see that you can basically forgive an amount of interest up to the amount of gift tax each year, the gift tax exemption.
Phil DeMuth: Well, I wouldn’t put it that way. I would just say you’re just generous with your gifting, the $15,000 tax exempt gift every year, and you’re also generous in making it a nice loan for the kids.
Dr. Jim Dahle: Yeah.

Phil DeMuth: They don’t really have anything to do with each other, it just works out that way.
Dr. Jim Dahle: Yeah, and it’s far more than 15,000, too. If there’s two spouses, your kid, and your kid’s spouse, and you and your spouse, I mean, we’re now talking about $60,000 in gifts that wouldn’t be included in the gift tax.
Phil DeMuth: That’s right. That’s right.
Dr. Jim Dahle: That covers an awful lot of interest on a pretty large loan at 2%.
Phil DeMuth: Yeah. You can do two and a half million dollars of a loan roughly with that. I wouldn’t have them line up exactly, but if they lined up approximately, that’s probably fine.

Dr. Jim Dahle: Yeah. Now, you have some rather controversial ideas about charities. You have argued that supporting charities is often a bad idea, and your argument is basically that the most efficient system is when people spend their own money on their own behalf, and the least efficient system is when people spend other people’s money on behalf of a third group of people, and you argue that, in that respect, charities are inefficient like government is. In fact, you say it’s safe to assume that any charity is a well-meaning scam until your own research proves otherwise.
Phil DeMuth: Yes.
Dr. Jim Dahle: How can a charity that is not a scam be identified a priority?
Phil DeMuth: Right. Well, I realize this is a minority view. Unfortunately, it happens to be correct. The state of philanthropy in our country is just a national scandal, and I think it will receive increasing attention in the future, certainly from me if nobody else. When you look around, charities basically operate in a pre-scientific world, where they’re trading in anecdotes, and photographs of them doing good, and pictures of animals, and basically what they’re selling is a feel-good moment for the donor, so the donor can relieve themselves of their guilt, and they can feel like, oh, yeah, I’m an important difference maker in the world. This is the state of things.

Phil DeMuth: The outfit that I actually do recommend in this regard is called GiveWell, and they are a charity evaluator group, and their story is interesting. I’m sure I’ve got it mostly wrong, but it basically began in San Francisco, I don’t know, 10, 15 years ago, by a series of hedge fund kids, and they realized they were all making too much money, and they wanted to do something constructive with it, so they would meet at lunchtime, and they’d each research a charity, because they spent their whole day researching business. They’d research some charities to try to find some good charities to give money to, and what they quickly found is there weren’t any. They’d go to the charity websites, and it was nothing but pictures and cute stories, but they’d say, okay, but what’s the research that shows that you guys are actually accomplishing something, and there isn’t any. Or if there is some, it’s incredibly biased, flawed, and absurd. They’ve been doing this. They broke off from the hedge fund world, started their own evaluation service. They’ve been doing it for years, and I think they’ve got a list of about eight charities that they think are worthwhile now.
Dr. Jim Dahle: Which is amazing considering there’s 8,000 or 80,000 in the world.

Phil DeMuth: Oh, yeah. There’s a lot of charities and private foundations and all this stuff. It’s horrible. Now, they don’t focus on should I give money to the opera. They don’t focus on should I give money to the University of Alabama football team, which a lot of people do. They focus on, if I want to give away a dollar, where is my dollar most likely to do the most good in terms of humanitarian work? For example, one of their charities is called Give Directly, and what Give Directly does it takes your dollar and it gives it to poor people in Kenya and Uganda, because the idea is you give poor people money, and they’re less poor, and it’s not like some patronizing event where you’re saying, oh, take this particular approach that we think is going to help you. You can figure out best how to use this dollar if you’re poor, so they don’t have any strings to it, and they’re also researching how they do, how it’s working, to try to be as transparent as possible. That’s one of them.

Phil DeMuth: The other thing I should just point out, I think that many physicians are not aware that the entire existence of modern medicine is due to the philanthropy of J.D. Rockefeller, because Rockefeller was a golfing buddy of a homeopathic physician, and this homeopathic physician was hitting him up to build him a medical school, and Rockefeller turned to his adviser, Frederick Taylor Gates, and said should I give this guy some money or not. Gates went to the library, he took out all of the medical textbooks that he could find, and he read them.

Phil DeMuth: He came back to Rockefeller and said, “That guy you’re playing golf with is a complete quack, and not only that, these other doctors you know that also want you to build them hospitals and medical schools, they’re also quacks. They just don’t know it, because the field of medicine knows practically nothing. They’ve got descriptions of thousands of diseases. They have treatments of some kind for some of them, and they have cures for a few. We don’t need more doctors, hospitals, medical schools. What we need is bench research. We need medical science about the causes and treatments of disease. You ought to put all your money into that. Forget about this other stuff,” and Rockefeller took his advice, and he did it, and that basically changed the world, and that’s why we’re all here today, because of the great things that came out of that fundamental change in direction of medicine that philanthropy caused.

Dr. Jim Dahle: All right. Let’s talk a little bit about these tax tricks, these tax tactics. We’ve talked about them a lot on my blog. On this podcast, we’ve talked about them. What I’ve never really done, though, is tried to rank them. If you looked at all of these tactics, you know, maximizing the use of retirement accounts, using tax efficient investments in your taxable accounts, getting your asset location right, making sure your dividends are qualified and that your capital gains are long-term, depreciating real estate, tax loss harvesting, donating appreciated shares to charity to flush those capital gains out of your portfolio, getting a step up in basis at death. If you had to rank those, which two or three would you put at the top of the list that give you the biggest bang for your buck? If you’re one of these 80/20 kind of people, which of those do you need to get right in order to get most of the bang for your buck out of investing tax efficiently?

Phil DeMuth: Well, unfortunately, each of them make a small positive contribution, so they’re all worth doing, especially keeping the turnover of your accounts low is extremely important in getting rid of these gratuitous capital gains. If you can minimize dividends where possible, that’s a great thing to do. They’re all good. I think that there are a couple of things that are even beyond that list that are worth paying close attention to, and that has to do with the fact that there are a couple of periods in your life where you can make a big difference to the overall curve, your overall financial curve, if you take advantage of them when you’re there, and they’re both periods when you’re likely to be relatively low income, and so you’re going to have more options.

Phil DeMuth: One of them would be when you’re starting out in your career. Very early stages. You’d have opportunities to donate to the extent to which you can save any money. That’s incredibly valuable, because it gives you the gift of long-term compounding, you might have a chance to open Roth IRAs and be able to do long-term compounding tax-free. That’s extremely important. In fact, I’ve come up with a phrase. It’s called live like a resident. I’m going to trademark this because I think it’s so cool. Actually, I got that from you. But if you can do that for just a little bit, and get your financial ducks in a row early, and put more money away in savings, and have that roll through your lifetime, that’s going to make a difference between night and day for you.
Phil DeMuth: The other interesting time is after you retire but before you have to start taking required minimum distributions and social security. This is a time when you can perhaps do Roth conversions of your IRA, and really try to optimize your whole tax picture in retirement, which, again, could be tremendously beneficial, especially if you also look through to the next generation, and figure out, well, who am I going to ultimately leave these assets to? What’s their tax bracket? And try to arrange the whole thing so you’re paying as much of the money as possible, paying as much of the tax as possible, in as low a rate as possible.

Phil DeMuth: I think that that’s very important to take advantage of if you can do it, and it’s worth getting some help. If you’ve never used financial planners at any other time in your life, it’s worth talking to them when you’re starting out, and I think it’s worth talking to them when you’re on the cusp of retirement, to figure out a game plan. You don’t have to stick with them forever, but at least try to get everything set up correctly then, because it could really pay off.
Dr. Jim Dahle: Now, let’s talk a little bit about a current event. In Congress right now is the Secure Act, and I’m hearing a lot of politicians saying they think it’s going to pass, at least in some form. You recently wrote a very widely-read article about that. Can you talk briefly about the basic changes that will occur if it passes and how much they really matter?
Phil DeMuth: Sure. Happy to, because I’m really riled up about this. The Secure Act is a bunch of different provisions that affect our retirement plans, and it passed the house 417 to three. It was slated to pass the senate, and I looked at the provisions, and I sort of analyzed them, and I said this is horrible. This is a nightmare. So I wrote about it for The Wall Street Journal, and then The Wall Street Journal, bless their souls, on their editorial page the following week, they basically just said that, “Yeah, we think this is a bad idea too.” Simultaneously, I emailed all of my clients, everybody I knew, to please contact your senators immediately. I sent their senator’s email contacts. I said, “Tell Congress to keep their hands off of your IRA.”

Phil DeMuth: What this bill does is it pays for itself by taxing our retirement plans when we leave them to our kids. Inherited IRAs. As it stands now, your kids can stretch out the payments over their actuarial lifetimes, and this is great. But the new plan is going to be, oh no, the kids have to take the money out in 10 years, and once your kid inherits a substantial IRA, that’s going to be a lot of money they’re pulling out over those 10 years, which means about an extra third of it is going to be taxed. If your kid happens to have kids that are going to college of his own, and needs to apply for some kind of financial aid, it’s going to completely screw up the financial aid package, so their expected family contributions is going to be much higher. It’s going to either go to the government or some college administrators. It’s not going to go to your kid’s retirement.

Phil DeMuth: The fact that this bill could have gotten so far, it was just sailing through, no discussion, no debate, I was just completely … and I am still baffled. I understand that the bill is being promoted by the insurance industry because one of the provisions is that all of these retirement plans, your 401K, your IRAs, they’re going to have to have an annuity as a potential offering, so the insurance companies love this. But the financial press is pretty much asleep at the switch. I’m just astonished that nobody has picked up on this and what a terrible consequence it is, and of course the people it hits the worst are not … it doesn’t hurt the 44% of people that don’t pay any federal income tax. They’re not bothered. Really rich people aren’t bothered by this either, interestingly, because they don’t have very much of their wealth in retirement plans. They’re all in these huge taxable accounts.

Phil DeMuth: But who is hurt by it are the upper income, people that have big salaries and have big IRAs. There’s another word for those people. Those are physicians and professionals. Lawyers, doctors, CPAs. These people are all going to just be hosed by this, and everybody thinks it’s great. Yeah, let’s have them pay for it. But there’s no constituency that’s represented these people in saying no, let’s not do this. AARP. Well, they get half their funding from the insurance industry, so they don’t think it’s a problem.
Phil DeMuth: My suggestion for the listeners is that they’re going to try again this fall. We got it derailed for the summer. They’re going to try again to get it passed. It’s going to probably be attached to a spending bill, and if they can succeed in doing that, those bills always pass. This is going to be the new order. If you want to have a third of your money from your IRAs going to the government, that’s fine. Do nothing. It’ll probably happen. If you’d like to have that money go to your families instead, I would encourage you to just Google contact my senators, send them an email saying please do not vote for the Secure Act in any form. Have the government keep their hands off my IRA, because I’ve been planning on this for the last 20 years, and now you want to change the goalposts in the last minutes of the game. It’s not fair.

Dr. Jim Dahle: Now, a lot of people would criticize that argument and say hardly anybody’s stretching their IRAs longer than 10 years anyway, and most people don’t have much of an IRA to start with. If you look at the average IRA or 401K balance in this country, it’s pitifully low, and most of it is spent in the first half of retirement by the typical middle class folks, so that’s kind of the criticism, is that this is a way to tax the rich, if you’re taxing anybody, but that really nobody’s stretching these things out anyway because the heirs can’t seem to keep their paws off the money longer than 10 years. They all pull it out in the first 10 years anyway. Do you think it’s really going to affect all that many people?

Phil DeMuth: Well, according to The Wall Street Journal’s editorial on this … First of all, you’re completely correct. If you have a small IRA, you’re going to spend it all yourself, and even if you have a $500,000 IRA, you’ll spend that yourself in retirement, pretty much, and anything the kids get, they’ll buy a new car immediately the day the check clears. That money is gone. But if you have a million dollar IRA, or a multimillion dollar IRA, that’s going to be a problem. Your million dollar IRA, when you’re 65 could still be a million dollar IRA after withdrawals at age 85, and so that’s going to your kids, and where it’s going to be quickly dispersed and taxed heavily.

Phil DeMuth: I think The Wall Street Journal’s editorial suggested that maybe one in every four dollars in retirement plans would be affected by this, and in fact, I downloaded the government’s own paperwork, own financial analysis, and what I realized is the way they sold it was that the bill would pay for itself over the course of 10 years. But if you extrapolate the cost versus the income, the new revenues they’re going to be bringing in, every year after those 10 years, the gap widens hugely, so by 2050, the government is going to be taking in an extra $4 billion a year in tax revenue from our IRAs and using it for its own secure projects, and this has completely screwed up, again, our ability to make any kind of financial plans. This is not the deal that we were promised when we were told we could contribute to these tax-free accounts.
Dr. Jim Dahle: Now, you’re obviously clearly against the limitation on the stretch IRA provision. It doesn’t sound like you’re very happy about the provision that requires an annuity option to be in the retirement accounts, as it sounds like a big boon to the insurance industry. There’s one other provision that’s pretty widespread and well-known about it, and that’s increasing the RMD age to 72, maybe 72 and a half, I don’t recall. You see that as good, bad, indifferent, doesn’t matter? What do you think?
Phil DeMuth: I’m all in favor of it. I would be happy to extend the RMD age any chance I can get. What I found so bitterly humorous is that there’s no possible way of extending the IRA RMD age from 70 to 72, which, by the way, for some people that’s one year, for other people that’s two years. It depends on which month of the year you were born in whether or not you get to collect a one year or two year benefit. But that somehow compensates for getting the entire IRA ultimately taxed over a 10-year period when it’s inherited. It’s just a complete joke.

Phil DeMuth: But to get back to the insurance companies for a minute, my favorite provision in the bill is there’s a safe harbor given to the plan administrators. They don’t need to worry about making sure that it’s the lowest cost annuity that they offer. They can offer the highest price annuity if they think that’s what’s going to be in the best interest of their plan holders. I can just imagine the lavish seminars that are going to be held in Hawaii, where the insurance industry invites the plan sponsors to come and spend the week, and will spend a couple of hours a day talking about investing, then they’ll play golf, then go to the beach. It’s going to be a great time. It’s a great time to be in a 401K plan sponsor. You’re going to have some real benefits.
Dr. Jim Dahle: Now, we’re starting to get a little bit long on the podcast now, but I wanted our listeners to get a chance to hear about what you’re working on now, your next project. What’s going on in your life now?

Phil DeMuth: I’m revising the tax book like a madman, trying to figure out how to put the Secure Act into it one way or the other, and then I want to get that off my plate, because what I really want to talk about is charity. It’s the last big wealth management topic that I haven’t really … I’ve sort of hinted at it here and there, but I’d like to talk about it in more detail, because I know I’ve given money to charity in my life. I see how foolishly I’ve been with my donations and how they’ve basically amounted to very little versus what they could have done if I’d been smarter about it, so I want to atone for my sins. That’s why I wanted to write a book about how to be a more effective giver when you do give to charity.

Dr. Jim Dahle: I want to give you a chance to wrap this up a bit. Somewhere between 20 and 30,000 people will listen to this podcast in the next month after it’s published, and then perhaps a thousand a month after that, going forward, so you’ve got the ear of a lot of doctors and other high-income professionals. What would you like to tell them that we haven’t already covered in the podcast?

Phil DeMuth: I have a couple of books I would like to recommend to them, apart from your book, which of course is a classic, but one would be … this is a political book, really. We all know Atul Gawande’s book The Checklist Manifesto, but I think The Checklist Manifesto of 2019 is going to be Marty Makary’s book The Price We Pay, which is about how to fix the healthcare system. I think that doctors are getting hurt by association with the healthcare system. People are having lots of problems with it, and while I’ve always revered doctors, and that was always the attitude growing up, people seem to have a lower opinion of doctors these days, and I think doctors should get out in front of how to fix the healthcare system, rather than just be bystanders while it gets fixed elsewhere, especially if you don’t … I think that Medicare for All is going to be a big campaign idea, and if you think there’s a better way of doing it, and I think Makary does, that I would get behind it.

Phil DeMuth: The other thing I want to say more personally, financially, is this. Doctors live rich, but they don’t die rich. They mistake having a big salary for wealth, and this is the problem. You want to think about, when you go to sleep at night, how can I transform this high salary into a form of wealth? How can I transform my practice into a business? That’s the way that your family is going to benefit most, and a good book about this is called The E Myth by Michael Gerber. I would recommend that.
Dr. Jim Dahle: All right. We’ve been talking to Phil DeMuth, PhD. He is the managing director of Conservative Wealth Management, the author of The Affluent Investor and The Overtaxed Investor, both books which I highly recommend. Thank you so much, Phil, for coming on the podcast today.
Phil DeMuth: Dr. Jim, thank you so much for having been. It’s been great being here.

Dr. Jim Dahle: That was great having Phil on. Like I said, he has a few controversial ideas, and I dwelt a lot on those, but if you read his books and his writings, you’ll realize that we agree on about 99% of everything financial, and so it’s just fun to talk about the controversial stuff. Make sure you’re signed up for our email list. You can get that at whitecoatinvestor.com/email. You can get the monthly newsletter, you can get a weekly summary of the blog posts, you can get every blog post we’ve published directly in your email box.

Dr. Jim Dahle: We’re also starting a list for real estate investing opportunities, and to learn more about that. That one will feel a little bit more marketing like than the main newsletter in the blog post, but if you were interested in some of those private real estate opportunities, you can sign up for that as well. Like anything, it’s free, and you can unsubscribe from it at any time. Thanks to those of you who have given us a five star review on the podcast, and for telling all your friends about it. Those reviews really do help spread the word of physician financial literacy.

Cindy: Cindy back again. Check out this episode’s show notes at whitecoatinvestor.com. You’ll find a full transcription of this episode, as well as links to everything that was discussed. This podcast was brought to you by contract diagnostics. This is a company that specializes in physician contract reviews. Specialization is something we can all appreciate here, so again, when you have contract needs, give them a call, even if it’s to talk about your current compensation structure in the group you’re in. They have helped many of your White Coat Investor colleagues. Check out contractdiagnostics.com, or give them a call at 8885745526.
Dr. Jim Dahle: Head up, shoulders back. We’ll see you next time on the White Coat Investor podcast.

Disclaimer: My dad, your host, Dr. Dahle, is a practicing emergency physician, blogger, author, and podcaster. He is not a licensed accountant, attorney, or financial adviser, so this podcast is for your entertainment and information only. It should not be considered official, personalized financial advice.