By Dr. James M. Dahle, WCI Founder
I met a well-to-do physician not too long ago. Now retired, the doc has a mid-eight figure net worth. Yes, you read that right.
Despite massive wealth, the doc only spends a couple hundred thousands per year. That did not surprise me too much; many very wealthy people spend “like doctors,” i.e., $200,000-$300,000 per year. What surprised me was to learn the doc was paying less than $100,000 per year in taxes. While I didn't get the details from the doctor, I did spend some time thinking about how someone worth $50 million could get away with paying less than $100,000 in taxes. I came up with several different scenarios and discovered it was actually really easy to have a low tax bill. Certainly, the less you spend, the easier it is to do. But even with significant spending, it can still be done.
Case Study #1: A Relatively Frugal Real Estate Investor with a Very Tax-Efficient Portfolio
Imagine a 60-year-old retired doctor who:
- Lives in a tax-free state,
- Spends $300,000 per year, and
- Gives away $100,000 per year.
This doctor has the following assets:
- $5 million in a primary and secondary home
- $10 million in muni bonds
- $15 million in a Total Stock Market Fund (TSM)
- $5 million in a Roth IRA and a traditional IRA
- $30 million in investment real estate with $15 million in mortgages against it
- Net worth = $50 million
How much is this doctor going to pay in taxes?
Remember, there is no wealth tax (at least until death), only an income tax. So, how much taxable income is this doctor going to have? Let's look at those assets again:
- Homes: $0
- Muni bonds: Let's say the average yield on these bonds is 2%, so $200,000 and none of it is taxable.
- TSM: The fund made almost 30% in 2021, but the yield is only 1.14% and it is all qualified dividends. 1.14% x $15 million = $171,000
- Retirement Accounts: No money taken out, no taxes due. No RMDs due for 12 more years.
- Real Estate: We'll assume a yield of 5% ($30 million x 5%) = $1.5 million, but it is entirely possible that it is all covered by depreciation.
- Total spendable income: $1,871,000
- Total taxable income: $171,000 – $95,500 in deductions = $75,500
- Tax due: $12,359 single ($8,662 MFJ)
See how that works? While it is hard to have a lot of earned income and avoid massive taxation, it's relatively easy to have a lot of unearned income and assets and pay almost nothing in tax. Now, you can quibble about little things in this case study, but the bottom line is the tax bill is incredibly low for the level of wealth.
Case Study #2: A Big Spender Takes Advantage of Loans
Let's imagine another 60-year-old doctor. This doc:
- Lives in a tax-free state,
- Is retired,
- Is married,
- Gives away $500,000 a year, and
- Spends $2 million a year.
Never mind the fact that I somehow found two doctors with $50 million a year. It's pretty rare but it happens. This doctor's assets look like this:
- Homes (4 of them!): $15 million
- Roth IRA: $1 million
- Traditional IRA: $9 million
- $15 million in Total Stock Market Fund
- $10 million in Total International Stock Market Fund (TISM)
- Net worth: $50 million
How much is this doctor going to be paying in tax? Let's take a look.
- Homes: $0
- Roth IRA: $0 no matter how much is withdrawn
- Traditional IRA: $0 if none is withdrawn
- TSM: $15 million x 1.14% = $171,000
- TISM: $10 million x 3% = $300,000
- Total spendable income: $471,000
So far, so good . . . except the doc is going to spend $2 million and give away another $500,000 this year. But there is only $471,000 in income? What is the doc to do? Here are the options:
- Borrow against the home (Costs future interest but no taxes)
- Withdraw from the Roth IRA (No taxes due, but only $1 million in there)
- Withdraw from the traditional IRA (Some taxes due)
- Sell some mutual fund shares (Some taxes due)
- Borrow against the mutual fund shares with a margin loan (Costs future interest but no taxes)
There are lots of options and no reason that one cannot use a combination to meet the $2,029,000 shortfall. Here's one option:
- Withdraw from the Roth IRA: $29,000
- Withdraw from the traditional IRA: $200,000
- Sell mutual funds shares: $800,000 but with these high-basis shares, basis = 3/4 of value
- Donate low-basis mutual fund shares: $500,000
- Take out a 2% margin loan: $500,000
- Total spendable income: $2,000,000 + $500,000 charitable gift
- Total taxable income: $471,000 in qualified dividends + $200,000 ordinary income + $200,000 in long-term capital gains – $250,000 qualified charitable contribution = $621,000
- Total tax: $147,798 – $27,900 foreign tax credit = $119,898
The doc will also pay about $10,000 a year in interest going forward on that loan. Hard to complain too much about spending $2 million and giving away another $500,000 while only paying $120,000 in tax.
Case Study #3: The Roth Doctor
Now let's talk about a 75-year-old doctor who
- Is retired,
- Is married,
- Is debt-free and plans to stay that way,
- Spends $500,000 a year,
- Receives $50,000 a year in Social Security,
- Gives away $1 million a year.
This doctor was pretty aggressive about Roth contributions and conversions during a long career and the assets now look like this:
- Home: $2 million
- Roth IRA: $15 million
- Traditional IRA: $8 million
- Muni bonds: $10 million
- TSM fund: $15 million
- Net worth: $50 million
This doctor has lots of options. Let's start by calculating the RMD. At age 75, it's 4.4%. Take the 4.4% x $8 million = $349,345. Let's call it $350K. The doctor takes $100K of that as a Qualified Charitable Distribution (QCD) and donates the other $250K to charity after withdrawing it. The doc also donates another $650,000 in Total Stock Market Fund. Those are two very tax-efficient ways to give to charity. So, that takes care of the giving. Now, what about the spending?
- Social Security: $50,000
- Muni Bond Interest: $200,000
- TSM Dividends: $171,000
The doc is still $79,000 short. So, the doc takes it out of the Roth IRA. Or sells some muni bonds. The basis is about equal to the value, so there's no tax due there. The muni bond interest is tax-free. Only 85% of the Social Security is taxable so the adjusted gross income = $42,500 in ordinary income plus $171,000 in qualified dividends. However, half of that taxable income is wiped out by the tax deduction from the charitable donation (and the doc carries the rest forward). I think that applies first to the ordinary income, so that is wiped out completely, leaving $96,750 in qualified dividends. At 15%, that's a tax bill of under $15,000. Practically non-existent compared to $50 million in wealth.
As these three case studies demonstrate, a very wealthy person can spend quite a bit of money without paying much at all in tax, especially if they are charitable. Whether you think that's good tax policy, that is the way the tax code is written. Structure your retirement income accordingly.
What do you think? Are you surprised by how low the tax bill can be for very wealthy people? If you are retired, how much do you pay in taxes? What are you doing to reduce your future tax bills? Comment below!
Curious – did this person get to 50 million doing medicine for a long time or was it made from outside businesses?
I actually know several. They all owned some type of business, although one was just a multi-doctor medical practice that really grew large.
I had another very wealthy doc email me after this post stating they didn’t have any sort of side business.
I think it would be a challenge for a physician to amass that much wealth solely from practicing [legitimate] medicine and index funds unless one did it for 50+ years, saved a ton, and had several nice bull markets along the way.
How would one build a $15 million Roth IRA given the contribution limits? Successful speculative bets in a self-directed account?
That would be the fast way. Another method would be getting a fast growing small business in there somehow a la Peter Thiel.
A slower route would be taking advantage of all kinds of Roth accounts and then eventually rolling them over into a Roth IRA. For example, consider someone that put $69K a year into a Roth IRA for 30 years and earned 10% on it.
=FV(10%,30,-69000) = $11,350,087.57
Let that ride a few more years and it would hit $15 million
Yet another way is to do a huge Roth conversion.
Or a combination of the above.
Slight clarification above – you said, “consider someone that put $69K a year into a Roth IRA for 30 years”. I am sure you meant “Roth products” (like you said in the sentence before).
Thanks for the correction. $63K into a Roth 401(k) + MBDR + $6K into a Roth IRA would be $69K.
Where are those states that have no property taxes?!
California. Thanks to prop 13. There are tons of people in coastal communities who bought their houses decades ago. Their houses are now worth millions, but they only owe about 1% of the value from when they bought it (increased by 2% cagr)
And if they have a hobby farm or hobby conservation easement they get further relief.
I know this doesn’t apply to a no state income tax situation as above, but don’t forget how progressive CA income tax is. If you earn under 100k per year of “taxable income” you don’t actually pay much California income tax.
Not entirely true. Singles hit 8% before $50K in taxable income.
Even if you don’t make much, you’re still paying a lot of income tax in CA. From HR block below.
==================================
California Tax Brackets for Single Taxpayers
Taxable Income Rate
$0 – $8,809 1.00%
$8,809 – $20,883 2.00%
$20,883 – $32,960 4.00%
$32,960 – $45,753 6.00%
$45,753 – $57,824 8.00%
$57,824 – $295,373 9.30%
$295,373 – $354,445 10.30%
$354,445 – $590,742 11.30%
$590,742 – $999,999 12.30%
$1,000,000+ 13.30%
California Tax Brackets for Married/Registered Domestic Partner (RDP) Filing Jointly Taxpayers (and Qualifying Widowers)
Taxable Income Rate
$0 – $17,618 1.00%
$17,618 – $41,766 2.00%
$41,766 – $65,920 4.00%
$65,920 – $91,506 6.00%
$91,506 – $115,648 8.00%
$115,648 – $590,746 9.30%
$590,746 – $708,890 10.30%
$708,890 – $1,181,484 11.30%
$1,181,484 – $1,999,999 12.30%
$2,000,000+ 13.30%
This is from my comment below:
The way an average doctor can do this is via 401k/Cash Balance plan contributions. Depending on the duration, you can easily end up with $5M-$7M in assets in both of these plans, and then aggressively convert those prior to taking RMDs to Roth to end up with $7M-$10M Roth account by age 72. One can have a maximum of about $3M in a CB plan, and possibly another $3M-$4M in a 401k over one’s career. It will be much harder to do this with only a 401k plan, that’s for sure. You just need a way to get a lot of tax-deferred assets into an IRA, and while 401k can get you there over time, it is only half of the puzzle for a typical doc. Cash Balance plan is the second half of this puzzle.
However, $15M can be a challenge, but the question is at what age? If you make all of the conversions by age 72, by age 80 you might have $15M. If you are asking about having $15M by age 65, I’d say you have to get very lucky with investing. Using index funds you will get there over time.
Should the Roth doctor have converted more of the Traditional IRA? I would be interested to know how he/she got 15 million into a Roth.
You want to know how the hypothetical doctor got the money in there? I guess we could ask them.
“Hey, hypothetical doctor in this WCI blog post, how’d you get all that money in a Roth IRA?”
“Well, I contributed to Roth IRAs and 401(k)s for a long time, I invested them very aggressively, I did a big Roth conversion when I was 57 and invested that very aggressively and then never actually took anything out of that Roth IRA and now I’m 75.”
Happy? I’m not sure why 2 of the first 3 comments are aimed at that particular data point. We’ve got 7 figures in Roth IRAs already, can put 6 figures into them every year, and could do a multimillion dollar Roth conversion tomorrow and we’re only in our mid 40s. The doc in the example is 30 years older than us. Nobody questions a multi-million dollar IRA. But guess what? When you have $50 million, a multi-million dollar IRA very easily becomes a multi-million dollar Roth IRA.
Now I guess you could nitpick that a 75 year old TODAY didn’t have access to a Roth IRA or Roth 401(k) for very long in his or her career. That would be a fair criticism. But it won’t apply in 20 or 30 years when a lot more of today’s readers are 75.
Whether the doc should do more Roth converting depends. More information here:
https://www.whitecoatinvestor.com/roth-conversions-and-contributions-principles/
I am interested in getting more money into my Roth hence the question.
I think you’re retired now as I recall, so your only option is Roth conversions and/or investing more aggressively in your Roth accounts.
How do you get 6 figures in a Roth per year?
$61K into a Roth 401(k) (using Roth 401(k) and Mega Backdoor Roth Contributions) plus $6K for a Backdoor Roth IRA plus $20.5K for a Roth 457. That’s $84K. Double it for the other spouse. That’s $168K. And you can add another $15K if they’re both over 50.
Can most people do that? No. But it’s at least theoretically possible. Katie and I can do it because we designed our own 401(k) to allow us to do that. We don’t have 457s of course.
Mega backdoor Roth 401k x2. See his post on the WCI “world’s best 401k”
The way an average doctor can do this is via 401k/Cash Balance plan contributions. Depending on the duration, you can easily end up with $5M-$7M in assets in both of these plans, and then aggressively convert those prior to taking RMDs to Roth to end up with $7M-$10M Roth account by age 72. One can have a maximum of about $3M in a CB plan, and possibly another $3M-$4M in a 401k over one’s career. It will be much harder to do this with only a 401k plan, that’s for sure. You just need a way to get a lot of tax-deferred assets into an IRA, and while 401k can get you there over time, it is only half of the puzzle for a typical doc. Cash Balance plan is the second half of this puzzle.
After 2025 the estate limit I think is $12.06 million for a married couple so upon death wouldn’t a married couple pay over $15 million in federal taxes on the $50 million in most of these scenarios?
No, because they’d be dead.
The estate MIGHT. But that is offset by not owing any capital gains tax. And if they title the assets properly they can get the step up in basis when the first one dies. But these docs are clearly charitably inclined and so I’d bet there will be a large charitable bequest. Heck, these estates are large enough to start thinking about family foundations.
Isn’t there a Net Investment Income tax of 3.8% after a threshold? How do you avoid that?
NIIT only applies to income tax, not estate tax. If there is no taxable income due to a step-up in basis, there is no taxable income, no income tax, and no NIIT.
I’m curious about what might be reasonable spending for docs in retirement. I’ve factored in about 200K per year in spending based on our history, but have always struggled with determining spending budgets. The annual spending is one of the biggest factors in planning retirement, but is probably the least written about.
I think it is important to enjoy your hard earned wealth and enjoy life, but hedonism is ultimately as unsatisfying as undue austerity. Where is the best balance to be found?
We all struggle with that. Spending is very individual. Some docs are very happy with $60K a year in retirement spending. Others of us like to go heli-skiing and that’s not going to work on $60K a year.
It would be interesting to see a bar graph of monthly or annual spending habits of physician families, if the data could be obtained. We see that kind of thing all the time on the income side, and many docs like to see where they fall relative to their colleagues. I’ve even seen these depictions of average income based on age or decade.
If we had similar data on the spending side it would be very helpful, and perhaps allow spouses who don’t necessarily agree on budgets to have some data points for comparison. At least it would be easy to identify outliers one way or another.
In Case 3, his QCD is limited to $100k annually.
I noticed that also.
You’re right. Thanks for the correction. For our purposes, let’s just assume that that the amount above $100K is withdrawn as an RMD then donated to charity. It’ll be almost the same thing in this case.
In Case 2-living on portfolio loans is appealing when interest rates are low and equities have been appreciating. Your 2% example is unlikely even today. Portfolio margin loans can be raised spuriously as base rates rise ; typically base + 3%. The strategy fails when base rates reach 3-4%.
Correct. Lower returns and higher interest rates make that less appealing, but not necessarily completely unappealing especially in the short run.
Interesting article. Good examples of how taxes can be quite low even on high asset balances. Shows why it rarely pays to go to great lengths to avoid income taxes on index funds
I would note that it is misleading to discuss Roth accounts as if they were tax free. It is more accurate to realize that the tax was paid when the money went in. No more taxes paid when it comes out, but the total is lower than it would have been had there been no taxes upfront.
An interesting and simple example:
No state income tax.
$50,000,000 in assets
$30M in VTI
$20M in munis
VTI throws out $342,000 in qualified dividends. Only other taxable income is SS, taxed as ordinary income. Call it $400,000 in muni interest. Total income $792,000. If married, they will not even get to the 20% rate on qualified dividends.
No charitable deductions, no depreciation, no Roth, no investment real estate, no debt, no cash value life insurance. As simple a financial life as possible.
$50M in assets, can do their own tax return in a few minutes, low taxes. No temptation to complicate life to save a bit on taxation.
Give the couple a house, say $2M, and reduce the taxable income a bit.
There would still be property tax. There might be sales taxes, but those one can control by not buying much.
The federal bite with be a tiny fraction of net worth.
Only when compared to a tax-deferred account.
I come up with a completely different question when I hear this scenario.
Why have $50 million?
If they live on $300000 comfortably, even at 2% you would be very safe at $15 million. Certainly legacy is part of it. If there are charitable goals, why wait.
I love in a medium col area, a modest pension and SS. I have a hard time seeing how I could every need more than $3 million. I’ll never have more cause I give away whatever exceeds that.
I’m now saying we all need to be the same, just haven’t been able to understand having multiples more than one could ever use.
That’s what I used to think too: that I’d just give away everything over the estate tax amount and never have to worry about that stuff. But how do you give away an illiquid asset like a small business or farm? That’s what most people worth $50 million own. If you give away what you can give away, you’ll really be “business poor” or “farm poor” since you’ll no longer have any liquid assets.
Certainly you’re right that nobody really “needs” more than a relatively small amount (although most would argue your $3 million is far beyond that). This website is all about first world problems though. But you seem to have a problem with anyone ever actually having that much. Like it’s some sort of moral failing. Why should someone have to give away their business just because it’s particularly valuable? Seems wrong. Take away Tesla and Facebook and Amazon from some of our more financially successful citizens and there’s not all that much left, at least as a percentage of assets.
As I stated it was a question that I had as I read the post.
I said we aren’t all the same. Certainly that includes how our assets are arranged.
Never said someone had to give away anything. Including me.
I believe in this case they were retired, not business owners.
At the end I still wonder (for myself, and I agree I have more than enough) why keep more than I can use.
You can be both. If you retire from medicine but still own a bunch of rental properties producing income, you may have liquid and illiquid assets in the tens of millions but “giving away” the rental properties probably isn’t a good way to generate income–especially if you plan on donating part of that income to charity. I think the bottom line is that sometimes we know (or at least believe) that the assets generating income are better served under one’s own stewardship, and then you can donate the proceeds on an ongoing basis as one sees fit. Similar logic applies to large companies and dividends–if the company can increase its earnings and share price with that money, then it should stay in the company. If they have run out of opportunities to meaningfully reinvest in the business or the cash balance is so great that it’s essentially impossible to use it all, then you return that money to investors as a dividend. I think WCI, for example, does more good from both a financial and non-financial standpoint in Jim’s hands than it does if he simply off-loaded it. I suspect he feels the same way, which is why when it came time to choose whether to walk away from it or expand it, he chose the latter.
Three sample reasons to accumulate assets beyond expected needs:
1. Unexpected needs.
2. Severe downturns in asset prices depleting the value of a portfolio.
3. Both.
Needs:
People who get chronic illnesses that require care not covered by health insurance can find themselves in deep financial holes.
Need to care for a loved one.
Not difficult to come up with examples.
Assets
Victims of fraud.
There were many people in 1929 who thought they had more than all the money they would ever need. Did not work out that way.
The unexpected happens. If you think you will need $5 million, it is safer to accumulate $20 million. Better to have it and not need it than the alternative.
Hard to argue it isn’t safer. But it’s easy to argue it may not be worth what it takes to do it work wise and risk wise.
There does not seem to be any magic here.
1. If one gives away significant assets to charity there will be corresponding deductions applicable to income and therefore lower taxes.
2. Qualified dividends are taxed at a relatively low rate
3. If most of your assets are not in 401ks, IRAs or pension plans the tax rate will be your capital gains tax, which, in most cases, will be much lower than ordinary income tax rates.
That’s the point. It doesn’t take magic to do this.
Great blog, just WAY too complex an analysis.
First , achieving mid 8 figures doesn’t require a side business,Medicare fraud, or even any real estate exposure ( beyond personal houses) . Heck, not even a Roth. Simply live with a small footprint and invest with Mr. Bogle’s guidelines. Despite making multiple mistakes along the way, compounding does it all for you.
As for taxes, live in a tax free state, and have munis ( preferably laddered individual bonds) for portfolio balance. Dividends from your taxable equities will amount to taxes significantly less than the quoted $100,000.
And most importantly, live with a small footprint.
Updating my hypothetical with estimated taxes.
$50M total assets
$30M in VTI
$20M in munis
Total income $792,000
$400k of that is tax free muni
$342k is qualified dividends frim VTI
$50k from Social Security
Before any charitable contributions, fed tax would be $47,966, or a rate of 6%.
Take the contributions to $250k and the fed tax goes to
$14,561,
less than any of the illustrated totals, with a vastly simpler financial life, no taxes waiting when RMDs come due, not ignoring taxes already paid to put the money into a Roth, heirs get the full amount at a stepped up basis, no mandate to withdraw and pay taxes on the inherited amounts.
Expenses are vastly lower than example 1, where there are mortgage payments, taxes and maintenance costs of the rental property. Expenses are lower than example 2, which includes interest costs on loans for spending money.
With low cost, plain, simple investments, the doc pays a 6% effective tax with no charitable contributions and a bit over 4% with them. Pays lower total tax bill than any of the examples.
Pays no one to do the complicated taxes. Pays no interest on loans.
Unfortunately, I will never have $50M. If I did, I would prefer my hypothetical to any of the others.
Very close analysis of what my portfolio holds. I’m currently 80/20 . S&P/ laddered munis. The muni interest kicks off slightly more than you note, but your numbers are very close.