By Dr. James M. Dahle, WCI Founder
In chess, there is an opening, a middle game, and the end game. Given our mortality, there is also an end game in investing. Today, we'll address it.
Our subject comes from an email:
“There are several people in my peer group who fit the following profile:
- Physicians
- 65+ and still working
- Investment assets of $5 million-$10 million
- Home value of $2 million-$4 million
- No debt
These docs like working but recognize the impending need to wind it down. Looking at this, we should (I think) feel lucky and comfortable. We are fortunate but we’re actually terrified that we’re making mistakes we will have to pay for later when we’re most vulnerable. Some of us even have between $1 million-$2 million sitting in cash because they don’t know what to do and don’t want to get ripped off. What should we do?”
Congratulations!
The first thing to do is to congratulate these fine folks on a job well done in the accumulation phase. With net worths in the $7 million-$14 million range and a job they still enjoy, these docs have triumphed in a game that few can win. Remember that only 10% of doctors have $5 million or more in assets and a quarter of docs in their 60s aren't even millionaires. That does not mean the docs mentioned in that email don't have any problems, but it does mean they don't have the usual (much more difficult to solve) problems.
You Still Need a Written Investing Plan
The cure for paralysis by analysis, whether you're 30 with a net worth of negative-$200,000 or you're 65 with a net worth of $10 million, is a written investing plan. When I hear that someone is “terrified of making a mistake” and has “$1 million-$2 million sitting in cash because they don't know what to do,” I know they don't have a written investing plan. The solution? Get one. As I've written before, there are three ways to get it:
- Write it yourself (this is best for capable, frugal hobbyists)
- Take the Fire Your Financial Advisor online course (it has a cost, but you'll receive a framework and a high-yield education to assist the process)
- Hire a financial planner to help you draft it up and possibly implement it (this is the most expensive but requires the least expertise)
Whichever option you choose, you need that plan. Our most recent poll of white coat investors indicated that only half of them have a written financial plan. If you “don't know what to do,” get one.
Remember that a 65-year-old still has a long-term investing horizon for a significant chunk of their money: 20 years on average with some as much as 30 or 35 more years. Traditionally, about half of your investing career is before retirement, and half is after. That portfolio will still need to produce some growth and keep up with inflation, so there still needs to be some risky assets (such as stocks and/or real estate) in the portfolio—even if the ratio of risky assets to less risky assets is lower than it was during the early accumulation years. Don't fall into the trap of “income investing.” Assets and income are both fungible and you can convert one to the other at any time (although it may require some costs and/or taxes to be paid.)
A cash holding is not necessarily a bad thing, although 10%-40% of your portfolio is almost surely excessive. However, many retirees are just fine keeping 1-2 years' worth of spending in cash. Like an emergency fund, this money allows them to invest the rest more aggressively knowing their short-term spending needs are taken care of. It's more about the return of the principal than the return on the principal for that money.
A Spend-Down Plan
One thing an investor in the distribution phase needs that one in the accumulation phase does not necessarily need yet is a spend-down or distribution plan. This is a plan that describes how you are going to spend your money in retirement. The plan needs to detail several things:
- How much you will spend
- How that amount will be adjusted if necessary
- The order in which assets will be spent
A book could be written about each of those three topics. The classic guideline for how much to spend is the 4% rule. That rule says you can spend approximately 4% of a portfolio—adjusted upward with inflation each year—and expect with a very high probability that the money will last 30 years. If you're FIREing at 50, perhaps you need to adjust that 4% down a little bit. Conversely, if you work until 70, you can probably adjust that number up a bit.
The more flexible your spending and your withdrawal plan, the more of your assets you can spend. The ultimate flexibility comes when all mandatory spending is covered by guaranteed income (Social Security, pensions, Single Premium Immediate Annuities [SPIA]) and only discretionary spending comes from the portfolio. Then, in the event of a severe market downturn, no money must be spent from the portfolio. A variable withdrawal strategy that “adjusts as it goes” is clearly a better option than any sort of hard-and-fast rule.
However, most of the docs mentioned in that email probably aren't even going to need to spend 4% of their portfolio to maintain their pre-retirement lifestyle. Their worry is almost surely misplaced given that 4% of $5 million-$10 million is $200,000-$400,000 a year. That goes a long way when there are no student loans, no mortgages, no need to save for retirement/college, no kids' expenses, and no disability/life insurance premiums. Plus, they've already worked through most of the “go-go years” of retirement.
The order of spending assets is also an important part of the distribution plan. The general rule here is to first spend that money that is going to be taxed anyway before moving on to other income. This includes earned income, Social Security, pension payments, SPIA payments, required minimum distributions, rental income, interest, dividends, and capital gain distributions. If you need to spend additional money, it should usually be taken from high basis assets in the taxable account. Only after that point should additional retirement account withdrawals, low basis taxable investments, and borrowing against assets (portfolio, cash value life insurance, house) be considered.
As you design a spending plan beyond that income you must pay tax on, keep the estate planning implications of that plan in mind. Some assets are best left for heirs and others to charity if you are inclined to leave money to either.
If you need assistance with a distribution plan (whether just drafting it or drafting/implementing/maintaining it), book an appointment with a competent, fairly-priced financial advisor.
Estate Planning
Everyone should have an estate plan, but the wealthier you get and the closer you get to your likely time of death, the more important it becomes. Writing a will ensures your assets go where you want them to when you die. A revocable trust helps you to avoid probate. If you anticipate an estate tax problem (i.e. an estate larger than the estate tax exemption of $12.06 million for a single person and $24.12 million for a married couple in 2022), some additional planning can help minimize the bite of estate, inheritance, and even income taxes for you and your heirs. If a large portion of your estate involves illiquid assets such as farms, businesses, or properties, liquidity planning can also be important.
The truth is that many of these wealthy but somewhat anxious docs are highly likely to have an estate tax problem. Using a 4% withdrawal rate, you will, on average, die with 2.7X the amount you retired with. While the estate tax exemption is indexed to inflation, it is scheduled under current law to be cut in half starting in 2026. That means it will be approximately $6 million single/$12 million married, climbing slowly with inflation. If these docs do not want Uncle Sam to be among their primary heirs, they should seriously consider giving money away to heirs and charities now and/or at death.
Asset Protection
Mo' money, mo' problems. The more you have, the more you can lose. Asset protection concerns become more and more important as assets grow. However, most retirees actually have less asset protection risk than younger people, especially as malpractice concerns recede and dangerous habits are eliminated. Insurance is still the first line of defense, including a seven-figure umbrella policy. You are still far more likely to lose money to your spouse than anyone else, just like earlier in your career. Date night is the best asset protection technique. Well-developed spend-down plans and estate plans tend to maximize asset protection naturally.
As you accumulate wealth, you need a way to protect your assets. WCI’s newest book is The White Coat Investor's Guide to Asset Protection, and it provides the techniques you can use to safeguard your money AND the most comprehensive list of state-specific asset protection laws ever published. Pick up the book today and protect your wealth!
Planning for Senility
This group of docs has serious fear that they're “making mistakes we will have to pay for later when we’re most vulnerable . . . and don’t want to get ripped off.” 
I addressed some of this fear above when I talked about an investing plan and a spend-down plan. Some of this fear is addressed by learning enough about the financial services industry to recognize when you are getting good advice/service at a fair price. However, there is some additional fear due to our own decreasing capacity as we age.
There are lots of people that can rip you off, ranging from investment managers and financial advisors to scam artists to your own family. A certain amount of fear of that is probably healthy. While these risks cannot be eliminated completely, they can be minimized. If you are a do-it-yourself investor, then have a backup plan, especially if your spouse is not comfortable. The backup is often an advisor whose role goes from occasional check-ins to full management over time. Make sure your plan includes triggers of when that will occur. Perhaps your plan involves a trusted family member or friend with a financial power of attorney. Redundancy is key. It is best to have multiple people involved who can all watch each other. The elderly are particularly vulnerable to scams. If something seems even a little off, run it by somebody knowledgeable before acting. Also, be sure to treat your kids well since they'll be choosing your nursing home!
Even successful, experienced, wealthy doctors have money worries, and they might be concerned that they're going to get check-mated off the chessboard while they're still playing the game. Financial planning should address most of these concerns. If that isn't sufficient, you can always add a coach or even a therapist to your advisory team.
What do you think? What advice would you give to these anxious, late-career, multimillionaire docs? Comment below!
Thank you for a great article! The description in the email largely fit me so this seems very relevant. I do hope that I haven’t passed most of my go-go years though!
(An action item for me is to plan for managing the investments when I become unable.).
I am curious about what can be done to reduce estate taxes. Currently this is not a big problem but if the investments do well and the scheduled cuts in the size of the exemption occur then there will be a major estate tax haircut!
I have started to give away money but am reluctant to go too far down that road. I have reviewed the Monte Carlo simulations. As you point out there is a likelihood that we will die with a lot of money. On the other hand there is a chance that markets will do badly and that we will spend down a lot of our capital. I don’t want to give away a lot of money and then be unable to support ourselves. Frankly we do spend a lot of money but it’s a reasonable amount to me given our level of income and savings.
I fear these may be irreconcilable goals – I want to keep enough money so as to have financial security in the future yet avoid a major loss to inheritance tax. Any thoughts or suggestions?
First, you won’t be around if there is an estate tax to be paid. Your estate gets the haircut, not you.
To the extent your non-qualified assets have appreciated in value and you are charitably inclined, you could purchase charitable gift annuities. Those appreciated assets would then be out of your estate, there are income tax benefits for you, and you have income to spend during your lifetime. Of course, those assets might have enjoyed a step up in basis at death but who knows if that will be around.
You could buy life only SPIAs (Single Premium Immediate Annuities) with your non-qualified and/or qualified assets removing them from your taxable asset and having lifetime income to spend.
Buying charitable gift annuities and SPIAs to reduce your estate to an amount that won’t be taxed also has the benefit of generating monthly or quarterly payments that you can spend freely that will likely (depending on the age you buy them) provide income greater than the so-called 4% rule would generate.
A final benefit buying charitable gift annuities and SPIAs is that should you lose the interest or the mental capacity to manage your investments, there is nothing to manage with charitable gift annuities and SPIAs…you (or whomever is your agent in your PoA or your Trustee if your non-qualified assets are owned by a revocable trust) just have income to cover your expenses. So this could be one part of an action plan for managing your “assets” if/when you become unable.
You have an estate plan in place now, right? What does the attorney who drafted your estate planning documents recommend?
This article was written for me. Thanks for writing it. I wish it had more answers than it does as after reading it I don’t feel like I know anymore than I did before. It would be nice to read other people who are in the same boat plan of action and thoughts.
I could have written Hicoladoc’s comment so it will be interesting to see what other people like us say about what their plans and thoughts are.
Please write more about this group off people. This is a good first step but we do need more articles and advice geared toward us as even though where we are is a good place to be we still want advice on what to do. Thanks again for writing. And O in these times I think one to two million in cash is a good idea. My thought is inflation may make it lose value but investing can actually cause a greater loss than having it sit and it does increase your security when stock market fluctuations occur that decrease ,hopefully for the short term, your overall portfolio.
Do you have a specific question I could answer? It’s hard to guess what you’re wondering about and write an article about what I think your questions are.
Agreed there is a bit of a strategy shift between accumulation to spending down assets. If I may suggest, this Bogleheads podcast below (Dr. Wade Pfau interview) may be worthwhile for the commenters above for some strategy ideas:
https://podcasts.apple.com/us/podcast/bogleheads-on-investing-podcast/id1436401528?i=1000486733905
Goodness people.
Keep working if you want, but cut back.
Things went well for you.
Learn how to stop worrying. Start to spend, give, and enjoy. Develop hobbies, connect with others, exercise, travel, and live well.
Most of these folks could burn through $400k per year and still be fine!
Thanks for raising this subject, Jim. I’m a physician in my late 40s and retired from clinical medicine within the last year, while my physician wife retired a few years ago (for additional background, readers can check out my three guest posts on WCI – the “Super-saving for an early retirement” series). Until very recently, I was bringing in enough income through two side gigs to cover almost all of our yearly expenses. Then, very recently, my more lucrative side gig evaporated, and now we are faced with spending down our (very sizable) portfolio.
By any rational evaluation of the numbers, we should be living and spending pretty much however we want, as our yearly expenses would have constituted less than 1.5% of our portfolio at its zenith in 11/2021. With the stock and bond markets both down now and our portfolio a couple million dollars lighter, our withdrawal rate will be a little over 1.5% of the current total – still very conservative.
The thing is, we’re feeling a little stressed about money right now. This reaction to our current circumstances is not rational – it’s emotional. We know that we don’t really need the income stream I lost. We also know that our portfolio should easily withstand a sequence of bad returns. But it is very hard to transition from an accumulation mindset to a decumulation mindset. With 40-45 years of retirement stretching out in front of us, it was incredibly reassuring to spend income from my side gigs plus some dividends and interest to make up the rest, without ever touching our principal.
Yes, I know that this is pathologic thinking. Yes, I know that the whole point of saving so much money was so that we could spend it and enjoy our retirement. But I’m still feeling a bit stressed. Our expenses are already up considerably from baseline this year due to several very expensive trips (had to make up for missed international travel during the height of covid), some renovations, and one or two bigger ticket expenses. Over the last few years, we also let our 3-4 year cash cushion dwindle to 1-2 years, mainly because the CD ladder we previously used became a poor vehicle once interest rates were no longer worth locking money up for any length of time. We figured that our bonds would function as a cash cushion-equivalent, but obviously they’ve taken a fair hit.
Yes, I know that none of this really matters when it comes to our statistical probability of portfolio “success” over the next 45 years. I don’t need anyone to explain this to me. As Derek Sivers says, “If more information was the answer, we’d all be billionaires with perfect abs.” My whole point in writing this comment is to let people know that I think it’s understandable – if not exactly normal – to have some anxiety about spending down the portfolio, especially during a bear market. My belief is that it will simply take time – living through a few bear markets in retirement – to gain comfort with the idea that everything is going to be fine.
For now, my wife and I have talked about whether we need to tighten our belts a bit and perhaps cancel, delay, or modify some of these upcoming (discretionary) expenses. We talked for over three hours, and I think it was very helpful to lay all of our thoughts and feelings out on the table. As this comment is already too long, I won’t go into everything we discussed here. Suffice it to say, we have given ourselves about two weeks to cool off and then reconvene, to see how we feel about it all once the emotions settle. This may not be the “right” approach for everyone – or even anyone – but it’s what we feel comfortable with at the moment, so it’s what we’re doing.
Is it sad that two people who engineered such a remarkable financial position at such a young age cannot enjoy it carefree? Perhaps. I’m sure there are many readers who will tsk at us and think we’re myopic fools. But I think it’s important to let others who share our pathology know that they’re not alone.
Thanks for sharing. Your thinking/feeling is extremely common, probably the majority.
Thanks for the thoughtful comment. I’m in a somewhat similar position and understand where you are coming from. I wonder if you ought to consider sitting down with a flat fee financial planner – I know you understand all the theory and what you should do etc., but just have them act as a sounding board, like a psychologist, to go through the behavioural aspects of this. It may help with the anxieties and help you started on a reasonable spending plan. Good luck!
I am getting money out of my estate:
1. Fully funding 529s and planning to pay their educational expenses in whole. My parents did the same for me, and I want to pay it forward as long as my kids understand what a gift this is: financial freedom.
2. Contributing to my kids’ UTMA account.
3. Getting assets in irrevocable trusts: ILIT (life insurance trust), IDGT
4. Loosening the purse strings on vacations and experiences with family.
The problem I have is convincing my parents to do the same. My father in law created a dynasty trust to keep assets out of our estate. If not planned correctly, inheritances will be eaten up by taxes.