Tax Diversification means not having all of your assets subject to the same tax effects, particularly during the withdrawal phase in retirement.  Proponents of this concept argue you should have some tax-free money, some tax-deferred money, and some taxable money.  This not only provides against against the possibility of severe changes in tax brackets, but also allows you to control your tax bill to a certain extent in retirement.  Consider the following two physicians in retirement, both 65 years old, married and taking the standard deduction, and both wanting $100,000 to spend this year.

Physician A has a portfolio that is entirely in a traditional, tax-deferred rollover IRA into which he rolled his 401K after he retired.  He really enjoyed the tax break over the years as he stuffed money into the account.

Physician B took advantage of some opportunities to build up a Roth IRA when he was a resident, did a little bit of Roth 401K contributions during his career, and did a few small Roth conversions in the last few years as he cut back on his hours at work. He also had a good size HSA he built up by using a high deductible health insurance plan and a taxable investing account he built up during a few of his highest earning years.

Now they both need to spend $100K, let's say $80K on living expenses, $10K on health care, and $10K as charitable donations.

Physician A pulls $100K out of his IRA.  There are no penalties, since he's over 59 1/2, and there's no required minimum distribution, since he's not yet 70.  But he still has to pay tax.  His adjusted gross income is $100K.  He gets to deduct the married standard deduction of $11,600, and personal exemptions of $7300, leaving a taxable income of $81,100.  The first $17,000 of that is taxed at 10%, the next $52,000 is taxed at 15%, and the final $12,100 is taxed at 25%, for a total tax bill of $1700+$7800+$3025=$12,525.

Physician B has a lot more options.  The stock market actually had a bad year, and he was able to tax loss harvest some of his taxable account assets, and thus will deduct $3000 of losses from his taxes.  He then uses some of his appreciated taxable account assets to make his $10K charitable contribution, all tax free.  He then uses his HSA to pay his $10K in health care costs.  This leaves him $80K in expenses which he needs to pull out of his traditional and Roth IRA.  He then decides to pull $40K out of the traditional IRA and $40K out of the Roth IRA.  The $40K out of his Roth comes with no tax bill, just like the taxable account and the HSA money.  He is able to deduct his $3K in investment losses from the $40K in income he generated by pulling it out of the IRA.  What is his overall tax bill on the same income?  Well, he has $37K in adjusted gross income.  He then deducts his standard deduction and personal exemptions, leaving him a total of $18,100.  The first $17000 of this is taxed at 10% and the last $1100 is taxed at 15%, for a total tax bill of $1700+165=$1865.

One doc pays an overall tax rate of 12.5%, the other pays 1.9%.  The beauty of tax diversification.

Now, there are two lessons worth learning here.  First, there is some real benefit in having some options when it comes time to start spending your retirement stash.  Second, and a bit more of a subtle point, is that most attendings ought to have the lion's share of their retirement assets in a tax-deferred account anyway.  Consider even the doc with no tax diversification.  His overall tax rate is only 12.5%, even though his marginal tax rate is twice that.  But when he put this money away, he was probably in the 28-33% bracket.  So he got a tax break up front of 33%, and then only paid taxes on that money at 12.5%.  There's a real arbitrage opportunity there.

So, at the end of the day, try to get some tax diversification.  Just don't be stupid how you get it.  As blogged by The Finance Buff, you don't necessarily want to be missing out on tax-deferred savings by doing Roth IRAs and putting all your savings into a Roth 401K during your peak earnings years.  Here are some ways you can ensure you have some tax diversification:

1) As a resident, max out that Roth IRA.  If you spend a few years with a lower salary, as a fellow or a military doc, continue doing Roth IRAs or a Roth TSP.  When you're in a relatively low marginal bracket, the up-front tax break of a traditional IRA or 401K isn't worth that much.  These are the years to get that Roth IRA going.  Then you have your whole career for this money to compound into a significant sum.  By the time you retire, hopefully it will be a decent portion of your total portfolio.  But even if it is only 10-20%, that'll give you some options to reduce taxes in retirement.

2) If you retire early, get disabled, have a sabbatical, get laid off, become a stay-at-home parent, or for whatever reason have a lower income one year, consider doing some Roth conversions.  If you can convert tax-deferred assets to Roth assets at 0, 10 or 15% , you'll likely be glad you did.  Not only will it give you tax diversification, but it will also reduce the required minimum distributions you will have to pay after age 70, and possibly provide a great estate planning vehicle.  (Stretch Roth IRAs are far better than stretch traditional IRAs.)  Roth conversions are also a wonderful way to minimize state income tax.  Consider a doc who works in California during his career, deferring his income into a 401K where he saves 33% on federal taxes and also 9.3% on state taxes.  Then, during a period of early retirement when he lives in Nevada, he does some conversions at 10-15%, without any state tax at all.  He may later return to California and spend the money from his Roth IRA, both federal and state tax free.

3) Save a lot of money.  If you've already maxed out your $49K 401K/403B/SEP-IRA/Solo 401K limit, you can then do backdoor Roth IRAs for another $5K ($10K if married, $12K if married and over 50).  I wouldn't recommend doing this instead of getting that upfront tax deduction, but it is far better than investing in a taxable account.  Once you max that out, you may consider investing in a taxable account.  A taxable account has lots of unsung benefits including paying lower capital gains tax rates (and even that only on the gains), a step-up in basis at death, tax-loss harvesting opportunities, and charitable contributions where neither you nor the charity has to pay the capital gains.  Those who save a lot may also consider using a Roth 401K, even at a relatively high bracket, for some of your savings, since the contribution limits are the same, and after-tax money is worth more than pre-tax money.

4) Use an HSA.  Ideally, you'll want to spend money in your checking account on health care and let the HSA grow until you're in retirement.  But if you use it for health care then, you'll get not only the up-front tax break and decades of tax-free growth, but also tax-free withdrawals.  Triple-tax free.  That's the stealth IRA for you.

Taxes Made Simple: Income Taxes Explained in 100 Pages or LessAccounting Books)