The Value of Debt in Retirement – A Review

By Dr. James M. Dahle, WCI Founder

Earlier this year, Thomas J. Anderson published a follow-up to his “NYT Bestselling” book The Value of Debt called The Value of Debt in Retirement. (Interestingly, both books are ranked well below my book on Amazon, maybe I should call the NYT.) At any rate a reader wrote in and asked me to review it. Accordingly, I wrote the publisher asking for a review copy and received not the 1-3 I asked for, but 7 hardback copies of the book. Must not be that many people asking for them, but I suppose that's your gain and my loss since it not only costs me money to ship them to you, but I also miss out on the Amazon commission I would get if you actually bought it from the links on this page! At any rate, after looking at the portion available for free on Amazon, I knew I was going to enjoy this book because the author not only writes about a subject rarely been touched by other personal finance and investing authors, but argues convincingly for this unconventional strategy.

The conventional wisdom (which I have also generally subscribed to) is to go into retirement with as little debt as possible. The benefits of this include the elimination of a number of risks and frequently an improvement in cash flow. As one Boglehead said about paying off his mortgage, “I don't know if it was the right decision or not, but I do know that I only need $6,000 a month where I needed $8,000 before.” The entire contention of Mr. Anderson is that for some people, and he is very careful to describe those people, that is not the right move. In fact, he advocates taking on even more debt as you go through retirement. I love books that challenge my current thinking like this one, especially when the arguments are well-written. I found myself agreeing with him on certain points and disagreeing on others. I'll try to illustrate most of the basic concepts in the book, and then offer a few criticisms of his ideas (many of which he readily acknowledges in the book).

 

Basic Concepts in the Book

 

Three Types of Debt

Per the author, there are three types of debt. The first is oppressive-think payday loans, credit card loans, high rate auto loans etc. The second is working debt-think mortgages, student loans, business loans etc. These are loans that are often thought of as “good debt,” meaning debt that helps you build wealth. The third type is enriching debt. He considers this to be debt that you have the assets to pay off at any time. It should be low interest and have excellent terms. Preferably, it should be non-callable, require no mandatory payments, be tax-deductible, and be relatively low interest. His usual examples are interest-only mortgages, margin loans against your taxable portfolio and an “Asset Based Loan Facility.” He's not entirely clear on what that is, but as near as I can tell, it's a loan from a bank where you offer some type of asset as collateral. Most of the time he uses the phrase it sounds like a margin loan.

 

An Optimal Debt Ratio

Mr. Anderson suggests that people should operate their financial lives more like businesses operate. Sometimes it is best for businesses who need capital to issue stock, and sometimes it makes sense to issue bonds (i.e. borrow money) depending on the cost of capital. But very few large corporations carry no debt at all. He suggests that there is an optimal ratio between your total assets and your debt (which should be less than the optimal ratio for most corporations, of course.) For anyone within 20 years of retirement or retired, that ratio is between 15% and 35%. So, for someone with $2 Million in assets, he's suggesting carrying something like $500K in debt is optimal. His ratio does not really change much as you get older.

 

Debt Can Help Reduce Taxes

One of the big benefits of using debt instead of equity for an individual is the tax treatment. You don't pay taxes on money you borrow. This isn't a particularly new concept to me or most readers of this blog. Real estate investors know they can borrow equity out of their investments tax-free (but not interest-free) and spend it on whatever they like. Cash value life insurance works the same way. However, this book is the first time I've seen somebody really advocate the technique for someone using a more standard stock and bond portfolio. The idea is that you borrow rather than sell any of your assets. Then, when you die, all the debt is paid off and your heirs get the step-up in basis. Thus no capital gains taxes are ever paid. They're entirely eliminated. I find that idea appealing, especially since it isn't combined with all the downsides of investing in insurance products, nor the transaction costs and hassle associated with refinancing your residence or even an investment property (especially difficult after retirement.) However, just like with whole life insurance, the interest rate and terms of the loan really matter. You're still trading taxes for interest.

 

Leverage Risk May Be Lower Than Your Other Risks

While Mr. Anderson is quick to acknowledge that there are risks associated with leverage (mainly that it works both ways,) he makes a strong argument that this risk may be lower than the other risks an investor is entirely too comfortable taking. His idea is that you may be better off leveraging up a portfolio with a 6% expected return than having an unleveraged portfolio with a 9% expected return.

 

Debt Might Help You Sustain a Higher Withdrawal Rate

He makes a number of arguments in the book that using debt in retirement can actually help you to sustain a withdrawal rate of higher than 4%. In fact, he even provides a “Trinity-like” graph in the appendix that suggests that using debt on a 100% equity portfolio may provide a success rate of 49% on a 12% withdrawal rate. That's not a typo-12%. That's almost four times the success rate on an unleveraged portfolio.

 

Many (Most?) People Shouldn't Do This

One of the greatest strengths of the book are his warnings that this strategy is not for everyone. Perhaps the best explanation he gives about who shouldn't do this is in his 7 Rules for Being a Better Debtor

1. Honestly assess whether you can handle any debt at all, including better debt. Some might consider Anderson the “Anti-Dave Ramsey.” I don't think he necessarily is, although Ramsey feels that all debt is bad. However, they both agree that the right answer is to get rid of all of your oppressive debt as soon as possible, and that most people can't really handle any debt safely.
2. Never overextend yourself, even for better debt. Anderson would argue that when Ramsey went bankrupt, it was not because he was using debt, but because he overextended himself.
3. Make sure any new debt you take on actually is high-quality (better debt). For example, low rate, flexible terms, and perhaps some tax advantages.
4. Only take on better debt in the context of a thoughtful, holistic, professionally vetted plan. Like most books written by financial advisors, the book is filled with suggestions to go hire one that understands this stuff. I would say you either need a good advisor, or you need to become as knowledgeable on the subject as one. Personally, I found it irritating that all the examples where he used debt to show just how low your taxes could be seemed to include paying an advisor $20-50K a year (and deducting the payment).
5. Get rid of all low-quality and oppressive debt as soon as it's feasible to do so.
6. Don't necessarily rush to pay off existing debt.
7. If you do take on debt, be conservative and scientific.

It's hard to argue with him when he gives those caveats.

 

The Need-Want-Have Matrix

Although the name is a little goofy, this section of the book was one of the better parts. It helps the investor know if this strategy is for him. It basically comes down to four questions:

Do you need to take on enriching debt? If you have enough assets that a 3-4% withdrawal rate will provide for all your needs and wants, then congratulations, you have no need to add debt to the mix. If you need a 6% withdrawal rate, you will need to take more risk, and perhaps you are better off with leverage risk than additional market, small, value, momentum, quality, interest rate etc risks.

Do you want debt? If you're a Dave Ramsey disciple, or have a religious objection to debt, or even just happen to be relatively debt averse, then you shouldn't follow a debt strategy in retirement. Mr. Anderson suggests you be more open-minded, but if the debt is going to gnaw at your soul for decades, then this isn't for you.

Do you have debt? If so, what is your ratio? If you're a typical recent residency graduate, your debt to asset ratio may be something way over the 15%-50% Anderson recommends for anyone. It might be 20,000% if all you have is $200K in student loans and $1,000 in your checking account. The idea is that if you are way above the recommended ratios, taking on more debt is not wise. You need to pay down your oppressive and working debt as you are way overextended relative to your assets.You will also want to look at what your debt ratio is likely to do in the future. Here is where you decide that perhaps you won't pay down your low-interest mortgage any faster than necessary in order to keep your ratio optimal.

Do you currently have, or can you create access to better debt? The idea here is that for someone who is already retired with a paid off home whose only other assets are all in retirement accounts that it will be very difficult to create better debt, and thus this strategy is not an option for them.

 

Another Great Pearl – Second Home Rent vs. Buy Conundrum

I thought one other great section of the book is when he discusses the rent versus buy conundrum for a second home, as I don't really recall ever seeing guidelines for this sort of thing before. He says you should rent if:

• You are at a property less than 30 nights per year
• You are only at a property for a consecutive series of dates and not in and out throughout the year
• You plan on being at that property for less than three years
• You do not intend to rent that property when you're not there.

You should buy if:

• You intend to rent it out (which case you should evaluate it first as an investment)
• You will be there more than 30 nights (and especially if more than 120 nights) per year
• You will be there throughout the year.
• You believe there is a likelihood of appreciation at least equal to or greater than inflation (i.e. you're buying at a great price.)

 

How It Works

So practically speaking, what is Anderson suggesting you do? He suggests you carry an interest-only mortgage on your house, get an interest only mortgage on any second homes you buy, and borrow against your taxable investment account (rather than liquidate it) for everything else (cars, boats, children's education, down payments for your kids' homes, gifts to children, charitable donations, vacations, routine spending, etc.) until your debt ratio is optimal. However, he is careful to limit that amount of borrowing to 1/4 of the taxable account to try to avoid margin calls in a big bear market. He never comes right out and says it, but a less than careful reader may think he is saying that you should pass up opportunities to max out retirement accounts in order to build a sizeable taxable account. I think Mr. Anderson would prefer to see that you have both sizeable retirement accounts AND a sizeable taxable account. Thus, in retirement, you're living off of Social Security, the dividends from your taxable account, the RMDs from your tax-deferred accounts, and then borrowing (up to the mentioned limits) against your taxable account.

 

My Criticisms

The reader that wrote in to ask me my opinion of this book had already read it. He wanted to know what I thought of it. I admit the book certainly got me thinking. I'm a relatively debt-averse person, despite the fact that I've never really had a problem managing the debt I've had. I simply don't like owing anybody anything. But I also don't like pissing money away, and am certainly open to strategies that help me increase return, lower risk, spend more, and give more. At any rate, here are the criticisms I have of the book and of Mr. Anderson's advocated techniques.

 

#1 The Wrong People Will Use the Techniques

The people who can profitably use these techniques are those who are most repelled by them. Those who are attracted to them are exactly the people who should not use them. I run into docs all the time who are way too comfortable with debt. It is not uncommon for me to hear of somebody with $400K in student loans plus a nothing down $600K mortgage plus a $500K buy-in/practice loan, who might have a high income but who has a five figure net worth. The person who can profitably use these techniques is probably the debt-averse supersaver who will have no problem living off of 3-4% of his stash in retirement, and thus doesn't need these techniques. So it is really just a very tiny percentage of the population who has both the ability, and the need (or at least the desire) to use these techniques.

 

#2 You Need a Massive Taxable Account

Along those same lines, Mr. Anderson's main loan type are these margin loans against the taxable account. But he (wisely) says you should only borrow an amount up to 1/4 of your taxable account. So if you have a $1M IRA, a $500K house, and a $500K taxable account, he's saying you should only borrow up to $125K. Well, that ratio is 6%, way below his minimum recommended 15%. So now we're looking for people who not only can handle debt very wisely AND have a need to use it, BUT ALSO have a huge taxable account. Slim pickings now unless you pull money out of your home by refinancing just before retirement in order to have a taxable account.

 

#3 Margin Loans Are Both Callable and Variable

While it may be true that the risk of leverage is less than the risk of short-term and long-term market losses,  (I'm not sure I know how to quantify any of those in a meaningful way,) margin loans have two very serious risks associated with them. The first is the call risk already discussed. In a big market downturn, if your ratio of debt to equity gets too high, you get a margin call and the company just starts selling all of your assets at the worst possible time. The second is that you introduce interest rate risk, since these loans are almost all at a variable interest rate. Mr. Anderson would counter that this enriching debt can be paid off at any time, but the way it is paid off is by selling the assets, which are likely to be doing poorly in any scenario in which you need to sell them-in a market downturn or in a time of rapidly rising interest rates. Buy high, sell low is not a winning investment strategy.

 

#4 Leverage Still Works both Ways

In every scenario in the book, leverage seems to only be a good guy. As anyone who bought a home with a mortgage in 2006 knows, leverage can be a very bad guy too, even over a very long period of time.

 

#5 Loans Are Negative Bonds

Now, I'm no expert in getting margin loans, and I'm sure Mr. Anderson is better at it than I am. But on the day I wrote this post I went to Charles Schwab to see what margin rates were available. I was surprised to see they ranged from 6-8.5%, especially when every example in the book seemed to suggest I could get these loans for something like 3.25%. I did find Fidelity offering 3.75%, but only on amounts over $500K. Vanguard is 5.25-7.75%. I found 3.89% if you borrow more than $1 Million from Etrade. I suspect many of the examples in the book wouldn't look nearly as good if the cost of the loan was assumed to be 6% instead of 3%. But my main point is this- a loan is a negative bond. If you have a portfolio with bonds in it, chances are very good those bonds aren't even paying 3%, much less 6%. Their yield is probably 2%, and if it were to rise significantly, margin rates would probably also go up. It seems awfully silly to me to borrow money at 6% in order to invest it at 2%. Mr. Anderson always looks at the expected return of the entire portfolio (and seems to like 9%, which seems awfully optimistic for a portfolio with a large percentage of bonds.) But in reality, if I have a 6% loan, or even a 3.25% loan, I'm much better off paying it down than investing in a 2% bond.

 

#6 Retirement Accounts Probably Provide More Benefits Than Debt

Perhaps my biggest criticism of the book is his failure to acknowledge the massive benefit of using retirement accounts. It would be fun to hear his thoughts on the cogent arguments made by James Lange in Retire Secure in favor of using a retirement account over a taxable account. I suspect it is far wiser to use your taxable account, even if you have to pay some LTCG taxes, to pay for Roth conversions than to try to increase your debt ratio. His many examples where he uses debt to keep taxes low also never seem to include a Roth account. Roth withdrawals are pretty cool- they're not only tax-free, but interest-free as well. But even a tax-deferred account provides a huge boost to the size of your nest egg, not only from the tax-protected growth but also from the arbitrage between the marginal tax rate(s) at which you contribute money and the marginal tax rate(s) (which may include 0%, 10%, and 15%) at which you withdraw it. And this isn't even considering the asset protection and estate planning benefits of retirement accounts.

 

My Takeaway from the Book

Overall, I highly recommend the book. It is an eye-opening exploration into a topic rarely if ever covered in the personal finance and investing literature. The math doesn't lie in that if you can somehow borrow at a low rate and at good terms while investing at a higher rate of return you will come out ahead (as long as you can survive any rough periods.) The tax law indeed provides for real tax advantages for those who borrow while awaiting the step-up in basis at death. There may indeed be times when using enriching debt may be the ideal decision. But I probably won't be going all out for this strategy for a number of reasons:

1. I have no need to take on debt- 3-4% should provide me a very comfortable retirement
2. I have little desire to take on debt- I'm generally debt-averse for personal, cultural and religious reasons
3. I will probably never have a very large taxable account- I tend to use it for my charitable contributions or invest it in relatively illiquid real estate.
4. I can't seem to find margin rates higher than what my bonds are paying, and I'm not psychologically able to handle a 100% equity portfolio, much less a leveraged 100% equity portfolio.
5. I probably won't be buying any vacation homes.
6. I highly value the luxury and improved cash flow of having my main home paid for.

All that said, I'm a little more comfortable with debt than I was before and I've learned a lot of really academically creative techniques where it can profitably be used. I do feel a little better about my plan to save up an amount equivalent to my mortgage (after-tax rate of 1.6%) in a taxable account rather than pay it off. I wouldn't be surprised if many readers find some of the ideas in the book to be very handy, even if they don't buy into them wholesale. While it should not be the first (and especially not the only) investing book you read, advanced readers of this site looking for something new and interesting will not be wasting their time or money.

Want a free hardback copy of this book (signed by me but not Mr. Anderson?) All you have to do is  “Like” this site on Facebook, and we'll choose six random Likers to get the books. The winners will be announced in a few days in the comments section.

If you didn't win, buy The Value of Debt in Retirement on Amazon!

What do you think? Do you believe there is an optimal debt ratio? Is your current debt ratio between 15% and 50%? Why or why not? Would you consider carrying “enriching debt” into retirement? What did you think of his second home buy vs rent recommendations?