Should you pay down that mortgage?

TL;DR: You can reach Ramen Retirement sooner, and generate greater long term wealth by taking out a mortgage on your primary home and investing those proceeds in yield generating assets like real estate. But there are times when it might make sense to pay it down instead. Depending on the size of the mortgage or the age of the mortgage, you might be better off paying it down sooner to reach Ramen Retirement. Also, there can be a number of other reasons to put your money towards your mortgage if you can’t find good investment alternatives. Paying down your mortgage is like a virtually risk-free investment that can generate solid returns while balancing out the risk of your portfolio.

I’m a big fan of primary home ownership – it’s the best way to lock in your housing costs and get neutral real estate. But one question I struggled with for some time was whether I should pay down my mortgage early or not. There is plenty of commentary on this topic. Essentially the debate boils down to whether you can get a better return investing that money elsewhere.

From my analysis below, you’ll see that taking out a mortgage to invest the proceeds can be a good decision – though you’ll need to focus on higher yielding investments like cash generating real estate. Stocks and bonds won’t do. But while you might be able to get better returns, that doesn’t tell the whole story. Paying down your mortgage is like buying a risk-free bond with returns in the 6-8% range pre-tax (way better than US Treasuries!). This can be a great way to balance out the risk in your portfolio and get yourself one step closer to Ramen Retirement. Moreover, there are some situations where it might make sense to pay down the mortgage – particularly if the markets are overheated, and compelling returns are hard to find.

The ultimate goal is to free your time and energy to discover your gift and share it with the world. There are many ways to do that. Let’s dig into the details below to understand the tradeoffs, and figure out what makes the most sense for you!

The $1M mortgage

We’ll start first by looking at a couple that owns a $1.25M home with a $1M mortgage (the max that can be deducted). Assume the couple earn $400K in personal income, and file taxes jointly in California.

To understand the tradeoffs, we first need to break down the mortgage economics.

Mortgage payment

In early 2018 you can get a 30 year fixed mortgage with a ballpark 4.5% interest rate. This translates to ~$61,391 in mortgage payments per year: ~$45K in interest, and ~$16K in principal. On the surface, the 4.5% interest rate is a good metric to compare against alternative investments, and if you make that a pre-tax amount (assuming a marginal tax rate of 43.65%), then a competing investment would need to make ~8% per year. This is awfully close to the long term S&P 500 performance, so on the surface it doesn’t seem like a great deal simply to take the mortgage proceeds and invest them in the stock market. Similar return for much greater risk.

Value of mortgage interest tax deduction 

But… the 4.5% interest rate is not the effective interest rate due to the mortgage interest tax deduction. This allows homeowners to deduct the interest expense from their mortgage against their personal income. This deduction can be valuable for people in a high tax bracket – for example, if you’re paying a marginal tax rate of 50%, then the mortgage tax deduction would theoretically reduce the true cost of your mortgage from 4.5% interest to 2.25%. But there are a few catches. First, the deduction disappears for any mortgage amount over $1M. Also, people often don’t account for the fact that this deduction is not additive to their standard deduction – i.e. they can either take their standard deduction ($24K for couples filing jointly), or they can take an itemized set of deductions that would include their mortgage interest expense and their state / local taxes (note: state / local tax deduction is capped at $10K). So in this example, eligible mortgage interest of $45K and $10K state / local deductions totals $55K in potential tax deductions. To get the true value to this couple, we should only look at the ‘incremental’ tax deduction above and beyond the standard deduction they would get anyhow – so subtract $24K from $55K and we get $31K in incremental deductions for having the mortgage. Now, with $400K in income, this couple is paying a marginal tax rate of 43.65%, which implies a value from the deduction of $13,209… this brings the true cost of the mortgage interest from $45K down to $31K – making for an effective interest rate of 3.1% (vs. the 4.5% stated interest rate). So the true interest cost in this situation is 3.1%. Not bad. To outperform that from an investment perspective, this couple would need something yielding ~5.6% pre-tax. That’s feels achievable in the public markets, but it’s not a slam dunk – long term S&P 500 returns of 7-8% vs. 5.5% is not a HUGE difference. So it begs the question of whether it’s worth it on a risk-adjusted basis. You can take a 5.5% virtually risk free return by paying down your mortgage, or you can roll the dice at the 7-8% long term market returns – much higher risk. If you’re looking to balance your portfolio risk, it seems like paying the mortgage down isn’t a terrible idea.

Interest cost vs. cash cost of mortgage

Of course, in addition to long term returns, I am also quite interested in reaching Ramen Retirement as soon as possible. With a mortgage you pay both the interest and the principal. This means the effective cash cost to cover the mortgage is closer to 4.9% after tax ($61K less the $13K tax savings means a true cash outlay of $48K to carry the mortgage)… which equates to a ~8.69% pre-tax yield, which means this couple would need substantially higher yield today to cover that carrying cost. The stock market will likely yield ~2% in dividends, so in this case, using the mortgage proceeds for buying stocks will push out the time it takes to reach Ramen Retirement. If you wanted ~8% yields, it is possible to do with certain types of real estate investments, but they definitely have higher risk than simply paying down your mortgage.

So to summarize, in theory, if the married couple is able to invest their money elsewhere and get a ~5.6% or greater return, they will end up ahead over the long term. This seems achievable with a low cost index fund that tracks the market, but it doesn’t feel like a slam dunk. If they want to cover the carrying cost of their mortgage to reach Ramen Retirement, they’ll need an even higher cash yield of ~8.7%, which requires some pretty damn good investments in things like real estate. But how about we run a few more numbers to show what this looks like in terms of years to Ramen Retirement, and net worth over time.

Control group – no mortgage

For the control group, assume this couple is 30 years old, with some retirement savings, but no stocks or bonds. Assume they have no mortgage on their home, and invest future savings 50/50 into stocks and real estate. In this scenario they could be Ramen Retired by 38, and by 60 their net worth would be $48M.

Screen Shot 2018-02-28 at 9.27.59 AM

Invest mortgage money in stocks 

If the couple instead takes out a mortgage for $1M on the home and invests it in stocks, Ramen Retirement jumps to age 41, and net worth at 60 would be $49M… in this case, the financial outcome is basically the same, while the couple has to wait an additional 3 years to reach Ramen Retirement.

Screen Shot 2018-02-28 at 9.29.42 AM.png

Invest mortgage money in real estate

If the couple instead invests that $1M in real estate, they reach Ramen Retirement at 34 – 4 years sooner! By 60 years of age, they’re worth $70M.

Screen Shot 2018-02-28 at 9.32.02 AM.png

What can we conclude?

There are a few key insights here. First, the only difference between paying down your mortgage, or investing that money in stocks is that with the stock investment, you reach Ramen Retirement 3 years later. This makes sense, because stocks typically yield in the 2% range, which isn’t enough to offset the monthly payments to service the mortgage – so naturally Ramen Retirement will be pushed out. Not a great outcome. In addition, the long term returns from stocks don’t really outperform the returns achieved by simply paying down the mortgage. At least not by much in this example.

If, instead, you invest that mortgage money into cash generating real estate investments (8%+ cash-on-cash yields), you can reach Ramen Retirement 4 years sooner than the base case, and make more money over the long term. Of course, this is all dependent on the assumptions we make around financial returns to stocks and real estate, but the numbers I’ve used here are quite achievable given historical experience and the current environment in 2018 (to learn more about potential real estate investments generating solid cash returns, drop me a line at ramen@ramenretirement.com).

But not everyone owns a home that could use a million dollar mortgage. What if you only have $500K in mortgage debt?

A more modest mortgage – $500K

For this example, we’ll look at a $1.25M home with a $500K mortgage. Assume the owners are a married couple who earn $400K in personal income, and file taxes jointly in California.

Mortgage payment

The mortgage payment on a $500K loan would be ~$30,695 per year: ~$22K in interest, and ~$8K in principal

Value of mortgage interest tax deduction

In this case, the incremental deduction beyond the standard deduction is only $8,500 for having the mortgage. This implies a value from the deduction of $3,710… this brings the true cost of the mortgage interest from $22.5K down to $18.8K – making for an effective interest rate of 3.76% (vs. the 4.5% stated interest rate)

Interest cost vs. cash cost of mortgage

To outperform the interest cost from an investment perspective, this couple would now need something yielding ~6.6% pre-tax (vs. ~5.6% in previous example), which is much closer to the long term S&P 500 performance. The effective cash cost to cover the mortgage is closer to 5.4% after tax, which equates to a ~9.5% pre-tax yield for this couple – that’s a pretty high bar from a cash flow perspective, even for real estate investments.

So you can see that in an example of a more modest mortgage – one where you haven’t maxed out the deductible mortgage debt, it starts to look less attractive. The interest cost alone would be hard to outperform with the long term returns from the stock market, and the cash cost of the mortgage would definitely be higher than any sort of yield you could get from stocks. You could invest in real estate and potentially get yield to offset the cost of the mortgage, but it’s starting to get pretty close to the high end of expected real estate returns as well. You might do better over the long term, but probably not by much, and you’ll have the added risk of those real estate investments performing poorly.

See this chart below for a summary of the outcomes for this same couple with no mortgage, and various levels of mortgage debt. You can see that as the mortgage amount goes down, the cost of debt and cash coverage goes up. A modest $300K mortgage barely outperforms the ‘no mortgage’ scenario.

Screen Shot 2018-03-04 at 9.41.43 PM.png

So should you pay it down?

Well, the numbers would suggest there is a long term benefit to keeping the mortgage and investing elsewhere, but depending on where / how you invest, keeping the mortgage might push out your time to financial independence and Ramen Retirement. But, there are a number of other reasons you might want to pay it down anyway!

Can’t find great real estate deals:  

Can you find profitable real estate investments that will yield 8%+ cash-on-cash? If you can’t get access to the deals that will generate a strong, consistent return, then the strategy won’t work. If you need help learning more about how to get into real estate and find those types of deals, drop me a line at ramen@ramenretirement.com and we can discuss options that might work best for you.

Don’t like the risk-reward of the current market

In early 2018, assets of all types are way up. Things are euphoric. While nobody can predict a recession, we can certainly say that valuations look stretched, and the mood is euphoric. This is not the ideal environment to be deploying capital. Prospective returns are likely to be lower than the last ~8 year run has delivered. As always, we should move forward, but in this sort of environment move forward with caution. One way to make a more cautious bet in this environment is to pay down your mortgage. If the markets tank, it won’t affect you in the slightest. Instead, that’s exactly when you could refinance your home and pick up some assets on the cheap. In this way, paying down your mortgage is like buying a risk free bond that will yield anywhere from 5-8% pre-tax on the interest cost alone… and the best part is that you sell that bond whenever the next recession strikes. Usually when the economy slides into recession, the federal reserve responds by lowering interest rates to stimulate the economy – this means you’ll be able to take out cheap debt at exactly the right time to have capital available to you… when there is blood in the streets!

Have a lump sum payment

Sometimes we get hit with a windfall of cash. Stock options vest. A company gets acquired. When you have a lump sum payment that makes up a sizable share of your net worth you might not want to invest it all at once – for reasons of diversification, you don’t want to put all your money into one deal, or invest it all in a single timeframe. Particularly if it feels like valuations are stretched (as in early 2018). By paying down your mortgage, you’ll get that incremental cash flow available to you spread out over a longer time frame, giving you a chance to diversify your investments, mitigating the risk of buying at the ‘peak’.

Want to reduce your risk:

The one thing that I don’t model in the above scenarios is risk. The risk that real estate markets fall, and rental rates from your properties go down. Assets don’t always rise (despite what 2009-2018 has shown us). In the above mentioned scenario, paying down your mortgage is a much more certain way to reach Ramen Retirement. It serves as the conservative, low risk anchor to a ‘barbell’ investment strategy. If instead you keep the mortgage and invest the proceeds, it’s possible it will perform poorly in the early years, and affect your ability to reach Ramen Retirement, while hurting your long term net worth. Employing a ‘barbell’ investment strategy is a useful way to manage risk by covering the downside, while exposing yourself to large upside potential. Paying down your mortgage is one of the more conservative investments you can make, reducing risk and getting you much closer to Ramen Retirement.

So what’s the answer?

I hope you can see that this decision to pay down the mortgage isn’t so black and white. It depends entirely upon your goals, risk preference, and the market environment for assets you might buy. If there is blood in the streets (much like March 2009) it could be a great time to take out a mortgage and invest that money in a range of assets. If the markets are buoyant and euphoric (much like early 2018), then it might make sense to be more conservative and pay off that mortgage for the time being.

As you can imagine, my goals are largely focused on reaching Ramen Retirement as soon as possible, while maximizing long term wealth. From my experience, the best way to do this is through a portfolio of cash generating real estate. But I’m also sensitive to the fact that you can’t build out a real estate portfolio like that over night, and it’s always good to balance a portfolio with a range of assets with varying degrees of risk. So when you’re hit with a large amount of cash you can’t or don’t want to invest all at once, it can make a lot of sense to pay down the mortgage instead. Also, when markets look overheated, take a breather and pay down your mortgage. It’s not an irreversible decision, and you can always get that money out to invest should you need it. In the meantime, you’ll have it working for you at a pretty decent rate of return.

As with everything in life, your mileage may vary. If you want to run some scenarios for yourself, take a spin through my Ramen Retirement Lifetime Earnings model where I also model out the tradeoffs of paying down the mortgage. Request access and you’ll be able to customize the model to your specific situation. If you want help thinking about the decision, you can always drop me a line at ramen@ramenretirement.com… or comment below. I hope this helps you with your decision, and gets you one step closer to your Ramen Retirement!


Addendum

As you pay down your mortgage, the amount of interest you pay goes down, and the amount of principal goes up. This has two effects:

  1. Decreasing value of mortgage tax deduction: When you pay less interest, your mortgage interest tax deduction goes down, and the effective interest rate goes up
  2. Increasing cash cost of mortgage: Over time the effective mortgage payment as a percent of the mortgage balance goes up – this makes sense because the mortgage balance is decreasing each year while the mortgage payment remains the same (more principal, less interest over time). This is fine from a returns perspective, but when thinking about Ramen Retirement, the yields you would need from your investments to outweigh the reduced payment of the mortgage start to get quite large. In the below example, by year 8 of the mortgage, the effective cash payment is 5.81% of the mortgage balance, which is equivalent to a 10.31% pre-tax cash yield, which is close to what you might get with a good real estate investment

Effective interest expense and cash cost of mortgage, as percent of outstanding mortgage balance

Screen Shot 2018-03-04 at 10.29.38 AM

So even in this most extreme example with a million dollar mortgage (the largest mortgage amount that can benefit from tax deduction), there are reasons and situations where you might want to pay down early as the mortgage matures. Of course, one alternative is to constantly refinance your home and use that added cash for investments. This would keep your property in the ‘sweet spot’ from a tax and payments perspective, but would also incur higher costs of refinancing every ~5 years, in addition to the potential risk that interest rates will rise and that refinancing will not look as attractive.

Leave a Reply