12 terrible things that could happen if you don't do your taxes

Pay interest

"Statutorily, the IRS can't waive interest," explains Green. "They want the time value of the money you owe them." If you fail to pay, you may be paying a penalty plus interest, which is usually determined by the federal short-term rate (anywhere from 1%-4%), plus 3%, for a total of 4%-6%.

Get notices from the IRS

It's probably fair to assume that no one wants mail from the IRS. But if you don't file or don't pay, that's exactly what could happen. "The IRS gives you multiple opportunities to get it right," says Green. "They have to send you a notice before taking any action, and usually they need a response in 30-60 days. But many people in this situation know it's coming, so they panic when they get their notice and shove it in a drawer to deal with when they have the money."

But here's the thing: Whether you have the money or not, if you don't reach out to the IRS upon receiving a notice, they could start taking action. What kind of action? Well, they can make you …

Forfeit your refund

It makes sense when you think about it. If you owe the IRS money, the agency is not going to hand over any until you pay. For example, if you didn't file taxes in 2014 and the IRS is after you, but you did file for 2015 and are due a refund, you may never see that money. The IRS could simply hold onto it. They can also make you ...

Give up your Social Security

"Through what's called the Federal Payment Levy Program, the IRS has the ability to attack certain assets after going through the appropriate notification process," explains Green. "While they can't inhibit your ability to earn money, take your work tools, or appropriate certain benefits like those paid to your children, Social Security is one thing they can seize."

Receive a federal tax lien

It sounds technical, but basically, a lien is a claim the IRS makes to your property. This claim, however, isn't another notice you can shove in a drawer. According to IRS Publication 594, a lien is a public declaration of the agency's claim to your property in relation to your other creditors. Not only may it be filed to employers, landlords, and creditors, but the lien can make you ...

Lose ground on your credit report

An unpaid debt to the IRS is just like an unpaid debt to anyone else, and it will appear on your credit report. "People don't realize that your credit report reflects your tax liens as much as any other outstanding debt," says Green. We won't even pretend that it could be considered "good debt."

Have your property seized

A lien is a claim to your property; a levy is the actual taking of it. IRS Publication 594 makes it clear that in some cases, the agency can appropriate your house or car, not to mention your income or bank account.

They might restrain themselves if it's agreed that you're suffering "economic hardship," which means their seizure would hinder your ability to meet "basic, reasonable living expenses." Plus, the publication reads, "If there's money left over from the sale [of your assets] after paying off your tax debt, we'll tell you how to get a refund." Make of that what you will.

Receive a summons

If the IRS is having trouble sorting out the taxes you owe, you could get a summons — that's a legal requirement to appear — to meet with an IRS officer, and bring appropriate records, documents, and possibly even testify.

It won't necessarily be you who is asked to meet with the agency: A third party with information relevant to your case, such as a record keeper from a financial institution, could be summoned instead. If the IRS is simply gathering info, you'll be informed of the third-party summons, but if it's in reference to money it's already clear you owe, you might not even find out.

Declare bankruptcy

Let's hope it doesn't come to this. "People who might declare bankruptcy are the people who couldn't pay their taxes because they couldn't afford to pay their mortgage or expenses and get caught in a bit of a bind," explains Green. "Usually it's people who are caught for three or four years not filing, spending the money they didn't pay the IRS on things to try and stay above water."

Remember that bankruptcy isn't magic: While in certain cases, a tax debt can be discharged, if it has turned into a tax lien, it might not be erased. "Instead," clarifies Green, "the IRS will generally suspend the debt and seek to collect it after bankruptcy."

Serve jail time

While jail is unusual for most well-meaning citizens, it is a possibility. "If the government deems that you've willfully failed to file or filed fraudulent returns, they could see it as an attempt to defraud the government," says Green. "In cases where jail time becomes an issue, you typically see two things: a lot of income being hidden from the IRS, and a pattern or some evidence of wrongdoing." Unless you're a dishonest high roller, it's unlikely that the IRS will pursue a jail sentence.

Deal with the IRS for a decade

Did we mention that the government has the right to pursue unpaid taxes for 10 years? While there are certain appeals and exceptions for individual cases, if you've been a negligent taxpayer (or rather, non-taxpayer) you can look forward to a long and close relationship with the IRS.

However, there's hope

The absolute best thing you can do if you've neglected to file or pay, says Green, is reach out to the IRS immediately. It may seem counterintuitive, but the agency is more likely to look kindly on someone who admits they're off track and wants to work it out than someone who has been lining the litter box with their notices. You may be able to negotiate a payment plan or even a reduction of the total owed.

"You shouldn't panic upon getting a notice from the IRS," says Green. "There is some recourse, but your options are more limited the longer you wait to engage."