Let’s be real, shall we? Getting a mortgage can be a little frightening. Actually very frightening. Who doesn’t get jittery at the prospect of coughing up a huge down payment, of letting lenders probe their financial past, and of ultimately committing to paying back what’s owed, month after month, regardless of what curveballs life might throw? In fact, 46% of potential home buyers fear they won’t qualify for a mortgage to the point that they don’t even try.

But guess what? Many of people’s mortgage fears are largely unfounded. As proof, here’s a closer look at the top things that give home buyers the heebie-jeebies. So, if one of these fears is stopping you, here’s a quick reality check: They aren’t actually as nerve-wracking as their bad rap suggests.

Fear #1: Not having enough money for a down payment

Being able to assemble a down payment is often the most daunting concern among home buyers, but that’s largely because a lot of home buyers think they absolutely need to save for a 20% down payment to get approved for a mortgage, which isn’t the case.

There is, of course, a benefit to putting more money down. If you’re getting a conventional loan a 20% down payment enables you to avoid paying private mortgage insurance. However, the prospect of paying PMI shouldn’t completely dissuade you from buying a home. There are several alternative mortgages available, even a conventional option with as little as 3% down and no monthly mortgage insurance.

There’s a stigma about needing to pay mortgage insurance. If you absolutely cannot qualify for a no mortgage insurance loan then paying PMI in order to get a home loan today instead of buying a few years from now can enable you to take advantage of the current low interest rates.

Still, there are ways to avoid paying PMI. If you’ve served in the military, one way is to get a VA loan, which requires no PMI. Another option worth pursuing is qualifying for down payment assistance; there are 2,290 down payment assistance programs across the country that offer financial assistance, kicking in an average of $17,766.

Fear #2: Not qualifying due to poor credit

Lenders will check your credit score when you apply for a mortgage. Ideally, you’re on the higher end of the credit spectrum—anywhere between 760 and 800—since being in that range will enable you to qualify for the best interest rates. (If so, congrats!) But most lenders will still approve conventional loans for borrowers who have credit scores of at least 650.

Meanwhile, there are some mortgages that have less strict credit requirements. A VA loan, for military service members and veterans, typically requires a credit score of only 620; and a FHA (Federal Housing Administration ) loan requires a credit score of only 580. (Granted, you’ll need to meet other requirements to qualify for one of these loans.)

You should also bear in mind that credit is just one factor that affects the strength of your loan application. Lenders look at your financial picture as a whole. How much money have you saved, what is your debt to income ratio, etc.

Fear #3: Not being able to make monthly mortgage payments

Although foreclosures dropped to a 10-year low last year, foreclosure is still a valid concern for home buyers given the potential for an unexpected layoff. The best way to create a safety net is to build a solid emergency fund before you purchase a house.

Build an emergency fund that will cover living expenses for at least six months which includes your potential monthly mortgage payment. By having a sufficient rainy day fund available, you’ll be able to continue making mortgage payments if you become unemployed for a few months.

Fear #4: Getting in too much debt

Today the average household with credit card debt has balances totaling $16,748, a recent NerdWallet study found. Meanwhile, student debt has surged to an average $28,950 per borrower according to the Institute for College Access & Success. However, having debt doesn’t automatically mean you won’t be able to qualify for a mortgage.

What mortgage lenders care about is your debt-to-income ratio, or DTI, which compares how much money you owe (on student loans, credit cards, car loans, and more) to your income. For a conventional loan, most mortgage lenders require a borrower’s DTI to be no more than 50%. For example, if you make $6,000 a month but spend $500 a month paying off student loan debt, you would qualify for a new mortgage payment of $2500. ($500 + $2500 = $3000,  divide $3000 by $6000 to get a DTI of 50%.

If you’re above the 50% ceiling, there are ways that you can lower your DTI and there are programs like FHA and VA that allow you to have a 55% DTI. The alternative would be to apply for a smaller mortgage, meaning you’ll have to lower your price range. But hey, even a smaller house might be better than none at all.