A sign hangs from a branch of Banco Santander in London, U.K., on Wednesday, Feb. 3, 2010.

The Federal Reserve's interest rate hikes have caused banks and other mortgage providers to lose business.

Tim Wennes, CEO of the U.S. division of Santander, said that some firms will be forced to leave the industry.

He knew that the product was going to be dropped by the small player in the mortgage market.

Wennes said in a recent interview that other people are doing the same math as they were when they first moved to this area. The majority of mortgage volume is refinance activity which is drying up and will likely cause a shakeout.

The mortgage business boomed during the first two years of the epidemic due to low financing costs and a preference for suburban houses. Black Knight said the industry had a record $4.4 trillion in loan volumes last year.

Many Americans can't afford a home because of surging interest rates and home prices that have yet to decline. Black Knight reported that rate-based refinances sank through the first four months of the year.

The move by Santander, part of a strategic pivot to focus on higher-return businesses like its auto lending franchise, now seems like a good idea. The Madrid-based global bank with operations across Europe and Latin America has about $150 billion in assets.

The largest banks in home loans have cut mortgage staffing levels to adjust to the lower volumes. Smaller nonbank providers are trying to sell or even partner with rivals.

Wennes said that the sector was as good as it gets last year.

He said that they looked at the returns through the cycle, saw where they were headed, and decided to exit.

Home loans have been a central part of the American mortgage business since the 2008 financial crisis, but banks have played a less important role. Market share has been soaked up by nonbank players like Rocket Mortgage.

Three of the top ten mortgage providers are traditional banks: Wells Fargo, JP Morgan and Bank of America.

United Wholesale Mortgage and Freedom Mortgage are new players. Many of the firms took advantage of the boom to go public.

To keep up with customer expectations, banks have to plow money into technology platforms.

The business will be less attractive due to the capital rules forcing it to purge mortgages from its balance sheet.

The dynamic could cause some banks to offer mortgages via partners, which is what Santander now does.

Wennes said that banks would need to ask themselves if they considered this a core product.