Since the last Fed meeting, the stock market has rallied. Markets misinterpreted the Fed's approach to fighting inflation, and the central bank will likely push its benchmark rate past 4%, according to a former New York Fed President.

The S&P 500 has rallied 15% from its mid- June low of 3,666, a sign that investors are starting to turn bullish after the Fed hiked rates last month. July's Consumer Price Index shows inflation cooling off from June's high.

Dudley thinks that the Fed still has a lot of work to do.

Dudley said in an interview with CNBC that he thinks the Fed is going to be higher for a longer time.

He said investors misinterpreted Powell's comments after the July meeting of the Federal Open Market Committee, where he suggested the policy rate had hit a neutral level. Powell said in the July 27 press conference that it would likely be appropriate for the FOMC to slow the pace of its rate hikes.

Markets have interpreted that to mean that the Fed will change its policies. The current unemployment rate is too low for the Fed to loosen policy.

The Fed has more room to tighten before it gets close to its 2% inflation target because of the elevated price growth in other areas of the economy.

The market doesn't believe Powell when he says he wants to bring inflation down to 2%. If inflation is 3% in the middle of next year, the Fed will relent, according to Dudley. It will take time for the market to understand what Powell is saying.

Powell's message has been reinforced by other Fed officials in the wake of the bull rally. Mary Daly, president of the San Francisco Fed, said in an August interview that the forecasts of the Fed cutting rates are a puzzle to her.

Evans said it will take more time and data to know if we have a lot more ahead of us.

Dudley said that the Fed would likely need to raise interest rates past 4%, and that he doesn't think rate cuts are going to happen soon. He said that the Federal Reserve needs to see a strong downwards trend toward the 2% target before it makes any sort of dovish pivot.

Dudley warned against premature optimism because Chair Powell doesn't want to do that.