There were a number of stumbling blocks that led to the collapse of the move to the other side.
A fee of more than 25m with add-ons was agreed on Saturday by the two parties. He was going to have a medical in the city.
There were problems when the two sides began to talk about the details of the deal.
Sky Sports News has been told that Villa's bosses walked away from the deal due to growing reservations about it.
He was left out of the squad for Saturday's game because of an injury, according to the manager.
He has 25 goals in 95 appearances since joining the Hornets from Rennes, with five of them coming in the premier league.
The Hornets were second in the Sky Bet Championship table but did not have an adequate number of attackers after Dennis joined Forest. The Nigerian scored 10 goals for the Hornets in the top-flight but they were not able to keep up.
If you want a more accessible video player, please use the chrome browser.
The transfer window will close at 11pm on September 1st.
Sky Sports has a dedicated Transfer Centre website where you can keep up-to-date with all the latest news. Sky Sports News gives you the ins, outs and analysis of sports.