Only the second time in their history, Senegal advanced to the World Cup's knockout stages with a nerveless volley by Kalidou Koulibaly in the second half.
All three of the goals were scored by players based in England, with the only exception of the penalty by the Hornets.
It was a point that was needed to progress, but it was also a point that was paid the price for a passive display.
The African champion scored from the spot after being upended by a defender, after being clumsily upended by another player.
Caicedo scored the first goal of the tournament for Ecuador, but it wasn't long before the lead was back for the Africans, who knew only victory would suffice.
They were runners-up in Group A behind the Dutch, who won the tournament with a 2-0 victory over the hosts.
The winner of Group B will face Aliou Cisse's side in the last 16 of the World Cup.
Much has been made of the absence of Sadio Mane from this tournament due to injury, but they have found goals from other places as well.
As the South Americans began to benervily, they found a lot of space in and around the Ecuador area, but both Boulaye Dia and Idrissa Gueye missed great opportunities.
When Caicedo leveled from a corner, the heads of the players did not fall even when they were in front.
Victory secured a first last-16 appearance since 2002 when they shocked France in the opening game on their way to the quarter-finals and before the game, they dedicated their efforts to the memory of Papa Bouba Diop, who died.
Gueye, who would have been his 100th cap, will miss the second round after picking up a second yellow card, meaning he will not play.
Caicedo had to be physically hauled from the turf by the team after he lay prostrate on the pitch at the final whistle.
Under the pressure of a virtual winner-take-all environment, they failed to reproduce the level of performance they had shown in their first two games.
Even though Enner Valencia was taken off on a stretcher late in the game against the Netherlands, he failed to make a difference.
Four points they had accrued prior to the game meant they could play for a draw, but they did not look like the side that had kept seven successive clean sheets earlier in the year.
Get the latest results and goal notifications for any team at the Fifa World Cup by downloading the BBC Sport app: Apple - Android - Amazon
You can get a daily dose of the World Cup on the radio.