Quay Walker was ejected for the second time this season.
Walker was thrown out of the game for shoving a member of the Detroit Lions medical staff who was tending to an injured runner.
Swift was struck in the head by a forearm from Packers defensive tackle Jarran Reed.
The penalty gave the Lions a first down. The Lions running back scored on a one-yard run to give Detroit a 20-16 lead.
Green Bay never recovered as Detroit moved down the field and converted a fourth-down pass to win the game. The Packers could not get into the playoffs because of the 20-16 loss to the Broncos.
Walker has been ejected twice this season. He was ejected from the game at Buffalo for shoving a player.