The Carabao Cup was knocked out of the top flight by the Seagulls.
Eddie Nketiah scored the first goal of the game after a run from the halfway line by Reiss Nelson.
Danny Welbeck made it 2-2 with a penalty after Karl Hein was called for a foul.
Kaoru Mitoma putBrighton ahead after a neat passing move and Tariq Lamptey made it a three point game with a low strike.
The loss in this third-round tie was only the third time this season that the team had been beaten.
Nketiah's 20th-minute strike was the only goal of the first half as Mikel Arteta made a number of changes.
Their lead only lasted a short time.
Hein, a 20-year-old goalkeeper who had a loan spell at Reading last season, was making his first senior appearance for the club when he brought down Welbeck, who then converted the penalty.
Nketiah fired a shot against the post from outside the penalty area, but Albert Sambi Lokonga missed the target.
After scoring in the 3-2 win at Wolves on Saturday, Mitoma got his second goal in five days, before Lamptey added a third.
This is the third year in a row that they have reached the last 16 of the cup, and their best performance was in 1978 when they reached the quarter-Finals.
The draw for the next round will be held on Thursday night after the match between Manchester United and Villa.