It was a game Arsenal would be kicking themselves for losing after leading by two-goal margins on two occasions in the game.
They also had the chance to win the game but failed to hold on as Divock Origi snatched a late equaliser in added time to force the game to penalties.
Liverpool took the lead on six minutes following an own goal by Shkodran Mustafi.
Arsenal drew level on 19 minutes and raced to a 3-1 lead after Gabriel Martinelli scored twice in the 26th and 36th minutes.
Liverpool were awarded a penalty with two minutes before the end of the first half which was converted by James Milner to bring the score to 3-2.
In the 54th minute, Arsenal restored their two-goal lead as Aisley Maitland-Niles got on the score sheet which made it 4-2.
Former Arsenal player Alex Oxlade-Chamberlain scored to make it 4-3 before Origi levelled the score at 4-4 in the 62nd minute.
Again Arsenal went in front thanks to Joseph Willock on 70 minutes which made it 5-4 but the Gunners could not hold on as Origi scored in the 94th minute to force the game to penalties.
And in the shootout Dani Ceballos missed for Arsenal before youngster Curtis Jones converted the decisive kick, to send Liverpool into the last eight.
At Stamford Bridge, Manchester United secured their place into the next round following a 2-1 win against Chelsea.
The win means United have now recorded back-to-back away wins at Chelsea for the first time since February 1998.
