David Noble put Bristol ahead on 23 minutes but Australia's Mark Viduka headed an equaliser from close range in the second half before Yakubu Aiyegbeni put the top-flight side in the lead in extra time having earlier missed a penalty.
Jamie McCombe then levelled, before Middlesbrough won 5-4 on penalties despite Yakubu missing again in the shootout. They now have a home tie with West Bromwich Albion on Saturday.
Second division Norwich also came from 1-0 down to beat third division Blackpool 3-2 after extra time to set up a fifth-round tie at Premier League champions Chelsea.
Bolton Wanderers host Arsenal in their replay on Wednesday.
