The initiative swung this way and that but with around 35 minutes to go it seemed that Colchester would be going back to Essex with three points courtesy of a 3-1 lead.
However, Cheltenham's new look team have only been together for a short time, but they have already got enough spirit and determination about them to storm back with three goals to win the match.
The first half was packed with goalmouth incident with Alex Russell and Ashley Vincent going close in the opening minutes for Cheltenham and Colchester's Mark Yeates seeing one saved by goalkeeper Scott Brown and placing another just wide of the target.
Cheltenham almost took the lead when Stuart Fleetwood crossed for Vincent wide open in space, but he was not able to make contact.
Instead it was the visitors who grabbed the advantage in the 20th minute when defender Michael Townsend mis-kicked in the penalty area, Brown had to react quickly to parry the ball on the line, but striker Anthony Wordsworth was on hand to tap in the rebound.
Cheltenham equalised 16 minutes later when defender Darren Kenton headed in powerfully from a Russell cross, but the home defence was breached again two minutes before the break.
This time Yeates made a driving run into the box and crossed for the arriving Wordsworth to provide the finishing touch.
Chances came and went at both ends at the start of the second half, but Colchester put themselves in control with 51 minutes played when midfielder Johnny Jackson picked up a loose clearance from a corner, drove into the box and beat Brown with a low shot.
Cheltenham 's response was immediate, Lloyd Owusu poking home from close range after goalkeeper Dean Gerken's punch had only gone above his head, Gerken claimed he had been fouled.
Then two minutes later referee Andy Penn awarded a penalty for handball when a shot from James Wesolowski struck defender Gary Borrowdale.
Stuart Fleetwood planted the spot-kick between the posts and the game continued to ebb and flow until deep into injury-time, when Alan Wright's free-kick was headed on by Owusu and substitute Barry Hayles sent a deflected header past Gerken from eight yards.