A penalty for Higgs' 87th-minute challenge was harsh but the 18-year-old striker was swamped by congratulations from delighted Rovers players.
He had been sent on only three minutes earlier in place of Junior Agogo who had made no impression on Cheltenham's tight defence.
Wayne Carlisle hit the spot kick wide of Higgs' left hand but the margin of defeat hardly matched the visitors' contribution to this West Country derby.
A short-range header from Paul Tait had put Rovers ahead just before half time but they were often made to struggle before beating Cheltenham for the first time.
This derby had an extra relish stirred into it because six of Cheltenham's squad - seven if you include manager Bobby Gould - had been or are on Bristol City's books.
Until Tait's strike Cheltenham had kept the home side very much on a watchful back foot because of the pace of their front runners Damien Spencer and Kayode Odejayi.
But it was a long run from their full back Kevin Amankwaah who covered all of 75 yards that had Gould on the touchline with a double thumbs-up sign.
For all their energy though Cheltenham were not producing any shots although they had a great chance to go ahead when Kevin Miller came off his line for a corner and missed the ball.
Rovers manager Ray Graydon had at last listened to the fans by playing with width and this eventually led to them going ahead.
An opening came first for Lee Hodges on the left who emerged from a cluster of defenders in possession. There was no one up in support so the little winger tried his luck, sending an angled shot over the bar.
The breakthrough came on the other flank when Dave Savage lifted the ball over Cheltenham's back line.
Defender Daniel Boxall had sped down the flank and from the byline knocked in a well judged cross for Tait to head in at the far post.