The Robins made the perfect start to the game when Martin Devaney latched onto a mammoth clearance from goalkeeper Shane Higgs and fired his team in front after only three minutes.
The lead was doubled on the quarter-hour when Grant McCann's corner was met with a near-post flick from Mark Yates and the ball travelled into the net off a Yeovil defender.
The visitors maintained their composure with some accurate passing and clever movement but missed a fine chance to pull one back when Gavin Williams hit the post from only five yards out.
A Yeovil goal did arrive after a spell of concerted pressure five minutes before the break when Kevin Gall rammed a shot past Higgs from the edge of the area following a lengthy build-up.
Having got themselves back into the game Yeovil would have been confident going into the half-time break but a strange set of circumstances conspired against them.
Cheltenham's record signing McCann stayed in the dressing room at half-time with an injury and he was replaced by striker Paul Brayson.
The move forced the Robins to abandon their five in midfield and play with two strikers, which proved far more effective.
Brayson, who has been rarely called upon this season, put his team back in front within 50 seconds of the re-start when he beat Chris Weale after racing onto a throughball from Jamie Victory.
The home side held out after that with Kevin Amankwaah holding the defence together. Yeovil applied plenty of pressure but the closest they came to a goal was a long-range effort from Gavin Williams that hit the crossbar.
"I'm pleased for the players, for the directors and for the loyal supporters that understand football," said Gould.
"We had the rub of the green and it's a fine dividing line between winning and losing but I thought we deserved the win in the end." Yeovil manager Gary Johnson said: "It was a bad day at the office for us but the game could quite easily have been 3-3.
"You go 2-0 down and you're chasing the game and the task is much harder then. We closed the door towards the end of the game but couldn't get back into it."