Long-range strikes from club captain John Finnigan and winger John Melligan earned the points, although the Robins also had to thank their excellent goalkeeper Shane Higgs for his inspired second-half performance.
Cheltenham took the game to their visitors for most of the first half and almost took the lead in the sixth minute when veteran right-back Jerry Gill rocketed a 25-yarder just wide.
Another defender, this time Michael Townsend, mis-kicked when in a good position in front of goa,l but the home side edged themselves in front with 13 minutes played.
A lengthy spell of pressure on the Orient goal ended with Damian Spencer squaring a pass from the left and Cheltenham captain Finnigan smacked a low shot just inside goalkeeper Glynn Garner's right-hand post from at least 20 yards out.
Gill was required to make a brave and timely block from Orient striker Gary Hooper midway through the half, but Cheltenham almost added a second shortly before the break when Kayode Odejayi raced onto a pass from Townsend but volleyed wide with only Garner to beat.
After the break it was a different story with Orient holding much of the advantage.
Higgs pulled off an incredible triple save from Orient strikers Ryan Jarvis and Hooper at the start of the second half.
Both players thumped in shots from close range after a Wayne Corden corner fell for them but Higgs was in the way each time.
Cheltenham doubled their lead on 61 minutes when Melligan raced on to a measured pass from Finnigan before lashing a shot past Garner from the right hand corner of the penalty area.
However, the lead lasted only two minutes before Orient substitute Gary Alexander got his head to a Corden cross to glance his team back into it.
Orient kept banging on the door but were held at bay until deep into injury time when Jarvis raced on to a through ball from Jabo Ibehre and bore down on goal only to see his blasted shot saved by hero Higgs.