The Galashiels outfit – bossed by Martin Scott – had to play virtually the whole of the second half with 10 men after the dismissal of defender Kelvin Daniel, a recent signing.
And there was further drama in the dying minutes when Liam ‘Jinky’ Campbell also got his marching orders.
Before either of the red cards, a first half shot by Gala’s Danny Galbraith rattled the home crossbar at East Peffermill.
Allan Smith’s 26th minute strike then had Rovers ahead before David Maskrey levelled for the home team.
In photos: Lauderdale Limper Marc Wilkinson makes his mark at fundraising hill run near Selkirk
In pictures: Sun brings crowds out for Peebles Sevens
Borders amateur footballers getting back on the ball this weekend
Peebles swimmer Gregor Swinney brings back bronze medal from Birmingham’s Commonwealth Games
Hawick drawn against Jed-Forest in first round of rugby's Scottish Cup next year
Zander Murray converted a 43rd minute penalty kick to put Gala 2-1 ahead at half-time.
There was a major blow for Gala early in the second half when Daniel – in for injured skipper Gareth Rodger – was sent off for a second booking.
And Edinburgh made it 2-2 on 60 minutes when Maskrey scored from the spot.
There was to be one final twist as Gala won it within five minutes with Gregor Jordan’s header.
Rovers managed to hang on despite having Campbell sent off for a late lunge.
The victory sees Gala move top of the table with 10 points from four games.
Rivals Bo’ness United and University of Stirling – who are playing each other at Newtown Park tonight (Wednesday) – both have the chance to leapfrog Gala with a win in that match.