The Galashiels outfit – bossed by Martin Scott – had to play virtually the whole of the second half with 10 men after the dismissal of defender Kelvin Daniel, a recent signing.
And there was further drama in the dying minutes when Liam ‘Jinky’ Campbell also got his marching orders.
Before either of the red cards, a first half shot by Gala’s Danny Galbraith rattled the home crossbar at East Peffermill.
Allan Smith’s 26th minute strike then had Rovers ahead before David Maskrey levelled for the home team.
Injury-hit Southern Knights muster reinforcements ahead of trip to Fosroc Super6 Championship basement side Stirling Wolves
Galashiels coach Dougie Anderson has gone from Borders amateur football ranks to mingling with Mourinho
In pictures: Capital gain at Peebles as Jed-Forest’s Kings of 7s title defence falters
Borders amateur footballers getting back on the ball this weekend
Chirnside team bowled over by success at national championships
Zander Murray converted a 43rd minute penalty kick to put Gala 2-1 ahead at half-time.
There was a major blow for Gala early in the second half when Daniel – in for injured skipper Gareth Rodger – was sent off for a second booking.
And Edinburgh made it 2-2 on 60 minutes when Maskrey scored from the spot.
There was to be one final twist as Gala won it within five minutes with Gregor Jordan’s header.
Rovers managed to hang on despite having Campbell sent off for a late lunge.
The victory sees Gala move top of the table with 10 points from four games.
Rivals Bo’ness United and University of Stirling – who are playing each other at Newtown Park tonight (Wednesday) – both have the chance to leapfrog Gala with a win in that match.