Peebles dropped to fifth in the East of Scotland League on Saturday, after scraping a draw with 11th-placed side Ormiston.
The Borderers took the lead in the 10th minute with a Dale Richardson header.
And the Peebles men soon found themselves two up thanks to a 30-yard screamer from Brendan Edwards.
However, a nice double from their top scorer Michael Osbourne brought Ormiston back into the match.
After the break, the momentum with the home side, Mark Riley made it three after a shocking goalkeeper error.
But with two minutes to go, Paul Murray rescued a point for the visitors, netting the rebound after Jason Darling battered a 25-yarder off the bar.