At a meeting of the council this morning Mr Rowley was nominated and appointed unopposed.
He replaces outgoing leader Shona Haslam, who has stepped down after four years to take up a role at a think tank.
Mr Rowley, who was the council’s executive member for economic regeneration and finance, now has the task of leading the authority through testing times ahead.
He said: “My election in 2017 was probably the most humbling day of my life, being elected by neighbours and constituents, and this probably kind of trumps it.
"We are in incredibly difficult times, our staff are very stretched, every officer in the council is working incredibly hard, we are under-resourced in terms of funds from government. We are in difficult times and I want to reach out to colleagues right across the council to help us get business through.
"Councillor Haslam has big sparkly red shoes to step into. She smashed through the glass ceiling, the first first female leader to be on that board in the chamber. That is very impressive.”
Mr Rowley, who described himself as “probably the first openly gay leader of Scottish Borders Council”, also revealed he planned to carry out a partial reshuffle of the administration next month.
He was nominated by fellow Conservative councillor Simon Mountford, who said: “I know Councillor Rowley is well-respected across the chamber for his mastery of detail and for his bi-partisan approach where that is required. He faces a tough job following in the footsteps, I’m tempted to say high heels, of Councillor Haslam, who has been a truly exceptional leader. I’m confident Councillor Rowley will lead the council with panache and wisdom.”
Council deputy leader Sandy Aitchison, of the Independent Alliance group, seconded the nomination, saying: “I’m very happy on behalf of my colleagues to support the nomination.”