Rose, who hit a hole in one on his first day, carded a final round four-under-par 67 to beat Sweden's Henrik Stenson by two shots in a competition that went all the way to the final putts. American Matt Kuchar claimed bronze.
"I've been thinking about Rio for a long, long time," Rose said. "To come out of it with a medal is incredible, and to come out of it with gold, unbelievable."
New Zealand's golfers finished midfield in their competition. Danny Lee fired a final round two under par 69 to finish 2 under for the tournament - a tie for 27th. Ryan Fox was three shots back, finishing with a 3 under par 68 in a tie for 39th.
Rose had been neck-and-neck with Stenson throughout much of the final round but hit a lovely approach shot to within feet of the pin at the last hole.
Stenson put his long putt past the hole and missed his next shot to card a bogey six. Rose, whose only major victory was the US Open in 2013, birdied and then punched the air and grabbed the British logo on his shirt to the acclaim of the crowd.
Matt Kuchar, who was only included after world number three Jordan Spieth pulled out, finished one stroke behind Stenson to win bronze.
Spieth and the other three men in the world's top four - Jason Day of Australia, American Dustin Johnson and Ireland's Rory McIlroy - withdrew from the Olympics, some of them citing concerns over the Zika virus.