Alberta Federal Court Justice Robin Camp's decision came after a scathing Canadian Judicial Council review called for his removal.
The review concluded he acted in a manner that seriously undermined public confidence in the judiciary.
The federal justice minister has accepted his resignation.
Mr Camp's comments made during a 2014 rape trial sparked outrage and drew sharp criticism from sexual assault victims and their advocates.
The judge had tried to hold on to his position and told the Council he had undergone education and had apologised for his comments.
The Council, which oversees the judiciary, found that Mr Camp's actions during the trial were "so manifestly and profoundly destructive of the concept of impartiality, integrity and independence of the judicial role that public confidence is sufficiently undermined to render the Judge incapable of executing the judicial office".
It also found he spoke to the complainant in a manner "that was at times condescending, humiliating and disrespectful".
Federal Justice Minister Jody Wilson-Raybould said on Thursday she is confident Mr Camp received due process in his disciplinary review and accepts his decision to step down as of 10 March.
"Sexual assault and gender-based violence is in no form acceptable and we will continue to stand up for victims," she said.
In the 2014 case, he asked a 19-year-old woman: "Why couldn't you just keep your knees together?"
He also said, "pain and sex sometimes go together" and had referred to the complainant as "the accused," court records show.
The woman alleged she was raped over a bathroom sink at a house party in Calgary.
A new trial was ordered for the accused, who was acquitted a second time in January.