He said he did not want his illness to get in the way of decision making, and apologised to the Japanese people for failing to complete his term in office.
The 65-year-old has suffered for many years from ulcerative colitis, an inflammatory bowel disease, but he said his condition had worsened recently.
Last year, he became Japan's longest serving prime minister. His current period in office began in 2012.
He will remain in his post until a successor is chosen.
