The announcement by Defence Minister Itsunori Onodera comes amid regular missile tests by North Korea and increased tensions between the nations.
The missiles will be deployed on Japan's fleet of fighter jets and have a range of up to 1,000km (621 miles).
The move is likely to be controversial, as Japan limits its military to self-defence.
But conservative Prime Minister Shinzo Abe has pushed in recent months to loosen the country's military restrictions, imposed in the wake of World War Two.
New laws introduced by Mr Abe will soon allow a broader interpretation of what the constitution does, and does not, permit.
At the moment, Japan's longest-range missiles can reach just 300km.
Mr Onodera said the new missiles would be for defence purposes, with Japan still relying on the United States to strike any enemy bases.
"We are planning to introduce the JSM (Joint Strike Missile) that will be mounted on the F-35A (stealth fighter) as 'stand-off' missiles that can be fired beyond the range of enemy threats," he told a news conference.
The JSM has a range of about 500km. Japan also plans to arms its US-made F-15 fighters with the US-made Joint Air-to-Surface Standoff Missile (JASSM-ER), which has a range of 1,000km.
Mr Abe's move to water down the country's anti-aggression rules have faced opposition in parliament, but government officials say Japan needs an effective deterrent to North Korea, which has tested long-range ballistic missiles over Japan.
North Korea is seeking to mount a nuclear warhead on a ballistic missile, raising international alarms.
Photo copyright: GETTY IMAGES. Caption: Japan plans to mount medium-range missiles on its US-made F-15 fighters