Some 20,000 troops are being deployed in the operation in Kyushu region after the magnitude-7.3 quake at 01:25 on Saturday (15:25 GMT Friday).
At least 15 people have been killed and hundreds injured, media reports say.
Dozens of people are feared trapped in the rubble of collapsed buildings. The quake on Thursday killed nine people.
Roads have been damaged and big landslides have been reported over a wide area. Some 200,000 households are now without power.
The extra troops are being sent to Kyushu to help police and firefighters. "We are making every effort to respond," Chief Cabinet Secretary Yoshihide Suga said.
The second quake - which was at a depth of 10km (six miles) near Kumamot - was much bigger and hit a wider area than the one that struck the city of Kumamoto on Thursday night.
Thousands of people spent the night on the streets and in parks - where they were huddled under blankets looking dazed and afraid.
There are many reports of people trapped inside buildings, including at least 60 inside an old people's home.
One village has been evacuated after a dam collapsed as a result of the quake, public broadcaster NHK says.
Japan's nuclear authority said the Sendai nuclear plant was not damaged.
A small eruption occurred at Mt Aso following the tremor, media reports say.
Gavin Hayes, a research geophysicist with the US Geological Survey (USGS) in Colorado, told the BBC the latest earthquake would hamper the earlier rescue operation that was already under way.
He said more damage could be expected as the earthquake had been shallower and the fault-line had been much longer.
"The ground surface would have moved in the region of 4-5m (yards). So, you are talking very intense shaking over quite a large area. And that's why we'll probably see a significant impact from this event."
Thursday's magnitude-6.2 quake caused shaking at some places as intense as the huge earthquake that hit the country in 2011, Japan's seismology office said.
That quake sparked a huge tsunami and nuclear meltdown at The Fukushima power plant.
Most of those who died in Thursday's quake were in the town of Mashiki where an apartment building collapsed and many houses were damaged. More than 1,000 people were injured.
Some 40,000 people initially fled their homes, with many of those closest to the epicentre spending the night outside, as more than 130 aftershocks had hit the area.
Japan is one of the most seismically active areas on Earth, accounting for about 20% of global quakes of magnitude 6.0 or greater. Seismometers are recording some kind of event every five minutes, on average.
It is through bitter experience that Japan has learnt the strategies to mitigate damage, injury and death. Not only does it implement some the best building construction practices but it has also established an early warning network.
This system relies on the lightning analysis of the developing quake, establishing its location and strength. Alerts are then broadcast that can give people more distant from the epicentre vital seconds' notice.
Just 10 seconds is more than sufficient to drop and get under a sturdy table or open the doors of a fire station.
The prospect of buildings already damaged in Thursday's quake toppling over in this latest tremor will be a concern.