The vessel capsized in the storm Monday night with 458 people aboard, touching off a frantic rescue effort. At least 15 people were brought to safety, including three pulled from the overturned hull Tuesday, and five people were confirmed dead.
The vessel was carrying mostly elderly tourists from Nanjing to the southwestern city of Chongqing when it overturned in China's Hubei Province.
Divers rescued a 65-year-old woman and, later, two men who had been trapped, state broadcaster CCTV reported. It said more people had been found and were being rescued, but did not say whether they were still inside the overturned hull.
CCTV video showed rescuers in orange life vests climbing on the hull, with one of them lying down tapping a hammer and listening for a response, then gesturing downward.
"We will do everything we can to rescue everyone trapped in there, no matter they're still alive or not, and we will treat them as our own families," Hubei military region commander Chen Shoumin said at a news conference shown live on CCTV.
The survivors included the ship's captain and chief engineer, both of whom were taken into police custody, CCTV said. Relatives who gathered in Shanghai, where many of the tourists started their journey by bus, questioned whether the captain did enough to ensure the passengers' safety and demanded answers from local officials in unruly scenes that drew a heavy police response.
Xinhua quoted the captain and the chief engineer as saying the four-level Eastern Star sank quickly after being caught in what they described as a cyclone. The Communist Party-run People's Daily said the ship sank within two minutes.
Zhang, the tour guide, said in an interview with Xinhua from his hospital bed that he grabbed a life jacket with seconds to spare as the ship listed in the storm, sending bottles rolling off tables and suddenly turned all the way over.
The 43-year-old Zhang said he drifted in the Yangtze all night despite not being able to swim, reaching shore as dawn approached.
"The raindrops hitting my face felt like hailstones," he said. "'Just hang in there a little longer,' I told myself," he added.
Some survivors swam ashore, but others were rescued after search teams climbed on the upside-down hull and heard people yelling for help from within more than 12 hours after the ship overturned.
Thirteen navy divers were on the scene and 170 more were joining them, Chen said.
The 65-year-old woman was rescued by divers who took an extra breathing apparatus up into the bowels of the ship and spent about five minutes teaching her how to use it before bringing her out to safety, Chen said.
"That old woman had a very strong will and learned very fast, and after 20 minutes she surfaced to the water and was rescued," he added.
Chinese Premier Li Keqiang traveled to the accident site about 180 kilometers (110 miles) west of the Hubei provincial capital of Wuhan.
The overturned ship had drifted about 3 kilometers (almost 2 miles) downstream before coming to rest close to shore, where fast currents made the rescue difficult.
The fact that the capsized ship drifted downstream was a good sign for rescuers because it meant there was enough air inside to give it buoyancy, and could mean there were enough air pockets for survivors, said Chi-Mo Park, a professor of naval architecture and ocean engineering at South Korea's Ulsan University.
"It all depends how much space there is inside the vessel," Park said.
CCTV said the vessel had been carrying 406 Chinese passengers, five travel agency employees and a crew of 47. The broadcaster said most of the passengers were 50 to 80 years old.
Many of the passengers took a bus from Shanghai to Nanjing for the departure to Chongqing. Their relatives gathered in Shanghai at a travel agency that had booked many of the trips, and later went to a government office to demand more information about the accident before police broke up the gatherings.
Huang Yan, 49, an accountant in Shanghai, wept as she told a reporter that she believes that her husband, 49, and his father, who is in his 70s, were aboard. But she said she couldn't be sure because she hadn't seen a passenger manifest yet.
"Why did the captain leave the ship while the passengers were still missing?" Huang shouted. "We want the government to release the name list to see who was on the boat."
A group of about a dozen retirees from a Shanghai bus company were on the trip, said a woman who identified herself only by her surname, Chen. Among them, she said, were her older sister and brother-in-law, both 60, and their 6-year-old granddaughter.
"This group has traveled together a lot, but only on short trips. This is the first time they traveled for a long trip," Chen said.
The ship sank in the Damazhou waterway section, where the river is 15 meters (50 feet) deep. The Yangtze is the world's third-longest river and sometimes floods during the summer monsoon season.
More than 50 boats and 3,000 people were involved in the search.
The Eastern Star was 251 feet (76.5 meters) long and 36 feet (11 meters) wide, and could carry a maximum of 534 people, CCTV reported. It is owned by the Chongqing Eastern Shipping Corp., which focuses on tourism routes in the popular Three Gorges river canyon region. The company could not be reached for comment.
CCTV reported that 6 inches (150 millimeters) of rain had fallen in the region in the past 24 hours. Local media reported winds reached 80 mph (130 kph) during the accident.