Hurricane Matthew, a Category Four storm, swept over the west of Haiti and is now heading towards eastern Cuba.
Southern Haiti has effectively been cut off after the bridge linking it to the capital, Port-au-Prince, collapsed.
The deputy mayor of the southern coastal town of Les Cayes described the scene there as "catastrophic".
Marie Claudette Regis Delerme said the city of 70,000 people was flooded and many houses had lost roofs. She herself had to flee a meeting when a gust ripped off the building's roof.
One man too ill to leave his home was killed when waves struck in the town of Port Salut.
Haiti's Interim President Jocelerme Privert said earlier that some people at sea or who had not "respected alerts" had died, but he gave no more details.
Officials said they were trying to restore access to the southern peninsula after the collapse of the La Digue bridge but admitted it would be difficult to find an alternative route.