The men were moved to the island state of Pohnpei at the end of June after arriving with a boat load of others from Nepal and India on neighbouring Yap.
They were detained on Yap for 18 months and were the only ones among the 34 men determined to be refugees.
An American living on Pohnpei, Mark Dvorak, who has been visiting the men said they were now confined to Pohnpei's dock on a boat that needed constant pumping to stay afloat.
He said he was concerned for their welfare.
"You know you see what looks like sadness and loneliness in their expressions. You could see that this is their third new years that they've spent in captivity, that they're not seeing the light at the end of the tunnel."
Mr Dvorak said the men should be allowed to leave the port while they waited for a third country to accept them.
"Go to the island, go to the library, go to the campus, to the university, or just hiking in this beautiful land too. Just give them some exposure to the area outside of that."
He said the police raided the boat and confiscated electronics the men had been given.
He said they had been given food and other care by the UN.
Lawyer Marstella Jack, who is a former Attorney General for the FSM, said she met with the men and would likely start taking legal action.
"It's arbitrary detention. Number one they have not been charged with anything. FSM is not a party to the refugee convention but we are a member of the United Nations and we subscribe to the principles under the UN Declaration on Human Rights."
Ms Jack said it was inhumane to keep people in such conditions for so long.
She said the boat was rat-infested, the men were sleeping on the boat's hard boards, with just a stove and a few pots and pans to cook food provided by the International Organization for Migration.
Ms Jack said she was unsure why the refugees had been waiting so long to be found a new home.
"Nevertheless you should not subject people to those kind of conditions. They're not prisoners but they're being treated worse than prisoners. Our jails are much better than the vessel these guys are stuck on."
Ms Jack said a laptop and phone they had been provided with had been confiscated by the police.
She said she would meet with the authorities this week about the men.
RNZ International has contacted the FSM government for comment.
Photo: Supplied/ Denise Oen