On Friday, Mr Dutton was making small talk with Australia's Prime Minister Tony Abbott, who had just returned from the Pacific Islands Forum in Papua New Guinea.
Noting that a meeting was running late, Mr Dutton remarked it was running to Cape York time, to which Mr Abbott replied: We had a bit of that up in Port Moresby.
Mr Dutton then quipped: Time doesn't mean anything when you're about to have water lapping at your door, a comment caught on a television microphone.
President Tong says as long as there is this kind of attitude, this kind of arrogance in any position of leadership, we will continue to have a lot of tension.
The Kiribati leader was referring to varying viewpoints on climate change at the Pacific Islands Forum in Port Moresby.
Mr Dutton later refused to comment about his quip, describing the exchange as a private conversation.
Mr Tong told the ABC the joke shows a sense of moral irresponsibility quite unbecoming of leadership in any capacity.
He also warned Mr Dutton that a future Australian immigration minister will have to deal with a wave of Pacific refugees from low-lying countries like Kiribati, if sea levels continue to rise.
The Foreign Affairs Minister of the Marshall Islands, Tony de Brum, also expressed his dismay over Mr Dutton's remarks.
He wrote on twitter that he was dismayed Australians Ministers joking about sea level rise in Pacific.
Gary Juffa, the governor of Oro province in PNG, also tweeted, saying Mr Abbott must apologise for insensitivity towards all for laughing at climate change.