Prime Minister Scott Morrison said the decision was in response to "human rights abuses" in China's Xinjiang province and "many other issues that Australia has consistently raised".
Athletes would still attend, he added.
China has condemned the US announcement and threatened to retaliate, without giving further details.
On Monday, the US said it would not send diplomats to the Games in Beijing over concerns about China's human rights record.
Mr Morrison said it was "no surprise" that Australia had joined the boycott, given relations with China had deteriorated in recent years.