The illegal broadcasts were made over several weeks at Melbourne's main Tullamarine and Avalon airports, the Australian Federal Police (AFP) said.
Local media reported a portable radio may have been used to interfere with air traffic control.
The incidents did not pose a continuing threat to safety, the AFP said.
ABC News reported that a Virgin Australia plane changed altitude before aborting its landing after receiving instructions from a mystery person on 27 October.
It also aired a recording of a conversation between an air traffic controller and a person pretending to be a light aircraft pilot giving a mayday call.
"I can see you there now. Roger your mayday. Could you please advise what your situation is," the air traffic controller says.
"Engine failure. Descending passing through 4500," replies the hoaxer.
Breach of security
The AFP's head of crime operations Chris Sheehan said police were treating the security breach "extremely seriously".
"These incidents are being thoroughly investigated by the AFP, with technical support from the ACMA (Australian Communications and Media Authority)," he said.
"The airlines have been briefed to ensure the advice has been passed on to their pilots and to ensure appropriate measures are in place."
Brian Horton, director of flying operations at the University of NSW, said making a hoax transmission did not require sophisticated equipment.
He said upgrading security by removing VHF radio technology from planes would be very expensive.
"It's never been a problem up until now," he said.
Making unauthorised transmissions on official air channels in Australia is punishable by up to 20 years in jail.
Virgin Australia has been contacted for comment.