"We are reviewing exactly what has happened in these circumstances because they cannot be repeated," Prime Minister Jacinda Ardern said.
Yesterday compassionate leave for those in quarantine had been suspended.
In a Facebook message last night, Ardern said that would not change unless testing was "rigorous".
There was an expectation that anyone who came into from overseas was in quarantine for two weeks and would not leave without being tested, she said. "That was where there was a failure in this case."
Rigorous testing was underway of everyone they had been in contact with, even those remotely, she said.
"These individuals have not had contact with wider members of the public."
She said the government was taking the issue seriously and referred to the instance it was taken to court by a man in mandatory quarantine wanting to visit his dying father.
However, while the prime minister says rigorous contact testing is underway, Health Minister David Clark told Morning Report he hasn't yet received information about the number of people who need to be contact traced.
"...obviously these two women did everything that they were asked to do so there's a very small job there."
And he said he doesn't know how many others have left managed isolation without being tested.
"It was my expectation that the testing would be done before people left the facility not after."
He said he's not yet had an answer why this didn't happen.
"We want to be assured that the right protocols are being followed in every case."
Medical staff are on site and these things need to happen, Clark said.
One of the women had a pre-existing condition which meant the symptoms she experienced were not unusual for her, he said.
"It meant that the right questions weren't being asked, the right information wasn't solicited, the appropriate follow-up didn't happen."
Clark said he's concerned by reports people recently arrived in the country have being mingling with others.
"These are supposed to be self-isolation facilities, mixing and mingling is not supposed to be part of the regiment there, there should be social distancing, appropriate precautions being taken and that's something else I want assurances about."
Before compassion leave can be reinstated, the minister said he wants to be sure someone is tested routinely before leaving a self-isolation facility, that symptom checks are done and that people aren't mixing a mingling.
The women are now in self-isolation, he said.