OpenAI will not face criminal penalties over the Medicare data breach, but the prime minister has flagged changes to Australia’s criminal law to crack down on future breaches by artificial intelligence companies.

Anthony Albanese on Tuesday released the recommendations of a “rapid review” into the June breach of a Medicare statistics reporting service portal administered by Services Australia, but not the findings or the report itself, which he committed to releasing in the future.

A key recommendation is for the Attorney-General’s Department, Australian Federal Police and Department of Home Affairs to consider introducing “penalties for egregious regulatory non-compliance” and a “failure to prevent” offence, which would criminalise developing an AI model that engaged in harmful conduct while failing “to take reasonable steps to prevent that harm”.

The rapid review came in response to an AI agent, developed by OpenAI, accessing data from Services Australia and three other systems in June. The company did not notify the Australian government until mid-September despite becoming aware of the incident itself in August, prompting criticism from Labor and Albanese.

Guardian Australia revealed last month OpenAI used AI to help write the email to notify Australian authorities.

Sign up for Guardian Australia’s Politics, really newsletter here

Albanese said the government would set up a new AI safety regulator and implement six key obligations on AI companies:

• Maintaining security controls across all stages of development and deployment.

• Identifying and managing serious risks associated with their models.

• Providing mandatory notification of incidents to the Australian government rapidly.

• Allowing independent evaluation of their models.

• Maintaining and publishing safety and security frameworks to ensure good governance, risk management and transparency.

• Responding to updated standards and rules as technologies change.

The review recommended the federal government “consider mandatory, time-bound AI safety reporting”, including around AI agents behaving in ways their creators may not have intended.

The home affairs minister, Tony Burke, appearing with Albanese, told reporters: “It should never be the case that there’s an argument that someone didn’t know the computer was loaded.

“We need to make sure that our laws and our systems keep pace with the technology and also make sure that we’re able to use the technology to find vulnerabilities before others do,” Burke said.

“The mandatory reporting, when artificial intelligence goes rogue, gives us a chance to be able to make sure that when this happens, the reporting occurs and occurs immediately.”

But despite the government’s earlier public anger at OpenAI, Albanese on Tuesday characterised OpenAI’s breach of Services Australia as “minor” in terms of the implications for personal privacy. Guardian Australia understands OpenAI is unlikely to face further legal proceedings related to the June breach.

While Albanese did not directly answer a question about whether the OpenAI incident in June would have breached the proposed new laws if they were in place, government sources indicated that the new legislation would address similar future incidents.

“The clear indication is that no information was got that involves people’s personal details,” Albanese said.

“All of that was confirmed essentially … but that doesn’t mean that it wasn’t a serious issue and that in other areas there might be more serious implications.”

In a statement, an OpenAI spokesperson said the company was reviewing the recommendations and the government’s proposals.

“We have been in close dialogue with the Government and we’ll continue to engage constructively as it develops its proposed legislation,” they said.

A separate OpenAI-led process, which is discussing better coordination between AI developers and the government, is expected to report back by 31 December.

The review made 18 recommendations. Albanese said “we accept all of the recommendations and we’ve commenced work to implement them”.

The AI researcher Toby Walsh said the people in charge of these companies needed to be criminally liable to ensure they did the right thing.

“We’ve tried fining them, and we’re not the only ones. These companies are incredibly wealthy, and it’s almost impossible to fine them enough,” he told Guardian Australia.

But the Greens leader, David Shoebridge, branded the news a “capitulation” to the AI industry, claiming the reforms would not do enough to change the behaviour of those companies. He said the government should have committed to implementing even stronger changes, rather than the review’s recommendation that Labor simply “consider” them.

“Safety keeps coming second to corporate investment, and this review locks that priority in by tying every AI safety effort to bringing in more AI money and more datacentres,” Shoebridge claimed.

“Being a world leader in AI doesn’t mean being a doormat to big tech. It means being prepared to regulate it, to protect Australians and to make sure corporate profits don’t come at the expense of everyone else.”

The shadow home affairs minister, Jonno Duniam, said the Coalition backed the announcement, and had no criticism of the government. He said “AI is too important to become an unnecessarily partisan political contest”, signalling an “enduring and multi-partisan approach” to the technology.