A NSW high school has been rocked by allegations a student used artificial intelligence to create sexualised images and videos depicting fellow students.

About 10 female students were allegedly depicted in the material, which was shared on social media, a source familiar with the incident told 7NEWS.com.au.

The source alleged a Year 9 student created the material, and affected students reported the matter to the principal around the beginning of September.

Know the news with the 7NEWS app: Download today

The school subsequently addressed the year group about the incident, telling students the matter was being handled and had been reported to police, according to the source, who asked to remain anonymous.

The source said they had spoken with some of the girls involved and described them as feeling annoyed and “creeped out”.

The source said some students believed further action should be taken and that the student allegedly responsible should not be allowed to return to the school.

7NEWS.com.au understands the student was expected to return to school on Thursday after being suspended for several days and was expected to apologise to some of the affected students.

The school is in contact with the families of students involved.

About 10 female students were allegedly affected. Credit: Cravetiger/Getty

The NSW Department of Education confirmed to 7NEWS.com.au that the school was aware of the incident and had taken disciplinary action.

“The school took immediate disciplinary action as soon as it became aware of this incident and reported it to NSW Police and the eSafety Commissioner,” a spokesperson said.

“Counselling and wellbeing services are available at the school for any student who has been affected by this incident.”

It is understood the alleged incident occurred outside school hours.

Australia’s online safety regulator has also confirmed it is investigating.

“eSafety is unable to comment about the details of individual complaints but can confirm it is investigating this incident and has been in contact with the school,” eSafety Commissioner Julie Inman Grant told 7NEWS.com.au.

7NEWS.com.au has contacted NSW Police for comment. 

‘Devastating’ rise of AI-generated abuse

Inman Grant said the issue extended well beyond the reported incident.

She said eSafety had been aware since early last year of reports of intimate image-based abuse and child sexual abuse material involving students under 18 circulating in schools as a result of “nudifying” services.

“eSafety is engaging with police and has strong working relationships with education sector leaders through the National Online Safety Education Council (NOSEC), to support the management of online safety issues,” she said. 

Inman Grant warned that young people using “nudify” tools may not appreciate the seriousness of what they are creating or sharing.

“Deepfakes, especially deepfake image-based abuse, can be devastating to the person whose image is taken and altered without their knowledge or consent, no matter who they are,” she said.

“And new laws introduced by the Australian Government impose serious criminal penalties on those who share sexually explicit material without consent. This includes material that is digitally created using artificial intelligence or other technology.”

eSafety Commissioner Julie Inman Grant said reports of intimate image-based abuse and child sexual abuse material involving students under 18 have been circulating in schools since last year. Credit: Creative Images Lab/Getty Images

She said eSafety had a “very high success rate” in having such material removed through its image-based abuse scheme, but argued greater responsibility needed to fall on AI services to prevent their technology being misused in the first place.

Enforceable industry codes and standards require online services, including AI tools, to take steps to prevent the creation and spread of harmful material, including synthetic child sexual exploitation and abuse material.

“This includes the use of so-called ‘nudify’ services to target children under 18, and the online marketplaces that offer these generative AI models,” she said.

Services that fail to comply with the codes and standards can face civil penalties of up to $54.6 million.

eSafety has published guidance for schools on responding to incidents involving AI-generated deepfakes, including advice on reporting incidents to police and the regulator.

The watchdog has also issued an online safety advisory warning of the increased risks associated with AI manipulation of school photos posted online.

Anyone affected by image-based abuse can report it to eSafety and access support through eSafety.gov.au.