AI content: wake | Photo credit: bl-online administrator
The Ministry of Electronics and Information Technology’s response to the controversy is welcome, but the pervasiveness of the issue requires more research. Moreover, reactive measures rather than proactive willingness to prevent the creation of such material with AI will not solve the problem either.
Unfortunately, the proliferation of explicit material created by generation AI poses a threat to women and minors, as any image posted online may be distorted. The question is therefore not whether enforcement exists on paper, but whether women and children can participate publicly without fear.
Two factors
Two drivers explain why action in the wake of circulation will not completely solve the problem. First, weak guardrails pose a design problem because safety controls are often too narrow, applied inconsistently, or easily circumvented. As a result, filters that catch obvious cues may fail to detect coded wording, regional slang, mixed-language requests, or workflows where a user starts with a normal image and gradually pushes the system toward a sexualized result through successive operations. Consequently, jailbreak strategies that exploit these loopholes, using euphemisms, step-by-step directions, and indirect instructions to bypass restrictions and elicit outputs, are designed to deny the system, rendering the “policy” meaningless.
Second, dataset management remains a relevant concern, as many open-source AI training datasets contain sexually explicit material. AI models can internalize these patterns and increase the likelihood of sexualized outcomes.
This creates a paradox: when an AI tool is trained with content and rules are established to prevent the production of explicit images, the training itself predisposes the tool to such content. This makes the superficial filters brittle, as they can block obvious keywords but may fail when users target their weaknesses.
Reactive response
India’s regulatory response is more focused on responding to harm. The draft amendments on synthetically generated information move in the direction of disclosure, requiring platforms to request a declaration from the uploader as to whether the content was generated by AI. If declared synthetic, the platform must ensure it is prominently labeled or contains a persistent identifier or metadata.
MEITY’s advisories have emphasized the importance of interim due diligence and takedown compliance under IT regulations, which is an important step, although it does not serve the purpose holistically. The AI-enabled sexual synthetic content can spread through platforms like WhatsApp and Telegram without any control as there is no system to monitor the synthetically generated content on such platforms.
This policy and legislative stance also determines how the law is currently used to address this; it only responds after it has been put into circulation. The plausible remedy available under the IT Act, Section 67, which focuses on publishing or transmitting obscene material in electronic form, may apply at the time of circulating an altered sexual image.
They don’t solve the replication problem, because even a quick takedown on one platform doesn’t stop screenshots being taken and forwarding through closed groups, where the image continues to travel, and such distribution is rarely reported.
Safety by design
The default mechanism should be Safety by Design throughout the lifecycle of these systems, including controlled training data practices, structured red teaming that tests whether models can be exploited to cause harm, and provenance measurements that make results traceable rather than easily launderable.
Regulation must also shift from post-hoc punishment to capacity control, meaning independent audits before deployment of wide-reach systems, mandatory detection of input and output abuse with accessible user reporting, and a high-risk classification for nudging functions should be eliminated, demonetized, and blocked rather than simply managed after their creation.
Gupta is a final year law student at National Law University, Jodhpur; Yadav is a final year law student at RMLNLU
Published on January 8, 2026
#Grok #conundrum #solved