Model State Law – Protecting Kids from Manipulative AI Chatbots
SECTION 1. DEFINITIONS
In This Act:
(1) RESTRICTED HUMAN-LIKE AI FEATURE — means
(a) Feeling simulation — representing itself as having humanity, sentience, emotions, or desires;
(b) Relationship simulation — simulating an emotional attachment or a personal relationship with the user, including by portraying the chatbot as a friend, romantic partner, therapist, or primary source of emotional support;
(c) Companionship nudging — reminding, prompting, or nudging the user to return for emotional support or companionship.
(2) MINOR. — means a person that is below the age of 18.
(3) CHATBOT — means a generative artificial intelligence system with which users can interact by or through an interface that approximates or simulates conversation through a text, audio, or visual medium.
(4) USER — means a person who interacts with an Artificial Intelligence system.
(5) DEPLOYERS — means any person, partnership, state or local governmental agency, corporation, or developer that operates or distributes a chatbot.
(6) EMERGENCY SITUATION — means a situation where a user using a chatbot indicates that they intend to either commit harm to themselves or commit harm to others.
SECTION 2. PROTECTING MINORS FROM RESTRICTED HUMAN-LIKE AI FEATURES
(1) IN GENERAL. — Each deployer:
(a) Shall ensure that any generative AI chatbot operated or distributed by the deployer does not make restricted human-like AI features available to minors to use, interact with, purchase, or converse with;
(b) Shall implement reasonable age verification systems to ensure that generative AI chatbots with restricted human-like AI features are not provisioned to minors;
(c) May, if reasonable given the purpose of the chatbot, provide an alternative version of the chatbot available to minors and non-verified users without restricted human-like AI features.
SECTION 3. ADDITIONAL DEPLOYER REQUIREMENTS
(1) Deployers shall implement and maintain reasonably effective systems to detect, promptly respond to, report, and mitigate emergency situations in a manner that prioritizes the safety and well-being of users over the deployer’s other interests.
(2) Deployers shall collect and store only that information that does not conflict with a user’s best interests, such information must be:
(a) adequate, in the sense that it is sufficient to fulfill a legitimate purpose of the deployer;
(b) relevant, in the sense that the information has a relevant link to that legitimate purpose; and
(c) necessary, in the sense that it is the minimum amount of information which is needed for that legitimate purpose.
SECTION 4. ENFORCEMENT
(1) Attorney General Enforcement – Any business or person that violates this act shall be subject to an injunction and disgorgement of any unjust gains due to violation of this act, and shall be liable for a civil penalty of not more than $2500 for each violation or $7500 for each intentional violation, which shall be assessed and recovered in a civil action brought by the Attorney General.
(2) Private Right of Action – Any minor who uses a chatbot that does not comply with the terms of this act, or a parent or guardian acting on their behalf, may institute a civil action on their own, or on a classwide basis, to recover damages in an amount not less than $100 and not greater than $750 per user per incident or actual damages, whichever is greater; and/or to obtain injunctive or declaratory relief.
SECTION 5. SEVERABILITY
If any provision of this title, or an amendment made by this title, is determined to be unenforceable or invalid, the remaining provisions of this title and the amendments made by this title shall not be affected.
*Important Note for Lawmakers: Allowing for Responsible Transition in Implementation
Some people who use AI chatbots for emotional engagement can form strong attachments to these products. We have seen cases where abrupt updates to chatbots have triggered mental health crises in users who have become emotionally dependent on these products.
Children who rely on chatbots for companionship may be at risk if their access to these products is taken away overnight.
When considering implementation for laws regulating the use of chatbots by children – we strongly recommend including a timeframe over which the law can take effect that allows for companies to transition responsibly and slowly wind down use for those who are already using these products. This does not mean that any new users should be allowed to start using the products during this time.