AI Frontier Institute

AI Chatbots - Licensing/Safety/Privacy

View primary source →

North Carolina Senate Bill 624 (2025) has not been enacted. It was filed March 25, 2025, passed first reading, and was referred to the Senate Committee on Rules and Operations. If enacted, it would do two things, both effective January 1, 2026 (Sections 1.(b) and 2.(b)). Section 3 makes the rest of the act effective when it becomes law. Part I (new G.S. Chapter 114B, the Chatbot Licensing Act) would bar anyone from operating or distributing a chatbot that deals substantially with health information unless they hold a license from the Department of Justice (G.S. 114B-3). Applicants would have to submit technical, data-handling, security, privacy, testing, risk, compliance and insurance documentation. The Department would review applications for technical competence, data protection, regulatory compliance, risk management, evidence of efficacy, expert endorsement, and public safety. Licensees (G.S. 114B-4) would have to: - carry professional liability insurance; - use encryption and run security audits every six months; - report data breaches within 24 hours to the Department and within 48 hours to affected consumers; - get explicit user consent and give users access to their data and the ability to delete it; - make specified disclosures, including that the chatbot is artificial; - show effectiveness through peer-reviewed controlled trials and comparison with human experts; - undergo an annual third-party audit; - monitor safety continuously and file quarterly reports. The Department could inspect physical sites and digital systems, including source code and models (G.S. 114B-5). Trade secrets would be kept confidential. Violating G.S. 114B-5 would carry a $50,000 civil penalty (G.S. 114B-6). Part II (new G.S. Chapter 170, the Chatbot Safety and Privacy Act) would apply to "covered platforms." These are chatbot providers with more than $100,000 in annual revenue or more than 5,000 monthly active U.S. users. Educational or research services that are not monetized, and government services, are excluded. Covered platforms would owe users a duty of loyalty (G.S. 170-3), with specific duties covering: - emergency situations; - emotional dependence; - chatbot identity disclosure; - influence; - data collection; - personalization; - gatekeeping of personal information. They would also have to: - use a clear terms of service agreement with affirmative consent (G.S. 170-4); - run a chatbot identification and consent process at the start of each session, separate from the privacy policy (G.S. 170-5); - de-identify user data before storage and analysis; - take reasonable care to keep sensitive personal information out of AI training datasets; - retain conversations without sensitive information for at least 60 days; - use self-destructing messages with a 30-day destruction period in the sectors listed in G.S. 170-6(c); - use transport encryption (G.S. 170-6). Enforcement (G.S. 170-7) would be by the Attorney General, who could sue on behalf of the State. Injured individuals could also sue for the greater of actual damages or $1,000 per violation, plus attorneys' fees. They would have two years from discovery to sue, and these rights could not be waived.

← Back to the tracker