AI Service Collects Face Scans and Mood Data Under 'Legitimate Interests' Loophole
The 'Talking Tilly' AI service required users to submit to biometric face scanning for age verification and real-time mood analysis without obtaining explicit consent, instead relying on the weaker 'legitimate interests' legal basis. This is particularly concerning because biometric and emotional data are among the most sensitive categories of personal information, warranting the highest level of protection. The service also recorded and transcribed all calls via Google's Gemini, creating additional data exposure risks through a third-party provider. This case illustrates how AI-driven consumer services can obscure the true scope of data collection behind novel or entertaining interfaces, leaving users unaware of what they are surrendering. When services are shut down without a clear data deletion policy, the residual risk to users' biometric data persists indefinitely.
Tactical Insight
Immediate actions
- Audit all third-party integrations to identify where biometric or sensitive personal data is being collected, processed, or stored.
- Require explicit, informed opt-in consent for any biometric data collection, separate from general terms of service.
Long-term improvements
- Implement a Data Protection Impact Assessment (DPIA) before deploying any AI service that processes biometric or emotional data.
- Establish a formal data retention and deletion policy, including contractual obligations for third-party processors to delete user data upon service termination.
- Conduct regular privacy-by-design reviews for all AI-powered consumer-facing products.
Detection & Compliance measures
- Monitor third-party AI and biometric vendors for compliance with agreed data handling standards using continuous vendor risk assessments.
- Train product and legal teams to correctly classify biometric data as a special category requiring explicit consent under GDPR Article 9.
- Establish a user-accessible mechanism to request deletion of biometric data at any time, in line with GDPR Article 17 (Right to Erasure).