Sign-language AI leaves the lab and lets Deaf users sign instead of type
Google DeepMind is bringing sign-language-to-text AI into Gboard and Live Transcribe on Pixel 11, beginning with ASL to English. Users can sign for searches, messages, documents, and Gemini interactions or translate a nearby signer at no added cost. The underlying SL2T model was trained on more than 100,000 hours across over 50 sign languages, about one quarter of it ASL, but the launch itself supports only ASL-to-English, with more languages and devices planned. On-device MediaPipe Holistic converts video into geometric pose landmarks; only those coordinates are sent to the server and raw video is discarded immediately. The system bypasses gloss transcription and is designed for streaming latency, left-handed signing, one-handed phone use, and suppression of text when nobody is signing. DeepMind also discloses current limitations including rare signs, fast fingerspelling, passive constructions, classifier details, and tense. The product was developed with Deaf employees, data partners, experts, user studies, and an advisory committee.