- Sign-Speak aims to use machine learning to make communicating through sign language easier.
- Cofounder Nikolas Kelly's own challenges communicating while deaf prompted the idea.
- The software translates between English and sign language and is now in some zoos and restaurants.
When Nikolas Kelly got a potential cofounder — Yamillet Payano — on a video call to hear out his startup idea, the circumstances of their meeting alone turned out to be enough of a sales pitch.
Kelly, who is deaf, had an idea to create software that would translate sign language for non-speakers in real time with machine learning. When the two met virtually, Payano and Kelly had difficultly communicating since she didn't know sign language, and Zoom's caption service didn't offer
"You find that thing that you know you're supposed to do, and you get this epiphany moment of, 'I need to do something about this problem,'" Payano told Insider. "That's what happened to me."
Together, with help from Kelly's friend Nicholas Wilkins — who was working as a software engineer for Google at the time — they started Sign-Speak.
According to the World Health Organization, there are nearly 430 million people globally who are deaf or have disabling hearing loss. Despite this, Payano said that a lack of sign-language datasets is just one of the reasons why translation products like theirs haven't reached the mainstream. Now, Sign-Speak is building its own dataset of ASL signs to power its translation. The software — which is designed to work with any camera-enabled device — currently only supports English, but the company has plans to expand to other languages.
"Think of us as like Siri for deaf people," Payano explained.
Sign-Speak ran a beta test in late 2021 at a pizza restaurant in Washington, DC, to assist customers with ordering. Tablets with Sign-Speak's software provided spoken translation as customers signed orders to enable them to easily interact back and forth.
Earlier this year, Sign-Speak also tested the product at a zoo in Rochester, New York, to help guide visitors. At both locations, Payano said an average of 90 people a day used the software, allowing businesses to better serve a community they hadn't considered before. The tests have both ended, but Sign-Speak hopes to launch with some enterprises and small businesses sometime this year.
"They didn't have to write back and forth. They didn't have to beg, plead. They didn't have to come overly prepared," Payano said. "Deaf and hard-of-hearing individuals felt seen for the first time."
Having to be overly prepared, according to Payano, means that deaf and hard-of-hearing people understand certain businesses are not designed to serve them. She cited banks as an example of businesses that deaf and hard-of-hearing clients feel do not accommodate for them. In 2011, Wells Fargo settled an Americans with Disability Act, also known as ADA, complaint from the US Department of Justice alleging that the bank refused to do business with deaf clients. Payano envisions a future where Sign-Speak could deploy its software in banks to help ease transactions with deaf and hard-of-hearing individuals.
Currently, many deaf and hard-of-hearing individuals rely on translators, lip reading, and writing notes on paper to engage with different businesses. Having translators and interpreters can be cost-prohibitive, Payano said. Still, Sign-Speak says it doesn't want to replace translators and interpreters, but rather serve deaf and hard-of-hearing individuals in more accessible ways. Above all, Payano said the company has a responsibility to the community it serves.
This also means expanding the technology so that people can use sign-language translation anywhere. Payano talked about a future where two people — one deaf or hard-of-hearing, and one not — will be able to communicate on a train platform using an app.
As any company grows, Payano says it's important to remember the intent around a product and who that product is serving.
"We believe in building with the deaf community, not for the deaf community," she said. "And we have a long way to go."