What started as a 36-hour hackathon project last weekend could empower the open source community to revolutionize the smart glasses industry. Five team members built a $20 pair of smart glasses, called Open Glass, that connects what you see and hear to an AI chatbot, like Meta’s La Llama 3.
Why do all hurricanes spin in the same direction?
Scott Fitsimones headed to downtown San Francisco on Saturday morning to meet Nik Shevchenko, not knowing he was about to spend the next 36 years. hours at an AI hackathon building a new device with it. At the time, Fitsimones thought he was picking up an AI pendant made by Shevchenko, whom he describes as a leader of San Francisco’s growing AI wearables movement. At the end of the weekend, his team won the hackathon and earned approximately 1,500 people on waiting list to pre-order your open source smart glasses.
“I had no idea about this hackathon, and it was just very serendipitous,” Fitsimones said. “And then, you know, I think we just started improvising and building on the initial prototype.”
For her part, Shevchenko participated in the hackathon knowing that she wanted to build the hardware element for a type of smart glasses, according to her teammates (Shevchenko did not respond to Gizmodo’s interview request). He was joined by Stepnan Korshakov, who solved the most difficult software. project challenges. These two chose Fitsimones, Shreeganesh Ramanan and Jatin Gupta to form a winning team.
In a large, airy room overlooking the blue waters and green mountains of the Bay Area, software engineers coded on plush couches with stacks of La Croix on one side. Cerebral Valley hosts hackathons like this often, bringing together San Francisco’s thriving AI startup scene. Shevchenko was one of the few who used a soldering pen instead of a laptop, while the rest of the team continued working with the software. At one point on Saturday afternoon, Shevchenko left the event to 3D print the Open Glass computer case.
After about 36 hours of hacking, Shevchenko and the team proudly presented a pair of cheap sunglasses with a black box protruding from the right side. side. The glasses included a camera that took a photo every five seconds and a microphone that constantly transcribed audio. This compiles a database of photos and text to reflect what your eyes and ears take in. Press a button on the side of the glasses and you can ask Meta’s Llama 3 chatbot to answer questions about your own life.
“What’s that person’s name?”, “Where did I leave my keys?” and “How many calories do these fruits have?” some questions the AI answered during the demo. The technology has useful applications for many people, but it might excel in helping someone with poor vision or hearing.
Shevchenko’s team won first prize at the hackathon, despite a bug in the glasses’ speech-to-text capability during the demonstration. from Meta and Groq executives, as well as Hugging Face CEO Clem Delangue, who judged the final projects. Within hours, Shevchenko’s entrepreneurial mind kicked into gear and she created a waiting list to reserve a version of the prototype.
“Woah, woah, woah, woah, that’s 1,300,” Korshakov said on the phone with me on Monday when he found out how many pre-orders they had received. People around the world want to build things with this. Now they have some way to own and contribute to the success of this project. themselves”.
While there are other smart glasses on the market today, such as the Meta Ray-Ban, they are not open source or as cheap. Glass offers a relatively inexpensive kit that allows developers to choose which LLM they want to use and decide what they want. For example, not all Open Glass have to constantly take photos or record audio. This offers an affordable option. , a hackable option for smart glasses, a form factor that was previously wildly expensive and limiting in what could be done.
“You can connect it to OpenAI, you can connect it to Gemini,” Ramanan said in a phone interview. having the ability to mix and match the best options and then create your own interesting applications and frameworks.”
Smart glasses have not caught on like other wearables have. However, the advancement of multi-modal AI models makes this an exciting time for smart glasses. It’s easy to imagine how OpenAI’s new GPT-4 Omni, which can process video, audio, and text simultaneously, could be used in glasses like this. Google even showed off a prototype of new Google Glasses in a demo of its latest AI on Tuesday. Open Glass hopes that giving the open source community access to this technology will allow for greater innovation in the space.
One thing that smart glasses are plagued with is privacy concerns. Meta’s Ray-Bans don’t constantly record audio and video to use your life as a database, and that’s probably a good thing. But there is a growing community of AI device fans in Silicon Valley who are interested in the idea of constantly recording your life to create the ultimate personal assistant. Rings, pendants and now glasses are emerging from startups in San Francisco, all fascinated by this potential.
While open source technology will allow developers to innovate on these ideas in a more unique way, there are still important concerns to be resolved. addressed with privacy and costs. These questions are important, but they are not exactly the focus for developers in the early stages of this technology. The most important thing for them is to make it useful.
Some non-technical people may purchase Open Glass to simply use a cheap pair of smart glasses. The team is still analyzing the product, but it looks like it will come pre-built with large language models built in and an accompanying mobile app. The actual price of the device is also subject to change, although all source code is available. free on GitHub.
The story of Open Glass is a testament to the thriving AI startup culture in San Francisco. The open source community could unlock key advances in stagnant technology, such as smart glasses. More practically, it could also offer non-technical people a pair of smart glasses for the price of a movie ticket.
This content has been automatically translated from the original material. Due to the nuances of machine translation, there may be slight differences. For the original version, click here.
