Categories: Innovation

Mark Zuckerberg Shows Off Jarvis-Inspired AI Home Assistant

As the founder and CEO of one of the world’s biggest and most successful technology companies, Facebook’s Mark Zuckerberg is a man who likes a challenge.

In January this year, he revealed his 2016 aim of building a fully-functional, artificial intelligence (AI) system to help “run my home and help me with my work.” Just under 12 months later and he has achieved his goal.

After around 100 hours of work over the course of the year, Zuckerberg has spent this week showing off his AI-powered personal assistant inspired by Jarvis, the virtual assistant designed by Tony Stark in Marvel’s Iron Man.

Jarvis comes to life

“So far this year, I’ve built a simple AI that I can talk to on my phone and computer, that can control my home, including lights, temperature, appliances, music and security, that learns my tastes and patterns, that can learn new words and concepts, and that can even entertain Max,” he wrote in a blog post.

“It uses several artificial intelligence techniques, including natural language processing, speech recognition, face recognition, and reinforcement learning, written in Python, PHP and Objective C.”

The first issue he encountered was connecting all of the different systems in his house, which he described as being “much more complicated than I expected”. As the systems are based on different languages and protocols, Zuckerberg had to first write code that would connect all of the disparate systems together.

“We use a Crestron system with our lights, thermostat and doors, a Sonos system with Spotify for music, a Samsung TV, a Nest cam for Max, and of course my work is connected to Facebook’s systems,” he explained. “I had to reverse engineer APIs for some of these to even get to the point where I could issue a command from my computer to turn the lights on or get a song to play.”

“Further, most appliances aren’t even connected to the internet yet. It’s possible to control some of these using internet-connected power switches that let you turn the power on and off remotely. But often that isn’t enough.” Some appliances actually required hardware modifications in order for them to work, such as a toaster from the 1950’s which he fitted with a “connected switch” to enable it to automatically start toasting when the power comes on.

He continues: “For assistants like Jarvis to be able to control everything in homes for more people, we need more devices to be connected and the industry needs to develop common APIs and standards for the devices to talk to each other.”

Zuckerberg also explains the subtle nuances involved with natural language requests for actions such as playing music, as well as how he incorporated vision and face recognition so the system can identify which room of the house people are in or recognise who someone is when they knock at the front door.

So, what’s next? “In the longer term, I’d like to explore teaching Jarvis how to learn new skills itself rather than me having to teach it how to perform specific tasks. If I spent another year on this challenge, I’d focus more on learning how learning works,” said Facebook’s fonder.

Zuckerberg would also like to find ways to make this sort of system “available to the world,” something he was unable to do with this version as it is “too tightly tied” to his home, appliances and network.

“Building Jarvis was an interesting intellectual challenge,” he concludes. “This year I thought I’d learn about AI, and I also learned about home automation and Facebook’s internal technology too. That’s what’s so interesting about these challenges.”

Quiz: How much do you know about the world of AI?

Sam Pudwell

Sam Pudwell joined Silicon UK as a reporter in December 2016. As well as being the resident Cloud aficionado, he covers areas such as cyber security, government IT and sports technology, with the aim of going to as many events as possible.

Recent Posts

Intel ‘Playing Politics’ Over Delayed Ohio Chip Factory, Alleges Governor

Ohio Governor Mike DeWine alleges Intel's Ohio factory delay is a negotiating tactic, despite Pat…

2 hours ago

Steve Jobs Posthumously Awarded US Medal Of Freedom

President Joe Biden has named Apple co-founder and former CEO Steve Job, as a posthumous…

3 hours ago

Twitter Seeks Judicial Review Of Indian Takedown Order

Clash continues, Twitter court challenge against Indian government order to remove certain content it deems…

4 hours ago

TikTok ‘Halts E-Commerce Expansion Plans’

TikTok reportedly scraps plans to expand TikTok Shop livestream commerce in Europe and US after…

23 hours ago

European Parliament Passes Landmark Tech Regulations

European Parliament votes to adopt Digital Markets Act and Digital Services Act, but campaigners warn…

23 hours ago

Indian Economic Police Raid Offices Of Smartphone Maker Vivo

Indian economic crime agency Enforcement Directorate raids dozens of locations across India belonging to China's…

1 day ago