Facebook Developing ‘Egocentric’ Artificial Intelligence

A Facebook-led artificial-intelligence research project hopes to make machines think more like the people who use them.

The Ego4D programme is training AIs to interact with the world from an “egocentric” or first-person perspective.

The aim is to allow wearable devices such as augmented reality (AR) glasses to assist their users in everyday tasks such as cooking or remembering where they put the keys.

The project is training systems using a massive trove of video recorded from a first-person point of view.

Image credit: Facebook

‘Centre of the action’

“Next-generation AI will need to learn from videos that show the world from the centre of the action,” Facebook said in a blog post.

It said it wants “immersive devices” such as AR glasses and VR headsets to become as useful as smartphones.

Facebook owns VR headset maker Oculus and is also developing AR spectacles.

At the centre of Ego4D is a “massive-scale egocentric video dataset” developed with 13 universities in nine countries.

It includes 3,025 hours of video of daily life activities from 855 individuals wearing cameras, researchers said.

Massive data set

Users filmed while doing construction work, baking, playing with pets, socialising and carrying out other activities in countries including Saudi Arabia, Tokyo, Los Angeles and Colombia.

The company said the dataset includes 20 times more footage than any other, and will be available from November to researchers who sign a data use agreement.

Facebook also developed benchmarks for what it hopes to achieve, including “episodic memory” (“Where did I leave my keys?”), forecasting what the user is likely to do next (“Wait, you’ve already added salt to this recipe”), hand and object manipulation, keeping a record of who said what when (for instance, during a class), and social interaction.

But the project is likely to raise privacy concerns, based as it is on the idea of a computer system that’s constantly recording data from the user’s surroundings.


The company has been heavily criticised and fined for its previous record on privacy.

And its Ray-Ban Stories smart glasses, which allow the user to record short video clips, prompted a response from the data protection regulators of Ireland and Italy earlier this year.

Technology news site The Verge noted that Facebook’s benchmarks for the project don’t include privacy parameters, such as refraining from recording people who don’t wish to be recorded when creating diaries or recording social interactions.

Facebook told the site that safeguards would be brought in as applications were developed.

Matthew Broersma

Matt Broersma is a long standing tech freelance, who has worked for Ziff-Davis, ZDnet and other leading publications

Recent Posts

Lawsuit By Former Google Staffers Cites ‘Don’t Be Evil’ Moto

Three out of the 'thanksgiving four' staff fired from Google in November 2019, hit back…

17 hours ago

Amazon’s Alabama Warehouse To Vote Again On Trade Union

Not the right outcome? Workers and staff at Amazon fulfilment centre in Bessemer, Alabama to…

18 hours ago

Meta Ordered To Sell Giphy By British Regulator

But will Zuckerberg obey? The UK's CMA watchdog orders Facebook to sell Giphy, after concluding…

22 hours ago

Clearview AI Faces £17m Fine For ‘Serious’ Data Protection Breaches

American facial recognition firm Clearview AI is facing a possible £17 million fine over “serious…

23 hours ago

Bye Jack. Twitter Co-Founder And CEO Jack Dorsey Steps down

End of the road for Jack. Twitter's Jack Dorsey steps down from CEO role for…

23 hours ago