The controversy surrounding the recently announced recall of Windows 11 continues. The UK Data Protection Supervisor already is confirmed that it is «inquiring» with Microsoftwhile the billionaire is the CEO of SpaceX Elon Musk warned millions of X followers to opt out of the feature— all this despite Recall not yet being available on any Windows 11 PC.
Recall is one of several artificial intelligence (AI) features that come exclusively to Copilot+ computers. These devices, which include the brand new Surface Laptop 7 from Microsoft, require a dedicated neural processing unit (NPU) to process artificial intelligence tasks. Samsung, Dell and Asus, among others, have promised to launch Copilot+ computers.
But ahead of the launch of the first batch of Copilot+ PCs on June 18, cybersecurity expert Kevin Beaumont flagged a number of potential flaws in the Revocation feature. Given that Beaumont used to work for Microsoft, he’s in a good position to comment on this upcoming AI feature.
Rollback is the main feature of this Copilot+ PC list, allowing you to scroll back through everything that happened on your Windows 11 PC and go back in time with one click. Everything is searchable because artificial intelligence searched the images and text on the device
MICROSOFT PRESS OFFICE
After a week of testing, Kevin Beaumont discovered that Recall stores data in the database in plain text. Without encryption, this could make it trivial for a hacker to extract data about all that you did on the computer.
“Screenshots are taken every few seconds. Azure AI, running on your device, automatically OCRs them and writes them to a SQLite database in the user’s folder,” explained Kevin Beaumont. in a long blog post criticizing the feature.
«This database file contains a plain text record of everything you’ve ever viewed on your computer.»
OCR — optical character recognition — is the practice of digitizing images of typed, handwritten, or printed text. Microsoft will use artificial intelligence to transcribe text from web pages, Word documents, PDFs, handwritten notes, and anything else displayed on your computer screen so that all can be searched immediately.
For example, if you know you searched for flights to Spain last month — you can search the destination to find the exact website. With one click, Windows 11 will call up a document, image, video or web page to pick up where you left off. You can also scroll back through the screenshots, which are taken hundreds of times every hour and can be stored for months.
Yusuf Mehdi, director of marketing at Microsoft, described how Recall works company blog: «We set out to solve one of the most frustrating problems we face every day – finding something we know we’ve seen before on our computer. Today we have to remember what folder a file was stored in, what web page it was on, or scroll through hundreds of emails you’re trying to find, now with Recall you can access almost everything you’ve seen or done on your computer in a way that feels like you have a photographic memory.»
But cybersecurity expert Kevin Beaumont labeled the feature a «disaster,» warning that «the theft of everything you’ve ever typed or looked at on your own Windows PC is now possible with two lines of code.»
As a proof of concept, Beaumont claims to have «automated exfiltration and built a website where you can upload a database and search it instantly,» so anyone can search the full history of everything on the screen using the Recall feature.
Microsoft told media outlets that a hacker could not pull off Copilot+ Recall activity remotely.
Reality: How do you think hackers will exfiltrate this plain text database of everything a user has ever looked at on their computer? Very simple, I have it automated.
HT detective pic.twitter.com/Njv2C9myxQ
— Kevin Beaumont (@GossiTheDog) May 30, 2024
«I’m purposely withholding technical details until Microsoft ships the feature because I want to give them time to do something. I actually have a bunch of stuff to show and I think the wider cyber community will have a lot of fun with this when it’s widely available.. but I also think that that’s really sad, because real-world damage will follow,» a former Microsoft employee wrote in a detailed blog post about the flaws.
Revocation is enabled by default on all Copilot+ computers, although it can later be turned off in the settings. By default, it won’t take screenshots whenever you use a private browsing mode, like Google Chrome’s incognito mode, Microsoft reassured PC owners. You can also disable the feature for each app.
Kevin Beaumont advised Microsoft not to issue a recall with the first batch of Copilot+ PCs this month because it could undermine customer confidence in the operating system and desktop feature. He writes: «In my opinion — they should recall Recall and rework it to be the feature it deserves to be, delivered later. They should also review the internal decision-making that led to this situation, as such things should not happen.
«Earlier this month, Microsoft’s CEO sent an email to his staff saying«If you’re faced with a trade-off between security and another priority, your answer is clear: Do security.» We’ll find out if he was serious about that email.
«They need to eat some humble pie and take the hit now or risk customer confidence in their Copilot and safety brands. Frankly, few if any customers will cry foul over Recall not being immediately available – but they arewill absolutely be seriously concerned if Microsoft’s response is to do nothing, ship the product, fumble a bit or try to skirt the issue in the media.»
The recall is one of a number of features that will only appear on Windows 11 devices labeled as Copilot+ PCs, which have a chipset with an NPU to handle all the AI tasks
MICROSOFT PRESS OFFICE
Several other privacy campaigners also reacted strongly to the Revocation announcement.
«This could be a privacy nightmare,» Dr Kris Shrishak, an AI and privacy consultant, told the BBC. «The mere fact that screenshots will be taken while using the device could have a chilling effect on people.»
Jen Caltrider, who works as a program director for the *Privacy Not Included team at Mozilla, warned that Recall would provide anyone with access to your laptop or desktop computer with a treasure trove of personal information since it contains a categorized list of recent activity.
Caltrider warned: «[This includes] court orders from law enforcement, or even from Microsoft if they change their minds about keeping all this content local and not using it to target advertising or train their AI down the line.»
Microsoft says that Recall will not censor or delete data from the screenshots it takes, even when passwords or bank account information are visible on the screen.
In a blog post about the new functionality, which will arrive with a new Surface Laptop next monththe American company wrote: «Recall uses your personal semantic index, built and stored entirely on your device. Your recordings are yours; they remain locally on your computer. You can delete individual recordings, adjust and delete the time range in the settings, or pause in any moment directly from the taskbar icon on the taskbar.
«You can also filter apps and websites from saving. You’re always in control with privacy you can trust.»
Recall and a number of other AI features will be exclusive to Windows 11 running on so-called Copilot+ PCs, including the newly announced Surface Laptop 7th Edition, pictured above MICROSOFT PRESS OFFICE
But Jake Moore, global cybersecurity advisor at software security firm ESET, said creating and storing more private data through the feature could be a tempting prospect for cybercriminals.
«Enabling a feature that has the ability to record data from the screen not only offers even more data to the company behind the software, but also opens up another avenue for criminals to attack,» he said.
THE LATEST DEVELOPMENT
“Although this feature is not turned on by default, users should take care to allow any content to be analyzed by AI algorithms for a better experience.
“While it may produce better results, there is a balance that must be struck between functionality and privacy, so users must remain aware of the potential risks if any sensitive data is ever compromised. Creating and storing more private data seems unnecessary when cybercriminals are constantly looking for any vulnerability they can exploit.”