Join my exclusive Dead Drop Digest with the most popular spy stories, tips, & tricks.

close
Breaking news

A Russia-linked cyber-espionage group has hacked into the controls of electricit...read more What is metadata, how is it used and why is it important? When we interact over ...read more Mass surveillance is the subjection of a population or significant component of ...read more There’s a good chance you were affected by the Equifax hack of 143 million nam...read more Facebook has the ability to spy on users in their homes and offices because of...read more

Facebook is listening to everything for more than conversation-based ads

Facebook has the ability to spy on users in their homes and offices because of the official Facebook policy, “growth at any cost,” says Cambridge Analytica whistleblower Christoper Wylie, appearing before a committee of British MPs.

The British parliament is investigating Cambridge Analytica’s involvement in the Brexit election. MP Damian Collins, who chaired the committee, asked Wyle whether Facebook has the ability to listen to what people are talking about in order to better target them with ads.

“There’s been various speculation about the fact that Facebook can, through the Facebook app on your smartphone, listen in to what people are talking about and discussing and using that to prioritize the advertising as well,” Collins said. “Other people would say, no, they don’t think it’s possible. It’s just that the Facebook system is just so good at predicting what you’re interested in that it can guess.” He asked for Wylie’s thoughts on the possibility.

“On a comment about using audio and processing audio, you can use it for, my understanding generally of how companies use it… not just Facebook, but generally other apps that pull audio, is for environmental context,” Wylie said. “So if, for example, you have a television playing versus if you’re in a busy place with a lot of people talking versus a work environment.” He clarified, “It’s not to say they’re listening to what you’re saying. It’s not natural language processing. That would be hard to scale. But to understand the environmental context of where you are to improve the contextual value of the ad itself” is possible.

Wylie continued: “There’s audio that could be useful just in terms of are you in an office environment, are you outside, are you watching TV?”

Facebook has long denied allegations that its app listens in on users in order to customize ads.

“I run ads product at Facebook. We don’t – and have never – used your microphone for ads. Just not true,” Rob Goldman, vice president of ads products at Facebook, tweeted on October 2017. “That includes Facebook-owned Instagram,” he added.

But users have raised concerns about the practice after observing that they’ve been targeted with ads for products they’ve never expressed an interest in online. Many Facebook users have reported examples and alleged evidence that the app is listening in on their conversations.

During the hearing, the former employee of Cambridge Analytica, who told MPs that he’s a “progressive Euroskeptic,”  was also asked whether the company used “coercion to achieve their desired electoral results.”

Citing the “Nigeria project” as an example, Wylie said he “got incredibly panicked phone calls” from people involved in the project. “Several people were threatened with their lives and they had to immediately leave the country,” he said.

Wylie told the committee that he was informed by Cambridge Analytica shortly after he began his position that Dan Muresan, his predecessor, had been poisoned and died in Kenya after a political deal went wrong. Emphasizing that what he had heard was “pure speculation,” Wylie said he had been told police had been bribed not to enter the hotel room where Muresan had been found.

Facebook has long denied allegations that its app listens in on users in order to customize ads.

“I run ads product at Facebook. We don’t – and have never – used your microphone for ads. Just not true,” Rob Goldman, vice president of ads products at Facebook, tweeted on October 2017. “That includes Facebook-owned Instagram,” he added.

But users have raised concerns about the practice after observing that they’ve been targeted with ads for products they’ve never expressed an interest in online. Many Facebook users have reported examples and alleged evidence that the app is listening in on their conversations.

During the hearing, the former employee of Cambridge Analytica, who told MPs that he’s a “progressive Euroskeptic,”  was also asked whether the company used “coercion to achieve their desired electoral results.”

Citing the “Nigeria project” as an example, Wylie said he “got incredibly panicked phone calls” from people involved in the project. “Several people were threatened with their lives and they had to immediately leave the country,” he said.

Wylie told the committee that he was informed by Cambridge Analytica shortly after he began his position that Dan Muresan, his predecessor, had been poisoned and died in Kenya after a political deal went wrong. Emphasizing that what he had heard was “pure speculation,” Wylie said he had been told police had been bribed not to enter the hotel room where Muresan had been found.

“My predecessor was found dead. One of my former coworkers had a massive head injury and is missing part of his skull. People do get hurt at this firm.” He added that people are afraid to come forward to talk about the company “because it’s that intimidating.”

In an apparent pushback, Cambridge Analytica tweeted that Wylie’s role in the company had been limited.

You can watch the entire hearing below. The discussion of Facebook’s alleged audio surveillance begins at 2:58.

Source


THE TEST

I gave it audible input of professional photography for about 45 minutes. Here are the ads I received the next time I opened Instagram.

Graph of activity

Ad 1

Profile of Ad 2

Profile of Ad 3

Profile of Ad 4

Method:

I connected my iPhone to my Mac via Internet Sharing, and the Mac is connected to the router via Ethernet. I started capturing packets of WiFi activity only with WireShark, so that I could see the packets going in and out of the phone.

I exposed the iPhone to YouTube tutorial videos about professional photography of commercial products (deodorant, shampoo, and watches), playing in a Chrome incognito window, so hopefully this helped separate any possibility of cookies playing a role. The videos played for about 40 minutes. The Facebook app was open and in the “post status” window the whole time.

I let the phone sit for another 40 minutes or so, locked and in silence, to see if it would do anything while the screen was locked. Answer – not this time.

After the experiment, I received multiple ads for photographers and graphic designers.

I noticed when Facebook is first opened, and put into the “post status” mode, there are some small spikes in activity. After it was put in “post status” mode the second time, there were seemingly no spikes in activity. Facebook supposedly always records what you type even if you don’t post it, so it’s interesting to me why the same screen would have two different levels of activity.

When I analyzed the raw data from WireShark, the majority of outgoing traffic to Facebook uses the Facebook Zero (FB_ZERO) protocol. This protocol is primarily used to send/receive data between you and Facebook without counting against a data plan limit, but also has the advantage of getting a 0-round-trip encryption handshake, speeding up the encrypted content delivery. The downside is, there is no way to tell what the contents of FB_ZERO packets are, because they are encrypted.

The other half of the traffic were TCP acknowledgements of having received data. These appeared even when there was seemingly no interaction, other than leaving the screen open into “post status” mode.

I had a few minutes where I had to delete some podcasts to make room for documenting videos of the experiment. No unusual activity to Facebook during this time.

There’s probably a lot of controls that were missed in this experiment, but it’s a start. Let me know what I can do to improve.

EDIT: some conclusions after some discussion here

  • There are packets being sporadically sent from me to Facebook while the phone is sitting idle in “post status” mode
  • The packets being sent are about 50% FB_ZERO encrypted packets, and 49% TCP acknowledgements of having received data (again, while phone is sitting idle). Other 1% is TLS 1.2 and SSL packets.
  • FB_ZERO being active seems to be a key to all this. I’ve heard a common rebuttal of “if Facebook were listening, your cellular data plan limit would be met very quickly.” FB_ZERO protocol avoids spending data against a limit.
  • If Facebook were listening, it could potentially listen and upload in brief “micro-samples.” It could possibly be a few seconds of low-quality compressed audio, recorded and uploaded in brief spurts. This would be similar to “wake words” for assistant devices

Variations for future experiments:

  • Try a different interest, particularly something with a very large marketing budget, that would be both unrelated to me, and unrelated to Instagram.
  • Try experiment with mic permissions on for Facebook. Currently mic permissions are off, as they always have been.
  • Try experiment with phone on mobile data. Although packets could not be captured with Wireshark, it would represent a more real-life application.
  • Try playing YouTube with phone in a different room, to test for IP-based or cookie-based data linking.
  • Separate phone and computer IP addresses. I have a laptop, which could play audio, and connect to the main router via WiFi. The phone could sit next to the laptop, connected to the Mac via Internet Sharing, and the Mac be connected to a VPN to change its apparent IP address. Hopefully this would provide enough separation to control for IP-based or location-based data linking. An air-gapped media device like a DVD player or tape player might also provide more control.
  • Try listening to a different language, to see if ads appear in a different language. There is a long-standing rumor that playing a Spanish-speaking radio station will convert ads to Spanish. I previously tried to bait Facebook with advertisements in Mandarin Chinese, featuring ads for Pepsi and KFC. That did not seem to work. The packet activity was significantly smaller as well.

source

Tags: ,

No Comments

Leave a reply

Story Page