Instagram's lax privacy practices let a trusted partner track millions of users' physical locations, secretly save their Stories, and openly flout its rules
- A buzzy San Francisco startup has been secretly saving what appears to be millions of Instagram users' Stories and tracking their locations.
- Marketing firm HYP3R has been scraping huge quantities of data off the Facebook-owned app and using it to build up detailed profiles of people's movements and interests.
- The situation highlights how Facebook is still struggling to protect users' data and oversee developers accessing its platform, more than a year after the Cambridge Analytica scandal revealed important privacy lapses.
- Instagram has now issued HYP3R with a cease-and-desist, kicked it off its platform, and made changes to its platform to protect user data.
- EDITOR'S NOTE: This story would normally be exclusive to BI Prime members. However, due to the public interest in this reporting, we're making this story free to read for a limited time.
A combination of configuration errors and lax oversight by Instagram allowed one of the social network's vetted advertising partners to misappropriate vast amounts of public user data and create detailed records of users' physical whereabouts, personal bios, and photos that were intended to vanish after 24 hours.
The profiles, which were scraped and stitched together by San Francisco-based marketing firm HYP3R, were a clear violation of Instagram's rules. Yet it all occurred under Instagram's nose for the past year by a firm that Instagram had blessed as one of its "preferred marketing partners."
On Wednesday, Instagram sent HYP3R a cease and desist letter after being presented with Business Insider's findings, and confirmed that the startup broke its rules. "HYP3R's actions were not sanctioned and violate our policies. As a result, we've removed them from our platform. We've also made a product change that should help prevent other companies from scraping public location pages in this way," a spokesperson said in a statement.
The existence of the profiles is a stark indication that more than a year after revelations that Facebook users' data was exploited by Cambridge Analytica to fuel divisive political ad campaigns, Facebook's struggles locking down users' personal information not only persist, but extend beyond the core Facebook app. Instagram, which is owned by Facebook but operated as a mostly separate business, has been largely insulated from the privacy backlash and scrutiny that has rocked its parent company.
But the wealth of the data contained in people's fleeting Instagram activity, from family vacation snapshots to restaurant appetizer photos, can provide valuable fodder for a variety of outside actors, who can repurpose the information in ways users never expected or agreed to.
Business Insider spoke to multiple former employees of HYP3R to learn about its practices, and reviewed public documents and marketing materials that outline its capabilities.
The total volume of Instagram data HYP3R has obtained is not clear, though the firm has publicly claimed to have "a unique dataset of hundreds of millions of the highest value consumers in the world," and sources say more than of 90% of its data came from Instagram. It ingests in excess of one million Instagram posts a month, sources say.
Data scraping is a persistent problem across the web for open platforms. Instagram is not the only service to have been affected over the years, and HYP3R is almost certainly not the only business scraping its data. But the nature of HYP3R's activity raises significant questions about the extent of the due diligence that Instagram and parent company Facebook conduct on partners using their platform as well as their own procedures to safeguard user data.
"For [Instagram] to leave these endpoints open, and let people get to this in a backchannel sort of way, I thought was kind of hypocritical," one former HYP3R employee said. It takes very little effort for Instagram to protect the location data accessed by HYP3R, they said: "Why they haven't done it remains a mystery."
HYP3R denied breaking Instagram's rules, arguing essentially that accessing public data on Instagram in this way is legitimate and justifiable and said it was confident that any issues with Instagram will be resolved shortly.
CEO Carlos Garcia said in an emailed statement: "HYP3R is, and has always been, a company that enables authentic, delightful marketing that is compliant with consumer privacy regulations and social network Terms of Services. We do not view any content or information that cannot be accessed publicly by everyone online."
'A location-based marketing platform'
HYP3R, founded in 2015, describes itself as "a location-based marketing platform that helps businesses unlock geosocial data to acquire and engage high-value customers."
In simpler terms: HYP3R is a marketing company that tracks social media posts tagged with real-world locations. It then lets its customers directly interact with those posts via its tools, and uses that data to target the social media users with relevant advertisements. Someone who visits a hotel and posts a selfie there might later be targeted with pitches from one of the hotel's competitors, for example.
To provide some of these capabilities, HYP3R made unauthorized use of Instagram data in 3 key ways:
- It took advantage of an Instagram security lapse, allowing it to zero in on specific locations, like hotels and gyms, and vacuum up all the public posts made from the locations.
- At these locations, it systematically saved users' public Instagram Stories - a type of content designed to vanish after 24 hours -including the individual photos that users shared in the Stories, in a clear violation of Instagram terms of service.
- It scrapped public user profiles on a broad basis, collecting information like user bios and followers, which it then combined with the other location information and data from other sources.
It also uses image recognition software on users' posts it collects to automatically analyse what they're depicting.
HYP3R did not access any non-public data from Instagram users who set their profiles' privacy settings to "private."
The result of the public information it gleaned was a sophisticated database about Instagram users, their interests, and their movements that HYP3R openly touted to customers as one of its key selling points, despite the fact that Instagram's policies were structured so that such a thing would not be possible.
HYP3R's data-scraping was a response to post-Cambridge Analytica changes
HYP3R is not a shady, boiler-room operation.
The buzzy startup has raised tens of millions of dollars, including a $17.3 million funding round in September 2018 from backers including Silicon Valley Bank and Thayer Ventures. It has won multiple awards - including a Most Innovative Company accolade from Fast Company in 2019 and 2018, and a Cannes Lions in 2017. It counts marquee brands like Marriott international, Pepsi, Hard Rock, and 24 Hour Fitness among its clients, and Jim Messina, a former Obama aide, sits on its board.
Some of HYP3R's behavior was once permitted by Instagram.
Like many big platforms, Instagram has an API, or Application Programming Interface, that allows developers to build services that can interact with its platform. (They're the reason you can save files to Dropbox from Microsoft Office, or see your Facebook friends on Spotify, for example.)
But revelations in March 2018 about political research firm Cambridge Analytica's misappropriation of 87 million Facebook users' data - data which was originally collected via a quiz app built on top of Facebook's API years prior - prompted a sea-change for Facebook, including at Instagram.
Prior to the scandal breaking, Instagram's API allowed developers to search for public posts for a given location. But in the aftermath Instagram began to deprecate (i.e. switch off) a bunch of its API's functionality, including location tools - causing chaos for companies like HYP3R that had been relying on it.
Publicly, HYP3R welcomed Instagram's API changes, writing a worthy blog post in which it said it "understand[s] and welcome[s] the changes that Facebook is making to protect the privacy of all of us," and promising its data would never be used for political purposes.
But behind the scenes, the company got to work building a system that could disregard Instagram's decision and keep on harvesting data anyway, sources told Business Insider.
HYP3R geofenced thousands of locations around the world, then slurped up public posts
HYP3R created a tool that could "geofence" specific locations and then harvest every public post tagged with that location on Instagram.
The result is a database of thousands of locations, including "hotels, casinos, cruise ships, airports, fitness clubs, stadiums and shopping destinations across the globe," as well as hospitals, bars, and restaurants.
If a user makes a post at one of these locations, it is, unbeknownst to them, saved to HYP3R's systems indefinitely, sources say, along with other information including a link to their profile picture, their profile bio, and their number of followers.
Ordinary users' Instagram Stories - posts that are supposed to disappear after 24 hours - have never been available through Instagram's API. But HYP3R built a tool to collect them too, sources said, saving the images indefinitely along with the associated metadata. (The official API only allows access to Stories of Business Accounts and Creator Accounts, a tiny fraction of the Instagram population, and these are not surfaceable by location.)
The posts and Stories HYP3R collected were available publicly - but viewable only as single pieces of content. By harvesting them systematically from popular locations, HYP3R became able to build up detailed profiles of huge numbers of people's movements, their habits, and the businesses they frequent over time.
Imagine visiting a new city, and sharing a geotagged story with friends of the hotel you visited. By itself, it doesn't tell viewers much about you.
But combine it with the story you posted from the hospital you visited for a check-up, and the selfie you made the next day at a sports stadium, and the story from the vegetarian restaurant you ate at, and so on, and an intimate picture of your life and interests begins to emerge over weeks and months.
The collection and preservation of Stories in particular appears to defy Instagram user expectations. People share Stories with the understanding they will disappear in a day's time; instead many are being saved indefinitely by a company without their knowledge, and used to profile them.
HYP3R argued that because the data it collects is already public, it does not require consent from Instagram users to harvest it, and that companies have legitimate business needs that justify knowing what is being shared from their properties.
How HYP3R uses its data
HYP3R has put this treasure trove of data to work in multiple different ways.
First, it lets customers easily engage with users that are at their properties via the app, using its tool "Engage." It means Marriott, for example, can see every post tagged at a Marriott hotel via the HYP3R app, including comments, and likes and respond to them where it wants to. This is not currently possible for apps built on Instagram's official API.
It can also target people with ads, based on their interests and the locations they've visited. Businesses can ask HYP3R to geofence their rivals' locations, then subsequently target people who have visited those rivals with ads on Facebook.
The harvested Instagram data can also be combined with data collected elsewhere on platforms like Salesforce and Adobe - creating ever-more detailed profiles about the people whose information is being scraped.
And you can view profiles of people whose data has been collected, and combine it with other data collected through other tools using other platforms like Salesforce and Adobe - building up ever-more detailed profiles about the people whose data is being scraped.
Salesforce and Adobe did not immediately respond to Business Insider's request for comment as to how they vetted HYP3R before partnering with the startup.
Why didn't Instagram spot this?
HYP3R has made no attempt to hide what it does.
The company's iOS App Store listing shows screenshots of an Instagram post in its app that it says it collected from a specific location - a capability that Instagram does not allow - and in its release notes from December 2018 it references adding "support for Instagram Stories across the app."
It publicly promises its customers features that far exceed what is available through Instagram's API, saying it "surfaces all public social activity from a location - regardless of hashtags and mentions - so you never miss an opportunity to dazzle your customers." (Instagram's current API allows users to view public posts if they have been mentioned in it, or retrieve some hashtagged posts subject to stricter limitations, but not because of their location.)
However, Facebook included HY3R on its exclusive list of Facebook Marketing Partners - a directory of vetted companies that "can give you superior insights and data for better marketing decisions."
A spokesperson for Instagram said that the company periodically reviews Facebook Marketing Partners to ensure compliance.
HYP3R's scraping appears to violate Instagram's rules on multiple points, including a requirement to store or cache content only "for the period necessary to provide your app's service" (HYP3R stored user data indefinitely, according to multiple sources), and a prohibition on "reverse engineer[ing] the Instagram's APIs" (HYP3R deliberately rebuilt its own version of an API that Instagram shuttered after Cambridge Analytica).
Similarly, Facebook's Automated Data Collection terms say: "You will not engage in Automated Data Collection without Facebook's express written permission."
Instagram also bans data from being transferred "to any ad network," but the Instagram data could be plugged into Facebook's own Ads Manager to target people with advertisements - meaning Facebook indirectly profited from HYP3R's data collection.
HYP3R disputed that it violated Instagram's terms of service and data policies. However, an Instagram spokesperson said its practices violated the company's rules on automated data collection.
The marketing firm's behaviour seems unlikely to be illegal under US law. Back in 2017, LinkedIn lost a legal fight against a company that had been scraping its publicly available data.
Instagram's data lapse
HYP3R also took advantage of a lapse in Instagram's security to boost its data collection.
When accessing Instagram through a web browser, there is a publicly available JavaScript package that bundles up various bits of data into an easy-to-access format. It's available by simply appending a short string of characters to any Instagram URL, and you don't need to log in, gain approval, or authenticate your identity in any way to access it.
At Instagram's request, Business Insider is not currently sharing the exact method of accessing the package, so the company has time to fix the issue.
Instagram displays public location pages, showing ordinary users posts from a given location, and this package appears on those pages. Sources say that it's through this that HYP3R was able to scrape some of the data it was illicitly collecting on users.
In other words: A year after Instagram disabled its location functionality for developers, the social network was still inadvertently providing an easy way for developers to keep on collecting this data, without any accountability.
The data would still have been technically possible to scrape had this JavaScript package not existed - but its exposure made it significantly simpler.
It's not clear why Instagram's automated tools for detecting bots on its platform failed to detect HYP3R's mass-scale scraping.
In response to HYP3R's actions, Instagram has made a change to prevent public location pages being available to logged-out users.
It has also completely revoked HYP3R's access to its APIs, and removed it from the list of Facebook Marketing Partners.
An Instagram spokesperson said they couldn't yet comment on whether they will notify affected users, or ask HYP3R to formally certify that it deletes the data. The social network has formally asked HYP3R to stop collecting Instagram data in its cease and desist letter, they said, and will ask it to explain itself in a phone interview and provide an account of all the data that was scraped.
Do you work at Instagram or HYP3R? Got a tip? Contact this reporter via encrypted messaging app Signal at +1 (650) 636-6268 using a non-work phone, email at rprice@businessinsider.com, Telegram or WeChat at robaeprice, or Twitter DM at @robaeprice. (PR pitches by email only, please.) You can also contact Business Insider securely via SecureDrop.
Read more:
- Mark Zuckerberg's personal security chief accused of sexual harassment and making racist remarks about Priscilla Chan by 2 former staffers
- Facebook says it 'unintentionally uploaded' 1.5 million people's email contacts without their consent
- Years of Mark Zuckerberg's old Facebook posts have vanished. The company says it 'mistakenly deleted' them.
- Car-bomb fears and stolen prototypes: Inside Facebook's efforts to protect its 80,000 workers around the globe