Disclosures: Let’s get this out of the way right up front. DataSift is a client of mine at Altimeter Group. I am closely connected to Twitter via my role as a board member of the Big Boulder Initiative, of which Chris Moody, VP Data Strategy at Twitter, is chair.
On Friday, Zach Hofer-Shall, head of Twitter's ecosystem, published a post on the Gnip/Twitter blog entitled "Working Directly With the Twitter Data Ecosystem." Shortly thereafter, Nick Halstead, CEO of DataSift, published a post (since updated) entitled "Twitter Ends its Partnership with DataSift – Firehose Access Expires on August 13, 2015". The next day, Tim Barker, DataSift's Chief Product Officer, added another take: "Data Licensing vs. Data Processing."
In a nutshell, this means that Twitter has ended its reseller agreements with third parties, and, going forward, will be the sole distributor of Twitter data.
To be clear, this does not mean that Twitter is discontinuing firehose access; in the future, Twitter will control its own data relationships rather than licensing data through resellers.
These posts have sparked a flurry of commentary across a wide spectrum, from support to vilification to philosophizing on the meaning of platforms, analytics and ecosystems. I've included links to a few at the end of this post.
It wasn’t a huge surprise for anyone watching Twitter go public, and subsequently disclose the revenues from the direct data business, to anticipate that Twitter might realize that this was an area ripe for a significant strategy shift. And it was a short hop from there to conclude that DataSift’s (and possibly others') days of reselling data received via the Twitter firehose might be numbered.
It also hasn't been a surprise to see Twitter enhance its analytics, re-evaluate the short-and-long-term value of its data and announce strategic partnerships such as the ones with IBM and Dataminr as it seeks to build its partner strategy and revenue potential.
Meanwhile, DataSift has continued to execute on its own strategy, which includes broadening its data sources far beyond social data, announcing VEDO, its data categorization platform, and via its developing its privacy-first PYLON technology (see its announcement with Facebook on how they are providing privacy-safe topic data).
Long story short: no one was shocked at the news. But the reaction to it has been polarizing in the extreme. What seems to have fanned the flames are a few different dynamics:
It also doesn’t help that Twitter has been (except for the original post) all but silent this past week. But that shouldn’t come as a surprise to anyone either. It’s a public company, and, as such, required to comply with regulatory requirements governing communications with the public. According to Nasdaq, Twitter is expected to report earnings on April 28. So they’re in quiet period, and, as will surprise no one, won't talk about confidential negotiations between parties.
As a privately-held company, however, DataSift has more leeway to comment publicly. I’m not going to repeat their position here; it is clearly stated in several posts on the DataSift blog.
But none of this gets at the most important issue: the impact of this decision on customers and users of Twitter data. Here are a few constituencies to consider:
1. Companies that sell social technology
To gauge the impact, you need to consider how these companies gained access to Twitter data before now:
|How they get Twitter data||Impact||Implications|
|Directly from Twitter, via firehose or public API||No change||None|
|From Gnip (and therefore now from Twitter)||No change||None|
|From third-party resellers such as DataSift||Ends August 13, 2015||Must re-assess how they migrate from DataSift to Twitter|
But this requires a bit of context.
Before the acquisition, there was a reason companies—social technology or enterprise—would select Gnip or DataSift (or, before its acquisition by Apple, Topsy) if they wanted direct access to Twitter data: they had different value propositions.
DataSift positioned itself as a platform in which the access to social and other types of data came with value-adds such as advanced filtering, enrichments, taxonomies, machine-learning and other sophisticated data processing capabilities to enable users to derive insights from the data.
Gnip, on the other hand, was a simpler, less expensive option: they offered enrichments, but the value proposition was historically more about simplicity and reliability than sophisticated data processing. This tended to be an easy calculation for a lot of social tech companies who wanted to add their own capabilities to the data.
So, speaking broadly, analytics or social technology companies (even brands) who could or wanted to handle raw data would have been better suited to Gnip. Those who wanted a more plug-and-play product that processed data consistently across multiple sources would more likely favor DataSift. Once Twitter acquired Gnip, it didn’t take a team of data scientists to conclude that Twitter had bigger plans for its data business, and that a lot of that development would happen under the waterline, as these things tend to do.
But that doesn't eliminate the very real issue that migration is going to be highly disruptive for customers of DataSift.
However, there is another data point that’s important to consider.
Unlike most industry shifts, it’s been very difficult to get any social analytics companies to talk on the record about this news. On background, some stated that they were never quite sure whether DataSift intended to be a partner or a competitor because they weren’t a pure reseller; the platform—with its ability to perform sentiment analysis, enrichments, and provide classifiers and taxonomies—pushed it uncomfortably into analytics territory for some.
Some said they’re concerned about Twitter’s plans as well. Now that Twitter has discontinued data licensing, what will it now monetize? Will they take more control of or develop their own analytics? If not, what then?
This is unsettling for some in the social analytics community, who are also being buffeted by business intelligence and marketing/enterprise cloud companies (think Oracle, Salesforce, Adobe) eager to wrap social data into a broader insight offering. It's a time of shifting strategies and shifting alliances.
2. End users of technology (brands and enterprise)
For the most part, end users of Twitter data don’t have much to worry about, unless they are current or potential DataSift customers and can’t (or don’t want to) ingest the firehose in its raw form. If they are, they’ll need to migrate to Twitter, and assess the extent to which Twitter is currently (or via roadmap) able and willing to provide the type of processing they need.
If enterprises get their Twitter and other social data from social analytics providers, they are more insulated from this news. The question I would ask is whether and how Twitter intends to normalize data from other social media platforms. Will users have a clear sightline across multiple social (and other) data sources? Will they analyze more than Twitter data? Will they handle that through partnerships (and if so, with whom?) The ability to normalize data across sources has been a clear value proposition for DataSift; less so from Gnip, especially since the acquisition by Twitter. And of course we can't discount the fact that Twitter likely has more up its sleeve than its able to disclose right now.
3. Agencies, consultancies and other professional services firms
Agencies can be affected in any number of ways, based upon their social (and other) analytics strategy and business model. Those who offer their own bespoke analytics would do well to learn more about Twitter's data roadmap; how much will be product and how much service. Those who use a mix of analytics tools would likely be less affected.
As for professional services firms, there is a tremendous amount of opportunity in custom analytics for enterprise. The challenge is that 1) data processing isn't core to that business 2) developing custom analytics doesn't scale well and 3) Twitter data, especially in the context of other social, enterprise and external data, is extremely complex. As a result, professional services firms will need to approach Twitter to better understand what the company will and won't offer in the future and where the synergies lie. Either way, it's going to be a delicate dance.
For all DataSift customers, Tim Barker’s post is a shot across the bow for Twitter; even if Twitter disagrees with his assessment of the impact of terminating the reseller agreement, it’s a starting point for customers to begin their conversations with Twitter and suss out what exactly a shift to a direct relationship might entail.
One other option is for customers to bring Twitter data into DataSift via a Gnip connector. DataSift is working on that now.
A few last thoughts
In the end, a lot is still unknown, and Twitter’s silence is enabling people to fill the void with speculation, which creates uncertainty and doubt among those whose businesses depend on to some extent on Twitter. But in my opinion, all of this is an inevitable if painful blip for both companies: DataSift will move on, and Twitter will continue to build out its data business, which will likely create even more uncertainty in the social data ecosystem for some time to come.
But, as one person who spoke off the record rather philosophically commented, “In a world where you’re dealing with third-party data, you can never be completely comfortable.”
Other points of view
The implications of Twitter turning off the tap for one of its biggest data partners.
Why the competition between the big marketing cloud vendors shouldn’t be the focus of their clients.
A look at the digital ethics and privacy conversations from this year’s SXSW conference.
A new law banning the collection of personal information in South Africa could influence legislation in other countries as well.
This year’s results have troubling implications for the technology industry.
A look at what we give up and gain when we allow our lives to be turned into sources for data.
Here are five data questions about the Super Bowl that we’d like the answer to.
Highlights of what the Big Boulder Initiative accomplished in 2014, and its plans for the new year.
This document is just a first step toward setting context for the many disruptions of ubiquitous and complex data, but it includes preliminary frameworks to help us examine these issues in more detail.
In my last post, I discussed some themes for 2015, one of which was an imperative for us as an industry to get serious about digital ethics.
I’m not generally a fan of annual predictions; they always remind me of a carnival in which you’re encouraged to “pay no attention to the man behind the curtain”; you almost never win the giant teddy bear.
During the past several years, the television industry has changed dramatically, spurred by device proliferation, changing distribution methods, and the increasing popularity of social media.
By now, you’ve probably heard that data scientists at Facebook recently published a study in The Proceedings of the National Academy of Science…
Late last year, I started wondering about social media command centers. Salesforce had launched one, as had Brandwatch, but I wondered: were they really still relevant? Were companies investing in command center deployments, or had interest subsided since their heyday in 2010?
In the past year, social data has continued to wend its way into organizations of all types, from large enterprise to small business to media and entertainment and the public sector. We’ve seen use cases far past marketing into product and service quality, entertainment programming, customer service, fraud detection and a host of other examples.
I spend a lot of time reading and thinking about social data: what it is, what it isn’t, how to measure it, where it’s going.
It’s a nightmare scenario. You get a frantic text or call from a co-worker that someone tweeted a tasteless joke or profanity from your corporate Twitter account.
Everyone talks about the challenges of measuring the revenue impact of social media, but how are top brands actually doing it? And are they successfully measuring ROI?
The run-up to Facebook’s IPO reminds me a bit of a wedding: everyone’s attention is on the big day (expected to be Friday May 18), without much regard for the weeks, months and years afterward.
Even though the topic of social media ROI may sometimes seem like an endless game of Whack-a-Mole [see previous post], there’s plenty of evidence to suggest we’re inching ever closer to accountability.
Wherever I go, the question I hear most often is this: “What is the ROI of social media?” Even though most companies we’ve surveyed have a brand monitoring solution in place, few have yet to crack the measurement code. It remains one of the most stubborn challenges for the social business.