WebProNews

Tag: realtime search

  • Google Algorithm Update (‘Colossus’ or ‘News-Wave’) Google Trends-Related?

    Google Algorithm Update (‘Colossus’ or ‘News-Wave’) Google Trends-Related?

    On June 17, Google’s algorithm seemed to get a mysterious jolt with a mysterious update that Google deemed a routine, non-major change. Google specifically said the update was not related to Panda, Penguin, or HTTPS. They wouldn’t comment further.

    They just said, “We’re always making improvements to our search algorithms and the web is constantly evolving. We’re going to continue to work on improvements across the board.”

    According to MozCast, which measures the “temperature” of patterns of the Google algorithm, the update came in at 101.8°F. For comparison, the first Penguin update only registered at 93.1°.

    Since initial reports on the update, Searchmetrics has analyzed it a bit, and found that news sites are benefiting most, and that it appears to be related to trending keywords and real time hot topics. Top winners, according to the company, were WSJ.com, USAToday.com, Dailymail.co.uk, BusinessInsider.com, Time.com, NBCNews.com, LATimes.com, NYPost.com, TechCrunch.com,FoxNews.com, Steampowered.com, BizJournals.com, TheVerge.com, Fortune.com, Gizmodo.com, Dict.cc, HollywoodLife.com, TechTarget.com, and WindowsPhone.com.

    Most of these sites publish “fresh and newsworthy” content on a regular basis, as Searchmetrics notes.

    The update coincides with a major refresh of Google Trends, which now provides data in real time, and takes into account trends on YouTube and Google News. Google hasn’t confirmed the connection here, but it seems like the most likely explanation at this point.

    As Searchmetrics notes, Google also has the Twitter fire hose now, so that’s more real time data it can use. It’s unclear whether or not that’s connected. The only use of this data that has actually been announced comes in the form of tweets appearing in Google’s mobile results, but it’s probably safe to say that Google can tap into this for other reasons, that have bigger implications than that specific feature.

    When the update was spotted by Moz, they dubbed it the Colossus update. Searchmetrics is calling it the seemingly more fitting “News-wave” update. I don’t know if either name will stick, but it does appear that freshness is once again a major priority of the Google algorithm. This has been taken too far by Google in the past, in my opinion, so we’ll see how it goes this time.

    Interestingly, Searchmetrics reports that Wikipedia has seen a bit of drop in SEO visibility as a result of the update. There was some speculation initially that the shakeup in tools like Mozcast was related to Wikipedia switching to HTTPS. Since Wikipedia is usually the top result for many pages, it any changes with the site could significantly change SERPs in general. According to Searchmetrics, however, Wikipedia’s placement has dropped a little due to news sites ranking for some terms.

  • Google Trends Goes Real-Time

    Google just announced its biggest update to Google Trends since 2012 when it merged with Insights for Search. The new version provides users with real-time data about trending searches ranked in order of popularity.

    On the homepage, there’s a ranked, real-time list of trending stories, which takes into account not only Google searches, but YouTube and Google News searches as well. It combines all of this to give you a better sense of what’s trending on the web.

    The new Google Trends homepage

    “You can now find real-time data on everything from the FIFA scandal to Donald Trump’s presidential campaign kick-off, and get a sense of what stories people are searching for,” says Nimrod Tamir from the Google Trends Team. “Many of these changes are based on feedback we’ve collected through conversations with hundreds of journalists and others around the world—so whether you’re a reporter, a researcher, or an armchair trend-tracker, the new site gives you a faster, deeper and more comprehensive view of our world through the lens of Google Search.”

    “You can now explore minute-by-minute, real-time data behind the more than 100 billion searches that take place on Google every month, getting deeper into the topics you care about,” he adds. “During major events like the Oscars or the NBA Finals, you’ll be able to track the stories most people are searching for and where in the world interest is peaking. Explore this data by selecting any time range in the last week from the date picker.”

    Google says it has expanded its breadth of coverage to include data for more niche topics isn smaller geographies.

    Google trends data

    In addition to the new Google Trends, Google is posting data sets on specific topics on Github with the Google Trends Datastore.

    trends4

    From there, you can download data sets listed by date, coverage, and subject. Current ones include sets about the Denmark elections and NBA finals.

    The redesigned Google Trends homepage is launching in 28 countries with more on the way.

    Images via Google

  • Is This How Google Is Going To Use Tweets?

    Is This How Google Is Going To Use Tweets?

    It looks like Google’s new Twitter integration might be starting to take effect, with one possible format seen in the wild on Android.

    Are you looking forward to Google utilizing Twitter more? Discuss.

    Earlier this year, it came to light that Google and Twitter struck a new deal that would see Google indexing tweets in real time and that tweets would become more visible in Google’s search results as soon as they’re posted. During Twitter’s earnings conference call last week, CEO Dick Costolo said that the new integration would roll out in May. That was as specific as he got. It looks like some users may already be seeing some of that integration.

    Read: Google’s Twitter Update Is About To Hit: Here’s What Businesses Should Be Thinking About

    Search Engine Land has a screenshot of what appears to be an eight-minute old tweet from Gary Valenciano about the Mayweather/Pacquiao fight, which was apparently surfaced through a related search. It appears under a CNN result and its accompanying “more news for mayweather pacquiao” option. The Twitter result comes with a “feedback” link, indicating that the feature is a work in progress. There’s also a screenshot with a tweet fro Steve Benfrey in results for Carly Fiorina.

    Barry Schwartz, who was tipped with the screenshot by Razorfish’s Dan Martin, writes, “He told me he sees it happening on and off on his Android device for many trending topics right now, such as [maythe4thbewithyou], [Carly Fiorina], [Jimmy Greaves] and other searches. I personally cannot replicate this on any of my devices, but it would not surprise me if Google is now testing this and may roll it out any day.”

    Trying these searches on my own Android device, I’m not seeing the style shown in the screenshot. I did get this for “maythe4thbewithyou,” but that appears to be more app indexing related:

    As you may know, Google also recently turned app indexing into a ranking signal (and even expanded that signal further a couple weeks ago).

    Read: Google Does More With Recently Added Ranking Signal

    The above screenshot actually illustrates how that particular update might also help Twitter, who made the Google deal to drive more traffic to its service, which has been struggling to grow its user base fast enough to please investors. Even if Google doesn’t show sufficient real time tweets as a result of its Twitter integration, it may still point users to fresh Twitter content by way of the Twitter app.

    It’s unclear if anyone is seeing the new Twitter integration on non-Android devices, and to what extent the feature is being seen by users at all.

    Asked about the integration, a Google spokesperson told WebProNews, “As mentioned in Twitter’s earnings call back in January, we’ve partnered to bring their real-time content to Google Search. We’ve started some small experiments in mobile search, but we don’t have more to announce at this point.”

    They didn’t elaborate beyond that. Is this the extent of what the deal is going to bring to the table, or is this just one of multiple possible integrations? My guess is the latter.

    Beyond integrations directly into the search results, it’s possible that Google will also be getting more data from the tweets when it comes to ranking the content that they point to.

    We recently looked at some findings from Stone Temple Consulting, which analyzed Twitter’s past use of tweets, and found links in tweets having a significant impact.

    “Google still loves links. 26% of the tweets with an inbound link from sites other than Twitter got indexed. That is nearly 4 times as much as the overall average rate of indexation,” said Eric Enge in the report, adding that link quantity correlates highly with a tweet getting indexed.

    They found that out of 21 accounts and 91 tweets with with over 100 inbound links, 46% were indexed. The number goes down the less inbound links there are. Those with less than ten links only saw a 7% index rate.

    We later had a discussion with Enge, who said, “As you may know, there are many sites out there that simply replicate lots of tweets on their sites. I am not sure what value they serve, or if any people actually visit such sites. But, some of the links tweets get come from such sites, and my bet is that Google ignores those.”

    “However, there are other sites that may reference tweets within a blog post or article, and link in a clean traditional web link based fashion to the URL for the tweet itself…it is these links that I believe that Google is placing a high value on.”

    It’s also possible that having access to Twitter’s firehose will enable Google to better rank fresh articles based on the Twitter activity they get. They’ll at least be able to look at that as a more significant signal with the real time element coming into play.

    Google completed the roll-out of its big mobile-friendly algorithm update late last week. At the time, the company indicated that it wasn’t finished indexing everything. Google also recently indicated that its doorway page update had completed its roll out.

    Do you see the Twitter integration as a positive for Google results? Share your thoughts in the comments.

    Note: This article has been updated from its original form.

  • Google’s Twitter Update Is About To Hit

    Google’s Twitter Update Is About To Hit

    Long story short, what you say and what others say about you on Twitter may start mattering a lot more.

    Do you expect to benefit from Google’s coming Twitter integration? Let us know in the comments.

    We learned in February that Google and Twitter struck a new deal that will see Google indexing tweets in real time. Initial reports indicated that tweets would become visible in Google’s search results as soon as they’re posted, starting in the first half of this year. Now, we’ve got a more specific timeframe.

    In a conference call to discuss Twitter’s earnings, CEO Dick Costolo revealed that the integration will begin its roll-out next month. Given that April is just about over, that could potentially be as early as this week.

    “I don’t have a specific date for you, but now I can at least give you a specific month,” said Costolo. “And that Google deal is all about…that relationship is about driving our total audience strategy. The goal is that people consume content and engage with that content whether they log in or not.”

    Twitter Needs More Eyeballs

    Since going public in 2013, Twitter has faced immense pressure to grow its user base, and that growth has been quite slow. While the company did manage to add 14 million users over the last quarter, it only just surpassed its 300 million monthly active user milestone. For comparison, Facebook just reported 1.44 billion. Twitter has been exploring different ways to grow its MAUs, and much of that has been focused on improving the experience for logged out users.

    Earlier this month, Twitter launched a homepage redesign. For logged in users, the homepage remains the user’s home timeline, but for logged out users, it’s now a user-friendly directory of categorized content. Twitter has for all intents and purposes turned its logged-out homepage into a news site.

    The Google deal is another way (and likely a pretty significant one) to get people to realize more value from the service even if they’ve not been frequent users in the past. The narrative around Twitter’s stock every time the company reports its earnings is always shareholder disappointment, so the importance of the Google deal cannot be overstated from Twitter’s corporate perspective.

    The two companies, as you may know, used to have a similar relationship, but when the original deal expired, the companies were unable to reach an agreement to keep it going.

    “I would say that the way we think about the Google deal now without again — without going into any of the details distinct from the kind of relationship we had in the past is that we’ve got the opportunity now to drive a lot of attention to an aggregate eye balls if you will to these logged out experiences, topics and events that we plan on delivering on the front page of Twitter,” Costolo said on a previous earnings call ahead of the new homepage roll-out. “And that’s one of the reasons this makes a lot more sense for us now.”

    What’s in store?

    Little is known about why exactly that relationship fell apart, but the most logical explanation is that the pre-public Twitter thought it was worth more than what Google was willing to pay for access to Twitter’s firehose of realtime tweets. Google had even built a feature for its search results pages -Realtime Search – around the deal. It used to display a box of scrolling realtime results, which included tweets as well as content from other sources, on search results pages for newsy queries.

    Once the Twitter deal went away, so did the feature, making it clear that the tweets were the only real valuable part of that. This also illustrated how valuable the deal was for Google, as its absence highlighted a failure of Google’s stated mission to organize the world’s information and make it universally accessible. With so much of the world’s information now flooding the internet in real time, Google could hardly make good on that mission without the access it once had.

    Google isn’t expected to implement tweets the same way it used to when the new integration goes into effect.

    “I don’t think that this is what Google is looking for,” search marketer Eric Enge recently told WebProNews. “I suspect that the UI impact will be minimal, but that more tweets will get indexed. However (and this is a big however), what will really be interesting to see is if Google uses tweet data to help drive personalization in one fashion or another. One simple way to do this? Simply favor content that people link to from their tweets in future related search results.”

    “This type of prioritization is similar to what they do with Google+ already,” he added. “This is just speculation on my part, but I think it could be a huge win for Google if this deal gives them enough visibility to allow them to do that.”

    We’ll see if either company makes a big announcement about how Google will handle tweets or if we’ll just start seeing the tweets surface more.

    How Google has been using tweets

    Earlier this year, Enge and his firm Stone Temple Consulting released some findings about how Google indexes tweets currently, which provides some insight into how things may change when the new deal goes into effect. His team analyzed over 133,000 tweets to see how Google indexed them, and found that about 7.4% of them were actually indexed, leaving 92.6% completely left out of the search engine.

    The findings suggested that Twitter accounts with larger follower counts are getting more tweets indexed, though it may be only a correlation. Enge said he doesn’t think Google is looking specifically at follower count, but that other signals are affecting which profiles get indexed more (i.e. links to those accounts’ profiles). Either way, he noted, more value is clearly being placed on the authoritative accounts.

    Out of the accounts with over a million followers that the research looked at, there were 13,435 tweets with 21% of them being indexed by Google. Out of 44,318 tweets in the 10K to 1M follower range, only 10% were indexed. For 80,842 tweets from accounts with less than 10,000 followers, just 4% were indexed.

    Stone Temple said images and/or hashtags seem to increase a tweet’s chances of getting indexed with percentages registering higher than average. Mentions, on the other hand, register negatively. It also points to another of its studies, which showed that links from third-party sites have a significant impact.

    “Google still loves links. 26% of the tweets with an inbound link from sites other than Twitter got indexed. That is nearly 4 times as much as the overall average rate of indexation,” Enge said in the report, adding that link quantity correlates highly with a tweet getting indexed.

    They found that out of 21 accounts and 91 tweets with with over 100 inbound links, 46% were indexed. The number goes down the less inbound links there are. Those with less than ten links only saw a 7% index rate.

    “What our study showed is that Google currently places minimal impact on freshness of tweets today,” Enge told WebProNews in February. “Perhaps when crawling needs to be done to discover them it’s just not worth it, and it might be that the new deal will change that. However, I suspect that it’s not the tweets themselves that Google really values the most, but the content they link to that Google wants to discover more quickly. That said, if they see a tweet getting major engagement, chances probably would go up that this tweet will show up higher in the results.”

    Your reputation on the line?

    Businesses may have reputation-related issues to be concerned about when the deal takes effect.

    “The biggest challenge and opportunity for businesses using Twitter for customer service is that every interaction is now amplified,” Conversocial CEO Joshua March told us last month. “Whether that’s a complaint from a customer or the company’s response, the agreement between Google and Twitter places a greater spotlight on each interaction.”

    “When a customer is searching on Google for a business, Tweets from customers about issues or bad service experiences could be on the front page,” he said. “If businesses have a social first approach to customer service then they can tackle these quickly and head on, creating positive engagements that will show up instead. This deal has the potential to accelerate the kind of service-related Twitter crises many brands have already experienced.”

    The key word there is potential. Until we see Google’s approach to the integration, any of this is only speculative. However, these are important points for businesses to keep in mind as the integration approaches.

    “For companies with a social first approach who are committed to delivering excellent, fast and authentic social customer service, the agreement between Google and Twitter has the ability to spotlight them, and make it very obvious to customers that they care,” said March. “Companies that have successfully integrated various social media into their customer service DNA should be very excited by the agreement. In addition, previously addressed concerns are now searchable, allowing customers to potentially self-help.”

    Businesses are going to have to consider that any tweets related to their brands could become more visible, as could tweets from employees. In fact, even beyond the Google deal, Twitter is doing other things that could inflate visibility through search.

    This goes beyond Google

    Google already has a firehose deal in place with Bing.

    Twitter is also working with Apple. During Tuesday’s conference call, Costolo said, “And finally, we are also working with Apple to surface great Twitter content and accounts directly in Spotlight Search on iOS and OS X, that also makes it easier and quicker to find great things on Twitter. So I would sum up by saying, there is absolutely an opportunity to go and monetize that attention and traffic. We want to make sure we iterate on the experiences to get them right first.”

    Update: Apple’s spotlight search is reportedly already surfacing Twitter content.

    Twitter clearly wants (and needs) to have its content surfaced as much as possible, so look for it to find other partnerships to fuel this as well.

    The company is also experimenting with its own search interface, which could emphasize the power of that to more people as it continues to court and retain new users. We recently looked at a redesign it’s been testing, and while Twitter would not confirm that it will roll out to all users, it’s clearly an improvement from the existing interface, so I would be shocked if it doesn’t.

    There’s not much change in terms of functionality, but the way things are laid out and labeled are significantly improved, and could draw some new attention to Twitter Search. For a deeper dive into this interface, read this.

    Twitter also recently started indexing every public tweet from the last 8 years, so there’s that too.

    Are you looking forward to the Google/Twitter deal taking effect? How should Google approach the data? Discuss.

    Images via Wikimedia Commons, Twitter

  • Facebook Is Messing Around With Its Trending Section

    Facebook is testing some changes to its Trending feature, which for some has become a staple of the Facebook experience since it launched on desktop early last year.

    In December, Facebook introduced some big changes, adding new sections to split up the types of content that users can peruse for each trend. These include Articles, In the Story, Friends and Groups, Near the Scene, and Live Feed. It also launched the feature on mobile devices.

    With the new test, Facebook is trying out some additional categorization – this time by actual topic. AdWeek shows the trending box displaying a few trends, with the ability to “see more” followed by categories like Politics, Business, Science and Technology, Sports, and Entertainment. Clicking any of these topics will reportedly show you a few trends from that category.

    When Facebook first added its trending feature, it was largely looked at as the latest (at the time) attempt to add more Twittereque flavor to the social network. Facebook has historically been more about personal social networking, but in recent years it has become much better for public information. The Trending section has certainly helped in that regard, not to mention the trending stories it inserts into the News Feed.

    Twitter itself has been messing around with its trends as well, electing to add some context.

    “We know that trends aren’t always self-explanatory, so now you’ll see a description below each trend,” the company said earlier this week. “Since trends tend to be abbreviations without context, like #NYFW, a description will make it clear that this trend is about New York Fashion Week. The new trends experience may also include how many Tweets have been sent and whether a topic is trending up or down.”

    That’s in addition to Twitter killing the discover tab and experimenting with how it lets users surface public information with search.

  • Twitter Confirms It’s Messing With Its Search Experience

    Twitter Confirms It’s Messing With Its Search Experience

    While covering Google’s closure of Google Moderator earlier, I searched Twitter on the desktop to see what people were saying about it (not much so far), and suddenly, just for a minute or two, Twitter was serving me a completely different search interface. Then, it was nowhere to be found. It actually went away before I left the page.

    Here’s what it looked like:

    I’m not the only one who noticed it:

    After seeing these tweets, I checked Safari (I was using Chrome), and I’m not getting it there either. I’ve reached out to Twitter for more information, and will update accordingly.

    As you can see from the screencaps, it lets you filter search results by the top ones, live results, accounts, photos, videos, news, those from everyone, those from people you follow, those from everywhere, and those near you. It also lets you easily save the search, embed it, or go to advanced search.

    Clicking around through the different options, I couldn’t get any of them to actually work. They all just directed me to the regular search results pages, showing results for “everything”. It was clear that the new interface wasn’t quite ready for primetime.

    What’s not so clear is if this is going to be rolled out to all users, or if it’s just a test. Hopefully, it’s the former, because this would be a huge improvement, and make Twitter better as a search tool. That combined with the fact that it now indexes every public tweet since Twitter began, make it a great resource for finding information. It should also keep Twitter itself as the best option for finding tweets even as it partners with Google.

    Update: It’s come back and gone away again.

    Update 2: A Twitter spokesperson tells us, “I can confirm that we are experimenting with a new search experience.”

  • What Google’s Twitter Deal Means For You

    What Google’s Twitter Deal Means For You

    News came out last week that Google and Twitter have struck a new deal to put real-time tweets back into Google’s search index. The companies aren’t providing much in the way of details about the deal at this point, and it’s possible that they never will, but they did confirm the deal, and indicate that it will go into effect in a few months.

    Do you expect to benefit from the deal? Tell us what you think about it in the comments.

    Years ago, when the two companies had a similar relationship, Google had a search feature called Realtime Search, which displayed a set of scrolling results at the top of the search results page on some queries (typically newsy ones). The feature didn’t rely solely on Twitter. It incorporated other sources, but it was clear that Twitter was the one that really mattered, especially when the whole feature went away upon the expiration of the companies’ initial deal.

    Ever since that fell apart, Google has been lacking in the real-time department. In the early days of Google+, it seemed like Google thought it might be able to replace Twitter with its own real-time content, but obviously that never materialized to the extent of what Twitter has to offer. Meanwhile, Google would continue to index tweets in its regular search results, but it would never be able to index them in real time, and the ones it did index would only be a small percentage of the larger tweet pool.

    Eric Enge’s Stone Temple Consulting released some new findings about how Google indexes tweets currently, which provides some insight into how things may change when the new deal goes into effect. His team analyzed over 133,000 tweets to see how Google indexed them, and found that about 7.4% of them were actually indexed, leaving 92.6% completely left out of the search engine.

    That tells us a great deal right there. Google’s mission is to “organize the world’s information and make it universally accessible and useful.” As we’ve discussed in the past, Google is essentially failing that mission without Twitter’s firehose. Today, the world’s information is coming at us in extremely rapid fashion, and as far as public information goes (Facebook is working to do more with the non-public stuff), Twitter is the best provider of that rapid-fire info. How can Google possibly succeed in its stated mission if it’s only organizing a little over 7% of that information?

    Stone Temple’s findings suggest that Twitter accounts with larger follower counts are getting more tweets indexed, though it may be only a correlation. Enge says he doesn’t think Google is looking specifically at follower count, but that other signals are affecting which profiles get indexed more (i.e. links to those accounts’ profiles). Either way, he notes, more value is clearly being placed on the authoritative accounts.

    Out of the accounts with over a million followers that the research looks at, there were 13,435 tweets with 21% of them being indexed by Google. Out of 44,318 tweets in the 10K to 1M follower range, only 10% were indexed. For 80,842 tweets from accounts with less than 10,000 followers, just 4% were indexed.

    Stone Temple says images and/or hashtags seem to increase a tweet’s chances of getting indexed with percentages registering higher than average. Mentions, on the other hand, register negatively. It also points to another of its studies, which showed that links from third-party sites have a significant impact.

    “Google still loves links. 26% of the tweets with an inbound link from sites other than Twitter got indexed. That is nearly 4 times as much as the overall average rate of indexation,” Enge says in the report, adding that link quantity correlates highly with a tweet getting indexed.

    They found that out of 21 accounts and 91 tweets with with over 100 inbound links, 46% were indexed. The number goes down the less inbound links there are. Those with less than ten links only saw a 7% index rate.

    Be sure to check out the research for additional findings.

    Following the release of this research, we did a Q&A with Enge:

    Do you think Google will re-implement the kind of real-time scrolling results feature at the top of search results like it used to have with its old Twitter deal?

    Enge: Not really, I don’t think that this is what Google is looking for. I suspect that the UI impact will be minimal, but that more tweets will get indexed. However (and this is a big however), what will really be interesting to see is if Google uses tweet data to help drive personalization in one fashion or another. One simple way to do this? Simply favor content that people link to from their tweets in future related search results.

    This type of prioritization is similar to what they do with Google+ already. This is just speculation on my part, but I think it could be a huge win for Google if this deal gives them enough visibility to allow them to do that.

    Under the deal, do you think we’ll see a lot more brand new tweets appearing HIGH in search results? Do you expect the freshness of a tweet to be heavily factored into Google’s ranking signals when indexing tweets?

    Enge: Great question. What our study showed is that Google currently places minimal impact on freshness of tweets today. Perhaps when crawling needs to be done to discover them it’s just not worth it, and it might be that the new deal will change that. However, I suspect that it’s not the tweets themselves that Google really values the most, but the content they link to that Google wants to discover more quickly. That said, if they see a tweet getting major engagement, chances probably would go up that this tweet will show up higher in the results.

    The study suggests that tweets with images and/or hashtags have a better shot at getting indexed, and those with mentions have less of a shot. It’s acknowledged that this may or may not be simply a correlation. What does your gut tell you?

    Enge: I think it’s real. Bear in mind that the study we published in December on Twitter engagement also shows that images and hashtags have a positive impact on user engagement. This means that people see them as more valuable, and Google wants to place more value on the content that users value the most. So, my gut tells me that this is actually a causal situation, not just a correlation.

    When the study is talking about the impact of 3rd party sites linking to tweets as something Google likes, are we talking primarily about tweets that are being embedded on these sites, just plain old links, or a combination of the two?

    Enge: As you may know, there are many sites out there that simply replicate lots of tweets on their sites. I am not sure what value they serve, or if any people actually visit such sites. But, some of the links tweets get come from such sites, and my bet is that Google ignores those.

    However, there are other sites that may reference tweets within a blog post or article, and link in a clean traditional web link based fashion to the URL for the tweet itself (what you referred to as “plain old links”). It is these links that I believe that Google is placing a high value on.

    How do you expect Google to react to promoted tweets? Let’s say Google indexes your tweet when it’s organic, but then you decide to promote it? At that point, Google is basically indexing an ad. Will Google shy away from indexing promoted tweets altogether?

    Enge: If a promoted tweet gets a ton of engagement, as well as external links, I think that it might still get indexed and rank, but I’d expect that the threshold will be higher than it is for organic tweets. I don’t have any science for that answer, but it is my sense as to how they will treat it.

    Your site uses “tweetable quotes” throughout its content. Has this been particularly effective for increasing Twitter traffic? Have you measured this specifically or are you familiar with any studies that have?

    Enge: Mark Traphagen pushed us into doing this, and makes sure all of our posts include these. He also tracks it very closely. Within 5 hours of the Twitter indexing study going live today, 67 people have already used the click to tweet boxes to generate tweets, and this has driven 207 unique clicks to the article. Pretty valuable I’d say!

    Would you recommend sites use this more in light of the Google deal?

    Enge: Yes! People do respond to the click to tweet boxes and that helps us get more tweet-love for our articles, and more visits. We use ClickToTweet.com for this, but there are other good services out there. Note that to make this look nice, Mark figured out a process to take the ClickToTweet link and embed it in an image as well.

    All great stuff to know. Enge gives us some incredibly valuable insight as usual. Again, don’t forget to check out Stone Temple’s study.

    Are you looking forward to seeing Google indexing Tweets in real time again? Let us know in the comments.

    Image via StoneTemple.com

  • Twitter Talks Google Deal, Says User Growth Trend Already Turned Around This Year

    Twitter Talks Google Deal, Says User Growth Trend Already Turned Around This Year

    Twitter reported its Q4 and fiscal year 2014 financial results on Thursday, with 97% year-over-year revenue growth for the quarter at $479 million. Ad revenue per thousand timeline views reached $2.37 in the quarter, which was up 60% year-over-year. The company managed to impress with its business, but continued to disappoint in the user growth department, which is an area the company has been heavily criticized over since going public.

    Average Monthly Active Users (MAUs) were 288 million for the fourth quarter, which was an increase of 20% year-over-year. The company noted that this reflects a loss of about 4 million net Monthly Active Users in the quarter due to changes in third party integrations (specifically iOS 8). Users grew at a rate of just 1.4 for the quarter compared to 4.8% the prior quarter. Average Mobile MAUs were 80% of total MAUs. Timeline views reached 182 billion for the quarter (up 23% year-over-year). Twitter sees 6,000 tweets per minute every day, according to the company.

    All in all, Twitter added 4 million users during the quarter and 47 million throughout 2014. CEO Dick Costolo said on the earnings call that user numbers the company saw in January of this year indicate its MAU trend has already been turned around. He indicated that new Twitter features like native video, group messaging, and Instant Timeline (which populates the timelines of new users) should contribute to an upward trend. He also mentioned the company’s recent acquisition of ZipDial, which he said it will use to bring more content (like key moments and commentary from the Cricket World Cup) to a much larger audience on Twitter.

    Twitter is also doing a lot to build its developer ecosystem, as Costolo also brought up. This should lead to new apps and services that can contribute to user growth. Last fall, the company unveiled its new Fabric developer kits and Digits sign-in. This year, the company has already embarked on a developer tour called Flock, which will see the company helping developers build apps.

    Costolo said during the Q&A portion of Twitter’s earnings conference call that he could confirm that Twitter has entered into an agreement with Google, but declined to elaborate on any more details about it. In the past, the two companies have worked together to show tweets in real time in Google’s search results. From the sound of it this is what’s going to happen again, though it remains to be seen if it will function the same way. Either way, it’s going to get tweets in front of people more often, and that can’t hurt user growth.

    During the call, Costolo was asked why a Google deal didn’t make sense any longer when the two companies grew apart a few years ago, and why it makes more sense now. His answer (via Seeking Alpha) was as follows:

    We’ve obviously had a relationship with Google over the course of the years with all the bunch of the executives here and a bunch of the executives there obviously know each other quite well.

    I would say that the way we think about the Google deal now without again — without going into any of the details distinct from the kind of relationship we had in the past is that we’ve got the opportunity now to drive a lot of attention to an aggregate eye balls if you will to these logged out experiences, topics and events that we plan on delivering on the front page of Twitter. And that’s one of the reasons this makes a lot more sense for us now.

    He also made clear that we won’t see a launch from the deal for several months.

    As one analyst mentioned, the Google deal should have an impact on Twitter’s revenue in terms of licensing, but the company declined to discuss that for the time being.

    Regarding the iOS 8 changes that impacted Twitter’s user base, CFO Anthony Noto said:

    So we said we lost four million monthly active users due to the iOS8 integration. One million of those monthly active users were Twitter owned and operated monthly active users and three million were on Safari, what we call Auto point MAU’s and we lost those.

    We don’t expect to get the three million Autopoint MAU’s in Safari back and that’s a non-Twitter owned and operated Autopoint MAU. The one million that number was actually higher at a different point in the quarter when we were able to bring it back down to just one.

    Costolo added:

    We obviously have a great relationship with Apple. I’ve talked about that at length over the course of the last two years. On the second part of what Anthony talked about there, there was unforeseen bug in the release of iOS8 as it relates to the specific Twitter integration into iOS that’s why it was particular to us. Once we understood the issue, we move just quickly as we could on multiple fronts to minimize its impact, but it wasn’t — it wasn’t a one size fits off fix, which is why you’ve seen some of the complexity that we talked about here in brining those users back. The problem was a complex and affected different users differently.

    The Wall Street Journal reports that while hitting a record low, Twitter’s user growth fell below Facebook’s for the first time. According to Re/code, Twitter has revoked access to user growth numbers from its employees.

    Image via Twitter

  • Google And Twitter Partner On Search Again

    Google and Twitter have finally reached a new deal that will see Google indexing tweets in real time, according to a report from Bloomberg (and apparently confirmed by Re/code). The deal hasn’t been formally announced, and the terms are unknown.

    The report says tweets will become visible in Google’s search results as soon as they’re posted starting in the first half of this year. It’s unclear if Google will restore the realtime search feature it used to have when it had access to those tweets in the past. In those days, there was a section at the top of some search results pages reserved for tweets and realtime updates from Twitter (though Twitter provided the bulk of them, and were often the most useful).

    When the two companies failed to renew their deal four years ago, Google ended up dropping its realtime search feature altogether. There was little point to it without Twitter. It also made put a kink in Google’s mission to organize the world’s information and make it universally accessible. The Twitterverse spews out a great deal of information every second, and it’s been a lot more accessible on Twitter itself than on Google.

    In fact, Twitter has taken steps to improve its own search features in the time since the two companies grew apart. As recently as November, Twitter announced that it indexed every public tweet since 2006, making its search engine much more comprehensive in terms of the content it can retrieve.

    “This new infrastructure enables many use cases, providing comprehensive results for entire TV and sports seasons, conferences (#TEDGlobal), industry discussions (#MobilePayments), places, businesses and long-lived hashtag conversations across topics, such as #JapanEarthquake, #Election2012, #ScotlandDecides, #HongKong, #Ferguson and many more,” Twitter’s Yi Zhuang said in a blog post.

    Even before that, Twitter said at an analyst event that it wanted to do more to generate more search traffic. The company revealed that earlier in the year it had made a change allowing Google and other search engines to crawl its top 50,000 hashtagged search pages.

    As far as Google is concerned, Twitter content is largely best as it happens, and that’s what this deal will enable. If you want to know what’s being said by people about something that’s happening right now, you’ll presumably be able to get a good sense of that right from Google rather than having to open up Twitter.

    It will be quite interesting to see how this works with mobile, as mobile search (Google in particular) has come a long way over the past four years.

    Neither company appears to be commenting on the news at this point.

    Twitter’s user growth often comes into question, and investors have grown impatient. This can only help the company in that department.

    Image via Twitter

  • Twitter Search Now Indexes Every Public Tweet From Last 8 Years

    Twitter’s search engine just got a lot more comprehensive. It’s long been a helpful tool for real-time search, but now it’s going to be a lot better at surfacing historic content.

    The company announced on Tuesday that it now indexes every public tweet since 2006. This functionality is rolling out to users over the next few days.

    Twitter’s Yi Zhuang says in a blog post, “This new infrastructure enables many use cases, providing comprehensive results for entire TV and sports seasons, conferences (#TEDGlobal), industry discussions (#MobilePayments), places, businesses and long-lived hashtag conversations across topics, such as #JapanEarthquake, #Election2012, #ScotlandDecides, #HongKong, #Ferguson and many more.”

    Complete results from the full index will appear in the “All” tab of search results on the Twitter web client and on iOS and Android. As time goes on, more tweets from the index will start appearing in the “Top” tab of search results, as well as in new product experiences.

    “The full index is a major infrastructure investment and part of ongoing improvements to the search and discovery experience on Twitter,” says Zhuang. “There is still more exciting work ahead, such as optimizations for smart caching.”

    Twitter takes a super-deep dive into the technical aspects of how the index was accomplished here, so if you’re into back-end, give that a look.

    Image via Twitter

  • Twitter Regretting Not Getting That Google Deal Done?

    Twitter Regretting Not Getting That Google Deal Done?

    Remember the days when you could search for something that was in the news on Google and get a set of realtime search results comprised mainly (but not solely) of updates from Twitter? It was a helpful feature that let you get a glance at what people were saying about a given topic at that moment, and in some cases, provided the absolute most relevant results Google could possibly deliver.

    The realtime search feature went away after Google and Twitter failed to reach an agreement that would have extended the partnership. Twitter continued a similar partnership with Bing, but that just doesn’t get the usage Google does, and it appears now that Twitter may have some regrets about letting that Google deal fall apart.

    This week, Twitter held its Analyst Day event, where it discussed some things it is doing to help promote user growth. One thing it didn’t really go into in its blog post about various projects it’s working on to accomplish that, is trying to get more search traffic. This is something that would obviously come if that deal were still in place. Marketing Land reports:

    During the event this morning, Twitter’s CFO Anthony Noto suggested that Twitter would do more to generate search engine optimization traffic, free traffic from Google and other search engines. It’s something Noto said Twitter hadn’t really done in the past.

    Trevor O’Brien, Twitter’s director of product management, expanded on this later to say that Twitter made a change earlier this year to allow Google and other search engines to crawl its top 50,000 hashtagged search pages, which has generated a 10-fold increase in the number of logged-out people coming to Twitter — helping that figure rise to 75 million per month.

    It’s unclear whether Twitter has approached Google about starting up that deal again, which would actually benefit users to some extent, though it’s hard to say if Google would be interested at this point.

    Truth be told, Google’s feature was never really as helpful for realtime searches as a search directly on Twitter anyway, but again, there are a lot more people using Google in the first place.

    It’s worth noting that Twitter wasn’t a public company that had to worry so much about pleasing investors concerned about user growth when it didn’t renew its deal with Google.

    Image via Twitter

  • ‘Too Many Cooks’ Reminds Us That Facebook Is Really Slow At Rolling Out Graph Search Features

    A lot of people are talking about “Too Many Cooks“. Maybe you’ve heard. It’s been trending on Twitter all weekend and throughout Monday.

    If you’re on Twitter, you can easily see what people are saying about it, from people you follow and celebrities to the common folk of the Twitterverse. A lot has been said, and some fun content is being created around it.

    I’m sure there will be plenty more, but Twitter is going to continue to be a lot better place to search for what people are saying and sharing about Too Many Cooks than Facebook, because Facebook still hasn’t rolled out Graph Search for posts, despite having announced it in September of last year.

    While they said it would be a slow roll-out, this is ridiculous. With Graph Search, you should be able to search for “posts about Too Many Cooks” or “Posts about Too Many Cooks by my friends” or things of that nature.

    Unfortunately, for most users, they still get something like this when they try such a query:

    Facebook is notorious for announcing product updates, and taking FOREVER to roll them out, if they do at all. In the case of Graph Search, one would think it would be a higher priority at the company considering it’s another ideal place to put ads.

    The company may not want to spoil the broth around privacy concerns, as the feature will potentially surface a lot of content that would otherwise be buried. It did say it would be taking feedback from the small amount of users it gave it to at the beginning. Unfortunately, we can only speculate as to what Facebook is doing with it.

    In August, Facebook was found to be letting more people search posts from mobile devices, “testing” a feature it announced nearly a year prior.

    Too Many Cooks is a perfect example of when people want to know what their friends are saying about it, and the fact that I can’t do this on Facebook right now is just a reminder that Facebook is really just lagging on this. It also implies that it could be a pretty huge feature once available.

    Image via YouTube

  • Apple Acquires Topsy, Which Recently Launched A Twitter Search Engine

    Apple has acquired social media analytics firm Topsy Labs for over $200 million, according to a report from the Wall Street Journal, citing “people familiar with the matter”.

    Topsy made headlines a few months ago when it launched a new search engine for Twitter, indexing every public tweet and making them searchable.

    Apple only confirmed the acquisition with its typical canned statement: “Apple buys smaller technology companies from time to time, and we generally do not discuss our purpose or plans.”

    The company has been on something of an acquisition spree lately. Last week, news came out that the company acquired PrimeSense. Other recent acquisitions include Cue (formerly Greplin), AlgoTrim, Matcha and Embark.

    It’s unclear what exactly Apple has in mind with the Topsy acquisition, but one thing worth noting for those interested in the search industry, is that Topsy has access to the Twitter firehose, and can therefore deliver on realtime search. Google no longer does this, and now one of its chief rivals has that search power in its clutches.

    Image: Topsy

  • Twitter And Bing Renew Search Deal

    Twitter And Bing Renew Search Deal

    Bing announced today that it has renewed its search deal with Twitter, which provides tweets in search results for a “near real-time” experience.

    Bing had this to say in a blog post:

    At Bing, we believe that for every search there is somebody out there who can help. That’s why we have been teaming up with Twitter for a few years to surface public tweets from people who may have something to contribute to what you’re trying to get done with Bing.

    Whether it’s a politician, celebrity, thought leader or friend, our renewed partnership with Twitter ensures that you have near real-time access to what people are tweeting tailored to what you’re searching for.

    As you may know, Google used to have a similar deal with Twitter, but that fell apart a few years ago, around the time when Google was getting Google+ off the ground. Google search has suffered as a result.

  • Twitter Shares Some Search Ranking Details

    As previously reported, Twitter has launched some new search-related changes for its iOS and Android apps.

    Here’s what Twitter said about search in the announcement: “Search results now surface the most relevant mix of Tweets, photos, and accounts, all in one stream (similar to the stream in Discover). We’ve also added a new search button to Twitter for iPhone, letting you search from anywhere within the app. (This button was already available in the Android and iPad apps.) Look for the magnifying glass icon next to the button you use to compose a Tweet.”

    In a separate blog post on the company’s engineering blog, Twitter talks a bit about how it ranks search results.

    Youngin Shin of Twitter’s search quality team (Twitter has a search quality team) explains, “When a user searches, different types of content are searched separately, returning a sequence of candidate results for each content type with a type-specific score for each. For certain content types that are displayed as a single group or gallery unit, such as users or images, we assign the maximum score of results as the representative score of this content type. The result sequences for some content types may be trimmed or discarded entirely at this point.”

    “Once results of different content types are prepared, each type-specific score is converted into a universally compatible score, called a ‘uniscore’,” adds Shin. “Uniscores of different modules are used as a means to blend content types as in a merge-sort, except for the penalization of content type transition. This is to avoid over-diversification of content types in the blended result.”

    Twitter Search

    As Shin explains, all pieces of individual content are assigned type-specific scores called “raw” scores by their corresponding services. Raw scores are then converted into uniscores using type-specific log-linear score conversion functions. According to Shin, the chance of a converted score taking its value in [0, 1] is at least 95%.

    Read the full post here. It’s an interesting look into the back end of Twitter Search.

  • How Twitter Understands Why You’re Really Searching For ‘Honey Nut Cheerios’ (And Decides What Ads To Serve You)

    Twitter’s Edwin Chen and Alpa Jain wrote an interesting post on the company’s engineering blog about improving Twitter search “with real-time computation”. They discuss the challenges of real-time search from an advertising perspective.

    I’ve written quite a few articles about real-time search, often comparing Twitter’s offering to Google’s lack of one since the deal between the two companies came to an end, effectively killing off Google’s. Yet, I’ve never really talked about the advertising-related challenges that must accompany such an offering. Twitter’s post makes you consider what’s going on behind the scenes.

    “The queries people perform have probably never before been seen, so it’s impossible to know without very specific context what they mean,” the two say. “How would you know that #bindersfullofwomen refers to politics, and not office accessories, or that people searching for ‘horses and bayonets’ are interested in the Presidential debates? Since these spikes in search queries are so short-lived, there’s only a small window of opportunity to learn what they mean.”

    “So an event happens, people instantly come to Twitter to search for the event, and we need to teach our systems what these queries mean as quickly as we can — because in just a few hours, the search spike will be gone,” they add.

    To tackle the problem of serving relevant ads on real-time queries, Twitter monitors for which queries are popular at the moment, tracks stats on them, and then sends them to human evaluators, who are tasked with answering a variety of questions about them. To make a complicated story simple, when a response from the evaluator is received, Twitter pushes the info to its backend systems, so the next time a user searches for the query, it can serve a relevant ad based on context.

    “For example, suppose our evaluators tell us that [Big Bird] is related to politics; the next time someone performs this search, we know to surface ads by @barackobama or @mittromney, not ads about Dora the Explorer,” Twitter explains.

    Today, it might be better to see ads for Knicks tickets than for cereal on a Honey Nut Cheerios query.

    So, when you see ads on Twitter as you’re searching for information, just know that a lot of work is going on behind the scenes to make it fit the scope of what you’re actually looking for. Check out Twitter’s post for a much more in depth explanation.

  • Twitter Adds Trends For 100 More Cities

    Twitter Adds Trends For 100 More Cities

    Twitter announced today that it has added Trends for 100 more cities around the world. Among them are Istanbul, Frankfurt, Guadalajara and Incheon. That puts the total number of locations at over 200.

    You can see the Trends for all the different locations by simply clicking “Change” in the Trends section.

    Twitter Change Trends

    “People around the world turn to Twitter to talk about a range of topics – from football to giving thanks to sharing information and resources during natural disasters and emergencies,” says Twitter search and relevance engineer Kostas Tsioutsiouliklis. “This is why Twitter uniquely captures the pulse of the planet, and the pulse of your city: by surfacing the topics that people care about.”

    Meanwhile, Google remains without a realtime search feature, forcing news-seekers looking for up-to-the-second updates to go elsewhere (like Twitter).

    Twitter says it plans on adding more locations for Trends.

  • Without Realtime Search, Google Risks Pushing News Seekers Away To Twitter

    Will Google ever restore its realtime search feature? Will Google and Twitter ever reach another agreement giving Google the access to Twitter’s firehose it needs to make the feature useful? Would the feature ever work without Twitter?

    These are questions we’ve asked repeatedly since the deal expired last year, and the realtime search feature went way. Given that Google’s mission is to organize the world’s information and make it universally accessible, with search being the flagship product, it seems that this is an area that Google should have nailed down. Unfortunately, that’s far from the case.

    Now, the best place to search if you want to find up to the second news and commentary about something that is happening “right now,” is undeniably Twitter. That might have still been the case even when Google had the feature, but Google had/has the luxury of being the starting point for search for most people.

    We recently had a conversation with Mark Schaefer, author of The Tao Of Twitter, and wanted to see what he thinks about this. We asked:

    How badly does Google need to get Twitter’s firehose back for realtime search? Do you think that Google is missing an important function without it? At the same time, is Twitter benefiting from people not going to Google for these kinds of searches?

    “Twitter is trying to reign things in as a way to create more opportunities for monetization,” Schaefer tells WebProNews. “To the extent they can do that, then yes, Twitter will benefit. The real-time results from Twitter are an irreplaceable, unique and highly valuable asset, especially when it comes to providing ‘warm’ search results based on timely comments from friends.”

    “It’s quite ironic that Google+ has been so conservative with providing access to their API,” he adds. “At SXSW last year, Google’s Vic Gundrota said before they made it available, they wanted to make sure it was the right thing to do so people would not be disappointed down the road. In hindsight, his comments seem prescient!”

    Since Google got rid of the feature, Twitter has done things to improve its own realtime search. If Google is not careful, this thing could snowball in Twitter’s favor. Twitter is already a major source of news on the web, and publishers are increasingly distancing themselves from Google News.

    “New data from The Social Habit project reveals that Twitter is benefiting from a youth movement,” Schaefer tells us. “People between 12-17 appear to be piling on to Twitter now. Are they using it for straight news? Probably not!”

    “By comparison, I have surrounded myself on Twitter with the brightest marketing experts I can find,” he adds. “To a large extent, on this topic, Twitter is my trusted RSS feed and it is a very effective one. Today, the news breaks on Twitter. So, yes, Twitter can be an excellent news feed if that is what you want it to be.”

    Could Twitter replace Google News for more and more people? Twitter cares about news. They even recently put out a set of best practices for journalists.

    “News breaks on Twitter, whether local or global, of narrow or broad interest,” Twitter analytics research scientist Jimmy Lin recently said. “When news breaks, Twitter users flock to the service to find out what’s happening. Our goal is to instantly connect people everywhere to what’s most meaningful to them; the speed at which our content (and the relevance signals stemming from it) evolves make this more technically challenging, and we are hard at work continuously refining our relevance algorithms to address this.”

    “Just to give one example: search, boiled down to its basics, is about computing term statistics such as term frequency and inverse document frequency,” he added. “Most algorithms assume some static notion of underlying distributions — which surely isn’t the case here!”

    “During major events, the frequency of queries spike dramatically,” Lin noted. ”For example, on October 5 [2011], immediately following news of the death of Apple co-founder and CEO Steve Jobs, the query ‘steve jobs’ spiked from a negligible fraction of query volume to 15% of the query stream — almost one in six of all queries issued!”

    It’s interesting that even Google is acknowledging Twitter’s growing role in news seekers’ content consumption habits. One of Google’s official twitter accounts tweeted this out today:

    That links to a Brand Republic piece, which says:

    The research, seen exclusively by Media Week, from Ipsos Media shows that 20% of top European businessmen, including chief executives and finance directors, are spending more time on Twitter in an average month, than on global business websites such as Reuters, Bloomberg, and the Economist.

    The data also reveals that the business elite have dropped off in their daily consumption of the Financial Times and The Economist.

    Beyond just news, there is also the social element of Twitter, and Google is increasingly looking to personalize search based on social connections. Twitter (not to mention Facebook) could play a significant role here too, even in real-time terms, when relevant, but it doesn’t look like that’s going to happen any time soon.

    By the way, Twitter (like other Google competitors) is poaching Googlers. They reportedly just got Google business development director, Matthew Derrella.

  • Twitter Takes A Hard Look At Real-Time Search Queries

    Google, which has essentially ruled the search world for years, once recognized the value of realtime search, and Twitter search in particular. The two companies had a deal, which would enable Google users to get quick, real-time results from Twitter for timely queries. Last year, that feature went away, as the two companies failed to come to terms to extend the deal.

    Since then, the lack of the feature has left a glaring hole in Google’s mission to organize the world’s information and make it universally accessible. One example where this was particularly evident, around the time of Muammar Gaddafi’s death.

    While Twitter has said that the two companies talk frequently, there hasn’t been much to indicate that the deal and feature will return in the near future. Meanwhile, people are searching the hell out of Twitter. And just think. Every time someone uses Twitter to do a search, it’s an example of an instance where they’re not using Google to search.

    There’s more to Twitter search, however, than just sifting through the most recent things people have tweeted.

    “News breaks on Twitter, whether local or global, of narrow or broad interest,” says Twitter analytics research scientist Jimmy Lin, Research Scientist, Analytics . “When news breaks, Twitter users flock to the service to find out what’s happening. Our goal is to instantly connect people everywhere to what’s most meaningful to them; the speed at which our content (and the relevance signals stemming from it) evolves make this more technically challenging, and we are hard at work continuously refining our relevance algorithms to address this. Just to give one example: search, boiled down to its basics, is about computing term statistics such as term frequency and inverse document frequency. Most algorithms assume some static notion of underlying distributions — which surely isn’t the case here!”

    Twitter has put out a new study (pdf) about “churn” in tweets and real-time search queries. The abstract is as follows:

    The real-time nature of Twitter means that term distributions in tweets and in search queries change rapidly: the most frequent terms in one hour may look very different from those in the next. Informally, we call this phenomenon “churn”. Our interest in analyzing churn stems from the perspective of real-time search. Nearly all ranking functions, machine-learned or otherwise, depend on term statistics such as term frequency, document frequency, as well as query frequencies. In the real-time context, how do we compute these statistics, considering that the underlying distributions change rapidly? In this paper, we present an analysis of tweet and query churn on Twitter, as a first step to answering this question. Analyses reveal interesting insights on the temporal dynamics of term distributions on Twitter and hold implications for the design of search systems.

    The study reaches the following five conclusions:

    • Churn. Term distributions change rapidly—significantly faster than in web search for the head of the distribution. Even after discounting trending terms promoted by the platform, churn rates of top real-time queries are up to four times higher than those of web searches. For the tail of the distribution, churn drops quickly, and appears to be lower than that observed in web queries.
    • Unobserved terms. Similarly, rates of out-of-vocabulary words are higher for top Twitter queries, but lower at the tail of the distribution. This translates to rapid changes in the top user interests, but relative stability in the topics for which users seek real-time results.
    • Update frequency. Although query churn is consistently high, during major events it can further increase dramatically, as queries change minute by minute. In fact, to maintain accurate collection statistics requires frequent term count updates—in intervals of 5 minutes or less, according to our data.
    •  Churn patterns. The time period in which a query remains a top one varies, as does its decay pattern; na¨ıve approaches such as fixed term frequency decays may not be able to correctly model frequency changes over time.
    • Predictability. Anecdotal evidence suggests that some query churn may be predicted from past observations, providing a potential source for addressing this issue.

    “During major events, the frequency of queries spike dramatically,” says Lin. ” For example, on October 5, immediately following news of the death of Apple co-founder and CEO Steve Jobs, the query ‘steve jobs’ spiked from a negligible fraction of query volume to 15% of the query stream — almost one in six of all queries issued!”

    Steve Jobs queries

    Lin talks about the Japan Earthquake as another example of Twitter’s power, highlighting yet another event, in which today’s Google would be playing second fiddle to Twitter search, in terms of rapid, realtime info.

    Twitter is presenting findings at the International Conference on Weblogs and Social Media (ICWSM 2012), a workshop on Social Media Visualization, and a workshop on Real-Time Analysis and Mining of Social Streams (RAMSS).

  • Twitter Improves Realtime Search While Google Lacks It

    Twitter announced some new improvements to its Discover tab today. It’s getting more personalized, and will begin surfacing content based on tweets that are popular among people you follow and who they follow.

    “The Discover tab’s new design shows who tweeted about particular stories,” explains Twitter VP of Product, Satya Patel. “You can click ‘View Tweets’ on any story to see popular Tweets from your network or recent, relevant Tweets directly below the story summary. This social context helps you understand why each story matters to you and makes it easier to join the conversation. You can reply, retweet or favorite these Tweets, or you can ‘Tweet this story’ to share your own perspective.”

    There’s an interesting post on Twitter’s engineering blog talking about the back-end of how the features work. “Behind the scenes, the new Discover tab is powered by Earlybird, Twitter’s real-time search technology,” explains Twitter Director of Engineering, Ori Allon. “When a user tweets, that Tweet is indexed and becomes searchable in seconds. Every Tweet with a link also goes through some additional processing: we extract and expand any URLs available in Tweets, and then fetch the contents of those URLs via SpiderDuck, our real-time URL fetcher.”

    “To generate the stories that are based on your social graph and that we believe are most interesting to you, we first use Cassovary, our graph processing library, to identify your connections and rank them according to how strong and important those connections are to you,” Allon adds. “Once we have that network, we use Twitter’s flexible search engine to find URLs that have been shared by that circle of people. Those links are converted into stories that we’ll display, alongside other stories, in the Discover tab. Before displaying them, a final ranking pass re-ranks stories according to how many people have tweeted about them and how important those people are in relation to you.”

    This might be useful information to know if you’re thinking about engaging in any realtime SEO strategies. Twitter is probably the go to place for a lot of people looking for realtime info and commentary these days, especially now that Google no longer has realtime search (thanks to the expiration of a deal with Twitter last year).

    Twitter’s efforts in this area represent a real point of competition for Google. Not that Twitter and Google are competing on search in general, but Google’s greatest threat is users’ decreased reliance on it for various search tasks. Google’s lack of realtime search has to be driving some amount of searches to Twitter. I know it’s driving plenty of mine there. If I need to find up-to-the-second info or commentary on a subject, I can’t rely on Google for that without realtime search.

    See: Gaddafi Shows Why Google is Failing Its Mission in Search

    Google has put a great deal of emphasis on freshness of search results with various algorithm changes over the last six months or so. That combined with Search Plus Your World (the personalization element, which draws significantly from Google’s own social network), I believe is Google’s attempt to compensate for the lack of realtime search, at least in the short term. Comments made by the company in the past indicate they’re hoping Google+ can eventually provide the same kind of realtime power that Twitter can’t, but the user numbers just aren’t there. Not for the “social destination” part of Google+ (as opposed to the “social spine” part).

    So, Google is jamming “fresh” content into organic search, sometimes at the cost of relevancy. It’s not working as a substitute for realtime search. For one, when Google did have realtime search, it was its own section in the search results, not unlike Google News results (or other types of Universal Search results) are today.

    “All of this happens in near-real time, which means breaking and relevant stories appear in the new Discover tab almost as soon as people start talking about them,” says Allon of Twitter’s offering.

    That’s important, especially for journalists. And it’s something Google can’t provide in search results without the realtime search element – without Twitter. Until people start using Google+ like they use Twitter, it’s just not going to happen, or unless Google can get back that Twitter firehose.

    Twitter says the Discover tab update is only part of the company’s ongoing development of the feature. In other words, Twitter is working on getting even better at finding ways to get relevant, realtime info to users.

    The Discovery tab updates will be rolling out over the coming weeks on Twitter.com and the Twitter iPhone and Android apps.

  • Yandex Gets Twitter Access, Launches Social Features. Should Google Worry?

    Yandex, the Russian search engine giant, announced a new social networking search program.

    The announcement doesn’t mention Twitter specifically, but Search Engine Land is reporting that the search engine has made a deal with Twitter to gain access to the Twitter Firehose. Little has been said about how Yandex will actually used this, but reporter Greg Sterling says, Yandex “has discretion over what it uses and how that content is ranked and displayed.”

    You may recall that last year, Google’s deal with Twitter for that same access expired, and the two companies failed to reach an agreement to extend it. This led to the disappearance of Google’s Realtime Search feature, making Google less useful for some searches.

    Google would later go on to release “Search Plus Your World,” favoring Google+ connections in search results, and souring the relationship between the two companies further.

    Bing has access to Twitter (and Facebook), and some have expressed intent to actually switch search engines over the whole thing. Could Yandex become a more significant global competitor with some more expansion? Beyond Russia, Yandex has sites in Turkey, Belarus, Ukraine, and Kazakhstan.

    Yandex’s new offerings include the roll out of a beta version of a “people finder” for its Russian site. Users can view all public profiles of a person with accounts on sites like VKontakte, Facebook, LiveJournal, and Odnoklassniki (popular services in that country).

    “It is so much more convenient to see multiple profiles of the same person grouped together,” says Yandex Product Manager Alexander Chubinskiy. “Yandex does this grouping with care — only those profiles that refer to one another get grouped. Web users can choose if they want their profiles on different websites to appear in search results separately, or as grouped together. So, if one of your personal profiles refers to others, the icons of those websites on which they are hosted will appear on the same thumbnail. Conversely, the user can remove cross-reference from their personal pages so that each of the profiles appears in Yandex’s search results independently.”

    Yandex claims about 61% of Russia’s search market share and 45 million monthly visitors to its Russian search engine.

    The company says it processes over two million people searches daily. About half of them, it says, are to find information about celebrities, while the other half ask about someone’s friend, a contact, an employee or a partner.