WebProNews

Tag: BigQuery

  • Google Cloud Announces Public Preview of BigQuery Native JSON

    Google Cloud Announces Public Preview of BigQuery Native JSON

    Google Cloud announced a public preview of their BigQuery Native JSON data type, bringing support for semi-structured data in BigQuery.

    JSON is a popular format for storing data, since it doesn’t require a structured scheme like a traditional database. This allows for storing, managing, and querying semi-structured data, an ideal option for cloud applications.

    Google announced the public preview via Twitter.

    Google Cloud customers are already looking forward to the ways this new feature will help them analyze data and gain insights.

    “Agility is key to our business. We believe Native JSON functionality will enable us to handle changes in data models more quickly and shorten the lead time to pull insights from our data,” says Ryoji Hasegawa, Data Engineer, DeNA Co Ltd.

    Customers interested in signing up for the public preview can do so here.

  • Google Cloud Platform Gets Price Drop, New Features

    Google announced on Tuesday at its Cloud Platform Live event that it is lowering prices on Cloud Platform, and launching cloud-based DevOps tooling, Managed Virtual Machines (VM) for App Engine, real-time Big Data analytics with BigQuery.

    On the pricing, Google SVP of Technical Infrastructure Urs Hölzle says, “The original promise of cloud computing was simple: virtualize hardware, pay only for what you use, with no upfront capital expenditures and lower prices than on-premise solutions. But pricing hasn’t followed Moore’s Law: over the past five years, hardware costs improved by 20-30% annually but public cloud prices fell at just 8% per year. We think cloud pricing should track Moore’s Law, so we’re simplifying and reducing prices for our various on-demand, pay-as-you-go services by 30-85%.”

    Compute Engine has been reduced by 32% across all sizes, regions and classes. App Engine gets a simplified pricing strtucture with significant reductions in database operations and front-end compute instances. Cloud Storage is priced at 2.6 per GB, which is roughly 68% less for most customers, according to the company. Finally, BigQuery on-demend prices have been reduced by 85%.

    They’ve also launched sustained-use discounts, which start automatically when you use a VM for over 25% of the month. When you use a VM for a whole month, Google knocks off another 30% over the new on-demand prices for a total reduction of 53
    53%.

    sustained discounts

    “We’re also introducing features that make development more productive,” says Hölzle. “Build, test, and release in the cloud, with minimal setup or changes to your workflow. Simply commit a change with git and we’ll run a clean build and all unit tests; Aggregated logs across all your instances, with filtering and search tools; Detailed stack traces for bugs, with one-click access to the exact version of the code that caused the issue. You can even make small code changes right in the browser. We’re working on even more features to ensure that our platform is the most productive place for developers. Stay tuned.”

    Managed VMs will let you run any binary inside a VM and turn it into part of your App Engine app, and App Engine will automatically manage them.

    On real-time big data, Hölzle says, “BigQuery lets you run interactive SQL queries against datasets of any size in seconds using a fully managed service, with no setup and no configuration. Starting today, with BigQuery Streaming, you can ingest 100,000 records per second per table with near-instant updates, so you can analyze massive data streams in real time. Yet, BigQuery is very affordable: on-demand queries now only cost $5 per TB and 5 GB/sec reserved query capacity starts at $20,000/month, 75% lower than other providers.”

    Compute Engine now supports Windows Server 2008 R2 in limited preview and Red Hat Enterprise Linux and SUSE Linux Enterprise Server for everyone.

    Image via Google

  • Google Adds Capabilities To BigQuery

    Google Adds Capabilities To BigQuery

    Google announced a new update to BigQuery today, with the ability to run queries that return large numbers of rows and save them as a new table.

    The update also adds window functions, like the ability to take advantage of Rank and Partition to create statistical analyses with simpler SQL. Additionally, recent queries that are re-run return a cached result when the underlying table is unchanged. Google says this provides more cost-effective analysis.

    Also included are some new user interface changes, such as the ability to validate a query and estimate its cost before running it, as well as the ability to save frequently used queries. Google has also doubled existing query quotas.

    The changes build on other recent features like the ability to connect tools like Tableau, BIME and Excel.

    There area also new pricing options, effective in July. Data storage is going from $0.12/GB/month to $0.08/GB/month.

    “Furthermore, in addition to the existing on-demand rate for interactive queries, customers with higher-volume usage will soon be able to opt in for tiered query pricing,” says product manager Ju-kay Kwek. “This provides more economical and predictable cost for interactive queries. Customers who are interested are encouraged to contact a sales representative.”

    More on the new stuff at Google’s Developers blog.

    See how Interactions used BigQuery and Tableau to measure the effects of weather events on retail sales with some research being presented in London today.

  • Google BigQuery Helps Show How Weather Influences Shopping

    Interactions, a marketing firm for supermarket chains and big box stores, announced some interesting findings based on analysis with Google’s BigQuery and Tableau, measuring the effects of major weather events on sales.

    “This case study is just the first analysis in an ongoing relationship between Interactions, Google and Tableau to provide retailers and CPGS with data management solutions and insightful analysis,” a spokesperson for Interactions tells WebProNews. “While the results of the initial study may seem obvious, this type of analysis provides retailers with specific detailed information not previously available that will help them launch more successful marketing efforts resulting in increased basket size and cannibalization of competitors’ shoppers.”

    Using BigQuery and Tableau’s visual analytics software, Interactions layered “fine-grain transactional-level sales data” with multi-source, detailed regional weather data, and is using it as a predictive tool during and prior to specific weather events in order to help retailers minimize (or eliminate) out-of-stock issues, optimize item assortments in high-demand categories, and increase sales in those categories.

    “Early results suggest that these insights could deliver significant benefits, including boosting the return on investment for marketing and advertising spend and improving targeted and individualized shopper communications resulting in increased basket size and cannibalization of competitors’ shoppers,” the company says.

    The analysis specifically focused on identifying negative weather events, classifying them by severity, and measuring the effects they had on sales before, during and after those events. The company says it was able to track new patterns in sales and shopper behavior.

    “Interactive performance of Google BigQuery, combined with Tableau’s intuitive visualization tools enabled our analysts to interactively explore huge quantities of data – hundreds of millions of rows – with incredible efficiency. In some cases taking analysis that would ordinarily require a week down to just hours and minutes. This time-to-insight was previously impossible,” said Giovanni DeMeo, Vice President of Global Marketing and Analytics for Interactions. “It enabled us to visually share that information with our retailer and CPG partners, and use it to enhance in-store activity and increase sales. This is only one of an infinite number of ways that we will now be using big data to improve the revenue and profitability of our partners.”

    “Retailers have access to massive amounts of complex data to help them make good decisions. The trick is to find a way to easily visualize and analyze it effectively,” said Francois Ajenstat, Director of Product Management at Tableau. “By combining the flexibility and horsepower of Google BigQuery with Tableau’s visual analytics, Interactions has delivered insights that were not previously revealed. Retailers and CPGs will now be able to make real-time data driven decisions to inform their business.”

    According to Interactions, it was able to reveal, down to the product level, which items had the most significant change in sales (both increases and decreases), and what varied in shopper behavior for similar weather events, considering the time of day, day of week, geographic location, and proximity to competitor locations. The data was actually able to identify 28 categories with significant changes in sales (compared to the control).

    Based on the data, one day before statistically similar weather events, sales in those categories spiked from 20% to 261% over the same day in the previous year, and dropped in sales during the peak of the event and for four days after. This happened not only in regions that actually experienced the event, but also in those where the event had been predicted, but never occurred.

    “If weather reports predicted a storm a week ahead, people still waited until the day before the event to do their event-specific shopping,” says Interactions. “In one scenario, and contrary to every other shopping behavior, this resulted in a huge spike in Monday sales over the preceding weekend for a predicted Tuesday weather event.”

    Interactions, Google and Tableau are discussing more details about their findings at the Tableau European Customer Conference in London on Tuesday.

  • Google Ups Its Cloud Platform Support Game

    Google announced the launch of a new set of support packages for services on the Google Cloud Platform today. These cover App Engine, Compute Engine, Cloud Storage, Cloud SQL, BigQuery, etc.

    “While you can go to Stack Overflow or Google Groups, we realize some of you may need 24×7 coverage, phone support or direct access to a Technical Account Manager team,” says Brett McCully from Google’s Cloud Platform team.

    There are four tiers:

    • Bronze: All customers get access to online documentation, community forums, and billing support. (Free)
    • Silver: In addition to Bronze, you can email our support team for questions related to product functionality, best practices, and service errors. ($150/month)
    • Gold: In addition to Silver, you’ll receive 24×7 phone support and consultation on application development, best practices or architecture for your specific use case. (Starts at $400/month)
    • Platinum: The most comprehensive and personalized support. In addition to Gold, you’ll get direct access to a Technical Account Manager team. (Contact Sales for more information)

    Here’s another way to look at it:

    Cloud Platform

    More info here.

  • Google Adds Batch Queries, Excel Connector To BigQuery

    Google Adds Batch Queries, Excel Connector To BigQuery

    Google announced today that it has added a couple of new features to BigQuery, its Big Data analysis service. It now supports batch queries, and has a connector for Excel.

    “While BigQuery specializes in getting insights quickly, we understand that there are important, non-interactive queries, such as nightly reports, that businesses also need to run,” says product manager Ju-kay Kwek. “Now, you can designate a query as a batch query and it will complete within a few hours. “If you’re using BigQuery via our standard self-service model, you pay 2 cents per GB processed for batch queries and 3.5 cents per GB processed for interactive queries.”

    “Analysts and executives use spreadsheets to explore large data sets,” adds Kwek. “Last year, we launched the ability for BigQuery users to execute queries inside Google spreadsheets using the Google Apps Script integration. With the new BigQuery Connector for Excel, we’re now making it simpler to execute BigQuery queries using Microsoft® Excel. This connector takes advantage of Excel’s standard web query feature to eliminate the extra work of manually importing data and running queries directly within Excel. For instructions on how to download and use the connector, see the BigQuery Connector for Excel page.”

    Ryan Boyd, Developer Advocate for Cloud Data Services has more on using the new features on Google’s Developers Blog.

  • Google’s BigQuery Means Analytics for all Businesses

    Google is introducing an analytics solution for businesses of all sizes without the need for investments in additional hardware or software. BigQuery is a web service that lets you analyze information from huge datasets and have on-demand access to the figures. The service is available for free right now.

    Google has been running a trial version of their analytical solution since November and many businesses and developers are using it with great success. Claritics, a mobile solutions provider, developed a mobile analytics application based on BigQuery.

    Claritics CEO Raj Pai comments on BigQuery:

    “Getting data into the system was cumbersome,”

    “Once the data was in, running queries – asking simple questions and getting responses – took a lot of time. Complex queries on large datasets could take more than 30 minutes.”

    “We can use the BigQuery Service infrastructure to create advanced queries and build these really quickly from a prototyping perspective,”

    “Google BigQuery Service is saving us time and resources. Since we don’t have to worry about setting up machines as we bring more clients onboard, we expect it will save us a lot of money as well.”

    Crystalloids is another company who streamlined their efforts and used BigQuery to help vacation destinations plan, predict, and target markets based on available data from past sales and inquiries.

    Richard Verhoeff, founder of Crystalloids Innovations comments on BigQuery:

    “We had to get all of the data over into Excel and then format the reports, which became very tedious,”

    “Then we would have stacks of papers on our desk, comparing this year’s occupancy rates with last year’s and trying to identify problems.”

    “Google BigQuery Service makes it possible to examine millions of records in seconds,”

    “Other business intelligence solutions could take eight minutes to come back with an answer. That’s much too long. Speed is an essential part of our application, and BigQuery Service gives us that.”

    Google+Enterprise+Blog%29″>Businesses and developers alike can sign up for the BigQuery service and get 100 GB of data per month for free. They also offer special pricing plans for storing and query up to 2 TB. Google wants you to try the service and share your success stories on Google+. It sounds like a great service. Having real-time business information can vital for reaching new customers and competing against other providers in the same market.

  • BigQuery: Google’s Data Analysis Platform

    BigQuery: Google’s Data Analysis Platform

    Let’s say you’re with a company that analyzes thousands of terabytes as part of its business. You can either build a giant data farm to sift all of the data, or you can, well, all you can really do is build a server farm. Google hopes to help those kind of companies with their data analysis by offering up their servers as an alternative.

    BigQuery, according to the Google Developers page, is a “Web service that lets you do interactive analysis of massive datasets – up to bilions of rows.” That’s pretty impressive and must take a lot of server space. That’s exactly the point though as Google is offering to let businesses and developers offload their data onto Google’s servers for analysis.

    ComputerWorld is reporting that Google product manager Ju-Kay Kwek spoke at the GigaOm Structure Data conference this week about the company’s plans for this new service. The boldest claim they can offer in regards to BigQuery is that they let businesses offload all the data onto the cloud. This prevents them from having to build data farms and it lets Google fine tune their data collection algorithms.

    Some of the trial users at the moment are using the service to collect data on how ads are doing or making fiscal projections. All this and more is done over the cloud at a price that is “quite reasonable” according to one company using the service.

    Interested companies can sign up for the BigQuery preview right now. This will put you on a waitlist and you will be notified when the service is ready.

    Here’s a talk from Google I/O 2010 that touched upon BigQuery: