#Datacanvas

[govindhtech]

BigQuery Data Canvas: Now More Powerful for Faster Insights

Turn data into insights faster with the updated BigQuery data canvas.

Today's data-driven world requires quick insights. Many struggle to transform raw data into meaningful intelligence. Complex SQL queries, tedious investigations, and the split between technical and non-technical users slow progress. BigQuery Data Canvas is a visual workspace that democratises data research and unleashes BigQuery data potential. At Google Cloud Next '25 this month, Google Cloud introduced Data Canvas, powered by Gemini, with AI-assisted conversation. One question covers data exploration, visualisation, and workflow analysis in this chat experience.

Data canvas is a paradigm change for data practitioners, not just a new function. By seamlessly combining visual processes with BigQuery and Gemini, Google Cloud is turning raw data into valuable insights.

Key traits: In-depth

Explore the data canvas assistant's powers.

Gemini fuels AI data agents.

Google has added Gemini, their powerful AI model, to Data Canvas to boost data exploration. It lets you ask data questions, enhance queries using natural language, and obtain informative and useful replies. The Gemini-powered data canvas queries and provides dataset information when you select “Show me the top 10 customers by revenue”. Gemini's data discovery recommendations help you find relevant datasets.

The Gemini-powered AI conversation experience captures workflow analysis from data exploration to visualisation with one prompt. Not sure where to start? Start analysing your data by following the directions. BigQuery Data Canvas uses Gemini to generate natural language enquiries about your data and SQL queries based on your favourite tables. To answer your questions, Gemini might use many data sources.

You may also direct the system to transfer domain knowledge about your data to increase answer accuracy. You may tell Gemini that your company's fiscal year is not January–December using system instructions. Some system directives, such “always present findings with charts, use green colour for positive and red colour for negative,” might influence how your replies are presented and returned.

Complex Python analysis for forecasting and anomaly identification will soon be available in chat. If your inquiry is complex, Gemini chat assistant will use a Python code interpreter to reply. Turn this option on in the settings bar.

Visual query building: Explore several perspectives in one place. Imagine a single hub where you can filter, link, aggregate, or display data from several tables, each in its own container, on the same page for data analysis. Data Canvas uses a DAG (Directed Acyclic Graph) method instead of a linear one, allowing you to explore alternative views, return to prior phases, or compare outcomes.

Data may be added easily by finding and adding tables to the canvas. Starting with natural language queries to your data, Data Canvas will automatically generate the SQL, which you can analyse and edit. Our node-based technique lets SQL professionals and rookie analysts pursue insights without having to cope with complex query syntax.

Interactive visualisations: Instant insights

Data Canvas offers interactive tables, graphs, and charts. Customising visualisations, interactively examining data, and identifying patterns and irregularities are easy. Do you want to see regional sales distribution? Data Canvas automatically creates a chart using the “Region” and “Sales” variables. Just choose the best visualisation or make your own to watch your data come to life. These visualisations may be downloaded as PNGs or imported into Looker Studio for customisation and dissemination.

Implementing data canvas in practice

New AI functionalities on BigQuery data canvas may be used in many ways. Here are some industry-specific ideas to inspire you.

Telecom diagnostics and support: Faster service restoration

Consider a telecom support team helping clients. Using a data canvas, support tickets may be queried for customer phone, postcode, service, timestamp, and nearby mobile tower. BigQuery processes support tickets hourly. Each data item is maintained in a distinct node inside a canvas, so analysts may perform this study without switching query tabs. This visual method helps them find local outages, send personnel to the right towers, and fix service interruptions faster.

E-commerce analytics: Expanding sales and customer engagement

Imagine marketing personnel analysing customer purchasing data to optimise adverts. Data canvas lets them graphically mix customer and product information, filter by prior purchases, and examine demographic sales tendencies. By quickly identifying high-value customer categories, best-selling goods, and marketing success, they can make data-driven judgements.

Supply chain optimisation: logistics simplification

Logistics management may utilise Data Canvas to track inventories, delivery routes, and bottlenecks. Visualising supply chain data may boost efficiency, cut costs, and enhance delivery dates. They may also create interactive dashboards to track performance indicators and make real-time changes.

Visual data exploration is the future, and BigQuery data canvas, powered by AI, makes data useful and accessible to everyone. It combines Gemini's intelligence, BigQuery's power, and visual processes to maximise your data. Start your future data exploration experience now.

[datayuan]

DataCanvas released DataCanvas Intelligent Computing Cloud Alaya NeW Cloud 2.0

[androidponselku88my]

NeutraDC Gandeng HPE, Cirrascale, dan DataCanvas untuk Perkuat Ekosistem AI di Indonesia dan Asia Tenggara http://dlvr.it/TCkBLR

[pendolaredigitale]

WKS at Frontiers of Interaction Conference - Sept 21-22, 2017

A snippet of the presentation about data project management given at Frontiers of Interaction conference on September 21th.

https://www.slideshare.net/topix/the-data-ring-a-canvas-for-data-project

[therumorsofdata]

Playing with DataCanvas data and d3.js

[swissforextrading]

DataCanvas: Sense Your City – Mikko Jarvenpaa from Infogr.am Shares Tricks & Tips: Mikko Jarvenpaa is Infogr.am’s Chief Operating Officer responsible for the daily operations of the online data visualization service. Mikko is Finnish and has been serving as Product Marketing Manager for AdWords at Google, worked with tech startups in both Silicon Valley and Europe, and has recently been the Chief Marketing Geek for HackFwd. In this […] The post DataCanvas: Sense Your City – Mikko Jarvenpaa from Infogr.am Shares Tricks & Tips appeared first on SwissForexTrading.ch. http://dlvr.it/9vGSVR

[govindhtech]

Start Using Gemini In BigQuery Newly Released Features

Gemini In BigQuery overview

The Gemini for Google Cloud product suite’s Gemini in BigQuery delivers AI-powered data management assistance. BigQuery ML supports text synthesis and machine translation using Vertex AI models and Cloud AI APIs in addition to Gemini help.

Gemini In BigQuery AI help

Gemini in BigQuery helps you do these with AI:

Explore and comprehend your data with insights. Generally accessible (GA) Data insights uses intelligent queries from your table information to automatically and intuitively find patterns and do statistical analysis. This functionality helps with early data exploration cold-start issues. Use BigQuery to generate data insights.

Data canvas lets BigQuery users find, transform, query, and visualize data. (GA) Use natural language to search, join, and query table assets, visualize results, and communicate effortlessly. Learn more at Analyze with data canvas.

SQL and Python data analysis help. Gemini in BigQuery can generate or recommend SQL or Python code and explain SQL queries. Data analysis might begin with natural language inquiries.

Consider partitioning, clustering, and materialized views to optimize your data infrastructure. BigQuery can track SQL workloads to optimize performance and cut expenses.

Tune and fix serverless Apache Spark workloads. (Preview) Based on best practices and past workload runs, autotuning optimizes Spark operations by applying configuration settings to recurrent Spark workloads. Advanced troubleshooting with Gemini in BigQuery can identify job issues and suggest fixes for sluggish or unsuccessful jobs. Autotuning Spark workloads and Advanced troubleshooting have more information.

Use rules to customize SQL translations. (Preview) The interactive SQL translator lets you tailor SQL translations with Gemini-enhanced translation rules. Use natural language prompts to define SQL translation output modifications or provide SQL patterns to search and replace. See Create a translation rule for details.

Gemini in BigQuery leverages Google-developed LLMs. Billion lines of open source code, security statistics, and Google Cloud documentation and example code fine-tune the LLMs.

Learn when and how Gemini for Google Cloud utilizes your data. As an early-stage technology, Gemini for Google Cloud products may produce convincing but false output. Gemini output for Google Cloud products should be validated before usage. Visit Gemini for Google Cloud and ethical AI for details.

Pricing

All customers can currently use GA features for free. Google will disclose late in 2024 how BigQuery will restrict access to Gemini to these options:

BigQuery Enterprise Plus version: This edition includes all GA Gemini in BigQuery functionalities. Further announcements may allow customers using various BigQuery editions or on-demand computation to employ Gemini in BigQuery features.

SQL code assist, Python code assist, data canvas, data insights, and data preparation will be included in this per-user per-month service. No tips or troubleshooting in this bundle.

84% of enterprises think generative AI would speed up their access to insights, and interestingly, 52% of non-technical users are already using generative AI to extract insightful data, according to Google’s Data and AI Trends Report 2024.

Google Cloud goal with Google’s Data Cloud is to transform data management and analytics by leveraging their decades of research and investments in AI. This will allow businesses to create data agents that are based on their own data and reinvent experiences. Google Cloud unveiled the BigQuery preview of Gemini during Google Cloud Next 2024. Gemini offers AI-powered experiences including data exploration and discovery, data preparation and engineering, analysis and insight generation throughout the data journey, and smart recommendations to maximize user productivity and minimize expenses.

Google Cloud is pleased to announce that a number of Gemini in BigQuery capabilities, including as data canvas, data insights and partitioning, SQL code generation and explanation, Python code generation, and clustering recommendations, are now generally available.

Let’s examine in more detail some of the features that Gemini in BigQuery offers you right now.

What distinguishes Gemini in BigQuery?

Gemini in BigQuery combines cutting-edge models that are tailored to your company’s requirements with the best of Google’s capabilities for AI infrastructure and data management.

Context aware: Interprets your intentions, comprehends your objectives, and actively communicates with you to streamline your processes.

Based on your data: Constantly picks up fresh information and adjusts to your business data to see possibilities and foresee problems

Experience that is integrated: Easily obtainable from within the BigQuery interface, offering a smooth operation across the analytics workflows

How to begin using data insights

Finding the insights you can gain from your data assets and conducting a data discovery process are the initial steps in the data analysis process. Envision possessing an extensive collection of perceptive inquiries, customized to your data – queries you were unaware you ought to ask! Data Insights removes uncertainty by providing instantaneous insights with pre-validated, ready-to-run queries. For example, Data Insights may suggest that you look into the reasons behind churn among particular customer groups if you’re working with a database that contains customer churn data. This is an avenue you may not have considered.

With just one click, BigQuery Studio’s actionable queries may improve your analysis by giving you the insights you need in the appropriate place.

Boost output with help with Python and SQL codes

Gemini for BigQuery uses simple natural language suggestions to help you write and edit SQL or Python code while referencing pertinent schemas and metadata. This makes it easier for users to write sophisticated, precise queries even with little coding knowledge, and it also helps you avoid errors and inconsistencies in your code.

With BigQuery, Gemini understands the relationships and structure of your data, allowing you to get customized code recommendations from a simple natural language query. As an illustration, you may ask it to:

“Generate a SQL query to calculate the total sales for each product in the table.”

“Use pandas to write Python code that correlates the number of customer reviews with product sales.”

Determine the typical journey duration for each type of subscriber.

BigQuery’s Gemini feature may also help you comprehend intricate Python and SQL searches by offering explanations and insights. This makes it simpler for users of all skill levels to comprehend the reasoning behind the code. Those who are unfamiliar with Python and SQL, or who are working with unknown datasets, can particularly benefit from this.

Analytics workflows redesigned using natural language

Data canvas, an inventive natural language-based interface for data curation, wrangling, analysis, and visualization, is part of BigQuery’s Gemini package. With the help of data canvas, you can organize and explore your data trips using a graphical approach, making data exploration and analysis simple and straightforward.

For instance, you could use straightforward natural language prompts to collect information from multiple sources, like a point-of-sale (POS) system; integrate it with inventory, customer relationship management (CRM) systems, or external data; find correlations between variables, like revenue, product categories, and store location; or create reports and visualizations for stakeholders, all from within a single user interface, in order to analyze revenue across retail stores.

Optimize analytics for swiftness and efficiency

Data administrators and other analytics experts encounter difficulties in efficiently managing capacity and enhancing query performance as data volumes increase. BigQuery’s Gemini feature provides AI-powered suggestions for partitioning and grouping your tables in order to solve these issues. Without changing your queries, these suggestions try to optimize your tables for quicker returns and less expensive query execution.

Beginning

Phased rollouts of the general availability of Gemini in BigQuery features will begin over the following few months, starting today with suggestions for partitioning and clustering, data canvas, SQL code generation and explanation, and Python code generation.

Currently, all clients can access generally accessible (GA) features at no additional cost. For further details, please refer to the pricing details.

Read more on govindhtech.com

[govindhtech]

How BigQuery Data Canvas Makes AI-Powered Insights Easy

A Gemini feature in BigQuery, the BigQuery Studio data canvas provides a graphical interface for analysis processes and natural language prompts for finding, transforming, querying, and visualising data.

A directed acyclic graph (DAG) is used by BigQuery data canvas for analysis workflows, giving you a graphical representation of your workflow. Working with many branches of inquiry in one location and iterating on query results are both possible with BigQuery data canvas.

BigQuery data canvas

The BigQuery data canvas is intended to support you on your path from data to insights. Working with data doesn’t require technical expertise of particular products or technologies. Using natural language, BigQuery data canvas and Dataplex metadata combine to find relevant tables.

Gemini in BigQuery is used by BigQuery data canvas to locate your data, build charts, create SQL, and create data summaries.

Capabilities

BigQuery data canvas lets you do the following:

Use keyword search syntax along with Dataplex metadata to find assets such as tables, views, or materialized views.

Use natural language for basic SQL queries such as the following:

Queries that contain FROM clauses, math functions, arrays, and structs.

JOIN operations for two tables.

Visualize data by using the following types graphic types:

Bar chart

Heat map

Line graph

Pie chart

Scatter chart

Create custom visualizations by using natural language to describe what you want.

Automate data insights.

Limitations

Natural language commands might not work well with the following:

BigQuery ML

Apache Spark

Object tables

BigLake

INFORMATION_SCHEMA views

JSON

Nested and repeated fields

Complex functions and data types such as DATETIME and TIMEZONE

Data visualizations don’t work with geomap charts.

A ground-breaking data analytics tool, BigQuery data canvas, a Gemini in BigQuery feature, streamlines the whole data analysis process from data preparation and discovery to analysis, visualisation, and collaboration – all in one location, all within BigQuery. You may ask questions in both plain English and a variety of other languages about your data using the BigQuery data canvas, which makes use of natural language processing.

Because sophisticated SQL queries don’t need to be developed using this easy method, data analysis is now accessible to both technical and non-technical people. You may examine, modify, and display your BigQuery data using data canvas without ever leaving the environment in which it is stored.

This blog post provides a technical walkthrough of a real-world scenario utilising the public github_repos dataset, along with an overview of BigQuery data canvas. Over 3TB of activity from 3M+ open-source repositories are included in this dataset. We’ll look at how to respond to inquiries like:

In a year, how many commits were made to a particular repository?

In a particular year, who authored the most repositories?

Over time, how many non-authored commits were applied?

Which users, at what time, contributed to a certain file?

You’ll see how data canvas manages intricate SQL operations from your natural language prompts, such as joining tables, extracting particular data items, unnesting fields, and converting timestamps. We’ll even show you how to use just one click to create intelligent summaries and visualisations.

BigQuery data canvas quickly overview

BigQuery data canvas is mostly used for three types of tasks: finding data, generating SQL, and generating insights.Image credit to Google Cloud

Find Data

To locate data in BigQuery using a rapid keyword search or a natural language text prompt, use data canvas.

Generate SQL

Additionally, you may use the BigQuery data canvas to have SQL code written for you using natural language prompts powered by Gemini.

Create Insights

At last, use a single click to uncover insights concealed within your data! Gemini creates visualisations for you automatically so you can see the story your data is telling.

Using the BigQuery data canvas

Let’s look at an example to help you better understand the potential impact that the BigQuery data canvas can have in your company. Businesses of all kinds, from big corporations to tiny startups, can gain from having a better grasp of the productivity of their development staff. Google Cloud will demonstrate in this in-depth technical tutorial how to leverage data canvas and the public dataset github_repos to provide insightful results in a shared workspace.

You’ll learn how data canvas simplifies the creation of sophisticated SQL queries by working through this example, which demonstrates how to create joins and unnested columns, convert timestamps, extract the month and year from date fields, and more. Gemini’s features make it simple to create these queries and use natural language to examine your data with illuminating visualisations.

Please be aware that using any LLM-enabled application successfully requires strong prompt engineering abilities, just like using many of the new  AI products and services available today. Many people might believe that large language models (LLMs) aren’t very excellent at producing SQL right out of the box. However, in our experience, Gemini in BigQuery via data canvas may produce sophisticated SQL queries using the context of your data corpus if you use the appropriate prompting mechanisms. It is evident that data canvas uses natural language queries to decide the ordering, grouping, sorting, record count limitation, and SQL structure.

The github_repos dataset, which is 3TB+ in size and can be found in Bigquery Public Datasets, comprises information in numerous tables regarding commits, watch counts, and other activity on 3M+ open-source projects. We want to look at the Google Cloud Platform repository for this example. As always, before you begin, make sure you have the necessary IAM permissions. In addition, make sure you have the necessary rights to access the datasets and data canvas in order to run nodes properly.

Using data canvas makes it simple to explore every table in the github_repos dataset. Here, Google  Cloud evaluate schema, details, and preview data in one panel while comparing datasets side by side.Image credit to Google cloud

After choosing your dataset, you can hover over the bottom of the node to branch it to query or join it with another table. The dataset for the following transformation node is shown by arrows. For clarity, you can give each node a name when sharing the canvas. You can delete, debug, duplicate, or run all of the nodes in a series using the options in the upper right corner. Results can be downloaded, and data can be exported to Looker Studio or Sheets. In the navigation panel, you can also inspect the DAG structure, restore previous versions, and rate SQL suggestions.

Google examine four main facets of their data while examining the github_repos dataset. They will attempt to ascertain the following:

1) The total number of commitments made in a single year

2) The quantity of written repos for a specific year

3) The total number of non-authored commits that were applied throughout time

4) Determine how many user commits there have been for a specific file at a specific time.

Utilise BigQuery data canvas to simplify data analysis

It might be challenging to interpret data for a new project or use case when working with large datasets that span multiple disciplines. This procedure can be streamlined by using data canvas. Data canvas helps you work more efficiently and quickly by streamlining data analysis using natural language-based SQL creation and visualisations. It also reduces the need for repetitive queries and lets you plan automatic data refreshes.

Read more on Govindhtech.com

[datayuan]

迎接算力普惠时代，九章云极DataCanvas算力包产品重磅发布！

9月24日，2024九章云极DataCanvas算力包产品发布会于北京盛大召开，倾力打造的全新算力产品—���「算力包」正式亮相。「算力包」产品的问世，成功打通九章云极DataCanvas公司向社会广大算力消费者提供更经济、更好用、高质量、高效率的一体化算力服务的“最后一公里”！

[datayuan]

DATACANVAS AIDC OS智算操作系统正式发布， 定义新AI时代“智算中枢”

4月18日，2024九章云极DataCanvas智算操作系统新品发布会于北京隆重召开，全新产品DATACANVAS AIDC OS智算操作系统（以下简称AIDC OS）正式官宣。

[datayuan]

一切为了应用！九章云极DataCanvas大模型系列成果重磅发布！

11月21日，「筑基赋能 智向未来」九章云极DataCanvas大模型系列成果发布会（以下简称“发布会”）在北京重磅召开，本次成果发布距离今年6月30日DataCanvas Alaya九章元识大模型公布仅4个多月，是九章云极DataCanvas公司大模型技术应用创新能力的实力印证。