#SpannerGraph

[govindhtech]

Use Spanner Graph And Full-text Search For Deeper Insights

Organizations struggle to glean insightful information from constantly growing amounts of data in today’s data-driven environment. Complex relationship modeling is a strength of graph databases, and full-text search effectively extracts relevant information from unstructured data. Separate system maintenance for these functions, however, may result in overhead and delayed insights. A single, tightly integrated solution combining both functionalities is provided by Spanner Graph. We’ll explore the features and benefits of using Spanner Graph with full-text search in this blog post.

Full-text search and graphing are an effective combination

Since graphs naturally convey links in data, they are an excellent tool for studying linked data, spotting hidden patterns, and enabling applications that depend on connection knowledge. Graph databases excel at handling data’s intricate web of links, making them useful for social networks, recommendation engines, fraud detection, and supply chain management.

- Advertisement -

Unstructured data emails, text documents, social network posts, and consumer reviews are also abundant. Full-text search engines can index, discover, and locate relevant material in massive unstructured data warehouses. Knowledge discovery, customer service, and content management require full-text search to swiftly and conveniently find information. While full-text search and graph databases are valuable individually, they work best together.

Assume you are creating an online store: Your website quickly finds related products when a consumer searches for “waterproof hiking boots” by using a full-text search on product descriptions. Next, using the power of graphs, your recommendation engine examines the customer’s previous purchases of hiking socks and backpacks as well as the purchasing patterns of other customers who have made comparable boot purchases. By combining these facts, your website may effectively cross-sell and improve the user’s buying experience by returning waterproof hiking boots that match the user’s search as well as recommending related products like trekking poles that the user hasn’t yet bought. Full-text search and graph combine to enable this customized method.

It is not ideal to use many specialized systems

Currently, managing full-text and graph data frequently requires using two different kinds of systems: search engines and graph databases. This presents a number of difficulties:

Problems with data duplication and synchronization: Complex ETL processes for data copying and transformation are required to maintain consistency across distinct graph and full-text search engines. This results in the waste of important resources and increases the possibility of mistakes and delays.

Increased operational complexity and maintenance costs are a result of operating a large number of specialized services. Every service requires configuration, security upgrades, monitoring, and sometimes troubleshooting, which takes time and specialized knowledge.

Performing integrated searches and analysis is hampered by the division of full-text and graph search functions. It is frequently necessary to manually combine and correlate the results of different queries in order to bring together insights from both areas.

Impact on real-time application responsiveness: Applications where quick insights are critical, such as fraud detection, customer support, and real-time recommendations, may be adversely affected by the inherent latency brought on by data synchronization and independent query processing.

Spanner Graph integrates full-text search and graph functionality into a single system

Spanner Graph integrates Spanner, its globally consistent, always-on database with built-in graph features, to create an almost infinitely scalable database. One system has a tight integration of graph, full-text search, and AI features. The integrated full-text search offers tried-and-true technology, the foundation of many current Google products. It incorporates fuzzy matching for character variations and names with similar sounds in addition to precise or partial matches, result scoring, and automatic language detection. Additionally, it makes use of AI to interpret search terms, managing synonyms and spelling checks. It also enables sophisticated search queries with logical operators, just like the web search function on Google.

- Advertisement -

Graph queries and full-text search are closely coupled so that you can use one to explore relationships inside graph structures and the other to retrieve nodes or edges based on text content. This unified capability, which serves as your single source of truth, unifies two complementary methodologies to reveal insights, patterns, and hidden linkages across unstructured and graph data, all within one system.

Establish a full-text search index on the edges and nodes of the graph

Creating a search index on the graph node and edge properties you wish to search on is the first step towards using full-text search with Spanner Graph. Tables and their columns are mapped to graph nodes, edges, and their attributes using the Spanner Graph. You can establish a search index on the related table columns to enable full-text search on a text property of graph nodes and edges.

You first build a retail graph using a condensed example from the e-commerce domain, in which the product and the user are entities and the relationships are past purchases:

Add a new column descriptionToken of type TOKENLIST to the underlying Product table to store the tokenized content of the description column in order to enable product searches based on this property:

Next, a search index on the tokenized description can be created:

To locate graph nodes, using full-text search

Full-text search on Spanner Graph can be accessed via the SEARCH function once the search index has been constructed. Two parameters are passed to the SEARCH function: the property to search on and the input search query. The product nodes that have the term “waterproof hiking boots” in their description are returned in the example below:

Integrate graph traversals with full-text search

Combining graph traversals with full-text search yields the true power. Full-text search can be used to rapidly identify pertinent graph nodes that serve as the foundation for more investigation. After that, to navigate relationships, you can use graph queries. It would be challenging to find hidden connections, patterns, and insights using each technique alone; but, this combination reveals them.

Expanding on the last example, the following scenario shows graph traversal in the context of recommendation engines, beginning with products found using full-text search. The graph query finds users who have already bought each product. It then finds other things that those users have purchased as well, and returns those results as product recommendations based on popularity. The number of distinct buyers who have previously bought the item is used to determine how popular the product is. Please take note that the things the current user has already purchased cannot be returned. This procedure successfully exposes the client base’s connected preferences and patterns of buying.

Combined table data and query graph

By enabling complete compatibility between SQL and the Graph Query Language (GQL), Spanner Graph connects the graph and relational languages. GQL allows you to navigate through your graphs and join the results with tables in a single query. This gives you the most freedom to select the appropriate tool for the task at hand and achieve the best possible results. An even more basic recommendation system that makes use of both graphs and full-text search is demonstrated in the example that follows. In addition to making product recommendations, this query uses SQL to get the suggested products’ historical pricing data from a table, giving consumers insightful information about price trends and fluctuations.

Begin now

Discover hidden insights in your data with the potent mix of full-text search, graph, and SQL interoperability. This will change the way you examine, evaluate, and comprehend your information landscape.

Read more on govindhtech.com

[govindhtech]

Spanner’s Standard Edition To Create Solid, intelligent apps

Spanner is the globally consistent, always-on database with almost infinite scalability that powers a plethora of contemporary applications in many industries. It is now the cornerstone of many forward-thinking companies’ data-driven transformations. Google is launching Spanner throughout graph today, along with new features like vector search, full-text search, and more. With the help of these advancements, businesses can create intelligent apps that seamlessly integrate several data models and condense their operational deployments onto a single, potent database.

From Spanner versions, you may choose the Standard Edition, Enterprise, and Enterprise Plus editions with the right set of capabilities to fit your needs and budget.

With Spanner Graph, you can easily and succinctly match patterns, navigate relationships, and filter results across interconnected data for popular graph use cases like fraud detection, community discovery, and personalised recommendations.

Building on Google’s many years of search experience, advanced full-text search offers stronger matching and relevance ranking than unstructured text.

Vector search builds on more than a decade of Google research and innovation in approximate closest neighbour algorithms to facilitate semantic information retrieval, the cornerstone of generative  AI applications.

Furthermore, in response to Google’s enterprise customers’ increasingly complicated needs regarding latency, cost, and compliance, Google has improved Spanner’s basic infrastructure.

With geo-partitioning, you can store some of your data in particular places to provide quick local access while still deploying globally.

Data sovereignty needs are respected and multi-regional availability attributes are provided via dual-region arrangements.

By dynamically adjusting the size of your Spanner deployment, auto-scaling frees you from having to overprovision and enables you to respond swiftly to changes in traffic patterns.

Presenting the Spanner editions

Google is pleased to present Spanner editions, a new tiered pricing structure that enables you to benefit from these advancements while improving cost transparency and cost-saving options. You can select the Standard, Enterprise, and Enterprise Plus editions from Spanner editions to match your demands and budget with the appropriate collection of features.

The Standard edition is a feature-rich package that comes with additional features like scheduled backups and reverse ETL from BigQuery in addition to all of the current capabilities that are generally available (GA) in single-region settings. The Enterprise edition leverages multi-model features, such as vector search, enhanced full-text search, and the new Spanner Graph, to enhance the Standard edition.

Moreover, it offers improved data security and operational ease with incremental backups and automated autoscaling. The new geo-partitioning feature and multi-region setups with 99.999% availability are included in the Enterprise Plus version for workloads that demand the greatest levels of availability, performance, compliance, and governance.

Let’s examine each of the Spanner editions in more detail

Standard Edition

Everything you adore about Spanner is present in the standard edition, and then some. Access to Spanner’s single-region configurations, which include transparent failover, synchronous replication, and automatic sharding for almost infinite scalability and 99.99% availability, is provided by the Standard edition. All of the features available in the GA today, including Google’s enterprise-grade security and data protection measures, are included in the standard edition.

Furthermore, Google is adding new features to the Standard edition. Google is growing their offering with additional features like reverse ETL and external datasets with BigQuery, in addition to Spanner’s strong connection with BigQuery, which includes features like Data Boost and BigQuery federation. Google is also added scheduled backups to further its data protection capabilities. These improved features in the Standard edition are available to all Spanner customers.

Customers can gain from Standard edition in the following ways:

Migrate your relational workloads to Spanner to save operational toil, minimise costly scheduled and unplanned downtime, and cut the overall cost of ownership.

Shard MySQL or other complicated application-level sharding using Spanner’s automatic sharding instead.

Spanner’s nativePostgreSQL interface lets you modernise your apps without being locked into outdated features.

Elevate your NoSQL workloads with multirow transactions and stable secondary indexes thanks to support for adapters for Cassandra and DynamoDB that are compatible with APIs.

Avoid complex ETL pipelines and data silos by consolidating your data into a single source-of-truth database and running analytics on up-to-date data with Spanner Data Boost and BigQuery federation.

Utilise reverse ETL to service complicated analytics use-cases from Spanner and BigQuery to compute them at high QPS and low latency.

Enterprise Edition

Expanded data modalities are available to you with the Enterprise edition, enabling you to combine your operational deployments and provide your applications with true multi-model access. Spanner Graph, full-text search, and vector search capabilities are available for your data with the Enterprise edition, in addition to the relational and NoSQL functionality that Spanner currently offers.

The best part is that by combining all of this functionality, Spanner’s primary advantages of high availability, scalability, and consistency can be preserved in a genuine, completely interoperable multi-model database. Having all of your data in one location eliminates the need to transfer data between transactional and speciality databases, allowing you to minimise data silos and duplication, streamline data governance, cut down on operational complexity, and eventually minimise total cost of ownership.

Customers can gain from Enterprise edition in the following ways:

By fusing the advantages of SQL and GQL, you can supercharge your applications with Spanner Graph capabilities at nearly infinite scale. This allows analysts and developers to query linked and structured data in a single action.

Use full-text search to modernise your search apps and streamline your operational pipelines. You may leverage sophisticated search features without copying and indexing data into a separate search solution.

By basing fundamental models on contextual, domain-specific, and real-time data with native vector search capability in your operational data, you may enhance the AI-assisted user experience.

Launch robust multi-model applications, such a generative  AI product that grounds LLMs using relational and knowledge graph RAG approaches, or a searchable product catalogue that mixes traditional and semantic information retrieval through the combination of full-text and vector search.

Enterprise Plus Edition

Spanner’s most feature-rich and widely available edition is called Enterprise Plus. Enterprise Plus includes new dual-region configurations in addition to our 99.999% multi-region options, which let you take advantage of Spanner’s unparalleled availability while also adhering to your data sovereignty needs. You may run a truly global deployment with Enterprise Plus and still retain low latency and data locality for parts of your data by utilising geo-partitioning.

Customers can gain from Enterprise Plus edition in the following ways:

By using Spanner’s multi-region settings for application deployment, you can achieve real 0-RTO and 0-RPO deployments along with 99.999% availability, eliminating the need to construct and manage complex disaster recovery solutions.

Use Spanner’s dual-region configurations in places like Australia, Germany, India, and Japan to meet data sovereignty needs without sacrificing availability.

Partition your data to make it always available where you need it, latency-free, and manage your worldwide deployment while preserving data locality.

Increased openness and cost efficiency

Google is also introducing a more straightforward pricing approach with the new Spanner editions, shifting to per-replica charging and separating the expenses of compute and data replication to improve efficiency and cost transparency. Each edition has various cost-saving features, and switching to a new edition is meant to be a smooth process.

Customers in a single region can purchase the standard edition for the same price. Instead than paying multiple prices for every function, customers can choose the Enterprise version, which has a single price point, if they want increased multi-model and search capabilities. They can also take advantage of the price-performance improvements like incremental backups and managed autoscaler.

Even more, the Enterprise Plus edition offers lower compute expenses, separates computation and replication, and does not charge for witness replica storage. Finally, since configurable read-only replicas will be priced the same as read-write replicas for each edition, customers employing these replicas will benefit from significant cost savings. Making the switch to Spanner editions is easy, and you can take advantage of all the improvements Spanner has to offer while cutting prices.

The availability of the Spanner versions is scheduled for September 24, 2024. See the overview page for Spanner versions for further details.

Read more on govindhtech.com

[govindhtech]

Presenting Graph Spanner: A Reimagining of Graph Databases

Spanner Graph

Operational databases offer the basis for creating enterprise AI applications that are precise, pertinent, and based on enterprise truth as the use of  AI continues to grow. At Google Cloud, the team strive to provide the greatest databases for creating and executing AI applications. In light of this, they are thrilled to introduce Graph Spanner, vector search, and sophisticated full-text search a few new features that will make it simpler for you to create effective gen AI products.

With the release of Bigtable SQL and Bigtable distributed counters, the team also revolutionising the developer experience by simplifying the process of creating at-scale apps. Lastly, they are announcing significant updates to help customers modernise their data estates by supporting their traditional corporate workloads, such as SQL Server and Oracle. Now let’s get started!

Using a graph database can be a valuable, albeit complex, approach for enterprises to gain insights from their connected data so they can create more intelligent applications. Google is excited to present Spanner Graph today, a ground-breaking solution that combines Spanner, Google’s constantly available, globally consistent database, with capabilities specifically designed for graph databases.

Graphs are a natural way to show relationships in data, which makes them useful for analysing data that is related, finding hidden patterns, and enabling applications that depend on connection knowledge. There are several applications for graphs, including route planning, customer 360, fraud detection, recommendation engines, network security, knowledge graphs, and data lineage tracing.

But implementing separate graph databases to handle these use cases frequently comes with the following drawbacks:

Data fragmentation and operational overhead: Separate graph database maintenance frequently results in data silos, added complexity, and inconsistent data copies, all of which make it more difficult to conduct effective analysis and decision-making.

Bottlenecks in terms of scalability and availability: As data volumes and complexity increase and impede corporate growth, many standalone graph databases find it difficult to fulfil the expectations of mission-critical applications in terms of scalability and availability.

Ecosystem friction and skill gaps: Adopting a fully new graph paradigm may be more difficult for organisations because of their significant investments in infrastructure and SQL knowledge. They require more resources and training to accomplish this, which could take resources away from other pressing corporate requirements.

With almost infinite scalability, Graph Spanner reinvents graph data management by delivering a unified database that seamlessly merges relational, search, graph, and  AI capabilities. Spanner Graph provides you with:

Native graph experience: Based on open standards, the ISO Graph Query Language (GQL) interface provides a simple and clear method for matching patterns and navigating relationships.

Unified relational and graph models: Complete GQL and SQL interoperability eliminates data silos and gives developers the freedom to select the best tool for each given query. Data from tables and graphs can be tightly integrated to reduce operational overhead and the requirement for expensive, time-consuming data transfers.

Built-in search features: Graph data may be efficiently retrieved using keywords and semantic meaning thanks to rich vector and full-text search features.

Scalability, availability, and consistency that lead the industry: You can rely on Spanner’s renowned scalability, availability, and consistency to deliver reliable data foundations.

AI-powered insights: By integrating deeply with Vertex AI, Spanner Graph gains direct access to a robust set of AI models, which speeds up AI workflows.

Let’s examine more closely what makes Spanner Graph special

Spanner Graph provides a recognisable and adaptable graph database interface. Supported by Graph Spanner is ISO GQL, the latest global standard for graph databases. It makes it simpler to find hidden links and insights by providing a clear and simple method for matching patterns, navigating relationships, and filtering results in graph data.

Spanner Graph functions well with both full-text and vector searches. With the combination, you may use GQL to navigate relationships within graph structures and search to locate graph contents. To be more precise, you can use full-text search to identify nodes or edges that include particular keywords or use vector search to find nodes or edges based on semantic meaning. GQL allows you to easily explore the remainder of the graph from these beginning locations. This unified capability enables you to find hidden connections, patterns, and insights that would be challenging to find using any one method by combining various complimentary strategies.

Spanner Graph provides industry-leading consistency and availability while scaling beyond trillions of edges. Since Graph Spanner inherits Spanner’s nearly limitless scalability, industry-best availability, and worldwide consistency, it’s an excellent choice for even the most crucial graph applications. Specifically, without requiring your involvement, Spanner’s transparent sharding may leverage massively parallel query processing and scale elastically to very huge data sets.

Spanner Graph’s tight integration with Vertex  AI speeds up your AI workflows. Spanner Graph has a close integration with Vertex AI, the unified, fully managed AI development platform offered by Google Cloud. Using the Graph Spanner schema and query, you may immediately access Vertex AI’s vast collection of generative and predictive models, which can expedite your AI workflow. To enrich your graph, for instance, you can utilise LLMs to build text embeddings for graph nodes and edges. After that, you can use vector search to extract data from your graph in the semantic space.

You may create more intelligent applications with Spanner Graph

With practically limitless scalability, Spanner Graph effortlessly combines graph, relational, search, and AI capabilities, creating a plethora of opportunities:

Product recommendations: To create a knowledge network full of context, Spanner network represents the intricate interactions that exist between people, products, and preferences. By fusing full-text search with quick graph traversal, you may recommend products based on user searches, past purchases, preferences, and similarity to other users.

Financial fraud detection: It is simpler to spot questionable connections when financial entities like accounts, transactions, and people are represented naturally in Spanner Graphs. Similar patterns and anomalies in the embedding space are found by vector search. Financial institutions can minimise financial losses by promptly and correctly identifying possible dangers through the combination of these technologies.

Social networks: Even in the biggest social networks, the Graph Spanner model logically individuals, groups, interests, and interactions. For individualised recommendations, it facilitates the quick identification of trends like overlapping group memberships, mutual friends, or related interests. Users can quickly locate individuals, groups, posts, or particular subjects by using natural language searches with the integrated full-text search feature.

Gaming: Player characters, objects, places, and the connections between them are all natural representations of elements in game environments. Effective link traversal is made possible by the Spanner Graph, and this is crucial for game features like pathfinding, inventory control, and social interactions. Furthermore, even at times of high usage, Spanner Graph’s global consistency and scalability ensure a smooth and fair experience for every player.

Network security: Recognising trends and abnormalities requires an understanding of the relationships that exist between individuals, devices, and events over time. Security experts can employ graph capabilities to trace the origins of attacks, evaluate the effect of security breaches, and correlate these findings with temporal trends for proactive threat detection and mitigation thanks to Graph Spanner relational and graph interoperability.

GraphRAG: By utilising a knowledge graph to anchor foundation models, Spanner Graph elevates Retrieval Augmented Generation (RAG) to a new level. Furthermore, Spanner Graph’s integration of tabular and graph data enhances your AI applications by providing contextual information that neither format could provide on its own. It is capable of handling even the largest knowledge graphs due to its unparalleled scalability. Your GenAI workflows are streamlined with integrated Vertex  AI and built-in vector search.

Read more on govindhtech.com