Google Cloud Launches BigLake, a Unified Data Storage Option
Google has taken a progressive step in establishing cross-platform data harmony with the BigLake launch. The new tool is a data lake storage engine that aims to ease the data management for enterprises storing their data in warehouses and data lakes.
Google’s BigLake storage engine is to propel Google’s BigQuery data warehouse usage experience and further its reach to Google Cloud’s data lakes. The tool would effectively combine the warehouse experience along with data lakes usage.
In addition, the tool would wick away the issues such as storage formats incompatibility and other such issues.
Useful Link: 7 Most-Productive Google Cloud Tools One Must Have!
What is BigLake?
What makes BigLake better than its competition is its cross-platform compatibility. The data can be stored in any one of the warehouses. Be it AWS S3 or Azure Data Lake Storage Gen 2, or Google’s BigQuery; the user would be able to access the data using BigLake as their single point of contact.
This option of one uniform storage engine eliminates the requirement to duplicate or migrate data to avoid compatibility hassles.
“Managing data across disparate lakes and warehouses creates silos and increases risk and cost, especially when data needs to be moved,” said Gerrit Kazmaier, VP and GM of Databases, Data Analytics and Business Intelligence at Google Cloud, notes in today’s announcement.
“BigLake allows companies to unify their data warehouses and lakes to analyze data without worrying about the underlying storage format or system, which eliminates the need to duplicate or move data from a source and reduces cost and inefficiencies.”
BigLake is easy to configure as it provides the admins with policy tags that allow them to incorporate their changes at table, row, or column level.
The policy tags would allow the administrators to make their changes on data stored on Google Cloud Storage and even on third-party applications, which is made possible by BigQuery Omni, Google’s multi-cloud service.
BigLake’s Approach to Data Management
The security mechanism allows the data to pass seamlessly and securely into the tools used by the user. In addition, while supporting tools such as Spark and TensorFlow, Google went the extra mile by integrating the superior data management capabilities of Google’s Dataplex offering.
BigLake is endowed with controls and access, which allow users to extend the application programming interface to Google Cloud. Furthermore, additional control is lent to the users, which allows them to operate file formats that are even compatible with Apace Parquet and Apache Spark.
“The volume of valuable data that organizations have to manage and analyze is growing at an incredible rate,” Google Cloud software engineer Justin Levandoski and product manager Gaurav Saxena announced.
“This data is increasingly distributed across many locations, including data warehouses, data lakes, and NoSQL stores. As an organization’s data gets more complex and proliferates across disparate data environments, silos emerge, creating increased risk and cost, especially when that data needs to be moved. Our customers have made it clear; they need help.”
While Google Cloud is flexible, it is arduous to navigate through the cloud as it is pretty expansive. It is wise to rope the services of Veritis. From Fortune 500 to emerging companies, we have advised and tailored solutions to clients with various unique needs.
So, reach out to us and walk away with a solution that suits you best.