Databricks has recently confirmed that it has acquired Redash Ltd., a startup based in Israel that offers data scientists an open source dashboarding and visualization tool for exploring their vast data easily. Databricks has also introduced a new and high performance query engine, Delta Engine tool, aimed towards cloud-based data lakes.

Apparently, data lakes can be deemed as systems or repositories of data which is stored in their natural format. They represent a single place to store all the data of an enterprise, which includes raw copies of source system as well as transformed data utilized for work like visualization, reporting, machine learning and analytics.

While data lakes are valuable, they could usually be inaccurate or unreliable for various reasons. These might include failed writes, data inconsistencies and schema mismatches. Databricks’ latest Delta Engine tool has been developed for working with the company’s Delta Lake, an open source structured transaction layer introduced in the previous year which helps to improve enterprise data lake efficiencies.

Databricks had claimed last year that the aim behind the Delta Lake project was to make sure data is kept more reliable and accurate. It enables that through the management of transactions on both batch and streaming data, along with multiple simultaneous writes.

Also Read: Microsoft completes takeover of ADRM Software on data lake generation

By enabling firms to build curate data lakes comprising of both structured and semi-structured data, the tool provides quality and reliability to the data lakes for allowing companies to perform faster data analytics.

According to vice president of marketing at Databricks, Joel Minnick, the Delta Engine launched by Databricks will enable faster analytics for enterprises across data stored within their Delta Lakes. It facilitates analyzing data without the need to move it out of the Delta Lake, along with a boost to the query speeds by nearly 8 times, owing to a vectorized query engine.

Minnick added that companies must also understand what these queries communicate and acquiring Redash is a step towards helping them. An open source project, Redash allows data scientists to visualize the results of their queries through charts, cohorts and funnels, which can easily be shared with the users, Minnick noted.

Source Credits: https://siliconangle.com/2020/06/24/databricks-debuts-delta-engine-fast-query-tool-big-data-lakes/