Today at AWS re:Invent in Las Vegas, Amazon made a few announcements that contribute to the future of data management without the need for extract, transform, and load, or ETL.
Every data scientist and the team struggles with ETL as they strive to prepare the data for use. You may have data in a variety of locations, such as your application usage data in a database and customer reviews in your data lake, as discussed by AWS CEO Adam Selipsky. It has been quite difficult to put them together until now.
To enable users of the Aurora database and the Redshift data warehouse to transport data without performing ETL, AWS introduced the Aurora zero-ETL interface with Redshift.
Selipsky told the re:Invent audience, “We’ve been working for a few years now and establishing connectors amongst our services to make it easier to do analytics and the machine learning without having to deal with ETL.”
What if we could accomplish more? What if we could completely do away with ETL? We would all like that kind of world. This is what we mean when we talk about a future with no ETL. Data integration is no longer a manual process in the future, he added. The preview of a completely managed new ETL-free connection between Aurora and Redshift is what I’m excited to announce today.
While he was at it, he also disclosed a comparable integration between Apache Spark, a well-known open-source big data processing tool, and Amazon Redshift. Without needing to extract, process, and load data first, it gives equivalent capabilities to transport data between the two platforms.
Aurora and Redshift have a preview integration. All regions now offer the Redshift-Apache Spark integration.
Visit CxO Global FORUM or CxO News Live for all the latest updates.
