From the course: End-to-End Real-World Data Engineering Project with Snowflake

Unlock this course with a free trial

Join today to access over 24,900 courses taught by industry experts.

Create the task to merge customer changes in the Silver layer

Create the task to merge customer changes in the Silver layer - Snowflake Tutorial

From the course: End-to-End Real-World Data Engineering Project with Snowflake

Start my 1-month free trial Buy for my team

Create the task to merge customer changes in the Silver layer

“

- [Presenter] In the silver layer, when we are going to load the data, we have to do some kind of a cleanup and transformation as per the business logic. For the customer, the transformations and the cleanup needed is as follows. First, we have been asked to do the email validation to remove all the email, which have a email ID as null. We also expected to do a customer type standardization. For some rows it might be regular, some rows it could be premium or unknown. Maybe sometimes they use RM or something like that. So let's standardize that as well. Do an age validation. We only need customers between 18 to 120. If age is beyond this, probably that's a wrong value. Similarly to the standardization for the gender type as well. And finally, we need to check the total price. The total price cannot be negative. If it is negative number, set it to be zero. Great. Now to automatically do this kind of a transformations and load the data into the silver table, we're going to create this…

Contents