0

I want to create DLT pipeline like following. I have streaming bronze table which looks like:

Name City Country
John Tokyo Japan
Mary New York USA
Sam Tokyo Japan

Now I need two more tables. Main dimension and address codebook. Generated identity column or something like this. So, the pipeline should continue with following results:

streaming table silver_address

ID City Country
1 Tokyo Japan
2 New York USA

streaming table silver_people

ID Name Address_ID
1 John 1
2 Mary 2
3 Sam 1

How should the streaming declarations for the next two tables look in SQL?

Improve this question
2
  • If you're just trying to generate a foreign key, take a look at the uuid function. Or build a hash from your key columns. Commented Aug 1 at 17:32
  • thanx for the idea, but I believe the performance of joins etc is poor compared to sequential integer... any other options? Commented Aug 1 at 22:24
Related questions
0
Databricks DLT Syntax for Read_Stream Union
0
DLT - Views v Materialized Views syntax and how to declare?
0
count the number of rows in query
Load 3 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.