Skip to main content
Klaster
Level 2
May 27, 2022
Solved

Union All

Hi,

 

I want to simply combine ALL results from two data streams - Union joins data based on reconciliation set chosen.

 

Speaking in SQL - I want to simply UNION ALL data.

 

regards

 

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
CedricRey
CedricReyAccepted solution
Level 5
May 30, 2022

Hi @klaster 

If you speak about a workflow, you have the UNION activity :

But the datas source must be the same type. For example you can't union deliveries and recipients (you'll get an error).

If the datas are the same type (if it is not but linked, you can try to use the "Change Axis" activity to get same type) you'll get all the line in the output transition.

 

Cedric

View in thread

3 replies

CedricRey
CedricReyAccepted solution
Level 5
May 30, 2022

Hi @klaster 

If you speak about a workflow, you have the UNION activity :

But the datas source must be the same type. For example you can't union deliveries and recipients (you'll get an error).

If the datas are the same type (if it is not but linked, you can try to use the "Change Axis" activity to get same type) you'll get all the line in the output transition.

 

Cedric

Translate
    KlasterAuthor
    Level 2
    June 6, 2022

    Hi @cedricrey ,

     

    but union internally uses joins which are terrible with performance on really large datasets.

    I need something which can be compared to "UNION ALL" in SQL - no joining data, no grouping.

     

     

    Translate
    CedricRey
    CedricRey
    Level 5
    June 8, 2022

    Hi @klaster 

    I just tested with the SQL Logs activated, the query is a union between 2 SELECT, and not a "JOIN" (it's not an intersection, wich will search the same key into the 2 tables, but a real union, wich will get ALL the lines from all the input tables).

     

    INSERT INTO wkfXXXXX_W (iId) SELECT iId FROM wkfXXXXX_Y_Z union SELECT iId FROM wkfXXXXX_X_U

    You should test with samples if you're not sure about performance (use the extract activity to get samples of your populations). This is usually a good way to get a performance overview.

    Translate
      Terms & ConditionsAccessibility statement

      Loading footer...