Parse CSV question (column with comma separated values) | Community
Skip to main content
V
viovi
Level 4
January 25, 2024
Solved

Parse CSV question (column with comma separated values)

  • January 25, 2024
  • 1 reply
  • 2202 views

Hi everyone,

 

I have a question reg. parsing CSV. While processing data, it's replacing " with empty string to get clean data.

Everything is working fine until we get comma separated values in one of the columns, for example "Senior Manager Audit, Compliance, and Risk Management" as then Fusion throws an error as it sees it as multiple columns and it does not match the expected number.

I tried to escape it in different ways but was not able to make it work.

Any idea how to do this? 

Could you please advise?

 

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by cverges

I'm attaching a blueprint that you can import and test the functionality of the CSV create/parser mechanism.  Hopefully this helps show you how it works.  If you still have problems, I'd recommend creating an isolated scenario like this that reproduces and then post the blueprint here.

Here's the overall pipeline:

Each of the configs for the modules:

 

 

 And here's what the test run shows:

 

Note how the input values match the ultimate output values.

1 reply

C
cverges
Level 5
January 26, 2024

Ensure your input file conforms to RFC-4180 (https://www.rfc-editor.org/rfc/rfc4180.html).  This is the standard for CSV files and ensures what you're talking about is properly handled.  Fusion's CSV parser is RFC-4180 compliant, and we import files like this all the time.

V
vioviAuthor
Level 4
January 26, 2024

Thank you for suggestion, but the thing is that data is being processed within scenario: at some point it being collected to Datastore and then Fusion is using this data to create the csv file to store and generate report and notices.  And here we have issue with quotes in the a.m. case, that they are either being removed and then we get error with extra columns or if we check "Preserve quotes inside unquoted field", it goes through, but all labels are quoted as "field1", "field2", etc and it shows as empty value in the following modules.

All is working with regular values though, if we remove the field under question.

C
cvergesAccepted solution
Level 5
January 26, 2024

I'm attaching a blueprint that you can import and test the functionality of the CSV create/parser mechanism.  Hopefully this helps show you how it works.  If you still have problems, I'd recommend creating an isolated scenario like this that reproduces and then post the blueprint here.

Here's the overall pipeline:

Each of the configs for the modules:

 

 

 And here's what the test run shows:

 

Note how the input values match the ultimate output values.

    Terms & ConditionsAccessibility statement

    Loading footer...