Creating JSONL in Fusion | Community
Skip to main content
Rafal_Bainie
Community Advisor
Rafal_BainieCommunity Advisor
Community Advisor
March 28, 2023
Solved

Creating JSONL in Fusion

  • March 28, 2023
  • 2 replies
  • 962 views

Hi everyone,

I'm seaking advice on how to create a JSONL file using Fusion.

I start with CSV file and my current flow is very simple, I get JSON for every CSV row and essentially I want to aggregate them into such a sample output:

{"id":1,"name":"Johnson, Smith, and Jones Co.","amount":345.33,"Remark":"Pays on time"} {"id":2,"name":"Sam \"Mad Dog\" Smith","amount":993.44,"Remark":""} {"id":3,"name":"Barney & Company","amount":0,"Remark":"Great to work with\nand always pays with cash."} {"id":4,"name":"Johnson's Automotive","amount":2344,"Remark":""}

Any advice would be appreciated.

For the context I need JSONL to feed into OpenAI and train the model with our own WF data.

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by Rafal_Bainie

Thanks Randy,
I ended up converting the file outside of Fusion as it turned out to be way simpler than I expected.

It now looks different, but basically let's say input CSV had 2 columns called Prompt and Completion and expected output is such JSON for every row:

{"prompt":"Company: Loft conversion specialists\nProduct: -\nAd:Straight teeth in weeks!\nSupported:", "completion":" no"}

2 replies

Rafal_Bainie
Community Advisor
Rafal_BainieCommunity AdvisorAuthor
Community Advisor
March 29, 2023

While I still need to work on the data set I want to upload here is HTTP module configuration if anyone was looking for something similar:

 

RandyRoberts
Community Advisor
RandyRobertsCommunity Advisor
Community Advisor
March 29, 2023

What does the output look like from the CSV? 

I just had something similar where I was getting an array returned (but with a blank record) and I needed to turn it into separate bundles so I could filter out the empty one(s) or get a parse error every time.

I had to use an iterator, set a variable and then iterate that as well, in order to parse it.

Your's isn't the exact same situation but all you need for a JSONL return is each collection on a new line (a "\r \n" at the end of each line), right?

In this thread, I documented the process I used. I hope this helps give you some ideas.

Rafal_Bainie
Community Advisor
Rafal_BainieCommunity AdvisorAuthorAccepted solution
Community Advisor
March 30, 2023

Thanks Randy,
I ended up converting the file outside of Fusion as it turned out to be way simpler than I expected.

It now looks different, but basically let's say input CSV had 2 columns called Prompt and Completion and expected output is such JSON for every row:

{"prompt":"Company: Loft conversion specialists\nProduct: -\nAd:Straight teeth in weeks!\nSupported:", "completion":" no"}
Terms & ConditionsAccessibility statement

Loading footer...