Expand my Community achievements bar.

SOLVED

Creating JSONL in Fusion

Avatar

Community Advisor

Hi everyone,

I'm seaking advice on how to create a JSONL file using Fusion.

Rafal_Bainie_0-1680013196794.png

I start with CSV file and my current flow is very simple, I get JSON for every CSV row and essentially I want to aggregate them into such a sample output:

{"id":1,"name":"Johnson, Smith, and Jones Co.","amount":345.33,"Remark":"Pays on time"}
{"id":2,"name":"Sam \"Mad Dog\" Smith","amount":993.44,"Remark":""}
{"id":3,"name":"Barney & Company","amount":0,"Remark":"Great to work with\nand always pays with cash."}
{"id":4,"name":"Johnson's Automotive","amount":2344,"Remark":""}

Any advice would be appreciated.

For the context I need JSONL to feed into OpenAI and train the model with our own WF data.

1 Accepted Solution

Avatar

Correct answer by
Community Advisor

Thanks Randy,
I ended up converting the file outside of Fusion as it turned out to be way simpler than I expected.

It now looks different, but basically let's say input CSV had 2 columns called Prompt and Completion and expected output is such JSON for every row:

{"prompt":"Company: Loft conversion specialists\nProduct: -\nAd:Straight teeth in weeks!\nSupported:", "completion":" no"}

View solution in original post

3 Replies

Avatar

Community Advisor

While I still need to work on the data set I want to upload here is HTTP module configuration if anyone was looking for something similar:

Rafal_Bainie_0-1680089978809.png

 

Avatar

Community Advisor

What does the output look like from the CSV? 

I just had something similar where I was getting an array returned (but with a blank record) and I needed to turn it into separate bundles so I could filter out the empty one(s) or get a parse error every time.

I had to use an iterator, set a variable and then iterate that as well, in order to parse it.

Your's isn't the exact same situation but all you need for a JSONL return is each collection on a new line (a "\r \n" at the end of each line), right?

In this thread, I documented the process I used. I hope this helps give you some ideas.

Avatar

Correct answer by
Community Advisor

Thanks Randy,
I ended up converting the file outside of Fusion as it turned out to be way simpler than I expected.

It now looks different, but basically let's say input CSV had 2 columns called Prompt and Completion and expected output is such JSON for every row:

{"prompt":"Company: Loft conversion specialists\nProduct: -\nAd:Straight teeth in weeks!\nSupported:", "completion":" no"}