How to zip .csv file over SFTP or in Export Activity

[#1]

Hi,

How to Zip .csv file which is on SFTP or

is it possible to zip .csv file in export activity so as to compress it.

Thanks

[#2]

Hi Ketan,

There is an unzip utility called gunzip which can be installed in a linux server and then used to compress and decompress files in Adobe campaign.

For example you can use gzip -filename to compress the file or gzip -d filename.gz to uncompress it.

Please note that the gunzip utility needs to be present on the server and it will zip files in the .gz format and not.zip format

Regards,

Adhiyan

[#3]

Hi Ketan,

Apart from what Adhiyan suggested you can use a script to zip your file in export activity.

var zip = new ZipFile("fileName.zip");

var content = new MemoryBuffer();

content.fromString("string if you have data as string else other options to get files content");

zip.setEntry(fileName, content)

content.dispose();

zip.save("", true);

//Use file io to export if you do not want to use export activity

Regards,

Amit

[#4]

Thanks Adhiyan,Amit.

I have tried both ways first solution requires gunzip utility which is currently not installed over server.

I tried second solution and made some changes to it, below is the script

var zip = new ZipFile("/path/Data.zip")

var content = new MemoryBuffer();

content.load("/path/Data.csv");

zip.setEntry("/path/Data.csv",content);

content.dispose();

zip.save("", true);

In above script it is throwing file too big error. As file size of CSV is 1.2GB.

Is there any way out for this ?

Thanks

[#5]

You should resolve this problem considering split the file in differents chunks and the merge everything at the end.

There are many ways, libraries to implement  this:

• Java Scanner streaming
• Streaming Apache Commons IO
• Java reader.lines()
• Any library, fmk for big data works as well. 

[#6]

Hi Diego,

I am using following code and I am getting error when the records in the file is above 1 million. The error is "Function 'fromString', argument 0: cannot convert argument to a string". 

var content= new MemoryBuffer();

var x=loadFile(vars.filename);

content.fromString(x);

var zipFileName= "Name of File";

zip.setEntry(zipFileName,content);

zip.save(zipFileName);

zip.dispose();

Can you please let me know in brief how to use these libraries

[#7]

Hi Ketang,

Regarding the Javascript size limitation of Amit's solution (although powerful in many cases), perhaps you can raise the MAX memory of Javascript parameter in serverConf.xml or instance conf.xml.

But, to have no limitations at all and have much more options to do all compression tasks, what I used to do for my own cases is to use 7zip tool (free tool) installed on the server, and because my Adobe Campaign servers are under Windows it is more common to use such tools rather gzip. Windows integrated tool for compressing/uncompressing is not so efficient as 7zip, most of all 7zip command lines options are powerful.

This solution is similar of gzip/gunzip suggested by Adhiyan, and supposed that you have a On Premise hosting and able to install/execute system tools.

So, in your workflow, after the file extraction activity, you can add an external task activity and set it as is:

In the command line of the External task, you write this:

7z a [pathname]\[filenamewithoutextension].zip [pathname]\[filenametocompress]

For instance

7z a D:\Temp\campaignXXX_<%= formatDate(new Date(), "%2D_%2M_%4Y") %>.zip  D:\Temp\campaignXXX__<%= formatDate(new Date(), "%2D_%2M_%4Y") %>.csv

Please note that since build 8795 it is not allowed by default to use External task / System script, for security reasons, except by allowing specifically in serverConf options. So see with your system administrator if he can allow to install and run 7zip, and only On Premise hosting mode is allowed (not Cloud hosting):

Adobe Campaign Release Notes (v7.0)

Adobe Campaign Release Notes (v6.1.1)

Regards.
J-Serge