How to reject a corrupted file if a column is not found? | Community
Skip to main content
KotiSyamala
KotiSyamala
Level 5
June 30, 2021
Solved

How to reject a corrupted file if a column is not found?

  • June 30, 2021
  • 3 replies
  • 2015 views

Dear Adobe Team,

We are unable to reject a file in a technical workflow if the specified csv file columns are not found in a file. Is there a way to reject the file to corrupted file in the file transfer or in data loading to send as a complimentary file?

 

Any suggestions would be helpful.

 

Thanks,

Koti.

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by CedricRey

Hi @kotisyamala ,

 

There is not easy way to manage corrupted file with the fileImport activity.

If you know how many line the file could conains, one way could be to  :

 - make mandatory all colunms of the file (don't allow null values for each column instead of "Adobe Campaign Default", 5th columns of the file format wizard)

 - put the maximum errors on the max error count parameter (large number if possible), check the option to get the reject in a file that will create a new transition. If the main transition count 0 lines that could indicate the input file has a missing column. This could be easy to test (test activity, vars.recCount == 0) and you'll get all the lines into the error file (indicated by the vars.filename of the complementary transition).

 

I'll be interested if anyone has a more elegant solution.

 

Cedric

3 replies

CedricRey
CedricReyAccepted solution
Level 5
June 30, 2021

Hi @kotisyamala ,

 

There is not easy way to manage corrupted file with the fileImport activity.

If you know how many line the file could conains, one way could be to  :

 - make mandatory all colunms of the file (don't allow null values for each column instead of "Adobe Campaign Default", 5th columns of the file format wizard)

 - put the maximum errors on the max error count parameter (large number if possible), check the option to get the reject in a file that will create a new transition. If the main transition count 0 lines that could indicate the input file has a missing column. This could be easy to test (test activity, vars.recCount == 0) and you'll get all the lines into the error file (indicated by the vars.filename of the complementary transition).

 

I'll be interested if anyone has a more elegant solution.

 

Cedric

    Krishnanunni
    Krishnanunni
    Level 4
    July 1, 2021

    Hi @kotisyamala ,


    I don't think you might be able to check the file header efficiently using data loading itself. I would suggest you use a shell script activity or a JS activity for this evaluation. The shell script could be as following:

     

    fileA="path to the file downloaded"
    fileB="path to file with header"

    head -1 $fileA | grep "$(cat $fileB)"
    if [[ $? -ne 0 ]] ;
    then
    echo "fileA header does not match with fileB"
    exit 1;
    fi

     

    You may also use an Advanced JavaScript code activity with the following code:

     

    var fileA="path to the file downloaded" var fileB="path to file with header" var file = new File(fileA); file.open("r", File.CODEPAGE_UTF8); var headerOfDownload = file.readln(); var originalHeader = loadFile(fileB,"utf-8") logInfo(headerOfDownload); logInfo(originalHeader); if(headerOfDownload.trim() == originalHeader.trim()) { task.postEvent(task.transitionByName('match')); } else { task.postEvent(task.transitionByName('nomatch')); }

     

     

    You can use the 'nomatch' transition to alert the respective operators. Hope it helps!

    Krishnanunni
    Krishnanunni
    Level 4
    July 1, 2021

    Hi @kotisyamala ,

    I don't think you might be able to check the file header efficiently using data loading itself. I would suggest you use a shell script activity or a JS activity for this evaluation. The shell script could be like
    fileA="path to the file downloaded"
    fileB="path to file with header"

    head -1 $fileA | grep "$(cat $fileB)"
    if [[ $? -ne 0 ]] ;
    then
    echo "fileA header does not match with fileB"
    exit 1;
    fi

     

    You may use an Advanced JavaScript code activity instead of shell script activity, with the following code:

    var fileA="path to the file downloaded" var fileB="path to file with header" var file = new File(fileA); file.open("r", File.CODEPAGE_UTF8); var headerOfDownload = file.readln(); var originalHeader = loadFile(fileB,"utf-8") logInfo(headerOfDownload); logInfo(originalHeader); if(headerOfDownload.trim() == originalHeader.trim()) { task.postEvent(task.transitionByName('match')); } else { task.postEvent(task.transitionByName('nomatch')); }

    You can use the 'nomatch' transition to alert the respective operators. Hope it helps.

    I
    Indra123
    November 24, 2021

    Hi @krishnanunni  ,

    How can we read sample file and input file, is there any way to count no of filled column??.

     

    Thanks,

    Indra Verma

    Krishnanunni
    Krishnanunni
    Level 4
    November 25, 2021

    Hi @indra123 ,

    Using shell script, you could read a csv file from SFTP and check the number of empty columns as 

    cat file.csv | head -n 2 | tail -n 1 | grep -c ',,'

    As the number of columns would be static, you could calculate the non empty columns.

    Terms & ConditionsAccessibility statement

    Loading footer...