How to download and save a file under our dam directly using document management systems like Dropbox ?

[ArpitBora]

Hi All,

Is there any way to download a file directly from a document management system like dropbox and save it in AEM dam location like "/content/dam/".

I referred this article http://www.tothenew.com/blog/making-dropbox-documents-available-in-aem/ for "Making Dropbox documents available in AEM".

Using above article I'm able to fetch dropbox file metadata information and asset node creation with metadata but I want to download and save the complete file under dam location.

I referred the Dropbox Java Core API v1 as well and found following code for download file from dropbox  https://www.dropbox.com/developers-v1/core/start/java

FileOutputStream outputStream = new FileOutputStream("magnum-opus.txt"); try { DbxEntry.File downloadedFile = client.getFile("/magnum-opus.txt", null,outputStream); System.out.println("Metadata: " + downloadedFile.toString()); } finally { outputStream.close(); }

But here is one problem "How to use OutputStream in AEM" OR is there any other way to download file directly from dropbox and save it in AEM dam.

Please provide your suggestion and answers.

I'm Using AEM 6.2 instance.

Thanks,
Arpit Bora

[smacdonald2008]

You can write a custom service to get assets from the DAM and make them available. See this article to point you in the right direction. (it shows how to handle files - this example use case places the assets into a ZIP and them downloads)

Downloading Adobe Experience Manager DAM Assets using Sling Servlets and the Query Builder API

Hope this helps... 

[ArpitBora]

smacdonald2008 wrote...

You can write a custom service to get assets from the DAM and make them available. See this article to point you in the right direction. (it shows how to handle files - this example use case places the assets into a ZIP and them downloads)

Downloading Adobe Experience Manager DAM Assets using Sling Servlets and the Query Builder API

Hope this helps... 

Thanks for reply Scott, I updated the question.

I want to download file from dropbox and then save it in my AEM dam loacation.

I'm able to get file metadata information and node creation like below :

But instead of storing metadata i want to download file directly from dropbox in my dam.

[smacdonald2008]

Ok - this would still be a Java application that uses the DropBox API and HTTP APIs to post files to the DAM.

You can use DropBox Java API to get file from DropBox. See: 

https://www.dropbox.com/developers-v1/core/start/java

Then once the App has the file, you can post the file to AEM Servlet: 

http://scottsdigitalcommunity.blogspot.ca/2014/03/creating-java-swing-applications-that.html

Notice that you can specify the JCR path  and the MIME type in this example. 

This is how you can address this AEM/DropBox use case. 

[kautuk_sahni]

Hi 

You need to create a custom Service/Component in AEM.

Step 1 :- Custom component to upload asset to DAM by AssetManager API.

//     Uploading files to Adobe Experience Manager DAM using AssetManager API

    Link:- https://helpx.adobe.com/experience-manager/using/uploading-files-aem1.html

Step 2: here in Step 1, you need to modify some bit of code to work on Dropbox API.

//     The Dropbox API is far too complicated for what you need. It's actually extremely simple to download a file from dropbox.

        The first step is to put the file that you want to download somewhere inside your dropbox's Public Folder.

        Next you want to right click that file and choose "copy public link." You can do this from the web interface or even right there in your computer-sync-folder-thing. This will give you a unique download url for the         file.

    Next, use this code:

        String url="https://dl.dropboxusercontent.com/u/73386806/Prune%20Juice/Prune%20Juice.exe";         String filename="PruneJuice.exe";         try{            URL download=new URL(url);           ReadableByteChannel rbc=Channels.newChannel(download.openStream());           FileOutputStream fileOut = new FileOutputStream(filename);          fileOut.getChannel().transferFrom(rbc, 0, 1 << 24);           fileOut.flush();           fileOut.close();           rbc.close();         }catch(Exception e){ e.printStackTrace(); }

    Link:- http://stackoverflow.com/questions/13557630/downloading-file-from-dropbox-in-java

Step 3: Not you can save these assets from Dropbox to DAM.

I hope this will help you.

~kautuk

[ArpitBora · Accepted solution]

Thanks for spending your valuable time @Scott and @Kautuksahni.

Using addRendition(String name, InputStream is, String mimeType)method from Asset interface i'm able to download dropbox file and save/upload it on my dam location.

Following is the code snippet :

InputStream inputStream = null; try { inputStream = getInputStream(downloadUrl); if (is.available() > 0) { Resource resource = resourceResolver.resolve(documentNode.getPath()); Asset asset = DamUtil.resolveToAsset(resource); if (asset != null) { asset.addRendition("original", inputStream, mimeType); session.save(); } inputStream.close(); } } catch (IOException io) { log.info(":::::=============== IOException ===============:::::"); log.info("-------------------> Exception ==> {}", io.getMessage()); io.printStackTrace(); }

In above code, I'm passing downloadUrl and mimeType, these values I'm fetching from dropbox file using "Dropbox Java Core API v1".

Thanks and Regards,

Arpit Bora

[tyronet42925573]

package com.adobe.cq.sling.download;

/*

Code from

https://helpx.adobe.com/experience-manager/using/downloading-dam-assets.html

*/

import java.io.BufferedReader;

import java.io.IOException;

import java.io.InputStream;

import java.io.InputStreamReader;

import java.io.BufferedInputStream;

import java.io.PrintWriter;

import java.net.HttpURLConnection;

import java.net.URL;

import java.rmi.ServerException;

import java.util.Dictionary;

import java.util.HashMap;

import java.util.Map;

import java.io.*;

import java.util.ArrayList;

import java.util.zip.ZipEntry;

import java.util.zip.ZipOutputStream;

import java.util.Iterator;

import javax.servlet.ServletOutputStream ;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

import org.apache.felix.scr.annotations.Properties;

import org.apache.felix.scr.annotations.Property;

import org.apache.felix.scr.annotations.Reference;

import org.apache.felix.scr.annotations.sling.SlingServlet;

import org.apache.sling.api.SlingHttpServletRequest;

import org.apache.sling.api.SlingHttpServletResponse;

import org.apache.sling.api.servlets.SlingSafeMethodsServlet;

import org.apache.sling.commons.osgi.OsgiUtil;

import org.apache.sling.jcr.api.SlingRepository;

import org.apache.felix.scr.annotations.Reference;

import org.osgi.service.component.ComponentContext;

import javax.jcr.Session;

import javax.jcr.Node;

//Sling Imports

import org.apache.sling.api.resource.ResourceResolverFactory ;

import org.apache.sling.api.resource.ResourceResolver;

import org.apache.sling.api.resource.Resource;

//QueryBuilder APIs

import com.day.cq.search.QueryBuilder;

import com.day.cq.search.Query;

import com.day.cq.search.PredicateGroup;

import com.day.cq.search.result.SearchResult;

import com.day.cq.search.result.Hit;

//DAM API

import com.day.cq.dam.api.Asset ;

@SlingServlet(paths="/bin/myDownloadServlet", methods = "GET", metatype=true)

public class DownloadAssets extends org.apache.sling.api.servlets.SlingAllMethodsServlet{

    //Set up References

    /** Default log. */

    protected final Logger log = LoggerFactory.getLogger(this.getClass());

    private Session session;

    //Inject a Sling ResourceResolverFactory

    @Reference

    private ResourceResolverFactory resolverFactory;

    @Reference

    private QueryBuilder builder;

    @Override

    //The GET Method uses the AEM QueryBuilder API to retrieve DAM Assets, places them in a ZIP and returns it

    //in the HTTP output stream

    protected void doGet(SlingHttpServletRequest request, SlingHttpServletResponse response) throws ServerException, IOException {

        try

        {

            //Invoke the adaptTo method to create a Session

            ResourceResolver resourceResolver = resolverFactory.getAdministrativeResourceResolver(null);

            session = resourceResolver.adaptTo(Session.class);

            // create query description as hash map (simplest way, same as form post)

            Map<String, String> map = new HashMap<String, String>();

            //set QueryBuilder search criteria

            map.put("type", "dam:Asset");

            map.put("path", "/content/dam/car");

            map.put("property.value", "image/png");

            builder= resourceResolver.adaptTo(QueryBuilder.class);

            //INvoke the Search query

            Query query = builder.createQuery(PredicateGroup.create(map), session);

            SearchResult sr= query.getResult();

            //write out to the AEM Log file

            log.info("Search Results: " +sr.getTotalMatches() ) ;

            //Create a MAP to store results

            Map<String, InputStream> dataMap = new HashMap<String, InputStream>();

            // iterating over the results

            /*

            for (Hit hit : sr.getHits()) {

                //Convert the HIT to an asset - each asset will be placed into a ZIP for downloading

                String path = hit.getPath();

                Resource rs = resourceResolver.getResource(path);

                Asset asset = rs.adaptTo(Asset.class);

                //We have the File Name and the inputstream

                InputStream data = asset.getOriginal().getStream();

                String name =asset.getName();

                //Add to map

                dataMap.put(name, data); // key is fileName and value is inputStream - this will all be placed in ZIP file

            }

            */

            /* This version also adds folders, and subfolders and files with in the folder*/

            // iterating over the results

            for (Hit hit : sr.getHits()) {

                //Convert the HIT to an asset - each asset will be placed into a ZIP for downloading

                String path = hit.getPath();

                Resource rs = resourceResolver.getResource(path);

                if (rs.getResourceType().equals("dam:Asset"))

                {

                    Asset asset = rs.adaptTo(Asset.class);

                    //We have the File Name and the inputstream

                    InputStream data = asset.getOriginal().getStream();

                    String name =asset.getName();

                    //Add to map

                    dataMap.put(name, data); // key is fileName and value is inputStream - this will all be placed in ZIP file

                }

                else if (rs.getResourceType().equals("sling:OrderedFolder")) {

                    //Add Folder

                    String folderName=rs.getName();

                    addFolder(folderName,rs, dataMap, resourceResolver, response);

                }

            }

            //ZIP up the AEM DAM Assets

            byte[] zip = zipFiles(dataMap);

            //

            // Sends the response back to the user / browser. The

            // content for zip file type is "application/zip". We

            // also set the content disposition as attachment for

            // the browser to show a dialog that will let user

            // choose what action will he do to the sent content.

            //

            ServletOutputStream sos = response.getOutputStream();

            response.setContentType("application/zip");

            response.setHeader("Content-Disposition", "attachment;filename=dam.zip");

            // Write bytes to tmp file.

            sos.write(zip);

            sos.flush();

            log.info("The ZIP is sent" ) ;

        }

        catch(Exception e)

        {

            log.info("OH NO-- AN EXCEPTION: " +e.getMessage() );

        }

    }

    private void addFolder(String folderName,Resource folderRS, Map<String, InputStream> dataMap, ResourceResolver resourceResolver, SlingHttpServletResponse response) throws IOException {

        Iterator<Resource> it = folderRS.listChildren();

        try {

            while (it.hasNext()) {

                Resource rs = it.next();

                if (rs.getResourceType().equals("dam:Asset")) {

                    Asset asset = rs.adaptTo(Asset.class);

                    if (asset != null) {

                        //We have the File Name and the inputstream

                        InputStream data = asset.getOriginal().getStream();

                        String name = folderName + "/" + asset.getName();

                        //Add to map

                        dataMap.put(name, data); // key is fileName and value is inputStream - this will all be placed in ZIP file

                    } else {

                        log.error("Asset does not exist or the user does not have permission to access the resource ");

                        //Todo throw an exception

                    }

                } else if (rs.getResourceType().equals("sling:OrderedFolder")) {

                    //Recursively add the files in the folder

                    String subfolderName=folderName+"/"+rs.getName();

                    addFolder(subfolderName,rs, dataMap, resourceResolver, response);

                }

            }

        } catch (Exception ex) {

            log.error("An Exception occurred {} ", ex);

            response.sendError(SlingHttpServletResponse.SC_INTERNAL_SERVER_ERROR);

        }

    }

    /**

     * Create the ZIP with AEM DAM Assets.

     */

    private byte[] zipFiles(Map data) throws IOException {

        ByteArrayOutputStream baos = new ByteArrayOutputStream();

        ZipOutputStream zos = new ZipOutputStream(baos);

        byte bytes[] = new byte[2048];

        Iterator<Map.Entry<String, InputStream>> entries = data.entrySet().iterator();

        while (entries.hasNext()) {

            Map.Entry<String, InputStream> entry = entries.next();

            String fileName =(String) entry.getKey();

            InputStream is1 =(InputStream) entry.getValue();

            BufferedInputStream bis = new BufferedInputStream(is1);

            //populate the next entry of the ZIP with the AEM DAM asset

            zos.putNextEntry(new ZipEntry(fileName));

            int bytesRead;

            while ((bytesRead = bis.read(bytes)) != -1) {

                zos.write(bytes, 0, bytesRead);

            }

            zos.closeEntry();

            bis.close();

            is1.close();

        }

        zos.flush();

        baos.flush();

        zos.close();

        baos.close();

        return baos.toByteArray();

    }

}