service to monitor / poll file system for new assets

[MPRobertson]

Hello,

I have a requirement to monitor / poll a path on filesystem of author instance for new asset files.  When new file assets are SFTP'ed to that file system path / directory, my AEM service should add them to the DAM.

Question:  how best to implement this in AEM?  Should it be a servlet or service or another approach? 

With a servlet, I believe it would need a post/get to initiate the process.  This is not what I need.  I need it to always be running (when instance is running) and monitor / poll the filesystem every minute or so to look for new files ( maybe a listener?).

With a service, I believe I would need to instantiate the service in a page to initiate the process.  Again, this is not what I need.  I need it to always be running and monitor / poll the filesystem every minute or so to look for new files ( maybe a listener?).

I very much appreciate some ideas / suggestions.

Thanks in advance.

[smacdonald2008]

Are you taking about a Watched Folder system on your file system. That is - when an image is added to the folder (ie - C:\WatchedFolder) - the image is automatically posted to the AEM DAM? 

[Jörg_Hoh · Accepted solution]

Hi,

the easiest way is to mount the JCR repository via webdav to a local folder, and then configure the FTP server in a way, that the client can directly place the asset into the right folder structure.

Haven't tried it yet, but you can give it a shot :-)

kind regards,
Jörg

[MPRobertson]

smacdonald2008 wrote...

Are you taking about a Watched Folder system on your file system. That is - when an image is added to the folder (ie - C:\WatchedFolder) - the image is automatically posted to the AEM DAM? 

Yes, that is the basic idea.  Only not just post to AEM DAM.  There is other "processing". 

But that is the basic question.  We want a service running in AEM that watches a folder for files and then processes new files added to watched folder.

I don't want to limit the watching of the folder by a servlet that only runs when posted to.  And I don't want to limit the watching of folder by a service that only runs when service is instantiated.

Basically, I need a service always running and watching folder, regardless of it being instantiated or not.  Is there a way to do this?  Maybe a service will do this without every being instantiated?

[smacdonald2008]

Build a service in AEM that uses a thread that runs. You can have an operation that turns the thread on and off.

Read this that talks about threads and OSGi bundles:

http://collaboration.cmc.ec.gc.ca/science/rpn/biblio/ddj/Website/articles/DDJ/2006/0609/060801og01/0...

The job of the thread is to watch a configured folder.

https://docs.oracle.com/javase/tutorial/essential/io/notification.html

http://andreinc.net/2013/12/06/java-7-nio-2-tutorial-writing-a-simple-filefolder-monitor-using-the-w...

When a new image is dropped into the folder, the service gets the file and processes it. You can write an AEM service that uses Java File classes to perform operations - as long as the dir is on the same box. 

[MPRobertson]

smacdonald2008 wrote...

Build a service in AEM that uses a thread that runs. You can have an operation that turns the thread on and off.

Read this that talks about threads and OSGi bundles:

http://collaboration.cmc.ec.gc.ca/science/rpn/biblio/ddj/Website/articles/DDJ/2006/0609/060801og01/0...

The job of the thread is to watch a configured folder.

https://docs.oracle.com/javase/tutorial/essential/io/notification.html

http://andreinc.net/2013/12/06/java-7-nio-2-tutorial-writing-a-simple-filefolder-monitor-using-the-w...

When a new image is dropped into the folder, the service gets the file and processes it. You can write an AEM service that uses Java File classes to perform operations - as long as the dir is on the same box. 

Thanks again.  I appreciate the quick replys.

I have doubts about needing to implement a thread for this requirement. So just want to explore this a bit further:  Is there any other way to have a service always running and watch a folder in AEM without invoking or instantiate/call that service from JSP ?

What if I want a service to run once every hour and do XYZ processing.  Looks like I could use sling scheduler to run a job once an hour and check folder for new files, etc...

https://helpx.adobe.com/experience-manager/using/aem-first-components1.html

[smacdonald2008]

WHy i suggested thread is because you mentioned "need it to always be running"

Back in Adobe Data Services - this is how a service that was always *running* was implemented.  Thread just keep looping and checking a folder until it was turned off. This could be implemented in OSGi if you need that functionality.