SFTP from project firefly worker

[jstadler]

My company is looking to integrate SFTP import and export with Adobe Experience Manager as a Cloud Service. We have run into issues where previous Java implementation in on-premise AEM no longer is possible in Cloud Services due to restrictions in network ports to/from AEMaaCS. Our team is now exploring using Adobe IO and project firefly worker to be the SFTP client. Are there any restrictions with connectivity to/from Project Firefly workers?

[dr_venture]

There are no outbound restrictions that I know of.  The only limitation I can think of there would be the max Action run time.  Serverless Actions (functions) are not long running by design so each Action by default would need to complete in under 1 minute before it would be timed out.  This is configurable up to 1,800,000ms per the documentation but 1 min is the default and complies with general serverless principles.

https://www.adobe.io/apis/experienceplatform/runtime/docs.html#!adobedocs/adobeio-runtime/master/gui...

As for in bound connections.  Since there are no long running processes you will be limited to Event handers via Adobe IO custom events or REST calls to Actions.

Events have a payload max body size but you could use them to trigger pull Actions listening and ready to Pull in data for processing.

Here are some codelabs that might be handy for you
https://adobeio-codelabs-alarms-adobedocs.project-helix.page/
https://adobeio-codelabs-journaling-events-adobedocs.hlx.page/

https://adobeio-codelabs-custom-events-adobedocs.project-helix.page/

If you tell us more about your use case maybe we can help you figure out a workable approach.  Given the data you have shared I would use a Custom Event to kick off a Firefly Action that would use SFTP to get some data from your systems and store it in the Firefly File store.  That Action when complete would send off another custom event saying upload was "done" and there is data to process.  That "done" Event would give you a nice audit trail.  That same "done" Custom Event would be mapped to another Firefly Action which would process the data you have stored in the File Store and do the next step in your process. Something like that...

[jstadler]

dr_venture,

Thanks for your very quick and helpful response. I will check out the documentation you have referenced.

Our use cases are fairly straight forward.

1) Import files to AEM Cloud Services from SFTP servers (as assets). The SFTP servers are external to our company, generally managed by our customers.

2) Export assets from AEM Cloud Services to SFTP servers. Again, the SFTP servers are external to our company, generally managed by our customers.

[dr_venture]

Your decision point, I think, is Event or Alarm based execution. 
Alarms are like Cron jobs(timer execution)

Your deployed Firefly Actions would run in our Adobe cloud, so they should be able to access any publicly available endpoint. 

Firefly Actions are written in nodejs which has a ton of librarys for calling SFTP endpoints.  On the AEM side I would imagine your going to use the Assets API to get and put files. There are also ways to hook into AEM Asset events.  I have not done it yet but I know its a thing one can do.

[jstadler]

Alright, we will explore Events and Alarms.

The main issue that we are working around is that AEM Cloud Services has tightly restricted network controls. We are unable to connect over SFTP ports from AEM in order to use the Assets API or any other Java libraries. We had a working AEM-based Java solution prior to Cloud Services, when our AEM servers were on-premise.

We are hoping that Firefly project would be able to "pickup" and "deliver" assets to AEM without network restrictions.

Thank you.

[dr_venture]

Well that is one of the reasons we built firefly.  To help provide easy extention points for our products.  I hope it helps with your use case.  Let us know here in the forum if you need more help.