Adobe Experience Platform Query Service | Community
Skip to main content
S
SundarSritharan
August 6, 2021
Solved

Adobe Experience Platform Query Service

  • August 6, 2021
  • 3 replies
  • 2174 views

Hi Team,

We have huge dataset with TBs of data in AEP . In order to fetch data from project firefly , we are using query service api calls. Seems like it is very slow in performance. Is there any other way to fetch AEP dataset's data very faster or to boost up query performance?

 

@dbenge.

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by dbenge

There will always be a small additional call time added when using Firefly actions to access the AEP query service.  It is another layer between the client and the data.  That being said it should be a small addition when compared to calling query service directly.
It depends on your use case but the value added by using Firefly Actions is the ability to use the State and File storage layer to do caching of data. 

 

Please see this thread on caching of data

https://experienceleaguecommunities.adobe.com/t5/project-firefly-questions/action-result-max-size-and-good-practice/qaq-p/416323

 

It depends on your use case but the slight call time hit might be worth it if you can cache some of the data and increase your applications response time greatly.

3 replies

D
Adobe Employee
dbengeAdobe EmployeeAccepted solution
Adobe Employee
August 6, 2021

There will always be a small additional call time added when using Firefly actions to access the AEP query service.  It is another layer between the client and the data.  That being said it should be a small addition when compared to calling query service directly.
It depends on your use case but the value added by using Firefly Actions is the ability to use the State and File storage layer to do caching of data. 

 

Please see this thread on caching of data

https://experienceleaguecommunities.adobe.com/t5/project-firefly-questions/action-result-max-size-and-good-practice/qaq-p/416323

 

It depends on your use case but the slight call time hit might be worth it if you can cache some of the data and increase your applications response time greatly.

    M
    mdonke
    August 19, 2021

    I've seen similar issues with query service.

    Its not about the minor time which is added by your firefly application. Its about the performance of the query service itself.

    Yes, you can try using some caching concept. But even with that, at some point you need to refresh. Right? And if it then takes 2 mins (yes, I've seen this!) in any case your custom marketing application's user experience is destroyed.

    The overarching question for me is: Is it recommended by Adobe to use the query service at all for custom applications? (no matter if Firefly or other)

      D
      Adobe Employee
      dbengeAdobe Employee
      Adobe Employee
      August 19, 2021

      good points

      you could use firefly and put in an Alarm (cron) job that would refresh the cached dataset independent of the users interactions.  The user would never hit the 2min refresh but it would still be there.

       

      Not trying to under play the need for fast preformant API's of course.  It would be ideal if this caching was not needed at all.  I understand the fustration. I don't work in AEP engineering or product management so I dont really know the answer to your question about correct use.  I would recomend asking this question in their forums.  


      BTW if you get a good answer that changes what we know about the use of that API please circle back and let me know.  I would love to understand it better as well. 

        Jacob-DDdev
        Jacob-DDdev
        August 30, 2021

        I'd love to see the ability to define a custom index on AEP datasets. Not sure if Adobe is worried about the extra storage costs involved, but the performance gains would make the tool way more approachable.

        Idea submitted, here.

        Terms & ConditionsAccessibility statement

        Loading footer...