Thanks for all the responses they definitely help, we probably need to speak to an Adobe Engineer for Performance Tuning our AEM instance before go-live, its coming up fairly soon.
smacdonald2008
Can you point me to any hotfixes we might have missed around Performance? As far as I know we have everything up to SP2 (AEM 6.1) and every hotfix except the March 24, 2017 one for OAK here: AEM 6.1 hotfixes (but suprisingly enough the JCR queries doesn't seem to be our bottleneck, no single query is even taking more than 400ms, I was hoping to see some smoking guns in the AEM monitoring/diagnostics tools but nothing).
@kaushal_mall
No, we are not really activating many pages. Our content won't be updated in PROD that often so its only a handful of tiny changes going on simultaneously with the Performance Testing. We are not bothering to Performance Test our Author at all since we won't have that many users in there (< 50). We had a caching problem with static images some how bad quotes got into the "Cache-Control" header, but its been fixed and image caching seems to be fully working again. Don't suppose you have any other ideas? If it helps, we have a lot of external dependencies on 3rd party APIs due to the nature of our applications, that's all our Thread Dumps are telling Adobe right now, but we are sure there's more to the story, hitting the APIs in isolation is not performing as badly as the end-to-end test cases.
We are in an Active-Active (Primary Data Center & Backup Data Center) arrangement with 2 Dispatchers, 2 Publishers in each, and a single Author with a cold backup Author ready to get flipped on.
Our performance troubles have been across the board, Authors are complaining it is slow and the website is just barely meeting SLAs without any Performance Testing scripts running. Once we fire up JMeter towards our (I thought relatively modest) concurrent threads/users goal of 2500, it croaks before hitting 1000.
