Notifications: Performance issues at scale?

[Gdubz-57m2mu]

Did some local testing and noticed that if, for example, 3 people are following a blog post and someone leaves a comment, there are 5 Mongo documents created:

• 1x Comment
• 1x Activity, for the user who commented
• 3x Notifications, for each user following this blog post

What happens if we had, again just for example, 15,000 users following a blog post...!

1. Would that generate 15,002 Mongo documents each time a comment was posted?
2. Would this slow down the time it takes to post each comment?
3. Couldn't this quickly get out of hand, if you had a large number of blog posts that users were actively following and commenting on?

Version: AEM 6.2 with Communities FP1 & FP2
SRP Config: Mongo DB SRP (MSRP)

[danmcweeney · Accepted solution]

This is true. But what it comes down to is, do you bias a system for read performance or data size? We have biased the system for read performance ( feel free to read up on fan out on write vs. fan out on read ). Also, the notifications system creates node async to the blog creation process so that "throughput" is not slowed by the fan out on write. 

[Gdubz-57m2mu]

Thank you so much for the reply, Dan. Makes a bit more sense after reading a couple articles on it.

For anyone else that may come across this post in the future, this is the article I found, and the links that it contains:

• Write-based fan-out vs read-based fan-out : http://rubanm.tumblr.com/post/59561467766/write-based-fan-out-vs-read-based-fan-out
• The Architecture Twitter Uses to Deal with 150M Active Users : http://highscalability.com/blog/2013/7/8/the-architecture-twitter-uses-to-deal-with-150m-active-user...
• Raffr Krikorian's Timelines at Scale : https://www.infoq.com/presentations/Twitter-Timeline-Scalability
• TAO: Facebook’s Distributed Data Store for the Social Graph : http://0b4af6cdc2f0c5998459-c0245c5c937c5dedcca3f1764ecc9b2f.r43.cf2.rackcdn.com/11730-atc13-bronson...