Managing a large number of collection items

Avatar

Avatar

Aaronius9er9er

Avatar

Aaronius9er9er

Aaronius9er9er

18-06-2009

Hey folks,

I'm just starting an application and I need some help understanding how to architect it.  My application could potentially have thousands of users, so I'm trying to build for it.  Let's say there's User A, User B, User C, User D, etc.  User A may be "following" (the word subscribed could work...but I don't want to get anyone confused) User C, but not User B or User D.  If User A is logged in and then User C logs in, User A needs to retrieve an attribute from User C.  Likewise, User C needs to retrieve an attribute from User A.  User A doesn't care about User B or User D's attribute.  A single user can be following multiple friends.  So User A could follow both User C and User X.  On the other hand, User C and User X aren't required to follow User A in return.

So, right now I have a Users shared collection and when a user logs in he adds himself to the collection.  That's working fine and all the users are getting added when they log in as expected, but I'm not sure what to do when the application hits thousands of users.  If User A is subscribed to the users shared collection (which it needs to be in order to see when User C logs in), isn't it also getting ALL the thousands of user objects that are on the Users shared collection?

Here's the second part of my question: Each user has a "library" of information (this is different than the "attribute" I mentioned earlier) which is an array of objects that will likely change over the period of the user's session.  So User C has a library and when an item gets added to his library User A needs to know about it.  If I have a shared collection of users and the library is an attribute of the user end node, I don't think User A would receive notification when an item gets added to User C's library unless User C re-adds himself to the Users shared collection.  I haven't actually verified that but I read about it somewhere so I could be wrong.  Here's my workaround: User C creates a UID for his library, creates a shared collection for his library using the UID for the node name, and then stored the UID on his user node.  So when User C logs in, User A hears about it, pulls the library UID from User C's user node, then subscribes to User C's shared collection that happens to have the same UID.

So both of these strategies are working, but I'm concerned on the scalability or that there's a better way to do what I'm trying to do.  Any tips?

Thanks,

Aaron

View Entire Topic

Avatar

Avatar

Aaronius9er9er

Avatar

Aaronius9er9er

Aaronius9er9er

23-06-2009

Nigel, thank you for the great response.  I really appreciate it.  I'd be glad to share my app details and see what you think.

At its core the app is essentially a peer-to-peer file-sharing application.  A user can add other users as friends/co-workers and pull files off their machines.  AFCS would be used for:

(1) Letting a user know when his/her friends have logged into the service.

(2) Providing details about the files the user's friends have made available.  If a friend adds/removes/renames a file, the user needs to know about it.

(3) Providing a means for swapping stratus ids.

That's pretty much it.  The actual file transfer will be done p2p using Stratus.

The nice thing about using AFCS is the push strategy.  The app doesn't have to continually poll the server for new info.

What ideas does that bring to mind?  I can't figure out how one user would know if one of his/her friends is connected to the service unless everyone is in the same room and the user is analyzing each new user that connects to see if he/she is a friend.

Thanks for any guidance.

Aaron