Expand my Community achievements bar.

Enhance your AEM Assets & Boost Your Development: [AEM Gems | June 19, 2024] Improving the Developer Experience with New APIs and Events

Feature Request: Clear All, Multi-Item Recieve, Multi-Item Retract

Avatar

Level 2
I am not sure where to put this but maybe at least we could
get some discussion out of it. Here is my list of 3 changes to the
CollectionNodeEvent (really changes it how the server works on the
back end I think). There are currently work arounds of course so
this isn't critical but it would be "nice" to have.



1)
Clear All Items - An event type of clear all Items that
would completely clear out all of the items in the message bin.


Justification: It is overly cumbersome and network intensive
to clear all of the items out an existing message queue or manual
message bin. One simply message would allow a user to for instance
completely "wipe" a whiteboard. Without a event type like this we
need to manually retract every item which can be slow and bottle up
the network quite a bit. Plus it is ackward to watch as items one
by one disappear as they are retracted. If my whiteboard had 100+
items on it, wiping it generates 100+ "itemRetract" messages.



2)
Multi-Item itemReceive - Allow multiple Item message to be
sent as a group (an Array). a Model would simple trap and process a
"multiItemReceive" event type and process multiple messages in the
recieved message array the same way it processes one message with
"itemRecieve".


Justification: This allows multiple messages to be processed
as a group and can lend itself to support for transactional type
data processing (IE: this group of messages needs to process or
fail to process as a singular atomic group). It also more
importantly speeds up the distribution of automated "generated"
messages. For instance say I added some code to the whiteboard to
generate a grid of 4x4 grid of squares. In code we would generate
the positions and properties of those simpleshapeitems and add each
one individually to the shared whiteboard data model. This would in
the background generate 16 (4x4) itemRecieve messages to each
client. Each message would be sent individually and since the
SharedWBModel is setup as STORAGE_SCHEME_MANUAL not a Queue, there
is no order to the reception of those messages. As the messages
came in the squares of the grid would pop into existence in a
seemingly random order. One message for this entire block of added
shapes would be far superior.



3)
Multi-Item itemRetract- Allow multiple Item message to be
retracted as a group (an Array). Similar to the Multi-Item
itemReceive suggestion above but for groups of "itemRetract"
Messages.


Justification: Like the Multi-Item itemReceive for automated
processes such as group selections and deletion this would be
faster and far superior. For mass deletion of model data items that
tracked and syncronized by the message itemID this would be
invaluable.



I think what I really maybe suggesting is for
CollectionNodeEvent to change

public var item:MessageItem;

into

public var items:Vector<MessageItem>;



I hope I was clear. If someone has some suggestions that
helps alleviate some the problems these event types would seemingly
help solve please don't hesitate to mention them. If it is apparent
anywhere that I have misunderstood some key aspect of the framework
please don't hesitate to mention that either.



Ves
2 Replies

Avatar

Former Community Member
Hi _vesuvius,



Thanks for taking time to write this - it's very well
thought out. I admit I've had some thoughts along these lines as
well, but in general ended up de-prioritizing them. Let me answer
these by number :



1) Clearing all items is, according to
XEP-60,
something only a UserRoles.OWNER is capable of doing. We've been
pretty persistent in following the XMPP rules as much as possible,
for reasons that will be clear later =). Likewise, only an OWNER
can delete a node, which clears all items in the node. So for cases
where we've needed to delete all items, we simply delete the node
and re-create it. It meant one less client-to-server RPC to handle,
which in the balance of things is actually a real benefit - we're
pretty stingy about the number of client-server operations we want
to support (while, hopefully, being generous with the number of
features we support).



2) I hear what you're saying here - I think the most
important point being the atomicity of pass/failure of a batch of
events. I don't think I agree with the whole justification you lay
out, however. There would be some bandwidth

savings, but I'm not convinced it's significant; it avoids
repetition of the RPC name ("publish!tem"), the node name,

and the publisherID for each item, but that's essentially it.
More importantly, to your 4*4 grid example, there is actually
guaranteed ordering to messages sent within the same
collectionNode, so they would definitely appear in the order in
which you sent individual items. This wouldn't prevent interleaving
of other publishers' items, which might be a use-case that's
important, but I'd thought about it and couldn't come up with
anything concrete. Any ideas there?





3) More or less the same answer as 2). The biggest case I can
make here is for bandwidth savings, and I think I dispute that it's
"far superior". If I'm missing something here, definitely let me
know.



We do take these sorts of suggestions very seriously - in
fact, it makes me happy to have someone to talk to about
CollectionNode type stuff =). In this case these were all features
we'd considered as part of the design; they got cut as optional,
since they fell more along the lines of "optimizations" rather than
"features" that unlock functionality that couldn't be accomplished
any other way.



Again, thanks for the great suggestions - please keep
digging, and if we're missing anything that wouldn't be possible
with what we have today, we're all ears.



nigel



[PS, while trying to submit this post, I was stopped by a
very unhelpful alert message telling me "Censored word(s)
detected". But of course, it didn't tell me which word(s) I needed
to eliminate. So I had to binary search through the post, cutting
out halves, until I discovered the censored word was "publish!tems"
(replace ! with I). This forum software is total bullsh!t.]

Avatar

Level 2
Nigel,



Thanks for the reply. I never expected interaction on a
Saturday of all times. I have just a few further comments :).



1) I thinks I can live with that. That makes some sense. I
didn't realize you were following the XMPP spec, that gives me some
insight. I think deleting and restarting the Node seems like a good
way to accomplish that. Let's run through it so i understand this
better. If i want to completely clear a node I issue a:



collectionNode.removeNode(nodeName); as the Owner followed by
a

collectionNode.createNode(nodeName, nodeConfig);



All of the non-Owning subscribers should be listening for a
CollectionNodeEvent.NODE_DELETE and in

onNodeDelete wipe all the objects associated with that Node.
Since they never lost sync with the collectionNode, they simply
keep on listening itemRecieves even ones for "nodeName". That seems
pretty simple now that I lay it out. I wouldn't have thought about
doing it that way but that doesn't seem too bad actually.



2&3) I may have misstated what I think is the biggest
problem here. I would agree with you that bundling messages as
opposed to sending them single file won't likely save much
bandwidth. They all still need message envelopes anyway so its not
a big deal from a bandwidth standpoint. Allowing for automic groups
of messages I still think is worthy feature but the real issue that
sparked that request has to do with poor performance. I should have
been a bit more clear, it's not bandwidth per say that concerns me
as much as latency.



As for a use case take the following: I modify the shared
whiteboard and add two special buttons. One turns every object on
the screen Red the other turns every object on the screen blue.
Lets say one client to hits the "change all to red" and another
separate client hits the "change all to blue" button couple of
seconds later. Depending on the number of objects and latency
issues it would be possible for some of the objects to end up blue
and some red.



In an ideal, non laggy, non spiking, non multiple resend TCP
world, we should see all of the objects change to red then a few
seconds later they should all change to blue. This won't happen and
here is why: Order can't be gaunteed until the message hits the
server.



I know you said:


quote:



"More importantly, to your 4*4 grid example, there is
actually guaranteed ordering to messages sent within the same
collectionNode, so they would definitely appear in the order in
which you sent individual items.





However in this case that doesn't help us at all, all that
can do is garuntee that the last object doesn't turn red before the
first object. Even though the second client started sending a few
seconds later they could still finish the blue color message for
all of the objects before all of the red color change messages had
finished (its the internet). In this situation the server would
still be receiving red color changes AFTER all blue color changes
have processed.



Now that I have re-read that, that really just makes the the
case for allowing automic groups. Which was the merit you really
saw in the suggestion in the first place. So you probably already
thought of this.



Thanks for the reply anyway, even if I don't change any minds
it is still very useful to know what the work arounds are.



Ves