When you say "we have set up the experience fragment template", is this a custom experience fragment template or the one out of the box? Can you also share more details on the component you are using, is it running specific functions at runtime to obtain product detail information and then placing it a list as an example?
I think at face value what I am thinking is happening is you have a race issue between the js and the markup because of how Target parses the markup from an experience fragment and places it into the selector.
Hope this helps!