When there is a change in header and footer then all of the site pages gets invalidated in dispatcher. This causes an increase in load on publish instances as dispatcher has to fetch all the pages again to serve new requests. What is the best way to avoid this instant load on publish instances ? Do you guys use re-fetch flush agents ? Are there any issues or any known problems in using the re-fetch agents?
To me, "re-fetch" feature would work fine in some cases based on your dispatcher-cache design.
If the stat file level is zero or you flush a huge number of pages "during business hours" then this feature could backfire and put the system under load. It would make sense to use it outside business hours with limited volume of content flushes
You may also want to restrict it with "Resource-Only" to limit down on the volume of flushes and re-fetching
For your use case, it would be better if the header/footer are saved as a separate fragment in the cache so that there is no need to flush entire site just for two shared fragments