You could set up a separate document root in your apache configuration for static content and have it live outside of the AEM dispatcher cache.
For example, if your AEM cache is under /var/www/aem/dispatcher/cache, you could setup a new directory /var/www/static and place your static files there, then create the appropriate configuration to serve content from /static when certain directories/pages are requested.