Notice the "async" word/attribute in the end of link tag above. It should be possible to use "defer" too.
I have found a library that can do this at Github (nateyolles/aem-clientlib-async), but unfortunately this code is not compatible with the latest AEM version and it also breaks the ACS Commons feature of Versioned Clientlibs.
a) Used the `nateyolles/aem-clientlib-async` library, available at GitHub;
b) This lib is very old so you will need to make some changes to the code to use it with the latest AEM version (there are some hints in how to implement the fixes in the Pull Request tab from his Github);
c) The last change you have to do, to make it compatible with ACS Commons Versioned Clientlibs feature is this:
Open the `apps/clientlib-async/sightly/templates/ClientLibUseObject.java` file and add the type attribute to the JS and CSS constants. The contants should stay like this:
private static final String TAG_STYLESHEET = "<link type=\"text/css\" rel=\"stylesheet\" href=\"%s\"%s>";