@JakeCham there are two scenarios.
Scenario 1, where the client library has no dependencies
With scenario one, simple implementation with no dependencies, you can simply call the absolute path to the client library, for example:
<script defer src="/etc.clientlibs/my-site/clientlibs/clientlib-site.js"></script>
Scenario 2, the client library has dependencies
then like what the others say, nateyolle's solution looks like an option.