Self-hosted libraries - Same file multiple times

[Antti_Ko]

I have talked this with ClientCare (ping Navdeep Pandey​), but we decided to wrote public forum post and bit pressure for product management to react to this one. There was also some discussions about this in Measure Slack. Anyway, would be great to get this information to forum too.

If you want to choose self-hosted files in Adobe Launch you need to download scripts in zip file. There are normal and minified versions available. After unzipping you get e.g. minified files:
(I modified folder/files, but you get the idea)

launch-EN2a37something-development.min.js

xxxxx/subfolder/launch-EN2a37something-development.min.js

xxxxx/subfolder/subfolder2/launch-EN2a37something-development.min.js
xxxxx/subfolder/subfolder2/EXed123456789de2d-libraryCode_source.min.js

xxxxx/subfolder/subfolder2/hostedLibFiles/EP971123456789xxxxxxx/AppMeasurement.min.js

My questions:

1. Why is there same file 3 different times on different folders? And if I see browser only using one of those could I then delete other files or should I keep those always on our server?

2. Is there plans to simplify this? My wish is to have only 1 file and maybe I need to wait this https://medium.com/adobetech/simplifying-customer-workflows-with-adobe-experience-platform-web-sdk-4... ?

3. Before we get things to simplified format, could there be detailed information available what these different files really contain and do you need to update every single one after making updates in Launch? For example, maybe you don't need the last AppMeasurement file at all if you have code in custom code etc?

Now you have to manually upload and do different kind of security tests for 5 different files and this takes too so much time compared to only 1 file (or max 3 files).

I already noticed that these folder and script names are nowadays shorter, because windows had some problems to unzip too long folder structure. Good!
I would also hope that unzipping the file would result into 2 different zip files containing minified version in separate file. Now you have to be very careful that you only upload minified version and not the other one.

[thebenrobb · Accepted solution]

1. Backwards compatibility.  The structure of the package has changed over time.  In the environments that we host for you, these are symlinks to a single file, but in the archived builds, we put identical copies of the file in because symlinks are not supported on all systems where you might unpack these files.  Pick whichever one you want and use that one, just make sure your embed code points to the right place.  The others are superfluous.
2. There are no plans to simplify it since we aren't willing to break the backwards compatibility.  If you use Launch to deploy the upcoming webSDK, you'll probably see a new file in the hostedLibFiles folder (depending on how those developers choose to structure their extension), but it won't change the underlying structure of the archived build.
3. Yes, this will end up in the product docs.  There's a backlog of documentation items that we're working through.

[thomas_amsler]

I still would root for an additional flat zip option, where all files are directly part of the zip without subfolders as they are only relevant for CDN and not self hostet.

[thebenrobb]

If this is something that you want, I'd suggest logging an idea for the enhancement, and then lobbying the community members to vote for it.  To be completely transparent, we probably wouldn't take it on without a bunch of community support for a couple reasons: 

1. Currently all builds have the same stucture which makes it much simpler to develop against and maintain.
2. The subfolders are still relevant even if you're self-hosting.  The main library file contains internal references to the sub-files within their subfolders, so we'd then have to update the client library (adding size and weight) to be able to reference the different build structures that might be there.

[Antti_Ko]

@thebenrobb  To continue my original question... we noticed something weird. Bit diffucult to explain, but I´ll try:

If you go to environment tab and click the package icon below “install” column for environment X you will see there is new url structure for non async url like this:

src="//ourdomain.com/our-folders/a1b2c3/12345678/launch-censored-development.min.js"

I believe those 2 folders before the script are new ones and not have been there before. Just curious to know why this change,
but I guess it doesn’t really matter that we use shorter url on our dev site like this:
"//ourdomain.com/our-folders/launch-censored-development.min.js"

Everything is working just fine, because we know what url we have hard-coded to our site and we can modify self-hosting folders in any way we like.

But let's consider situation that something goes wrong in our implementation and we need to start over and our new analyst is checking from launch that we need to hard-code
url src="//ourdomain.com/our-folders/a1b2c3/12345678/launch-censored-development.min.js" to our site. However, our scripts are really hosted in url
"//ourdomain.com/our-folders/launch-censored-development.min.js" because this was the original url structure in Launch when we made the launch property.

We are investigating this with Clientcare and as said, just curious about the logic and why the sudden change in Launch settings and is there something
we need to watch out in the future.

I guess this would be ok for new launch properties, just weird it changed to our old/current property or does it make sense?

Any comments or is it better that we continue with Clientcare? There is also small change that we have done internally
something that explains this behaviour, will see.

[thebenrobb]

@Antti_Ko, these changes happened several months ago. The folders themselves were introduced because of a limitation on the number of unique items per folder with our CDN partner. We needed a less-flat folder structure on the CDN in order to continue growth. The names of those folders are tokens that represent the unique company and property where these files exist. They only take affect for newly created environments, all existing environments were kept as is with no changes.

[Antti_Ko]

Ok, thanks for the clarification. Bit weird from client/user experience, but for now just glad to hear there is a real reason for this and this not a weird bug where folder structure is changing all the time etc. And as we know, there is already plans for single js file and everything should be easier in the future as mentioned in Adobe Summit 2020.

[Antti_Ko]

Ok, thanks for the details and hopefully we get the documentation in the near future.

[Urs_Boller]

thebenrobb​ I strongly support the idea of thomas.amsler​ to have a flat folder structure for zip file contents. and I don't think there is a lot to change, but might be wrong.

In fact, I think you just need to skip a bunch of folders and directly include the "ExtensionFolders" directly below the main file. so basically all the "libPath" could be something like "{myLocalPath}/EPXXX"

the desired final structure would look something like this:

{myLocalPath}
|- launch-ENXXXX.min.js

|- EPXXXX1 (folder for extension files 1)

|- EPXXXX2 (folder for extension files 2)

And since all the extension files are similar in all properties, you could take it even a step further: define a "globalSharedLibrary". just giving you an idea what I'm doing right now on our prod environment: when downloading the zip file I replace the "relative path beginning" of the extension files by a global "sharedLibPath". that means I have a shared library for all extension files which can be used by all properties. so my local folder structure looks something like this:

{localPath}
|- Property1
   |- launch-ENXXXX1.min.js

|- Property2

   |- launch-ENXXXX2.min.js

|- sharedLibs

   |- EPXXXX1 (folder for extension files 1)

   |- EPXXXX2 (folder for extension files 2)

the advantage for the "shared library" is a simpler update process: if you do not upgrade any extensions, there is just one file to be replace on prod (the min.js file). all extension files remain unchanged...

and the benefit for the end user? if someone moves between different properties, only the main file needs to be loaded, all shared lib files can be fetched from client cache (remark: caching for the shared libs could be even set to 1 year since upgrade of extensions would result to a new include file/folder)

I would hope to see an option for this "shared library" in the future - it would make update process for local deployment so much easier.

[thebenrobb]

This is good info, thanks =)

I'd reiterate that somebody should post this as an idea and collect votes.

[Urs_Boller]

I upvoted an existing idea: Simplify archive (zip) folder structure

[Antti_Ko]

I marked my initial questions as answered.

Now we have bit more understanding about this and hopefully we get official and detailed help documentations around this topic in the near future. I also found similar discussion thread in here https://forums.adobe.com/thread/2603462 and Launch self-hosting: Why are there duplicate files? if you want to learn more.

Also new idea post around this topic as Urs already mentioned: https://forums.adobe.com/ideas/11195

And long path names have been already fixed, great!

Thanks Urs and others for great ideas to modify scripts and paths. In some ways, it is risky to change default ways or at least that might confuse new analytics people when trying to understand how measurement is built on your site. Yes, you can and should always do documentation and it helps. Anyway, hopefully Adobe will develop this to better direction and something like this https://medium.com/adobetech/simplifying-customer-workflows-with-adobe-experience-platform-web-sdk-4...

sounds interesting too.