Why the "uber-jar-6.5.5-sources.jar" does not contain...

[software_engineer]

Hi, 

I have two questions which are related to the usage of the "uber-jar" in an IDE like Eclipse: 

Question 1) Why the file "uber-jar-6.5.5-sources.jar" does not contain at least the source code of the open source libraries which are included in the uber-jar? Why it is either empty or not provided by Adobe at all? 

For example, why it does not contain the source code of "org.apache.jackrabbit.api.security.user.UserManager"? 

At least the file "uber-jar-6.5.5-javadoc.jar" contains the Javadoc of the above-mentioned class, but why the file "uber-jar-6.5.5-sources.jar" does not contain its source code? It is an open source project and you can see links to its source repository here: https://jackrabbit.apache.org/jcr/source-repository.html

Therefore I do not understand why Adobe did not include it (and the source code of other open source libraries) in the file "uber-jar-6.5.5-sources.jar". 

In case if Adobe does not provide the "sources" file for its uber-jar at all (it is currently unavailable under the Adobe's public repo, as of 4th March 2021), then why Adobe does not provide it? 

Question 2) Why the file "uber-jar-6.5.5-javadoc.jar" does not contain at least the Javadoc of some proprietary classes which are not open source? (I don't mean their source code here as these seem to be some proprietary "closed" code of Adobe, but at least their Javadoc should be available.)

For example, why it does not contain at least the Javadoc of all classes which are located in the package "javax.jcr" and its sub-packages, like for example the class "javax.jcr.security.AccessControlManager"? 

It makes the life of an AEM developer who uses Eclipse more difficult. 

The problem with the file uber-jar-6.5.5-apis.jar is that in Eclipse you can only attach one jar file as a source of the Javadocs for it and also you can attach only one jar file as a source of the source code for it. 

For example, I can't specify for the file uber-jar-6.5.5-apis.jar in Eclipse multiple jar files with the source code of the Apache Jackrabbit library and multiple other open source libraries which are included in the uber-jar, if I already use the file "uber-jar-6.5.5-sources.jar" (or any other single jar file) as the source of the source code for it. 

I am used to being able to display all possible Javadocs and source codes of all libraries using the keys F2 and F3 in Eclipse. It disturbs my workflow and costs me additional time when I have to search for the Javadoc or source code of a given method/class in other places (e.g. on the Internet for each single open source library which is included in the Adobe's uber-jar). 

Does anyone know a solution to the above-mentioned problems? Or even better, will you Adobe fix these issues and make the life of the developers using your product easier? 

Thank you in advance for your answers. 

[Jaideep_Brar]

Hi,

I don't believe Adobe shares a file named "uber-jar-6.5.5-sources.jar". I could not find it under the public repo[1]

Also, user manager API details can be found at [2]. Is there anything specific you are looking for?

[1] https://repo.adobe.com/nexus/content/repositories/releases/com/adobe/aem/uber-jar/6.5.5/

[2] https://jackrabbit.apache.org/api/2.12/org/apache/jackrabbit/api/security/user/UserManager.html

[software_engineer]

Hi @Jaideep_Brar, I can't find it neither under the public repo [1], because it is currently unavailable. When I go to that URL, using the Chrome browser, I receive an error "Error: Request Entity Too Large: head". I know that I had indeed accessed this repo in the past, because I also have it in my bookmarks, but now it doesn't work. I hope that Adobe will fix this issue also. Anyway, I am almost sure that we have received the file "uber-jar-6.5.5-sources.jar" from Adobe. However, if you know a different source of the source code for the uber-jar, then please tell me - hopefully it will be better than the file "uber-jar-6.5.5-sources.jar". Regarding the link [2] from your post - yes, that's actually what I had written in my question that it is available there. I asked why it is not included in the jar from Adobe if it is an open source project which is generally available.

Update: I have just checked that the file "uber-jar-6.5.5-sources.jar" (which is in my local Maven repo) is almost empty. It doesn't contain any source code at all. So, well, it's actually worse than I thought. Only the jar with Javadoc does contain something, but also with the limitations about which I asked in the question. Where can I get a jar file with all possible source code for the uber-jar? I don't expect Adobe to provide me its secret proprietary code, but at least the source code of all included open source libraries should be included in it, shouldn't it? I mean, in Eclipse I can set only one jar file as a source of the source code for the uber-jar (which includes lots of open source libraries), so I hope that Adobe provides it. - If it does not, then why? 

Update #2: I have found out that I can access the public repo [1] using the Firefox browser. I still get the above-mentioned error "Error: Request Entity Too Large: head" when I try to access it using the Chrome browser, but it works using Firefox. The file "uber-jar-6.5.5-sources.jar" is indeed currently (as of 4th March 2021) not listed there. So it's a mystery to me where it came from into our company's Nexus repo. The other people involved in our project didn't put it there manually. - Its last modification date there is on the 4th November 2020, similar to the other uber-jar files there. Maybe this file had been temporarily provided by Adobe earlier, but has been removed, because it's empty and useless? The source Adobe's repo for our Nexus repo is the following URL (the current contents there are the same as under the repo [1], i.e. currently the sources jar is unavailable there): 

[3] https://repo.adobe.com/nexus/content/groups/public/com/adobe/aem/uber-jar/6.5.5/

Anyway, even if we assume that Adobe currently does not provide the file "uber-jar-6.5.5-sources.jar", then why Adobe does not provide it?

[Jaideep_Brar]

I checked Internally with a wider adobe audience and "uber-jar-6.5.5-sources.jar" is not a valid jar file. So, not sure how you got it in the first place. API and Javadoc jar is your best option.

[jkpanera]

This is an extremely lazy reply. He was asking for source code. It's actually a pretty reasonable request. Most open source projects make source code easily available and much of AEM is open source. There is no reason that it shouldn't be easily available.

Adobe should take developer's concerns more seriously.

[Jaideep_Brar]

You can log a ticket with AEM support to get more details on this if you have any followup questions

[software_engineer]

Hi @Jaideep_Brar, as I wrote above, the file "uber-jar-6.5.5-sources.jar" which we have is empty - it does not contain any source code, not even the source code of the open source libraries which are included in the uber-jar. - Please note that exactly these open source libraries and not the Adobe's proprietary code are the point about which I have asked in my question #1 regarding the source code. It's interesting to hear that the software developers at Adobe are able to use such a file with the source code internally and I wonder if it includes only the Adobe's proprietary (closed) source code or also the source code of all the open source libraries which are included in the uber-jar. This could explain why the software engineers at Adobe don't understand the pain of the technical customers of their AEM product, because they simply don't have the same problems in their everyday work.

You wrote: "it ["uber-jar-6.5.5-sources.jar"] was never supposed to be available to customers as it[']s Adobe's code". I suppose that we have some misunderstanding here. I am sure that the software engineers at Adobe are smart enough to know that the Apache Jackrabbit open source library and multiple other open source libraries which are included in the uber-jar are not Adobe's code. Therefore what you have written doesn't explain at all why Adobe doesn't provide a jar with the source code of all open source libraries which are included in the uber-jar.

I hope that some other people here will reply to my questions. - Maybe other software developers who managed to find a good solution for these problems. I hope also that someone at Adobe will read this and decide to make the AEM product better for its technical customers, i.e. for the developers who use this product outside of Adobe, by providing a jar with the source code of all open source libraries which are included in the uber-jar.

[Jörg_Hoh]

(Update)

I am using Eclipse as well, and I don't recall to have a problem with accessing the sourcecode of the opensource parts of AEM, even though they are not shipped. Honestly, I don't think that it is necessary, because by using maven and exact version numbers you (or Eclipse) can get the source packages directly from Maven central. 

Also, I haven't heard complaints about this yet, you are the first I know which requests the inclusion of these libraries.

[software_engineer]

(Updated)

Hi @Jörg_Hoh, thank you for updating your comment. It's great to hear that you have managed to solve this problem for you. Could you please share your knowledge with us and write how you managed to configure Eclipse so that after pressing the key F3 (for the command "Open Declaration") in your Java code on the type "org.apache.jackrabbit.api.security.user.UserManager", the source code of this class is displayed and at the same time (without changing the Java Source Attachment for the uber-jar before the next action) the same command can be used successfully on a class from a different open source library (from the uber-jar) in order to display also its source code?

Another, even better example, would be to press the key F3 on a call to a method of the class "org.apache.jackrabbit.api.security.user.UserManager" in your own Java code, for example on a call to the method "UserManager.getAuthorizable(Principal principal)". - Does Eclipse in your case jump to the source code of this method after pressing F3 and can you do the same for a method of a class from a different open source library (but also from the uber-jar), without changing the Java Source Attachment for the uber-jar between these two commands? 

In other words: How did you manage to attach in Eclipse the source code of more than one library for the single, big "uber-jar-6.5.5-apis.jar" which includes multiple open source libraries?

Did you maybe build your own sources jar for the uber-jar? - I mean, we thought about building a sources jar for the uber-jar on our own, but it requires some effort, has to be automated (extraction of the version numbers of the open source libraries from the uber-jar etc.) in order to build such a sources jar automatically for each new version of the uber-jar and most of all - we don't understand why Adobe doesn't provide such a sources jar which includes the source code of all open source libraries, considering that it would be much easier for Adobe to do it (because Adobe must have a build process for the uber-jar anyway, so adding a step for building a sources jar for it would be simple) than for each single development team using Adobe's product around the world to do it by themselves. 

Our team has configured (already earlier) our maven project configurations so that the source code of the required open source libraries (for example the source code of the open source Apache Jackrabbit library) is available to us in Eclipse in the "Open Type" window. - But this is the "best" solution that we know so far and it has the downside that we have to "manually" search for a given method in a class, instead of being able to see it immediately after pressing the F3 key. In other words - something what a developer in every usual Java project (at least in every Java project at which I have worked in my career until now) can do in a single step ("press F3 key on the method which source code you want to see"), has to be done in case of the open source libraries from the uber-jar in at least three steps (press Ctrl+Shift+T" to open the "Open Type" window, search for a class, select the found class (and remember not to select the one from the uber-jar, because it doesn't contain a source attachment...), press Enter or click on the selected class, search for a method which is interesting you in the opened source code of this class).

Another solution which I sometimes use is to attach in Eclipse a source code jar of a single open source library as the source code for the uber-jar. But this means that only the source code of this single library is available when pressing the F3 key for anything what comes from the uber-jar, until I change the source attachment for the uber-jar to another open source library. - This is obviously inefficient, because we use usually multiple open source libraries from the uber-jar in our code. 

- I hope that you found a better solution than these which we know. 

I also saw a partially similar, older question of another user "@jkpanera" here: https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/why-isn-t-debugging-easier... - Unfortunately, it does not have any accepted answer. It is not the same question as my two questions, but is partially similar (regarding the source code). 

[software_engineer]

Please note that the answer of @Jaideep_Brar does not answer the question #2 from my main "question" at all (regarding the javadoc jar). It only partially answers my first question (regarding the source code) and therefore I find this answer insufficient (please see my comment from ‎04-03-2021 above for more information regarding this part). I still hope that someone else will be able to provide me some helpful hints. I actually also hope that Adobe will decide to make its product better for the developers who use it.

[Jörg_Hoh]

@software_engineer: I think you are right. I just checked and found, that Javadoc is not displayed inline when hovering over a class- or method-name (complaining about missing source file). But I have a decompiler configured so jumping into the source code (decompiled on the fly) is possible. But IIUC that's not what you want to achieve.

After a bit of search I found https://stackoverflow.com/questions/310720/get-source-jar-files-attached-to-eclipse-for-maven-manage... which pointed me to some eclipse settings, which is currently downloading a ton of source and javadoc jar files. That seems to be quite promising.

[software_engineer]

Hi @Jörg_Hoh, which decompiler are you using in Eclipse? - However, using a decompiler to decompile the source code of open source projects is indeed not what I expect to achieve and not what is usually being done.

[Jörg_Hoh]

@software_engineer: Please check out that Stackoverflow link I provided, I turned on that feature in eclipse and it's working now, I get Javadoc displayed in the mouse hover effects. As Decompiler I have Procyon configured

[software_engineer]

Hi @Jörg_Hoh, of course I had already earlier enabled the Maven settings in Eclipse for downloading the artifact sources and Javadoc (this was one of the first things which I had checked in my Eclipse), but this didn't solve the problems which are described in my original two questions above. Otherwise I wouldn't post my original questions at all. - Here we come back to the questions #1 and #2 regarding the problems with the missing sources (for open source projects) and Javadoc (for some proprietary code of Adobe) of the uber-jar artifact which I described in detail in my "main question". - Could you please read my two questions in my "main question" and let me know if these particular problems are now solved for you? - I mean the following particular examples from my two questions: Example from question #1: Can you see the source code of "org.apache.jackrabbit.api.security.user.UserManager" (not from the decompiler)? Example from question #2: Can you see the Javadoc of the class "javax.jcr.security.AccessControlManager"? And an additional question: Which version of the "uber-jar" are you using?

[Jörg_Hoh]

@software_engineer: I cannot answer the first question, because I don't know the "why". I am not sure if there's anyone who can answer that question here. If you want to have that feature (and I think that is the question you might want to ask), you can either raise a support ticket and create post at https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager-ideas/idb-p/adobe-experien....

To answer your second question: I cannot answer this question as well. I don't know the reason. 

To comment your statement below the questions: I can totally relate that it makes your life harder than it could and should be. Nevertheless and as already wrote, this is the first time, that this is mentioned to me. I am myself an Eclipse user, and I hardly ever noticed it. I don't mind if the sourcecode of a class opens directly or via the decompiler, for me there is not difference. F2 opens here javadoc, F3 opens the .class file via the decompiler (including all comments and everything).

If you really would like to have that sources jar and and the javadoc as jar, please post it to the mentioned ideas section (link above). But just a warning: The pure uber.jar is already more than 50 megabyte in size, so I assume that including source and javadoc will increase the size to > 300 MB.

[software_engineer]

Hi @Jörg_Hoh, thank you for your answer. - I will post an idea regarding the sources jar and extended javadoc jar for the uber-jar to the ideas section mentioned by you. I have installed a decompiler plugin (https://marketplace.eclipse.org/content/enhanced-class-decompiler) in my Eclipse today, but it makes things partially worse, because when I open with F3 the decompiled source code for a class from the uber-jar for which no real source code is attached, then afterwards I can't see the Javadoc of that class and its methods anymore. - The decompiled source code seems to override in Eclipse the attached real Javadoc for the classes from uber-jar as soon as I view their decompiled source code and therefore no Javadoc is visible for all these classes for which it had been visible before I viewed their decompiled source code. (This problem does not occur only for classes for which a real source code is attached as sources for the uber-jar.) I have tried setting all possible default class decompilers in the Decompiler preferences (Procyon, FernFlower, JD-Core, Jad, CFR) and all of them have the same problem. How is it possible in your case that you see all comments etc. in decompiled code? Maybe what you see is not decompiled code, but the real source code is shown to you? Because in my case, when I press F3 on a class, then I see its source code with all comments and everything only if I had previously attached for the uber-jar a sources jar which includes the real source code of this class. Otherwise, I see a decompiled source code, i.e. without any comments etc, like one would actually expect a decompiled source code to look like, i.e. stripped from any additional information. Is it possible that - due to the fact that you work at Adobe - the real sources for the uber-jar were downloaded to your local maven repository from your company's internal repository and this is why you can now see them with comments etc.?

[Jörg_Hoh]

I checked with the Opensource parts, and there it works fine. It does not seem to work for product classes, but that's expected, as the uber.jar does only contain the full classes. But in both setups I see javadoc working. The project I have uses "aem-sdk-api (it's a CloudService project), but I am not aware of fundamental changes between AEM 6.5 and the CS packages (besides the naming).

[software_engineer]

Hi @Jörg_Hoh, you mentioned that you have Procyon configured as the Decompiler. Are you using the "Enhanced Class Decompiler" plugin for Eclipse (https://marketplace.eclipse.org/content/enhanced-class-decompiler)? If not, which other decompiler plugin are you using? I ask, because I wonder if you use some other decompiler plugin which works better than the one which I installed. When I select Procyon as the decompiler (under Preferences > Java > Decompiler) and afterwards press F3 on the class "org.apache.jackrabbit.api.security.user.UserManager", then an error is being shown "Editor could not be initialized." Only when I select another decompiler (under Preferences > Java > Decompiler), e.g. "CFR", then I can see the decompiled source code of a class from the uber-jar, but without any Javadoc. And after decompiling a given class on the fly by opening its source code, I can't see its Javadoc anymore when pressing F2 on that class, until I restart Eclipse. I.e. in my case the decompiled source code overrides the attached Javadoc which is a big downside for me. I wonder if the decompiler plugin which you use or its specific configuration integrate the attached real Javadoc into the decompiled source code, so that afterwards you see the source code together with the Javadoc. - This could be one of the possible explanations why it works in your case and I would like to know how you configured it to work. Could you please share your knowledge on this subject? 

I have configured the above-mentioned decompiler according to the recommendations from its Github page (https://ecd-plugin.github.io/ecd/) under the heading "How to check the file associations", i.e. I configured the following file associations in my Eclipse: 

• Click on “Window > Preferences > General > Editors > File Associations”
• “*.class” : “Class Decompiler Viewer” is selected by default.
• “*.class without source” : “Class Decompiler Viewer” is selected by default.

[software_engineer]

Hi @Jörg_Hoh, based on the information provided by you in your last comment, I have now found a reason why the problem regarding the Javadoc of all classes which are located in the package "javax.jcr" and its sub-packages, like for example the class "javax.jcr.security.AccessControlManager" (described in my question #2 from my "main question"), is not a problem for you. The reason is that the Javadoc jar of the "aem-sdk-api" (for the CloudService project) contains the Javadoc of these packages, in contrast to the Javadoc jar of the uber-jar (for AEM) which does NOT contain them.

You can see it here:
Javadoc of the "aem-sdk-api": https://javadoc.io/doc/com.adobe.aem/aem-sdk-api/latest/index.html (you can find the packages starting with "javax.jcr" there)
Javadoc of the "uber-jar 6.5.5": https://javadoc.io/doc/com.adobe.aem/uber-jar/6.5.5/index.html (you will NOT find the packages starting with "javax.jcr" there)

Moreover, the Javadoc of the latest uber-jar version (6.5.8 currently) also does NOT contain the Javadoc of these packages: https://javadoc.io/doc/com.adobe.aem/uber-jar/latest/index.html

For me, it's a clear proof that it is a bug in the "uber-jar-6.5.5-javadoc.jar" that it does not contain the Javadoc of all classes which are located in the package "javax.jcr" and its sub-packages, because the Javadoc of the "aem-sdk-api" does contain it. - I suppose that thanks to our exchange of information we have just found the answer to my question #2 from my "main question": It seems to be quite simple: "because it is a bug". Now, another question is: What will Adobe do about this bug? 

[Jörg_Hoh]

I cannot answer "what Adobe will do about this bug" (I am not "Adobe"). Have you reported this as bug?

[software_engineer]

Hi @Jörg_Hoh, I understand that you are not the right Adobe employee to report a bug to, are you? How can I report a bug? Is a support ticket the right way of reporting a bug to Adobe or should it be done in a different way? My boss had already reported earlier in the past the problems described in my "main question" as a support ticket to Adobe (he had done it before I have posted my questions here on Adobe Community), but the reply from Adobe had been the same as at the beginning of this discussion, i.e. that we have to live with what Adobe provides to us... 

I suppose that now, after finding out that it's a bug regarding the Javadoc jar, I shouldn't post it as an idea in the ideas section anymore, should I? (Do I assume correctly that an idea is treated with a lower priority than a bug?)