Deploying Oak Indexes to AEM as a Cloud Service

[evancooperman-rp]

Hi everyone! 

I was struggling a bit with this topic, so figured I'd post a discussion to help any others who are having issues.  The setup to load new or updated oak indexes into the cloud is fairly simple, and there's already documentation on how to do this...however, with the latest version of the filevault-package-maven-plugin plugin (v 1.0.5 - 1.2.2), you will get validation errors if you try to include an oak index in your ui.apps module.  If you've tried doing this, you have probably seen an error like

[INFO] Using 10 validators for package of type APPLICATION: jackrabbit-accesscontrol (org.apache.jackrabbit.vault.validation.spi.impl.AccessControlValidator), jackrabbit-filter (org.apache.jackrabbit.vault.validation.spi.impl.AdvancedFilterValidator), jackrabbit-properties (org.apache.jackrabbit.vault.validation.spi.impl.AdvancedPropertiesValidator), jackrabbit-docviewparser (org.apache.jackrabbit.vault.validation.spi.impl.DocumentViewParserValidator), jackrabbit-dependencies (org.apache.jackrabbit.vault.validation.spi.impl.DependencyValidator), jackrabbit-emptyelements (org.apache.jackrabbit.vault.validation.spi.impl.EmptyElementsValidator), jackrabbit-mergelimitations (org.apache.jackrabbit.vault.validation.spi.impl.MergeLimitationsValidator), jackrabbit-oakindex (org.apache.jackrabbit.vault.validation.spi.impl.OakIndexDefinitionValidator), jackrabbit-packagetype (org.apache.jackrabbit.vault.validation.spi.impl.PackageTypeValidator), jackrabbit-nodetypes (org.apache.jackrabbit.vault.validation.spi.impl.nodetype.NodeTypeValidator)

[ERROR] ValidationViolation: "jackrabbit-filter: Filter root's ancestor '/oak:index' is not covered by any of the specified dependencies nor a valid root.", filePath=META-INF/vault/filter.xml

[ERROR] ValidationViolation: "jackrabbit-oakindex: Package '/xxxx/ui.apps/target/xxxx.ui.apps-1.0.0-SNAPSHOT.zip' contains filter rule overwriting a potential index definition below '/oak:index/xxxx' but the according property allowIndexDefinitions is not set to 'true'", filePath=META-INF/vault/filter.xml

[ERROR] ValidationViolation: "jackrabbit-packagetype: Package of type 'APPLICATION' is not supposed to contain content outside '/libs' and '/apps'!", filePath=jcr_root/_oak_index/xxxx, nodePath=/oak:index/xxxx

The info on how to set up oak indexes in an AEMaaCS codebase is here: https://experienceleague.adobe.com/docs/experience-manager-cloud-service/operations/indexing.html?la...

Although the documentation is correct as far as how to structure the index in the codebase, it just doesn't take into account that the jackrabbit-packagetype validator will not let you deploy the code as it stands today on December 2, 2021.  Note that this will likely be fixed in the not so distant future with an update to the filevault plugin to use the latest validator module (where this is fixed), so this article might not be relevant for too much longer (hopefully!).  The answer I've seen on the forums has been to downgrade the filevault plugin to version 1.0.4, but this comes with its own set of problems as there are bugs in that version that will prevent you from doing things in your filter.xml files that you should be able to do.

So, if you want to stick with the current version of the filevault plugin, and still be able to deploy oak indexes to the cloud, this article is for you, so read on!

The latest versions of the validator do take oak indexes into account (3.5.x) and you can override the filevault plugin to use a newer version of the validator.  Because filevault-package-maven-plugin version 1.1.6 does not include the latest validator, you'll have to override that one dependency.  To do this, in the root pom.xml, you need to update the validator dependency within the filevault plugin:

<dependency>
    <groupId>org.apache.jackrabbit.vault</groupId>
    <artifactId>vault-validation</artifactId>
    <version>3.5.6</version>
</dependency>

 and then you also need to include the custom config that allows the oak:index node:

<validatorsSettings>
    <jackrabbit-packagetype>
        <options>
            <immutableRootNodeNames>apps,libs,oak:index</immutableRootNodeNames>
        </options>
    </jackrabbit-packagetype>
</validatorSettings>

By doing both of these things, you should be able to continue using version 1.1.6+ of the filevault plugin, and still deploy your oak indexes without downgrading.

Here's the full plugin configuration:

<plugin>
    <groupId>org.apache.jackrabbit</groupId>
    <artifactId>filevault-package-maven-plugin</artifactId>
    <extensions>true</extensions>
    <version>1.1.6</version>
    <configuration>
        <filterSource>src/main/content/META-INF/vault/filter.xml</filterSource>
        <validatorsSettings>
            <jackrabbit-packagetype>
                <options>
                    <immutableRootNodeNames>apps,libs,oak:index</immutableRootNodeNames>
                </options>
            </jackrabbit-packagetype>
            <jackrabbit-nodetypes>
                <options>
                    <!-- use the nodetypes and namespaces from the aem-nodetypes.jar provided in the plugin dependencies -->
                    <cnds>tccl:aem.cnd</cnds>
                </options>
            </jackrabbit-nodetypes>
        </validatorsSettings>
    </configuration>
    <dependencies>
        <dependency>
            <groupId>org.apache.jackrabbit.vault</groupId>
            <artifactId>vault-validation</artifactId>
            <version>3.5.6</version>
        </dependency>
        <dependency>
            <groupId>biz.netcentric.aem</groupId>
            <artifactId>aem-nodetypes</artifactId>
            <version>6.5.7.0</version>
        </dependency>
    </dependencies>
</plugin>

Also note that you'll still need to add 

<allowIndexDefinitions>true</allowIndexDefinitions>

 to your ui.apps/pom.xml file.  You won't need this with the most current version of the package type validator, but you will for the oak index validator.

Hope this helps!

Best,

Evan

[jamiec4451712]

Thank you! I was struggling with this for a while and the temporary solution was to set packageType to "mixed", which is obviously not a good approach. Followed your instructions and it resolved the issue immediately.

[evancooperman-rp]

That's awesome, I'm glad it worked!  This was driving me nuts, and I happened upon the Jackrabbit Jira ticket where this was discussed and resolved, so I was bent on making it work as soon as I saw that the validator was fixed!

[AMANATH_ULLAH]

@evancooperman-rp  @jamiec4451712 

I am getting the below error after following all the above mentioned steps. Please suggest

ValidationViolation: "jackrabbit-filter: Filter root's ancestor '/oak:index' is not covered by any of the specified dependencies nor a valid root.", filePath=META-INF\vault\filter.xml

My filter.xml in Ui.apps consists of below

<filter root="/oak:index/testsample-custom-1"/>

[NJ4]

Yes For me too. After doing all above steps all errors resolved but still getting ValidationViolation: "jackrabbit-filter: Filter root's ancestor '/oak:index' is not covered by any of the specified dependencies nor a valid root.", filePath=META-INF\vault\filter.xml

[Sean-McK]

Is the only way to solve the config.xml issue is to change it to mixed?

[ERROR] ValidationViolation: "jackrabbit-filter: Filter root's ancestor '/oak:index' is not covered by any of the specified dependencies nor a valid root.", filePath=META-INF\vault\filter.xml
[INFO] ValidationViolation: "jackrabbit-docviewparser: This file is not detected as docview xml file and therefore treated as binary", filePath=jcr_root\_oak_index\damAssetLucene-7-custom-1\tika\config.xml

?

[icegoffy]

Hi i made the same changes but i got this error when building : 

[ERROR] ValidationViolation: "jackrabbit-accesscontrol: Found no access control list, but there is supposed to be one contained as the property 'acHandling' is set to 'MERGE'!"

[icegoffy]

got it fixed, looks like i have to add this code pointed by the green arrow:

but now i got another error : 

[evancooperman-rp]

@icegoffy it looks like you're deploying wknd code.  Is there a reason you're needing to update indexes on the wknd project?  Normally you would start building project code from a fresh codebase generated from the latest archetype?  It's possible there's something in the wknd codebase that is causing issues, I'd have to dig in there further but not sure I'll have time to do that in the near future.

Also, I'm not sure that configuration is in the right place.  Here's the section of the pom.xml that should have the allow index definition property set to true:

<plugins>
    <plugin>
        <groupId>org.apache.jackrabbit</groupId>
        <artifactId>filevault-package-maven-plugin</artifactId>
        <extensions>true</extensions>
        <configuration>
            <allowIndexDefinitions>true</allowIndexDefinitions>
            <properties>
            .
            .
            .
        </configuration>
    </plugin>
</plugins>

[icegoffy]

thanks for the quick response, not sure why i didnt get a email notification

i was using wknd as a test implementation before adding it to my project

its quite confusing, on one hand, adobe said to add custom index to ui.apps (https://experienceleague.adobe.com/docs/experience-manager-cloud-service/content/operations/indexing...) , on the other hand this blog from adobe architect says to add it under ui.content (https://medium.com/adobetech/using-search-and-indexing-in-aem-as-a-cloud-service-cda0e10c944b)

so which one to follow, i tried both it failed to deploy the index