AEM Search Solution

[PhilipBae]

Hi all,

Has anyone implemented an enterprise level search solution for AEM?

The out-of-box Lucene is just terrible and not scalable as it has to be served up from Publisher. I tried Apache Solr with OOTB integration. This is also not a solution as query HAS to come back to Publisher and it is Publisher that instead of using its Lucene, it just extends Solr for queries, which helps Publisher, but Publisher still becomes the bottleneck.

I could write up a sling module to spit out XML structure of the content, then use Solr to index separately, but this requires a lot of development.. Is there any search solution that has this integration already built?

Much appreciated!

Philip

[daniel_henriqu1]

Hi, Philip.

Are you aware of the thread Implementing site search?

Regards,

Daniel.

[PhilipBae]

Hi Daniel,

Been through all that Solr <-> AEM integration. But like I explained, search has to go through Publisher, which causes bottleneck and is not scalable solution.

An ideal search solution would be that there is a standalone search server that hooks into AEM Publisher, indexes all it needs to, at the meta data level (nodes) (just like how Lucene does with AEM), and then be able to hit it externally via API.

This way, we can simply use AJAX to grab search results by hitting this external standalone search server. AEM/Solr integration OOTB does not provide this.

Thanks!
Philip

[karthickky2k4]

Search and Promote is an option worth spending your time on. Do check.

[daniel_henriqu1]

Hi, Philip.

Sorry, but I couldn't follow you...

As far as I've understood, the recommend solution is to use a 3rd party service/platform to index AEM content. This way your website (I mean its client side) would hit that service/platform to search for a specific content instead of hitting the publish instance .

Or do you have a hard requirement that demands accessing the search service through the publish instance?

Regards,

Daniel.

PhilipBae wrote...

Hi Daniel,

Been through all that Solr <-> AEM integration. But like I explained, search has to go through Publisher, which causes bottleneck and is not scalable solution.

An ideal search solution would be that there is a standalone search server that hooks into AEM Publisher, indexes all it needs to, at the meta data level (nodes) (just like how Lucene does with AEM), and then be able to hit it externally via API.

This way, we can simply use AJAX to grab search results by hitting this external standalone search server. AEM/Solr integration OOTB does not provide this.

Thanks!
Philip

[PhilipBae]

Hi, didn't that product become Adobe Target? From what I know, Target does not serve up search results but targeting content doing A/B testing.

Lmk!
Philip

[PhilipBae]

Hi Daniel,

I think we are on the same page. That is exactly what I meant. My requirement is to stay off of Publish instance because it becomes a performance bottleneck and was hoping if anyone knew a good enterprise-level search engine that already has good integration built that we can purchase.

There are a few search solutions out there like: Apache Solr (free), Oracle Endeca, Microsoft FAST, Google Search Appliances, to name a few, but they do not have any prebuilt integration with AEM.

Lmk! Thanks so much!
Philip

[daniel_henriqu1]

Yes. I believe we are on the same page :-)

Here, for example, we've developed a custom solution using Amazon CloudSearch:

• some preprocessors index the page content (I think this could be done in a Replication Agent instead);
• the page templates and components include a custom client library;
• this client library makes AJAX calls to the CloudSearch endpoint;
• some portions of the page/component are re-rendered based on that results (we use an Angular-like framework to do it).

Regards,

Daniel.

[valcohen]

@PhilipBae, I'll second @karthickky2k4's recommendation to look at Adobe Search and Promote (SnP). We're using it on two sites. I'm not sure what you mean by "search has to go through Publisher", but in our case load on the Publish servers is low and controllable, because it's only hit when indexing content, not when conducting the search or presenting results.

We're using SnP to search our product catalog, not (yet) for full-text search of AEM pages. Catalog inventory can change throughout the day. An SnP crawler comes along every half hour and queries the JCR for nodes that have changed since the last crawl, and updates the SnP index. We also do a full re-index every night.

To search, we present a form to get the search terms, then issue an AJAX call from the page to SnP, which in our case returns JSON data that the front-end consumes and formats for presentation. So, our solution is heavy on the JavaScript side, but not on the Publish server.

SnP is not without problems (but what is?): documentation is very sparse, it's a cloud-hosted solution with limitations on testing environments, no ability to export settings from one environment and import into another, and some limitation on how you can search -- e.g. from what I understand, it's not possible to have multiple conditions on the same "column" like "SquareFootage > 1000 AND SquareFootage < 5000". We get around this by creating extra fields and mapping the same value to them: SquareFootage -> SFMin and SFMax , and the query becomes "SFMin > 10000 AND SFMax < 5000". In other instances we do only very simple filtering in the SnP query, and further filter the results on the page in JavaScript.

The benefits are that it doesn't impose a lot of load on the Publisher (again, depending on what you mean by "a lot of load"), and is supported by Adobe, which may outweigh the limitations.

[maruthid1 · Accepted solution]

Ideally your enterprise search solution should not depend on whether integration with AEM available or not. You will end up using this search solution across different web properties which could be built using different platforms.

AFAIK AEM don't have any integration with any of the search solutions you mentioned but for most of them either passing url or xml feed will suffice on content publish from AEM. In case of GSA it is pretty simple (but note Google is changing licensing model it might cost you more). 

For front end integration with AEM , make sure views are still managed in AEM and responses are from search engine. Use one of the data binding java script frameworks like angular so that load is on the search server where as still the page layout changes can be published out of AEM.

[PhilipBae]

I agree! Thanks for the feedback!!

Philip

[PhilipBae]

@valcohen -- I had no idea Search&Promote was a cloud-based search solution that Adobe offered. I just could not find anything about it from search on web. After attending one of the session at the Adobe Summit 2017, I now know! I ended up grabbing a bunch of screenshots.

Thanks!!
Philip

[PhilipBae]

@daniel_henrique

I had no idea AWS had CloudSearch. It looks pretty robust solution very comparable to Search&Promote. Will look into it. Thanks so much!

Philip

[valcohen]

@PhilipBae heh, I was at Summit, and I had the opposite impression -- I was afraid Search&Promote was on its way out, since I didn't hear it mentioned once! I guess they're emphasizing the "Target" part of the name. So far we're using it to good effect -- I hope it works out for you.

[PhilipBae]

@valcohen -- Yeah.. Search&Promote is definitely going through a few transition. There isn't a good web page for it other than the SnP documentation. My understanding is that instead of promoting SnP alone, they want to provide it as Adobe Target brand where that suite of product offers advanced targeting/personalization. When I asked the SnP product manager about the licensing, he said that I probably will need to buy it as an add-on to Adobe Target Standard and Premium. Definitely confusing and vague.