SRP API - "Contains" Search or Full Text UGC Filtering?

[Gdubz-57m2mu]

Is it possible to filter strings by "contains" or maybe a full text search for UGC?

I've tried a ValueConstraint<String>[1] but looks like I can only perform a "begins with" search based on the available ComparisonTypes[2]. Even still, this doesn't seem to work...

final UgcFilter filter = new UgcFilter(); // ... final String titleFilterParam = request.getParameter("title"); ConstraintGroup optionalFilters = new ConstraintGroup(Operator.Or); Constraint titleFilter = new ValueConstraint<String>("postTitle", titleFilterParam, ComparisonType.BeginsWith); optionalFilters.addConstraint(titleFilter); filter.and(optionalFilters);

So then I tried a FullTextConstraint[3] on a specific property and that doesn't seem to work either.

final UgcFilter filter = new UgcFilter(); // ... final String titleFilterParam = request.getParameter("title"); ConstraintGroup optionalFilters = new ConstraintGroup(Operator.Or); optionalFilters.addConstraint(new FullTextConstraint(titleFilterParam, "postTitle"); // not sure which wildcard is valid (if any), no documentation on this optionalFilters.addConstraint(new FullTextConstraint("*" + titleFilterParam + "*", "postTitle"); optionalFilters.addConstraint(new FullTextConstraint("%" + titleFilterParam + "%", "postTitle"); filter.and(optionalFilters);

I've looked at the only filtering examples I've seen from Adobe on the github here[4] but that only shows an example of a ValueConstraint<Boolean> and a PathConstraint[5]. String value comparison using ValueConstraint<String> also seems to have a hard time with capitalization and spaces, so I've had to lowercase and replace all spaces with hyphens for string comparison, for now. Is there a better way to do this?!

I guess I could always just filter in my code after the fact, but figured we should get the UgcFilter correct the first time. This is extremely frustrating though, considering I can't get a printout of what the final UgcFilter looks like before querying Solr (like what the final XPath/JCR-SQL2/etc query string is).

[1] https://docs.adobe.com/docs/en/aem/6-2/develop/ref/javadoc/com/adobe/cq/social/ugc/api/ValueConstrai...
[2] https://docs.adobe.com/docs/en/aem/6-2/develop/ref/javadoc/com/adobe/cq/social/ugc/api/ComparisonTyp...
[3] https://docs.adobe.com/docs/en/aem/6-2/develop/ref/javadoc/com/adobe/cq/social/ugc/api/FullTextConst...
[4] https://github.com/Adobe-Marketing-Cloud/aem-communities-todomvc-sample/blob/master/bundles/aem-comm...
[5] https://docs.adobe.com/docs/en/aem/6-2/develop/ref/javadoc/com/adobe/cq/social/ugc/api/PathConstrain...

[calvin_wong]

You can get the search queries, SQL2 for JSRP or Lucene for MSRP, if you enable DEBUG logging for com.adobe.cq.social.ugc.impl.LuceneUgcSearch. With the debug queries, you can try them directly in CRX/DE for JSRP or the Solr Admin console for MSRP.

[Gdubz-57m2mu]

calvin.wong wrote...

You can get the search queries, SQL2 for JSRP or Lucene for MSRP, if you enable DEBUG logging for com.adobe.cq.social.ugc.impl.LuceneUgcSearch. With the debug queries, you can try them directly in CRX/DE for JSRP or the Solr Admin console for MSRP.

That's helpful, thank you Calvin!

I'll try to figure out what I'm doing wrong from there. Will update this thread with my findings.

[Gdubz-57m2mu]

So, in the Solr Admin panel, I can do a search like:

postTitle_s:*atta*

And it'll return results whose "postTitle_s" property CONTAINS "atta" in it, like one particular post example title: "Post with Attachments". This is what I was hoping for. 

However, when I try to create that query in Java using ValueConstraint<String>, it's escaping the wildcard, making it not work. This is the code:

final UgcFilter filter = new UgcFilter(); // ... other required filters ... final String titleFilterParam = request.getParameter("title"); ConstraintGroup optionalFilters = new ConstraintGroup(Operator.Or); Constraint titleFilter = new ValueConstraint<String>("postTitle_s", "*" + titleFilterParam + "*"); optionalFilters.addConstraint(titleFilter); filter.and(optionalFilters);

And this is the debugged query log, in which you can see the wildcard "*" is being escaped with a "\", thus making the query not work:

29.11.2016 08:38:34.574 *DEBUG* [0:0:0:0:0:0:0:1 [1480430314573] GET HTTP/1.1] com.adobe.cq.social.ugc.impl.LuceneUgcSearch Component: null lucene query is: +jcr:primaryType:cq:Comment +:path:\/content/usergenerated/asi/mongo/content/myproject/en/home/jcr\:content/posts/* +(+*:* -isDeleted_b:true) +postStatus_s:APPROVED +postCategory_s:for\-sale +postTitle_s:\*atta\*

Any tips on what to do? Any idea if the escaping is happening with even the most basic AbstractConstraint?

[calvin_wong]

The wildcard search in Solr is a known bug and it has been fixed. The fix is in AEM-6.2-COMMUNITIES-FEATURE-PACK-2, which will be uploaded to package share in the next few days. Will you be able to use this new feature pack? The same fix is also available for AEM 6.1 in AEM-6.1-COMMUNITIES-FEATURE-PACK-6, which will be uploaded to package share at the same time.

[Gdubz-57m2mu]

calvin.wong wrote...

The wildcard search in Solr is a known bug and it has been fixed. The fix is in AEM-6.2-COMMUNITIES-FEATURE-PACK-2, which will be uploaded to package share in the next few days. Will you be able to use this new feature pack? The same fix is also available for AEM 6.1 in AEM-6.1-COMMUNITIES-FEATURE-PACK-6, which will be uploaded to package share at the same time.

Yes, we'll be able to use it. Currently using AEM 6.2 with Communities FP1 on my local. Looking forward to the release of Communities FP2!

Thank you for the information, yet again, Calvin.

[Gdubz-57m2mu]

calvin.wong wrote...

The wildcard search in Solr is a known bug and it has been fixed. The fix is in AEM-6.2-COMMUNITIES-FEATURE-PACK-2, which will be uploaded to package share in the next few days. Will you be able to use this new feature pack? The same fix is also available for AEM 6.1 in AEM-6.1-COMMUNITIES-FEATURE-PACK-6, which will be uploaded to package share at the same time.

Calvin,

Did this not make it into the Communities FP2 release[1]?

Took an update, installed the code in my local, performed the ValueConstraint with wildcard and still having the same escaping issue.

[1] https://helpx.adobe.com/experience-manager/kb/aem62-available-hotfixes/aem62-communities-hotfixes.ht...

[calvin_wong]

Can you use FullTextConstraint and give it a try?

If you're searching for jcr:title or jcr:description, can you also try it in the search component http://host:port/content/community-components/en/search.html?

[Gdubz-57m2mu]

calvin.wong wrote...

Can you use FullTextConstraint and give it a try?

If you're searching for jcr:title or jcr:description, can you also try it in the search component http://host:port/content/community-components/en/search.html?

In the Solr admin panel, on the "query" page under the specific collection, what would the fulltext search filter look like? I'm seeing that the FullTextConstraint is logging without escaping the wildcards now, which is great, but it's still not returning anything...

This is the code (just because I wasn't sure which one would work):

ConstraintGroup titleFilterGroup = new ConstraintGroup(Operator.Or); titleFilterGroup.addConstraint(new FullTextConstraint(titleFilterParam, "postTitle_s")); titleFilterGroup.addConstraint(new FullTextConstraint("*" + titleFilterParam + "*", "postTitle_s")); titleFilterGroup.addConstraint(new FullTextConstraint("%" + titleFilterParam + "%", "postTitle_s")); titleFilterGroup.addConstraint(new FullTextConstraint(titleFilterParam));

This is the debug log:

05.12.2016 13:50:10.727 *DEBUG* [0:0:0:0:0:0:0:1 [1480967410721] GET HTTP/1.1] com.adobe.cq.social.ugc.impl.LuceneUgcSearch Component: null lucene query is: +jcr:primaryType:cq:Comment +:path:\/content/usergenerated/asi/mongo/content/myproject/en/home/jcr\:content/posts/* +(+*:* -isDeleted_b:true) +postStatus_s:APPROVED +:fulltext:postTitle_s:(pend) +:fulltext:postTitle_s:(*pend*) +:fulltext:postTitle_s:(%pend%) +:fulltext:(pend) +postCategory_s:for\-sale

[calvin_wong]

Enable DEBUG logging for com.adobe.cq.social.srp.impl.SocialSolrConnector.

You should then get the actual Solr query in the debug log. Should look something like this:

Query to solr is: sort=timestamp+desc&bl=en&pl=en&start=0&rows=10&q=%2Btitle_t:(*hello*)+%2Bprovider_id:\/content/usergenerated/asi/mongo/content/*+%2Bresource_type_s:*&df=provider_id&trf=verbatim&fq={!cost%3D100}report_suite:mongo

The query is the "q=" part. It's URL encoded, so use a web tool like http://meyerweb.com/eric/tools/dencoder/ to decode it and you should be able to paste the query into the Solr Admin Query tool.

[Gdubz-57m2mu · Accepted solution]

Thank you sooooo much for sticking with me on this one, Calvin! You're incredible.

Got it working with the following code:

final UgcFilter filter = new UgcFilter(); // ... required filters ... final ConstraintGroup optionalGroup = new ConstraintGroup(Operator.And); // ... other optional filters ... // Title "Contains" Filter final String titleFilterParam = request.getParameter("title"); if (StringUtils.isNotEmpty(titleFilterParam)) { optionalGroup.addConstraint(new FullTextConstraint("*" + titleFilterParam + "*", "postTitle_s"); } filter.and(optionalGroup);

Again, can't thank you enough!

[Shubham_borole]

It looks like we still have an issue with the "contains" text search.

The FullTextConstraint API does work partially with the wildcard, but if the search text contains a space it doesn't return any results.

For eg:- If we have an entry as "Testing rich text contents", we can use FullTextConstraint with the wildcard and if the user enters "rich" or "text" it returns us the matching record which is "Testing rich text contents", but if we enter the search term as "rich text" there are no records returned (The expected result is "Testing rich text contents").

We have tried the solution that was provided here and additionally we have tried the following as well (as suggested here),

final ConstraintGroup optionalGroup = new ConstraintGroup();
if (StringUtils.isNotEmpty(searchText))
{
optionalGroup.or(new FullTextConstraint("/.*" + searchText + ".*/i", PropertyConstants.TITLE_T));
}
requiredGroup.and(optionalGroup);

Not sure if we still have a bug that is prohibiting the API to search for text's with a space or if there is a workaround to implement wildcard searches that can accept space?

Can anyone from the community group help and provide a solution for this issue?