UTF-8 fix breaks content-type headers?

[TB3dock]

By default, AEM queryBuilder, which is how servlets can lookup content, responds with the data encoded in iso-8859 char set instead of the required UTF-8.  The solution, which seems improbable, but works, is to create /osgiconfig/config/org.apache.sling.engine.impl.SlingMainServlet.cfg.json file with the following content:

{
"sling.additional.response.headers":[
  "X-Content-Type-Options=nosniff",
  "Content-Type=text/html;charset=utf-8",
  "X-Frame-Options=SAMEORIGIN"
  ]
}

This fixes the queryBuilder returning data with the wrong charset in the servlet, but has the side effect of making the response header always "text/html".  We want our servlets, which are used by the front end to search for and pull page pages as json, to have a content-type of application/json.  however, if we change the above file, it will then break AEM.

We tried overriding the content type in the servlet itself, but this gets ignored (or overwritten).

Any ideas how to support both UTF-8 and application/json just for our servlets?

[Asutosh_Jena_]

Hi @TB3dock 

Please check "Apache Sling Request Parameter Handling" configuration in /system/console/configMgr and see what value "sling.default.parameter.encoding" property is holding. If it's set to "ISO-8859-1" please change it to "UTF-8"

Default value should be "UTF-8".

[TB3dock]

Hi this is for parameter encoding, so has nothing to do with the servlet response I believe. This value was set to UTF-8 when we started.

[Jörg_Hoh]

I just checked and the encoding for the output of /bin/querybuilder.json is set to UTF-8 since ages explicitly. What AEM version are you using?

[TB3dock]

Its the cloud version. Without that content type setting, anything we read from query builder is in iso-8859, even if writing to log files or writing to response. With the setting, we get utf-8, but lose the ability to set the content type to application/json which is a real pain.

[Jörg_Hoh]

When you say "it's the cloud version", you mean the version hosted at Adobe or the SDK running on your local machine?

[TB3dock]

its the cloud SDK running locally, but I assume its the same issue on the hosted version.  Ultimately, both have to work, and currently I can only chose between getting a UTF-8 char response, or a JSON response, I cant have both which is an issue for the API integrations which expect application/json response type.

[Jörg_Hoh]

I am very sure that the hosted environments run in an fully UTF-8-ified environment, so there must not be any other encoding. Also I am not aware of any other report of broken encoding in this code, you are the first one. Can you please check your local character encoding and also please validate on a DEV environment in CS, if you see the behavior there as well?

[mauricio_rodri2]

+1 having the same issue. Could solve this using the code below (be sure to maintain the order! doesn't take the value otherwise):

response.setStatus(statusCode);
response.setContentType(ContentTypeUtil.TYPE_JSON);
response.setCharacterEncoding(CharEncoding.UTF_8);
response.getWriter().write(json.toString());

[talkeshwar]

mauricio_rodri2 

Thanks for the solution I had the same issue and resolved with your answer  

[TB3dock]

This solution didnt work for us, because we have so many servlets, maintained by different teams, returning responses in hundreds of places in different ways.   AEM forces you to choose between UTF-8 or application/json, you can't have both (without a significant rewrite of the code and very large cost of retesting every aspect).  I would say that the "side effect" of setting UTF-8 causing the response type to change to text is a pretty serious bug.