How to write custom link-checker java code and exports the json

[keshava219]

how to write a custom link-checker java code and exports the json weather its valid or not just like in HTML with broken link icon

-I want this validation/link checker functionality in custom java code format then i can export json and integrate with UI

-Exporter inform of JSON format as mentioned in below format ""isValidLink":true," any refrence please.

{
"cta":{
"InHref":"www.google.com",
"lnTarget":false,
"isValidLink":true,
}
}

[partyush]

Hi @keshava219

You Must try this solution it worked for me ! also let me know if this solves your issue or not. 

1. Project Setup:

• Make sure you have an AEM instance set up and ready for development.
• Set up your Java development environment with necessary tools and dependencies.

2. Create a Custom Service in AEM:

• Create an OSGi service in AEM that will handle the link checking functionality. This service will be responsible for link validation and JSON generation.

3. Link Validation Logic:

• Inside your AEM service, implement the logic to validate links. Use Apache HttpClient to make HTTP requests and check for valid responses (HTTP status codes 200-299).
• Handle potential exceptions and error cases gracefully.

4. JSON Generation:

• Create a Java class to represent the JSON structure for link validation results.
• Use a JSON library like Gson or Jackson (which are commonly used in AEM projects) to serialize your Java objects into JSON format.

5. Expose an API Endpoint:

• Create a servlet or a sling model to expose an API endpoint in AEM. This endpoint will trigger the link checking process.
• This endpoint can be a RESTful API that accepts a list of URLs to check.

6. Link Checking Implementation:

• When the API endpoint is called, your AEM service should iterate through the list of URLs and validate each link using the link validation logic you implemented earlier.

7. JSON Output:

• As you validate each link, populate instances of your Java class representing the JSON structure.
• Serialize these instances into JSON format using Gson or Jackson.

8. UI Integration:

• On the frontend/UI side, create a component or script that can interact with your API endpoint.
• Make an AJAX request to the API endpoint and handle the JSON response to display the validation results to users.

9. Deployment and Testing:

• Deploy your AEM project with the custom link checker functionality to your AEM instance.
• Test the functionality thoroughly with various types of URLs and scenarios to ensure accuracy and reliability.

Here's an example of what the AEM service and JSON structure might look like:

LinkValidationService.java:

import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.osgi.service.component.annotations.Component;

@Component(service = LinkValidationService.class)
public class LinkValidationService {
    private final CloseableHttpClient httpClient = HttpClients.createDefault();

    public LinkValidationResult validateLink(String url) {
        // Implement link validation logic using httpClient
    }
}

class LinkValidationResult {
    private String url;
    private boolean isValidLink;

    // Constructors, getters, setters
}

With this approach, you're leveraging AEM's capabilities to create a custom service that integrates seamlessly into your AEM instance. This allows you to perform link validation, generate JSON output, and integrate with the UI effectively. As always, thorough testing and consideration of potential performance bottlenecks are important steps before deploying the solution to a production environment.

Thanks 

[keshava219]

Hi @partyush ,

 Thanks for the reply and steps of solution, But im stuck on implementation of linkchecker method could you refer me any exact method of implementation of link checker. 

                
          cause i implemented method for link checker, model.json taking more time than i expect to load

CloseableHttpClient httpClient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet(url);
httpGet.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3");
try {
CloseableHttpResponse httpResponse = httpClient.execute(httpGet);
log.info("Https:::::::::::::::::"+httpResponse.getStatusLine().getStatusCode());
} catch (ClientProtocolException e) {
log.error("ClientProtocolException::::::::::----------{}",e);
} catch (IOException io) {
log.error("IO::::::::::----------{}",io);
}

[partyush · Accepted solution]

@keshava219 

in that case try using Timeouts 

Timeouts:Set reasonable timeouts for your HTTP requests to avoid waiting indefinitely for unresponsive URLs.

I have updated the service for timeouts : 

import org.apache.http.HttpResponse;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.osgi.service.component.annotations.Component;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.stream.Collectors;
import java.util.List;

@Component(service = LinkValidationService.class)
public class LinkValidationService {
    private final CloseableHttpClient httpClient;
    private final ExecutorService executor;

    public LinkValidationService() {
        RequestConfig requestConfig = RequestConfig.custom()
                .setConnectTimeout(5000) // 5 seconds
                .setSocketTimeout(5000) // 5 seconds
                .build();
        
        this.httpClient = HttpClients.custom()
                .setDefaultRequestConfig(requestConfig)
                .build();
        
        this.executor = Executors.newFixedThreadPool(10);
    }

    public List<CompletableFuture<LinkValidationResult>> validateLinks(List<String> urls) {
        return urls.stream()
                .map(url -> CompletableFuture.supplyAsync(() -> validateLink(url), executor))
                .collect(Collectors.toList());
    }

    private LinkValidationResult validateLink(String url) {
        HttpGet httpGet = new HttpGet(url);
        httpGet.addHeader("User-Agent", "Your User-Agent String");
        
        try {
            HttpResponse httpResponse = httpClient.execute(httpGet);
            int statusCode = httpResponse.getStatusLine().getStatusCode();
            return new LinkValidationResult(url, statusCode == 200);
        } catch (Exception e) {
            return new LinkValidationResult(url, false);
        }
    }

    public void shutdown() {
        try {
            httpClient.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
        executor.shutdown();
    }
}

class LinkValidationResult {
    private String url;
    private boolean isValidLink;

    public LinkValidationResult(String url, boolean isValidLink) {
        this.url = url;
        this.isValidLink = isValidLink;
    }

    // Getters and setters
}

With this example you can get Idea, how to proceed further !, Let me know if this helps

Thanks. 

[mahi1729]

I am assuming, that you are looking for the capability in SPA based pages. Here is what I can recommend. Just like what @partyush you will need a custom service to eventually do this. However, to invoke the custom link checker, for model.json calls, just add a sling filter that processes all model.json calls and maintain a configuration with all potential attributes that capture path type values. And probably use something like pathprop_valid as you may have more than 1 property that is of type path in your model.json. This combination of a custom service in form of a sling filter and a custom config gives you more control. Hope these insights help. Thanks

[keshava219]

Hi @mahi1729 ,
       Thanks for the reply can you give me any reference impl cause i already have some model.json invoked with few components 

            Could you refer me any method of link checker implementation

[mahi1729]

It's not that hard. Just take a look at any sling filter. You have ability to add type selector extension. So you can attach model and Json as selector and extension. Now traverse through your Json, check if runmode is author and if wcmmode is edit (assuming this is only for inside author). Now for all configured properties check the paths. If the paths can resolve to a valid resource inside aem you will leave it as is, but if it's a non existing resource add addl property to indicate it is invalid. Now in your react component before rendering any link pass this second property also to indicate it's a broken link. Understand that here we are determining if path is inside aem. But if it's an absolute url starting with http or so u can take inputs from @partyush response. Hope this helps. Thanks 

[Mahedi_Sabuj]

Hi @keshava219, The AEM Core Navigation (v2) Component has provided the response for the link that matches your expectations. You can verify this by adding the Navigation Component to any page and examining the model response using the Model API like below: http://localhost:4502/content/aem-demo/au/en/jcr:content/root/responsivegrid/navigation.model.json 

The response from the Model API will response the JSON structure below:

{
  "id": "navigation-bc9f0148e6",
  "items": [
    {
      "id": "navigation-bc9f0148e6-item-c7b45e7771",
      "path": "/content/aem-demo/us/en/test-page",
      "link": {
        "valid": true,
        "url": "/content/aem-demo/us/en/test-page.html"
      },
      ":type": "aem-demo/components/content-page"
    }
  ],
  ":type": "aem-demo/components/navigation"
}

 The Navigation Component utilizes the LinkManagerImpl.java to generate the response for the link.

[keshava219]

Hi @Mahedi_Sabuj ,

 Its working for internal links like starts with "/content" so on.

but im looking for external client side urls validation do you have any reference for that please let me know