Thanks all !!
I have successfully integrated ACDL with AEM. Posting the steps here, so that it can help everyone. Happy learning 🙂
1. Upgrade core.wcm.components version to 2.9.0 or higher if not already done.

2. Since was enabling this capacity in older codebase, hence had to manually enable the context aware configuration for this for our sites using below steps. Or you can copy this node from wknd site to your site node.

3. Now if the page component has been extended for your site, make sure it would include the below script in body tag. You can refer ootb page component rendering html -> /apps/core/wcm/components/page/v2/page/page.html
id="${page.id}"
data-cmp-data-layer-enabled="${page.data ? true : false}">
<script data-sly-test.dataLayerEnabled="${page.data}">
window.adobeDataLayer = window.adobeDataLayer || [];
adobeDataLayer.push({
page: JSON.parse("${page.data.json @ context='scriptString'}"),
event:'cmp:show',
eventInfo: {
path: 'page.${page.id @ context="scriptString"}'
}
});
</script>
4. This would enable acdl for pages in your site. To test this go to published view of the page and type window.adobeDataLayer.getState() in console.
5. With this you will get empty page object. Now in order to populate this, you will have to override getData() method in your extended custom model. Below could be used, if we just need whatever is exposed ootb->
@Self
@Via(type = ForcedResourceType.class, value = "core/wcm/components/page/v2/page")
private Page superTypePage;
@Override
public ComponentData getData() {
return superTypePage.getData();
}
6. Thi same could be achieved for any extended ootb components, by overriding the get data method in extended slingmodel and adding datalayer data attribute in extended rendering html using below code ->
data-cmp-data-layer="${<model sly use reference>.data.json}"
Reference articles ->
https://experienceleague.adobe.com/en/docs/experience-manager-core-components/using/developing/data-layer/overview
https://github.com/adobe/adobe-client-data-layer/blob/master/README.md
https://unpkg.com/browse/@adobe/adobe-client-data-layer@2.0.1/dist/adobe-client-data-layer.min.js
https://experienceleague.adobe.com/en/docs/experience-manager-learn/sites/integrations/adobe-client-data-layer/data-layer-customize#set-up-wknd-site
https://github.com/adobe/aem-guides-wknd
https://medium.com/@abhaymishra12/customizing-adobe-client-data-layer-during-adobe-analytics-implementation-9dc357c8068e