since ‎18-10-2016
‎10-12-2019
michaelh2862615
Level 2
Re: Disk Usage Report much smaller than disk space consumed
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
Sure. I have just submitted a ticket. Thanks for everyone's help. I will update if Adobe comes back with anything insightful.

Views

1.3K

Like

1

Replies

0
Re: Disk Usage Report much smaller than disk space consumed
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
Yes, I ran offline compaction multiple times and even tried removing all checkpoints at some point as recommended above.

Views

867

Like

1

Replies

0
Re: Disk Usage Report much smaller than disk space consumed
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
I am not sure what the best way to provide a report about the file differences. I can tell you a quick summary comparing a difference in files between the old and new datastore:diff -qrN datastore/ datastore-bak/ | wc -l534635The AEM Usage Report (/etc/reports/diskusage.html) is basically identical -- I do not have an exact comparison on numbers, but the difference is negligible for our purposes.

Views

868

Like

1

Replies

0
Re: Disk Usage Report much smaller than disk space consumed
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
Here is what I have done that seems to have resolved the issue. If someone could let me know if there is some issue I am not seeing here but otherwise, it seems to have worked beautifully.java -jar crx2oak-1.8.6-all-in-one.jar segment-old:/content/aem/crx-quickstart/repository segment-old:/content/backup/ --include-path=/ --src-datastore=/content/aem/datastore --datastore=/content/backup/datastore/Running this command to pretend as if I am upgrading, but using "segment-old" for source and target...

Views

864

Like

1

Replies

0
Re: Disk Usage Report much smaller than disk space consumed
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
Unfortunately, I cannot do that since the server is remote and does not have a display.Anything else I can try? I have even updated to Oak 1.4.24 on a cloned server and deleted indices and compacted twice. I have not been able to free any space of the datastore. Is there some way to clone the datastore through some command (maybe the oak-run jar) that won't just copy byte-for-byte all the files on the disk and might produce something trimmed down?

Views

864

Like

1

Replies

0
Re: Disk Usage Report much smaller than disk space consumed
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
We have a cloned server already. So I stopped the application. I deleted crx-quickstart/repository/index. I ran java -jar oak-run-1.4.17.jar checkpoints /content/aem/crx-quickstart/repository/segmentstore/ rm-all. I started the server. I waited for the index to be built. I ran garbage collection. I checked the Disk Usage report. Still the same space is used in the datastore on the disk and the disk usage report is still showing the same, much lower number.

Views

863

Like

1

Replies

0
Re: Disk Usage Report much smaller than disk space consumed
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
Wouldn't a repository consistency check, which we also run daily, catch these corruptions? I will try this and follow up.

Views

865

Like

1

Replies

0
Re: Disk Usage Report much smaller than disk space consumed
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
All files over 100 MB are zips. I have inspected one of the files over 6 GB. They look like packages that were backups, but these packages do not exist anymore. I have checked all packages from the Package Explorer multiple times to confirm. Is there something I can do about these files in the meantime? Why would they not be garbage collected?This would be helpful in cleaning up at least a few GB, but the question about the over 150GB+ will still remain. 😕

Views

865

Like

1

Replies

0
Re: Disk Usage Report much smaller than disk space consumed
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
Not a shared datastore. How can I answer your question about embedded or external TarMK datastore? I believe it is a TarMK datastore, the run modes include crx3tar.

Views

866

Like

1

Replies

0
Re: Disk Usage Report much smaller than disk space consumed
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
I have seen that post and the linked maintenance document before posting and performed all those operations, including audit log purge (nothing older than five days).

Views

862

Like

1

Replies

0
Re: Disk Usage Report much smaller than disk space consumed
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
Yes. I performed offline compaction. Then I did various ways of garbage collection. I turned off the application, ran compaction, turned it on, and ran garbage collection, including the following command which should be clear that I have.curl -silent -u username:password -X POST --data markOnly=false http://localhost:4502/system/console/jmx/org.apache.jackrabbit.oak%3Aname%3Drepository+manager%2Ctype%3DRepositoryManagement/op/startDataStoreGC/boolean27.06.2019 18:10:36.159 *INFO* [qtp1670602695-...

Views

862

Like

1

Replies

0
Re: Disk Usage Report much smaller than disk space consumed
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
I appreciate the assistance, but it's not helpful if my message is not read. We run garbage collection...daily.I have searched for hours before posting and read all of Adobe's documentation on maintenance and many articles on this forum. However, I will repeat that we run version cleanup, workflow cleanup, and data store garbage collection daily. I will also repeat we have performed compaction.The data store has grown to this over 2 years or more, it is not suddenly growing rapidly, so a TRACE w...

Views

858

Like

1

Replies

0
Disk Usage Report much smaller than disk space consumed
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
We have an AEM 6.2 instance with Hotfix 17578 (cq-6.2.0-hotfix-17578) installed so we are on Oak 1.4.17 and CFP18.We run garbage collection, version cleanup, and workflow cleanup daily. We have a separate datastore and segment store. We have performed offline compaction but that only applies to the segmentstore.The disk usage report in AEM (/etc/reports/diskusage.html) reports we are using ~18 GB of data. However, our datastore has grown to ~170 GB of data. I cannot find any way to figure out ho...

Views

6.1K

Likes

0

Replies

25
Re: Issues with XPath Queries
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
That resolved the issue, thanks. I found the function to escape illegal characters from that exact page, but I was not sure why I would need to to replace characters if they were properly escaped in the first place. The document doesn't discuss why the replaceAll is there after either. A little odd. Now, however, the query contains:jcr:like(fn:lower-case(@jcr:title), '%ma''a%salama%'Will this be able to match ma'a salama using two single quotes between the a's?

Views

273

Likes

0

Replies

0
Re: Issues with XPath Queries
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
I would agree but I do escape it, using org.apache.jackrabbit.util.Text.escapeIllegalXpathSearchChars. I have also tried Text.escapeIllegalJcrChars. Is there no existing function to escape the query automatically or do I have to write something?

Views

271

Likes

0

Replies

0
Issues with XPath Queries
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
I have a servlet that I use to search the JCR based on a user-provided query. I log the string that is to be used as the query before running it. However, I get a parse error when running the query and the error shows a differently formatted query than the string logged. Am I doing something wrong or is there a bug in the stack that parses/formats the query? Below is a servlet that should produce the error and the log messages I get. The query used is "ma'a salama". req below is the object from ...

Views

459

Likes

0

Replies

6
Re: Cannot Create New Project in AEM 6.2 with Dispatcher 4.2.0
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
For anyone else having this issue, installing Hotfix 12785 and Service Pack 1 resolves the issue.

Views

363

Likes

0

Replies

0
Cannot Create New Project in AEM 6.2 with Dispatcher 4.2.0
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
I have a completely new installation of AEM 6.2 with a dispatcher, 4.2.0, running on the same server.When I access the application directly, I can create a new project (/mnt/overlay/cq/core/content/projects/wizard/newproject.html/content/projects).When I access the application through the dispatcher, using the default author dispatcher configuration, I cannot create a new project. If I swap out a new installation of AEM 6.2 with AEM 6.1, I can create a project in AEM 6.1.In response to the AJAX ...

Views

457

Like

1

Replies

1
Re: Richtext spell checker does not work with locale?
Avatar

michaelh2862615

michaelh2862615
- Adobe Experience Manager
This does not address the OP's question. How has this been marked as solved? To re-iterate, the question was: "Please suggest if there is something wrong in what I am doing or it is a bug in CQ?"I am having the same issue and it sounds like a bug. If there is a bug and therefore the only solution, rather than releasing a fix for the bug, is to create our own servlet to spellcheck, then that would be a clearer response, albeit a still mostly unhelpful response as this would then require knowledge...

Views

184

Likes

0

Replies

0
Likes from