[AEM 6.5 LTS] SegmentNotFoundException after offline compaction — root cause, hotfix, and fix (OAK-12119) | Community
Skip to main content
Adobe Employee
April 8, 2026
Solved

[AEM 6.5 LTS] SegmentNotFoundException after offline compaction — root cause, hotfix, and fix (OAK-12119)

  • April 8, 2026
  • 1 reply
  • 38 views

Hi community,

  We want to share details about a critical issue affecting AEM 6.5 LTS SP1 and AEM 6.5 LTS SP2 customers who perform offline compaction using oak-run.

  ---

 What is happening

  After running offline compaction via `oak-run compact` followed by two rounds of online garbage collection (GC), AEM instances were throwing SegmentNotFoundException (SNFE) and becoming inaccessible. 

 Who is affected

  Customers on AEM 6.5 LTS SP1 or SP2 who have run offline compaction with an oak-run jar that does not include this fix. This was a blocker for all customers upgrading to AEM 6.5 LTS SP1/SP2.

Best answer by RishabhKu1

Root cause

  The bug OAK-12119 is in how oak-run compact works internally:

  - Offline compaction calls compactFull() and cleanup() as two separate operations.
  - The CompactionResult from compactFull() was silently discarded before cleanup() ran.
  - cleanup() then substituted a SKIPPED placeholder result, which has requiresGCJournalEntry() = false.
  - As a result, no entry was written to gc.log after offline compaction.
  - On the next two rounds of online GC, Oak's reclaimer had no knowledge of the compacted segments and began reclaiming them — causing SNFE when AEM tried to read those deleted segments.

  The fix (apache/jackrabbit-oak#2779, merged March 2026) stores the CompactionResult after compaction and passes it correctly to cleanup(), ensuring gc.log is
  always written.

  Fix — AEM 6.5 LTS SP2

  1. Install the hotfix from Software Distribution:
     https://experience.adobe.com/#/downloads/content/software-distribution/en/aem.html?package=/content/software-distribution/en/details.html/content/dam/aem/public/adobe/packages/cq660/hotfixes/cq-6.5.lts.2-hotfi
  x-GRANITE-65388-1.0.zip

  2. For offline compaction, use the matching oak-run jar from Software Distribution:
     https://experience.adobe.com/#/downloads/content/software-distribution/en/aem.html?package=/content/software-distribution/en/details.html/content/dam/aem/public/adobe/packages/cq660/hotfixes/oak-run-1.88.1-B00
  6.jar

  3. Add the following system property to your AEM JVM start arguments to avoid the related disk usage regression (OAK-12134):
     -Doak.compaction.legacy=true

  ---

 Fix — AEM 6.5 LTS SP1

  1. Install the hotfix from Software Distribution:
     https://experience.adobe.com/#/downloads/content/software-distribution/en/aem.html?package=/content/software-distribution/en/details.html/content/dam/aem/public/adobe/packages/cq660/hotfixes/cq-6.5.lts.1-hotfi
  x-GRANITE-65385-1.0.zip

  2. For offline compaction, use the matching oak-run jar from Software Distribution:
     https://experience.adobe.com/#/downloads/content/software-distribution/en/aem.html?package=/content/software-distribution/en/details.html/content/dam/aem/public/adobe/packages/cq660/hotfixes/oak-run-1.78.1-B00
  6.jar

  ---

  Important: Always use an oak-run jar that matches the exact Oak version bundled in your AEM installation. Using a mismatched or older oak-run jar will leave gc.log unwritten and cause SNFE after the next two
  online GC cycles.

  References
  - OAK-12119: https://issues.apache.org/jira/browse/OAK-12119
  - Fix PR: https://github.com/apache/jackrabbit-oak/pull/2779
  - Related disk usage regression: https://issues.apache.org/jira/browse/OAK-12134

1 reply

RishabhKu1Adobe EmployeeAuthorAccepted solution
Adobe Employee
April 8, 2026

Root cause

  The bug OAK-12119 is in how oak-run compact works internally:

  - Offline compaction calls compactFull() and cleanup() as two separate operations.
  - The CompactionResult from compactFull() was silently discarded before cleanup() ran.
  - cleanup() then substituted a SKIPPED placeholder result, which has requiresGCJournalEntry() = false.
  - As a result, no entry was written to gc.log after offline compaction.
  - On the next two rounds of online GC, Oak's reclaimer had no knowledge of the compacted segments and began reclaiming them — causing SNFE when AEM tried to read those deleted segments.

  The fix (apache/jackrabbit-oak#2779, merged March 2026) stores the CompactionResult after compaction and passes it correctly to cleanup(), ensuring gc.log is
  always written.

  Fix — AEM 6.5 LTS SP2

  1. Install the hotfix from Software Distribution:
     https://experience.adobe.com/#/downloads/content/software-distribution/en/aem.html?package=/content/software-distribution/en/details.html/content/dam/aem/public/adobe/packages/cq660/hotfixes/cq-6.5.lts.2-hotfi
  x-GRANITE-65388-1.0.zip

  2. For offline compaction, use the matching oak-run jar from Software Distribution:
     https://experience.adobe.com/#/downloads/content/software-distribution/en/aem.html?package=/content/software-distribution/en/details.html/content/dam/aem/public/adobe/packages/cq660/hotfixes/oak-run-1.88.1-B00
  6.jar

  3. Add the following system property to your AEM JVM start arguments to avoid the related disk usage regression (OAK-12134):
     -Doak.compaction.legacy=true

  ---

 Fix — AEM 6.5 LTS SP1

  1. Install the hotfix from Software Distribution:
     https://experience.adobe.com/#/downloads/content/software-distribution/en/aem.html?package=/content/software-distribution/en/details.html/content/dam/aem/public/adobe/packages/cq660/hotfixes/cq-6.5.lts.1-hotfi
  x-GRANITE-65385-1.0.zip

  2. For offline compaction, use the matching oak-run jar from Software Distribution:
     https://experience.adobe.com/#/downloads/content/software-distribution/en/aem.html?package=/content/software-distribution/en/details.html/content/dam/aem/public/adobe/packages/cq660/hotfixes/oak-run-1.78.1-B00
  6.jar

  ---

  Important: Always use an oak-run jar that matches the exact Oak version bundled in your AEM installation. Using a mismatched or older oak-run jar will leave gc.log unwritten and cause SNFE after the next two
  online GC cycles.

  References
  - OAK-12119: https://issues.apache.org/jira/browse/OAK-12119
  - Fix PR: https://github.com/apache/jackrabbit-oak/pull/2779
  - Related disk usage regression: https://issues.apache.org/jira/browse/OAK-12134