How to cleanup unavailable blob id?

Avatar

Avatar

GK-AEM

Avatar

GK-AEM

GK-AEM

26-04-2018

Hi All,

Right now , i am encountering a problem that there is one dummy blob id which is trying to be migrated but throwing an exception.I think, this BLOB id is of DummyBlobStore.

Is there a way to exclude these blob store while doing the migration?

26.04.2018 18:08:05.557 ERROR  c.a.g.c.e.MigrationEngine: The internal error occurred during migration. The migration process has been aborted and it is not complete!

java.lang.RuntimeException: javax.jcr.RepositoryException: Failed to copy content

        at com.google.common.io.Closer.rethrow(Closer.java:149)

        at org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.migrate(OakUpgrade.java:81)

        at com.adobe.granite.crx2oak.engine.MigrationEngine$2.doMigration(MigrationEngine.java:67)

        at com.adobe.granite.crx2oak.engine.MigrationEngine.process(MigrationEngine.java:92)

        at com.adobe.granite.crx2oak.pipeline.Pipeline.run(Pipeline.java:103)

        at com.adobe.granite.crx2oak.CRX2Oak.run(CRX2Oak.java:66)

        at com.adobe.granite.crx2oak.CRX2Oak.main(CRX2Oak.java:51)

Caused by: javax.jcr.RepositoryException: Failed to copy content

        at org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copy(RepositorySidegrade.java:285)

        at org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.sidegrade(OakUpgrade.java:92)

        at org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.migrate(OakUpgrade.java:78)

        ... 5 common frames omitted

Caused by: java.lang.IllegalStateException: Attempt to read external blob with blobId [66f1d39a64998bf2e641765057509c086f2474202d9fd24fa654dabc01aba988#95174] without specifying BlobStore

        at org.apache.jackrabbit.oak.segment.SegmentBlob.getBlob(SegmentBlob.java:243)

        at org.apache.jackrabbit.oak.segment.SegmentBlob.getLength(SegmentBlob.java:252)

        at org.apache.jackrabbit.oak.segment.SegmentBlob.length(SegmentBlob.java:110)

        at org.apache.jackrabbit.oak.plugins.memory.AbstractBlob.equal(AbstractBlob.java:52)

        at org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.equals(SegmentBlob.java:227)

        at com.google.common.base.Objects.equal(Objects.java:60)

        at org.apache.jackrabbit.oak.plugins.memory.AbstractPropertyState.equal(AbstractPropertyState.java:59)

        at org.apache.jackrabbit.oak.plugins.segment.SegmentPropertyState.equals(SegmentPropertyState.java:242)

        at org.apache.jackrabbit.oak.plugins.migration.NodeStateCopier.copyProperties(NodeStateCopier.java:144)

        at org.apache.jackrabbit.oak.plugins.migration.NodeStateCopier.copyNodeState(NodeStateCopier.java:212)

        at org.apache.jackrabbit.oak.plugins.migration.NodeStateCopier.copyNodeState(NodeStateCopier.java:208)

        at org.apache.jackrabbit.oak.plugins.migration.NodeStateCopier.copyNodeState(NodeStateCopier.java:208)

        at org.apache.jackrabbit.oak.plugins.migration.NodeStateCopier.copyNodeState(NodeStateCopier.java:208)

        at org.apache.jackrabbit.oak.plugins.migration.NodeStateCopier.copyNodeState(NodeStateCopier.java:208)

        at org.apache.jackrabbit.oak.plugins.migration.NodeStateCopier.copyNodeState(NodeStateCopier.java:208)

        at org.apache.jackrabbit.oak.plugins.migration.NodeStateCopier.copyNodeState(NodeStateCopier.java:160)

        at org.apache.jackrabbit.oak.plugins.migration.NodeStateCopier.access$200(NodeStateCopier.java:75)

        at org.apache.jackrabbit.oak.plugins.migration.NodeStateCopier$Builder.copy(NodeStateCopier.java:465)

        at org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copyWorkspace(RepositorySidegrade.java:468)

        at org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.migrateWithoutCheckpoints(RepositorySidegrade.java:417)

        at org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copyState(RepositorySidegrade.java:317)

        at org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copy(RepositorySidegrade.java:278)

        ... 7 common frames omitted

Thanks,

Kishore.

Accepted Solutions (1)

Accepted Solutions (1)

Avatar

Avatar

GK-AEM

Avatar

GK-AEM

GK-AEM

02-05-2018

I am able to successfully migrate the content by running below command.

java -jar crx2oak-1.8.4-all-in-one.jar segment-old:C:/Users/xxxx/Documents/xxxx/Softwares/AEM/Chris/AEM_Author_60/crx-quickstart/repository C:/Users/xxxxxx/Documents/xxxx/Softwares/AEM/6.4/crx-quickstart/repository --datastore C:/Users/xxxx/Documents/xxxx/Softwares/AEM/6.4/crx-quickstart/repository/datastore --include-path=/content --exclude-path=/tmp --copy-versions=false --copy-orphaned-versions=false > migration_new.txt

Thank you all for your support.

Thanks,

Kishore

Answers (17)

Answers (17)

Avatar

Avatar

Jörg_Hoh

Employee

Total Posts

3.0K

Likes

910

Correct Answer

1.0K

Avatar

Jörg_Hoh

Employee

Total Posts

3.0K

Likes

910

Correct Answer

1.0K
Jörg_Hoh
Employee

26-04-2018

Ok, then you have a plain 6.2 installation, which did not have a datastore or blobstore by default (has change with 6.3)... So now you have the chance to reduce the size of your segmentstore and create a fileblobstore 🙂

If your instance isn't that large and it's just testing, can you start with this set of parameters

java -jar crx2oak-1.8.4-all-in-one.jar segment-old:C:/Users/xxx/Documents/xxx/AEM-Others/OOTB/crx-quickstart/repository C:/Users/xxx/Documents/xxx/Softwares/AEM/6.4/crx-quickstart/repository

just to eliminate the chance that one of them is causing this issue?

(When you want to have AEM 6.4 with a segmentstore you can add the parameter --fileblobstore C:/Users/xxx/Documents/xxx/Softwares/AEM/6.4/crx-quickstart/repository/blobstore

Jörg

Avatar

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K
smacdonald2008

02-05-2018

I am glad that you got it working!

Avatar

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K
smacdonald2008

26-04-2018

Have you looked here -- Data Store Garbage Collection

Does this report inform you of missing data?

Avatar

Avatar

GK-AEM

Avatar

GK-AEM

GK-AEM

26-04-2018

i have already tried that approach but getting error.

Avatar

Avatar

GK-AEM

Avatar

GK-AEM

GK-AEM

26-04-2018

I have tried with both commands but ended up with the same error

Avatar

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K
smacdonald2008

26-04-2018

Support stated: "I believe that they have an issue on a source AEM. Datastore GC should help to fix"

Avatar

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K
smacdonald2008

26-04-2018

Did you run the Datastore GC on the source AEM ? Was it completed with no issues?

Avatar

Avatar

GK-AEM

Avatar

GK-AEM

GK-AEM

26-04-2018

i see below message in the logs.

I doubt it ,implicit blobstore comes up with the when installing AEM.

" 26.04.2018 18:18:29.818 INFO   o.a.j.o.u.c.p.DatastoreArguments: Destination blob store: DummyBlobStore ".

Right now,i have only two folders inside /crx-quickstart/repository

1474112_pastedImage_4.png

Avatar

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K
smacdonald2008

26-04-2018

These parameters are discussed in this doc -- Using the CRX2Oak Migration Tool

Avatar

Avatar

GK-AEM

Avatar

GK-AEM

GK-AEM

26-04-2018

i don't see any blobstore folder in my source repository(6.2).Not sure, how to use parameters "src-fileblobstore" and "fileblobstore" in this case?

Is there any chance to have implicit blobstore, comes up with the AEM installation?

Avatar

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K
smacdonald2008

26-04-2018

THat provides a bit of background. Can you follow Joerg's suggestion here.

Avatar

Avatar

GK-AEM

Avatar

GK-AEM

GK-AEM

26-04-2018

Yes.Right now,i am following that doc and that's my reference for this migration.

Avatar

Avatar

Jörg_Hoh

Employee

Total Posts

3.0K

Likes

910

Correct Answer

1.0K

Avatar

Jörg_Hoh

Employee

Total Posts

3.0K

Likes

910

Correct Answer

1.0K
Jörg_Hoh
Employee

26-04-2018

Hi Kishore,

For me it looks like you used an blobstore on your AEM 6.2 instance; can you experiment with the parameters "src-fileblobstore" and "fileblobstore"?

Avatar

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K
smacdonald2008

26-04-2018

Did you go through all the information in the docs i pointed you to?

Avatar

Avatar

GK-AEM

Avatar

GK-AEM

GK-AEM

26-04-2018

Thank you.

Right now,i  am migrating 6.2 repository to 6.4.Below is the command i am using for migrating.

java -jar crx2oak-1.8.4-all-in-one.jar segment-old:C:/Users/xxx/Documents/xxx/AEM-Others/OOTB/crx-quickstart/repository C:/Users/xxx/Documents/xxx/Softwares/AEM/6.4/crx-quickstart/repository --include-path=/content --exclude-path=/tmp --copy-versions=false --copy-orphaned-versions=false --merge-paths=/content

Avatar

Avatar

Jörg_Hoh

Employee

Total Posts

3.0K

Likes

910

Correct Answer

1.0K

Avatar

Jörg_Hoh

Employee

Total Posts

3.0K

Likes

910

Correct Answer

1.0K
Jörg_Hoh
Employee

26-04-2018

Hi Kishore,

Looks like you are doing a repository migration. What is the source version and what is the target version? What command line are you using for this call?

regards,

Jörg

Avatar

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K

Avatar

smacdonald2008

Total Posts

12.7K

Likes

1.4K

Correct Answer

2.3K
smacdonald2008

26-04-2018

I have asked others to look here as well.

In mean time - have you gone through all the AEM migrations Docs?

Upgrading to AEM 6.4