Content migration using Crx2oak ( AEM 6.3 to 6.3)

Avatar

Avatar

Krisgum

Avatar

Krisgum

Krisgum

16-08-2017

Hi There ,

Using the setps mentioned in this article , http://www.sravan.co/blog/aem-convert-blob-datastore-to-file-datastore  I was able to migrate my repository from AEM 6.1 (all in segment store) to AEM 6.3 (segment store + binary data to separated file store) .

Now I'm trying out the migration  from AEM 6.3 (segment store+data store) to AEM 6.3 (segment Store + data store)

I'm executing it as below  , however migration seems to fail with exception

15:51:48.226 ERROR [main] MigrationSpecGenerator.java:57    I/O error during processing migration specification: org.apache.jackrabbit.oak.plugins.segment.file.InvalidFileStoreVersionException

java -jar crx2oak-1.4.6-standalone.jar --copy-binaries --src-datastore=crx-quickstart/old63repository/datastore --datastore=/crx-quickstart/new63repository/datastore crx-quickstart/old63repository crx-quickstart/new63repository

Am i missing any parameter when running the jar ?

Accepted Solutions (1)

Accepted Solutions (1)

Avatar

Avatar

sureshr19072451

Avatar

sureshr19072451

sureshr19072451

28-01-2018

Here's what solved the problem. This is not documented in adobe docs.

-Take a fresh backup on running 6.1 environment and perform the pre-upgrade maintenance tasks.

- Move the segmentstore from /repos folder and move it under /crx-quickstart/repository

- Navigate to /crx-quickstart/install folder and edit SegmentNodeStoreService config and remove the 'repository.home' parameter

- Copy 6.3 Jar along side /crx-quickstart

- Run the command to unpack, java -jar 6.3_quickstart.jar -unpack

- Backup and Remove …/crx-quickstart/logs

- Complete the migration by running command java -Xmx4096m -XX:MaxPermSize=2048m -jar aem-quickstart.jar -v -x crx2oak -xargs -- --load-profile segment-fds

- Monitor the logs

- Once completed, Login into the new 6.3 environment and do a quick round of smoke testing and sanity tests

- Shutdown the instance or now.

- Once ', Check the /crx-quickstart/install folder for the newly generated configs

- Move back the segmentstore and the blobids folder from the /crx-quickstart/repository folder to the /repos folder

- Edit 'org.apache.jackrabbit.oak.segment.SegmentNodeStoreService.cfg' and add repository.home=/Users/abc/Desktop/WorkArea/Instances/CIBC_repos (without the quotes) and Save. (Adjust the correct path per your systems)

- Start the instance and Voila the upgrade has been completed successfully.

Thanks to Kunwar from adobe day care team to help us on this.

Answers (5)

Answers (5)

Avatar

Avatar

sureshr19072451

Avatar

sureshr19072451

sureshr19072451

30-12-2017

Hello Kautuk,

I am trying to perform an inplace upgrade on AEM 6.1 with external file datastore to AEM 6.3 following our adobe docs - Performing an In-Place Upgrade. I am now stuck at content repository migration. Here's the configuration I have used to set up the external file datastore on 6.1

org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore.cfg

path=C:\\mytech\\Adobe\\repos-ln\\datastore

minRecordLength=4096

org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.cfg

customBlobStore=true

pauseCompaction=true

repository.home=C:\\mytech\\Adobe\\repos-ln

tarmk.size=256

My repo in 6.1 is available as per the defined configurations above.

1383482_pastedImage_4.png

Before running the content repo migration, I kept these two files under crx-quickstart/install folder as is with no changes. Followed the other steps like getting new 6.3 jar file in right location and unpacking etc.

And then, I ran the command - java -Xmx4096m -jar AEM-author-p4504.jar -v -x crx2oak -xargs -- --load-profile segment-fds --disable-mmap --src-datastore=C:\mytech\Adobe\repos-ln\datastore

I am getting following error - 

1384303_pastedImage_0.png

It seems like its not picking up the external datastore location. I have then tried copying the external datastore folders both "datastore" and "segmentstore" into crx-quickstart/reposiroty location. Running the same command again without -src-datastore parameter worked. Later I tested by starting the AEM instance using quickstart jar file [double clicking]. Everything seems ok.

Please confirm these steps are ok to follow in UAT like environemnts [Linux]

Any help on correcting the commands or advising on raising a daycare ticket would be much appreciated.

Avatar

Avatar

Krisgum

Avatar

Krisgum

Krisgum

17-08-2017

Thanks kautuk.

Im using higher version of crx2oak as 6.1 to 6.3 was successful.

As part of our deployment model for every deployment , im trying to spin up a fresh instance and migrate known content paths from old repo.

The way im running now for a repository with file store is by explicitly mentioning paths  src-datastore and target datastore. Is it not the right way ? When you mentioned FS copy , did you mean simply copying the directory like via rsync to target directory or running crx2oak with some different input params ?

Thanks

Avatar

Avatar

kautuk_sahni

Community Manager

Total Posts

5.8K

Likes

1.0K

Correct Reply

1.1K

Avatar

kautuk_sahni

Community Manager

Total Posts

5.8K

Likes

1.0K

Correct Reply

1.1K
kautuk_sahni
Community Manager

17-08-2017

Reply for the internal expert on this is:

crx2oak 1.4 & aem 6.3 don't match up. This requires crx2oak 1.6.x (otherwise crx2oak won't be able to read the segment store format) . Also , there is no reason to copy -binaries from fds to fds. You could just have a FS copy and reuse the FS. i assume this is a workaround to rebuild a clean repo (instead of compacting) - the only valid reason I know of is when you want to create a significantly shrunk down instance with the exclusion of a lot of paths (where copying DS + subsequent ds gc is not scaling- eg for tb of blobs)

I hope this would help.

~kautuk

Avatar

Avatar

kautuk_sahni

Community Manager

Total Posts

5.8K

Likes

1.0K

Correct Reply

1.1K

Avatar

kautuk_sahni

Community Manager

Total Posts

5.8K

Likes

1.0K

Correct Reply

1.1K
kautuk_sahni
Community Manager

17-08-2017

I will recommend you to follow official documentation:- Upgrading to AEM 6.3

~kautuk