Content migration using Crx2oak ( AEM 6.3 to 6.3)

[#1]

Hi There ,

Using the setps mentioned in this article , http://www.sravan.co/blog/aem-convert-blob-datastore-to-file-datastore  I was able to migrate my repository from AEM 6.1 (all in segment store) to AEM 6.3 (segment store + binary data to separated file store) .

Now I'm trying out the migration  from AEM 6.3 (segment store+data store) to AEM 6.3 (segment Store + data store)

I'm executing it as below  , however migration seems to fail with exception

15:51:48.226 ERROR [main] MigrationSpecGenerator.java:57    I/O error during processing migration specification: org.apache.jackrabbit.oak.plugins.segment.file.InvalidFileStoreVersionException

java -jar crx2oak-1.4.6-standalone.jar --copy-binaries --src-datastore=crx-quickstart/old63repository/datastore --datastore=/crx-quickstart/new63repository/datastore crx-quickstart/old63repository crx-quickstart/new63repository

Am i missing any parameter when running the jar ?

[#2 · Accepted solution]

Here's what solved the problem. This is not documented in adobe docs.

-Take a fresh backup on running 6.1 environment and perform the pre-upgrade maintenance tasks.

- Move the segmentstore from /repos folder and move it under /crx-quickstart/repository

- Navigate to /crx-quickstart/install folder and edit SegmentNodeStoreService config and remove the 'repository.home' parameter

- Copy 6.3 Jar along side /crx-quickstart

- Run the command to unpack, java -jar 6.3_quickstart.jar -unpack

- Backup and Remove …/crx-quickstart/logs

- Complete the migration by running command java -Xmx4096m -XX:MaxPermSize=2048m -jar aem-quickstart.jar -v -x crx2oak -xargs -- --load-profile segment-fds

- Monitor the logs

- Once completed, Login into the new 6.3 environment and do a quick round of smoke testing and sanity tests

- Shutdown the instance or now.

- Once ', Check the /crx-quickstart/install folder for the newly generated configs

- Move back the segmentstore and the blobids folder from the /crx-quickstart/repository folder to the /repos folder

- Edit 'org.apache.jackrabbit.oak.segment.SegmentNodeStoreService.cfg' and add repository.home=/Users/abc/Desktop/WorkArea/Instances/CIBC_repos (without the quotes) and Save. (Adjust the correct path per your systems)

- Start the instance and Voila the upgrade has been completed successfully.

Thanks to Kunwar from adobe day care team to help us on this.

[#3]

Upgrading to AEM 6.3 : Important lessons | LinkedIn

Thanks.

[#4]

Hello Kautuk,

I am trying to perform an inplace upgrade on AEM 6.1 with external file datastore to AEM 6.3 following our adobe docs - Performing an In-Place Upgrade. I am now stuck at content repository migration. Here's the configuration I have used to set up the external file datastore on 6.1

org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore.cfg

path=C:\\mytech\\Adobe\\repos-ln\\datastore

minRecordLength=4096

org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.cfg

customBlobStore=true

pauseCompaction=true

repository.home=C:\\mytech\\Adobe\\repos-ln

tarmk.size=256

My repo in 6.1 is available as per the defined configurations above.

Before running the content repo migration, I kept these two files under crx-quickstart/install folder as is with no changes. Followed the other steps like getting new 6.3 jar file in right location and unpacking etc.

And then, I ran the command - java -Xmx4096m -jar AEM-author-p4504.jar -v -x crx2oak -xargs -- --load-profile segment-fds --disable-mmap --src-datastore=C:\mytech\Adobe\repos-ln\datastore

I am getting following error - 

It seems like its not picking up the external datastore location. I have then tried copying the external datastore folders both "datastore" and "segmentstore" into crx-quickstart/reposiroty location. Running the same command again without -src-datastore parameter worked. Later I tested by starting the AEM instance using quickstart jar file [double clicking]. Everything seems ok.

Please confirm these steps are ok to follow in UAT like environemnts [Linux]

Any help on correcting the commands or advising on raising a daycare ticket would be much appreciated.

[#5]

Thanks kautuk.

Im using higher version of crx2oak as 6.1 to 6.3 was successful.

As part of our deployment model for every deployment , im trying to spin up a fresh instance and migrate known content paths from old repo.

The way im running now for a repository with file store is by explicitly mentioning paths  src-datastore and target datastore. Is it not the right way ? When you mentioned FS copy , did you mean simply copying the directory like via rsync to target directory or running crx2oak with some different input params ?

Thanks

[#6]

Reply for the internal expert on this is:

crx2oak 1.4 & aem 6.3 don't match up. This requires crx2oak 1.6.x (otherwise crx2oak won't be able to read the segment store format) . Also , there is no reason to copy -binaries from fds to fds. You could just have a FS copy and reuse the FS. i assume this is a workaround to rebuild a clean repo (instead of compacting) - the only valid reason I know of is when you want to create a significantly shrunk down instance with the exclusion of a lot of paths (where copying DS + subsequent ds gc is not scaling- eg for tb of blobs)

I hope this would help.

~kautuk

[#7]

I will recommend you to follow official documentation:- Upgrading to AEM 6.3​

~kautuk