Transfering large strings between QPACs

[Former Community Member]

Hello



I think I'm having a simple but serious problem. I want to transfer a huge string (up to 3 MB) between a custom QPAC and the workflow context (that an other QPAC can use the string variable).



The length of the string variable is limited to 65535, right? How can I transfer such large data strings? Any ideas?

It would be great to see any feedback...



Thank you!

Nico

[HowardTreisman]

Hi Nico

The length of a string variable is determined by the underlying data type that is used to store strings in the database. The limits will depend on the particular database type you're using. For example, I believe the standard Oracle varchar2 is limited to just 2K.



I have two suggestions for you:

1. Use a binary varible. This is generally implemented in that database as a Long Raw value, and should be effectively unlimited in size. The downside is that many of the existing QPACs may not be able to deal very well with binary variables.

2. Use a Document variable. An Adobe document variable is a little like a String (or really a binary) on steroids. It basically works like a binary, but has a threshold value (default of 64K), over which it will store the file in the file system rather than in the database. This has some significant advantages, including limiting the growth of your database. It also has some clever caching mechanisms, support for clustering, etc. A document variable is a little more difficult to deal with in your custom QPAC, but not much -the com.adobe.idp.Document type should be fairly self-explanatory. Documents are good, because they tend to be the default type that most of the Adobe QPACs use when dealing with large amounts of data.



Please also be aware that some databases (eg MySQL) have other limitations, such as the total number of binary/longvarchar columns that you can define in any one table. So you need to do some experimentation in that area as well.



I hope this helps.



Let us know you you get on.



Howard

http://www.avoka.com

[Former Community Member]

Thanks Howard.<br /><br />I've stored my data in a document variable. But how do I work with this process variable? I'm trying to prepare a XML Message with the Queue Sender QPAC as follows:<br /><br />><...><br />><ATT><br />> <BINARY>{$/process_data/@datadoc$}</BINARY><br />></ATT><br />><...><br /><br />@datadoc is the document variable containing the binary data (about 1 MB). But the generated XML message does only contain the byte array address (8 bytes).<br /><br />How can I embed the binary data in this XML message text? Is there a useful XPath String function to wrap binary data (XPathExpressionBuilder)?<br /><br />Nico

[HowardTreisman]

Hi Nico

To the best of my knowledge, there is no way to embed a document variable into a string.

This is at least partially because the document variable does not necessarily contain a string, it may contain all sorts of binary data.

I think you'd be looking at (another) custom QPAC for this.

Let us know if you'd like us to build it for you :)



Howard

[Former Community Member]

Hi Howard



Yes, you are right. There is no way to embed a document variable into a string. But there is a way to embed a XML variable into a string. And the XML variable does also not have any size limitation.



Of course this is only a "workaround". I use the XML variable as an output variable in my custom QPAC. The only condition is that the transferring data is packed into a XML syntax block. Afterwords the data can be unpacked with the XPath string functions... :-)



This works fine for me. Now I'm able to transfer very large string data between QPACS...



Nico

[HowardTreisman]

Very creative and elegant workaround, Nico.

Thank you for sharing this with all of us.

Howard