How to list ALL the data of a table which has over 1 million in the Javascript?

[felixsongfei]

I have table has over millions of records, I wanted to use the queryDef to list all the records, but the queryDef can only fetch 1000 records one time. If I use the loop syntax to use the queryDef for many times, how can I make sure it will stop after I list all the records? Is there any other functions I can use except the queryDef? Thanks!

[geoffreyl240454]

Hi felixs51477447​,

I don't know if another function exists but what I would do:

- Set a lastId parameter to 0

- Sorting your records by Id in your queryDef

- Adding a condition tag "id > lastId" in the where tag of your queryDef

- Getting the last id recovered for each queryDef execution

- Exit the loop when the result of the queryDef equal to 0

Geoffrey.

[Venu_Reddy]

Hi felixs51477447​,

When you use Javascript to process large amounts of data you must take some precautions to avoid

memory leaks. Otherwise the server may become unresponsive after a while.

Queries in Javascript (queryDef) results should be limited in number. Do not create a querydef result

of more than 10,000 (ten thousand) records. Use the “lineCount” attribute when loading a large

table. If you have more than 10,000 records to process, it's possible, but requires a special technique.

Exit the Javascript interpreter every 10,000 lines, then start again from the line where you stopped.

Use parseInt(), parseFloat() and toString() to cast the XML types to normal Javascript types before

using them.

For instance : var iRecipientId = parseInt(ctx.recipient.@id); process(iRecipientId).

Otherwise it could create errors on comparison of values, because they are not of the same type.

Regards,

Venu

[Jonathon_wodnicki]

Hi,

The for-loop will terminate when the db cursor reaches the end of the result set, i.e. when all records have been fetched.

I think Venu is speaking to the Tomcat memory limit, which defaults to 2 mb per js process in Campaign installations. Avoid buffering the full table to memory if possible.

Thanks,

-Jon