Journal API response size

[jgrubbs]

Hi all,

Is there a way to retrieve all entries in a Journal, or at least force it to deliver a more than whatever it chooses to give me?

I know I can limit the size, but I want to do the opposite. Also, is there a certain optimal cadence for querying the Journaling API? Once a second, once every hour?

[shikhartanwar · Accepted solution]

@jgrubbs: Thanks for reaching out and asking these questions.

---

Is there a way to retrieve all entries in a Journal, or at least force it to deliver a more than whatever it chooses to give me?

I know I can limit the size, but I want to do the opposite.

Currently this is not possible. See this and this in our FAQ section.

---

Also, is there a certain optimal cadence for querying the Journaling API? Once a second, once every hour?

The cadence should ideally depend on the rate of events that are being published against your event registration and how fast you would want to catch up. We also return Retry-After headers with 204 responses, so the best thing would be to follow that.

Hope this information helps.

[jgrubbs]

Thank you for the response Are there any plans that you know of which would allow me to query the full journal or at least force a larger response size?

[shikhartanwar]

No plans as such, but I am interested to know what is the use-case where you would want the read the full (or a large part of) journal at once?

[jgrubbs]

Just looking for a more efficient way to utilize our resources. The iterative process seems slow and it's a gamble as to how many events the API will return for each request. There's no way for me to know how close to the end I am in order to be "caught up", therefore I have to leave a container running nonstop. And if the container fails for whatever reason, that puts me even further behind. Yes, I can start where I left off with the "since" parameter, so at least I don't have to start at the end of the journal. It would just make life easier to query larger chunks at the very least, or get an idea of the current size of the journal and my current location in the journal relative to the end

[shikhartanwar]

I am afraid there is no short-term solution for the scenarios you mentioned. We do have the feature of "counting events in a journal" in our backlog, but it's not prioritized in our short-term roadmap.

[jgrubbs]

Well I do appreciate your time, so thank you again