Hi guys,
I am fairly new to using the Data Sources API and have jusr created a NodeJS importer that queries GSC data and is pushing that data into a data source.
Data import works great so far, and I have imported all 2023 data.
My client has a fairly big website with large numbers of entries per day (25k+) that requires breaking the import into multiple chunks per day (typically 3) which leaves me at 1000k import jobs so far.
To not accidentally import the same day data twice, I have created a failsafe that is first querying previous job names to see whether the planned day to import has already been processed.
This I do using the GetJobs function
https://api.omniture.com/admin/1.4/rest/?method=DataSources.GetJobs
And here is where the issues start.
This function only returns a maximum of 1000 jobs and my imports run twice because previous imports cannot be detected. Luckily, this is currently on a non-prod report suite.
Does anyone know whether
Cheers
Björn
Solved! Go to Solution.
Views
Replies
Total Likes
Hi @fhusain,
support said it's not possible, disabled my checks, filters only work on exact values i.e. no regex or pattern matching.
Don't think it's ever gonna be implemented in a 1.4 API.
Cheers
Björn
Views
Replies
Total Likes
Hey @bjoern__koth Are you still facing challenges with this issue?
- On the pagination part, if I check the body of the API it seems we don't any such parameter explicitly.
- We can check on the filter name part as well and if you haven't log a support ticket yet; can you please create one via Admin Console such that this can be looked at?
Views
Replies
Total Likes
Hi @fhusain,
support said it's not possible, disabled my checks, filters only work on exact values i.e. no regex or pattern matching.
Don't think it's ever gonna be implemented in a 1.4 API.
Cheers
Björn
Views
Replies
Total Likes