REST Folder API Duplicates while Paging | Community
Skip to main content
X
xceled
Level 2
November 27, 2024
Solved

REST Folder API Duplicates while Paging

  • November 27, 2024
  • 2 replies
  • 2316 views

Hello,

 

While paging through /rest/asset/v1/folders.json using the offset parameter, we are seeing duplicate folder ids in later pages that were returned in earlier pages. 

E.g. With a maxReturn=200 and offset=0, we get folder id 130 in the results. Then with the same maxReturn=200 and offset=200, we again get folder 130 in the results. This is breaking our existing integration we built and not sure why this API is now providing records shown in earlier pages in later pages. Any idea what might be behind this? Has Marketo broken paging in their API to now not page properly?

    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
    Best answer by Jo_Pitts1

    @xceled ,

    in addition to what @sanfordwhiteman has already said, don't forget to check the folderType value.

    It is possible for a folder of type 'Program' to have the same id as a folder of type 'Folder'.

    Cheers

    Jo

    2 replies

    SanfordWhiteman
    SanfordWhiteman
    Level 10
    November 28, 2024

    Seeing the same folder ID on a different page is always something you should build for, for the simple reason that the entire pageset is not fetched in advance. So if folder 123 and folder 456 appear on page 1 and then someone deletes both of those folders, by definition everything is going to be shifted back 2 positions, so 2 other folders will appear on 2 different pages.

     

    The fact that you fetch page 1, page 2, and page 3 over just a couple of seconds apart makes this less likely but by no means impossible, and your integration must take this case into account.

     

    Now as for a deeper, recent regression in the API, that’s possible. I’d have to run a bunch more tests to try to reproduce it. But since your integration already needs to be ready for this possibility, that would be the place to fix it.

      X
      xceledAuthor
      Level 2
      December 2, 2024

      That is fair to a degree (mainly for delete/adds during a paging operation) however paging requires consistent sorting or else there is no guarantee that I will retrieve all items from all pages if the order changes every time I call the API. While I agree that handling duplicates is something I can handle on my side (for the deletes/adds scenario affecting page content) that is not what is happening here and makes me doubt the consistency of the sorting of the results being returned by the API and if so, then do I really have any guarantee that I'm getting all records if records can just change pages randomly and I don't know if a record will appear ever in the paging process due to changing position?

        SanfordWhiteman
        SanfordWhiteman
        Level 10
        December 2, 2024

        Did you check the Audit Trail to see if indeed there were adds/deletes during processing? Very unlikely that any random sorting has been introduced.

          Jo_Pitts1
          Community Advisor
          Jo_Pitts1Community AdvisorAccepted solution
          Community Advisor
          November 28, 2024

          @xceled ,

          in addition to what @sanfordwhiteman has already said, don't forget to check the folderType value.

          It is possible for a folder of type 'Program' to have the same id as a folder of type 'Folder'.

          Cheers

          Jo

            X
            xceledAuthor
            Level 2
            December 2, 2024

            Thanks Jo, yes I confirm we have the identity properly defined to include id and folder type, it is not that causing the new issue we haven't seen before.

              Terms & ConditionsAccessibility statement

              Loading footer...