Workfront API issue - scalability concerns with task API and project fields

We are facing scalability challenges with the current API design.

Our use case requires that when a request is made to retrieve content metadata, the API should return metadata only for unrestricted content. In addition, project-level and related task-level fields need to be available as part of the response.

However, we have identified several limitations with the current approach:

• The task API is limited to approximately 2,000 records per request.
• Retrieving task data requires making separate calls per project ID, resulting in a very large number of API requests for full-load scenarios.
• The volume of task data returned per project is limited, which impacts overall efficiency.
• Project fields are restricted in the current design, making it difficult to retrieve comprehensive project-level details alongside task data.
• While the current solution may be sufficient for incremental loads, it does not scale effectively for larger datasets or bulk extraction requirements.

We are currently evaluating alternative approaches and would appreciate any insights from others who may have faced similar challenges.

Has anyone implemented a scalable solution to efficiently retrieve both project and task data at scale? Specifically, we are interested in approaches that reduce the number of API calls, support bulk/full-load scenarios, and provide access to both project and task-level fields. Any recommendations, best practices, or lessons learned would be appreciated.