Open APIs

Hi,
I would like to ask about X-Total-Count header restrictions.
In my understanding, the query-based pagination described in TMF630 suggests that some snapshots of data should be created for each session when the iteration starts.
If I am right, then that solution seems to not live up to the expectations of modern cloud architectures. It assumes that a single database session will be held for each sequence of iterations, however nowadays service instances should be added or removed on the fly depending on the current load and they should not be obligated to run for a specified time.
As a counter-example, please take a look at the GraphQL solution https://graphql.org/learn/pagination/
It does not force the system to hold a snapshot of data and it allows the offset to be a String instead of an Integer which gives the possibility for more flexible queries (not only iterating from the start to the end of data).

Assuming that we are not creating a data snapshot after the first API call, I am wondering if it is right for the X-Total-Count header value to change with each iteration.
In our case (TMF642 Alarm integration), the data changes very quickly on our side so most probably, the X-Total-Count will differ between the first and last iteration.
It would be more intuitive to rely only on the "next" link presence but the X-Total-Count header is mandatory according to TMF630. What is your recommendation for our case?

------------------------------
Marcin Roman
Comarch S.A.
------------------------------

Hi Marcin,
Your point is perfectly valid on current architectures, which come with various flavors / approaches, that all need to be supported / fit in what TMF exposes.
This being said, keep in mind the following:

• As a general statement, TMF provides guidelines around the design of APIs as an contract between consumers and provider, and doesn't enforce anything around the underlying implementation (as an interface); this especially means under the hood / behind this interface you have the flexibility to implement as per your own back-end ecosystem, taking into consideration its specificities / capabilities / limitations
• Focusing on pagination specifically, one implementation option indeed consists in keeping some 'cache' which will can be browsed, usually guaranteeing data stability (as snapshot) and optimized access (regular cache); this cache option applies as long as the filter criteria are unchanged; other option is to reach everytime the actual data, and in such case, as you stated, this data - including the total count - may change between successive 'page' accesses (note in such case the variation of X-Total-Count returned in the response can be useful indication to consumers)
• Regarding the rationale of making X-Total-Count mandatory, I understand your scenario is perfectly right assuming consumers will access iteratively fixed amount chunk of the data, then the next one (i.e. [page 1] > [page 2] > ... > [last page]), and so on (until there is no 'next'), but this is not the only flow pagination pattern supports and consumers may actually need to 1) access directly to specific offset (i.e. scroll to position), 2) access variable amount of data chunk (increase / decrease limit). This makes X-Total-Count madatory upfront for consumers to get to know the boundaries of the paginated data.

Hope this helps clarifying.

------------------------------
Christophe MICHEL
Amdocs Management Limited
------------------------------

Original Message:
Sent: Mar 24, 2022 07:31
From: Marcin Roman
Subject: TMF630 pagination recommendations

Hi,
I would like to ask about X-Total-Count header restrictions.
In my understanding, the query-based pagination described in TMF630 suggests that some snapshots of data should be created for each session when the iteration starts.
If I am right, then that solution seems to not live up to the expectations of modern cloud architectures. It assumes that a single database session will be held for each sequence of iterations, however nowadays service instances should be added or removed on the fly depending on the current load and they should not be obligated to run for a specified time.
As a counter-example, please take a look at the GraphQL solution https://graphql.org/learn/pagination/
It does not force the system to hold a snapshot of data and it allows the offset to be a String instead of an Integer which gives the possibility for more flexible queries (not only iterating from the start to the end of data).

Assuming that we are not creating a data snapshot after the first API call, I am wondering if it is right for the X-Total-Count header value to change with each iteration.
In our case (TMF642 Alarm integration), the data changes very quickly on our side so most probably, the X-Total-Count will differ between the first and last iteration.
It would be more intuitive to rely only on the "next" link presence but the X-Total-Count header is mandatory according to TMF630. What is your recommendation for our case?

------------------------------
Marcin Roman
Comarch S.A.
------------------------------