Open APIs

 View Only
Expand all | Collapse all

TMF GET / Query Operations

  • 1.  TMF GET / Query Operations

    TM Forum Member
    Posted Nov 07, 2023 07:27
    Edited by Vivek Mittal Nov 08, 2023 03:51

    Hello All,

    We have been working with TMF API's and have seen different flavours or implementation done across different teams on how they interpret and implement the TMF API's.

    If we consider TMF637 API's, we have a lot of entities like relatedParties (array), productCharacteristics (array).

    my element response looks like:

    [
        {
            "id": "123",
            "href": "/productInventory/v4/product/123",
            "name": "new product",
            "startDate": "2023-10-30T11:39:44.000Z",
            "terminationDate": "2023-11-30T22:59:59.000Z",
            "status": "ACTIVE",
            "@type": "ProductExtended",
            "@baseType": "Product",
            "productCharacteristic": [
                {
                    "name": "identifierOnMarketPartner",
                    "value": "marketpartneridentifier"
                },
                {
                    "name": "identifierOnFinancePartner",
                    "value": "financepartneridentifier"
                }
            ],
            "productOrderItem": [
                {
                    "orderItemId": "123",
                }
            ],
            "relatedParty": [
                {
                    "id": "123456",
                    "name": "mobile",
                    "role": "Customer",
                    "@referredType": "Individual"
                },
                {
                    "id": "Partner",
                    "role": "Partner",
                    "@referredType": "Organization"
                }
            ]
        }
    ]

    Our question was primarily on how the GET query parameters shall be used for instance, if I want to expose a GET operation on the above, what is the recommendation on how to expose the filter elements i.e. shall we adhere to the structure as per TMF637, so in this case shall this be queried as:

    /tmf-api/productInventory/v4/product?%24.relatedParty%5B%3F%28%40.name%3D%3D%22mobile%22%29%5D.id=123456

    which is messy to produce and complex? And as we add the filters using query, the query string goes very complex and is not client friendly!

    Other suggestion is if we make this simple and let the team decide and may be they can use any queryParameter say "customerIdentifier" and can use the simple logic:

    /tmf-api/productInventory/v4/product?customerIdentifier=123456

    Any views or guidance, would be helpful?

    Regards,

    Vivek



    ------------------------------
    Vivek 
    ------------------------------



  • 2.  RE: TMF GET / Query Operations

    TM Forum Member
    Posted Nov 08, 2023 06:08
    Edited by Vance Shipley Nov 08, 2023 06:10

    The query example you provided, after percent decoding, was: 

    /tmf-api/productInventory/v4/product?$.relatedParty[?(@.name==\"mobile\")].id=123456

    This appears to be a mix of JSON Path and simple query syntax, but isn't valid either way.  The correct query, using TMF630 Part 6 JSONPath, would be:

    /tmf-api/productInventory/v4/product?filter=relatedParty[?(@.name=='mobile'&&@id='123456')]

    ... and percent encoded as:

    /tmf-api/productInventory/v4/product?filter=relatedParty%5B%3F%28%40.name%3D%3D%27mobile%27%26%26%40id%3D%27123456%27%29%5D

    You describe this as "messy to produce" and "complex and is not client friendly".  This is not an uncommon sentiment, however this is a specification for a machine-to-machine (MMI) interface, not meant for humans to type in at run time.  The simpler query syntax may be used when it is sufficient to accomplish your goal (percent encoded):

    /tmf-api/productInventory/v4/product?name=new%20product

    However your example is a case where that option isn't available, and a more expressive syntax is required.  Here we may avail ourselves of the JSONPath syntax, as described in TMF630 Part 6.  This option is necessarily more complex however it provides great flexibility to craft a query which accomplishes arbitrarily complex queries.

    Your alternative suggestion example would be equivalent to a simple query on a top level attribute:
    /tmf-api/productInventory/v4/product?customerIdentifier=123456

     You could support this through polymorphism by adding that attribute to your MyProduct and including @type=MyProduct. In any event you have created a unicorn for you and your friends.



    ------------------------------
    Vance Shipley
    SigScale
    ------------------------------



  • 3.  RE: TMF GET / Query Operations

    TM Forum Member
    Posted Nov 08, 2023 06:22
    Edited by Vivek Mittal Nov 08, 2023 06:22

    Thanks @Vance Shipley, for your reply!

    In case the TMF layer is being created is more like a wrapper for an underneath group of services, which use a different API interfaces. In that case should

    • the use of filters should be made more easy and should be left to the team implementing those?
    •  Or we shall just extend the existing API's and make it very custom to our implementation?

    Thanks again for the inputs!



    ------------------------------
    Vivek
    ------------------------------



  • 4.  RE: TMF GET / Query Operations

    TM Forum Member
    Posted Nov 08, 2023 23:46

    Hi Vivek

    I would not recommend extending the APIs unless you are adding features that are not currently supported by the specific API.

    Filtering the relatedParty/characteristics and other arrays is a bit fiddly, I would recommend supporting the standard (more complex) filtering methods but also implementing an additional more basic filter directive.

    Good luck!



    ------------------------------
    Dan d'Albuquerque
    Individual
    ------------------------------



  • 5.  RE: TMF GET / Query Operations

    TM Forum Member
    Posted Nov 09, 2023 00:14

    I recommend against it however, If you are going to go your own way, I would suggest that you use your own query parameter name (i.e. myFilter):

    /tmf-api/productInventory/v4/product?myFilter=[customerIdentifier=123456,status=ACTIVE]

    You must choose a query parameter name which won't conflict with any object attributes. The right hand side value may take any form you wish, subject to rules on reserved characters including /, ;, and ?. Above I suggested a list of attribute value pairs, delimited with , and surrounded by [], but you do you.



    ------------------------------
    Vance Shipley
    SigScale
    ------------------------------



  • 6.  RE: TMF GET / Query Operations

    TM Forum Member
    Posted Nov 09, 2023 01:12

    A general observation about the JSON Path filtering mentioned by Vance - where in the call chain do you apply the filtering, in terms of performance. If your underlying data store supports JSON path natively, you're on to a good thing. In Vivek's case, where the TMF Open API is being used on top of a mediation layer, getting good performance might be more challenging. You would somehow have to map arbitrary JSON Path expressions into the capabilities of your underlying API endpoints.

    There is an alternative approach, which might be worth considering. This is to POST a task resource to express your query. The query inputs and outputs are strongly-typed properties in the resource.

    The advantages are:

    • Contract is "strong"
    • Query is easier to use by the consumer
    • You can map the query very closely to the underlying API capabilities.

    Disadvantages:

    • Less flexible
    • No inherent pagination support in the contract, you'll have to roll your own model to deal with multiple pages
    • No potential benefits from HTTP caching, as against GET

    Good luck



    ------------------------------
    Jonathan Goldberg
    Amdocs Management Limited
    Any opinions and statements made by me on this forum are purely personal, and do not necessarily reflect the position of the TM Forum or my employer.
    ------------------------------



  • 7.  RE: TMF GET / Query Operations

    TM Forum Member
    Posted Nov 09, 2023 01:36

    Yes, the use of JSONPath here is to communicate the query over the wire, no assumption should be made about any other part of the implementation.  It's unlikely that you have a persistence layer for your REST Collection which accepts JSONPath.  The onus is on the API interface implementor to parse a received query and use it to (efficiently) retrieve the requested Collection items.

    I see no point in creating your own Task resource for queries just because you can't handle the full expressiveness of JSONPath. You are much better off mapping the queries you do support onto JSONPath syntax and supporting (just) those queries.  No one said you had to support arbitrarily complex queries, however your supported queries should use the API specification.



    ------------------------------
    Vance Shipley
    SigScale
    ------------------------------



  • 8.  RE: TMF GET / Query Operations

    TM Forum Member
    Posted Nov 09, 2023 05:28

    Great inputs everyone!

    Thanks for this!

    We also internally are aligned and see that using Json query path has its benefits and provides more flexibility, hence we have that as our default at the moment!

    Downsides

    • complexity it brings!
    • url length
    • if the query parameters and values are special characters which are not allowed as part of uri, it becomes a limitation!

    Although its machine to machine communication, but the teams that are our consumers are not very happy and convinced of using these json path based query params!

    There was yet another suggestion, where as part of GET resources, would there be a recommendation to use a payload which would adhere to a key, value pair schema. We understand that this is not a recommendation, but would like to get your views on this as well!

    ## Request payload to accommodate Get query parameters
      Product_Request:
        type: array
        description: Describes a list of query parameters through a name/value pair.
        items:
          type: object
          properties:
            name:
              type: string
              description: Name of the characteristic
            valueType:
              type: string
              description: Data type of the value of the characteristic
            value:
              $ref: '#/definitions/Any'
              description: The value of the characteristic

    Thanks again for your valuable insights!



    ------------------------------
    Vivek
    ------------------------------



  • 9.  RE: TMF GET / Query Operations

    TM Forum Member
    Posted Nov 09, 2023 06:59

    @Vivek Mittal writes:

    >  Downsides
    >    - complexity it brings!

    This is a feature.

    >    - url length

    This is mostly irrelevant.  There is not limit to the length of a URL in any specification.  You may encounter an implementation limit somewhere (e.g. Apache) but you can also configure it to handle the length required (default in Apache is 8177 characters).

    >    - if the query parameters and values are special characters which are not allowed as part of uri, it becomes a limitation!

    Nope.  A URL is parsed as per RFC1738 including percent encoding of reserved characters. You have to be careful that the rules are followed in the correct order however there is no limitation.

    I find none of your alternatives compelling nor needed.  Folks just need better training. 

    My rule is: Make it as simple as possible, but no simpler.  



    ------------------------------
    Vance Shipley
    SigScale
    ------------------------------



  • 10.  RE: TMF GET / Query Operations

    Posted Feb 05, 2024 04:07

    Hi, I understood that JsonPath expression gives great flexibility in selecting our filters. 

    But I've a universal set (let's say some  4 million records at db level) and need to filter some 100 records out of it, is it going to work without any performance issues  ? 



    ------------------------------
    Nagarjuna Muddineni
    TO BE VERIFIED
    ------------------------------



  • 11.  RE: TMF GET / Query Operations

    TM Forum Member
    Posted Feb 05, 2024 04:33

    Performance is something entirely up to the implementor.  A filter, whether JSON Path expression or simple query parameters, is merely a way to express an intention.  The implementor of an Open API producer will have to accept the intent and perform some actions to retrieve the collection items which match the query filter. The CPU time to parse a filter will be negligible compared to the item retrieval.

    Now, many get confused about how to use a JSON Path expression, expecting that they will use a library to execute the filter, however they immediately realize the tool requires a JSON array of the entire collection (i.e. catalog, inventory).  Now that certainly won't perform well at all!  Some data persistence technology will accept JSON Path (i.e. Redis) in which case you'll have an easier time.

    The right way to look at it is this:

    1. you need to use a TMF Open API to query for matching items
    2. a JSON Path expression provides the expressiveness required for your advanced query
    3. you need a server implementation which knows how to perform your query

    You don't have to support every possible filter expression, just the ones you need.  Don't think about the JSON Path expression as doing the work, just expressing the intent.



    ------------------------------
    Vance Shipley
    SigScale
    ------------------------------