Open APIs

 View Only
Expand all | Collapse all

Duplicate attributes when using '@'

  • 1.  Duplicate attributes when using '@'

    Posted Oct 12, 2018 05:34

    I've posted this on the GitHub for the TMF657_ServiceQualityManagement repo on GitHub, but thought it worthy of posting here, as it would impact on the API Design Guidelines.

    When generating models using the Swagger definition using the swagger-codegen-maven-plugin (v2.3.1), there are some entities where the use of the @ to prefix attribute names is causing issues where there is another attribute named the same. eg. on ServiceLevelSpecParameter @type & type.

    By default, when the model code is generated, in our case into Java, the preceding @ symbol is stripped, resulting in duplicate methods being generated in the code.

    So, the use of @ to proceed the attributes is problematic. Would it be possible to review the use of @ to prefix attribute names? Or, should we raise this as an issue with swagger.io to modify the code generator?



    ------------------------------
    Lee Walton
    KCOM Group PLC
    ------------------------------


  • 2.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 12, 2018 10:19
    Hi Lee,

    randomly I tried exactly the same with a different API today. 

    I am not sure if changing the Standard is the best choice.

    Changing the generator might need a surgery within the generator framework - So I would go first for fixing the generated files. So you could try to rename the duplicated properties to "atType" (or similar) and see if the "@JsonProperty of the Spring framework will help you to respond "@type".

    I'll try the same over the weekend and will state the result here. 
    Of course correcting generated stuff is not one of the fancy jobs.

    Best regards

    Michael 

    P.S.: YOGO! (you only generate once ;-)


  • 3.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 13, 2018 13:17
    This is indeed possibly problematic. I will raise this with the architecture team @pierre gauthier @Steve Harrop and @Mariano Belaunde - it could be that we will have to revisit our design guidance.​​​​

    ------------------------------
    Jonathan Goldberg
    Amdocs Management Limited
    ------------------------------



  • 4.  RE: Duplicate attributes when using '@'

    Posted Oct 15, 2018 03:54

    Michael, you closing tag YOGO (You Only Generate Once), is an interesting point.

    In the proof of concept I am building, because of the ease with which the Java code can be generated (excluding the @ issue), I've added a Maven project that just builds the API & model classes, and it deliberately does not commit the generated source. So, if another developer checks out the code from git, they too would end up generating the source again. So it is a no go for yogo.

    It would be a simple enough matter to change this, so that we only generate once, but I consider this to be somewhat sub-optimal. You should always be able to regenerate the code and use it without modifying it.

    If we can address this issue through updating the TM Forum API design guidelines, I do think it would be the best outcome. I really don't like seeing UML models where a special character denotes some kind of additional trait for an entity or attribute, so if we can avoid this, I think it would be the best outcome.

    I am also considering forking the swagger.io code generator, to see if we can get the Maven plugin to address this through prefix-replacement, as I am sure that the TM Forum model is not the only model to be affected by these sort of design decisions.



    ------------------------------
    Lee Walton
    KCOM Group PLC
    ------------------------------



  • 5.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 16, 2018 16:09
    Lee,

    I did some research. The TMForum APIs are Swagger 2.0 which defined for all objects (including fields) to be JSON compliant. JSON spec (JSON) states that the object/field may be a string which includes all unicode characters. 

    That is why swagger validators like goswagger.io do not complain about TMForum APIs (like we do ;-).

    If TMForum moves to Open API 3.0 they need to follow the specification for Components which helps us because they define a regular expression (^[a-zA-Z0-9\.\-_]+$) for validating field names: OAI/OpenAPI-Specification 

    This slight difference is not explicitly mentioned in blog posts like A Guide to What's New in OpenAPI 3.0 .

    What I do not really understand is, that if you use swaggerhub to migrate an TMForum Open API to 3.0 the "@" remains and no validator complains (I tried swagger ui, swagger inspector and usabillabv/openapi3-validator). So either they did not implement a check using the regular expression or I should not study specifications at the later evening.

    @Steve Harrop is quite experienced if it comes to defining interfaces - maybe he can tell us, how it works and maybe if the next generation of APIs may use a smaller set of characters for defining fields.   

    Best regards
    Michael


    ​​

    ------------------------------
    Michael Frandsen
    Vodafone Group
    ------------------------------



  • 6.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 17, 2018 02:42
    Reverting to the issue about duplicate field names, @pierre gauthier has indeed confirmed that we should not be using an un-decorated ​attribute named type in Open API resources. The attribute should be named businessType or <entity>Type or some other name that makes sense in the context of the entity. The decorated @type attribute will stay as part of our support for polymorphism as described in the design guidelines.


    ------------------------------
    Jonathan Goldberg
    Amdocs Management Limited
    ------------------------------



  • 7.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 17, 2018 05:38

    I think that the standard will be better if fit will take into consideration other standards in same subject area. Very often we need to convert JSON to XML or vice versa. As soon as symbol @ is not allowed at the beginning of attribute in XML, then better don't use it also in TMForum OpenAPI.

    With such unsolved issue companies need to adapt the standard locally, which makes standard less attractive, and forces to think about alternatives

    ------------------------------
    Yury Bury-Burymski
    Telekom Austria AG
    ------------------------------



  • 8.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 17, 2018 12:51
    Thanks for keeping us posted here!

    Here two different movement are colliding. The API movement with the key drivers simplicity and customer centricity and the data-model driven roots of the TMForum APIs.
    As everybody may read (https://projects.tmforum.org/wiki/display/AP/API+Data+Model+SID+Mapping+Rules+and+Guideline) the move towards APIs led to a set of rules to simplify (or reduce the feature richness of data modeling) the models to integrate the entities within the APIs. One rule is flattening hierarchies and an other rule states: if an entity only includes metadata (starting with @) the entity should be removed - what is not exactly our case - but showing the importance of metadata.

    ​So if there are reasons to keep these metadata for the client of the data, I do not want continuing arguing. But there are a lot of APIs out there which do not rely on metadata which states the base entity of a structure included in an API. Maybe it helps systems consuming APIs to map these response payloads into their data model. But should they not be able to the same without metadata information?

    Open API spec 3.0 somehow heard that users miss inheritance & polymorphism. So allof, oneof, anyof were introduced and even discriminators to identify the field which delivers the type (https://swagger.io/docs/specification/data-models/inheritance-and-polymorphism/).

    So I stop complaining and offer my help moving to wards Open API 3.0 as a volunteer (if there are activities) - but I am not the born API guy.

    Best regards
    Michael

    P.S.: We are lucky: Polymorphism is allowed in Germany. I checked this.

    ------------------------------
    Michael Frandsen
    Vodafone Group
    ------------------------------



  • 9.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 23, 2018 11:07
    This is interesting and seems to reinforce the point that no change to @type is required.​

    ------------------------------
    pierre gauthier
    TM Forum
    ------------------------------



  • 10.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 15, 2018 02:07
    dears,
    nice to see such genuine info!




    Kind Regards

    Mohammed Kedir Hassen

    SR Network Perforrmance operation supervisor
    Web.www.ethiotelecom.et
    E-mail mohammed.kedir@ethiotelecom.et
    phone number 0930349994
    IMPORTANT. This e-mail message and any attachments are strictly confidential and may be protected by law. This message is intended only for the named recipient(s) above. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender and delete this e-mail message. Any unauthorized view, usage or disclosure of this message is prohibited. Since e-mail messages may not be reliable, ethio telecom shall not be liable for any message if modified, changed or falsified. Additionally the recipient should ensure they are actually virus free.




  • 11.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 17, 2018 09:38

    Hi, if touching the swagger codegen is an option, take a look at the function  sanitizeName in DefaultCodegen.java (package io.swagger.codegen).

    It should be possible to do stuff like

            if (name.startsWith("@") && name.length()>1) {
                name = "With" + name.substring(1,2).toUpperCase() + name.substring(2);
            }
            if (name.contains(".@")) {
                String[] parts = name.split(".@");
                name = parts[0] + "with" + parts[1].substring(0,1).toUpperCase() + parts[1].substring(1);           
            }

     
    Br,
    Knut Johannessen
    Telenor




  • 12.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 19, 2018 04:11
    Thanks!
    It will help to perform manipulations with specifications.

    I would suggest avoiding symbol@ in element names to avoid additional effort and complexity.



    ------------------------------
    Yury Bury-Burymski
    Telekom Austria AG
    ------------------------------



  • 13.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 22, 2018 08:06
    The problem with @Only required if both the @ prefix and non @ properties are present in the same class thus our recommendation to not use type but use something more context bound.

    Moving forward we will make sure that there will be no occurrences of @type and type in our specifications. 

    Note that this problem seems to be specific Java code generator. When both @x and x attributes are present then
    mapping for the x attributes must instrumented
    So in Java…current mapping is 
    @JsonProperty("@type")
      private String type = null;
    public String getType() {
        return type;
      }

      public void setType(String type) {
        this.type = type;
      }
    No problem if there is no other attribute called "type" (same for baseType an schemaLocation).
    If there is also an attribute like type then we have the duplication.
    @JsonProperty("type")
      private String type = null;
    public String getType() {
        return type;
      }

      public void setType(String type) {
        this.type = type;
      }
    The workaround (only required when the "type" and "@type" attribute are present ) is
    @JsonProperty("@type")
      private String atType = null;
    public String getAtType() {
        return;
      }

      public void setAtType(String type) {
        this.atType = type;
      }
    Hope this clarify.
    ​​​​

    ------------------------------
    pierre gauthier
    TM Forum
    ------------------------------



  • 14.  RE: Duplicate attributes when using '@'

    Posted Oct 23, 2018 03:03
    The problem isn't only with java. If we transform json to xml we'll obtain an error for @field because symbol @ isn't valid for a xml node name.

    ------------------------------
    Igor Murzich
    Telekom Austria AG
    ------------------------------



  • 15.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 23, 2018 09:10
    We only support json.

    ------------------------------
    pierre gauthier
    TM Forum
    ------------------------------



  • 16.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 23, 2018 09:16
    To be more clear.

    The TMF Open APIs support only JSON.

    I'm not sure if you want to extend them to support other representation formats 

    I assume that in your mapping code/script you can map your XML node to type instead of @type.

    I see that as a backend  mapping issue​.


    ------------------------------
    pierre gauthier
    TM Forum
    ------------------------------



  • 17.  RE: Duplicate attributes when using '@'

    Posted Oct 25, 2018 06:52

    @pierre gauthier, just picking up on your comment "TM Forum only supports JSON". That is an interesting point.

    The TM Forum has stipulating that for all APIs, a Swagger definition should be produced. Swagger.io produce a code generator specifically to generate code in a variety of languages. But, in those instances where an attribute name clash exists due to the use of an @ prefix, the generated code fails to compile and/or execute for numerous languages.

    I am happy to concede that this is essentially an issue with the Swagger code generator, and that we are defining a perfectly valid Swagger.

    However, I have to question whether a design which includes two attributes, one called 'type' and the other called '@type', is sound.

    At the very least, the use of 'type' as an attribute name should be avoided through prefixing with a disambiguator, eg. exampleType.

    However, it would be better IMHO to avoid the use of the @ prefix altogether, as it is conveying a semantic in a rather opaque fashion.



    ------------------------------
    Lee Walton
    KCOM Group PLC
    ------------------------------



  • 18.  RE: Duplicate attributes when using '@'

    TM Forum Member
    Posted Oct 22, 2018 08:07
    The problem with @Only required if both the @ prefix and non @ properties are present in the same class thus our recommendation to not use type but use something more context bound.

    Moving forward we will make sure that there will be no occurrences of @type and type in our specifications. 

    Note that this problem seems to be specific Java code generator. When both @x and x attributes are present then
    mapping for the x attributes must instrumented
    So in Java…current mapping is 
    @JsonProperty("@type")
      private String type = null;
    public String getType() {
        return type;
      }

      public void setType(String type) {
        this.type = type;
      }
    No problem if there is no other attribute called "type" (same for baseType an schemaLocation).
    If there is also an attribute like type then we have the duplication.
    @JsonProperty("type")
      private String type = null;
    public String getType() {
        return type;
      }

      public void setType(String type) {
        this.type = type;
      }
    The workaround (only required when the "type" and "@type" attribute are present ) is
    @JsonProperty("@type")
      private String atType = null;
    public String getAtType() {
        return;
      }

      public void setAtType(String type) {
        this.atType = type;
      }
    Hope this clarify.
    ​​​​

    ------------------------------
    pierre gauthier
    TM Forum
    ------------------------------