Open APIs

Hi everyone,

right now we implemented parts of the STM API regarding executing tests and delivering measurements back.
But now we have a usecase where we need to validate/rate the measurements of a test (like if measurement a is below 50 and measurement b is over 60 the service should be "ok").

Should this be implemented in STM aswell or would it make more sense to place it somewhere else (like ServiceProblemManagement) ?

Right now I think there are  3 options:

• just deliver the raw measurements and let SPM do the validation/rating
• deliver measurements (and adding threashold rules/violations) and SPM does the final summary/validation of the service status taking the violations into account
• add an additional measurement (like summary) and aggregate the "result" there in sth like "serviceStatus=ok" or "serviceStatus=not_ok"

Any ideas what the best solution could be?

------------------------------
Mike Joost
EWE TEL GmbH
------------------------------

Hi Mike,

let me reply to you question as regular TM Forum member and express my opinions. First, what do you mean about raw measurements? Because raw measurement can be realy mess which can read only NT professional and you will probably need also some (business) user or customer readable measurements. I think that this transformation should be done at one place inside this service. If needed, I'm sure there is a way  how to publish both sets of measured data (raw and user friendly).

Regarding rules violation calculation, I also think it should be done at one place and as I understand TMF653 correctly, TMF guys thinks the same and therefore there is ruleViolation section in Swagger and MeasureThresholdRuleViolation in Resource model.

Jan

------------------------------
Ján Krištof
T-Mobile Czech & Slovak Telekom, a.s.
------------------------------

Original Message:
Sent: Apr 06, 2022 08:28
From: Mike Joost
Subject: Valuation of Measurements from TMF653 (ServiceTestManagement)

Hi everyone,

right now we implemented parts of the STM API regarding executing tests and delivering measurements back.
But now we have a usecase where we need to validate/rate the measurements of a test (like if measurement a is below 50 and measurement b is over 60 the service should be "ok").

Should this be implemented in STM aswell or would it make more sense to place it somewhere else (like ServiceProblemManagement) ?

Right now I think there are  3 options:

• just deliver the raw measurements and let SPM do the validation/rating
• deliver measurements (and adding threashold rules/violations) and SPM does the final summary/validation of the service status taking the violations into account
• add an additional measurement (like summary) and aggregate the "result" there in sth like "serviceStatus=ok" or "serviceStatus=not_ok"

Any ideas what the best solution could be?

------------------------------
Mike Joost
EWE TEL GmbH
------------------------------

Yes, the Violation calc. should be done in the same service of course. We just havent implemented it yet. I just wanted to point out that an option could be to "split" the validation process up a bit in a) do validation violation/rules in the 653 "service" and b) validate/process the violations somewhere else (SPM for example).

With "raw" measurements I meant values like "bandwith" or "port status". In contrast to "serviceStatus" = "green" :-)

Right now in a different project we have a Customer/Service Problem Management solution running (where you can define complex workflows very well) and this solution gets the measurements and then the workflow calculates, what the problem could be. But this solution wont be used in the project I'm working on right now. So here we need to decide where to put this validation/rules engine best while trying to avoid to write any complex "rules" in plain java code :-)

------------------------------
Mike Joost
EWE TEL GmbH
------------------------------

Original Message:
Sent: Apr 07, 2022 05:03
From: Ján Krištof
Subject: Valuation of Measurements from TMF653 (ServiceTestManagement)

Hi Mike,

let me reply to you question as regular TM Forum member and express my opinions. First, what do you mean about raw measurements? Because raw measurement can be realy mess which can read only NT professional and you will probably need also some (business) user or customer readable measurements. I think that this transformation should be done at one place inside this service. If needed, I'm sure there is a way  how to publish both sets of measured data (raw and user friendly).

Regarding rules violation calculation, I also think it should be done at one place and as I understand TMF653 correctly, TMF guys thinks the same and therefore there is ruleViolation section in Swagger and MeasureThresholdRuleViolation in Resource model.

Jan

------------------------------
Ján Krištof
T-Mobile Czech & Slovak Telekom, a.s.

Original Message:
Sent: Apr 06, 2022 08:28
From: Mike Joost
Subject: Valuation of Measurements from TMF653 (ServiceTestManagement)

Hi everyone,

right now we implemented parts of the STM API regarding executing tests and delivering measurements back.
But now we have a usecase where we need to validate/rate the measurements of a test (like if measurement a is below 50 and measurement b is over 60 the service should be "ok").

Should this be implemented in STM aswell or would it make more sense to place it somewhere else (like ServiceProblemManagement) ?

Right now I think there are  3 options:

• just deliver the raw measurements and let SPM do the validation/rating
• deliver measurements (and adding threashold rules/violations) and SPM does the final summary/validation of the service status taking the violations into account
• add an additional measurement (like summary) and aggregate the "result" there in sth like "serviceStatus=ok" or "serviceStatus=not_ok"

Any ideas what the best solution could be?

------------------------------
Mike Joost
EWE TEL GmbH
------------------------------

Hi Mike,

it's hard to give you answer, we don't see full picture, but you do. Best solution is the one which best fits your needs.

If you implement validations in SPM, be aware of that if one of consumer (customer portal, technical call centrum, automated call machine, web chat bot) need to perform "healt check" of some service, it will be dependent on two other components (SPM and test management).

I got the idea behind "plain java coding" and I think I agree with you, only basic check should be done in test management, complex checks should be done somewhere else (SPM, AI and BigDATA, etc.).

Jan

------------------------------
Ján Krištof
T-Mobile Czech & Slovak Telekom, a.s.
------------------------------

Original Message:
Sent: Apr 08, 2022 08:47
From: Mike Joost
Subject: Valuation of Measurements from TMF653 (ServiceTestManagement)

Yes, the Violation calc. should be done in the same service of course. We just havent implemented it yet. I just wanted to point out that an option could be to "split" the validation process up a bit in a) do validation violation/rules in the 653 "service" and b) validate/process the violations somewhere else (SPM for example).

With "raw" measurements I meant values like "bandwith" or "port status". In contrast to "serviceStatus" = "green" :-)

Right now in a different project we have a Customer/Service Problem Management solution running (where you can define complex workflows very well) and this solution gets the measurements and then the workflow calculates, what the problem could be. But this solution wont be used in the project I'm working on right now. So here we need to decide where to put this validation/rules engine best while trying to avoid to write any complex "rules" in plain java code :-)

------------------------------
Mike Joost
EWE TEL GmbH

Original Message:
Sent: Apr 07, 2022 05:03
From: Ján Krištof
Subject: Valuation of Measurements from TMF653 (ServiceTestManagement)

Hi Mike,

let me reply to you question as regular TM Forum member and express my opinions. First, what do you mean about raw measurements? Because raw measurement can be realy mess which can read only NT professional and you will probably need also some (business) user or customer readable measurements. I think that this transformation should be done at one place inside this service. If needed, I'm sure there is a way  how to publish both sets of measured data (raw and user friendly).

Regarding rules violation calculation, I also think it should be done at one place and as I understand TMF653 correctly, TMF guys thinks the same and therefore there is ruleViolation section in Swagger and MeasureThresholdRuleViolation in Resource model.

Jan

------------------------------
Ján Krištof
T-Mobile Czech & Slovak Telekom, a.s.

Original Message:
Sent: Apr 06, 2022 08:28
From: Mike Joost
Subject: Valuation of Measurements from TMF653 (ServiceTestManagement)

Hi everyone,

right now we implemented parts of the STM API regarding executing tests and delivering measurements back.
But now we have a usecase where we need to validate/rate the measurements of a test (like if measurement a is below 50 and measurement b is over 60 the service should be "ok").

Should this be implemented in STM aswell or would it make more sense to place it somewhere else (like ServiceProblemManagement) ?

Right now I think there are  3 options:

• just deliver the raw measurements and let SPM do the validation/rating
• deliver measurements (and adding threashold rules/violations) and SPM does the final summary/validation of the service status taking the violations into account
• add an additional measurement (like summary) and aggregate the "result" there in sth like "serviceStatus=ok" or "serviceStatus=not_ok"

Any ideas what the best solution could be?

------------------------------
Mike Joost
EWE TEL GmbH
------------------------------

Hi Mike,
You can do that in the service test management itself. If you look at service test specification, there is the concept of ThresholdRule, Consequence etc. You can define your thresholds and consequential action also. Ofcourse, as a starting step, you need to define a metric and all these checks are against those collected metric value.



------------------------------
Varun Nair
Telstra Corporation
Note: This is an opinion based on my research and not an official TMF response.
------------------------------

Original Message:
Sent: Apr 06, 2022 08:28
From: Mike Joost
Subject: Valuation of Measurements from TMF653 (ServiceTestManagement)

Hi everyone,

right now we implemented parts of the STM API regarding executing tests and delivering measurements back.
But now we have a usecase where we need to validate/rate the measurements of a test (like if measurement a is below 50 and measurement b is over 60 the service should be "ok").

Should this be implemented in STM aswell or would it make more sense to place it somewhere else (like ServiceProblemManagement) ?

Right now I think there are  3 options:

• just deliver the raw measurements and let SPM do the validation/rating
• deliver measurements (and adding threashold rules/violations) and SPM does the final summary/validation of the service status taking the violations into account
• add an additional measurement (like summary) and aggregate the "result" there in sth like "serviceStatus=ok" or "serviceStatus=not_ok"

Any ideas what the best solution could be?

------------------------------
Mike Joost
EWE TEL GmbH
------------------------------

Hello Mike,

I have been thinking about the exact same thing. To me the 653 spec does no look like it meets the requirements for "diagnostics". Being able to tell us if the collection (and/or) of atomic test results (pass/fail, up/down) in a brother context are good or bad. Meaning if testMeasure 2, 4 and 8 are out of bounds/bad, how does this affect the overall result of the test. What component is supposed to say: because you have these failures the overall test is a pass or fail. I do not see a way to add AND and OR clauses.

I only see Service Problem being responsible for orchestrating the test and diagnosis but not actually performing it. For me it feels like there is a missing diagnostic API that would include composite atomic tests managed by 653. This Diagnostic API would take the 653 result and apply some rules to detect what is the likely cause of the failed test and this can then be used in Service Problem for further orchestration.

Have you found a way forward? I am battling with the same question. I would be happy to contribute.

------------------------------
Marc Landry
Bell Canada
------------------------------

Original Message:
Sent: Apr 06, 2022 08:28
From: Mike Joost
Subject: Valuation of Measurements from TMF653 (ServiceTestManagement)

Hi everyone,

right now we implemented parts of the STM API regarding executing tests and delivering measurements back.
But now we have a usecase where we need to validate/rate the measurements of a test (like if measurement a is below 50 and measurement b is over 60 the service should be "ok").

Should this be implemented in STM aswell or would it make more sense to place it somewhere else (like ServiceProblemManagement) ?

Right now I think there are  3 options:

• just deliver the raw measurements and let SPM do the validation/rating
• deliver measurements (and adding threashold rules/violations) and SPM does the final summary/validation of the service status taking the violations into account
• add an additional measurement (like summary) and aggregate the "result" there in sth like "serviceStatus=ok" or "serviceStatus=not_ok"

Any ideas what the best solution could be?

------------------------------
Mike Joost
EWE TEL GmbH
------------------------------