TM Forum Community

Expand all | Collapse all

Measuring customer experience

  • 1.  Measuring customer experience

    Posted Nov 01, 2018 09:08
    How do you effectively measure your customer experience in the current environment? I am also interested, how do we balance the economics of delivering great customer experience?
    #CustomerExperience

    ------------------------------
    Zaheer Ahmed
    KPMG International
    ------------------------------


  • 2.  RE: Measuring customer experience

    TM Forum Member
    Posted Nov 02, 2018 02:44
    One method is to correlate the NPS to churn rate, and based on churn rate work out the dollar value impact.

    ------------------------------
    Dominic Law
    PCCW Global
    ------------------------------



  • 3.  RE: Measuring customer experience

    Posted Nov 05, 2018 03:45
    ​So here's my debacle - NPS is great an is a great tool to really understand where you stand as an organization but what if I want to dig deeper. Say I've spent a good US$20m on my customer experience initiatives, NPS isn't necessarily going yo give me the impact of that initiative or is it? I mean if I am delivering poor service in specific pockets within my organization in areas in which my customers' experience is being impacted, enhancing through other initiatives won't solve it and NPS won't necessarily give me the answer. Is there a more innovative way?

    If I keep spending money on my customer experience and adding initiatives to improve it  - how do I assess the impact and when do you get to the point that the economics and impact just does not balance?

    I'm thinking from the perspective of any investment or business. You can keep funding it but if you do not truly understand or can accurately measure where your cash flow is being lost well then ROI alone is not going to solve your issue or identify your problem.

    ------------------------------
    Zaheer Ahmed
    KPMG International
    ------------------------------



  • 4.  RE: Measuring customer experience

    TM Forum Member
    Posted Jan 09, 2019 09:35
    Edited by Alfonso Miranda Jan 09, 2019 10:55
    Zaheer

    For most service provider they associate service quality with customer experience, therefore most investments targeting customer experience will go toward either measuring it better or improving the quality of the service, however, these decisions are still taken from a network vantage point, considering the experience that a particular node or platform is providing an assuming that the end customer perception will be somehow correlated to the average quality of the network.

    However, human beings perceptions are not linear and we are not impacted by every improvement of the network, just by those that are perceivable within the ecosystem of element that serve us specifically. In order to have a quantifiable impact on customer perception from a network investment it needs to be done using the end customer perspective holistically, by identifying the ecosystem of elements serving each customer according to their mobility and consumption patterns, using this information to asses the potential impact of the investment and deciding on the one that will bring the best return (hint! most of the time is not the highest traffic volume, it is not about managing more traffic, it is about managing better the most valuable traffic), this is a high dimensionality big data problem.



    ------------------------------
    ------------------------------
    Alfonso Miranda
    Bell Labs Consulting
    Alfonso.Miranda@Bell-Labs-Consulting.com
    ------------------------------
    ------------------------------



  • 5.  RE: Measuring customer experience

    Posted Jan 09, 2019 15:02
    Yes, I agree completely that this is big data issue. Also, I think customer experience cannot be accurately measured. However, we could somewhat measure the difference between good and bad experience. Based on my experience, customer experience should be measured by the data collected when customer is using the network. For example, if the network is not in good shape but I am not using it, my experience is not bad. If the network is measured as good by performance or service quality indices but I am playing a real-time game which is very sensitive to RTT, jitter, etc. my experience might still be bad. So, I believe using xDR (from probe, netflow, or whatever) to measure user experience based on performance, quality, availability, reliability, usability, etc. should be able to get close measurement of user experience. With good dimensioning, we can find out who have bad experiences and where they are. Combine with good OSS tools, we can find out why customer experience is bad therefore resolve the problem. This way will deliver better customer experience improvement. This is just my 2 cents.

    ------------------------------
    Fazhong Deng
    OSSEra, Inc.
    ------------------------------



  • 6.  RE: Measuring customer experience

    TM Forum Member
    Posted May 08, 2019 06:12
    Hi

    Customer Experience is a broad area, and it should be splitted in different subtopics.

    You should measure any interaction the customer has with the operator (visit to stores, customer care interactions, ...). At Optare Solutions we have develop a tool to measure measure QoE (quality of experience using the services) for each customer. WE can measure and optimize the subjective perception of the customer using the services (Voice, video streaming, web browsing, VoIP, online gaming...).  We measure QoE on the end user devices, to ensure we have the real experience of the customer. The information could be integrated into the operator systems (CRM, DataWareHouse, marketing automation, campaign management...).

    This tool helps to improve customer experience, but you will also need to measure and act in other processes on the operator (customer care, call center, physical stores...).

    You can read a couple of blog posts I wrote some months ago:

    https://augura.optaresolutions.com/why-we-should-worry-about-qoe-instead-of-other-kpis/
    https://augura.optaresolutions.com/how-can-you-optimize-qoe-with-augura/


    ------------------------------
    Sergio Pereira
    CMO
    Optare Solutions
    ------------------------------



  • 7.  RE: Measuring customer experience

    Posted Nov 02, 2018 03:50
    ​Relating Telco KPIs (Network, Billing, Customer Service etc) to Chrun is the most optimal approach. Most of the operators limits NPS relating to Chrun, since NPS usually collected for very small set of customer and at times customer feedback will be sarcastic and contradicting.

    Also, Telco KPIs are in internal data points purely managed by Operator, he can predictive the Customer Experience in real time and perform corrective measures quickly.


    ------------------------------
    praveen vemula
    Aricent
    ------------------------------



  • 8.  RE: Measuring customer experience

    Posted Nov 05, 2018 01:47
      |   view attached
    Hi, I recently came across the attached 7 Step Planning Guide to Successful Customer Experience which you might find useful in helping you get started.

    ------------------------------
    John Brunkard
    TO BE VERIFIED
    ------------------------------

    Attachment(s)



  • 9.  RE: Measuring customer experience

    Posted Jan 07, 2019 10:31
    Hi,

    as mentioned in the above comments I'd agree that (decreasing) Churn is an effective KPI. Those happier (existing) customers should be easier to up and cross sell to. A KPI linking improving NPS performance with winning more new customers is tricker ...

    One tip is to visit Bain & Company site* whose Fred Reichheld originated NPS and where you should find new material as the methodology matures.

    regards,
    Steve

    *http://www.netpromotersystem.com/about/how-is-nps-related-to-growth.aspx

    ------------------------------
    Steve Belgraver
    TO BE VERIFIED
    ------------------------------



  • 10.  RE: Measuring customer experience

    TM Forum Member
    Posted Jan 08, 2019 09:11
    In Telco , and other industries , generating add-on , cross &upsell revenue from existing Customers ​is enabled by a higher NPS. Customer acquisition is costly hence churn reduction is a top KPI.

    Tying increase in CEI .. , which in telco should be a predictive NPS to revenue growth is an extremely difficult. We are currently rolling out a CEM Solution , and so far have had very limited celebrated victories. Majority of the use-case has been around moving 'happy' customers to faster data connections where we predict that they will consume more data and other services.

    ------------------------------
    thembinkosi ndebele
    MTN Group Limited
    ------------------------------



  • 11.  RE: Measuring customer experience

    TM Forum Member
    Posted Jan 09, 2019 09:35
    Edited by Alfonso Miranda Jan 09, 2019 10:58
    Thembinkosi

    Considering that for most SP the following two premises will be true:

    - There is significant segment of the customer base (around 10% typically) that have a negative margin contribution
    - About 80% of the profitability will come from 40% of the customers

    Under these premises then churn it is not on it self a bad thing, that is why it is a variable that needs to be managed, not minimized. To be able to do it though, you need big data and analytics approach that enable you to target the investment on the right customers at the right time, this assumes that you are able to actually quantify who (end customers) will be impacted  by every investment and to what degree as apposed to the most common approach that just use aggregated numbers (upgrading site X will impact 10.000 customers) where we are bundling together high value customers with negative margin ones.

    ------------------------------
    ------------------------------
    Alfonso Miranda
    Bell Labs Consulting
    Alfonso.Miranda@Bell-Labs-Consulting.com
    ------------------------------
    ------------------------------



  • 12.  RE: Measuring customer experience

    TM Forum Member
    Posted Jan 10, 2019 06:19
    ​Great customer service is one of the biggest issues across all Corporates. Unfortunately the employees that are direct customer facing are also always the lowest paid.
    It takes a special person to only deal with negativity all day everyday and only a small percentage of call center employees can maintain a positive attitude when dealing with customer complaints.
    No company will throw money at this problem as it isn't the answer.
    My opinion is that The solution to good customer service and improved NPS is providing the call centers and staff that are direct customer facing, excellent support.
    I am a manager in this environment and have experienced this first hand.
    The better support the call centers have when dealing with any customer issue, they become more confident of helping the customer knowing that they can resolve the issue quickly and efficiently. This environment creates a positive vibe in the call center area and everyone becomes more confident of answering the next call.
    When you have this situation, the customer experience improves and the company gets better ratings.


    Shane Nel
    MTN Manager: Network Quality and Assurance


    ------------------------------
    Shane Nel
    MTN South Africa
    ------------------------------



  • 13.  RE: Measuring customer experience

    TM Forum Member
    Posted Jan 10, 2019 13:59
    ​Hi,

    This is very complex and complicated topic. It requires cross-domain data collection , data processing and analytics.
    Then cross domain execution. We are used to work in silos and it is hard to accept change.


    Most of the use cases are individual and applicable just for specific scenario. CSP can have great network quality but then lose a lot of customers in short period due to great marketing by competitor initiative which was based on cheap device and price.
    Network quality KPI's and user experience is every year more difficult to correlate even with advanced analytics . Every new service is making correlation between network PI's and customer experience more difficult. In GSM only era of voice and sms was quite easy to measure user experience directly from Network PI's.

    I believe NPS and Churn are good PI's to work with combined and correlated with other metrics could give good picture of current customer experience.




    ------------------------------
    Tomislav Ivankovic
    Amdocs Management Limited
    ------------------------------



  • 14.  RE: Measuring customer experience

    Posted Jan 11, 2019 01:58
    I agree with you on complexity is high. However, the technical side is not so difficult to handle. The hard part is cross domain execution. It is hard to get all data into one platform and build solution for multiple departments without clear owner. I believe operators will be able to overcome the complexity as it is very clear that silo solutions are not longer good for this digital age.

    ------------------------------
    Fazhong (David) Deng
    OSSEra, Inc.
    ------------------------------



  • 15.  RE: Measuring customer experience

    TM Forum Member
    Posted Jan 11, 2019 12:49
    ​The lack of OLA's in a lot of companies is a big contributor to poor customer service.
    Trying to improve customer experience by only focusing on the departments that are directly customer facing is long outdated.
    Customer Service needs to be part of everyone's KPI's, irrespective of whether they deal directly with customers or not.
    Networks have become too complex for single departments to be responsible for excellent customer service.

    ------------------------------
    Shane Nel
    MTN South Africa
    ------------------------------



  • 16.  RE: Measuring customer experience

    TM Forum Member
    Posted May 03, 2019 02:39

    Some how in my mind I think it is so difficult to understand customer. As I understand tmf recommend using a complete solution and ofcourse SP need to improve any process to satisfy their customer.

    After having MNP, we found many customer change to competitor but no complain in the CRM in 1 year.That is terrible and we need to know why?



    ------------------------------
    Nguyen Tuan Anh
    Vietnam Posts and Telecommunications Group (VNPT)
    ------------------------------



  • 17.  RE: Measuring customer experience

    Posted Jan 11, 2019 12:49
    I agree that managing churn is an important way to keep on top of customer experience, and especially tracking why customers are leaving the service. I wrote a paper on that a while ago. Here's a page where you can get it without filling out a form. Just click the "download" link: Download White Paper: Managing Customer Churn.  There's another paper referenced on that page. If you want to check it out without completing the form you can get it here: Best Practices for Providing Uncompromising Broadband Support

    Of course, it's better to handle customer support or service issues before a subscriber winds up leaving you because of poor service.  I agree that maintaining a good network is critical.  The better the network and quality of service is, the fewer calls your call center will get because of service problems. Fortunately, there are a lot of diagnostics tools now that can help operators find issues in their broadband network faster. In fact, in a DOCSIS network, Proactive Network Maintenance (PNM) tools even let you find problems like micro-reflections and group delay before service is impacted. These and other kinds of diagnostics applications can significantly reduce operational costs and improve the subscriber experience, which is a win-win.

    ------------------------------
    Rick Yuzzi
    ZCorum
    ------------------------------



  • 18.  RE: Measuring customer experience

    Posted Feb 14, 2019 04:27
    Hi Zaheer,

    To measure the customer experience, first, check the churn rate with your sales team. Then conduct a client satisfaction survey and ask for the feedback of your clients. You'll definitely get to know their experience with your service. Customer delight and customer success can create a great CX balance in the current environment.

    Thanks,
    Anand Mahajan
    Owner of Sphinx Solutions

    ------------------------------
    Anand Mahajan
    TO BE VERIFIED
    ------------------------------



  • 19.  RE: Measuring customer experience

    Posted Apr 30, 2019 06:21
    I can share some of our findings regarding how to effectively measure customer experience regarding QoS.

    We have an application called SubSearch that collects data on each subscriber in the network so you will se KPIs on Voice, Data and Coverage it is built for the customer care dpt. We have all this data but does it correlate with NPS? Does a bad data KPI actually impact the NPS, we did not know until recently when we got a real time NPS survey stream (only asking for QoS NPS).

    We can see clear correlation between Voice, Data and Coverage. You need a system to collect QoS KPIs and NPS figures from the customers, you combine them and then you will get the customer experience in the current environment.

    I will not go in to what financial impact an improved NPS means, there is so much written on the subject.

    See you in Nice :-)

    ------------------------------
    [Fredrik] [Edwall]
    [EVP Sales & Marketing/Co-Founder][Subtonomy]
    ------------------------------



  • 20.  RE: Measuring customer experience

    TM Forum Member
    Posted May 02, 2019 04:16
    Am interested in subsearch application. Pls share if it is free application or paid one and it cost and license model

    ------------------------------
    Regards
    Hani
    ------------------------------



  • 21.  RE: Measuring customer experience

    Posted May 07, 2019 05:46
    Hi Hani,

    Thank you for your interest in SubSearch (save >15min/ QoS call)!

    We provide SubSearch under a SaaS model so it is actually a service we provide with all the benefits surrounding it. We charge a system and an application cost for SubSearch based on the amount of IMSIS=subs you have in your network. Note: we don't export out the network data to a Subtonomy cloud, the data will reside within your own secure premises.

    Included in the price
    • Right to use the ordered application
      • All use cases out of the box
    • Unlimited concurrent users of the applications
    • Functionality for 2G, 3G, LTE, VoLTE & VoWifi, 5G environment
    • Functionality according to specification
    • Upgrades of the applications & system
    • Patch upgrades of the applications & system
    • Support & Maintenance
    • Operational support of the SW
    If you would like to discuss a price for your specific network please ping me on fredrik.edwall@subtonomy.com.

    ------------------------------
    Fredrik Edwall
    EVP Sales & Marketing/Co-Founder
    fredrik.edwall@subtonomy.com
    +46 705 66 06 85
    Subtonomy
    ------------------------------



  • 22.  RE: Measuring customer experience

    TM Forum Member
    Posted May 02, 2019 04:21
    Hi Fredrik,
    Interesting correlations and I am sure you get a good view of the quantitative measures/standards that you and your client want to achieve.
    However, I will question if this is fully reflecting your customers' experiences.
    NPS, is more about branding and recommendation of the "brand", and whether the branded company help customers generate value.
    Are you and your customers helping/supporting your customers generate value, or are you generating revenue on customer expense? 
    In today's world, surveys do not have the same meaning as it used to. The emotional state of a person when answering questions is very different from the state a person is in immediately after an experience,
    something that has been proven right in general elections and referendums in recent years.  Including your home country.
    I think as an industry, telecommunications, and also vendor communities have to base customers' experiences on a wider, and recent contextual correlations including dialogues with customers.
    Correlating NOC data and NPS only is too tinted. 


    ------------------------------
    Cato Rasmussen
    Independent
    ------------------------------



  • 23.  RE: Measuring customer experience

    Posted May 02, 2019 08:35

    Hi Cato,

    Thank you for commenting on my post!

    We are experts in our domain QoS = interpret network data and translating it to KPIs (from a customer perspective) regarding QoS, aggregated on single subscriber in a mobile network. We use this data in our applications that are purposely built to cater for different departments at an MNO e.g. customer service. They use it to assist and solve their customers issues with their services while the customer is on the call. In addition we have APIs, so we can distribute the data on each and every subscriber to e.g. a customer fronting app, where the subscriber can do their own customer self service test and see selected QoS KPIs themselves.

    We know exactly per subscriber how the different services have worked  for Voice, SMS, Data and coverage. So what does that mean? It means that we know how the device and network has performed but we don't know to what degree this has impacted the subscribers feeling towards the service. It may be fine that you have 5 call drops in a months time or not.

    You can do extremely complex calculations to try figure out the perceived customer experience from KPIs but without surveys you will not know, even with surveys you will not get a 100% accurate picture, but it will be the best you can get. Most NPS surveys don't address QoS, so you need to address this explicitly. I believe in simplicity so I beg to differ when it comes to NPS, I like it.

    We will not solve the complete customer experience puzzle, network quality is 30-40% according to different studies but that part is where we are experts and can assist.

    Regards Fredrik Edwall



    ------------------------------
    [Fredrik] [Edwall]
    [EVP Sales & Marketing/Co-Founder][Subtonomy]
    ------------------------------



  • 24.  RE: Measuring customer experience

    TM Forum Member
    Posted May 13, 2019 03:56
    To effectively measure customer experience, you need to fully understand all interactions each customer has along his/her respective journey. This is only effective if you're capturing digital touchpoints like the customer experience on the website, mobile app, or webchat session, physical touchpoints like interaction with representatives in a store or phone, service (not network) experience, and lastly validation through CSAT or NPS. Once these key ingredients necessary to properly evaluate and continually improve customer experience are captured, reduced churn would be a byproduct, especially when applying machine learning and predictive analytics.
    The good news is there's a platform, Splunk, that can help capture all of these customer insights in one spot and allow for that correlation to begin. More good news…we'll be at booth #311 this week to show you a demo on Customer Experience and help answer any questions you have, so stop by!


    ------------------------------
    Khalid Ali
    Industry Tech Lead, Communications and Media
    Splunk Inc.
    Mobile: +1 703.203.5582
    khalid@splunk.com
    ------------------------------



  • 25.  RE: Measuring customer experience

    TM Forum Member
    Posted May 13, 2019 06:56

    Dears

    Am interested to be part from this , thanks to keep me updated Splunk implementation and use cases in the telecom industry

     

    Regards

    Hany

     






  • 26.  RE: Measuring customer experience

    TM Forum Member
    Posted May 13, 2019 14:34
    Hi Hany,

    I'd be happy to touch base with you. Feel free to shoot me an email or if you're at the event, drop by and we can have a discussion, whiteboard session, etc.

    Best,
    Khalid

    ------------------------------
    Khalid Ali
    Splunk
    ------------------------------



  • 27.  RE: Measuring customer experience

    TM Forum Member
    Posted May 15, 2019 02:27
    I would like to have a go at this one.

    Summary:
    1. create inventory of customer goals per channel and/or application
    2. measure succesrate per customer goal on all channels and/or applications
    3. measure satisfaction(or effort score) per customer goal on all channels and/or aplications
    4. decide on strategic omnichannel approach with new insights

    We see that most companies have a broad range of applications and channels lined up to interact with our customers with the sole purpose of facilitating customers to acheive their goal. This goal could be something like, setting up an account, buying a productor add-on, getting help with installation, or simply installing something.  As a company we should aim to facilitiate this as effective and seemless as possible and in the most cost-efficient way. Right?

    If so, we should start measuring exactly that. for tis, it is key to know  what platform, channel, application or front-end in your business is facilitating what set of goals. When you have your inventory ready on your most valuable goals on all your channels, you can start measuring the performance per goal, per channel or platform.

    Do this by measuring Succes Rate of you goals on all touchpoints and then ask your customers how they experience the journeys your teams build  towards it, or how they experience they way that your stores or call-centers facilitatethem towards those goals(there could be more than one journey leading towards the same goal).

    Once you have these insights, you can start quantifiying your channels and platforms and start diverging you customers to the most cost efficient channel or the channel with the highest satisfaction.

    My 2 cents. :)


    ------------------------------
    Kevin Quast
    VodafoneZiggo
    ------------------------------



  • 28.  RE: Measuring customer experience

    Posted May 21, 2019 09:00

    This depends on a great deal on the data available to over measurement over the user experience. Then from this prospective, many analysis may be drawn, for example, studying the received complaints & based on that analyzing the respective KPI that is highly impacting the user experience, then next step to design thresholds on different dimensions, network level, device level, user level etc.

    ------------------------------
    Mohammed Al-Kalbani
    Ooredoo Oman
    ------------------------------