Insights and Adventures: Leading a Global Benchmarking Study for a Communication Giant

Embarking on a usability benchmark study for a major communication platform has been one of the most enlightening experiences of my career as a UX researcher. This longitudinal study spanned across diverse markets, including India, the US, and the UK, and provided me with a unique opportunity to delve into the nuances of user behaviors and needs across different cultures and technological literacies.

The study aimed to evaluate various aspects of user interaction with the communication platform through a series of meticulously designed tasks. Time on task, success, failure, friction, and user ratings were the key metrics used to gauge usability. This comprehensive approach ensured a holistic understanding of how users navigate and utilize the app’s features.

It was my first encounter with conducting usability testing for over 30+ tasks within a one-hour session. The complexity and breadth of the tasks provided a comprehensive view of user interactions but also demanded a rigorous approach to data collection and analysis.

This project took me to tier 1 and tier 2 cities in India, including Delhi, Bangalore, Jaipur, Nagpur and Indore. Each city offered a unique blend of language, cuisine, and culture. Jaipur, with its delicious street food and bustling markets, stood out as a particularly memorable experience. Imagine juggling user interviews and usability tests while resisting the temptation of mouth-watering street delicacies—definitely a challenge, but one that added a delightful flavor to the study!

Conducting the study in-person added a valuable dimension, allowing us to observe firsthand the behaviors, expressions, and subtle cues of the participants. This direct interaction was crucial in uncovering insights that would have been missed in a remote setting.

Conducting the study in India involved significant localization and translation efforts. Balancing the need to adapt tasks to local languages while maintaining consistency was a delicate task. Ensuring that the essence of each task remained intact despite linguistic and cultural differences was challenging but essential for the study’s success. It was learning on the go.

Being a constant resource on this project allowed me to develop a deeper understanding of the platform’s usability issues. I learned how to effectively present findings, analyze quantitative data, and integrate it with qualitative insights.

After the first couple of rounds, clear patterns began to emerge. Users often struggled with discovering certain features, and there were recurring issues with navigation and terminology comprehension. These insights were crucial in shaping the recommendations for improving the app’s usability.

The fourth round marked a pivotal moment where we consolidated data across markets. This analysis revealed fascinating contrasts and commonalities in user behaviors, providing rich insights. The research report required balancing quantitative findings with qualitative insights. The goal was to clearly convey the unique behaviors observed in each market, supported by data and real user verbatims. This holistic approach provided a comprehensive view of the platform’s usability, highlighting both strengths and areas for improvement.

Working on this project has not only enhanced my professional skills but also deepened my appreciation for the diverse ways people interact with technology. It was a privilege to contribute to improving a platform that plays such a vital role in connecting people worldwide.