artnomity

Click "Block Editor" to enter the edit mode. Use layers, shapes and customize adaptability. Everything is in your hands.

Tilda Publishing

create your own block from scratch

THE APPROACH

THE RESULTS

UX RESEARCH

STRATEGY

PROTOTYPING

THE CHALLENGE

How might we bring value for all of the customers across the market in agile product development?

Kaplan was rebranding and migrating products to a new platform. The purpose of the platform is to be the same across different products. Although, there are certain commonalities across the products, there is also an important distinction between the products and the type of resources they offer.

The old model did not include any customer research support, and was built solely on the internal business needs and wants.

MCAT business unit introduced a new 8th edition revision involving both live course experience and online resources with new features and in order to enable the MCAT business unit to sell their institutional products more effectively, as well as expand other potential “build-on” opportunities, it was decided that the MCAT team would be the first one to migrate their content and user experience(s) onto the new platform. However, little to no research was conducted.

One of the challenges I had to face was starting this project in the midst of its development. Very soon I found out that there was no in-depth or discovery research done or documented apart from analytical data that came from the old platform. In fact, there were no personas being developed that our agile teams could refer to. Although some data was still helpful, it was not enough for me to grasp a deeper understanding of our customers and their mental model about studying.

Second challenge was not knowing how the infrastructure at Kaplan works during onboarding. I didn’t know how Kaplan recruits participants. To counteract that, I spent two weeks narrowing down the scope to successfully start initiating the project.

MCAT was the project I lead from the brief to the final analysis before MCAT was fully migrated onto a new platform.

Being the only UX researcher on the team, my role for this project was to conduct the discovery and optimize research to gather and understand attitudinal and behavioural findings from the students. I worked closely with business leaders to identify and address internal wants and constraints, as well as data analytics team to update key stakeholders on quant and qual data that we were finding on weekly basis. Design interactions and usability testing were done with the UX designer team throughout research findings.

I first started to look at the literature review research that I could find that’s been done at Kaplan. Then, I signed up for a weekly meeting with the engineering team and product team to understand the data quality constraints and tech limitations. I also started working with the data analyst to understand the data infrastructure. I was able to narrow down the scope and find the strategic direction that our team can investigate to bring immediate impact.

In order to meet January's deadline for migration, I proposed to conduct the Beta testing of the full MCAT course with 24 live online MCAT students for the period of 3 months. The website was already developed and ready to be tested & the MCAT content was completed with new curriculum structure. I decided that beta testing would be easy to implement with the agile teams to test out the product with small sample size, test for bugs and quickly improve new features in a limited way before launching to the full MCAT population. It also helped determine the overall engagement, outcomes, and satisfaction associated with the new program. Feedback throughout the beta testing was through surveys & moderated as well as unmoderated interviews. I also conducted competitive analysis closer to the end of the beta experience.

Methodology

Constraints

Approach

Participants

MCAT students were recruited through an email newsletter because it was the quickest and cheapest way to reach thousands of our students within a limited timeframe, as well as offering students a gift card incentive. Sample size of 24 was chosen in order to accommodate precision, fixed budget and timeframe I had.

Research Goals

Key metrics

I collaborated with Data Analyst to set up the key metrics for this project. Through weekly meetings with key stakeholders including myself, we looked at the data around student's engagement, allowing us to track key data such as enrollment & retention rate, questions answered per student, days active per student, sequences started per student against our set goals for each of these metrics that was determined by the Data Analyst. With this data, we learned what students are doing on the new platform and how they interact with new features.

The metrics that I was responsible for is NPS score and Usability Score. it was captured closer to the end of the beta testing because we needed to give some time for students to interact with all the resources that MCAT program offers.

Administrative process

I created a tracker on Smartsheet to make sure I track each participant’s engagement and who I need to pay the incentive to. I also used google excel doc to communicate any methods and questions about beta testing with MCAT business leaders outlining. I wanted to make sure business leaders are on the same page with me from the curiosity aspect of what I want to learn vs business leaders from each stage of the research. It was also important to share this document to show the importance of beta testing to the outcome of the project.

Beta process

Started off with the surveys to target a larger volume of responses, which created the opportunity for more in-depth analysis discussion. This was the quick and cheap way of understanding if our onboarding process made sense to students or whether there is an opportunity to enhance that experience, same applied to the calendar. There were a couple of tech issues identified with the calendar and I received some interesting comments of opportunities which prompt me to conduct interviews around the calendar to dive into key comments such as understanding what students meant by the word 'flexibility', what is the current problem that they are experiencing with the schedule, understand what their behaviour looks like if that flexible feature would have been available and how would that solve their current pain point. The number I was targeting for interview recruitment was at least to talk with 5 students because usually this is when the saturation point is reached meaning little new information is being learnt about the topic.

What is affecting students’ confidence?

I designed the survey specifically related to online classes to understand what affects students' confidence, the amount of information covered in each class, the pacing of each class. So as the result of the final class, I was able to map out the emotional state of our students and depict how their confidence is being affected which you can see here.

Analysis + Synthesis

Personas

After having multiple conversations throughout the beta testing with different type of students I created several personas. In fact, there are 3 main personas identified for the MCAT program, those who are Overachieving Oswaldo or Monica who do not necessarily follow the calendar outline that we provide as they know where their strengths and weaknesses lie. Then we have Ingrid who trust the calendar outline and Kaplan course process. And finally we have optimizing ophelia who is a busy person and is making the most of her time with studying, who wants help with test strategies and wants the product that is flexible to meet her personal lifestyle.

This is specifically helpful for designers to make sure they think about who they are designing for.

Prioritization features

Throughout this testing, I understood that the calendar was the biggest area that needed design improvements as students wanted to see extra features. Hence, I used Kano Model to prioritize top features to increase student’s satisfaction because it helps to reveal how bad existing pain points are and why these features are important to students. The Kano Model prioritizes your feature ideas into a clear development plan based on performance enhancement and customer satisfaction levels. As a result, out of 12 features that students wanted to see on their calendar, only 4 were identified as the top priority.

I used affinity mapping to analyse data. I use Dovetail to transcribe my interviews as it saves me time by using tags and then helps me to place them on the affinity map. It helps me to uncover themes around problems and needs, and quickly make sense of my research findings when I group the information together. I then can see which trends, thus the most important insights that came out from my research study. This helps to report on crucial information and let the team know what to prioritise from the research.

I was also curious to see how our competitor, Blueprint, is doing by already providing the flexibility feature in their calendar to their students. I recruited students who are currently studying for the MCAT exam and who are using the Blueprint platform. For anonymous recruitment I used the tool called User Interviews. I then provided the link to the Userlytics platform which the qualified screening participants would access to begin answering specific questions while sharing their screen with me of the Blueprint platform. The majority of questions were designed around the calendar as Blueprint currently provides the flexibility that allows its students to drag and drop assignments to the days they want to. I was particularly curious about that feature and if Blueprint users actually use it and how frequently if so. What other features do they use on a daily and weekly basis? What are their thoughts about the monthly view calendar?

There were 10 participants in total because when I initially targeted 5 students, i was getting conflicting feedback, so i recruiting more participants until I established enough confidence in trend which validated my assumption that students are using drag/drop feature to move their assignments, particularly their long practice exams to another day which is most suitable for them.

Competitive Analysis

The outcome