Context & Problem
About Faculty Success
Watermark’s Faculty Success (FS) solution creates a central source of faculty activity, giving institutions deeper insights into their faculty’s contributions to support their growth.
The Problem
Based on past research and ongoing client feedback, we knew Faculty Success was challenging and time consuming for faculty to use, and got little faculty buy-in. Before this project kicked off, the only way for end users to add their publications to the system was through a manual form or search and find process when they log into the product.
Our goal with this feature was to ease faculty’s workload and provide incentive for faculty to log into the system by introducing a feature that completes an ongoing search on faculty’s behalf. We aimed to optimize impact of this feature by determining the timing and criteria of each search, the type and timing of notification to faculty and/or administrators, the process for accepting or rejecting a search result, and then finally, creating an efficient flow for inputting the publication data into faculty’s CV and other necessary parts of the system.
Business Goals
Business Outcome Hypothesis: By proactively presenting faculty with their accomplishments for confirmation rather than requiring their entry, this solution will increase admin confidence in the data, and reduce faculty buy-in friction.
Leading Indicators:
Client implementation times should be shorter by at least 10% (down to <152 days)
Graduation surveys should be more positive for launch by increasing CSAT score to >4.60%
Increase sales bookings by 10% (>47 bookings)
Increase Moonshot to >2.0
My Role
For this project, kicked off in May 2023 with MVP completed December 2023, I was the lead product designer for all aspects of the project.
Responsibilities
Research
Data Synthesis
UI/UX Design
User Testing
Design Toolkit
Sketch & Abstract | Mural | Overflow
Research
Goals
Who are we building this for?
Based on past research, we understand that there are multiple data management practices that exist for our users:
administrators do all the work up uploading CVs and maintaining/adding CV data,
administrators upload CVs initially and have their faculty maintain it, and
faculty upload their CVs and maintain their own CV data
We wanted to gain a better understanding of each use case and how this new feature might affect each. This understanding would hopefully guide us on which use case(s) take priority to solve for, and what that solution looks like.
How does a search work?
In order to best serve our users, we needed to find out exactly what we needed to search for on their behalf. What data points should we automatically include in a search? Do the faculty or admin set this search up initially or do they edit preset search criteria to their preferences? What does a completed search look like?
Timing and Notifications
We needed insight into the optimal timing for both the search functionality and notifying the users. How often are we looking for publications on their behalf? How should we notify someone that we found something matching their search criteria? How often should we notify them? What does the notification look like for admin that are maintaining data for their faculty?
Methodology
Discovery User Interviews
Competitive Analysis
Feature Prioritization
User Testing
After discovering critical pain points and opportunities in regards to a proactive search and publication match, we began working on sketches and wireframes, which will impact our goals. Based on the findings from our discovery interviews, we were able to begin prioritizing needs in order to design a comprehensive solution for proactive publication search. Once those needs were identified and prioritized they were added as backlog tickets.
Remote user interviews during discovery research
User research affinity mapping and feature prioritization
What We Learned
Through the discovery interviews, we were able to answer most of our questions and gleaned some key insights that would help guide our solutioning, including:
framing this feature’s notifications as a value add for faculty, rather than another chore, would be key to faculty’s engagement with the feature;
right now, administrators are very concerned with data integrity so most don’t trust faculty to manually input this data correctly; admin were excited about the assurance of clean data since this feature would pull from trusted sources, and about their reduced effort on behalf of faculty;
for publications, a once per month search and notification to faculty of possible matches sounded ideal across the board - it seemed it would be enough to entice faculty to log in regularly without annoying them;
faculty are very concerned about “Big Brother” and transparency about where their information is going and why;
and finally, a recurring topic brought up during our interviews was the concept of predatory journals and graded/weighted journals, and the impact this had on both the faculty member and institution.
Concepts & Testing
Based on our discovery interviews, we determined that MVP state would include email and in-app notifications for potential publication matches, a match reviewing flow where the user could accept, edit then accept, or reject a publication, and a duplicate publication detection flow where users could replace some, all, or none of a publication already created in their profile.
We decided that later iterations would include an administration review flow to ensure data integrity, a predatory journal flagging feature, and a journal grading or pointing tag that matches industry standards.
Userflow & Initial Concepts
User flow and prototype created in Overflow for testing
Concept Testing & Validation
Publication match and duplicate match user flows were tested as part of a larger Faculty Success user testing initiative, and included 10 clients with a mixture of both faculty users and administrator users.
Additionally, in-person user testing and feedback sessions were held with Indiana University in June 2023 as part of a partnership for developing this feature.
Findings
In general, most users found and completed this task quickly and easily, users were excited about this feature, felt it was comprehensive, and loved that it flows into the next match after reviewing one. “That was awesome”
Most users were hesitant to click the Add to Profile dropdown button without being able to edit the record first, they didn’t really notice the drop down part - should solution for this before launch
For future research and iteration of this: flagging predatory journals and ability to add rankings to journals (brought up multiple times throughout testing)
First Release (Expected January 2024)
Updates were made to the designs following user testing, including a more apparent way to “edit first” before accepting, as well as a way for users to save a duplicate match as a new activity, and MVP is expected to be released January 2024.
Next iterations will include an expanded search to more databases, for more scholarly activity types including funded research, presentations, and service. Design for the predatory and ranked journal opportunities discovered during our research is also in the works and planned for development in PI2 of 2024.