A Usable Skill Diagnostic
THE PROBLEM
The challenge is to design a computer-based simulation of the pipeline heater system that field personnel can use successfully.
Field technicians are asked to take a computer-based simulation of a piece of field equipment called a pipeline heater. Over 300 of these heaters are used to keep regulators malfunctioning and forming frozen hydrates that can damage piping systems. The challenge is to design a computer-based simulation of the pipeline heater system that field personnel can use successfully. Field technicians seldom used desktop computers at work or at home. Finding the right navigation and screen interactions during a testing or diagnostic situation would be a challenge. The following outlines the contextual research required to build the computer simulation.
Duration of Project
Two Months
MAKE OF THE TEAM
Thirty-two employees with a core team of several managers, 3 senior field technitians and one researcher-designer.
KEY GOAL
Deliver a computer-based skill diagnostic that field technicians will find usable.
MY ROLE
My role was to design, develop, program, and execute the front-end research for a computer-based skill diagnostic that would be used by over 200 field technicians at an energy utility company.
I did the research design, which was mainly qualitative, with quantitative analysis of the survey and context analysis between the simulator and traditional methods of certifying field technicians on the equipment. My data collection included interviews, ride along with field employees, prototyping with usability observations and programming the skill simulator.
Pipeline Heater Simulation - User Persona
Take a Closer Look
About the Skill Diagnostic
It is a higher-fidelity simulation that incorporates animation, sound, contextual imagery, and advanced interactions to simulate the tasks that field technicians perform in the field.
The computer-based skill diagnostic, helps field technicians assess their troubleshooting and repair approaches for the Pipeline heater equipment. The skill diagnostic isn't a paper and pencil, multiple-choice tests of skills, so commonly used in online assessment. It is a higher fidelity simulation that incorporates animation, sound, contextual imagery, and advanced interactions to simulate the tasks that field technicians perform in the field. However, the maintenance procedures are performed on the computer.
Pipeline Heater Simulation - Sample Interface
Below is a capture of the skill diagnostic for a pipeline heater.
BREAKING DOWM THE PROCESS
Methodology:
1. Interviews
2. Observations and journaling of work site
3. Ride Alongs with field personnel
4. Co-design with users for adoption and empathy
-
Ethnographic research was needed to construct the Pipeline simulation and its interface.
-
It is important to set realistic content so that the technicians will accept the content and user experience.
-
To establish the actual work context, it was essential to travel to the field locations of these pipeline heaters. The energy utility company uses over 300 pipeline heaters. I consulted with supervisors and technicians to obtain a variety of pipeline heaters to review.
-
Photographs were used to capture the pipeline station apparatus and mechanics.
-
Sounds of the mechanisms and equipment were captured to reproduce sounds used in simulations.
-
-
Prototyping for usability
-
Eleven problem-based scenarios matched the work environment context and the equipment's fault events were programmed into the computer-based skill diagnostic.
-
Prototyping allows a "test drive" of the software and an opportunity to increase the adoption of users.
-
During development, five employees, comprised of supervisors and field technicians, evaluated smaller versions or prototypes of the test for accuracy and realism in capturing the context of their jobs.
-
Eleven problem-based scenarios matched the work environment context and the equipment's fault events were programmed into the computer-based skill diagnostic.
-
-
Surveys
-
The researcher and the specialist interviewed technicians and mechanics to learn what challenges they encountered in the field. They also performed a general needs analysis to assess the knowledge needs of Uco personnel who are responsible for maintaining these heaters.
-
Welcome visitors to your site with a short, engaging introduction. Double click to edit and add your own text.
Results
-
69% of users found the diagnostic easy to use
-
97% of users said they would use it again
-
For 31 participants, over $16.5K was saved when compared to traditional field assessments of skills.
Reflection
Limitations
A sample of only 32 employees to use during the build and final rollout limits generalization to larger populations. However, the skill diagnostic set the "bar" for how skill diagnostics should be done and continued to be used years after development.
Not a Multiple-Choice Test
In a time were knoweldge checks are used frequently, the skill diagnostic did not provide hints to users as most stems in skill checks do. It used robust branching and user automony to choose how much analysis should be done. This allowed for a more authentic experience.
Update with Emmerging Technology
It would be interesting to update this diagnostic with generative artificial intelligence. The guidance, used to help users in each section, could be replaced by a chatbot or AI agent. Simulations, fault programming could be enhanved by AI too.