VDK Logs Usability Testing

Table of Contents

• Purpose
• Background
  • Objective
• Participants
• Environment
• Data Job for Testing
• Task for Participants
• Usability Testing Process
• Post-Testing

Purpose

The purpose of this usability testing is to evaluate the effectiveness and clarity of our logging system. By engaging users in a hands-on experience with a pre-configured environment, we aim to observe how they interact with the logs to identify and resolve errors.

Background

Objective:

The primary goal is to assess how users interact with VDK, especially in terms of navigating logs and identifying as well as troubleshooting errors.

Participants:

• TO BE ADDED

Environment:

We will use this pre-configured environment for the usability testing:

The environment comes with a ready failing job for the participants to work with.

Data Job for Testing:

Job Design:

Data processing job within VDK that is intentionally designed to fail due to a common and identifiable error. The job is simple yet reflective of real-world tasks users might encounter.

Task for Participants:

1. Run the Job: Instruct participants to execute the data job.
2. Assess the Logs: Participants should closely examine the logs generated during the job execution to gain insights into the job's behavior and error specifics.
3. Identify the Error: Participants should use their assessment of the logs to identify the specific cause of the job failure.
4. Attempt to Fix: After identifying the error, participants are tasked with implementing a solution based on their understanding and available resources.
5. Re-run the Job: Once the fix is applied, participants should re-run the job to verify if their solution successfully resolved the issue.

Usability Testing Process:

• Introduction: Briefly introduce participants to the testing environment and the task at hand.
• Execution: Allow participants to run the job, examine the logs, and attempt to rectify the error.
• Observation: Observe and record their approach, focusing on how they interact with the logs and the troubleshooting steps they take.
• Feedback: Conduct a short post-task conversation to gather feedback on their experience, challenges faced, and suggestions for improvement.

Post-Testing:

• Report on Data Collected: Analyze common patterns, difficulties, and areas of improvement identified during the testing.
• Action Items for Improvements: Develop a list of actionable steps to enhance the usability and effectiveness of the logging system.