UX Measurement Bootcamp 2024 – MeasuringU (2024)

Week 1

Tuesday: Online Coursework (~4.5 hours)
This is the day with the shortest live sessions and longest requirement for online coursework to establish a strong foundation for the rest of the course (about 4.5 hours). We strongly recommend (though it is not required) that attendees start the online coursework before the first day of the bootcamp, then using this time to finish or review the lessons. The online topics for the day are:

• Intro to Methods and Metrics (30 min): Introduction to UX methods and metrics
• UX Methods (50 min): A review of common methods for measuring usability and user experiences, plus guidance on selecting the right UX methods for your needs
• UX Metrics (50 min): A taxonomy of 70 UX metrics including the difference between attitude and action metrics, and how to determine the metric you need
• Intro to Data Types (25 min): A tour of different types of data (qualitative vs. quantitative, discrete vs. continuous) and an introduction to levels of measurement
• Measurement Framework Part 1 (55 min): A description of a measurement framework for UX research with real-world examples and applications
• Measurement Framework Part 2 (39 min): Continuation of the description and applications of a measurement framework for UX research

Thursday: Online Coursework (~2 hours)
The online topics for the day are benchmarking and confidence intervals, covered in five lessons (about 3.5 hours):

• Benchmarking Part 1 (38 min): Introduction to benchmarking essentials including what it is, why use it, when to use it, what type to use, and defining and getting access to users.
• Benchmarking Part 2 (41 min): Additional benchmarking essentials including task definition, metrics selection, measurement precision, and study length.
• Confidence Interval Demonstration (12 min): Physical demonstration of the accuracy of confidence intervals
• Confidence Intervals – Intro and Rating Scales (48 min): Introduction to confidence intervals including their three ingredients, a quick stats review, and application to rating scale data
• Confidence Intervals – Binary and Time Data (28 min): Confidence intervals for binary data (e.g., task completion rates) and task time data

Week 2

Tuesday: Online Coursework (~1.5 hours)
The theme of the day is statistical hypothesis testing, covered in three lessons (about 1.5 hours):

• Hypothesis Testing Demonstration (25 min): Physical demonstration of hypothesis testing
• Hypothesis Testing Intro (36 min): An introduction to the Null Hypothesis Significance Testing (NHST) framework, including discussion of significance levels and difference between statistical and practical significance
• Conducting Tests (12 min): Considerations when conducting statistical tests (data types and experimental designs)

Thursday: Online Coursework (~2 hours)
The theme of the day is power and sample size estimation, covered in three lessons (about 2 hours):

• Power and Sample Size estimation – Intro and Problem Discovery (39 min): Introduction to three ways to determine the right sample size and discussion of sample sizes for problem discovery, including whether five users are enough and if more severe problems get discovered first.
• Power and Sample Size Estimation – Precision/Estimation (33 min): With a focus on sample sizes for confidence intervals, discusses precision requirements and different methods for different data types (e.g., binary vs. continuous)
• Power and Sample Size Estimation – Comparison (37 min): Discussion of how effect sizes and power affect sample size estimation for statistical hypothesis tests

Week 3

Tuesday: Online Coursework (~1.5 hours)
The theme of the day is observing/coding usability problems, covered in three lessons:

• Observing/Coding Usability Problems (26 min): At the heart of usability testing, whether it be the do-it-yourself kind or the more industrial strength kind, is documenting, diagnosing and reducing the impact of usability problems.
• Usability Problems Exercise (6 min): You’ll watch videos of users attempting to find the cost of dog grooming. As you watch the videos you’ll use a form to document observed usability problems.
• More on Observing/Coding UI Problems (62 min): This video from our 2018 bootcamp reinforces key points from the other lessons with analysis of a different set of videos (users attempting to rent cars on Budget and Enterprise).

Week 4