Utilities are numeric measurements that reflect an individual's beliefs about the desirableness of a health condition, willingness to take risks to gain health benefits, and preferences for time. This report discusses the approaches to assess and compare the validity of methods used to assign utilities for cost-utility analysis. Threats to validity include construct underrepresentation and construct-irrelevant variance. Construct underrepresentation occurs when a stimulus presented to a judge fails to fully represent the depth and complexity of information required in actual judgments. Construct-irrelevant variation occurs when factors irrelevant to preferences influence measurements of utilities. Among several factors that cause construct-irrelevant variation are cognitive abilities, numeracy skills, emotions and prejudices, and the elicitation procedure. Commonly used elicitation methods (visual-analog scales, time tradeoff, and standard gamble) capture different facets of utilities (desirableness of states, time preferences, and risk attitude) to different degrees. The validity of an elicitation protocol depends (1) on the degree to which its scaling method captures the relevant facets of utility and (2) on the degree to which measurements are influenced by construct-irrelevant variation. Discrete-state health index models provide an alternative to direct elicitation of utilities and work by attaching fixed preference weights to observable health states. The creation of discrete-state models with current technologies requires the adoption of strong assumptions about the scaling properties of utilities. Future research must refine methods of eliciting utilities and identify sources of construct-irrelevant variability that reduce the validity of utility assessments. Because of the impact of variation in techniques on measurements, we do not recommend the combination of utilities elicited with different protocols in cost-utility analysis and do not recommend the display of cost-utility ratios from different studies in comparison or "league" tables.