Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Biomarkers and surrogate end points—the challenge of statistical validation

Nature Reviews Clinical Oncology volume 7, pages 309–317 (2010)Cite this article

Subjects

Abstract

Biomarkers and surrogate end points have great potential for use in clinical oncology, but their statistical validation presents major challenges, and few biomarkers have been robustly confirmed. Provisional supportive data for prognostic biomarkers, which predict the likely outcome independently of treatment, is possible through small retrospective studies, but it has proved more difficult to achieve robust multi-site validation. Predictive biomarkers, which predict the likely response of patients to specific treatments, require more extensive data for validation, specifically large randomized clinical trials and meta-analysis. Surrogate end points are even more challenging to validate, and require data demonstrating both that the surrogate is prognostic for the true end point independently of treatment, and that the effect of treatment on the surrogate reliably predicts its effect on the true end point. In this Review, we discuss the nature of prognostic and predictive biomarkers and surrogate end points, and examine the statistical techniques and designs required for their validation. In cases where the statistical requirements for validation cannot be rigorously achieved, the biological plausibility of an end point or surrogate might support its adoption. No consensus yet exists on processes or standards for pragmatic evaluation and adoption of biomarkers and surrogate end points in the absence of robust statistical validation.

Key Points

  • Candidate prognostic biomarkers are relatively easy to identify, but multi-site validation has rarely been done

  • Predictive biomarkers require extensive data for validation, based on large randomized clinical trials and meta-analyses

  • Surrogate end points require data demonstrating both that the surrogate is prognostic of the true end point, and that the effect of treatment on the surrogate correlates with that of the true end point

  • The biological plausibility of a biomarker or surrogate might support its adoption even in cases where full statistical validation is lacking

  • No consensus exists on the best approach for pragmatic evaluation and adoption of biomarkers and surrogate end points when robust statistical validation is lacking

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Prognostic markers: initial findings with the Amsterdam 70-gene signature in breast cancer.
Figure 2
Figure 3

Similar content being viewed by others

References

  1. Rifai, N., Gillette, M. A. & Carr, S. A. Protein biomarker discovery and validation: the long and uncertain path to clinical utility. Nat. Biotechnol. 24, 971–983 (2006).

    Article  CAS  PubMed  Google Scholar 

  2. Biomarkers Definitions Working Group. Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clin. Pharmacol. Ther. 69, 89–95 (2001).

  3. Temple, R. J. A regulatory authority's opinion about surrogate endpoints. In Clinical Measurement in Drug Evaluation (Eds Nimmo, W. S. & Tucker, G. T.) 17 (Wiley, New York, 1995).

    Google Scholar 

  4. Ransohoff, D. F. Rules of evidence for cancer molecular-marker discovery and validation. Nat. Rev. Cancer 4, 309–314 (2004).

    Article  CAS  PubMed  Google Scholar 

  5. Goodsaid, F. M., Frueh, F. W. & Mattes, W. Strategic paths for biomarker qualification. Toxicology 245, 219–223 (2008).

    Article  CAS  PubMed  Google Scholar 

  6. Wagner, J. A., Williams, S. A. & Webster, C. J. Biomarkers and surrogate end points for fit-for-purpose development and regulatory evaluation of new drugs. Clin. Pharmacol. Ther. 81, 104–107 (2007).

    Article  CAS  PubMed  Google Scholar 

  7. Goodsaid, F. & Frueh, F. Biomarker qualification pilot process at the US Food and Drug Administration. AAPS J. 9, E105–E198 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Clarke, M. Meta-analyses of adjuvant therapies for women with early breast cancer: the Early Breast Cancer Trialists' Collaborative Group overview. Ann. Oncol. 17, 59–62 (2006).

    Article  Google Scholar 

  9. Sørlie, T. et al. Gene expression patterns of breast carcinomas distinguish tumor subclasses with clinical implications. Proc. Natl Acad. Sci. USA 98, 10869–10874 (2001).

    Article  PubMed  PubMed Central  Google Scholar 

  10. Sørlie, T. Molecular classification of breast tumors: toward improved diagnostics and treatments. Methods Mol. Biol. 360, 91–114 (2007).

    PubMed  Google Scholar 

  11. Slamon, D. J. et al. Use of chemotherapy plus a monoclonal antibody against HER2 for metastatic breast cancer that overexpresses HER2. N. Engl. J. Med. 344, 783–792 (2001).

    Article  CAS  PubMed  Google Scholar 

  12. Piccart-Gebhart, M. J. et al. Trastuzumab after adjuvant chemotherapy in HER-2 positive breast cancer. N. Eng. J. Med. 353, 1659–1672 (2005).

    Article  CAS  Google Scholar 

  13. Romond, E. H. et al. Trastuzumab plus adjuvant chemotherapy for operable HER-2 positive breast cancer. N. Eng. J. Med. 353, 1673–1684 (2005).

    Article  CAS  Google Scholar 

  14. Slamon, D. et al. Phase III randomized trial comparing doxorubicin and cyclophosphamide followed by docetaxel with doxorubicin and cyclophosphamide followed by docetaxel and trastuzumab with docetaxel, carboplatin and trastuzumab in HER2 positive early breast cancer patients: BCIRG 006 study. In Proc. 28th Annual San Antonio Breast Cancer Symp. 1 (San Antonio, Texas, USA 2005).

    Google Scholar 

  15. Joensuu, H. et al. Adjuvant docetaxel or vinorelbine with or without trastuzumab for breast cancer. N. Engl. J. Med. 354, 809–820 (2006).

    Article  CAS  PubMed  Google Scholar 

  16. Benjamin, R. S. et al. Gastrointestinal stromal tumors II: medical oncology and tumor response assessment. Semin. Oncol. 36, 302–311 (2009).

    Article  CAS  PubMed  Google Scholar 

  17. Gora-Tybor, J. & Robak, T. Targeted drugs in chronic myeloid leukemia. Curr. Med. Chem. 15, 3036–3051 (2008).

    Article  CAS  PubMed  Google Scholar 

  18. Amado, R. G. et al. Wild-type KRAS is required for panitumumab efficacy in patients with metastatic colorectal cancer. J. Clin. Oncol. 26, 1626–1634 (2008).

    Article  CAS  PubMed  Google Scholar 

  19. Di Fiore, F. et al. Clinical relevance of KRAS mutation detection in metastatic colorectal cancer treated by Cetuximab plus chemotherapy. Br. J. Cancer 96, 1166–1169 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Paik, S. et al. Benefit from adjuvant trastuzumab may not be confined to patients with IHC 3+ and/or FISH-positive tumors: Central testing results from NSABP B-31. J. Clin. Oncol. (Meeting abstracts) 25, 511 (2007).

    Google Scholar 

  21. Perez, E. A. et al. Updated results of the combined analysis of NCCTG N9831 and NSABP B-31 adjuvant chemotherapy with/without trastuzumab in patients with HER2-positive breast cancer. J. Clin. Oncol. (Meeting abstracts) 25, 512 (2007).

    Google Scholar 

  22. Buyse, M. Towards the validation of statistically reliable biomarkers. Eur. J. Cancer 41 (Suppl. 1) 89–95 (2007).

    Article  Google Scholar 

  23. van 't Veer, L. J. et al. Gene expression profiling predicts clinical outcome of breast cancer. Nature 41 5, 530–536 (2002).

    Article  Google Scholar 

  24. van de Vijver, M. J. et al. A gene-expression signature as a predictor of survival in breast cancer. N. Engl. J. Med. 347, 1999–2009 (2002).

    Article  CAS  PubMed  Google Scholar 

  25. Sotiriou, C. & Pusztai, L. Gene-expression signatures in breast cancer. N. Engl. J. Med. 360, 790–800 (2009).

    Article  CAS  PubMed  Google Scholar 

  26. Hayes, D. F., Trock, B. & Harris, A. L. Assessing the clinical impact of prognostic factors: when is “statistically significant” clinically useful? Breast Cancer Res. Treat. 52, 305–319 (1998).

    Article  CAS  PubMed  Google Scholar 

  27. Pepe, M. S., Janes, H., Longton, G., Leisenring, W. & Newcomb, P. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am. J. Epidemiol. 159, 882–890 (2004).

    Article  PubMed  Google Scholar 

  28. Royston, P., Parmar, M. K. & Altman, D. G. Visualizing length of survival in time-to-event studies: a complement to Kaplan–Meier plots. J. Natl Cancer Inst. 100, 92–97 (2008).

    Article  PubMed  Google Scholar 

  29. Buyse, M. et al. Validation and clinical utility of a 70-gene prognostic signature for women with node-negative breast cancer. J. Natl Cancer Inst. 98, 1183–1192 (2006).

    Article  CAS  PubMed  Google Scholar 

  30. Desmedt, C. et al. Strong time dependence of the 76-gene prognostic signature for node-negative breast cancer patients in the TRANSBIG multicenter independent validation series. Clin. Cancer Res. 13, 3207–3214 (2007).

    Article  CAS  PubMed  Google Scholar 

  31. US National Library of Medicine. ClinicalTrials.gov [online]. (2009).

  32. US National Library of Medicine. ClinicalTrials.gov [online]. (2009).

  33. US National Library of Medicine. ClinicalTrials.gov [online]. (2009).

  34. Peterson, B. & George, S. L. Sample size requirements and length of study for testing interaction in a 2 × k factorial design when time-to-failure is the outcome. Control. Clin. Trials 14, 511–522 (1993).

    Article  CAS  PubMed  Google Scholar 

  35. Mandrekar, S. J. & Sargent, D. J. Clinical trial designs for predictive biomarker validation: one size does not fit all. J. Biopharm. Stat. 19, 530–542 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  36. Mandrekar, S. J. & Sargent, D. J. Clinical trial designs for predictive biomarker validation: theoretical considerations and practical challenges. J. Clin. Oncol. 27, 4027–4034 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  37. US National Library of Medicine. ClinicalTrials.gov [online]. (2009).

  38. US National Library of Medicine. ClinicalTrials.gov [online]. (2009).

  39. US National Library of Medicine. ClinicalTrials.gov [online]. (2009).

  40. US National Library of Medicine. ClinicalTrials.gov [online]. (2009).

  41. Sargent, D. J., Conley, B. A., Allegra, C. & Collette, L. Clinical trial designs for predictive marker validation in cancer treatment trials. J. Clin. Oncol. 23, 2020–2027 (2005).

    Article  PubMed  Google Scholar 

  42. Karapetis, C. S. et al. K-ras mutations and benefit from cetuximab in advanced colorectal cancer. N. Engl. J. Med. 359, 1757–1765 (2008).

    Article  CAS  PubMed  Google Scholar 

  43. Bokemeyer, C. et al. Fluorouracil, leucovorin, and oxaliplatin with and without cetuximab in the first-line treatment of metastatic colorectal cancer. J. Clin. Oncol. 27, 663–671 (2009).

    Article  CAS  PubMed  Google Scholar 

  44. Van Cutsem, E. et al. KRAS status and efficacy in the first-line treatment of patients with metastatic colorectal cancer (mCRC) treated with FOLFIRI with or without cetuximab: The CRYSTAL experience. J. Clin. Oncol. (Meeting abstracts) 26, 2 (2008).

    Article  Google Scholar 

  45. Buyse, M., Molenberghs, G., Burzykowski, T., Renard, D. & Geys, H. The validation of surrogate endpoints in meta-analyses of randomized experiments. Biostatistics 1, 49–67 (2000).

    Article  CAS  PubMed  Google Scholar 

  46. Estey, E. H., Shen, Y. & Thall, P. F. Effect of time to complete remission on subsequent survival and disease-free survival time in, AML, RAEB-t, and RAEB. Blood 95, 72–77 (2000).

    CAS  PubMed  Google Scholar 

  47. Kern, W. et al. Early blast clearance by remission induction therapy is a major independent prognostic factor for both achievement of complete remission and long-term outcome in acute myeloid leukemia: data from the German AML Cooperative Group (AMLCG) 1992 Trial. Blood 101, 64–70 (2003).

    Article  CAS  PubMed  Google Scholar 

  48. Weir, C. J. & Walley, R. J. Statistical evaluation of biomarkers as surrogate endpoints: a literature review. Stat. Med. 25, 183–203 (2006).

    Article  PubMed  Google Scholar 

  49. Lassere, M. N. The Biomarker-Surrogacy Evaluation Schema: a review of the biomarker-surrogate literature and a proposal for a criterion-based, quantitative, multidimensional hierarchical levels of evidence schema for evaluating the status of biomarkers as surrogate endpoints. Stat. Methods Med. Res. 17, 303–340 (2008).

    Article  PubMed  Google Scholar 

  50. Prentice, R. L. Surrogate endpoints in clinical trials: definition and operational criteria. Stat. Med. 8, 431–440 (1989).

    Article  CAS  PubMed  Google Scholar 

  51. Sargent, D. J. et al. Disease-free survival versus overall survival as a primary end point for adjuvant colon cancer studies: individual patient data from 20,898 patients on 18 randomized trials. J. Clin. Oncol. 23, 8664–8670 (2005).

    Article  PubMed  Google Scholar 

  52. Burzykowski, T., Molenberghs, G. & Buyse, M. (Eds) The Evaluation of Surrogate Endpoints (Springer, New York, 2005).

    Book  Google Scholar 

  53. Buyse, M. & Molenberghs, G. Criteria for the validation of surrogate endpoints in randomized experiments. Biometrics 54, 1014–1029 (1998).

    Article  CAS  PubMed  Google Scholar 

  54. Buyse, M. et al. Relation between tumor response to first-line chemotherapy and survival in advanced colorectal cancer: a meta-analysis. Lancet 356, 373–378 (2000).

    Article  CAS  PubMed  Google Scholar 

  55. Alonso, A., Molenberghs, G., Geys, H., Buyse, M. & Vangeneugden, T. A unifying approach for surrogate marker validation based on Prentice's criteria. Stat. Med. 25, 205–221 (2006).

    Article  PubMed  Google Scholar 

  56. Buyse, M., Burzykowski, T., Michiels, S. & Carroll, K. Individual- and trial-level surrogacy in colorectal cancer. Stat. Methods Med. Res. 17, 467–475 (2008).

    Article  PubMed  Google Scholar 

  57. Prentice, R. L. Surrogate and mediating endpoints: current status and future directions. J. Natl Cancer Inst. 101, 216–217 (2009).

    Article  PubMed  Google Scholar 

  58. Molenberghs, G. et al. Statistical challenges in the evaluation of surrogate endpoints in randomized trials. Control. Clin. Trials 23, 607–625 (2002).

    Article  PubMed  Google Scholar 

  59. Alonso, A. & Molenberghs, G. Surrogate marker evaluation from an information theory perspective. Biometrics, 63, 180–186 (2007).

    Article  Google Scholar 

  60. Buyse, M. Contributions of meta-analyses based on individual patient data to therapeutic progress in colorectal cancer. Int. J. Clin. Oncol. 14, 95–101 (2009).

    Article  PubMed  Google Scholar 

  61. Shi, Q. & Sargent, D. J. Meta-analysis for the evaluation of surrogate endpoints in cancer clinical trials. Int. J. Clin. Oncol. 14, 102–111 (2009).

    Article  PubMed  Google Scholar 

  62. Piedbois, P. & Buyse, M. Endpoints and surrogate endpoints in colorectal cancer: a review of recent developments. Curr. Opin. Oncol. 20, 466–471 (2008).

    Article  PubMed  Google Scholar 

  63. Buyse, M. et al. Validation of biomarkers as surrogates for clinical endpoints. In Biomarkers in Clinical Drug Development (Eds Bloom, J. C. & Dean, R. A.) 149–168 (Marcel Dekker, New York, 2003).

    Google Scholar 

  64. Collette, L. et al. Is prostate-specific antigen a valid surrogate endpoint for survival in hormonally treated patients with metastatic prostate cancer? Joint research of the European Organization for Research and Treatment of Cancer, the Limburgs Universitair Centrum, and AstraZeneca Pharmaceuticals. J. Clin. Oncol. 23, 6139–6148 (2005).

    Article  PubMed  Google Scholar 

  65. Burzykowski, T. & Buyse, M. Surrogate threshold effect: an alternative measure for meta-analytic surrogate endpoint validation. Pharm. Stat. 5, 173–186 (2006).

    Article  PubMed  Google Scholar 

  66. Buyse, M. et al. Progression-free survival is a surrogate for survival in advanced colorectal cancer. J. Clin. Oncol. 25, 5218–5224 (2007).

    Article  CAS  PubMed  Google Scholar 

  67. Burzykowski, T., Buyse, M., Sargent, D., Sakamoto, J. & Yothers, G. Exploring and validating surrogate endpoints in colorectal cancer. Lifetime Data Anal. 14, 54–64 (2008).

    Article  PubMed  Google Scholar 

  68. Miller, K. et al. Paclitaxel plus bevacizumab versus paclitaxel alone for metastatic breast cancer. N. Engl. J. Med. 357, 2666–2676 (2007).

    Article  CAS  PubMed  Google Scholar 

  69. Sargent, D. J. & Hayes, D. F. Assessing the measure of a new drug: is survival the only thing that matters? J. Clin. Oncol. 26, 1922–1923 (2008).

    Article  PubMed  Google Scholar 

  70. Ransohoff, D. F. How to improve reliability and efficiency of research about molecular markers: roles of phases, guidelines, and study design. J. Clin. Epidemiol. 60, 1205–1219 (2007).

    Article  PubMed  Google Scholar 

  71. Pepe, M. S. et al. Phases of biomarker development for early detection of cancer. J. Natl Cancer Inst. 93, 1054–1061 (2001).

    Article  CAS  PubMed  Google Scholar 

  72. Altar, C. A. The Biomarkers Consortium: on the critical path of drug discovery. Clin. Pharmacol. Ther. 83, 361–364 (2008).

    Article  CAS  PubMed  Google Scholar 

  73. McShane, L. M. et al. Reporting recommendations for tumor marker prognostic studies (REMARK). Nat. Clin. Pract. Oncol. 2, 416–422 (2005).

    CAS  PubMed  Google Scholar 

  74. Masood, S. & Bui, M. M. Prognostic and predictive value of HER2/neu oncogene in breast cancer. Microsc. Res. Tech. 59, 102–108 (2002).

    Article  CAS  PubMed  Google Scholar 

  75. Tournigand, C. et al. FOLFIRI followed by FOLFOX6 or the reverse sequence in advanced colorectal cancer: a randomized GERCOR study. J. Clin. Oncol. 22, 229–237 (2004).

    Article  CAS  PubMed  Google Scholar 

  76. Allegra, C. et al. End points in advanced colon cancer clinical trials: a review and proposal. J. Clin. Oncol. 25, 3572–3575 (2007).

    Article  PubMed  Google Scholar 

  77. Green, E., Yothers, G. & Sargent, D. J. Surrogate endpoint validation: statistical elegance versus clinical relevance. Stat. Methods Med. Res. 17, 477–486 (2008).

    Article  PubMed  Google Scholar 

  78. Lathia, C. D. et al. The value, qualification, and regulatory use of surrogate end points in drug development. Clin. Pharmacol. Ther. 86, 32–43 (2009).

    Article  CAS  PubMed  Google Scholar 

  79. Rastelli, F. & Crispino, S. Factors predictive of response to hormone therapy in breast cancer. Tumori 9 4, 370–383 (2008).

    Article  Google Scholar 

  80. Jackman, D. M. et al. Impact of epidermal growth factor eceptor and KRAS mutations on clinical outcomes in previously untreated non-small cell lung cancer patients: results of an online tumor registry of clinical trials. Clin. Cancer Res. 15, 5267–5273 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  81. Bogaerts, J. et al. Gene signature evaluation as a prognostic tool: challenges in the design of the MINDACT trial. Nat. Clin. Pract. Oncol. 3, 540–551 (2006).

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. International Drug Development Institute, 30 Avenue Provinciale, 1340, Louvain-la-Neuve, Belgium

    Marc Buyse

  2. Cancer Center Statistics, Mayo Clinic Cancer Center, 200 First Street SW, Rochester, 55905, MI, USA

    Daniel J. Sargent

  3. Department of Medical Oncology, Mayo Clinic College of Medicine, 200 First Street SW, Rochester, 55905, MI, USA

    Axel Grothey

  4. Fondation ARCAD, 22 Rue Malher, 75004, Paris, France

    Alastair Matheson

  5. Hôpital Saint-Antoine, Pavillon Moïana, 184 rue du Faubourg Saint-Antoine, 75012, Paris, France

    Aimery de Gramont

Authors
  1. Marc Buyse
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel J. Sargent
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Axel Grothey
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alastair Matheson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Aimery de Gramont
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marc Buyse.

Ethics declarations

Competing interests

M. Buyse is a stockholder/director with the International Drug Development Institute. D. J. Sargent is a consultant with the following companies: Almac, DiagnoCure, Exiqon, Genomic Health, Precision Therapeutics. The other authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Buyse, M., Sargent, D., Grothey, A. et al. Biomarkers and surrogate end points—the challenge of statistical validation. Nat Rev Clin Oncol 7, 309–317 (2010). https://doi.org/10.1038/nrclinonc.2010.43

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nrclinonc.2010.43

This article is cited by

Search

Advanced search

Quick links

Nature Briefing: Cancer

Sign up for the Nature Briefing: Cancer newsletter — what matters in cancer research, free to your inbox weekly.

Get what matters in cancer research, free to your inbox weekly. Sign up for Nature Briefing: Cancer