Does the Cleveland Clinic Lymphedema Risk Calculator Really Work?

Wouldn’t it be wonderful if we could accurately predict which post-operative breast cancer patients will go on to develop lymphedema? An online tool proposes to do just that.

The Cleveland Clinic Risk Calculator for Lymphedema

In 2012 a lymphedema prediction model was published, called the Cleveland Clinic Risk Calculator (ref1).  It was designed to calculate the risk of lymphedema in early post-operative patients following axillary lymph node dissection. The authors also turned it into an online tool (http://www.lymphedemarisk.com/) where individuals can enter their own information and calculate their personal 5-year probability of developing lymphedema.

When this came out we were excited by the idea, but our excitement was tempered by the knowledge that although the statistical model underlying their risk calculator was built using a rather large sample of patients (which is great), the data they used relied on patient self-reporting surveys instead of objective data (not ideal), it came from a single Brazilian centre (not ideal), and the underlying statistical model was only ‘internally validated’ (not ideal). So we didn’t publicize their risk calculator, nor have we directed patients or physicians to it.  But we liked it, in principle.

In a study published just this month, researchers out of Pittsburgh decided to test the Cleveland Clinic Risk Calculator at their centre (ref2). This is to our knowledge the first time the risk calculator has been independently evaluated.

The original developers of the risk calculator found their tool to have an AUC score of 0.729 in their ‘internal validation’ (an AUC score of 1.0 would mean perfect predictive ability, while a score of 0.5 would be equivalent to randomly guessing who will get lymphedema). This score would make the tool reasonably effective by most standards. But when their model was applied to a smaller sample of patients in Pittsburgh, the AUC was found to be closer to 0.6, which makes it poorly accurate, and therefore not useful.

Unfortunately the Cleveland Clinic Risk Calculator doesn’t appear to be an effective tool for predicting breast cancer related lymphedema in patients post ALND after all. Two major factors may be contributing to the failure of their model:

1. When they built their risk calculator they did not validate it with an external data set. Whenever prediction models are BOTH created AND validated using the SAME data set (or portion thereof), the resulting prediction models almost always appear to perform better than they actually do in the real world.
2. Groups of patients and standards of health care can differ across health centres (and countries) and so risk calculators designed for one place may not apply elsewhere.

Hopefully the creators of the Cleveland Clinic Risk Calculator will continue to refine their model, we still really like the idea.

A prediction tool that could accurately identify patients who will go on to develop lymphedema would enable earlier detection and better management of the condition.  This could help patients avoid developing advanced lymphedema, and reduce the number and severity of health complications associated with lymphedema including cellulitis, compartment syndrome, and lymphangitis . Early detection could thereby decrease health care costs, and improve patient quality of life.

References:

1. Bevilacqua J.L., Kattan M.W., et al. Nomograms for predicting the risk of arm lymphedema after axillary dissection in breast cancer. Ann Surg Oncol. 2012. Aug; 19(8):2580-9.
2. Soran A., Menekse E., et al. Breast cancer-related lymphedema after axillary lymph node dissection: does early postoperative prediction model work? Support Care Cancer. 2015. Sep 9 [Epub ahead of print]