Guide to Using the SRTR Website

Below we detail nine changes made to the search results presentation. Click on each image for a larger view.

Description of the change: The measure “Transplant Volume” was renamed “Transplants in a year” for all organs except kidney and liver. Kidney and liver now include new separate living and deceased donor measures described in change #6 (eg, “Deceased Donor Transplants in a Year”). “Transplant Rate” was renamed “Getting a Transplant Faster” for organs other than kidney and liver. For kidney and liver, only deceased donor transplants are now included in the calculated transplant rate, and this is specified in the heading. “Outcome Assessment” was renamed “1-Year [organ] Survival,” and [organ] is replaced by the specific organ in each organ-specific search.

Response to feedback: Patient feedback included the suggestion to clarify that “volume” refers to numbers of transplants performed in a year, and the criticism that “transplant rate” and “outcome assessment” are not descriptive and do not help interpret the information. 

Description of the change: The new website (launched in December 2016) presented a 5-tier assessment for first-year survival with a functioning transplanted organ, and provided the observed (unadjusted) transplant rate as a number, eg, 17.1 transplants per 100 people per year. The updated site provides a simple 5-tier summary for three metrics: survival on the waiting list, getting a deceased-donor transplant faster, and 1-year organ survival. Further feedback received during public comment in 2018 requested that the SRTR remove the 5-tier summary metric for survival on the waitlist for kidney programs because kidney candidates are often not directly under the care of the transplant program. The SRTR Review Committee agreed and this version of the site does not present a 5-tier summary metric for survival on the waitlist for kidney programs.

Response to feedback: SRTR made this change in response to feedback that the original version of the website focused too much on first-year outcomes when getting a transplant faster is arguably more important to overall patient survival. The Consensus Conference also recommended enhancing reporting of “access to transplant and pretransplant outcomes.” SRTR agreed with this feedback and developed a methodology to provide 5-tier assessments for both the waitlist mortality rate (survival on the waiting list) and the transplant rate (getting a deceased-donor transplant faster). These assessments are adjusted for patient characteristics in a similar manner to how posttransplant outcomes are adjusted for patient and donor characteristics. Researchers at the Hennepin Healthcare Research Institute (Principle Investigators: Ajay Israni, MD, MS, and Cory Schaffhausen, PhD), through a grant from the AHRQ, have been conducting patient focus groups and randomized trials of website versions. Through a randomized controlled trial, they found that users more often selected programs with a higher transplant rate when the transplant rate was presented as a graphical icon rather than numerically. When a program did not have the highest 1-year organ survival, but did have the highest transplant rate, 45% of patients chose the program with the highest transplant rate when it was presented as a tier icon; only 26% did so when the numeric transplant rate was shown alongside the 5-tier icon for 1-year organ survival.

Description of the change: Because we added two new 5-tier assessments for pretransplant metrics, we changed the icons slightly to be more compact. In addition, the original version of the website included interpretive text below the 5-tier assessment icons to indicate whether higher numbers where better or worse than lower numbers. The interpretive text consisted of the following statements for tiers 1 through 5, respectively: “Worse than Expected,” “Somewhat Worse than Expected,” “Good (as Expected),” “Somewhat Better than Expected,” and “Better than Expected.”

Response to feedback: SRTR made this change in response to feedback that the interpretive text was inflammatory and did not add clarity; eg, what does ”as expected” mean? Further feedback noted that these statements did not convey the actual differences between the tiers with regard to actual survival percentages (in the case of the first-year outcomes assessments). The updated version of the website removed these statements in favor of a key placed at the top of the search results page that provides the actual numbers expected within each tier for each of the three outcomes presented in the search results, as detailed in change #4 next.

Description of the change: Because we removed the interpretive text below the icons as detailed in change #3 above, we added a key at the top of the search results page. The key begins by simply showing that tier 1 is “worse” and tier 5 is “better.” Without this, readers may not correctly interpret that a tier-5 is better than a tier-1 evaluation on each of the outcomes assessed. Below this simple key is a button labeled “Show National Rates >>”. Clicking on this button brings up a table that shows expected numbers within each of the five tiers for each of the outcomes. These provide context for what each tier means for each outcome and clarifies that tiers reflect a comparison to national outcomes. This also demonstrates that variation in getting a deceased donor transplant faster is generally larger than variation in first-year rates of organ survival. The numbers in this table reflect outcomes for an average-risk patient at an average program within the tier.

Response to feedback: This key was added in response to feedback that the tiers hide the actual numbers behind the evaluations and do not provide context. Additional feedback suggested that getting a transplant faster, ie, higher transplant rates, were often more important to overall patient survival after listing than posttransplant outcomes. The key allows readers to put each metric in context with the other metrics.

Description of the change: The original version of the website presented the unadjusted transplant rate as a number, eg, 17.1 transplants per 100 persons per year. This rate included both living and deceased donor transplants in the numerator for kidney and liver programs. The new 5-tier assessment of getting a deceased donor transplant faster includes only deceased donor transplants in the numerator of the rate, censoring candidate follow-up at the time of living donor transplant.

Response to feedback: This change was made in response to feedback that the “all-donor” transplant rate was misleading, especially for candidates without a living donor. Furthermore, OPTN policy mandates that living donor recipients be added to the deceased donor waiting list prior to undergoing living donor transplant. However, policy does not specify when candidates are added to the waiting list. Therefore, living donor candidates may be added to the waiting list early or late in the process leading up to the living donor transplant. The timing of adding candidates to the deceased donor waiting list when a living donor transplant is intended can bias the transplant rate high or low depending on the program’s practice. The new metric responds to this feedback by constructing the transplant rate evaluation based only on deceased donor transplants, censoring patient follow-up at the time of undergoing living donor transplant.

Description of the change: Because the transplant rate evaluation described in #5 above now describes only the program’s deceased donor transplant rate, we split the program’s transplant volume descriptors into deceased donor and living donor volumes, separately. This allows readers to see which programs perform more or fewer living donor transplants.

Response to feedback: This was done in response to feedback as described in #5 above, and to feedback from patient focus groups indicating that patients want to know which programs perform more or fewer living donor transplants.

Description of the change: An icon () has been added to specific columns indicating which component of care has the greatest impact on overall survival after listing. In searches for heart, kidney, and liver programs, this icon appears on the “getting a transplant faster” column. In searches for a lung program, this icon appears on the “1-year lung survival” column. For more information regarding how these columns were determined, click here.

Response to feedback: This was done in response to feedback that the original version of the site focused too much attention on first-year posttransplant organ survival when pretransplant outcomes may be more important for overall survival when choosing a transplant program, given larger variation in transplant rates across programs. A first step to remedy this was adding 5-tier evaluations for pretransplant outcomes, ie, waitlist mortality and transplant rate. With three evaluations, patients see more information but lack guidance regarding how to use the information. In testing various solutions, this presentation was the only one that succeeded in guiding users toward the columns of most importance to their survival. Without this icon, users often focused on 1-year organ survival as being most important.

Description of the change: The original version of the website sorted search results by the outcome evaluation tier, bringing tier-5 programs to the top of the search results. The updated version sorts initially by the column with the largest impact on patient survival after listing as described in change #7 above. For heart, kidney, and liver programs, the default sort is by the “getting a transplant faster” column. For lung programs, the default sort is by the “1-year lung survival” column. The default sort order for each organ is determined by the following algorithm:

• Heart results:
  • Primary sort: getting a transplant faster tier
  • Secondary sort order: first-year organ survival tier, survival on the waiting list tier, deceased donor volume.
• Kidney results:
  • Primary sort: getting a transplant faster tier
  • Secondary sort order: first-year organ survival tier, deceased donor volume, living donor volume.
• Liver results:
  • Primary sort: getting a transplant faster tier
  • Secondary sort order: first-year organ survival tier, survival on the waiting list tier, deceased donor volume, living donor volume.
• Lung results:
  • Primary sort: first-year organ survival tier
  • Secondary sort order: getting a transplant faster tier, survival on the waiting list tier, deceased donor volume.
• Other organs:
  • Primary sort: deceased donor volume
  • Secondary sort order: getting a transplant faster tier, first-year organ survival tier, survival on the waiting list tier.

Users can re-sort based on their preference by clicking on the column headings. The following algorithm is followed for sorts by a particular column:

• Distance:
  • Secondary sort order: getting a transplant faster tier, first-year organ survival tier, survival on the waiting list tier (if applicable), deceased donor volume, living donor volume (if applicable).
• Deceased donor volume:
  • Secondary sort order: getting a transplant faster tier, first-year organ survival tier, survival on the waiting list tier (if applicable), living donor volume (if applicable).
• Living donor volume (if applicable):
  • Secondary sort order: getting a transplant faster tier, first-year organ survival tier, survival on the waiting list tier (if applicable), deceased donor volume.
• Getting a transplant faster tier:
  • Secondary sort order: first-year graft survival tier, survival on the waiting list tier (if applicable), deceased donor volume, living donor volume.
• Survival on the waiting list tier (if applicable):
  • Secondary sort order: getting a transplant faster tier, first-year organ survival tier, deceased donor volume, living donor volume.
• First-year organ survival tier:
  • Secondary sort order: getting a transplant faster tier, survival on the waiting list tier (if applicable), deceased donor volume, living donor volume.

Response to feedback: This was done in response to feedback that the original version of the site focused too much on posttransplant outcomes when pretransplant outcomes may be most important to overall patient survival after listing. The AHRQ’s Best Practices in Public Reporting recommended rank ordering programs by performance, and the updated version of the site rank orders programs by the measure most important to overall patient survival after listing. For more information on how these columns were determined, click here.

Description of the change: The following statement was added to the site below the table of search results: “Note: Future outcomes may be different from past outcomes. Groups (numbers of bars) do not represent the outcome for each individual.”

Response to feedback: This change was made in response to feedback from transplant professionals requesting a statement that these evaluations are derived based on past experience and may not reflect current or future performance at the program. The AHRQ Best Practices in Public Reporting recommend against adding statements of uncertainty when reporting healthcare evaluations publicly, noting “research shows that consumers tend to discount information when a report communicates that there is uncertainty about the data” (Hibbard J, Sofaer S. Best Practices in Public Reporting No. 1: How to Effectively Present Health Care Performance Data to Consumers. AHRQ Publication No. 10-0082-EF, June 2010). However, Drs. Israni and Schaffhausen’s work with patients did not find a substantial downside to including this statement. SRTR also presented the concept to OPTN’s Patient Affairs Committee, which had no objections and unanimously approved the statement.

During the formal comment period from 5/14/2018-7/13/2018, SRTR received many comments and feedback about the beta site. SRTR extracted themes from the comments and this report contains responses to those themes.

You may continue to provide feedback at any time by contacting us.