|
|
HOME ► National Performance Rankings ► National Rankings for Hospitals ► Data Limitations and Analytic Considerations
Data Limitations and Analytic Considerations
Data source: We computed hospital rankings using publicly reported data downloaded from the CMS Hospital Compare web site (www.hospitalcompare.hhs.gov - last accessed 7/14/20101). This data set contains hospital-specific performance on 25 quality measures2 and 10 HCAHPS measures3 for over 4,400 hospitals nationwide. The data presented on Hospital Compare comes from hospitals that volunteered to submit their data for public reporting. Approximately 98% of eligible hospitals nationwide are participating. Those not participating have no data posted on the website and are not included in the rankings.
Hospital Compare displays data provided by acute care (prospective payment hospitals – PPS) and critical access hospitals (CAHs). Long-term acute and acute rehabilitation hospitals are not eligible to report data.
CAHs that have also agreed to participate in the HQA, do not receive any financial incentive to report their data. These hospitals can elect to submit data for any or all of the measures or can elect to withhold the data from display on Hospital Compare.
| Data limitations: |
- While a great deal of time and study has been devoted developing a valid and reliable public hospital performance reporting system, the system is still relatively new. CMS began reporting the Quality Measures on the Hospital Compare web site April 2005, and began reporting the HCAHPS Measures September 2008. Other national public reporting efforts – within and beyond health care – have experienced a period of instability wherein opportunities for improving data quality were recognized and acted upon.
- The Quality Measures used to compute rankings are drawn from four clinical topic areas – heart attack, heart failure, pneumonia and surgical infection prevention– even within these areas, they reflect only a portion of the patient care experience. Quality of care measures for other conditions are not reported and are not incorporated into the rankings.
- The Quality Measures and HCAHPS data submitted to Hospital Compare are subject to quality oversight activities including auditing procedures and edit checks.
- Because of differences in hospital size, the types of patients they care for, and sampling strategies, the numbers of patients used to compute reported performance rates from the Quality Measures varies by hospital, clinical topic, and measure. This impacts the precision with which performance can be ranked.
- Hospital Compare displays data from discharges that occurred between 9 and 18 months ago. More recent changes in the reliability of hospital performance are not reflected in the rankings.
|
| |
| Analytic considerations: |
- Performance rates and national rankings are calculated separately for the Quality Measures and HCAHPS because they reflect very different dimensions of patient care; this is confirmed by the fact that the correlation between them is very small.
- The Quality Measures reflect processes of care supported by clinical science rather than patient outcomes. For these types of measures, risk adjustment is not considered to be necessary. 100% performance is thought to be achievable for all measures used in the rankings.
- For the Quality Measures the extent to which each indicator contributes to a hospital’s overall ranking is proportional to the number of cases reported for that measure.
- For the HCAHPS all measures contribute equally to a hospital’s overall ranking.
- Our ranking approach attempts to emphasize and anticipate a hospital's ability to improve performance across their system rather than only within focused areas.
- While percentile scores, by definition, always range from 0-100, reported performance on a given indicator may cluster very tightly.
- A single composite performance score or ranking can mask either strong or weak performance in specific indicators.
|
1 This currently includes data from the 10/1/08 to
9/30/09 for the Quality Measures and HCAHPS and 7/1/06 to 6/30/09 for 30-day mortality and readmissions. To obtain a copy of the database, go to http://www.hospitalcompare.hhs.gov/Hospital/Static/Resources-
Links.asp?dest=NAV|Home|Resources|RelatedWebsites#TabTop
click on “Download the Hospital Compare database”.
| 2 These measures are: |
- Heart Attack (Acute Myocardial Infarction or AMI)
- Aspirin at arrival
- Aspirin at discharge
- ACE Inhibitor for Left Ventricular Systolic Dysfunction
- Beta Blocker at discharge
- Fibrinolytic medication within 30 minutes of arrival
- PCI within 90 minutes of arrival
- Smoking cessation advice/counseling
- Heart Failure
- Assessment of Left Ventricular Function
- ACE Inhibitor for Left Ventricular Systolic Dysfunction
- Discharge instructions
- Smoking cessation advice/counseling
- Pneumonia
- Initial Antibiotic Timing
- Appropriate Initial Antibiotic
- Pneumococcal Vaccination
- Influenza vaccination
- Blood culture prior to first dose of antibiotics
- Smoking cessation advice/counseling
- Surgical Care Improvement Project
- Preventative antibiotics one hour before incision
- Preventative antibiotics stopped within 24 hours after surgery
- Appropriate preventative antibiotics
- Cardiac surgery with controlled blood glucose post surgery
- Appropriate hair removal
- Surgery patients on a Beta Blocker prior to arrival who received a Beta Blocker during the perioperative period.
- Treatment to prevent blood clots within 24 hours before or after selected surgeries
- Doctors ordered treatments to prevent blood clots (venous thromboembolism) for certain types of surgeries
|
| Click here for a description of the quality measures for hospitals. |
| 3The HCAHPS Measures and the response levels reported by CMS are: |
- How often did nurses communicate well with patients? (Sometimes or Never; Usually; Always)
- How often did doctors communicate well with patients? (Sometimes or Never; Usually; Always)
- How often did patients receive help quickly from hospital staff? (Sometimes or Never; Usually; Always)
- How often was the patient’s pain well controlled (Sometimes or Never; Usually; Always)
- How often did staff explain about medicines before giving them to patients? (Sometimes or Never; Usually; Always)
- Were patients given information about what to do during their recovery at home (No; Yes)
- How often were the patients’ rooms and bathrooms kept clean? (Sometimes or Never; Usually; Always)
- How often was the area around the patient’s rooms kept quiet at night? (Sometimes or Never; Usually; Always)
- How do patients rate the hospital overall? [on a scale of 1-10: 6 or lower (low); 7 or 8 (medium); 9 or 10 (high)]
- Would patients recommend the hospital to friends or family? (No, Probably; Definitely)
|
Hospital Compare displays data provided by acute care (prospective payment hospitals – PPS) and critical access hospitals (CAHs). Long-term acute and acute rehabilitation hospitals are not eligible to report data.
CAHs that have also agreed to participate in the HQA, do not receive any financial incentive to report their data. These hospitals can elect to submit data for any or all of the measures, and can elect to withhold the data from display on Hospital Compare.
| Data limitations: |
- While a great deal of time and study has been devoted developing a clinically valid and reliable public hospital performance reporting system, the system is still relatively new. The CMS rollout of the Hospital Compare web site was in April 2005. Other national public reporting efforts – within and beyond health care – have experienced a period of instability wherein opportunities for improving data quality were recognized and acted upon.
- The measures used to compute rankings were drawn from four clinical topic areas – heart attack, heart failure, pneumonia, and surgical infection prevention – even within these areas, they reflect only a portion of the patient care experience. Patient satisfaction and quality of care for other conditions are not reported and are not incorporated into the rankings.
- Data submission to Hospital Compare is subject to auditing procedures and edit checks, which assess whether data submitted is consistent with defined parameters for sample size, outliers, and missing data. The data are subject to validation to verify that the data abstracted and reported by the hospitals is consistent and reproducible. The limitations of this process are such that for each quarter of data submitted, a random sample of five medical records across all topics are selected for each hospital, regardless of the number of cases submitted.
- Hospital Compare displays data from discharges that occurred between 9 and 18 months ago. More recent changes in the reliability of hospital performance are not reflected in the rankings.
- Because of differences in hospital size, the types of patients they care for, and sampling strategies, the numbers of patients used to compute reported performance rates varies by hospital, clinical topic, and measure. This impacts the precision with which performance can be ranked.
|
| |
| Analytic considerations: |
- Hospital performance measures reflect processes of care supported by clinical science rather than patient outcomes. For these types of measures, risk adjustment is not considered necessary. 100% performance is thought to be achievable for all measures used in the rankings.
- Our ranking approach attempts to emphasize and anticipate a hospital's ability to improve performance across their system rather than only within focused areas.
- The extent to which each indicator contributes to a hospital’s overall ranking is proportional to the number of cases reported for that measure.
- While percentile scores, by definition, always range from 0-100, reported performance on a given indicator may cluster very tightly.
- A single composite performance score or ranking can mask either strong or weak performance in specific indicators.
|
|
