Lori R. Pelletier, PhD, MBA; Sharon A. Johnson, PhD; Edward R. Westrick, MS, MD, PhD; Robert A. Klugman, MD, FACP
The Patient-Centered Medical Home (PCMH) is a comprehensive care delivery approach for providing primary care, which seeks to strengthen the physician-patient relationship and support the coordination of safe, timely and high-quality care. In this paper, we utilize the Pearson correlation coefficients to explore the relationship between PCMH practices focused on access and communication, and patient satisfaction. We use two sources of data: (1) 2008 PPC-PCMH survey results that measure adoption of PCMH structures and (2) patient experience data from Massachusetts Health Quality Partners (MHQP). In our results, the PPC-PCMH structures of access and communication were negatively correlated with the MHQP patient experience measure, an unexpected result, although early in PCMH implementation. Future work using more sophisticated statistical analyses should be pursued to explore this and other potential relationships more deeply. The broader contribution of the study is to explore how the efficacy of the PCMH might be tested by relating measures of process and care delivery structures to patient outcomes.
Effective patient care is one cornerstone of a healthy society; emphasizing preventive primary care and population health can positively affect outcomes (Schultz, 1995; Starfield, Shi & Macinko, 2005; Kaiser Family Foundation, 2009). The patient-centered medical home (PCMH) model is gaining national attention in the U.S. as a critical component in healthcare reform (Kaiser Family Foundation, 2009; Berwick, Nolan, & Whittington, 2008), based in part on demonstration projects that suggest the PCMH can improve quality of care, patient experience, and efficiency (Gilfillan et al., 2010; Reid et al., 2010; Rosenthal, 2008; Solberg, Asche, Fontaine, Flottemesch, & Anderson, 2011). As physician groups and individual physicians strive to deliver care that is based on new patient-centered models of care, feedback from patients is necessary for determining what to improve.
In this study, we examine the relationship between PCMH structures associated with access and communication and patient satisfaction for 16 practices that are part of a larger health system we refer to by the pseudonym “Health Clinic.” We use the results of two practice-level surveys, including the 2008 PPC-PCMH survey, which measures structures and practices associated with the PCMH; and a patient experience survey conducted by Massachusetts Health Quality Partners (MHQP). Exploring relationships between medical home constructs and measures of patient satisfaction is important because a true PCMH is designed to enhance the patient experience, as advocated by organizations that are members of the Patient-Centered Primary Care Collaborative (Patient-Centered Primary Care Collaborative, 2012). Our study also highlights the complexities of such analyses.
The idea of a PCMH was conceptualized by the American Academy of Pediatrics (AAP) in 1967 as a central location for archiving a child’s medical record. In 2006, the American Academy of Family Physicians, the American College of Physicians, American Osteopathic Association and the American Academy of Pediatrics published seven Joint Principles that form the basis of the current PCMH model (Solberg et al., 2011; National Committee for Quality Assurance [NCQA], 2012; Rogers, 2008), which address the Institute of Medicine’s vision of care that is safe, equitable, efficient, effective, patient-centered and timely (IOM, 2001). These principles include an ongoing relationship with a personal physician, who leads a team of individuals that collectively take responsibility for delivering high-quality patient care across all stages of life and across the complex healthcare system. A PCMH also includes enhanced access through systems such as open scheduling and new options for communication between patients and providers, as well as payment models that recognize the added value provided to patients.
The National Committee for Quality Assurance (NCQA) provides a formal PCMH recognition program, which awards Level 1, Level 2 or Level 3 Medical Home Recognition at the practice level. Recognition is based on meeting structural and process standards that are associated with excellent care. The 2008 Physician Practice Connection – Patient-Centered Medical Home (PPC-PCMH) is a self-evaluation survey (subject to audit) used by the NCQA in its evaluation process (NCQA, 2012). The survey was revised in 2011 (NCQA, 2012), based on feedback from early adopters that elements should focus more on patient-centeredness, care coordination, and engagement rather than information technology (Berenson et al., 2008; Nutting, Crabtree, Miller, Stange, Stewart, & Jaen, 2011).
Establishing whether recognition programs and associated care models, at the individual physician or practice level, are correlated with better quality of care is important because the results can help support investments in new models of delivering care. Solberg, Asche, Pawlson, Scholle, and Shih (2008) and Solberg et al. (2009) studied elements of the Chronic Care Model (Wagner, 1998) on diabetes care using Pearson correlations and found significant relationships. The effect of the PCMH on outcomes, including both quality of care and patient experience, is being evaluated in the context of several demonstration projects (Nutting et al., 2011). These projects have shown improvements in outcomes, but the results are based on just a few studies, and quality improvements in some cases are no better than those of non-PCMH practices (Solberg et al., 2011). In this study, we consider a specific aspect of the PCMH, and hypothesize that having and using processes within a practice for patient access and communication is positively correlated with patients’ satisfaction in getting an appointment or information when they need it.
This study used data provided by Health Clinic, a private, non-profit integrated healthcare system in Massachusetts. The study population included 16 Health Clinic practices, representing pediatric primary care, family medicine and general internal medicine.
Health Clinic provided two sources of data for this study, 2008 PPC-PCMH survey results and MHQP patient experience results. Thirty-three practices were surveyed using the 2008 PPC-PCMH survey tool. Twenty-four practices received MHQP patient experience results. The 16 practices included in both sources of data were used as the sample in this study.
The first data source used in our study results from the 2008 PPC-PCMH survey, which was used to assess 33 of Health Clinic’s primary care practices for their level of medical “homeness,” the degree to which a medical home has been implemented, from late 2008 through mid-2009. The surveys were conducted to provide feedback to practices about becoming a recognized medical home, rather than to directly support this study. Health Clinic invited all primary care clinics to participate. The 33 practices that chose to participate represent the diversity of sites at Health Clinic; they include small, medium and large practices; resident and non-resident practices, and rural and urban practices. Health Clinic assigned one surveyor, a nurse practitioner, who met with a team from each practice consisting of at least one physician, nurse, medical assistant, practice manager and nonclinical staff. Assigning one surveyor to administer the survey minimizes bias in how questions are asked and responses are interpreted. However, measurement bias by way of incorrect interpretation of the elements was possible. To minimize this possibility, the surveyor attended webinars and sought feedback from NCQA to clarify how questions should be interpreted.
The second data source provided by Health Clinic is patient experience data extracted from the Massachusetts Health Quality Partners (MHQP) Patient Experience Survey conducted in late 2009. Because the PPC-PCMH survey was carried out in late 2008 and early 2009, the timing of the surveys was appropriate for studying practice structures in place and the corresponding patient satisfaction of such structures. The MHQP Patient Experience Survey is a 50-question tool developed from two nationally recognized, validated surveys (MHQP Patient Experience Survey, 2012), the Ambulatory Care Experiences Survey (ACES) and the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Survey. The ACES survey was developed by MHQP and researchers from Tufts New England Medical Center (ACES, 2012). The CAHPS Survey was developed by the Agency for Healthcare Research and Quality (AHRQ) and endorsed by the National Quality Forum (NQF) (CAHPS, 2012).
Health Clinic regularly uses MHQP results to examine patient satisfaction; the survey was not conducted to support this study specifically. The MHQP surveys a sample of patients from participating healthcare organizations at the practice level. Sample size is determined from reliability targets and non-response rates from previous years. Of the about 4,400 surveys sent to patients by MHQP in 2009 for the 16 clinics included in this study, approximately 1,500 surveys were completed, ranging from 28 to 213 depending on the size of the practice. MHQP includes eight measures within two domains, quality of doctor-patient interaction and organization/structural features of care.
A matrix was created by medical management at Health Clinic to facilitate hypothesis generation regarding predicted correlations between the structures identified by the PPC-PCMH survey and measures in the MHQP patient experience survey. The matrix is shown in Table 1; rows represent elements of the 2008 PPC-PCMH survey while columns represent elements in the MHQP survey. Cells with three-plus signs in Table 1 indicate a strong hypothesized association between the elements. For these cells, scatter plots were generated to explore the relationships between the bivariate variables. The mean, standard deviation and range were calculated for each PPC-PCMH element and each MHQP patient experience measure.
In this study, we focus on one of the relationships identified in Table 2, specifically whether having processes/results within a practice for patient access and communication is related to patients’ assessment that they are able to get an appointment or information when they need it. Table 3 lists the detailed MHQP patient questions for the Organizational Access Measure in the first column, and the PPC-PCMH survey questions for elements 1A (processes) and 1B (results) in the second column. The PPC-PCMH standards for these elements have questions related to providing same-day access, as well as phone responses outside of office hours; the MHQP organizational impact measure asks patients whether they have been able to get timely appointments. A practice that indicates they have a process and are able to provide same-day appointments, resulting in a higher PPC-PCMH score on these elements, might be expected to provide patients with better access (e.g., same-day appointments) and who are thus more likely to express higher satisfaction.
We chose Pearson correlations over other methods of association because they are commonly used with scale data (such as the PPC-PCMH measure) and scatter plots suggested linear associations (Petruccelli et al., 1999). While MHQP patient experience data is continuous (in the interval 0 to 100), the small sample size and ordinal nature of the PPC-PCMH elements data weaken the underlying assumption of normality. Hypothesis testing was then used to determine the significance of the Pearson correlation coefficient with the statistical package SPSS. The test statistic, used to examine the null hypothesis that the Pearson coefficient equals zero, follows a Student’s t-distribution with degrees of freedom, v=n-2 (Petruccelli et al., 1999). We used a significance level α, equal to 0.05, in our tests. The tests are said to give a statistically significant result if the p-value is smaller than 0.05.
Descriptive characteristics for the 16-study practices are reported in Table 3. The number of physicians in each practice ranged from 3 to 26, with a median of 7. Most practices (59%) were general internal medicine and 44% of the practices included full- or part-time residents.
The access and communication measure (1A and 1B) of the PPC-PCMH survey tool had a mean of 3.5 and a standard deviation of 2.7 across the 16 practices, with a range between 0 and 7.8. The organizational access measure of the MHQP Patient Experience survey tool had a mean of 82.8 and a standard deviation of 4.2 across the 16 practices with a range between 75.6 and 89.5.
The scatter plot of the MHQP organizational access measure (100 maximum) and the PPC-PCMH access and communication elements (9 maximum) for each practice (Figure 1) and the calculated Pearson correlation coefficient of -0.72 indicate a strong negative linear association. The hypothesis results indicate that the MHQP Patient Experience scores for organizational access and the PPC-PCMH scores for patient access and communication (Elements 1A and 1B) are negatively correlated with a p-value of 0.002, significant at the 0.05 level. In this study of 16 practices, the correlation of -0.72 was detected with 27% power at the 0.05 significance level.
In this study, we investigated one relationship between the scores received by 16 primary care practices on a patient experience survey conducted by MHQP and the results from internal use of the 2008 PPC-PCMH survey, for one healthcare organization. Reid et al. (2009) found that as the structures of a Medical Home increase, patient satisfaction increases. The PPC-PCMH Recognition program, sponsored by NCQA, also suggests this relationship and uses levels of recognition to indicate the medical “homeness” of a practice. In our study, however, the hypothesized association was found to have a significant negative correlation, contrary to expectation and some recent studies (Reid et al., 2010; Rosenthal, 2008). That is, as the processes and results increase for patient communication and access, patient experience decreases. Nutting et al. (2010) also found a similar decrease in patient ratings of practices. We explore several possible reasons for the negative association, as well as study limitations.
First a potential cause of the unexpected negative correlation is that the processes of patient access (Element 1A) and the processes of the care team (Element 3C) may be newly implemented. The performance of such systems may worsen before getting better, or patient experience might be driven by earlier experiences. The phenomenon of worse-before-better is often seen in IT implementations and process redesign (Nutting et al., 2010; Ashurst, Doherty, & Peppard, 2008). Therefore, sampling patient experience during the transformation stage might bias the population statistics causing sampling errors across practices.
In addition, the negative association could represent either poor performance of the processes assessed or other external factors not examined by this study. Newly hired staff, a redesigned process for triaging, or a new call center layout are examples of issues that can strain a practice, and initially reduce patient satisfaction. In our study, we also used existing data sources, rather than designing instruments to specifically support the study. Elements and categories in the 2008 PPC-PCMH and MHQP are not completely matched. For example, the MQHP access measure includes questions about waiting time during visits, which is not addressed in PPC-PCMH survey questions on access. Even if patients are able to get same-day appointments (good access as defined by PPC-PCMH), they may rate a practice poorly if they have to wait to be seen at the appointment.
The study also has several important limitations. First, the limited number of practices in our sample made it difficult to test relationships for statistical significance. Even with the significant finding in access and communication, a sample size of 30 would be needed to achieve a 0.80 power. The sample size also limited the extent to which the effects of multiple covariates could be examined simultaneously. Larger studies that adjust these associations for multiple confounding variables would be valuable.
Second, the survey tools we used may not be appropriate for assessing the types of processes associated with PCMH principles, or the relationship between patient satisfaction and these elements. The 2008 PPC-PCMH survey, for example, equally weights the 12 items in Element 1A (Table 2), so providing an interactive practice website (item 10) has the same effect on the element score as scheduling each patient with a personal clinician for continuity of care (item 1). Such limitations with the 2008 PPC-PCMH have been noted by others (Solberg et al., 2011; Berenson et al., 2008). In addition, it is possible that the variation in PPC-PCMH scores does not differentiate structural characteristics for each element, and may be too process — rather than results — focused. In comparing practices participating in the PCMH National Demonstration project, practices were able to implement changes most consistent with episodic care, such as same-day appointments, electronic prescribing, and making results accessible to patients. Practices struggled with elements that required more fundamental changes to the care delivery model, such as electronic visits (e-visits), group visits, wellness promotion, proactive population management, and team-based care (Crabtree et al., 2010). The 2011 PCMH survey, which was developed in part to respond to such feedback, might be a better measurement tool (NCQA, 2012). The MHQP survey is also risk-adjusted (MHQP Patient Experience Survey, 2012), and the scores a practice receives account for its unique patient mix. The PPC-PCMH scores are not risk-adjusted, and may reflect the decisions that a practice makes in response to its unique patient mix. For example, a practice with many medically complex patients might triage appointment requests, creating scheduling delays for less complex patients, explaining a lower patient satisfaction.
Because an internal surveyor conducted the PPC-PCMH survey, an important limitation is lack of external validation (through a NCQA audit, for example). The nurse practitioner who conducted the surveys attended NCQA webinars to develop knowledge of the standards. The goal was to provide feedback to practices about the PCMH, rather than to evaluate them. Yet, a study of the Physician Practice Connections Readiness Survey (precursor to PPC-PCMH) reported that overall agreement with the on-site audit ranged from 40.9% to 96.7% among lead physicians and the accuracy of self-reports of practice systems varied by type of system being assessed and by type of respondent (Scholle et al., 2008). Internally, processing errors (e.g., mistakes in mechanical tasks such as arithmetic or data entry) and response errors (e.g., a subject gives an incorrect response and/or surveyor misinterprets) are also possible.
This paper provides an exploratory framework for understanding the relationships between the structures assessed by internal use of 2008 PPC-PCMH tool and MHQP measures of patient experience. We extend the work of Solberg et al. (2011) and Reid et al. (2010) by incorporating the PPC-PCMH as a measure of medical homeness, sharing their hypothesis that positive correlations between structures of a medical home and patient experience should be expected. However, similar to Nutting et al. (2010), we found a significant negative correlation at the 0.05 level between the processes and results of access and communication and the MHQP patient experience measure of organizational access. Healthcare organizations implementing a PCMH might expect immediate across-the-board improvements in their performance measures; our study illustrates some of the difficulties with this assumption.
Future studies could address the study limitations and expand the scope to include other performance measures, such as quality and efficiency, and other measurement tools, such as the 2011 PCMH survey. We used survey tools designed to measure either PCMH structure or patient experience; better tools might be specifically designed to explore their relationship. Much remains to be done to compare results across practices, controlling for critical differences, and to relate them to patient outcomes.
MHQP, Organizational Access: Getting Timely Appointments, Care and Information
PCMH: Access and Communication Processes (1A) and Results (1B)
|For patients, MHQP asked:
In the last 12 months…
|For each practice, PCMH asked:The practice establishes in writing standards and has results for the following processes to support patient access:
Table 2. MHQP and MH Access and Communication Comparison.
|No. of physicians||
|Type of practice|
|Pediatric Primary Care||
Table 3. Description of Participating Practices (n=16).
* One practice has both Internal Medicine and Pediatric Primary Care
Ambulatory Care Experiences Survey (ACES). (2012). Available at: http://18.104.22.168/icrhps/research/thi/aces.asp.
Ashurst, C., Doherty, N. F., & Peppard, J. (2008). Improving the impact of IT development projects: the benefits realization capability model, European Journal of Information Systems, 17(4), 352-370.
Berenson, R. A., Hammons, T., Gans, D. N., Zuckerman, S., Merrell, K., Underwood, W. S., & Williams, A. F. (2008). A House is Not a Home: Keeping Patients at the Center of Practice Redesign, Health Affairs (Project Hope), 27(5), 1219-1230. doi: 10.1377/hlthaff.27.5.1219
Berwick, D.M., Nolan, T.W., & Whittington, J. (2008). The triple aim: care, health, and cost. Health Affairs, 27(3), 759-769.
Consumer Assessment of Healthcare Providers and Systems (CAHPS) Survey. (2012). Available at: https://www.cahps.ahrq.gov/default.asp.
Crabtree, B. F., Nutting, P. A., Miller, W. L., Stange, K. C., Stewart, E. E., & Jaen, C. R. (2010). Summary of the National Demonstration Project and Recommendations for the Patient-Centered Medical Home, Annals of Family Medicine, 8 (suppl 1), 580-590. doi: 10.1370/afm.1107
Gilfillan, R. J., Tomcavage, J., Rosenthal, M. B., Davis, D. E., Graham, J., Roy, J. A., Pierdon, S. B., Bloom Jr, F. J., Graf, T. R., Goldman, R., Weikel, K. M., Hamory, B. H., Paulus, R. A., & Steele Jr., G. D. (2010). Value and the Medical Home: Effects of Transformed Primary Care, The American Journal of Managed Care, 16(8), 607-614.
Institute of Medicine (U.S.). (2001). Committee on Quality of Health Care in America, Crossing the Quality Chasm : A New Health System for the 21st Century, 1st ed., National Academy Press, Washington, D.C.
Kaiser Family Foundation. (2009). Side-by-Side Comparison of Major Health Care Reform Proposals, Focus on Health Reform, October 15, 2009. Available from: www.kff.org.
Massachusetts Health Quality Partners (MHQP) Patient Experience Survey. (2012). Available at: http://www.mhqp.org/quality/pes/pesMASumm.asp?nav=031600.
National Committee for Quality Assurance. (2012). Physician Practice Connections—Patient-Centered Medical Home (PPC-PCMH) Standards. Available at: http://www.ncqa.org.
Nutting, P.A., Crabtree, B.F., Miller, W.L., Stange, K. C., Stewart, E., & Jaén, C. (2011). Transforming physician practices to patient-centered medical homes: lessons from the national demonstration project, Health Affairs, 30, 439-445. doi: 10.1377/hlthaff.2010.0159
Nutting, P.A., Crabtree, B.F., Stewart, E.E., Miller, W.L., Palmer, R.F., Stange, K. C., & Jaén, C.R. (2010). Effect of Facilitation on Practice Outcomes in the National Demonstration Project Model of the Patient-Centered Medical Home, Hannals of Family Medicine, 8(Suppl 1), s33-s44.
Patient-Centered Primary Care Collaborative. (2012). Available at: http://www.pcpcc.net.
Petruccelli, J. D., Nandram, B., & Chen M., (1999). Applied Statistics for Engineers and Scientists, Prentice Hall, Upper Saddle River, N.J.
Reid, R. J., Coleman, K., Johnson, E. A., Fishman, P. A., Hsu, C., Soman, M. P., Trescott, C. E., Erikson, M.,& Larson, E. B. (2010). The Group Health Medical Home at Year Two: Cost Savings, Higher Patient Satisfaction, and Less Burnout for Providers, Health Affairs (Project Hope), 29(5), 835-843.
Reid, R. J., Fishman, P. A., Yu, O., Ross, T. R., Tufano, J. T., Soman, M. P., & Larson, E. B. (2009). Patient-Centered Medical Home Demonstration: A Prospective, Quasi-Experimental, Before and After Evaluation, The American Journal of Managed Care, 15(9), 71-87. doi: 10.1377/hlthaff.2010.0158
Rogers, J. C. (2008). The Patient-Centered Medical Home Movement–Promise and Peril for Family Medicine, Journal of the American Board of Family Medicine, 21(5), 370.
Rosenthal, M. B., de Brantes, F. S., Sinaiko, A. D., Frankel, M., Robbins, R. D., & Young , S. (2008). Bridges to Excellence–Recognizing High-Quality Care: Analysis of Physician Quality and Resource use, American Journal of Managed Care, 14(10), 670.
Rosenthal, T. C. (2008). The Medical Home: Growing Evidence to Support a New Approach to Primary Care, Journal of the American Board of Family Medicine, 2008, 21(5), 427.
Scholle, S. H., Pawlson, L. G., Solberg, L. I., Shih, S. C., Asche, S. E., Chou, A. F., & Thoele, M. J. (2008). Measuring Practice Systems for Chronic Illness Care: Accuracy of Self-Reports from Clinical Personnel, Joint Commission Journal on Quality and Patient Safety / Joint Commission Resources, 34(7), 407-416.
Schultz, D. V. (1995). The Importance of Primary Care Providers in Integrated Systems, Healthcare Financial Management, 49(1), 58.
Solberg, L. I., Asche, S. E., Fontaine P., Flottemesch, T. J., & Anderson L. H. (2011a). Trends in Quality during Medical Home Transformation, Annals of Family Medicine, 9(6), 515-521. doi: 10.1370/afm.1296
Solberg, L.I., Asche, S.E., Fontaine, P., Flottemesch, T.J., Pawlson, L.G., & Scholle, S.H. (2011b). Relationship of clinic medical home scores to quality and patient experience, Journal of Ambulatory Care Management, 34(1), 57-66. doi: 10.1097/JAC.0b013e3181ff6faf
Solberg, L. I., Asche, S. E., Pawlson, L. G., Scholle, S. H., & Shih, S. C. (2008). Practice Systems are Associated with High-Quality Care for Diabetes, The American Journal of Managed Care, 14(2), 85-92.
Solberg, L. I., Asche, S. E., Shortell, S. M., Gillies, R. R., Taylor, N., Pawlson, L. G., Scholle, S. H., & Young, M. R. (2009). Is Integration in Large Medical Groups Associated with Quality?, The American Journal of Managed Care, 15(6), 34-41.
Starfield, B., Shi, L., & Macinko, J. (2005). Contribution of Primary Care to Health Systems and Health, Milbank Quarterly, 83(3), 457-502. doi: 10.1111/j.1468-0009.2005.00409.x
Wagner, E. H. (1998). Chronic Disease Management: What Will it Take to Improve Care for Chronic Illness?, Effective Clinical Practice, 1(1), 2-4.
This research was supported in part by funding from Health Clinic (fictitious name), the organization studied. Any opinions, findings and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of Health Clinic. The authors thank Health Clinic for generously providing us with access to their data and the ability to attend PPC-PCMH data collection sessions, at a time when they were extremely busy. The authors would also like to thank anonymous reviewers who provided many helpful suggestions for improving the manuscript.
About the Authors
Lori R. Pelletier, PhD, MBA, is director of Performance Improvement at UMass Memorial Healthcare in the Center for Innovation and Transformational Change and Assistant Professor at UMass Medical School in the Division of Health Informatics and Implementation Science in the Department of Quantitative Health Sciences. She completed her PhD in Health Systems Engineering at Worcester Polytechnic Institute focusing on design, systems, processes and statistics. Her research focuses on using engineering approaches to improve healthcare delivery.
Sharon A. Johnson, PhD, is an associate professor in the School of Business at Worcester Polytechnic Institute (WPI), and Faculty Director and co-founder of WPI’s Healthcare Delivery Institute. Dr. Johnson received her PhD from Cornell University in Operations Research and Industrial Engineering. Her research focuses on using systems engineering approaches to improve healthcare delivery systems.
Edward R. Westrick, MS, MD, PhD, is vice president of Clinical Informatics at Verisk Health.
Robert A. Klugman, MD, FACP, is an associate professor of Medicine and Quantitative Health Science at the University of Massachusetts Medical School and senior vice president, chief quality officer and medical director of Quality and Patient Safety for UMass Memorial Health Care. Dr. Klugman graduated from the University of Massachusetts Medical School, and completed his training in Internal Medicine at UMass as well. He established a practice in primary care internal medicine in Westborough, MA, where he continues to actively care for patients.