Psychometric Validation of Patient Satisfaction Assessment Tool for Al

Psychometric Validation of Patient Satisfaction Assessment Tool for Al

Introduction

Patient satisfaction is a comprehensive concept of the cumulative effect of multiple interlinked domains, which include (a) healthcare provider’s assessment – Availability of medicine, side effects, (b) cost-effectiveness – out-of-pocket expenditure and affordability of treatment, (c) general health status – frequency of illness and sleep quality, Eating habits, medication adherence, health education and resources, (d) health awareness, health check-ups and preventive health measures, physical activity, stress management and pleasure assessment and (e) daily screen time and overall healthcare experiences. However, factors such as age, income, communication, employment status, gender, and education of the patient can also affect the perception of satisfaction. Therefore, it becomes important to understand the scope of the construct of ‘patient of health1 from the patients’ perspective. Patients carry certain expectations before they visit the hospital, and the resultant satisfaction or dissatisfaction is the outcome of their experience.2 All such information can be utilized effectively to identify barriers, address treatment gaps, enhance patient turnover, and build more sustainable healthcare services.3

Better service qualities are a means to achieve more support, competitive advantage, and long-term profitability for healthcare providers.4 Considering the significance of patient satisfaction in healthcare around a continuous exchange of knowledge, understanding, and cooperation between the patient and healthcare providers, the characteristics of the patient as well as the providers, can be the determinants of affecting this interaction.

India has a sizeable proportion of traditional complementary and alternative healthcare practitioners, collectively termed AYUSH practitioners. AYUSH is an acronym for Ayurveda, Yoga and Naturopathy, Unani, Siddha, and Homeopathy (spelled as homoeopathy in India). Of the estimated 5.76 million health workers in India, 1.16 million are modern medicine practitioners, 0.79 million are AYUSH practitioners and 0.27 are dentists, the remaining being healthcare workers such as pharmacists and nurses.5 The AYUSH practitioners work in both stand-alone and integrated settings. Homeopathy is an acceptable form of treatment being considered a holistic, safe medicine.6 It is also reported that 82.40% (95% confidence interval = 79.23, 85.19) of patients would prefer integrating Homeopathy services along with modern medicine setups.7

Within this multi-system provision of healthcare, apart from the process of defining patient satisfaction and further differentiating its subsets, its relationship to the holistic care provided by the practitioners of traditional and complementary medicine needs to be determined.

The purpose of this study is to develop a validated scale that can measure the quality of care/patient satisfaction from the services provided in independent standalone settings/outreach services in the community, as per our knowledge, no such standardized scale specific to alternative medicine delivery services is not yet available.

Materials and Methods

Study Design

This was a cross-sectional survey involving the development of a patient satisfaction questionnaire. The research team obtained approval from the scientific and ethics committee of the Central Council for Research in Homoeopathy (CCRH), New Delhi. The study was conducted in compliance with the Declaration of Helsinki.8 The development of the Patient Satisfaction Assessment Tool (PSAT) took place according to a previously reported procedure that included item development, validation, and pilot testing.9,10

Literature Review and Item Generation

Following a search of the literature, previously published studies were reviewed, and it was concluded that no survey questionnaire was available to address the objectives of our study. Thus, we screened all the relevant studies and extracted useful information to form the initial face of the questionnaire. The information was mainly from the evaluated dimensions and items from existing inpatient satisfaction questionnaires developed in countries all over the globe.

Of various fields, items were generated for developing our questionnaire, to evaluate the consultation quality, treatment process, and environment of care. Following literature survey and patient interaction, 39 items representing various domains including healthcare provider’s listening to health concerns, knowledge of disease, diagnosis, instructions provided, professionalism, treatment effectiveness, overall care, availability of medicine, side effects, out-of-pocket expenditure, affordability of treatment, financial stress and future treatment alternatives, general health status, frequency of illness and sleep quality, eating habits, medication adherence and health education and resources health awareness, health check-ups and preventive health measures which reflected the construct-concept of the tool were generated.

Face Validity

Face validity was achieved based on the appearance, format, and layout of the questionnaire. It is a subjective assessment of factors such as the relevance, formatting, readability, clarity, and appropriateness of the questionnaire for the intended audience. Two experts with more than 40 years of experience in patient care and healthcare management looked at the items in the questionnaire to support their relevance. Care was taken to include both positive and negatively worded items to avoid the chances of a submissive response by the study participants. The experts also examined the language of the questions, the intent of the questions, and the total number of questions (length of the survey). Once consensus was obtained from the experts, the questionnaire was finalized with 39 questions. Options for the questions were framed in the form of Likert-type items as respondents can choose one option that best aligns with their view. The questionnaire was developed in English language, and the surveyor’s script was developed bi-lingual in English and Hindi.

Psychometric Evaluation

Factor analysis (Extraction method Principal Component Analysis with rotation method varimax with Kaiser normalization) was done for construct validity testing to identify the domains affecting patient satisfaction and to reduce dimensionality. The construct validity determined if the tool measured the concept that it was meant to measure. Importantly, it determined if the measure is appropriately associated with other factors that are not directly included within the tool and the extent of correlation between related measures.

Participants

Participant recruitment took place in the outpatient department (OPD) of Dr. D.P. Rastogi Central Research Institute for Homoeopathy, Noida, Uttar Pradesh, India, a peripheral research center of CCRH, providing homeopathic treatment, with an average footfall of 450 patients per day. The target sample size was 200 participants considering a ratio of 1:5. The sample size was expected to be achieved over a period of maximum 4 days within a span of 10 working days. However, on the last day of data collection, a larger number of patients were enrolled in the study due to increased footfall of patients resulting in a total of 285 participants being interviewed. A team not affiliated with the center and not involved in patient care carried out the pilot survey. Following the participant’s visit with the physician and receipt of the prescribed medication, the survey was administered as an exit interview. The participants were included if they were adults above 18 years of age, coming to the institute for consultation for the second time or more and gave consent for participation. Participants who could not understand English or Hindi and, therefore, could not communicate with the interviewers or understand the questions or were short of time and likely to rush through the exit interview were excluded. Participants were made aware of the fact that the interviewers were free from any judgmental bias and the information provided by them, whether good or bad, will only help to improve the quality of care provided at the facility. They were informed that participant confidentiality will be maintained at the time of compiling the feedback responses.

Data Collection

Participants were provided with information regarding the survey, it was also made sure that the participants knew the interviewers were not from the hospital staff and that the answers provided by them would be used to improve the healthcare facility. Written informed consent was obtained from all participants, ensuring confidentiality and voluntary participation. Pen paper method was used to fill the paper questionnaires, where each question was read out in English or Hindi to the participant and their responses were marked. Data was collected for 4 days in a span of 10 working days in the month of July 2024.

Data Analysis

Statistical analysis was performed using SPSS version 21.0. Descriptive statistics were used to summarize the demographic data. Socio-demographic data included the name of the participant, age, sex, marital status, type of family, and education. Modified Kuppuswamy Scale11 was used which categorizes the occupation of participants in a range of unemployed/unskilled to professional level based on the kind of work they do on a daily basis.

The internal consistency of the questionnaire which reflects the extent of the correlations among the individual items included in the questionnaire was calculated. Factor analysis was performed to elucidate the construct validity, using the principal component analysis method as an extraction method, which creates uncorrelated linear combinations of weighted observed variables and accounts for the maximal amount of variance present in the data. The Varimax method was used as the rotation method. Items poorly associated with all other items were scrutinized if they were contributing to the overall measure of “Patient Satisfaction”, while items that were very highly correlated were retained. Internal consistency was assessed by measuring Cronbach’s alpha measures correlation within items and includes the association among all items within the questionnaire. A significant Bartlett’s test of sphericity was examined for possible inter-correlations of the items which is required for conducting principal component analysis and a determinant of correlation matrix <0.00001 revealed if there is any multi-collinearity.

Results

Out of the data of 285 participants who participated in the survey, 31 participants were excluded from the analysis as they deviated from the requisite inclusion criteria (Figure 1).

Figure 1 Flowchart of the survey.

Demographic data of the study participants are summarized in Table 1, and missing responses were excluded. The majority of participants were female (64%), and mean age of the participants was 37.63 ± 12.96 years (range 18 to 79 years).

Table 1 Demographic Characteristics of the Participants

Sequencing and grouping were done to further modify the questionnaire. Content validity was also supported by expert reviews and patient feedback. Finally, as shown in Table 2, 34 items were kept with an overall variation of 72.25% using factor analysis which is satisfactory.

Table 2 Rotated Component Matrix (39 Questions)

The analysis was conducted four times, initially including all 39 items. Additionally, separate analyses were performed for each of the four domains. A total of five items, pertaining to anxiety assessment, oral health, comparison with other hospitals, cleanliness and ambience, and staff friendliness, were removed during the process. Three items contributed minimally to the variance extraction, showing factor loadings below 0.50, and two items had factor loadings greater than 0.50 on more than one component, indicating potential issues with cross-loading and multicollinearity.

The 9 domains identified were consultation quality, treatment quality, cost-effectiveness, individual health status, self-care, health consciousness, habits, addiction, and lifestyle habits (Table 3).

Table 3 Distribution of Questions Into Various Domains (34 Questions)

Measure of sampling adequacy (KMO = 0.855) and Bartlett’s test of sphericity were found to be significantly satisfactory (Table 4). Overall internal consistency of the final questionnaire reflects good associations among the items with Cronbach’s alpha of 0.78.

Table 4 KMO and Bartlett’s Test

Discussion

Studies identifying satisfaction levels associated with alternative medicine when juxtaposed with modern medicine have not received due attention in India, despite the growing use of alternative medicine. There remains a need for standardized tools to measure patient satisfaction in these settings. This study addresses this gap by developing and pilot-testing a questionnaire, Patient Satisfaction Assessment Tool (PSAT), designed to assess patient satisfaction specifically at a homeopathic care OPD. Based on the initially identified 39 questions, the final questionnaire had 34 items covering 9 domains of Consultation Quality, Treatment quality, Cost-effectiveness, Individual health status, Self-care, Health consciousness, Habits, Addiction, and Lifestyle habits. The internal consistency of the PSAT was found to be satisfactory and so was the sampling adequacy. To our knowledge, this is the only scale to be validated in a homeopathic outpatient department and aims to assess if the conceptual differences in the practice of modern medicine practitioners and that of homeopathic practitioners are perceived differently and affect patient satisfaction.

Future demand for the services in healthcare settings is influenced by how patients assess the process of receiving care as well as the technicalities of it in the outpatient departments. Not surprisingly, it is increasingly important to understand and measure patient satisfaction in varied settings, including stand-alone complementary and alternative care, such as homeopathic clinics and outreach camps and settings providing integrated care. By doing so, healthcare providers can begin to address organizational and service delivery changes that can contribute to patient satisfaction, and potentially improve the health of their surrounding communities at the same time.12

According to the theory of constructed preferences, when people are in a situation that is both complex and unfamiliar, they likely do not have fixed ideas about what is important to them13 advocated that a person’s orientation determines satisfaction; dissatisfaction occurs where there was a mismatch in the relationship between the expectations and experience of the patients.

We see the importance of empirical surveys of patient satisfaction as a way of expressing the preferences of those who are most directly affected by medical care. While clinicians tend to agree that clinical skill, rapport, and health-related communication behaviors constitute key elements of “quality care”, patients view empathy, courtesy, respect, and “enough time” for care encounters as more important than healthcare providers.14 In alternative medicine, including Homeopathy, patient satisfaction can significantly impact the perception of care effectiveness. How the concepts of health and disease and principles of holistic care provided by homeopathic practitioners15 affect patients’ perception of quality care and satisfaction can be adjudged by the development of a questionnaire that assesses care parameters provided in a homeopathic setup. Further studies are, therefore, needed to contribute to our knowledge of the nature of patient satisfaction with health services, as evaluated by members of the population who use Homeopathy in the OPD and outreach services.

Satisfaction scales have been present and modified in the past, such one tool is SERVQUAL “A Multiple-Item Scale for Measuring Consumer Perceptions of Service Quality”, this 22-item instrument (SERVQUAL) was initially used for assessing customer perceptions of service quality in service and retailing organizations.16 It was later modified according to the healthcare assessment. Concerning the treatment conditions and cultural contexts, standard scales have been developed in different countries, in multiple languages, cultural context and care settings. These tools measure the patients’ perceptions and expectations of services in physical or concrete dimensions, reliability, responsiveness, assurance, and empathy. For instance, ServQual was used in Malaysia17 and Iran18 where patient satisfaction survey was used to assess the quality of care in a referral hospital, and then it was used to do a psychometric analysis of diabetes as well as stroke patients from six different European countries.19 Similarly in Spain, the Primary Care Satisfaction Scale (PCSS) of the EUprimecare,20 a cross-sectional survey with an age limit of 18–65 years was developed to assess patient satisfaction with primary care. In China, the In-Patient Satisfaction Questionnaire was formulated to measure the satisfaction of Conscious patients who had stayed in the hospital for over three days. It had four Dimensions – Doctors’ care quality, Nurses’ care quality, Quality of the environment and facilities, and Comprehensive quality.21 A cross-sectional study was conducted in a teaching hospital in Maharashtra,22 on a scale with multiple domains, viz., ten improvement in health, infrastructure, availability of services, services providers, time spent, communication, billing, cleanliness, and confidentiality that whose validity and reliability were not calculated based on statistical parameters.

These tools have been developed from time to assess the conventional mode of treatment but are not used to assess the alternative mode of treatment in healthcare facilities. PSAT examines the general characteristics of homeopathic OPD settings in India. It is in English language with surveyor script being in English and Hindi, which enhances the outreach of the questionnaire to a large population in the country. Though tested in North India, it can be applied to homeopathic OPDs across the country. PSAT, therefore, will help assess the care provided at homeopathic OPDs. The tool can also be adapted for other Ayush settings after pilot testing in different set-ups. The pilot testing of our patient satisfaction survey tool demonstrated its utility in capturing critical aspects of patient experiences in homeopathic care settings. The findings emphasize the importance of addressing operational aspects, such as waiting times and patient education, to enhance overall satisfaction.

Strengths of the study include the development of a tool which provides a framework for continuous quality improvement in alternative medicine OPDs. Firstly, knowing the health status of the patient and their awareness and consciousness about their health provides valuable insights for further health education and resource allocation. Second, the interaction between the patient and the consultant, especially when it comes to attending to the patient, plays one of the most effective roles for enhancing the quality of care and patient satisfaction. Also, patients who are aware of their health and conditions that can affect their health have higher-quality expectations.23 Another strength of our study is that it is designed and tested for homeopathic/alternative medicine settings and outreach camps because less emphasis on the attractiveness of the immediate infrastructure facility is given and more on the communication and rapport built between the patient and the consultant. Factor analysis requires a satisfactory value of the Kaiser–Meyer–Olkin test, which should be more than 0.7 only then the Factor analysis is considered to be applicable. In our findings, we observed a 0.85 value of KMO test and a significant Bartlett’s test.

The study limitation includes validation done in a single homeopathic settings, which may affect the generalizability of the findings independent of specific diseases. This tool provides both options – self-administering and response to a surveyor. Script for the surveyor is needed for smooth application and better results. Bilingual testing has not been done, but it can be performed with the help of a translation tool with ease. The use of the Likert scale provides freedom of a graded response rather than a dichotomous response. Future research should involve multiple alternative medicine settings. This alternative medicine approach is particularly pronounced among the rural part of the Indian subcontinent because the treatment is provided in modest settings with basic facilities and equipment and almost no laboratory findings.

Conclusion

This survey was successfully developed, and the pilot tested a patient satisfaction assessment tool for alternative medicine OPDs. The tool can be effectively utilized to identify strengths and areas for improvement in patient care at a homeopathic setting. Ongoing use of such tools can contribute to enhancing patient-centered care and improving overall satisfaction in alternative medicine settings.

Abbreviations

AYUSH, Ayurveda, Yoga & Naturopathy, Unani, Siddha and Homeopathy; CAM, Complementary and Alternative Medicine; CCRH, Central Council for Research in Homoeopathy; KMO, Kaiser-Meyer-Olkin; OPD, Out Patient Department; PSAT, Patient Satisfaction Assessment Tool.

Acknowledgments

We acknowledge Dr. Anil Khurana, Chairman, National Commission for Homoeopathy, Ministry of Ayush, Government of India, and Dr. Kumar Dhawle, Director Dr. M.L. Dhawle Trust, Palghar, Maharashtra, for their guidance in questionnaire development and reviewing the questionnaire. Dr. Pankhuri Misra assisted in the conduct of survey at Dr. D.P. Rastogi Central Research Institute for Homoeopathy, Noida, Uttar Pradesh, India. Ms. Nisha Devi and Ms. Neetu Tyagi provided secretarial assistance for the project.

Funding

The financial support was provided by the Central Council for Research in Homoeopathy, New Delhi, India, under the Scheduled Caste Sub Plan (SCSP) budget.

Disclosure

The authors report no conflicts of interest in this work.

References

1. WHO. The The World Health Report 1999: Making a Difference. World Health Organization. Available from: Accessed January 10, 2025.

2. Arshad S, Andrabi H, Masooda S, Masooda S. Measuring patients satisfaction: a cross sectional study to improve quality of care at a tertiary care hospital. East Afr J Public Health. 2012;9(1):26–28.

3. Hsieh MO, Kagle JD. Understanding Patient Satisfaction and Dissatisfaction with Health Care. Health Social Work. 1991;16(4):281–290. doi:10.1093/hsw/16.4.281

4. Nadi A, Shojaee J, Abedi G, Siamian H, Abedini E, Rostami F. Patients’ Expectations and Perceptions of Service Quality in the Selected Hospitals. Med Arch. 2016;70(2):135–139. doi:10.5455/medarh.2016.70.135-139

5. Karan A, Negandhi H, Hussain S, et al. Size, composition and distribution of health workforce in India: why, and where to invest? Human Resources for Health. 2021;19(1):39. doi:10.1186/s12960-021-00575-2

6. Furnham A. Attitudes towards homoeopathy in particular and beliefs about complementary medicines in general. Psychol Health Med. 2000;5(3):327–342. doi:10.1080/713690198

7. Manchanda R, Koley M, Saha S, et al. Patients’ Preference for Integrating Homoeopathy Services within the Secondary Health Care Settings in India: the Part 3 (PPIH-3) Study. J Evid Based Complementary Altern Med. 2017;22(2):251–259. doi:10.1177/2156587216650116

8. WMA. The World Medical Association-WMA Declaration of Helsinki – ethical Principles for Medical Research Involving Human Participants. Available from: Accessed January 2, 2025.

9. Chatterjee P, Rebok GW, Dwivedi SN, Kumar DA, Madan R, Dey AB. Development of integrated care tool – BRIEF for screening the unmet psychosociomedical needs of older Indians. Indian J Public Health. 2019;63(1):51–57. doi:10.4103/ijph.IJPH_187_18

10. Wilson A, Hewitt G, Matthews R, Richards SH, Shepperd S. Development and testing of a questionnaire to measure patient satisfaction with intermediate care. Qual Saf Health Care. 2006;15(5):314. doi:10.1136/qshc.2005.016642

11. Modified Kuppuswamy Scale | PSM Made Easy. Available from: Accessed October 16, 2024.

12. Cleary PD, McNeil BJ. Patient satisfaction as an indicator of quality care. Inquiry. 1988;25(1):25–36.

13. Hibbard J, Pawlson LG. Why Not Give Consumers A Framework for Understanding Quality? Joint Commission J Qual Safety. 2004;30(6):347–351. doi:10.1016/S1549-3741(04)30040-7

14. Bortoli A, Daperno M, Kohn A, et al. Patient and physician views on the quality of care in inflammatory bowel disease: results from SOLUTION-1, a prospective IG-IBD study. J Crohn’s Colitis. 2014;8(12):1642–1652. doi:10.1016/j.crohns.2014.07.004

15. Taneja D, Michael J, Nayak C. Research Challenges in Homoeopathy. Homoeopathic Links. 2022;35(01):56–61. doi:10.1055/s-0042-1742590

16. Parasuraman AP, Zeithaml V, Berry L. SERVQUAL A Multiple-item Scale for Measuring Consumer Perceptions of Service Quality. J Retailing. 1988;64:12–40.

17. Butt MM, de Run EC. Private healthcare quality: applying a SERVQUAL model. Int J Health Care Qual Assur. 2010;23(7):658–673. doi:10.1108/09526861011071580

18. Aghamolaei T, Eftekhaari TE, Rafati S, et al. Service quality assessment of a referral hospital in Southern Iran with SERVQUAL technique: patients’ perspective. BMC Health Serv Res. 2014;14(1):322. doi:10.1186/1472-6963-14-322

19. Konerding U, Bowen T, Elkhuizen SG, et al. Development of a universal short patient satisfaction questionnaire on the basis of SERVQUAL: psychometric analyses with data of diabetes and stroke patients from six different European countries. PLoS One. 2019;14(10):e0197924. doi:10.1371/journal.pone.0197924

20. Cimas M, Ayala A, García-Pérez S, Sarria-Santamera A, Forjaz MJ. The patient satisfaction questionnaire of EUprimecare project: measurement properties. Int J Qual Health Care. 2016;28(3):275–280. doi:10.1093/intqhc/mzw024

21. Wei J, Wang XL, Yang HB, Yang TB. Development of an In-Patient Satisfaction Questionnaire for the Chinese Population. PLoS One. 2015;10(12):e0144785. doi:10.1371/journal.pone.0144785

22. Javadekar S, Raje S. Patient satisfaction scale: development of a comprehensive tool for quality and efficacy of services provided. Pravara Medical Review. 2017;9:14–18.

23. Alemu W, Girma E, Mulugeta T. Patient awareness and role in attaining healthcare quality: a qualitative, exploratory study. Int J Africa Nursing Sci. 2021;14:100278. doi:10.1016/j.ijans.2021.100278

link

Leave a Reply

Your email address will not be published. Required fields are marked *