Design, Evaluation and Report of Psychometric Properties. Empathy Scale for Health Workers

Assessment, Performance & Reward

Design, Evaluation and Report of Psychometric Properties

Various personality traits contribute to the uniqueness of a person. However, certain traits are more important when working in certain sectors. With regard to the healthcare sector evidence suggests that empathic behaviour has a positive impact on healthcare outcomes and plays an important role in the everyday working environment of the healthcare professional. Therefore, having psychometrically sound empathy scales is highly important for the assessment of care-workers (Ioannidou & Konstantikaki, 2008; Williams et al., 2013).

Based on different theoretical assumptions various scales are available to measure empathy (Lawrence et al., 2004). However, none seemed to be perfectly suitable for a timely constraint recruiting process for care-workers dealing with elderly people. Either they were too long, too difficultly worded or just inadequately measuring the construct as defined for our purpose.

According to the literature, empathy is of multidimensional nature (Hogan, 1975). Social psychologists have conceptualised empathy as consisting of ‘cognitive empathy’ (i.e., the intellectual apprehension of another’s mental state) and ‘emotional empathy’ (i.e., an emotional response to emotional responses of others) (Lawrence et al., 2004). Davis (1994) states that emotional empathy can be further defined as parallel and reactive. Parallel refers to feeling one’s own emotions match the target emotions whereas reactive empathy refers to the feeling of sympathy or compassion. Moreover, recent research suggests that emotional empathy can be labelled as affective, referring to the ability to think about the contents of other minds and behave accordingly (Lawrence et al., 2004). In turn, emotional empathy is always a two way process that is self- and other-orientated (Gagan, 1983).

The daily work of care-workers consists of dealing with other people. Therefore, the ability to manage the emotional relationship with their patients is a key characteristic of a successful healthcare professional. In this case, possessing emotional empathy is key to increase the quality of the work (Ioannidou & Konstantikaki, 2008). Hence, by following carefully the process outlined by Hinkin (1998) this report deals with the development of a statistically valid emotional empathy scale, consisting of just 10 items (5-point Likert). The final scale provides a short but solid assessment tool to be used in the screening process to assess shortlisted care-worker candidates further (O'Meara & Petzall, 2013).

Item Generation

The first stage of our scale development was the creation of items that will assess emotional empathy. By using the deductive approach already available scales as suggested by the literature and the definition of emotional empathy were central in determining what our items needed to assess, e.g., the ability to share feelings and to tune into another person’s situation. As suggested by Harvey, Billings and Nilan (1985), four items per scale are needed to test the homogeneity of items within each latent construct. It is also known that approximately one half of the created items will be retained for use in the final scales (Hinkin, 1998). Therefore, we created 21 items to ensure at least 10 final items (see appendix 1).

The length and the vocabulary of the items were worded carefully. Ideally, since these were to assess the empathy of care-workers and respective applicants, our items should have been assessed within this target group. However, due to time constraints, a general academic audience was used instead. Nevertheless, based on research (Skills for Care, 2015) our perfect target sample would have consisted of middle-aged women (30-50 years old) with a low to medium level of education, stemming from various ethnics and nationalities. Therefore, we maintained a simple vocabulary and kept the items short sentenced. Moreover, to avoid “double­barreled” items each only looked at one single-issue to avoid confusion between two different constructs. To keep the item creation simple and efficient no reversed- scored items were used.

For the item scaling it was decided to use the 5-point Likert-type scales (strongly disagree, disagree, neither agree nor disagree, agree and strongly agree). Indeed, according to Kerlinger (1986), the Likert scale is the most useful in behavioural research and the most commonly used in survey questionnaire research (Cook et al., 1981). As there were no questions related to frequency, requiring another scale, a constant scale for all items could be used.

Benchmarking & Content Validation

In order to optimise the validity of the items the Big-Five Inventory-10 scale, consisting of 10 items, was included as benchmark scale (see appendix 2). The correlation between the results of both scales allow for the examination of a ‘nomonological network’ identifying traits that are associated with empathy. Considering existing theory, the definition of emotional empathy and its fit with the care-worker job description, it is expected that responses such as ‘generally trusting’ and ‘outgoing/sociable’ (Extraversion/Openness to Experience) will correlate positively with empathy items. Contrastingly, responses as ‘finding fault with others’ and ‘reserved’ (Neuroticism/Introversion) will show a negative correlation (e.g., Ashton et al., 1998; de Barrio et al., 2004; Jolliffe & Farrington, 2006).

To determine the content validity of the 21 items we used the views of subject- matter experts. Ideally, they would have been assessed on their relevance and match with the provided definition by current care-workers. Due to accessibility constraints instead HRMB, WOB and OB students validated all 21 items.

Questionnaire Administration

To measure the construct under examination, the 21 items along with the benchmark scale were administered in an online based questionnaire. Due to time constraints no pre-test was conducted. According to Cohen (1969) data has to be collected from an adequate sample size to appropriately conduct subsequent analyses. Generally, recommendations for item-to-response ratios rank between 1:4 and 1:10 (Hinkin, 1998). Hence, with 21 items at least 84 responses were needed which was with the final 81 respondents nearly attained.

Initial Item Reduction

Once the data has been collected, a factor analysis allows for the reduction of a set of variables of observations which provides evidence of construct validity (Guadagnoli & Velicer, 1998). The number of factors to be retained depends on underlying theory and quantitative results (Hinkin, 1998).

Prior to the factor analysis, a preliminary analysis (see appendix 3) shows the Pearson correlation coefficient between pairs of questions and thereby indicates significant patterns (Kim & Mueller, 1978). The determinant figure (with 0.01 > than the necessary 0.00001), the KMO statistic (with .736 > than the necessary .7) and the significant Bartlett’s test indicated a relationship between the variables and thus allowed to use a factor analysis.

Structural Validity

Generally, the objective of a factor analysis is to identify those items that most clearly represent the underlying construct. Thus, a simple structure is desired which is why only those items clearly loading on a single appropriate factor remain (Hinkin, 1998). According to Ford et al. (1986) a criterion level of .40 appears most suitable for judging factor loadings as meaningful.

So as a next step, the factor extraction was carried out. The Eigenvalues (Kaiser criterion) and a Scree plot test were used to identify the factors (Cattell, 1966). Moreover, to analyse the independence of the scales an orthogonal rotation was conducted. The Scree plot (see appendix 4) and Eigenvalues of greater than 1 suggested that 8 factors could be extracted. However, this is only accurate when there are less than 30 variables (we used 21 items) and the commonalities after extraction are greater than 0.6 (our mean was .656). Consequently, support for extracting 8 factors using the Kaiser criterion was shown.

The Pattern Matrix indicates which items load on which factor and appear to represent a sub-construct. To reduce the items to a final set whilst maintaining the integrity of the scale, the outlined analysis was conducted 5 times until a clear factor structure matrix was obtained. Items were deleted due to multiple factor loadings, a too low score or because they did not appear to measure a valid sub­construct as defined by our emotional empathy definition (Ford et al., 1986; Hinkin, 1998; Kim & Mueller, 1978). In conclusion, 3 factors with 10 items remained (see appendix 6 & 7). The Correlation matrix indicates with .073 an even higher determinant figure (see appendix 5). The final 3 factors ‘personal distress’, ‘perspective taking’, and ‘caring behaviour’ however, match our research findings and show the several sub-constructs of emotional empathy. Supported by the high significance of the scale no further reduction was carried out.

Internal Consistency

Reliability is the accuracy of a measuring instrument and a necessary condition for validity (Kerlinger, 1986). The most common measure used in conjunction with factor analysis is internal consistency reliability using Cronbach’s alpha (Cortina, 1993; Price & Mueller, 1986). Values of a = .70 or above are usually considered as satisfactory (Churchill, 1979; Nunnally, 1978). Hence, a result of a = .731 proves the reliability of the scale. As no item deletion would have improved the reliability

[...]