Examining EAP Effectiveness in China
By Peizhong Li, Matthew Mollenhauer and Jie Zhang
Employee assistance professionals face a global need for empirical assessment of their impact in the workplace (Sharar, et al, 2013). This need is particularly acute in emerging markets such as China, where employers and employees alike have had limited experience with EAP (Wang, 2005; Zhang, 2006).The study referred to in this article is the first to address this need.
More specifically, this article examines a brief pre- and post-instrument that was used to demonstrate that EAP counseling produced statistically significant improvements in workplace outcomes in a large sample of Chinese employees from both local and multinational companies.
EAPs have only recently been introduced in countries like China, as a result of rapid economic development and integration with the global economy. In fact, international and local EAPs are expanding operations in China’s major economic centers (Shi, 2006; Wang, 2005; Zhang, 2006), while large multinational and state-owned companies lead the way in purchasing services for their employees (Shi, 2006).
In China, as in the U.S., EAPs must make the business case to employers that the EAP’s interventions will have positive influences on workplace effectiveness (e.g. less absenteeism and presenteeism, higher levels of work engagement and lower levels of distress at work). More information on these scales appears later in this article.
Need for Outcome Research in Emerging Markets
As noted, the need for empirical assessment of EAP’s effectiveness is especially pronounced in emerging markets, where employers’ and employees’ exposure to EAP in particular and mental health services in general are relatively recent and not in-depth (Wang, 2005; Zhang, 2006). Research which shows that EAPs are effective in China is important for future development of the field in that country.
However, even in North America employee assistance professionals have noted that the “growth of EAPs has not been accompanied by an increase in rigorous evaluation of these programs” (Balgopal & Patchner, 1988, p. 17). In fact, typical assessments focus on metrics such as utilization rates, client satisfaction and case studies or testimonials – areas that have limited impact when used to demonstrate the business value of EAP to client organizations. In other words, they do not measure the program’s effectiveness in improving workplace outcomes.
To date, much of the evidence for EAP’s efficacy for improving workplace functioning has been presented in non-peer reviewed outlets, such as internal evaluations, conference presentations, and trade magazine articles (Attridge, 2010). The methodological rigor of these studies is weak or unknown (Arthur, 2001; Pompe and Sharar, 2008; Attridge, 2010).
The small number of outcome studies published in peer-reviewed journals that demonstrate positive organizational impact of EAP are not representative of the industry (Attridge, 2010). Moreover, they tend to focus on cases with serious mental health or substance abuse issues.
However, the majority of cases presented to EAPs are less severe and only offer brief counseling; i.e. six sessions or less (Sharar, 2009). It is not clear if the prevalent “brief counseling only” EAP model would yield the same outcomes as older “core technology-type” programs that focused on intensive case management and follow-up with more severe, higher-risk cases.
The cultural-specific nature of existing studies has been another barrier to research on the effectiveness of EAP. Most studies have occurred in North America, where EAP has had a relatively long history. The same findings may not be replicated in countries with different cultures, customs and levels of economic development. As EAPs expand into emerging markets, there is a strong need to understand how well their model works in an environment where the concepts and practices of EAP – or any form of mental health care for that matter – are relatively new. In addition, training and licensing for mental health professionals may not be adequate in other nations, and the prevalent attitudes toward psychological issues and their relationship to the workplace may differ from Western societies (Lim, Michael, Cai & Schock, 2010).
A significant hurdle that has prevented EAPs from producing adequate outcomes centers on how systematic program evaluations may be viewed as intrusive to “help-seeking” employees, as well as increasing costs and workloads for both providers and clients. But practical methods of collecting data using valid though brief instruments can help overcome this barrier.
The Workplace Outcome Suite (WOS) is an instrument specifically designed for quick and reliable assessment of EAP effectiveness (Lennox, Sharar, Schmitz, and Goehner, 2010). It is short, workplace-focused and easy to administer (Lennox, et al, 2010). As mentioned briefly earlier in this article, the WOS contains five scales that measure concepts central to understanding EAP effectiveness:
* Work Engagement;
* Life Satisfaction; and
* Workplace Distress.
With five items in each scale, the full suite has a total of 25 items. Lennox and colleagues have since developed a shorter version of WOS (WOS-Short) with one item on each scale; thus five items in total.
* Absenteeism asks the employee to report the total number of hours missed due to personal problems within the last month. The remaining four items use five-point rating scales for employees to evaluate their functioning in the last month.
* Presenteeism asks the employee to rate the degree in which personal problems keep him/her from concentrating at work. It measures productivity while the employee is at work, but not working to his/her potential due to unresolved personal problems.
* Work Engagement asks the employee to determine the degree to which he/she is “often eager to get to the work site to start the day.” It measures the extent to which the employee is invested in or passionate about his/her job.
* Life Satisfaction asks the employee to gauge the degree to which his/her “life seems to be going very well,” providing a global measure of the impact of work and life issues on one’s sense of well-being.
* Workplace Distress asks the employee to rate the degree to which he/she dreads to go to work.
In this study the Chinese translation of the WOS-Short was used to examine the degree to which EAP counseling improves workplace functioning in China. The hypothesis was that the participants would show better workplace functioning after EAP counseling compared with before the intervention.
Participants consisted of 1,707 employees from 34 local and multinational companies in different regions of China who received EAP counseling between May 2012 and September 2014. These companies represented a variety of industries, ranging from manufacturing to information technologies to banking. They use the same external vendor, a joint-venture between an American EAP and a Chinese partner. The sample included approximately 35% men and just over 65% women, a gender makeup that reflects typical EAP clients in China.
The WOS-Short was administered to employees both before (the pre-test) and after EAP counseling (the post-test). The pre-test was built-in as part of the routine phone intake, before arranging counseling with a staff member or affiliate. The post-test occurred 90 days after counseling was completed, thus allowing the intervention to run its usual course and have potential for sustained impact. Counseling was brief and solution-focused, with an emphasis on being helpful in six or fewer sessions.
Results replicated those from a study in the U.S. that used the full version of WOS (Sharar, et al, 2013), in which participants showed statistically significant improvements on four of the five scales of workplace functioning:
* The proportion of participants who reported not missing any work over the last month increased from 77.09% in the pre-test to 89.68% in the post-test;
* Presenteeism and workplace distress scores decreased by 39.37% and 19.23% respectively; and
* Life Satisfaction increased by 24.51% on average from pre- to post-test.
However, employees did not demonstrate any significant change from pre- to post-test on Work Engagement.
Researchers and client organizations are interested in not only whether an EAP intervention has any effect, but also how large the effect is, which was also reflected in this study. The effect sizes for the pre- and post-test differences were computed using the Cohen (1988) rule of thumb – 0.20, 0.50 and 0.80 – for small, medium and large effects, respectively. The rule of thumb is vital in interpreting results since no literature exists on the typical effect size for EAP intervention.
The EAP intervention produced large and medium effects on Presenteeism (0.90) and Life Satisfaction (0.57) while small effect sizes were observed for Workplace Distress (0.29) and Absenteeism (0.17). Again, no effect was observed on Work Engagement.
The results indicate that EAP counseling produces positive changes to Chinese employees. Moreover, these changes persisted 90 days after the employees received brief counseling, indicating that the effects of EAP intervention were lasting and sustainable. These are encouraging signs for EAPs and client organizations in China.
However, Work Engagement turned out to be an area that seemed resistant to change for both the Chinese workers in this study and in the U.S. There are different interpretations for this stagnation. Perhaps EAP counselors are not adequately equipped to influence the factors that contribute to employees’ enthusiasm for their jobs, such as compensation, work environments, characteristics of their tasks, or their relationships with co-workers and supervisors. Further studies are needed to examine what types of interventions are effective for this particular issue.
The results of this study have implications for using a brief instrument to measure workplace outcomes. It is worth noting that Sharar et al. (2013) obtained the same pattern of results with American employees using the full version of WOS. Moreover, the WOS-Short not only detected statistically significant effects of the brief intervention, but also differentiated various effect sizes in the five scales of workplace outcomes. As a result, there is reason to believe that the WOS-Short can serve as a valid and reliable tool for assessing EAP outcomes.
This study used a single group pre-test-post-test design. Without a matched comparison group, the results cannot claim to show that EAP service caused the improvements. However, revealing that EAP service is consistently associated with improved work effectiveness supports the hypothesis that EAP contributes to work effectiveness.
The study should be replicated among other EAPs and include different types of interventions. Moreover, this type of outcome evaluation should be seen as a routine form of monitoring among employers who purchase EAPs in China and elsewhere.
Peizhong Li, Ph.D., is a senior researcher with Chestnut Global Partners, China. Matthew Mollenhauer, MS, LCPC, is Vice President of Operations, Chestnut Global Partners, Bloomington, Illinois. Jie Zhang, MBA, is the General Manager of Chestnut Global Partners, China.
Attridge, M. (2010). Taking the pareto path to ROI. Journal of Employee Assistance, 40 (3), 12-15.
Arthur, A. (2001). Employee assistance programs: Do they work? EAPA Exchange. July-August, 21-23.
Balgopal, P. and Patchner, M. (1988). Evaluating employee assistance programs: Obstacles, issues, and strategies. Employee Assistance Quarterly, 95-105.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hilllsdale, NJ: Erlbaum.
Lennox, R., Sharar, D., Schmitz, E., and Goehner, D. (2010). Development and validation of the Chestnut Global Partners Workplace Outcome Suite. Journal of Workplace Behavioral Health, 25(2), 107-131.
Lim, S., Lim, B. K. H., Michael, R., Cai, R. and Schock, C. K. (2010). The Trajectory of Counseling in China: Past, Present, and Future Trends. Journal of Counseling & Development, 88, 4-8.
Pompe, J. and Sharar, D. (2008). Preparing for the challenges of research. Journal of Employee Assistance, 38(2), 7-9.
Sharar, D. (2009). General mental health practitioners as EAP network affiliates: Does short-term counseling overlap with general practice psychotherapy? Brief Treatment and Crisis Intervention, 8 (4), 358-369.
Sharar, D. A., Pompe, J., Lennox, R. (2013). Evaluating the Workplace Effects of EAP Counseling. Unpublished Manuscript.
Shi, K. (2006). Yuan Gong Yuan Zhu Ji Hua Zai Zhong Guo De Fa Zhan Yu Si Kao. [Observations on the development of employee assistance program in China.] Xinziben [New Capital. ] 3, 14-17.
Wang, Y. (2005). Yuan Gong Bang Zhu Ji Hua Zai Wo Guo De Fa Zhan He Qian Jing [The current status of future prospects of EAP in China.] Rencaikaifa [Talent Development,] 6, 10-11.
Zhang, X. (2006). Yuan Gong Bang Zhu Ji Hua, Zhong Guo EAP De Li Lun Yu Shi Jian. [Employee assistance programs: Theories and practice of Chinese EAP.] Beijing, China, Publishing House of Chinese Academy of Social Sciences.