By: Fabio Sala, Ph.D.	Download PDF
Publication Date 2002

he recent and widespread interest in the importance of emotional intelligence (EI) at work (Goleman, 1995) has led to the development of programs that are designed to (1) educate people about the relevance of emotional intelligence in the workplace, (2) assess their relative strengths and weaknesses, and (3) provide a framework to develop and enhance their ability to interact with others with greater emotional intelligence (Boyatzis, 1999). The present research will attempt to provide some evidence for the effectiveness of an emotional intelligence training program; specifically, whether participants' scores on a measure of EI improve after exposure to a program designed to increase emotional intelligence at work.

Methods

Samples

Two samples participated in a Mastering Emotional Intelligence (MEI) workshop. Sample 1 consisted of 20 participants who were assessed twice on a measure of emotional intelligence, with 8 months between assessments. Half of these participants (10) were Brazilian managers from a large consumer retail organization, undergoing the training as part of a developmental process. The remaining 10 participants were Brazilian consultants from a global HR consulting firm, attending the same training session to familiarize themselves with the MEI program. Sample 2 consisted of 19 participants from a large U.S. government accounting organization, with 14 months between assessments.

Emotional Intelligence Inventory (ECI)

The ECI is a multi-rater instrument that provides self, manager, direct report, and peer ratings on a series of behavioral indicators of emotional intelligence, based on the emotional competencies identified by Goleman (1998). The ECI encompasses 20 competencies, organized into four clusters (See Table 1): Self-Awareness, Social-Awareness, Self-Management, and Social Skills (Boyatzis, Goleman, Rhee, 1999). Previous research has shown the ECI to have high levels of internal consistency (Boyatzis & Burckle, 1999).

Mastering Emotional Intelligence Program (MEI)

Briefly, the MEI program is a one-year process that helps to better identify and address workplace emotional intelligence issues, and provides support for participants as they work to raise their emotional intelligence competencies.

Participants were first assessed on the ECI (T1) that served to establish baseline levels of emotional intelligence. After T1 ratings were collected, participants then participated in a "Building Awareness" two-day workshop that provided an introduction to EI. Participants met again for another 2-day "Deciding to Change" workshop. During this workshop, participants received individual attention on those EI competencies where their self assessments were dramatically different from "total others" ratings. Participants were encouraged to meet with group members who were committed to helping each other improve EI by providing support and feedback when situations arose that challenged their EI skills. Participants then met again for a 1-day "Practicing and Mastering" workshop that provided further opportunity to work on EI behaviors. Finally, participant's progress was measured by another multi-rater ECI assessment.

Statistics

Because of the nature of EI workshops (small n), differences between T1 and T2 scores were considered meaningful if they showed (1) moderate to high effect sizes, (2) and if differences were statistically significant based on the results of paired-samples t-tests (Nickerson, 2000; Cohen, 1988; Harris & Rosenthal, 1985). Effect size equaled the difference between the groups being compared, divided by the standard deviation of the combined groups. This is a direct measure of the size of the difference between the groups, that is not influenced by sample size (Nickerson, 2000).

Results

Sample 1 (large Brazilian consumer retail organization)

In general, scores were higher upon reassessment, suggesting that EI improved. T2 scores ranged from .02 SDs to .75 SDs higher than T1 scores (See Table 1). On average, T2 scores were .33 SDs higher than T1 scores, which corresponds to approximately an 11% improvement over T1. Participants showed significant improvement on T2 ratings on the following 8 out of 20 EI competencies: Self-Confidence, Organizational Awareness, Service Orientation, Conscientiousness, Adaptability, Initiative, Communication, and Conflict Management. Two others (Empathy and Change Catalyst) showed marginal levels of improvement (See Table 1).

These overall differences were overwhelmingly carried by the Brazilian sub-sample of consumer managers. The average magnitude of change for Brazil managers (.60 SDs) was considerably greater than the change for Brazilian consultants (.22 SDs). Although not discussed in detail here, these findings may provide further support for the effectiveness of the MEI program by providing a quasi-control group. The consultant sub-sample may not have been as sufficiently motivated and committed to change as was the client sub-sample.

Sample 2 (large U.S. government accounting organization)

Results with sample 2 were even more robust and promising than those found with sample 1. Scores were significantly higher upon reassessment on 19 of 20 competencies (see Table 2). Alternatively, T2 scores ranged from .28 SD units to 1.06 SD units higher than T1 scores. On average, T2 scores were .70 SD units higher than T1 scores, which corresponds to approximately a 24% improvement over T1.

Confounding Variables

Information was also collected on sample 2 regarding overlap of raters for T2 to determine whether being rated by the same or different people had an impact on ECI scores. Each participant received a percent overlap score indicating the proportion of raters that rated participants for both assessments. The % overlap scores ranged from 0% to 90%. The mean % overlap was 47% and the median overlap was 55%. To test whether % overlap influenced ECI scores, the following analyses were computed.

Percent of overlap was correlated with all 20 ECI "total others" competency scores for assessment two and no relationships were found. Gap scores were computed (difference between T1 and T2 scores) to determine EI change/improvement. Correlations were computed between all 20 gap scores and % overlap and again, no relationships were found. Also, a median split was computed creating two groups: low overlap group (<50%) and a high overlap group (>50%). Another series of pair-samples t-tests were computed and no differences were found between groups for all 20 competencies. These analyses suggest that multi-rater ECI ratings by the same people or by different people does not bias or influence "total others" scores.

Discussion

Increases in T2 ECI scores with both samples suggest that workshop interventions are effective at improving EI. However, it is important to underscore that these findings, although very promising, are preliminary. Without an adequate control group it is difficult to isolate the impact of the intervention versus that of other variables that may have contributed to the higher scores. For example, Goleman (1998) reported research demonstrating a positive correlation between EI and age. Also, future research will attempt to answer the following question: Does the focused effort on those competencies in participants' developmental action plans show particular improvement? Because each participant targets different competencies for development, powerful evidence for the efficacy of training would result from demonstrating the particular improvement of these targeted EI competencies relative to the level of overall improvement of non-targeted EI competencies.

Future Research Directions

There are several things we can do to improve the quality of research around the ECI. For example, access to larger samples of individuals that have been assessed twice on the ECI, inclusion of a control group to allow for a comparison of improvement, good demographic data (e.g., age, years of service) to control for other confounding variables, and accurate records of specific action plans for each participant would greatly strengthen our position. These steps will allow for firm assertions about our EI measures and programs that are supported by rigorous empirical research.

 

Table 1. Sample 1. Overall differences between assessment one and assessment two on participants' total-other scores (N=20).

Competency	Assessment				Effect Size	t	p
	One		Two
Self Awareness	Mean	SD	Mean	SD
Emotional Self-awareness	2.55	.34	2.62	.31	.22	-1.05	.31
Accurate Self-Assessment	3.42	.32	3.41	.28	.03	0.08	.94
Self-confidence	4.21	.38	4.42	.35	.58	-3.24	.004*
Social Awareness
Empathy	4.62	.70	4.80	.58	.28	-1.46	.16
Organizational Awareness	3.58	.31	3.70	.29	.40	-2.79	.01*
Service Orientation	4.21	.27	4.42	.29	.75	-3.07	.006*
Self-Management
Self-control	2.47	.31	2.52	.37	.15	-0.65	.52
Trustworthiness	2.66	.23	2.69	.21	.14	-0.88	.39
Conscientiousness	2.78	.26	2.90	.16	.57	-4.19	.001*
Adaptability	3.18	.23	3.32	.22	.62	-2.96	.008*
Achievement Orientation	4.28	.65	4.73	.56	.25	-1.23	.23
Initiative	3.25	.32	3.42	.26	.59	-2.36	.03*
Social Skills
Developing Others	3.17	.40	3.18	.46	.02	-0.15	.89
Leadership	3.20	.46	3.27	.39	.17	-0.83	.42
Influence	3.88	.37	3.98	.32	.29	-1.02	.32
Communication	3.19	.35	3.35	.33	.47	-2.24	.04*
Change Catalyst	3.85	.45	4.00	.53	.31	-1.73	.10
Conflict Management	3.06	.35	3.23	.35	.49	-1.91	.07*
Building Bonds	3.46	.36	3.50	.36	.11	-0.73	.47
Teamwork & Collaboration	4.06	.54	4.10	.51	.08	-0.37	.72

*Differences were considered meaningful, or significant, if effect sizes were moderate or large and if paired-samples t-tests statistically significant (p < .05).

 

Table 2. Sample 2. Overall differences between assessment one and assessment two on participants' total-other scores (N=19).

</

Competency	Assessment				Effect Size	t	p
	One		Two
Self Awareness	Mean	SD	Mean	SD
Emotional Self-awareness	2.75	.28	2.87	.18	.55	-2.80	.01*
Accurate Self-Assessment	3.72	.33	3.94	.11	1.01	-3.30	.004*
Self-confidence	4.43	.46	4.70	.31	.69	-3.22	.005*
Social Awareness
Empathy	5.16	.73	5.58	.40	.75	-3.81	.001*
Organizational Awareness	3.79	.30	3.92	.12	.60	-2.27	.04*
Service Orientation	4.65	.44	4.83	.25	.54	-2.52	.02*
Self-Management
Self-control	2.77	.36	2.90	.15	.50	-2.05	.05*
Trustworthiness	2.81	.20	2.92	.01	1.06	-2.74	.01*
Conscientiousness	2.92	.18	2.96	.12	.28	-1.19	.24
Adaptability	3.45	.42	3.71	.27	.75	-2.84	.01*
Achievement Orientation	5.21	.60	5.52	.42	.60	-3.43	.003*
Initiative	3.61	.39	3.78	.24	.55	-2.24	.04*
Social Skills
Developing Others	3.57	.48	3.84	.22	.77	-3.46	.003*
Leadership	3.52	.48	3.84	.20	.97	-4.45	.001*
Influence	3.32	.56	3.61	.35	.63	-2.28	.04*
Communication	3.60	.43	3.81	.21	.65	-2.46	.02*
Change Catalyst	4.39	.50	4.65	.27	.67	-2.86	.01*
Conflict Management	3.59	.42	3.82	.21	.70	-3.33	.004*
Building Bonds	3.65	.38	3.89	.18	.87	-3.35	.004*
Teamwork & Collaboration	4.46	.48	4.80	.28	.89	-4.32	.001*

 

References

Boyatzis, R. E. (1999). Developing emotional intelligence. In Cherniss, C. & Goleman, D. (eds.), Developments in Emotional Intelligence.

Boyatzis, R. & Burckle, M. (1999). Psychometric properties of the ECI. Boston, MA: Hay/McBer Group.

Boyatzis, R. E., Goleman, D., & Rhee, K. (1999). Clustering competence in emotional intelligence: Insights from the Emotional Competence Inventory (ECI). In Bar-on, R. & Parker, J. D. (eds.). Handbook of Emotional Intelligence. San Francisco: Jossey-Bass.

Cohen, J. (1988). Statistical power analysis for the behavior sciences. (2nd ed.). Hillsdale, NJ: Erlbaum.

Goleman, D. (1998). Working with emotional intelligence. New York: Bantam Books.

Grandey, A. A. (2000). Emotional regulation in the workplace: A new way to conceptualize emotional labor. Journal of Occupatoinal Health Psychology, 5 (1), 95-110.

Harris, R. J. & Rosenthal, R. (1985). Mediation of interpersonal expectancy effects: 31 meta-analyses. Psychological Bulletin, 97, 363-386.

Nickerson, R. S. (2000). Null hypothesis significance testing: A review of a old and continuing controversy. Psychological Methods, 5 (2), 241-301.