8/23/12:  The results of an SEC employee survey about the SEC's so-called "evidence based" performance management system ("EBP") are in -- and the SEC received a failing grade.

Last year, the Union negotiated an agreement with the SEC concerning a second "pilot run" of the EBP for the 2011 rating year. As part of that agreement, the parties agreed that an independent expert would review the EBP, including by conducting a survey and focus groups at the agency, and report back to the SEC and the Union.  By overwhelming numbers, SEC managers and staff reported that the proposed new system lacks transparency, fairness and credibility. The data generated demonstrates that, as currently configured, the EBP simply is not working for the SEC. Review the survey results here.

The most important survey finding is that the data backs the central concern the Union has raised since SEC management first developed it:  the new system lacks clear performance standards and fails to make meaningful distinctions between performance levels. Only 18.8% of SEC employees responded that they understood what their supervisor expected from them to be rated at the various performance levels. Only 41.4% of supervisors believed they could make clear distinctions in employee performance under the EBP, while almost the same percentage of supervisors (39.2%) said that they could not.  Nearly one of every two supervisors (44.5%) reported that they were not comfortable communicating to the employees they rate the differences between EBP performance levels. This is an important finding insofar as the agency’s interest in developing a performance management system is to develop a clear path to improved performance. 

The lack of meaningful distinctions between performance levels compromises the entire system.  For example, only 27% of bargaining unit employees believed that their ratings accurately reflected their performance. Only 51% of supervisors believed that the “highest performers” received the highest summary rating. And only 30% of employees believe that their rating is an accurate reflection of their performance. 

The survey also revealed structural problems in the system.  For example, only 27.6% agreed that the EBP included measures that can actually be used to track their performance.  And only 32.9% of respondents agreed that they were evaluated on factors that were “most relevant” to their jobs.  And less than half of supervisors agreed that the performance objectives in their work unit were a good reflection of the individual job in their unit. 

The so-called “calibration” process also scored poorly: 35.6% of supervisors disagreed with the final ratings given their employees after the calibration process and 38.7% believed that the calibration process added value to the credibility and fairness of the process. And less than 13% of employees believe that the EBP is "user-friendly and straightforward."

As to whether the EBP is an improvement over past systems, 56.8% of respondents disagreed that they had a better understanding of how they were performing and 57.6% disagreed that under the EBP they had a better understanding of what was expected of them than under past systems.

Finally, the survey reveals that SEC employees lack overall confidence in the system. Only about one in five (21.%) of employees believe that the EBP's 5-level rating scale is "beneficial to the SEC."  Only 31% of supervisors believe that the EBP is a tool that enables them to assist employees in improved job performance. Only 19.3% of employees believe that the EBP is fair, 15.8% believe that it is transparent and 15.4% believe that it is credible. 

The study report made a number of recommendations concerning the EBP, including the need to review the performance measures and objectives to make changes that will ensure that they accurately reflect the work actually performed by SEC employees, to address problems with the "calibration" process utilized by the agency, and to provide sufficient training.