This week, I am exploring issues related to compliance and ethics training, inspired by an article in the online publication, Slate, entitled “Ethics Trainings Are Even Dumber Than You Think”, by author L.V. Anderson. Today I tackle the issues of effectiveness and evaluation of your compliance training.

While most people tend to overlook the issue of attendance at training, it is an issue that should also be considered. You should determine that all senior management and company Board members have attended Foreign Corrupt Practices Act (FCPA) compliance training. You should review the documentation of attendance and confirm this attendance. Make your department, or group leaders, accountable for the attendance of their direct reports and so on down the chain. Evidence of training is important to create an audit trail for any internal or external assessment or audit of your training program.

One of the key goals of any FCPA compliance program is to train company employees in awareness and understanding of the law; your specific company compliance program; and to create and foster a culture of compliance. The testing and evaluation of your FCPA compliance training program is an important aspect not to overlook. In their book, entitled “Foreign Corrupt Practices Act Compliance Guidebook: Protecting Your Organization from Bribery and Corruption”, Martin T. Biegelman and Daniel R. Biegelman provide some techniques which can be used to evaluate ethics and compliance training.

The authors encourage post-training measurement of employees who participated. A general assessment of those trained on the FCPA and your company’s compliance program is a starting point. They list five possible questions as a starting point for the assessment of the effectiveness of your FCPA compliance training:

1. What does the FCPA stand for?
2. What is a facilitation payment and does the company allow such payments?
3. How do you report compliance violations?
4. What types of improper compliance conduct would require reporting?
5. What is the name of your company’s Chief Compliance Officer?

The authors set out other metrics, which can be used in the post-training evaluation phase. They point to any increase in hotline use; are there more calls into the compliance department requesting assistance or even asking questions about compliance. Is there any decrease in compliance violations or other acts of non-compliance?

What if you want to take you post-training analysis to a higher level and begin to consider the effectiveness through your return on investment (ROI)? Leona Lewis explored this issue recently on her podcast Masters of Disaster, where she interviewed Joel Smith, the founder of Inhouse Owl, a training services provider. He advocates performing an assessment to determine ethics and compliance training ROI to demonstrate that by putting money and resources into training, a compliance professional can not only show the benefits of ethics and compliance training but also understand more about what employees are getting out of training (effectiveness). The goal is to create a measurable system that will identify the benefits of training, such as avoiding a non-compliance event such as a violation of the FCPA. Smith admits that calculating legal ROI is very difficult as ethical and compliance behavior is an end-goal and of itself – not necessarily one that everyone feels should be subject to a ROI calculation.

Smith noted, “it is extremely difficult to isolate the training effect to calculate what costs you avoided due solely to your ethics and compliance training. Although each organization will have a unique ROI measurement due to unique training objectives, it is possible to use a general formula to calculate ethics and compliance training ROI.”

Smith’s model uses four factors to help determine the ROI for your ethics and compliance training, which are: (1) Engagement, (2) Learning, (3) Application and Implementation, and (4) Business Impact. These four factors are answered through posing the following questions.

1. Figure out what you want to measure (i.e. what’s the “benefit”?) Before you ever train an employee, you should have a goal in mind. What actions do you want employees to take? What risks do you want them to avoid? In the FCPA, you want them to avoid ethical and non-compliant actions that would lead to FCPA violations. So your goal is to train employees to follow your Code of Conduct and your compliance program policies and procedures rules so you avoid liability related to actions. Therefore the benefit to calculate for ROI purposes is the total amount saved by the company because employees now understand (due to the training) not to engage in unethical and non-compliant conduct around bribery and corruption.

2. Were employees satisfied with the training? What is their engagement? The next step is to get a sense of whether employees feel that the training you provided is relevant and targeted to their job. If it’s not targeted, employees will likely not be committed to changing risky behavior. Smith believes you can get data on employee engagement through a quick post-training survey. Although this factor does not produce a quantitative number to use in the ROI calculation, it will help you isolate and qualify the training benefit.

3. Did employees actually learn anything? Smith believes that a critical part of any employee training is the assessment. If you want to understand the “benefit” of training employees, you must know whether they actually learned anything during training. You can collect this data in a number of ways, but for compliance training, the best way is to measure pre and post training understanding over time. Basically, each time you train an employee, measure comprehension both before and after training.

4. Are employees applying your training? Smith says that for this point you will need to conduct a survey to determine employee application and their implementation of the training topics. To do so, you must conduct employee surveys to understand whether they ceased engaging in certain risky behaviors or better yet understand how to conduct themselves in certain risky situations. These surveys can provide a good sense of whether the training has been effective.

5. What’s the quantitative business impact of your training? At this point you are ready to determine the numerical business impact of your ethics and compliance training. Smith has an approach he calls the “Best Guess” approach. Smith believes there are two parts to the business impact calculation: (1) the benefit calculation and (2) the isolation calculation. Smith provided five questions he would pose.
   1. How often could a noncompliance event occur?
   2. How much revenue would be involved?
   3. What is the profit margin on the revenue?
   4. What are the other costs?
   5. What are the noncompliance hard costs?

The next step is to isolate the benefits of training so that you properly attribute the ROI to the ethics and compliance training. To make this determination, you need to know at a minimum (1) whether employees understood the training and (2) whether employees are applying the training. This information must be compared with other factors, namely: (1) the effects of any other company initiatives involving anti-corruption, (2) employee attitudes regarding the topic and training, and (3) any business factors such as decreasing/increasing international revenue, macro-economic trends, etc. that may contribute to avoidance of a noncompliance event. From these calculations, you should then apply a percentage of the benefit to the training. Here Smith suggests 25%.

6. ROI: bringing it all together. Now it is time to calculate the ROI. Here I turn to the formula as laid out on Smith’s company website: “Total FCPA Noncompliance Costs Avoided – Total FCPA Training Program Costs  ÷Total FCPA Training Program Costs ($20,000) x 100=ROI”. Smith concludes by noting, “Even though calculating training benefits is often difficult and imprecise, it’s incredibly important to make an attempt to quantify training ROI” to demonstrate not only effectiveness but also “so you can show business people the incredible effect that engaging training can have on the bottom line.”

The importance of determining effectiveness and the evaluation of your ethics and compliance program is becoming something that is emphasized more by the Department of Justice (DOJ). Beginning last fall, we started to hear that the DOJ wants to see the effectiveness of your compliance program. This is something that many Chief Compliance Officers (CCOs) and compliance professionals struggle to determine. Both the simple guidelines suggested by the Biegelmans and the more robust assessment and calculation laid out by Smith provide you with formulae you can use going forward.

[View source.]