[author: Lori Tansey Martens]

To build a robust ethical culture, ethics and compliance professionals need effective culture assessments to pinpoint vulnerabilities and proactively address them. The U.S. Department of Justice’s (DOJ) update to the Evaluation of Corporate Compliance Programs further underscored the importance of these assessments, raising questions about frequency, methodology, and efficacy.

Three common pitfalls

Based on the International Business Ethics Institute’s experience with more than 100 companies, culture assessments remain one of the most challenging aspects of ethics and compliance programs. Many organizations fall prey to three common pitfalls that hamstring assessment efficacy:

• Over-relying on employee opinion surveys;

• Emphasizing program elements versus the day-to-day experiences of employees; and

• Failing to collect relevant and actionable data makes interpretation and corrective action particularly challenging.

1. Over-relying on employee opinion surveys

Multiple studies demonstrate that employee surveys do not always accurately indicate employee sentiment. Despite corporate assurances of confidentiality, research shows pervasive distrust among employees concerning the privacy of their responses, particularly on sensitive questions. In fact, 21%^[1] to 47%^[2] of employees admit to deception, stemming both from fears of retribution and a belief that their organizations are not genuinely receptive to negative responses. Additionally, there is a known positive bias in responding to questions worded affirmatively, e.g., “my company is an ethical company,” “my manager acts consistently with our values”—a common survey format.^[3]

Further adding to the challenge is the striking disparity between employees’ self-perceptions of their conduct versus their actual behavior. According to a 2023 Ethisphere report, 93% of employees claim willingness to report observed misconduct; however, the number who actually report after witnessing wrongdoing falls by almost half, to 54%.^[4] Collectively, these findings raise serious doubts about employee surveys as the primary vehicle to capture the true ethical pulse of an organization.

2. Focusing too much on formal program elements

Ethical culture is often likened to an iceberg, with program elements such as codes, value statements, training, and hotlines representing the visible aspects of ethical culture. The true threats, however, reside in the submerged aspects of culture, those hidden from immediate view. These include the unwritten rules governing how business is really conducted, the level of psychological safety, on-the-job training, and role modeling. While challenging, developing metrics to peer below the waterline is possible and vitally important.

3. Not gathering relevant and actionable data

The third common pitfall is gathering information that may be interesting or benchmarkable but is neither useful nor actionable. Consider the question: “I know how to access company policies and the code when needed.” Assuming 75% agree, how do you improve that number? Is the issue generally accessibility or difficulty finding information within the documents?

Similar issues arise with questions such as, “My company’s ethics training and communications are effective.” First, this question combines two separate items: training and communications. If results are lower than desired, which one needs to be addressed? It’s also not clear what “ethics training” means. Is it the annual code of conduct refresher, everyday guidance from a manager, or specialized training for certain functions? The term “effective” is itself ambiguous. Effectiveness should be measured against specific objectives. Can employees remember training objectives months after the fact?

Toward a new paradigm for ethical culture assessments

There are tactics available to enhance survey effectiveness as a barometer of employee sentiment. Moreover, integrating other targeted data-gathering techniques can yield greater insights when assessing culture and targeting vulnerabilities. While professionals must ultimately customize assessments to suit their organization’s specific circumstances, here are several potential strategies and approaches worth considering.

Shift focus to measuring actual behavior

Collect behavioral data that reflects real actions—not just perceptions. Survey questions about actually witnessing misconduct, reporting it, and understanding the reasons behind not reporting will provide more realistic insights into employee conduct. Essential follow-up questions should include to whom employees reported the wrongdoing and their satisfaction using that particular channel.

But don’t stop with survey questions. During investigations, integrate assessments of how many employees likely knew of the misconduct without reporting it. If, for every report, five employees kept quiet, that is real data about employees’ willingness to speak up. If these employees are witnesses or interviewed as part of the investigation, ask about the specific reasons why they failed to speak up. However, companies need to balance the value of capturing this information against disciplinary actions for not reporting, as this will have a chilling impact on honest disclosures.

Another tactic to better gauge actual behavior is to add specific time frames to survey questions. For example, consider the difference between “My team considers our ethical standards when making decisions” to “Within in the last 30 days, my team made decisions based on our ethical standards.”

Be as specific as possible in questions

The desire to cover a wide range of topics within a limited number of questions can lead to over-broad surveys. A common example involves asking about senior leaders, e.g., “My senior leaders demonstrate a commitment to ethical conduct.” But definitions of senior leaders may vary based on the employee’s own level within the organization, making it difficult to interpret results. Another frequent example concerns values, e.g., “My manager acts in accordance with our values.” Assuming your company has multiple values, this question is impossible for an employee to accurately answer or for the results to be truly meaningful. It also assumes the employee has immediate recall of all corporate values, which, based on our experience, is typically rare. Instead, consider asking specifically about the core value most directly related to ethical climate, e.g., “My manager acts in accordance with our core value of integrity.” Another approach is to list the values and ask employees to identify the ones they feel require the most attention.

Build metrics into ethics and compliance interfaces

Instead of asking employees whether the code and policies are useful or accessible, observe their actual behavior. How often do employees access the code and/or company policies? Electronic documents can yield a treasure trove of information, such as how long employees stay on the site, what search terms are used, and the number of clicks required to find information. Consider adding a popup when employees exit asking, “Did you find the information you were looking for?”

Another approach is to gather volunteers in front of keystroke-recording computers. Provide several scenarios and ask employees to resolve them by referencing the code or policies. How do they navigate to the code? What are common search terms for various topics? This will provide a wealth of specific information to enhance the user-friendliness of your code and policies.

Ethics and compliance training also provide a great opportunity to both assess program elements and gauge sentiment. For example, incorporate physically accessing the code and policies into your annual training, which will provide concrete verification that employees can access key documents.

Training is also a great time to ask questions about disconnects between the specific policies covered in the training and real life. Additionally, inquire about future topics the training should feature, e.g., “Based on your experience, which of the following topics would be most useful for future programs?” This not only ensures your future training is relevant but is a nonthreatening method to determine likely areas of greatest concern and potential vulnerabilities.

Harness insights from social media

While not a perfect vehicle, current and former employees may have higher confidence in the anonymity afforded by external websites, thus encouraging more candid communications. Some social media sites, such as Glassdoor, allow you to track culture and values ratings over time. Also, reviewing comments on industry-specific bulletin boards—e.g., Cafepharma, Wall Street Oasis, or Blind—can provide valuable information. Similarly, search Reddit for threads about your organization. While employee sentiment may skew negatively on many sites, tracking the positive-to-negative ratio over time—or compared to competitors—can be a useful metric.

Searching the comments section on these sites using terms like code, ethics, integrity, or policy can yield surprisingly detailed observations and insights. For example, a cursory review of these channels for a major company showed multiple reports of a specific policy violation. In another case, a poster admitted firing a contract employee due to her pregnancy. This approach provides specific, actionable insights, such as communicating that nondiscrimination standards apply equally to contract workers.

Leverage hotline data for deeper insights

Hotline report analysis is a valuable tool, providing insights beyond just the overall percentage of annual contacts and recurring topics. For instance, if over half of your hotline calls are anonymous, it implies a lack of trust and a fear of retaliation. An unusually high percentage of calls related to human resources might signal concerns regarding human resources’ approachability or effectiveness. If your company offers hybrid or remote-working arrangements, it’s advisable to ascertain the reporter’s status during intake to incorporate these variables into the analysis.

For web portal reports, consider a question at the end of the form on whether the reporter was satisfied with the reporting process. Ask what could be improved. To better understand retaliation, follow up with reporters (if identified) at regular intervals to determine their perceptions of retaliation, including the source and the type they may have experienced. Track this over time.

Integrate performance metrics into assessments

Performance management is a significant driver of ethical culture and is an essential component of comprehensive culture assessments. Look closely at recent promotions— especially leadership positions—for trends and commonalities. What “opportunity” areas are overlooked in favor of which strengths or accomplishments? This provides considerable insights into actual corporate values, irrespective of published value statements. Even more detail into leadership issues is possible if your organization conducts 360-degree feedback reviews. By amalgamating this data, remarkably granular insights into vulnerabilities, such as communication problems or the absence of psychological safety, can emerge and serve as valuable guidance for leader-training initiatives.

Conclusion

Properly designed employee surveys can be a useful starting point for assessing ethical culture. However, surveys should be complemented by a sophisticated array of other tools to monitor the organizational landscape accurately. Metrics should be seamlessly integrated into all formal program elements, with discrete data points providing real-time insights into employee behavior. Only by embracing a multifaceted assessment strategy can professionals hope to achieve the strong ethical cultures that their leaders, DOJ, and society at large now demand.

Takeaways

• Employee surveys may be unreliable due to privacy concerns and affirmation bias. Self-perceptions often differ from actual behavior, necessitating more reliable assessment methods.

• Ethical culture extends beyond formal program elements, including hidden aspects like unwritten rules and psychological safety. Comprehensive assessments require metrics for these vital components.

• Collect meaningful and actionable data by focusing on specific behaviors and avoiding overgeneralized survey questions.

• Use diverse data-gathering techniques, such as actual behavior observations, analyzing social media, leveraging hotline data, and integrating performance metrics for deeper insights.

• Integrate metrics into formal program elements, including code and policy interfaces and training programs for real-time insights into employee behavior.

1 Rodd Wagner, “‘Tell Me Lies’ -- The Song One-Third Of Employees Now Hear,” Forbes, updated June 28, 2016, https://www.forbes.com/sites/roddwagner/2016/06/27/tell-me-lies-the-song-one-third-of-employees-now-hear.

2 AllVoices Team, “Impact Of HR On Employee Feedback: Trust, Communication, And Power Dynamics,” AllVoices, March 28, 2022, https://www.allvoices.co/blog/impact-of-hr-on-employee-feedback.

3 Samuel Messick, “The psychology of acquiescence: An interpretation of research evidence,” in Response set in personality assessment, ed. Irwin A. Berg (Chicago: Aldine, 1967): 115–145.

4 Ethisphere, 2023 Ethical Culture Report: Lessons From The Pandemic, accessed June 26, 2024, https://ethisphere.com/2023-culture-report/.

[View source.]