Crossing the second wall of HR

In the last couple of years, more organizations moved from descriptive analytics to advanced analytics (Insight222 People Analytics Trends 2022), crossing the first wall of HR (\"Investing in People\" by Cascio and Boudreau, 2010). Despite the growth in the adoption of advanced analytical methods in HR, most of the advanced analytics work is done on an ad hoc, one-off-project basis. Several years ago, we introduced the second wall of HR. The second wall of HR is about the maturity of deployment related to people analytics outcomes. By crossing this wall, an organization moves from ad hoc research-based insights captured in reports to more continuous personalized insights at scale that create value for employees.

Where HR is used to deploy descriptive analytics via dashboards, we observe that organizations do not (yet) deploy advanced people analytics outcomes at scale. Typical examples of advanced analytical research are based on, e.g., regression analyses that provide insights on the key drivers for, for instance, client satisfaction, sales, or retention. This type of research is highly valuable because of its strategic importance for an organization, and they provide insights for further decision-making. The outcomes of these types of projects are delivered in a one-off report or PowerPoint and communicated to senior management.

Deployed analytics deals with delivering analytical outcomes continuously and at scale in your organization. A specific machine learning example from our practice is using a support vector machine model to classify employee experiences. The precision rate of our classification model improved over time from 80% to above 90%. In other words, in more than 90% of our predictions, we are able to classify what employees are telling us about their experiences correctly. The insights are periodically renewed via a monthly survey and shared at scale in the organization via Power BI dashboards. This example is discussed in more detail in the book \"Excellence in People Analytics\" by Jonathan Ferrar and David Green.

 \"Crossing the second wall is about delivering continuous analytics at scale.\"

A specific case of deployed analytics is personalized predictive modeling, such as vacancy recommendations or personalized career advice to individuals. Machine learning techniques are most suitable to support these personalized, predictive models. We built a minimum viable model for vacancy recommendations within our practice. We used the EMSI framework to instantly create structured individual skill profiles to compare vacancies requirements with employee skills.

We acknowledge that HR is not the first function to cross the second wall. For example, risk and marketing analytics are clearly ahead regarding deploying risk assessment models and personalized marketing campaigns. However, we see value in sharing our experiences within the HR context. We do not imply that our team has already fully crossed the wall. Our considerations are merely food for thought for whoever is in the same process or thinking about moving to personalized analytics at scale within HR. We grouped our experiences while crossing the second wall into four topics.

More technical skills needed

In the context of personalized predictive modeling, machine learning skills (e.g., topic detection, support vector machine models, or tree-based algorithms like XGBoost) need to be even more in the skill toolset of a data scientist than before. An important part of applying machine learning techniques is thinking about the evaluation of a model. When is a model successful? In regression models, the most used performance metric is R-squared. Machine Learning models have a different way of checking performance. You can think of measures like accuracy, precision, XGBoost regressor, or mean squared errors. At the start of a use case, it is important to discuss which performance metric is most suitable for your data. Hence, you know when the model meets the predefined quality requirements.

Additional skills come from the machine learning engineer. This person ensures that the model built by the data scientist can be deployed in production in a sustainable, secure, and effective way. Not specifically related, but very important, when deploying analytics is the role of the data engineer. The data engineer is able to automate the data pipeline. These skills are generally important, for instance, in preparing data pipelines for dashboarding. But in the context of delivering deployed analytics continuously and at scale, this role is even more important. Furthermore, it is necessary to think about data ingestion, processing, and storing and to ensure at the same time that you comply with all the data privacy and security principles.

\"After the data analyst and the data scientist, people analytics needs a machine learning engineer.\"

Another aspect of Machine Learning is explainability. Machine Learning can be experienced as a black box and less transparent. Therefore, it is necessary to invest in the ability to describe an AI model in a way that our employees can understand it. Explainable AI (XAI) is a critical step to ensure that employees and other stakeholders, such as legal and works councils, trust the model before it goes into production.

IT as a partner

More than ever, IT has become an important partner for HR when crossing the second wall. First, HR infrastructure depends on vendor platforms such as SuccessFactors, Workday, or Degreed. With an IT architect, you need to design how the machine learning models fit in this IT infrastructure. For example, you have to determine how the data will be extracted from the different vendor platforms (if needed for your models) and what platform can be used best in the HR IT landscape to visualize the model outcomes to the employee. In other words, should a model on vacancy recommendations be implemented within the internal job portal, the learning system, or the personal intranet page?

 \"IT has become a vital partner in the people analytics process.\"

Secondly, once the model is built, the people analytics team works closely with machine learning and data engineers from IT to ensure a model is implemented in production in a secure, compliant, maintainable, and scalable way. In the exploration phase, our data scientists work on a model until it meets the predefined requirements. After this phase, the model is deployed as an application into production and hosted by IT.

Privacy and ethics

As always, you need to involve legal, compliance, and works councils within a people analytics process. We have spoken extensively with all stakeholders to see what we need to do to feel comfortable crossing the second wall regarding privacy and ethics regulations such as GDPR. Together with representatives from legal, privacy office, and works councils, we wrote an ethics report to extensively describe personalized analytics' vision and boundaries. The core of this report ensures that a) personal analysis must not limit, stigmatize and discriminate against employees, and b) individual results of the analyses are treated confidentially and are only visible to the employee himself. This HR ethics report is published on the intranet with an updated privacy statement, including the use of personalized HR analytics.

 \"We only apply personalized analytics if it benefits the employee.\"

When creating personalized insights for employees, it is important to increase their involvement. The involvement of employees can be maximized by, for instance, focus groups, test panels, or pilots. Employee involvement is particularly needed to identify potential ethical consequences when putting a model into production. Together with employees and experts, you identify these potential unwanted consequences, quantify them and consciously decide on the next steps. Ultimately, we only apply personalized analytics if it benefits the employee.

The People Analytics team as a gatekeeper

An HR IT infrastructure without cloud-based solutions maintained by external vendors is a rarity. More of these cloud-based solutions come with functionality, based on analytics, for example, a chatbot, natural language querying, driver analysis, or recommendations. These functionalities provide opportunities for the organization to profit from the intelligence designed by vendors.

However, there is a flip side. In most cases, the analytical models used by vendors are a black box. An organization needs to create a gatekeeper role to check analytical models created by vendors for biases, potential ethical issues and validate their performance (e.g., accuracy or precision). Model risk classifications are mandatory within our organization when analytical models are built internally. However, organizations should also apply these same rigor standards when dealing with analytical models provided by vendors.

\"Without a clear model strategy and gatekeeper role, an organization might end up with a vendor for every single use-case.\"

Another important role of the gatekeeper is to ensure the re-use of analytical models and to deliberately think about making models internally or buying from or allying with vendors. Without a clear model strategy and gatekeeper role, an organization might end up with a vendor for every single use case. As with your HR IT landscape, an organization should aim to keep its analytical models transparent, reusable, and easy to maintain. In general, this is easier to accomplish with a limited dependency on vendors. Since our people analytics team acts as a gatekeeper, the team is already involved in assessing and validating multiple analytical models of a vendor.

Final words

Many thanks for the continuous hard work of the HR analytics team at ABN AMRO and for continuously pushing to improve.

We hope you appreciate us sharing our experiences. Also, we hope it inspires some of you to seek the opportunities of people analytics to deliver value at scale to your employees. Do not hesitate; cross the second wall!

About Author

I am a true believer in making HR more evidence-based and data-savvy to improve decision-making. I am a senior manager and thought leader with experience in people analytics, continuous listening, organizational design, survey management, employee experience, employee engagement, HR management information, and strategic workforce planning in a global corporate organization. I am a frequent speaker on People/ HR analytics and strategic workforce planning summits and conferences (e.g., Unleash, Tucana, HRCoreLAB, PAFOW). I am also a Ph.D. candidate focusing on the adoption and internalization of workforce analytics.