Advances in ‘Big Data' and Analytics Can Unlock Insights and Drive HR Actions

Stay informed and ready to meet both everyday challenges and long-term planning and policy-making goals, with focused news, practical information, and strategic insights on all HR-related developments.


By Kurt Naasz

April 1 — Advances in “big data” are creating new opportunities for human resources professionals to extract meaningful information from previously untapped sources and make better use of HR analytics in their organizations, Ben Taylor, chief data officer for HireVue, a provider of digital recruiting services, said April 1 at the Human Capital Institute's Human Capital Summit in Orlando, Fla.

For instance, he said, data from Twitter and Instagram could play a role in looking at job candidates for things that might raise concerns about hiring them. “Is that a valid data set? How do you utilize that?” 

Turning to another social media platform, Taylor asked audience members if they thought LinkedIn endorsements could predict competency. “Someone has 99 endorsements—that's a really good score—does that mean anything?” According to Taylor, the answer to that question must be determined by the data. “We can't come up with our own personal ideas, because the data needs to decide,” he said. “That's what HR's been missing out on.”

Computers are fast and effective at sorting through information, but “HR has messy data,” Taylor said. He cited resumes as an example and asked participants to consider a hypothetical search for job candidates who meet certain criteria, such as graduating from a top school with a minimum grade point average of 3.5 and at least one internship. Combing through a large volume of resumes to find individuals meeting those criteria would be difficult and time consuming if it were done manually, Taylor said, and the same would hold true for a search done by computers were it not for the advances that have occurred in recent years.

“The whole thing with big data is it's enabled us to look at all of these different types of data that have never been looked at previously,” Taylor said. The idea of doing resume modeling 10 years ago would have been a very daunting and difficult task, but now it's “kind of child's play for a data services company to do something like that.”

‘Game Changer.'

Cloud computing, which “took off” in 2007, made big data possible by offering access to computer infrastructure that companies could rent and use as needed, providing an alternative to buying and maintaining their own supercomputers, Taylor said.

It was a “game changer” when cloud computing led to the availability of “infrastructure as a service,” he said. Companies no longer have to sink a lot of money into the hardware needed to do things like HR analytics.

Increased computing power, along with new techniques for sifting and sorting information, have addressed the issues of volume and velocity, Taylor said. But there's also a third challenge, which is “variety” or “inconsistency” of data. In that respect, Taylor said, “HR has some of the worst data I've ever seen.”

He again used resumes as an example, noting that job candidates don't always include the same pieces of information. Similar problems arise with other HR data sources, such as employee satisfaction surveys and performance evaluations. “You've got a ton of inconsistency,” Taylor said.

A key distinction, he said, is whether the data are “structured” or “unstructured.” To explain the difference, he compared “structured data” with the information in an Excel spreadsheet, where columns with headers define the various data elements. Examples of unstructured data, meanwhile, would be resumes, videos, images and audio recordings, Taylor explained.

“The way I like to think about it,” he said, “is that unstructured data is data that's waiting to be structured. So somebody—like a data scientist or a third-party vendor—has to take that data and convert it to a tabular format.”

In many cases, HR departments already have applicant tracking systems that take the information in resumes and convert it to a structured, tabular format. “Some of your ATS's are so advanced that they will actually auto-parse that resume for you,” he said, while others will force job candidates to input their information in a formulaic, structured manner.

Data-Driven Decisions

One of the key areas where HR can put data to use is in making better hiring decisions, according to Taylor. “We, as humans, are terrible at screening talent, and we have been for a long time,” he said.

When people are faced with the task of searching through a stack of resumes, a natural approach to speed the process would be to come up with “threshold methods,” but everyone's method would be different. For example, Taylor said someone might decide to weed out all candidates who have a GPA of less than 3.3. However, that threshold number would come from the screener's personal, limited scope rather than real data.

Another threshold method would be to exclude all those candidates who failed to mention the hiring company's name in their cover letters, he said. “How many great candidates are they missing out on, who just don't understand the standards around writing a resume and a cover letter?”

Taylor also said that we tend to hire people who are similar to us. Employers will “hire people based on different traits, similarities, their humor and things that don't map to competencies.” He said that's another reason to consider using algorithms “to add a lot more rigor to how you predict candidate success.”

Big data and HR analytics can produce better results, he said, because “they'll look at things that matter and match them to performance,” while disregarding immaterial factors.

‘The Pothole's Coming.'

Taylor said there are three categories or types of analytics—descriptive, predictive and prescriptive—and he offered an analogy involving a car and a pothole to illustrate how they differ from one another.

He said descriptive analytics are the easiest type. In the car analogy, this would be like driving while looking in the rearview mirror. “I just drove over a pothole. I look in the rearview mirror and, yep, that was a pothole.” This backward-looking examination of data typifies descriptive analytics, he said.

With predictive analytics, Taylor said, the driver of the car would be looking through the windshield while moving toward the pothole. “There's a pothole, there's a pothole, I hit the pothole. I didn't do anything to prevent that, I still hit it.”

Prescriptive analytics, on the other hand, involves forward-looking data, as well as action in response to the data, Taylor said. “The pothole's coming, and using my insight into how this system behaves, I'm going to do something different” to avoid hitting it.

Taylor said many HR professionals use descriptive analytics. But he warned that too much data can be bad, and he quipped about “death by dashboard.”

Moving along the continuum to predictive analytics, he said that's “a little harder now. We're actually going to make a prediction that this candidate is more likely to fit our culture and have these competencies.”

With regard to prescriptive analytics, Taylor said he hasn't seen any instances of this in HR yet. A hypothetical example would be an “attrition model” predicting who will be leaving the company. “I know who I might lose, and I'm going to do something about it.” Based on insights from the model, he said, HR could identify possible steps to take in order to retain valued employees who have been identified as an attrition risk.

While lauding the potential value of big data and HR analytics, Taylor said the “other thing to think about is ‘garbage in, garbage out.' There aren't these methods that are going to save HR and make us better. It all comes down to the data that we're dealing with.”

When extracting information from performance evaluations, for example, two key concerns are data consistency and the accuracy of performance metrics, according to Taylor. He said a “terrible” performance metric would be retention. “It needs to be something that's objective—are they meeting expectations, are they exceeding expectations, are they on a performance improvement plan? You need specifics, because when you're ready to start harnessing some of these different tools, you want to make sure your data is good.”

Next Steps for HR

For those HR departments that decide they want to get into big data and analytics, Taylor said there are four levels of action, which he ranked on a scale of “good, better, best.”

The “best” option, but also the one that's the most expensive and risky, is to hire an HR data scientist. “If you hire a good HR data scientist, they'll hunt through your whole system, and they'll be able to scope out where your biggest wins and successes could come from. Then they can just go down the line to tackle those different objectives,” he said.

Taylor said the next level down is hiring a consultant. “This is actually one I recommend if people are just putting their toes in the water.” The hourly rate for data scientists working as consultants can range from $120 to more than $300 per hour, he noted.

“What a consultant can do is they can come in and they can storyboard and go over some of these projects quickly to give you some validation that there is value here.” HR can then use that information in seeking approval to proceed with the projects, he said.

A third option would be to have an HR analyst on staff, Taylor said. “They're not necessarily data scientists, but they understand the statistics and they understand the subject matter,” he said.

The fourth option, which Taylor called the easiest, is to take advantage of third-party tools offered by vendors. Most vendors are eager to secure new clients, and “it's their job to convince you that there's value, so it's almost no risk for you,” he said.

“A lot of you are in different spaces right now. Some of you may already have an HR analyst on staff. Some of you may even have an HR data scientist,” Taylor said.

He added, however, that hiring a data scientist can be hard to do, especially if the individuals making the hiring decision don't really know what they're looking for. An HR person who interviews data scientists can have difficulty determining who is or isn't a good fit, Taylor said. “This is high risk, because you may commit to hiring an individual and then realize a year later that that was not the best fit and they haven't justified that expense.”

To contact the reporter on this story: Kurt Naasz in Melbourne, Fla., at

To contact the editor responsible for this story: Simon Nadel at