Selecting an Assessment Technology: Five Business Principles Vital to Your Success

Executive Summary

The goal within most organizations is to hire a happy, productive workforce that
stays on the job longer and produces more. That simple mission is often very
hard to execute without an HR tool that is proven to predict a candidate’s
on-the-job performance and tenure. Volumes of research show that an assessment
technology—when positioned and deployed correctly—will reduce turnover and
improve productivity while creating a reservoir of objective performance data
designed to identify prospective employees who are good fits in specific job
roles.

To fulfill the mission of hiring a productive workforce that stays on the job
longer and produces more, assessment technology has become a mission critical
component for organizations. With the right assessment technology, your company
should have the means to identify, develop, and retain a highly productive
workforce, which is one of the vital ingredients to business success.

I want to share with you lessons I’ve learned over the last decade on how to
most effectively select, deploy, and study the effectiveness of an assessment
technology solution. Equipped with these five principles, you possess the
fundamental components that must be top-of-mind when purchasing an assessment
technology solution.

The Principles

Principle #1: An assessment technology should be…

Proven to predict employee performance.

Assessment technologies are designed to assist organizations in identifying
candidates who will be successful on the job. To determine which assessment can
best meet your organization’s needs, you must be convinced of the system’s
ability to predict performance. From an objective, scientific perspective,
performance predictability of an assessment solution is most often documented
through two concepts: reliability and validity.

Reliability—Only Part of the Equation

I met a good friend of mine at a golf course in West Texas many years ago. Our
plan was to enjoy a round or two and catch up on old times. However, due to a
high volume of golfers waiting in line, the course officials paired us up with
two “local boys” (that’s a Texanism for two grown men you don’t know).

I was the last to tee off after watching my friend and the two local boys really
set the pace by crushing their drives. Embarrassingly, I “topped” the ball,
meaning I barely caught enough of the ball to send it gently skipping down the
middle of the fairway about fifty yards from the tee box.

As golf etiquette would have it, the player furthest from the hole must hit the
next stroke. As I took a couple of practice swings, I noticed the two local boys
waiting in front and just to the right of my position on the fairway.

In a neighborly fashion, I called out, “Hey, you boys might want to move. I have
a nasty slice.” (My ball always curls off to the right.) One of the two
nonchalantly called back, “Aw, don’t worry, you won’t hit us!” Not wanting to
disrupt the flow of the game, I warily continued to line up my shot. I tightened
my grip on the club, took one more practice swing, and then let it rip.

It really was a beautiful shot—featuring my standard beautiful slice in all its
glory. The ball curved so fast I did not have time to yell “fore.” Before I knew
it, the ball whistled straight at the local boys and struck one with a loud
thud! (I suppose he was fortunate—the ball struck that padded area between the
hamstrings and the lower back.) The golfer with the smarting backside shrieked
so loudly that everyone on the course felt his pain.

The ever-present slice in my golf swing provides the perfect illustration of the
concept of reliability in an assessment technology.

In golf, I reliably slice the ball to the right side of the course every time;
you can count on it, and, unfortunately, the local boys did not heed the
warning. To relate this to assessment terms, anytime you assess someone, you
want to receive a reliable result. The reliability of an assessment focuses on
the consistency of the responses, but not the accuracy. In practical terms, an
assessment that asks several similar questions—using slightly different
words—would yield similar answers. Put another way, if a person took an
assessment, then took it again later, the results should be very similar. By
contrast, if you receive a wide variety of responses, you would likely determine
that the measure is not reliable.

The statistical reliability of an assessment is measured in several different
ways. It would take a lengthy white paper to cover this topic to my
satisfaction, but, in simple terms, a rule of thumb for a behavioral assessment
instrument is to achieve reliability of .7 to .8. This range will vary due to
the type of assessment that was used. I would encourage you to not only ask
about the reliability of any assessment technology, but also the background data
that defines how that number was generated.

It is important to remember that reliability is only part of the equation.
Without validity, you will not have a full picture of the assessment’s
effectiveness. For example, to better understand the actual success of my golf
game (or lack thereof), we need to analyze my validity to determine how
accurately I can hit the ball in the hole. (At least I am reliable…one out of
two isn’t bad.)

Validity—Does the Assessment Work?

Validity answers a very different question. Does it work? In the game of golf,
the number of strokes to complete a round of golf provides a validity estimate
of a player’s golfing abilities. It is important to understand that one round of
golf at one golf course does not provide an accurate representation of one’s
golfing ability. Golfers attain different scores depending on the course played,
weather, type of course, difficulty of the course, the number of holes played,
the number of strokes required to make par, etc. It is not one round, but the
body of evidence collected over time that provides the validity of a player’s
golf game.

This concept translates nicely to assessment validity. When evaluating the
validity of an assessment technology, you should focus your evaluation efforts
on the volume of studies, types of roles, and the sample sizes of the various
studies. Generally, assessments should deliver a validity coefficient in the
neighborhood of .2 to .4. Like reliability, but even more so, the range of the
validity coefficient may vary due to the context of the study, sample sizes,
length of study, etc. Dig into the reported validity coefficient as well as the
supporting documentation that details the study process.

Collectively, discussions around reliability and validity should provide you
with the confidence you need to narrow the choices of possible assessment
technologies for your organization.

Principle #2: An assessment technology should be…

The catalyst to continuous workforce improvement.

To stay competitive, every company should desire to see continuous improvement
in the workforce. The advantages that an organization gains through the pursuit
of continuous improvement are numerous: more productive workers, better process
efficiencies, lower overall expenses, and higher revenues, to name a few. The
key to that kind of long-lasting improvement lies in bettering the performance
of every member of the organization. After all, individuals make up teams, teams
make up departments, departments comprise company divisions, and divisions form
corporations. Individual performers are the building blocks of the entire
structure.

Often the key role that individual performers play in creating a culture of
continuous improvement is overlooked. Traditionally, companies are very good at
monitoring and tracking performance of the masses at the company, regional, and
group levels. However, those same organizations often miss the mark when it
comes to tracking and monitoring performance at the individual level. Without
solid tracking of individual job performance, companies are unable to evaluate
performance on the front lines where it actually occurs: at the individual
level.

As part of your evaluation of assessment technologies, look for processes that
rely heavily, if not solely, on objective performance metrics to document the
effectiveness of individuals in the workforce. Individual performance numbers
will not only define “success” in your company and culture, but also serve to
link behaviors to performance when a behavioral assessment tool is introduced
into the hiring procedure.

This is how your assessment technology can become the catalyst for continuous
workforce improvement. If positioned properly, the assessment software will be a
crucial collection point of individual behaviors—and related performance
metrics—that dictate what great performers look like in specific jobs.

To derive the best results from an assessment technology, it is important to
understand performance in terms of data at the individual level. Understanding
individual performance will provide you with a clear performance picture
surrounding the objectives and desired outcomes for a position. The clearer the
performance picture, the more equipped you are to accurately capture the
behaviors and skills needed for success.

By installing an assessment technology, your organization’s maintenance will
include reevaluating the clarity of performance data on a continual basis in
order to improve the behavioral/skill capture. In this process, it is
commonplace for companies to focus on higher quality individual performance
metrics to better leverage their assessment technology. This effect will
automatically raise the bar in terms of selection, training, development, and
employee productivity across any position where an assessment technology is
deployed.

In summary, focusing on detailed, objective performance data collection methods
will inevitably lead to a better capture of behaviors and skills. A better data
capture through an assessment technology leads to the accumulation of workers
who are more aligned with desired business performance goals. Eventually, one
component improves the other, fueling an ongoing cycle of continuous
improvement.

Principle #3: An assessment technology should be…

Focused on fit; more is not always best.

Have you heard the saying, “More is better”? In the game of golf, you have a
variety of golf clubs designed for different situations. Some clubs are for
driving the ball great distances down the fairway, while other clubs are used
for shorter shots such as chipping or putting. Imagine how your golf game would
suffer if you believed that the bigger club was always better. On a par three
hole, you may overshoot the green with one swing. Even worse, once you make it
to the green, you will struggle putting the ball in the hole using your driver.
At that point, the bigger club actually hurts your ability to maneuver the ball
where you want it to go, which is in the hole. By that logic, more is not always
better.

The same concept applies when it comes to using an assessment. Typically,
assessments measure a collection of characteristics (referred to as factors,
dimensions, etc.). Many people assume—incorrectly— that it is always better to
be on the higher side of a characteristic (the More is Better Syndrome).

Let’s consider the implications of this thought process. Is being smarter always
better? What about filling a mundane job vacancy? How long would a brilliant
person stay in a non-thinking, repetitive job? Is being highly sociable a great
characteristic for every job? Consider an isolated role where interaction with
others is detrimental to good performance. Would a person who thrives on
socializing enjoy, or be driven to success, in this type of role?

Of course, I’m exaggerating these scenarios to drive home the point: it is
important to avoid the mistake of assuming more is always better. The key to
fully utilizing the power of the assessment is to find just the right amount of
many characteristics to predict future success in a specific role.
By fine-tuning the subtle shades of each assessment characteristic to best
describe your strongest performers, you will be better equipped to maximize the
predictive power of your assessment tool. Again, great caution should be taken
if your objective is to only use assessment characteristics in the context of
“more is better.” That method of evaluation often leads to selection tactics
based on incorrect assumptions. Additionally, you will effectively dismiss a
large amount of hidden insight that will increase your predictive power to
identify your future top performers who will stay in position longer.

Keep in mind that most assessment technologies are built according to the
assumption that more is better. Your evaluation of assessment technologies
should only include systems that measure a large group of behavioral
characteristics; moreover, the system must offer flexibility in specifying the
optimal amount of each characteristic an ideal candidate would possess to
succeed in the target job.

Principle #4: An assessment technology should be…

More than just a score.

When selecting an assessment technology, it is important that the usefulness of
the assessment goes far beyond a simple score or rating of the candidate.
Overall scores are helpful when sorting and sifting candidates and narrowing the
field, but the real value comes when you dig deeper and fully leverage all the
rich information gathered from the assessment. Specifically, you should be able
to apply the assessment information to areas such as enhancing the interview,
on-boarding, determining future career paths, and developing employees over the
long term.

Enhanced Interviews

Beyond providing a score, information gained from the assessment should
improve your interview process. A quality assessment can effectively produce
targeted interview questions designed to facilitate discussion around the
specifics of a position. These targeted interview questions also provide a means
to ensure consistency in your interviewing process regardless of the size or
geography of your organization. Additionally, by using the targeted interview
questions, you will maximize your time with the candidate. At a minimum, you
will have a better understanding of the strengths and opportunities revealed by
the assessment in relation to a specific position.

On-Boarding

On-boarding is the process of getting a new hire officially authorized for his
or her first day on the job. This hiring phase includes the completion of
various governmental and proprietary forms, plus any other paperwork required by
the hiring company. To expedite this procedure, an assessment technology will
typically be integrated with the company’s Human Resource Information System (HRIS)
to pass on all relevant data previously collected on the candidate. In essence,
the assessment platform should “fill in the blanks” required on electronic forms
in the HRIS database through a transfer of information from the candidate’s
original application. Without this integration (more on integrations in the next
section), on-boarding remains a manual process and any potential efficiencies
that could be driven from the assessment technology are negated. Direct your
evaluation of assessment technologies to only those systems with proven
integration success with common HRIS technologies.

Career Pathing

Future career paths are another area where an assessment technology should allow
you to go beyond a score. In companies with an eye to the future, the selection
strategy is to hire not only for the immediate need, but also determine each
employee’s viability for future positions. For example, if you are tasked with
hiring an assistant manager, you may also be interested in a candidate’s
potential to be a manager at some point down the road. Your assessment
technology should provide you with the insight to understand and evaluate the
potential for candidates to move into other positions, and not just the job for
which they applied.

Employee Coaching and Development

Companies are often asked to do more work with fewer people on the payroll.
Therefore, coaching and employee development programs have become an area of
emphasis in most organizations. Consider future coaching tools as an integral
part of the assessment technology purchase. The assessment process captures a
wealth of data, which should be used throughout the life cycle of an employee.
By scientifically examining the relationships between performance data and
assessment characteristic scores, the assessment technology provides specific,
detailed developmental targets to support continued growth of the assessed
individual.

One of the biggest hindrances to creating a quality coaching and development
program is finding specific content statistically related to performance on the
job. Assessment technology provides the perfect vehicle to supply accurate,
job-related content for training in the current position, as well as in future
positions.

Principle #5: An assessment technology should be…

A tool that makes your organization better.

Although this principle serves as number five, it fits the old adage, “Last but
not least.” Central to any new purchase or program decision is the need to
determine how your organization will ultimately define value. A great approach
to this question is to ask, “How will this assessment technology make us
better?” You will find that value comes in many forms; each organization has a
unique focus that is proven to breed success. Three universal ways in which an
assessment technology can better an organization are:

• Better processes.
• Better retention.
• Better performance.

Better Process

The primary function of an assessment technology is to address the fundamental
challenge of indentifying candidates who produce more and stay longer on the
job. In fulfilling that primary function, your assessment technology should not
hinder your overall HR process, but in fact should streamline the hiring
workflow. This is most often accomplished through integrations with existing
software systems designed to manage the flow of information as candidates move
from their initial applications to their first day on the job.

The advent of applicant tracking software (ATS) allowed companies to manage the
data generated during the hiring process. ATS tools—not to be confused with
assessment technology—were designed only to collect, organize, and move
candidates through the HR process. In other words, they simply manage bits of
information. Some applicant tracking tools provide a few features such as
pre-screens or light assessment functionality, but the central focus is on
organizing information. These features are handy, but secondary, to the primary
objective of hiring the right fit for the job.

To enjoy the functionality of assessment technology and an ATS, one business
option is to select an assessment technology that can co-exist side by side with
an ATS. However, this arrangement isn’t a requirement. Quality assessment
technology now provides features to categorize and sort people, collect résumés,
store applications, provide detailed reports, and do many other practical tasks
to manage your peopleflow—the path every candidate takes from the “Apply Now”
portal to the final hire/no hire decision. The focus must always be on selecting
the right candidate for the job, but be aware that an assessment technology may
build in enough information management features to ensure that your hiring
process is smooth, user friendly, and meets your peopleflow needs.

Assessment + ATS = Integration

If your organization has determined to use, or is currently using, an applicant
tracking software, then you want to make sure that the assessment technology has
the ability to integrate with that specific ATS. Integration is defined as the
process of connecting two or more technology solutions together to create a
seamless flow of information from one system to another. The seamless flow
should be present for both the applicant and the end-user. The objective of an
integration is to simplify and streamline the data collection and delivery
process.

Integrations are common in the marketplace today. Many systems such as tax
credit, background checks, performance management, applicant tracking, and
payroll or human resource information systems (HRIS) are connected through a
seamless integration. You should expect an assessment technology to provide you
with a history of integrations and examples of current clients already using the
assessment technology integrated with another ATS or HRIS.

Better Retention

A business objective that is directly addressed by an effective assessment
technology solution is improving employee retention. Excessive employee turnover
effects all organizations in the form of both direct and indirect costs. Direct
costs include the placement of job postings, plus the labor hours devoted to
screening and interviewing candidates. There are many indirect costs to consider
as well. A few examples are down time in the vacant position, lost
opportunities, overtime expenses for others to cover job vacancies, not to
mention the potential negative effect on company morale.

Regardless of your current retention issues, the stakes are high and worthy of
careful consideration. Cash America, an international financial services company
that studied its hire-termination trends over a two-year period, conservatively
calculated the direct and indirect costs for replacing a store manager at
$10,000 each, and around $2,500 for each customer service representative.
Whether your numbers are higher or lower, it’s readily apparent that for a
company with thousands of employees, significant reductions in employee turnover
equates to millions of dollars saved over time.

A common thread among much of the existing employment research is the fact that
candidates who are good behavioral fits to their particular jobs tend to stay
longer and turnover less frequently. It is important to recognize that employee
retention is a strong indicator of an improvement effect from an assessment
technology. Most companies keep detailed records of terminations for payroll
purposes, which makes good business sense. No company would willingly continue
to pay an individual who is no longer employed. These records may provide
important data for a quality hire-termination study. For example, as part of the
aforementioned Cash America study consisting of data on 3,248 employees, the
hire-termination data documented that the company experienced a 43% turnover
reduction in managerial positions after implementing an assessment technology.

Keep in mind that obtaining study-worthy results for all positions in the
organization simply may not be possible. Expectations for turnover studies
should be appropriate to the scope of the position. Roles with small
populations, lack of accurate hire and termination data, or an insufficient
amount of time for data collection can affect your ability to conduct a quality
study.

Better Performance

I have never met an executive who did not measure success in terms of
performance. Companies may evaluate performance in many different ways, but one
business rule is undeniable—improved performance comes from improving your
incumbents and selecting better people. Because so many companies desire to
improve their workforce, assessments are a great way to drive improvement. An
assessment technology modeled after actual performance data provides a strong
tool to select those who have the greatest potential to perform well in the
role.

When evaluating an assessment technology, a very common question is often posed
by company executives, included in requests for proposals (RFPs), and/or
submitted by committees: “What is your validity coefficient?” By latching on to
this statistical term, the organization is really asking, “Does it work?” Or,
“Can you prove it has made other companies better in target positions?” Let’s
take a moment to dissect the meaning of this question.

As we touched on in Principle #1, it is important to interpret any answer to the
validity question in the context of the particular situation. Remember my golf
game. If you ask me what I shoot, like any self-respecting person I am going to
tell you my best score. You might think I am a decent golfer based on that one
score. What I conveniently neglected to tell you was the situation surrounding
that score. I left out the part about all the holes being par threes with no
water, sand traps, or trees to get in the way. On an average competitive golf
course, my performance would be much worse.

Interpreting validity is more than just asking, “What is your validity
coefficient?” You should dig into the specifics of the situation. Pay attention
to specific items such as sample sizes, types of data being studied, types of
positions, or any other particular items of interest. Some studies may not, at
face value, seem impressive until you understand the situation and the results
based on the situation.

For example, by deploying an assessment technology, a large call center
enterprise hoped to identify job candidates who could reduce the average time
spent on incoming phone calls. After studying the performance of 704 employees
over their first 12 months on the job, employees hired using the assessment
process averaged call times that were 1.14% shorter than calls taken by their
non-assessed coworkers. That translates to a savings of approximately four
seconds per call, or about the time it took you to read this sentence.

At first glance, are you impressed with a 1.14% improvement? Before you answer,
consider this: across the entire corporation consisting of multiple call
centers, each second shaved from the average call time is valued at $175,000
over the course of a year. That four-second improvement saves over $700,000 per
year company-wide, and the assessment technology has paid for itself many times
over.

While there are plenty of success stories, be aware that the reverse can occur.
A study may appear very impressive at first glance, but when the situation is
exposed to the light, the results may be found lacking due to tiny sample sizes
or some other extreme set of conditions.

Breaking down the question, “What is your validity coefficient?” a bit deeper,
we find that the terms are in a singular context. Meaning, the person asking the
question is asking for only one number or one value that represents the entire
concept of “Does it work?” or “How has this made someone else better?” It is
important to realize that a solid, proven assessment technology should be able
to show many studies from different companies, positions, and situations. Each
study, based on the situation, should show a relationship (in one form or
another) between the assessment outcome and the performance metric. The
documented volume of evidence should go way beyond one “validity coefficient”
and provide massive amounts of ongoing research proving the technology has, and
continues, to make other companies better.

Just as with a hire-termination study, obtaining concrete performance results
for all positions may not be possible. Temper your expectations for performance
studies according to the scope of the position. Small sample sizes, a lack of
objective performance metrics, or an insufficient amount of time for data
collection can affect your ability to conduct a quality study.

When evaluating an assessment technology, ask to see multiple client case
studies that demonstrate significant performance improvements based on quality
sample sizes. Reputable assessment technologies should provide access to a
technical manual packed with studies that detail significant improvements in the
areas of turnover and performance.

Summary

There you have it…the list of five business principles that should guide your
decision on your next purchase, or upgrade, of an assessment technology. To
recap, here are the five principles:

• Principle #1: An assessment technology should be proven to predict
performance.
• Principle #2: An assessment technology should be the catalyst to continuous
workforce improvement.
• Principle #3: An assessment technology should be focused on fit; more is not
always best.
• Principle #4: An assessment technology should be more than just a score.
• Principle #5: Assessment technology should be a tool that makes your
organization better.

This is by no means an all-inclusive list, but if an assessment falls short on
one or more of these principles, keep shopping. Your efforts will deliver great
dividends for your company when the right assessment technology is
in place.

One tip I recommend to those evaluating different assessment technology tools is
to create a wish list of features and functionality. Be sure that the needs of
all levels of end-users are included in your wish list. Then categorize the list
into groups consisting of the “must haves” and the “like to haves.” This little
exercise will help you focus your efforts during the evaluation process to
ensure you achieve maximum improvement within the organization.