The search for skilled IT staff has gone global, with almost a quarter of those in some organizations now defined as “borderless” — sourced and based overseas, according to Gartner.
Casting such a broad net, however, makes it challenging to find the people you want and, crucially, onboard them quickly, reliably and securely.
It’s a job built for machine intelligence: finding and analyzing people with different skills and expectations and matching them to an infinitesimal number of openings while being governed by a multitude of requirements.
Both people and positions lack anything approaching a common syntax that might help. Add to that challenges such as language and time zones.
It’s a huge data analysis job, yet we built our AI-driven hiring and recruitment platform Andela Talent Cloud (ATC) without using a large language model (LLM). In fact, we took ChatGPT out of our pipeline and built a system founded on structured data: Our engineers developed a taxonomy equipped specifically for nuances of the recruitment process.
Engine Deconstructed
Our Talent Decision Engine (TDE) uses AI- and data-driven matching algorithms to pair people to positions, analyzing thousands of data points from skills and experience to location and language proficiency. This engine is fed by a number of related services: a Talent Response Service to prioritize people who are most likely to respond quickly; a recommendation engine to match and rank people based on their overall fit for a role; and AutoMatch that optimizes for successfully making the most matches possible while avoiding bid wars.
TDE delivers high-quality matches, something ChatGPT couldn’t provide. First of all, LLMs are unable to handle tabular data well and may struggle to extract meaningful insights from such data representations. ChatGPT also lacks adequate numerical processing capabilities. LLMs primarily deal with textual data and may not infer relationships between structured data containing numerical values, such as how well a talent’s time zone and working hours match with multiple job requirements, each from a different time zone and unique minimum working hours.
In addition, LLMs face explainability challenges, which is crucial for decision-making: While they can generate text outputs, understanding the reasoning behind their predictions on structured data is challenging and is a significant drawback when compared with techniques focused on tabular data, like XGBoost or alike, which can easily explain their predictions using SHAP (Shapley Additive exPlanations values), for example.
A fourth drawback is that LLMs typically have a limited context window, meaning they can only consider a fixed number of preceding tokens when generating text. This limitation makes it difficult for them to capture long-range dependencies and complex relationships present in structured data.
These are just four of the reasons we chose not to use a LLM for the tabular types of problems we faced. Basically, they cannot perform as effectively or efficiently in such scenarios compared to specialized models designed explicitly for structured data processing, such as graph neural networks or traditional machine learning algorithms like decision trees or support vector machines.
As a result, we created models based on tabular data, which respects a structured taxonomy to get around this problem. Our AI-driven approach models idiocratic elements inherent to our business domain. For example, it classifies skills based on their necessity — critical, mandatory and optional — to fine-tune the automatic matching process. Next, it analyzes the congruence between talent skills and the job’s requirements.
Then it meticulously evaluates the alignment of other job prerequisites with the characteristics in our talent pool, including considerations such as temporal constraints and time zone compatibility, previous experience, the candidate’s preferred role, etc. Additionally, our methodology incorporates robust protocols essential for curating and annotating databases vital for sustaining our training pipelines. Such meticulous procedures are paramount, particularly when confronted with job categories characterized by data scarcity.
Another challenge was quality: How could we ensure ATC’s algorithms would find the best candidate? We worked with recruiters and talent matchers to figure out which qualities to look for. This helped us develop what we call a Match Fitness Score, using information on more than 10,000 individuals and around 800 skills. The score is based on 50 attributes such as time zone, years of experience, skills and salary.
Dealing With Incomplete Data
Building a credible Match Fitness Score means we also had to overcome holes in people’s profiles – the absence of essential data. Some people, for example, don’t specify what they want to earn, which is important both in matching people and setting rates that meet customer budget expectations. In this specific case, we developed a talent rate recommendation service that generates an approximation of how much someone might seek based on their skills by identifying those with similar skills.
Building and refining the different components of ATC entailed using a number of different techniques, including dimensionality reduction, word embeddings, one hot encoding and data standardization. Often we’d use several techniques to solve one specific problem — comparing the results and selecting the most effective approach.
We discovered no shortage of useful machine-learning techniques and methods to draw on when solving technology problems. The real challenge was ensuring project participants had fully captured and articulated the business and processes involved in hiring. There are so many nuances that getting even the smallest detail wrong could result in flawed search outcomes.
To deliver Match Fitness to customer-facing applications, we developed the Extensible Recommendation Service (ERS), a Python-powered framework designed to offer endpoints that assess the suitability of talent for various job roles. Through ERS, our customer-facing applications gain insights including skill-based match fitness, talent responsiveness likelihood, estimated talent rates and more. That empowers our platforms to efficiently identify, engage and initiate dialogue with the most qualified candidates for each job position.
What’s next? LLMs will feature in the product roadmap, though they likely will support ATC rather than become a cornerstone. For example, we’re considering whether LLMs can be combined with talent profiles in areas such as skills to help improve the presentation of candidates to potential recruiters. We’re also using LLMs to parse job descriptions for skills to map to our taxonomy to simplify the job creation process.
Generate Insights from Structured Data
LLMs are getting a lot of publicity. While many people rush to skill up, it’s worth reflecting that success in AI comes not from the technology but from understanding the business problem you’re trying to solve. Based on our experience, for other industries to generate insights from data, they should:
- Recognize the limitations of language models (LLMs) like ChatGPT in handling structured data. By investing in specialized models designed explicitly for structured data processing, such as traditional machine learning algorithms like decision trees, you can develop high-accurate explainable models.
- Understand business nuances: Ensure project participants fully understand and articulate the business and processes involved in any domain-specific task. Pay attention to nuances, as even small details can significantly affect the effectiveness of data-driven solutions.
- Develop strategies to address data quality issues, such as the possibility to develop a structured taxonomy relevant to your business domain. This can generate insightful new data types, like categorical information, that would otherwise be noisy, missing or incomplete in their original text format. Some good examples of this in our domain are job roles, skills and spoken languages, among many others. When they are properly extracted and combined, more powerful machine-learning models can be built.
- Employ smaller models to estimate missing critical information to feed other relevant models or services. In our domain, we did this, for example, to estimate talent characteristics, like responsiveness or rate.
- Work closely with domain experts to identify relevant qualities necessary to develop scoring mechanisms like the Match Fitness Score. Incorporate human expertise to overcome gaps in data and ensure the relevance of features needed by algorithmic decisions.
- Explore a variety of machine learning techniques and methods to solve specific problems and compare results to select the most effective approach.
By incorporating these strategies, industries seeking to leverage structured data for insights can develop robust data-driven solutions tailored to their specific needs, improving decision-making processes and outcomes.
The post How Andela Built Its AI-Based Platform Without an LLM appeared first on The New Stack.
Its data-driven matching algorithms that pair people to positions employs a structured taxonomy to overcome other models’ limitations.