Tech skills analysis with resumes and dev. platforms

Every now and then each recruiter gets negative feedback on candidate’s abilities. It happens after a technical interview, and goes like “The candidate is not Senior, he doesn’t even know the framework!” Then you recheck his perfect resume with Senior React words in title, questioning what went wrong.

A nice simple algorithm:

1. Find a candidate who was Senior Something at company X;
2. Propose a similar but better position at company Y;
3. Profit!

mostly works but sometimes breaks for mysterious reasons. Some fail an interview, some are immediately rejected – and then it’s a bit of your fault as well.

Resumes impact and determine personal careers, but they are also trivial to fake. You may be underestimating the scale of the problem. According to a standout-cv.com survey more than 64% of people, on average, admitted to lying on their resumes. Considering the % people that lied without confessing, we can assume the real fraction is even larger. Per-industry statistics favors engineers, over other experts, as being relatively fair. But still the number of altered resumes is more than 34% in that field.

We’re talking about truth and lies, but often it’s hard to draw a line in-between. There are humble people, who lower their abilities, and there are their opposites. One way or another – it’s hard to exclude natural bias when writing a resume. Now, as recruiters, we evaluate such results of self-evaluation and hope for better.

Approaches to pre-validate skills

Skill checking is a permanent issue. You can’t say much without an interview, but you can’t interview everyone. In an interview, to properly validate tech competence, you’re expected to know what? All popular technologies?

Should you hire other people to help you with the task? For candidates – maybe, though very expensive, but what about prospects? There’re too many of them. Automated tasks, questionnaires? Not applicable. Until you had, at least, a single call – most people would reject such proposals.

There’re cases when we can be confident about a profile, though. One example is a specialist, focused on a single area. We see multiple titles with the same tech, we see a lot of related skills – we feel solid. The complexity here is that (ideally) we should know the field. VueJS experience does not directly translates to ReactJS, but they are still very similar tools. Experiences with Machine Learning and Frontend Development do no sum up nearly as well.

All that, however, is applicable to a subset of resumes. Profiles of Middles and Juniors may not contain enough information. Profiles of people who switched titles and roles are more “blurry”. As well as generalist profiles that might be the most challenging to interpret correctly. It’s hard to tell which technologies they are interested in at the moment: most recent, all mentioned, or something else.

Resumes alone do not provide enough trust and clarity, so we refer to other sources of information – sometimes provided near the resume, and sometimes on separate platforms.

One such example is certificates. It’s easier to believe in DevOps or Security engineer dedication when we see abbreviations like CKAD, DOFD, OSCP, ISC2.

Another source of truth is given by testimonials. Developers with better soft skills will collect more testimonials, on average. Regardless, it’s clearly good when the profile you’re validating has them.

DEV Platforms

There is yet another great source of information – underestimated and underused by many recruiters. A wide range of open source, tech, blogging, Q&A resources where people share their projects (not resumes), discuss, develop, contribute, and compete. For lack of a better term, we’ll call them DEV platforms. GitHub, GitLab, HackerRank, StackOverflow, Medium, Dev.To, Hashnode and many others belong to this category.

DEV platforms can often tell us more than resumes. They are social communities after all. And you don’t need a tech education to make grounded conclusions. An engineer with unusually many followers must be possessing some talent. A developer with a high number of contributions should be very productive and/or experienced to achieve that.

As Nietzsche wrote: “The proof of strength is an excess of strength.” There certainly are silent talent who focus all their effort on paid work, never going outside and beyond that. But it’s easier to believe in something that can be tracked, measured and proven by more than just a personal claim. Open source contributions, blogging, questions & answers, high ranks on platforms like CodeWars… all this activity reveals and confirms hard (and sometimes soft) skills with a confidence non-achievable by resumes alone.

Now you can say that, while you agree that it works in theory, – it’s too time consuming. And should be reserved for the most responsible roles, where you really want to gather all the available information. For Joe “The Average” Frontender quantity-first approach has kept you more productive so far.

We would still object that an angle of view is important. Looking for an “active developer”, you eventually find an active developer. Looking for a “perfect resume”, you eventually find someone “with a perfect resume”. Neither is guaranteed to be a great employee, yet still, fingers crossed, chances are in favor of the first. History is proof. And a resume is just a list of statements.

But, most of all, it’s a matter of available tooling. If only there existed an instrument to search and analyze DEV activity, a tool to accompany your resume search – you would definitely try it out, right?

Welcome DevScanr

DevScanr is a tech talent search and analytics platform that works on top of GitHub and other public profile pools. The platform is powered by machine learning, graphs, and custom algorithms. Which allows us to enter the unexplored territory of activity-based talent search.

With this new, alternative method you can start with data derived from open source, blogging, etc. activities and explore resumes afterwards. The idea is that you can visually scan profile cards, mostly represented by skills and activities, with the same speed as you can scan resumes represented by titles and work histories.

Unlike resumes, however, dev. activity records are objective and harder to falsify.

Regardless of which search method is your primary, you’ll probably agree that combined information from multiple (independent) sources will be the most objective, complete, and precise. And it’s best when sources have different vectors of strengths and weaknesses.

Resumes reflect personal, subjective view of someone’s experience. They are more prone to “false-positive” kind of errors. Even if we exclude liars, people overestimate and overemphasize certain skills. So you make wrong conclusions as a result.

DEV activities are more objective but more prone to “false-negative” kinds of errors. We miss a lot of developers who are not active in public areas. And we miss hidden parts of work experience that might be reflected in resumes.

DevScanr’s vision is that you should combine many sources of truth: resumes, activities, certificates, testimonials, hard and soft skills – as quickly as possible – to make an objective judgment about talent abilities.

Skills and Interest Highlights

All recruiters know how important it is to quickly assess experience and whether it matches the requirements. Search results on DevScanr are ordered by scores at several axes. If you enter PHP as a required skill – talent with most (and the most significant) PHP contributions will be at the top.

In comparison, resume platforms do not typically have the luxury of knowing how often some skill was applied, and to which degree. They have to rely on indirect inputs like number of skills mentions in work history and endorsements to sort their lists of profiles.

DevScanr combines different pieces of functionality and, in this article, we focus on profile analytics.

We tried to keep profile cards simple yet packed with important information. Searched terms are highlighted and ordered by their relative score, as you would expect. As for something novel – DevScanr features a unique distinction between skills and interests.

Here’s an example. A Security engineer must be interested in Security, by definition. But an interest in security from a Frontend engineer is less common and therefore a peculiar characteristic. It can be more or less important, depending on the exact project and the team you search for. If his/her task is to develop an authentication system – the knowledge, or even a casual interest in security, is a big plus.

Such extra pieces of information might be missing from vacancy descriptions. Some companies prefer to keep them lean. They will appreciate such facts or educated guesses about a person anyway. So we designed DevScanr cards to be useful both for recruiters and for company representatives. The interests, finally, can and should be used as communication and engagement points for candidates.

Skills Radar Charts

You probably saw and maybe even used one of the browser plugins that capture information from GitHub accounts and aggregate it to convenient cards. Such plugins might be useful to analyze a particular profile but they can’t help you to find a candidate. So you have to rely on (quite weak) builtin GitHub search.

We’ll review pros and cons of such plugins in a separate article. For now, one major issue to remember is that they are limited to what GitHub server immediately provides. GitHub detects and calculates “language usage” percentages, all other skill categories are ignored. Like frameworks, databases and so on... As long as such tools do not introduce a new “language” – they will be missing from those cards.

DevScanr goes much further. It parses texts with NLP tools, infers missing pieces with AI models, perfectly understands all important tech skills. It relies on custom algorithms to extract, analyze, and present the abundance of information available in code and surrounding content: documentation, commits, issues, comments, etc.

Pie-chart (as above) displays values in relation to each other. So it’s possible to compare personal skills but it’s not possible to compare skills of different people with each other.

Radar chart (as below), we have chosen for DevScanr, is much more flexible. One chart reveals not only skill distribution but also how well a particular developer benchmarks against the proper average. We calculate an average based on the same specialization, within the same seniority. So it’s good, apples to apples, comparison. And something that were never possible on resume-centric platforms.

Sure, DevScanr knowledge is derived from public data and does not take private contributions into account. Maybe some person is a vivid contributor to a single private repo that will “change the world”. Maybe – we don’t know that.

Each and every skill metric is incomplete and even flawed in some sense. Hey, even tech interviews yield false positives and false negatives. Still we should rely on something for a quick check and use the best tool that is available for the job.

Unlike forementioned plugins that use only data from public repositories, DevScanr considers talent star and fork activity. With custom logic for each category, unified to a single final “skill score”. You’ll be surprised to see how much information it’s possible to extract from there.

Activity Heatmaps

We already mentioned that it’s hard to judge which tech skills are the most relevant, especially for generalists. Some terms, like PHP from a job title related to early 2000-s, might demonstrate years of experience and represent the strongest skills. Or it might represent something that is long forgotten. Developers might be tired of this stack so your job proposal won’t interest them at all. It’s an art of striking a balance between “well known, but boring” and “exciting, but not time proven”.

You can certainly derive some clues from a well written resume. But the same, or different, clues can be derived from an open source activity. When it’s the same – it adds to your confidence about talent’s expertise. When it’s different – it can help to detect something interesting.

The strongest skills are both:

• began to be used long time ago
• are still in use or were used recently

Activity Heatmap displays that perfectly. Like analysis, like everything else, demands some understanding. The lack of open source activity does not equal the lack of interest or effort per se. But it’s still a signal that priorities have changed, that a person finished an education track, etc.

You’ll get more information with comparisons. A track of React activity interrupted by a track of Svelte, which then continues for months or years, is a very strong signal that the developer is no longer interested in React (red arrow on the screenshot above). And it’s pointless to propose such positions to them.

Again, you can change any line in your resume. You can “travel back in time” and describe 1990 differently. But you can’t spoof the createdAt field of repository on GitHub. Activity heatmaps are powerful indicators that are not used and not popular only because no one previously cared enough to extract and use them.

And if an abundance of technical terms feels overwhelming – DevScanr adds tooltips with descriptions for all technologies and concepts, so you just need to hover and refresh your memory about a particular meaning.

Summary

We introduced and reviewed a problem of talent assessment, hard tech skills in particular. We discussed why recruiters cannot trust resumes alone and should combine multiple independent sources of truth – whenever you want to be confident about your prospect/candidate picks. We saw how DevScanr platform provides an alternative way to search and an alternative way to analyze tech profiles. Be sure that we’ve just scratched the surface of what is possible to learn from data. There’re more insights and opportunities but we have enough material to digest for today. Good luck!