AI and remote interviewing have definitely exacerbated the problems that existed before though.

Leetcode was always weak, but now that it is easy to cheat on it is a negative selector, because the cheaters do best. Leetcode was originally supposed to be done in-person on a whiteboard to assess a candidate's collaborative problem solving skills, but with remote interviewing it has evolved into writing passing code with minimal or no feedback.

The real problem is that engineering departments are now filled with leetcode grinders and cheaters, who all live in permanent fear of being replaced by AI, and so any candidate who doesn't fit that paradigm is a threat that must be eliminated at all costs.

Honestly the problem is hiring teams -- they have created this whole issue. They ask for a resume and cover letter. Fine. But don't make applicants put in the work if you're not even going to provide a response, or any sort of feedback -- even when the position stays open for months. The result is that people looking for jobs have to submit huge numbers of custom cover letters, and tweaked resumes, with no feedback, all within a vacuum. Hence the feeling that they need to "pump up" their resume, just to get over the initial gate.

It goes the other way as well though. Companies are increasingly filtering resumes/candidates in a sufficiently aggressive fashion to the point that they're strongly incentivizing, if not actively selecting for, people that are gaming the system in some way or another. Quite odd this is all happening when ostensibly the unemployment rate is very low, which should make it an employee's market.

The bigger issue is the screening filters are flooded now (and also largely AI “enhanced”) so getting real signal through the noise is becoming basically impossible.

> This has always been a problem: Candidate applies with an amazing resume but then flails when you ask them questions or “can’t remember”.

Yeah, but it's now 1000x worse. Before you needed actual skill (or luck) to create a good looking CV, especially for niche positions.

Now you take their job description, the company's "About us" webpage, your old CV and have LLMs generate a CV with pretty solid grammar and most of the verbiage they expect.

In the past the average unqualified person wouldn't even know the right words for a specific niche domain, let alone how to use them.

Oh, and single LLMs are kind of inherently multilingual, this makes it even worse, because you can have people that barely understand the target language generate a reasonable CV in that language.

The CV quality floor has been raised but the candidate floor has fallen through the pits of hell.

Yes. People are gatekeeping. They are not interviewing with the aim of having an interesting conversation. They are trying to block hiring.

There's a spectrum of 'broken' and AI made it worse

Yes they just automated the broken now.

Much of it has been based on networking for a good 20+ years. Yes, there was a certain if you have a pulse era in tech at at a lot of companies--leet code notwithstanding though that was an issue--that has largely passed and a lot of people are reacting to tech hiring becoming a more normal multi-month process.

I had a similar experience before [1]. I fully agree that too many interview tests select for dishonest people.

[1] https://news.ycombinator.com/item?id=35496976

> Companies are increasingly filtering resumes/candidates in a sufficiently aggressive fashion to the point that they're strongly incentivizing, if not actively selecting for, people that are gaming the system in some way or another.

Which itself is a symptom of companies getting drowned in AI generated resumes. It's becoming more common for people to use AI tools that will operate browsers to mass-submit resumes for them. When you receive 1000 resumes you have to start filtering somewhere.

What I'm worried about now is that we're moving to a situation where some level of proof-of-work that an AI can't easily do is going to become necessary to have some filtering. I don't know what that looks like, but I don't like it.

> Quite odd this is all happening when ostensibly the unemployment rate is very low, which should make it an employee's market.

Unemployment rate is not evenly distributed. If you were a licensed electrician or qualified as a home healthcare aid then you could walk from one job to another in many cities.

If you're trying to get a $200K or more tech job, then you're competing with everyone else for a shrinking pool of openings.

> Before you needed actual skill (or luck) to create a good looking CV, especially for niche positions.

Sure, resume writting is a skill, but it's probably not relevant for the position unless the position involves a lot of grant writing or enterprise sales.

I think we'll just end up going back to referrals. It might generate more nepotism, but at least the company will feel like it's doing a better job and not cause it to overly focus on hiring to the detriment of its current employees.

We ask for something stupid like "3 years of Pascal experience." If the resume has it, it goes straight to the trash unless it has specific real-world Pascal experience.

Recommendation from a trusted 3rd party.

Bill Gurley has a great line about this:

"I use LinkedIn like this:

If Person A reaches out to me and there is a Person B that is a common connection between A and myself, I want to be able to call Person B and have 100% confidence in their evaluation. That's the bar I set to connect with someone on LinkedIn."

From:

https://www.youtube.com/watch?v=xmYekD6-PZ8

I think various ‘longer interview’ processes can be good by reducing the chance of particularly regretted hires. This could be internships (but note this goes two ways and you want interns to accept offers and recommend the programme to their friends even if they are not hired) or work sample tests. Both have the downside that they are more work for the candidate (especially internships or some other short-term-to-possibly-long-term position) and so experienced candidates who feel they have better options and less need to prove themselves typically won’t take part (this depends a bit on how much they want to work at your specific company of course). Potentially this isn’t so bad – competing to hire the same people as everyone else is going to be more expensive – or potentially it is bad – maybe there’s a reason those candidates are in high demand and you will suffer from only getting a look at people who didn’t fit the typical pattern. I think it’s going to depend a bunch on how good you are at sourcing candidates and how hot your firm is.

Yes. Figure out who your top performers are ask them for referrals. Some people will recommend "meh" people, but more often than not your top performers hang out with other top performers because they appreciate the same things.

We source our intern-to-Junior pipeline from a good state school from which we have a few graduates. We have about an 80% placement rate for the interns. We’ve yet to have any abusive or bad hires, this being a fully remote company. For Senior hires, a prior employee founded a Java User Group and sourced several high quality engineers from the pool of visitors. So, build a pipeline and play the long game?

Previously we’ve sourced candidates via a reputable recruiter from an in-town firm that our manager can routinely sit down with and build a relationship over the years. This had a good rate with only one bad placement. We ultimately traded time cost for money cost in that one, but I liked it.

The worst outcomes we’ve had were via LinkedIn jobs posts. By the time our in-house full-time recruiter would give us resumes half would be obvious frauds with most of the remainder being subtle frauds. I blame this in good part to having non-technical staff as the first filter in our pipeline.

Unfortunately the firm makes money hand over fist year on year so we are no longer a lean mean operation but a burgeoning beauracracy with room to hide, rest, and vest.

I think being successful makes hiring easier because you can source better candidates, and being big makes hiring easier because candidates are more likely to know people at the firm who can use referrals to work around otherwise broken systems.

It’s perhaps also worth noting that lots of companies used to copy how Microsoft did interviews and later they copied how google did interviews so clearly there were some ideas that those companies were good at hiring. (I’m not sure this strategy was that good. The problem for the Microsoft or google type companies is filtering out acceptable hires from a deluge of applicants with acceptably low errors and costs; the challenge for less desirable firms is sourcing candidates who are both high quality and not about to be hired by Microsoft or Google)

One company that comes to mind when I think about being good at hiring was one that recruited a bunch from my university around when I was graduating. Their particular specialty was hiring illegible graduates with a lot of potential (eg classicists, science students without little programming, etc), training them well, and effectively underpaying them a bit for how skilled they were (which only worked out because the UK has a pretty shit job market for tech and because those people liked working there). I think it was more effective for them than trying to hire the same computer science graduates as everyone else would have been.

Ask the best people you’ve ever worked with who the best people they’ve worked with are. Recurse. When names start repeating through different graph paths, make those people an offer they can’t refuse. Once they join, ask them to do the same, and give them the budget and role to make it happen.

Apprenticeship? Actually spend time working with them on real work.

It's both hard and doesn't scale.

Because it is really hard to reliably hire good people. Almost all typical signals & methods (CVs, experience based interviews, …) have very low reliability. An IQ test has the highest reliability according to studies but would be illegal in most jurisdictions. Plus, hiring managers frequently don’t know what they want or they believe they want something that they actually don’t

The best hiring is generally expected to happen through referrals, so there's not a ton of pressure to improve the public application pipeline beyond the minimum required to keep it functional.

Maybe companies should say “Wanna apply for a job? Come to our office during these hours for a pre-screening and to drop off your resume” you can’t email or apply online anymore

Absolutely. It turns out friction is important in the right places.

Lying on resumes is very common, so is lying on job postings. It's a really weird arms race where no one is getting what they want.

I will say that I'm not surprised by this at all. I think a ton of people have been convinced that basically all languages are more or less the same, so they are confident putting languages they barely know on their resumes. "I know python and Java, how hard can C++ be?". This isn't a new problem, or even a "coding bootcamp problem"

I studied computer science at a small university in 2006, several of my friends went to a much larger university and studied Software Engineering

They didn't learn pointers back then either. They learned Uncle Bob Java and that was basically it.

It's just how you have to structure a large organization to spend responsibly. You assign a budget to each function, you tell people to do the best they can within the budget, and if they come back and say "actually I'd like to spend even more money" you ask them to generate an explanation for why that's in the best interests of the business. And generating that explanation is kinda the whole purpose of the source article: the Harvard Business Review exists for managers and executives to discuss amongst each other as arguments for why their proposals and budget requests should be approved.

I have what I think is a good analogy for this problem.

In the Olden Days [tm], jobs were advertised through recruiters, physical media (eg the paper) and connections. You had to review applications and conduct interviews. The cost of applying was relatively high, your reach was relatively low and the investment per applicant was relatively high. So imagine that there were enough jobs for everyone in a simplified model. 10 people applied for 10 jobs. It's not the same 10 people for each job. But there was a decent ROI on effort. It kinda just worked.

Fast forward to now and the cost of applying is essentially zero in terms of registering interest and submitting a CV. And you apply for a lot more jobs so instead of 10 applicants for 10 jobs, you have 200 applicants for 200 jobs. Still the same applicant to job ratio but way more inefficient for everyone involved. Applicants can't put in the same effort for 200 applications that they did for 10 and employers can't review 200 applications the same way they did 10.

So what happens? Employers, who have the power in this relationship, put up roadblocks in the name of efficiency. Now you have to survive ATS before ever going in front of a human. That ATS uses inscrutable logic that may filter you out for not including enough keywords or some other specious reason. You now have hiring assignments.

The net effect is that an applicant puts in 200 applications, get automatically filtered out from 180 of them and then has to do upwards of 20 take home assignments.

Plus there are more and more layers added. More rounds of interviews. Phone screens. Remote interviews then on-site interviews. All of this wastes time and, like you allude to, I don't think it's effective. But it's a natural response to the illusion of choice.

Let me give an online dating analogy. In years gone by, you'd rely on meeting people in person. Now, less so. And speaking in a strictly heteronormative sense, how it tends to go is that women on average have hundreds of choices and men have on average far fewer. A gender imbalance plays into this.

So what does a woman in this scenario do? They start adding filters to just make the numbers more manageable. Height, salary, location, same interests and so on. So the net effect is that that a lot of people are indepndently applying filters and filtering down to a pool with a lot of crossover. Conventionally attractive men, for example, will tend have far more options.

So I think the same happens with hiring. If you're a Big Tech company, you start adding filters. Did this person go to a top school? What internships did they have? Do these things matter for on-the-job performance? Barely (IMHO). But what you'll probably find is that a handful of people have a ton of options while others struggle. And it's simply the product of employers trying to make their applicant pool manageable but they're all doing it in very similar ways.

And I honestly don't know what the solution is.

Because like a lot of things, metric of "What does recruiting cost us?" is very easy number to quantify so companies will attempt to reduce it.

"What does bad recruiting cost us?" is very hard number to quantify because it's just sand that gets thrown into so many gears, but cost of that sand is across a ton of departments and so measuring for it is very difficult.

Here's the secret: it's still just gambling. Elon musk isn't a trillionaire because he brought something special to the table; it's because he was able to perform the martigale enough times and he arbitrarily reached the top.

Hiring is exactly the same thing, even when trying to do it on merit, people are simply poor judges of character, ability and the rest.

Most of society is governed by people who simply kept getting lucky and kept doubling down because their ego demanded it and their last roll of the dice didn't drive them to poverty or happiness.

In the not too distant past (like 10 years ago) flying people to a final in person round was standard practice.

> It's very strange that the authors talk about how "making a bad hire is terribly expensive" but then call out "travel time and costs". Well, if B < A for each role filled, is it really so bad?

The cost of a bad hire they're referring to includes things like opportunity cost of not having a good hire in that position, damage they've done to the product (codebase, design, etc.), and second-order effects like demoralizing the rest of the team.

The actual hiring costs of a bad hire are a rounding error compared to the damage they can do.

Have you ever been on a team that was great until they hired one wrong person who made every work week a miserable slog? Attrition goes up as the good employees start to leave. The codebase starts accumulating a lot of tech debt. Even after they're gone it can take a long time to recover.

This is why it's so important to be able to fire fast, but that's another topic rife with difficulties.

I’ve worked with places that have had HR bungle the hiring and places that haven’t. The only difference is whether it’s HR or Engineering bungling the hiring. Writing a job description that actually matches what you want is hard work. Sifting through 300 applicants that don’t meet the requirements or lie on the application form is hard work. Doing 10 30 minute intro calls is hard work. Desigining “standard” questions for comparison is hard work. Wrangling 2 rounds of interviews per candidate, dealing with people who are too busy with work for hiring is hard work. Chasing people for interview feedback that isn’t just “yeah seems fine” is hard work. And then getting the group to stop saying “we want to speak to more people” is harder than any of the previous steps.

I’ve interviewed hundreds of people over the last few years as a peer, hiring manager, and as a “bar raiser”, and it’s just a lot of work no matter who does it…

I told the last recruiter who had a six week interview process “I’m not waiting that long when I can literally clone your product in a weekend for $50

You'll also filter out people smart enough to know that this is a bullshit keyword matching game and the only way to win it is to put the keywords on their resume.

Because they assume that the job posting was written by a non-technical idiot, and 95% of the time, they'd be correct, and they are just playing the game as the game expects to be played.

Look. If you're looking for 100% integrity and honesty from everyone in their communication, you shouldn't expect find it in a corporation's hiring and HR process. Everyone white-lies (or black-lies) all the time, both up and down the chain. The bones of this interaction do not value, reward, or even want honesty.

Is this really battle-tested, as in: did you see this work in practice or do you have a reliable source?

Or is this something you came up with?

The problem is that you have 100 applicants and one apprentice slot.

In my anecdotal experience talking to people applying for jobs right now, this practice has come back in full force. You can expect final round interviews to be on-site unless otherwise specified. The days of getting hired entirely remotely are over.

A friends' company has even ended remote hiring altogether after auditing their remote hires and discovering a lot of connections from countries they didn't expect.

There's even a growing scam where people get recruited to lend their identities and bank accounts to someone else to get the job. Then they're asked to install some software on the company laptop and leave it open and powered on during the workday so someone can operate it remotely. Remote work is wild right now.

But that's the point.

If the cost of a bad hire is huge (which I agree it is), why is the hiring process optimized, in part, around reducing the travel costs? It would seem that these costs are modest in comparison.

People really want _their_ position to be meritocracy because their ego can't handle alternatives of a chaotic world.

Huh? Hiring being broken has nothing to do with cost, it's a filtering problem. Even when there's no HR or bean counter in sight it's still hard. There's fundamentally limited signal you can extract from interviews, so there's very loose correlation to on-the-job performance. Saying it's a cost-cutting problem would just encourage more and longer interviews, which could actually work against you because high performers tend to have more options and will not jump through infinite hoops.

Standard and expensive (in terms of time and money) for both the company and the applicant

That's nice. But a lot of people looking for professional jobs may take a year+ to find a position.

> And then getting the group to stop saying “we want to speak to more people” is harder than any of the previous steps.

FOMO will keep them doing this in perpetuity until you find a way to make them feel the pain.

+1, referred candidates at my company perform way better in interviews and on the job compared to cold applicants

I am aware of several startups that started this way, and have been involved in some. The quality of the results depends on who you seed the graph with, of course; but I’ve seen it work well.

As requested by the original poster, it doesn’t scale.

No, this is 100% battle tested. I can't drop numbers but we just did a study at work comparing referrals to non-referrals... its night and day. Referrals are out performing across the board. the only problem is that eventually you run out of referrals

> High performers tend to have more options and will not jump through infinite hoops.

Biases are a strange thing. “High performers” aren’t one homogenous group; take a staff engineer at a FAANG and plop them in a role at a startup or vice versa and you’ll find very quickly that high performers are a product of environment (IME). The people you need to ship something at a big company will sink your startup, and the people who will lead a startup to unicorn levels of success will flounder in frustration in a big corp.

Finding high performers is really hard, as you said it’s a filtering problem, and it’s very much based on vibes and feelings. Leetcode, take home tests, on site tests, discussions about projects all filter for specific things - some or many of which aren’t related to the job at hand. If we removed the “risk of leaving current job element” the only way to do it would be to give someone a 3 month trial and see if they’re a fit. Honestly you probably know in your gut by week 2 if it’s going to work or not.

Well, maybe some things should be expensive.

FYI companies should be reimbursing travel expenses for this travel. As a candidate it's worth clarifying to confirm so you don't get some oddball startup trying to force candidates to pay their own travel, but every big company travel interview will be expenses paid down to your travel to/from the airport and the meals you eat along the way.

The time commitment is real, but on-site travel is almost always reserved for the last round on-site. Often as a final pass verification, or when the company is down to a couple of final candidates. Companies aren't flying every applicant out for all of the interviews. If you get to that point, you're close to the job.

If the cost of hiring the wrong person is huge, the cost (in terms of time and money) of conducting on-site interviews is almost certainly lower.

Also, in terms of the costs to the applicants, this touches on the oligopolistic nature of so many industries today, which has resulted in high concentrations of the most desirable jobs in places with the highest costs of living.

Basically, unless you already have a FAANG job or are independently wealthy, it's not easy to up and move to Silicon Valley, Seattle, etc. and job hunt.

IME the only way to make them feel the pain is have them responsible. Panel gets yes/no and hiring manager makes final decision. It’s not a consensus.

Been 3 years for me so far

Agreed, that's why I said used the phrasing "tend to". There's no silver bullet.

It doesn't even need to be startups versus FANNG. I've seen first-hand how people hired into various roles aren't a great fit for roles as a company grows and changes. Of course, they can adapt to various degrees but they'd probably never have been hired for the roles they're now in.

The problem with trials is that people often have a current job of some sort and having things not work out puts them in a difficult situation. May happen anyway but, generally, a new job is assumed to be at least a somewhat stable situation.

> The problem with trials is that people often have a current job of some sort and having things not work out puts them in a difficult situation. May happen anyway but, generally, a new job is assumed to be at least a somewhat stable situation.

Totally, and I’d never say “hey I think you might be a fit, let’s try it out for 3 months”. But if we interview someone I’m just doing my best to try and figure out do they get on with the team, do they have the right skill set match for the gap we’re trying to fill and will their working style work in the org. Everything after that is (unfortunately) up to how it goes when we’re working together.