The biggest coding bootcamps may exaggerate their success rates — here's how to choose one that will actually get you a job

• Coding bootcamps may have a financial incentive to overstate their effectiveness.
• Learning to code can still be life-changing for some so we spoke to insiders about how to find the best bootcamps.

Last month, I reported on what appears to be exaggerated reports on student outcomes by the coding bootcamp Lambda School. According to my reporting, the school's job placement rate for its graduates was around 30% — far lower than the 74% it currently advertises. My reporting showed that the school's advertised rates omit a large section of its graduates.

The school itself countered that any such discrepancies were simply a result of standard practices for bootcamp outcomes reporting. In this, they may be correct: General Assembly, for example, dropped more than 30% of its graduates for not "participating" fully with its career services. With these nonparticipants removed, they could then claim that the remaining 99.2% of their graduates successfully found jobs. Flatiron School, with over 2000 students, omitted 22% as non job-seeking. Other schools like Coding Dojo excluded as many as 40% of graduates.

Coding bootcamps are private, for-profit schools that typically take adults with some work experience and retrain them as computer programmers. They cost between $10,000 and $20,000, around 15 percent of an entry-level salary in tech — a price that is well-worth it if the student lands a job. Last year 35,000 students graduated from these bootcamps.

Uniquely among postsecondary schools, bootcamps have a cultural history of disclosing important numbers about how many of their graduates get jobs. However, not all bootcamp reports are equal. In their quest to compete with each other for enrollments, bootcamps have a potential financial interest in overstating their effectiveness, and prospective students should be wise to a few of the tricks the biggest bootcamps employ.

Cherry-picking data is rife among the largest bootcamps

According to Course Report's 2020 survey of bootcamp graduates, students primarily chose schools for the perceived outcomes of alumni, according to Course Report's 2020 survey of bootcamp graduates. Placement rates are the most visible estimate a student has for whether their career switch will be successful.

Looking closely at the aforementioned schools you'll find graduates are routinely omitted from their tally for a number of reasons including being unresponsive to career services or failing to apply to enough jobs in a week. Some schools omitted as much as 40% of its graduates from its placement rates.

A Lambda School instructor told Insider, the rate of non-responsiveness at bootcamps is alarming. "Maybe students aren't responding to you because they're angry with you. Maybe they're not responding because they're ashamed."

Typically, these "non-job-seeking" students are designated as such after they've graduated from the program — which is a red flag, according to Sheree Speakman, a former CEO of the Council of Integrity in Results Reporting (CIRR), a nonprofit that tracks bootcamp performance.

General Assembly spokesperson, Tom Ogletree, told Insider, omitted students chose to "go it on their own" and their 99% placement figure was one piece of data that prospective students seek before enrolling. Coding Dojo spokesperson, Luke Lappala, told Insider, that students who are unresponsive "are classified as opted out of Career Services due to nonresponsiveness" and this is "explicitly conveyed in our reporting." Flatiron School did not respond to a request for comment.

Speakman says nearly all boot-camp students classify themselves as "job-seeking" when they start their programs, because these programs are designed for students looking for a career in tech. The Course Report's survey similarly showed that 91% of graduates attended a bootcamp primarily to get a programming job.

For this reason, bootcamps that report to CIRR ask students whether they are job-seeking before they enter a program and are not allowed to remove those students from their placement rates once they've graduated. According to Speakman, prospective students should beware of any non-job-seeking rate above 10%, as this is a sign that the school is seeking to remove the students who did poorly. CIRR does not allow after-the-fact removal of students from their tally.

In general, students should be suspicious of results that are only published directly by the school. Prospective students should check to see whether placement performance for a particular school is available via a state regulator or on CIRR's website. Reports made to regulators or independent bodies are much more standardized and leave less room for creative accounting.

Income Share Agreements and tuition refund programs can be too good to be true

Lambda School was also a pioneer in tuition deferral programs called "Income Share Agreements," where students pay a percentage of their salary only after receiving a job that pays $50,000 a year or more. Since then, similar programs have become widespread in the bootcamp industry. Some schools, like the e-learning platform Thinkful, even offer tuition refunds to graduates who are unable to find jobs. But for many, these financing options can be a little too good to be true. In the case of Lambda School, even if a student withdraws, they can still be responsible for paying the school if they get a tech job within five years.

Recently, Income Share Agreements have also come under regulatory scrutiny for a variety of reasons. According to the Consumer Financial Protection Bureau, some schools misled students to believe ISAs were not actually loans and didn't comply with consumer finance law. Lambda School was also recently censured by the California Department of Financial Protection for claiming that its ISAs were nondischargeable in bankruptcy, when in fact they were.

Even Thinkful's tuition refund promise requires strict compliance with certain conditions such as the inability to say no to any "qualifying position." This can place graduates in a very uncomfortable bind. If a graduate receives an offer paying as little as $40,000 they must either accept the job and pay Thinkful's tuition, or reject the job and pay the tuition anyway as they no longer qualify for a refund. In tech hub cities like San Francisco or New York, this is barely within the living wage for single-person households and far beneath the liveable wage for anybody with dependents, as many bootcamp students are.

The best ways to choose a bootcamp

Despite these controversies, bootcamps are serving an ever growing share of students looking to break into tech. And for some, it can be a good career move.

Jesse Farmer, co-founder of Dev Bootcamp, one of the first coding bootcamps, sees a need for reform in the industry. Farmer, who now teaches programming at the bootcamp Adjacent, told Insider what prospective students should look for when selecting a school.

"Students should be shopping for bootcamps like they shop for houses, but they shop for bootcamps like they're shopping for cereal," he said. Farmer emphasized it's important to take time and make sure a school is a good fit rather than signing up for something immediately.

He recommended that prospective students ask schools: "What is the harshest thing ten random graduates from LinkedIn are going to say about your school?" A good school will be honest about the difficulties students in certain situations will face, and may also put prospective students in touch with graduates in similar situations to them. He recommends reaching out to former students independently as well, to be safe.

Farmer also stressed the importance of regular code review from experienced programmers in a school's curriculum. "Code reviews are at the intersection of all the important parts of the learning experience. There's the work, the tools, the student, the staff, your peers, and they're all converging into this place called the code review." Farmer encourages students to ask how many code reviews they can expect in a week, and if the answer is less than one, to forgo attending the school.

Lambda School offloaded code reviews from its professional instructors to teaching assistants. Eventually, a majority of them were done away with altogether. Farmer warned me that these cost cutting measures are a result of the pressures of trying to scale. "Once you try to scale past the limit of teacher to student, you become more and more just a video library and cool discord channel," he told me.

Despite their problems, Speakman told me it is noble to post placement results in public when the norm in education is to sidestep the question of job placement altogether. However, for this transparency to have meaning these results need to be standardized and trustworthy, which isn't possible so long as every large school posts results that are inconsistent in slightly different ways.

Unfortunately, large bootcamps are now stuck in a prisoner's dilemma — no school wants to be the first one to post "clean" stats while others continue to fudge their figures. As Speakman told me, "You better learn to self-regulate, because you're not going to like external regulation when it happens."