Codecov uses Sourcegraph to resolve incidents faster

In 2021, security engineers Mitchell Borrego and Jeff Holland joined Codecov with the goal of creating a cutting-edge security program. Their responsibilities include security tooling, compliance, and code review from a security perspective. Working with Jerrod Engelberg, CEO of Codecov, the team is developing a world-class, and ever-improving, security program.

Key to their efforts? Sourcegraph. According to Jeff, “Sourcegraph allows us to be more efficient with our time, whether it's code review, quickly answering security-related questions from clients, or searching for things in the code more easily than we could through other tools.”

Codecov reduced time to resolution with 100% confidence when facing Log4j

In December of 2021, researchers discovered that Log4j, an otherwise nondescript open source logging library, had a security vulnerability so severe it scored a 10/10 on the CVSS scale. The use of the library was, as the FTC put it, "ubiquitous."

Software companies the world over scrambled to figure out whether Log4j was somewhere in their codebases and how fast they could patch it. For many companies, it was a stressful, high-pressure event, one in which they had to ask developers to work extra hours. Despite their efforts, many companies were unable to have full confidence that they had remediated all affected code.

Not Codecov. With Sourcegraph, said Mitchell, Codecov was able to “get a quick reconnaissance and understand our possible exposure by doing some simple searches.”

“With Sourcegraph, we confirmed in 5 minutes, and sanity-checked in another 5 minutes, with 100% assurance, that we weren't exposed to Log4j in our codebase,” reported Mitchell.

“Confidence is everything,” said Holland. “It's extremely important, the 100% confidence that you can go out in good faith to your customers and report the absence of a vulnerability.” Other companies, according to Holland, instead had to temper expectations and some even had to walk back assurances. “It deeply affects the trust that you can provide to customers,” Holland said.

Log4j was a particularly severe instance of a common pattern: companies adopt an open source code component, often packaged with other components, not realizing it contains a vulnerability. These kinds of vulnerabilities are on the rise but with Sourcegraph, Codecov is prepared. The next time a vulnerability emerges, Codecov can find all instances of it-if any-in their codebase with just a search.

“If Log4j version 7 comes out and all of the sudden it affects us and we've got a can of worms on our hands, what do we do? Start searching with Sourcegraph,” said Jeff. “And pretty quickly, we can figure out if we're vulnerable and patch. And hopefully we avoid an incident versus floundering around trying to use other code search tools.”

Codecov makes code review secure and comprehensive

When he's not responding to the latest vulnerability, one of Mitchell's primary responsibilities is performing security reviews on PRs and ensuring that other developers don't accidentally merge insecure code.

Mitchell reported that before Sourcegraph, the ability to search their codebase or their PRs for security flaws “was either nonexistent because it was so inefficient or there was just no conceivable way to do it.”

Code reviews are now comprehensive, which is helpful over and above security reviews. “Sourcegraph makes code reviews a lot more thorough,” Mitchell said. “And you have faith that you don't have to go back and confirm, double check, and triple check.”

By the time he's done reviewing code, Mitchell is confident: “I have more assurance in the effectiveness of my own code reviews. I have more faith that it is an accurate code review and that nothing is getting past me.”

Codecov speeds up developer onboarding and saves senior developers time and effort

Having experienced the pain of onboarding in a new company without Sourcegraph, Mitchell is impressed by the speed at which new developers are now able to onboard and start shipping.

“With Sourcegraph, onboarding is certainly faster and certainly better,” said Mitchell. Even well-documented codebases aren't immediately intuitive, so Sourcegraph can help onboarding developers understand the big picture as well as the nuances of their codebases. “It provided a significant value to us in understanding our codebase at large,” Mitchell said.

Sourcegraph helps developers help themselves and each other. “I want developers to only have to help each other when necessary,” said Jerrod. Sourcegraph searches have helped Codecov developers answer questions ahead of time, before they have to ask for help and wait, often across time zones, for a response.

Efficiency is only half the benefit. According to Jerrod, “We increased the interpersonal trust and we increased the interpersonal dynamics in the company, making people feel more welcomed.”

Asking for help, of course, isn't a practice exclusive to early career developers. “Sourcegraph will not only help with their onboarding,” said Mitchell. “It'll help post-onboarding, increasing the developers' usability across the repositories we have.”