https://eric-fritz.com/Recent content on Eric FritzHugo -- gohugo.ioen-usFri, 13 Dec 2024 00:00:00 +0000https://eric-fritz.com/articles/2024-12-13-fix-a-top-cause-of-slow-queries-in-postgresql/Fri, 13 Dec 2024 00:00:00 +0000https://eric-fritz.com/articles/2024-12-13-fix-a-top-cause-of-slow-queries-in-postgresql/https://eric-fritz.com/articles/my-ai-junior-dev-still-needs-an-adult-in-the-room/Tue, 26 Nov 2024 00:00:00 +0000https://eric-fritz.com/articles/my-ai-junior-dev-still-needs-an-adult-in-the-room/The past two years has been a constant and exhausting barrage of AI hype. At the beginning, I was ready to dismiss the utility of AI coding assistance entirely. My employer at the time threw all available energy chasing this trend. It sucked all the air out of the room and extinguished all existing efforts. I didn’t understand why. Back then, GitHub Copilot was the state-of-the-art AI assistant, but my experience with it was a net negative on my productivity.https://eric-fritz.com/articles/box-muller/Thu, 24 Oct 2024 00:00:00 +0000https://eric-fritz.com/articles/box-muller/When trying to generate fake data for a database performance test, I found myself falling down a rabbit hole researching how to create random variables that fall into a specific probability distribution. Finding myself with a solution but without intuition, I ended up creating an interactive visualizer to help learn more deeply what was going on. I encourage you to both read on and take the visualizer for a spin!https://eric-fritz.com/articles/hosting-a-blog-like-its-2024/Mon, 07 Oct 2024 00:00:00 +0000https://eric-fritz.com/articles/hosting-a-blog-like-its-2024/This blog is built by the Hugo static site generator and hosted by Render, along with a number of other sites I maintain with an identical deployment. Before joining Render and discovering how freakin’ slick it is for hosting static sites, I had massively over-engineered my own patchwork solution. I had built a Zastava Yugo from spare parts when I could’ve just called an Uber. This article contrasts two approaches to deploying this blog:https://eric-fritz.com/articles/sourcegraph-went-dark/Mon, 19 Aug 2024 00:00:00 +0000https://eric-fritz.com/articles/sourcegraph-went-dark/Towards the end of my mid-2019 job search, I was down to joining the Google Go team or Sourcegraph. Sourcegraph ultimately won due to cultural factors - the most important of which was the ability to build 100% in the open. All documents were public by default. Technical and product RFCs (and later PR/FAQs) were drafted, reviewed, and catalogued in a public Google Drive folder. All product implementation was done in public GitHub repositories.https://eric-fritz.com/articles/i-am-abusive-to-git/Mon, 17 Jun 2024 00:00:00 +0000https://eric-fritz.com/articles/i-am-abusive-to-git/When I was working as a Staff Engineer at Sourcegraph, I performed a lot of “exploratory coding” that helped determine if some nascent idea is able to stand on its own legs. If the idea was validated with sufficient confidence, it would be polished, productionized, and shipped. Sometimes the polish ended up taking the majority of the effort. Often times these ideas were backtracked and abandoned and its learnings taken back to the team to amend our frontier ideas.https://eric-fritz.com/articles/2024-06-13-postgresql-stories-taking-random-samples-from-big-tables/Thu, 13 Jun 2024 00:00:00 +0000https://eric-fritz.com/articles/2024-06-13-postgresql-stories-taking-random-samples-from-big-tables/https://eric-fritz.com/articles/2024-04-03-postgresql-stories-a-simple-query-with-a-big-problem/Wed, 03 Apr 2024 00:00:00 +0000https://eric-fritz.com/articles/2024-04-03-postgresql-stories-a-simple-query-with-a-big-problem/https://eric-fritz.com/articles/2024-05-15-postgresql-stories-slow-query-to-fast-via-stats/Wed, 03 Apr 2024 00:00:00 +0000https://eric-fritz.com/articles/2024-05-15-postgresql-stories-slow-query-to-fast-via-stats/https://eric-fritz.com/articles/llm-antihallucinogen/Sun, 21 Jan 2024 00:00:00 +0000https://eric-fritz.com/articles/llm-antihallucinogen/This article was originally written on July 3rd, 2023 but was never officially published. I am publishing this here as a historic snapshot of an interesting technical project, though the details of this article are likely out of date and no longer indicative of the current capabilities or development methodology of Cody, Sourcegraph's AI Coding Assistant. Within the rapidly expanding domain of artificial intelligence, the capacity for Large Language Models (LLMs) to sift through extensive codebases and provide insightful responses has evolved into a critical necessity for today’s software developers.https://eric-fritz.com/articles/compressing-source-code-ranges/Wed, 17 Jan 2024 00:00:00 +0000https://eric-fritz.com/articles/compressing-source-code-ranges/Since the dawn of time, mankind hath sought to make things smaller. One of the favorite little pieces of code I wrote during my time at Sourcegraph was an EncodeRanges function that took a series of offsets into a text document and compressed them into a tight, tiny, opaque ball of bits. At the time, I was designing a new set of Postgres tables and indexes that would store a quick way to go from a globally unique symbol (name of a type, a variable, etc) to all of the source code locations that relate to it.https://eric-fritz.com/articles/home-office-setup/Fri, 29 Dec 2023 00:00:00 +0000https://eric-fritz.com/articles/home-office-setup/I recently ended a four (plus) year stint at a geographically distributed, all-remote, async-first startup, and began a similar remote role at another slightly earlier stage SF-based startup. I took the two weeks between jobs to tidy up my home office to clean my mental slate. It’s like cleaning your dorm between terms: new semester, new me. I snapped a few pictures to immortalize the absence of dust and clutter on my desk and was reminded of an old Sourcegraph blog post from 2020 where the entire company (which was very small at the time) shared their home office setup.https://eric-fritz.com/articles/2023-07-03-llm-antihallucinogen/Mon, 03 Jul 2023 00:00:00 +0000https://eric-fritz.com/articles/2023-07-03-llm-antihallucinogen/https://eric-fritz.com/articles/ranking-in-a-week/Mon, 21 Nov 2022 00:00:00 +0000https://eric-fritz.com/articles/ranking-in-a-week/Last week we announced that we are now ranking search results on https://sourcegraph.com in order to prioritize relevant as well as reusable code. We consider this Sourcegraph’s code intelligence platform’s first major victory of many, and a booming herald for a new era of code search. The effort to bring ranked results to our public instance was a concerted effort of four developers across two teams (Search and Code Intelligence), but it took only a week of wall-clock time to design, implement, and deploy.https://eric-fritz.com/articles/multi-version-upgrades/Tue, 25 Oct 2022 00:00:00 +0000https://eric-fritz.com/articles/multi-version-upgrades/It’s finally happened. The most obnoxious restriction (by a very wide margin) related to administration of a Sourcegraph instance no longer applies. Sourcegraph instances (v3.20 or newer) are now able to upgrade directly to a future release, without the requirement to hit every minor release along the way. A recent peek at our customer instance distribution showed that 70% of our self-hosted customers are on a “historic” release of Sourcegraph (older than two months).https://eric-fritz.com/articles/2022-05-18-how-to-stop-autovacuum-from-sucking-up-your-performance/Wed, 18 May 2022 00:00:00 +0000https://eric-fritz.com/articles/2022-05-18-how-to-stop-autovacuum-from-sucking-up-your-performance/https://eric-fritz.com/articles/fixing-broken-migrations/Tue, 22 Feb 2022 00:00:00 +0000https://eric-fritz.com/articles/fixing-broken-migrations/We have recently identified the root cause of a frequent, insidious pain point related to database migrations during Sourcegraph version upgrades. Many users handling the installation and upgrade of their Sourcegraph instance can likely attest to seeing an error similar to the following midway through an upgrade from one minor release to the next. ERROR: Failed to migrate the DB. Please contact support@sourcegraph.com for further assistance: Dirty database version 1528395797. Fix and force version.https://eric-fritz.com/articles/optimizing-commit-graph-part-2/Fri, 04 Jun 2021 00:00:00 +0000https://eric-fritz.com/articles/optimizing-commit-graph-part-2/In Part 1 of this optimization story, we detailed how Sourcegraph can resolve code intelligence queries using data from older commits when data on the requested commit is not yet available. The implementation lies completely within PostgreSQL, and the queries run with very low latency (< 1ms). We boldly claimed that our fears of scalability were no longer cause for concern. Turns out that claim was a half-truth (if not a lie) as Sourcegraph solves the problem well, but only at a particular scale.https://eric-fritz.com/articles/optimizing-commit-graph-part-1/Thu, 27 May 2021 00:00:00 +0000https://eric-fritz.com/articles/optimizing-commit-graph-part-1/Sourcegraph’s Code Intelligence team builds tools and services that provide contextual information around code. These enable users to perform fast, comprehensive, and accurate code navigation, and to surface dependency relationships across projects, repositories, and languages. In this post I’ll dive into how Sourcegraph can resolve code intelligence queries using data from older commits when data on the requested commit is not yet available. In Part 2, we’ll cover how additional scalability concerns presented themselves and how we tackled them.https://eric-fritz.com/articles/postgres-version-upgrade/Wed, 31 Mar 2021 00:00:00 +0000https://eric-fritz.com/articles/postgres-version-upgrade/As of Sourcegraph 3.27 (released April 20, 2021), we’re updating the minimum supported version of Postgres from 9.6 to 12. If you are maintaining an external database and your Postgres version is older than Postgres 12, you will need to update your database instance prior to upgrading from Sourcegraph 3.26 to 3.27. See the following instructions for a step-by-step guide. If you are using the Sourcegraph maintained Docker images, your instance is already using an appropriate Postgres version and no action is needed on your part.https://eric-fritz.com/articles/migrating-to-postgres/Sun, 18 Oct 2020 00:00:00 +0000https://eric-fritz.com/articles/migrating-to-postgres/At Sourcegraph, we have recently decided to migrate code intelligence data that has historically been stored in thousands of SQLite databases on disk into a single Postgres instance. Long story short, we were running into some limits of writing a persistence layer over SQLite databases in a consistent way. We were unable to horizontally scale, as one database can only be opened by one backend at a time. We would need to implement a sharding mechanism in which data is spread out evenly across multiple machines, and requests could be routed to (one of) the backend(s) with that data.https://eric-fritz.com/articles/optimizing-indexer/Mon, 07 Sep 2020 00:00:00 +0000https://eric-fritz.com/articles/optimizing-indexer/We (Sourcegraph’s Code Intelligence team) recently made Go code intelligence faster, especially on very large repositories. For example, we cut the indexing time by 95% for the huge Go AWS SDK repository, from 8 minutes to 24 seconds. Here’s how we did it. Background: what is code intelligence? Developers use Sourcegraph for code search and navigation. When you’re navigating code on Sourcegraph, you get hovers, definitions, and references to help you along the way.https://eric-fritz.com/articles/lsif-backend-evolution/Wed, 17 Jun 2020 00:00:00 +0000https://eric-fritz.com/articles/lsif-backend-evolution/Jumping to the definition of a symbol under your cursor and finding all its references are two of the basic mental mechanics of software engineering. Fast code navigation accelerates the rate at which you can build a mental model of the code, and when it’s available, you’re likely to use it hundreds, if not thousands, of times per day. Code navigation is the core of how Sourcegraph helps you understand the parts of the universe of code that are most relevant and important to you.https://eric-fritz.com/articles/optimizing-a-code-intel-backend/Wed, 17 Jun 2020 00:00:00 +0000https://eric-fritz.com/articles/optimizing-a-code-intel-backend/When it comes to developer tools, speed is a critical feature. The difference between a 100ms, 1s, and 10s delay fundamentally alters user psychology—it’s the difference between coding at the speed of thought vs. losing focus as your mind wanders while waiting for the UI to respond. One of Sourcegraph’s magic powers is its ability to provide compiler-accurate code navigation in completely web-based interfaces: Sourcegraph.com, private Sourcegraph instances, and on GitHub, GitLab, Bitbucket, and Phabricator via the Sourcegraph browser extension.https://eric-fritz.com/articles/works-on-my-machine/Sat, 28 Dec 2019 00:00:00 +0000https://eric-fritz.com/articles/works-on-my-machine/As described in a previous article, Sourcegraph’s main source of truth for all code data is gitserver, which is a sharded RPC service wrapping git commands. Requests to this service specify a repository name, which indicates the directory to perform the git command in, as well as the arguments that should be passed to the command. The LSIF service needs to fetch commit ancestry data from gitserver, which it gets through a formatting the output of git log.https://eric-fritz.com/articles/easy-peasy-sql-audit-tables/Fri, 27 Dec 2019 00:00:00 +0000https://eric-fritz.com/articles/easy-peasy-sql-audit-tables/Deposition (described in a previous article) is a tool used to track the dependencies and vulnerabilities of a software project. This product belongs to a class of tools that benefits greatly from the existence of an audit log. Most of the time, modifications to data via the API are performed by a build user in the context of a continuous integration system. The API and the UI does allow data to be altered and overridden.https://eric-fritz.com/articles/subtle-circular-import-bug/Tue, 24 Dec 2019 00:00:00 +0000https://eric-fritz.com/articles/subtle-circular-import-bug/Sourcegraph’s main source of truth for all code data is gitserver, which is a sharded RPC service wrapping git commands. The LSIF service requires git ancestry data in order to answer queries such as find the commit closest to $c$ for which we have LSIF data, or determine if commit $c$ (for which we have LSIF data) is the closest such commit to the HEAD of master. Each query to the LSIF server referencing a previously unknown commit forces a query to gitserver for the first $n$ ancestors of the commit.https://eric-fritz.com/articles/found-a-golang-bug/Sun, 03 Feb 2019 00:00:00 +0000https://eric-fritz.com/articles/found-a-golang-bug/I have to say, this was a cool milestone in my career. Before we start this story, I need to point out that if you think that your application is behaving in a strange way due to a compiler bug, a programming language runtime bug, an OS bug, or a hardware bug, you’re almost certainly displaying a disproportionate amount of hubris. But sometimes you just hit those weird edges. On January 24th, I was blocked by a Bamboo build that just wouldn’t finish.https://eric-fritz.com/articles/testing-patterns-in-go/Fri, 20 Jul 2018 00:00:00 +0000https://eric-fritz.com/articles/testing-patterns-in-go/This article outlines some patterns of unit testing in Golang that I have found, over time, to work extremely well. he testing code here assumes the use of the gomega assertion library and the sweet test-suite runner library. Mocking Interfaces Consider the following definition of the function WithCache that memoizes the result of a function using a cache instance that conforms to the (simple) interface defined in gache. The implementation should be non-surprising: the cache fetch always happens, the target function is only invoked when a cache miss occurs, and whenever the target function is invoked is value is written back into the cache.https://eric-fritz.com/articles/arrow-structured-concurrency/Tue, 01 May 2018 00:00:00 +0000https://eric-fritz.com/articles/arrow-structured-concurrency/This article adapts some ideas from Arrows for JavaScript for use as concurrency primitives for some high-level language as a response to Nathaniel Smith’s article Notes on structured concurrency, or: Go statement considered harmful published on April 25, 2018 and Martin Sústrik’s response Structured Concurrency in High-level Languages published on April 28th, 2018. In the first half of the article I detail some of the more interesting semantics of a ArrowsJS.https://eric-fritz.com/articles/deposition/Mon, 16 Apr 2018 00:00:00 +0000https://eric-fritz.com/articles/deposition/Impetus In Autumn 2017, we had an incident involving multiple services crashing and failing to re-initialize. We quickly correlated the time of the crashes with the time our Elastic Stack became unavailable. We then determined that our logging library did not put a maximum capacity on the number of log messages in the publish queue (see gomol issue #20), causing the memory usage of the application to rise when logs could not be pushed out.https://eric-fritz.com/articles/linear-mtf-hash-table/Wed, 30 Aug 2017 00:00:00 +0000https://eric-fritz.com/articles/linear-mtf-hash-table/Vehicular routing is a very interesting real-world problem that contains two distinct but interacting combinatorial optimization problems: the travelling salesman problem (TSP) and the 0-1 knapsack problem (KP). This unison of problems, unfortunately, creates a lot of opportunity for local maxima. Finding a better route for a particular truck may decrease the optimality of the contents fo the truck, and vice versa. RedPrairie’s approach to this problem was to approximate some solution, then iteratively and stochastically apply small transformations to the solution.https://eric-fritz.com/articles/an-adventure-in-unstable-apis/Mon, 14 Aug 2017 00:00:00 +0000https://eric-fritz.com/articles/an-adventure-in-unstable-apis/Antirez announced the Redis module system in May 2016. At this point in time, Charon’s Redis interface was a set of Lua scripts which was the best available solution for atomicity and bandwidth at the time. Because scripts simply invoke other Redis commands, the data model had to be designed carefully to fit into Redis’s supported datatypes. As a result, each script modified up to a few dozen keys and made debugging (which usually required inspection of values in the database) difficult.https://eric-fritz.com/articles/charon-simulator/Sat, 12 Aug 2017 00:00:00 +0000https://eric-fritz.com/articles/charon-simulator/Configuring rate limits Charon, as described in the whitepaper, is a cooperative system used to enforce limited access to resources on behalf of users. For example, an HTTP API server can choose to immediately return an HTTP 429 response if the limited resource has had a large number of requests from that user in its recent history. This can help save, for example, expensive database operations which are a much more precious resource than server cycles handling request (and rate limit) middleware.https://eric-fritz.com/articles/on-monads/Thu, 10 Aug 2017 00:00:00 +0000https://eric-fritz.com/articles/on-monads/Just kidding.