High Performance Git

I'm only on to chapter two and already it's explained some plumbing details that I somehow have missed all these years. This is great

> LFS adds its own operational overhead.

Seemingly seconds on every remote-touching command, even on a very small repo.

ted nyman: #1 most knowledgable college football fan in sf

and also git

which makes more sense i guess

I've been wanting to ask this:

Why isn't

    git clone --depth 1 ...

the default?

I would guess that for at least 90% of the repos I clone, I just want to install something. Even for the rest, I might hack on the code but seldom look into the history. If I do then I could do a `git fetch` at that point and save the bandwidth and disk space the rest of the time.

I've been wanting to ask this:

Why isn't

    git clone --depth 1 ...

the default?

A question: why is git involved at all in this? You don't want a repository.

What if that's only you? Git isn't made only for those who "just want to install something"

Its not the default because that'd be counter-productive to developers who use git with larger repositories, which is how git started life in the first place - your clone depth would be entirely useless for Linux kernel developers, for example, if it were default ..

I'm only on to chapter two and already it's explained some plumbing details that I somehow have missed all these years. This is great

Thanks much. I hoped that chapter would stand well on its own. That and the packfile chapter were the first I wrote, partly as a reference for myself!

> LFS adds its own operational overhead.

Seemingly seconds on every remote-touching command, even on a very small repo.

What is worse is that for about half a year or so, I now have to authenticate my ed25519-sk key with my Yubikey thrice (!) when using LFS. On every push.

That they didn't go with git annex was such a fit of NIH of a mistake.

ted nyman: #1 most knowledgable college football fan in sf

and also git

which makes more sense i guess

Of most things, really, he was on Jeopardy for a reason! https://thejeopardyfan.com/tag/ted-nyman

A question: why is git involved at all in this? You don't want a repository.

This! The default was to have a link to download a tarball of the source. And if the user wanted to contribute (or check the devel version), you would add a link to the vcs.

What is worse is that for about half a year or so, I now have to authenticate my ed25519-sk key with my Yubikey thrice (!) when using LFS. On every push.

Thanks much. I hoped that chapter would stand well on its own. That and the packfile chapter were the first I wrote, partly as a reference for myself!

Of most things, really, he was on Jeopardy for a reason! https://thejeopardyfan.com/tag/ted-nyman

That they didn't go with git annex was such a fit of NIH of a mistake.

Both have their advantages and disadvantages. git-annex is not strictly better, LFS just chose different tradeoffs.

What if that's only you? Git isn't made only for those who "just want to install something"

Thanks. That's great! I especially like that it then lazy loads the blobs as you need them.

I was going to ask if there's a way to set that as the default but I guess I'll just set up an alias like I have for most of the subcommands I use daily.

This! The default was to have a link to download a tarball of the source. And if the user wanted to contribute (or check the devel version), you would add a link to the vcs.

Grabbing git repos instead of just tarballs is useful.

A) You can update them, because you can git pull to fetch changes.

B) If you want to apply patches on top, its better to have version control so you can keep track of what you changed, especially useful if you want to rebase.

I think gitignore solves a problem that is hard to solve with the traditional tarball approach.

Downloading a tarball and running ./configure or make, editing a config file here or there, etc then running `make install` is the most common flow. Now days I find myself frequently editing the Dockerfile to make it to my liking. With a git repo, the owners of the repo have excluded all the local files, build caches, etc and you can keep pulling to get updates stashing and reapplying your local changes. With tarballs, you have to figure it out all over again. Lose your build cache (language dependent maybe), lose a change you made here or there, etc.

Both have their advantages and disadvantages. git-annex is not strictly better, LFS just chose different tradeoffs.

Thanks. That's great! I especially like that it then lazy loads the blobs as you need them.

I was going to ask if there's a way to set that as the default but I guess I'll just set up an alias like I have for most of the subcommands I use daily.

I think gitignore solves a problem that is hard to solve with the traditional tarball approach.

Grabbing git repos instead of just tarballs is useful.

A) You can update them, because you can git pull to fetch changes.

B) If you want to apply patches on top, its better to have version control so you can keep track of what you changed, especially useful if you want to rebase.

A) only valid if you want to stay with the devel version

B) See A

I use OpenBSD and before that, I was on alpine, debian, and arch. Of it was a software I want to try, I downloaded the tarball. if it’s something I wanted to keep for longer, I created a port or a custom packages.

A) only valid if you want to stay with the devel version

B) See A

Pencil sketch of a sailboat moored near a dock with shoreline buildings in the distance.

Git looks like a version-control tool. It is also a content-addressed database, a filesystem cache, a graph walker, and a transfer protocol.

This book is about those layers and the performance costs of each one. It starts with objects, refs, the index, and history traversal, then moves outward into packfiles, maintenance, sparse working trees, partial clone, transport, repository scale, diagnosis, configuration, and recovery.

It is written for engineers who need Git to stay fast as repositories, histories, and teams get larger: build and CI engineers, monorepo owners, developer-experience teams, and the people who wind up debugging strange Git behavior when the easy explanations stop working.

Section 0 · Introduction

Introduction

Section I · Foundations

Why Git gets slow, what Git stores, and how refs and the index steer through it.

Section II · History and Rewrite

How Git walks history and how rewrite commands reshape it without mutating commits.

Section III · Storage and Local Scale

Object storage, index cost, maintenance, and the techniques that shrink local state.

Section IV · Large-Repo Operations, Transport, and Scale

Clone shape, transfer policy, parallel work with worktrees, repository size, and ref scale.

Section V · Diagnosis and Recovery

How to instrument Git, find the slow layer, apply high-leverage settings, and recover when the repository is actually wrong.

Hacker Times