Using Git's rerere feature to escape recurring conflict hell

I never get conflicts during a merge because I only ever merge in one direction. I get all my conflicts on branches because I rebase before merging. I started doing this years and years ago because I kept coming across these mysterious silent regressions with my team. I searched something like "git merge silent regressions" and came across this stackoverflow answer:

https://stackoverflow.com/a/28510260

That completely fixed the problem. Now I only ever get conflicts on my feature branches. The rule is always: rebase away from main, and merge towards main. All conflicts are then on your branches, never on main, and always from rebase, never from merge. Then I set the pull behaviour to rebase, too.

I've never had a silent regression since, and never had a problematic conflict scenario.

I did recently learn about ORIG_HEAD though which was very cool, because I had accidentally rebased to main instead of to a dev branch from which I had created a bunch of worktrees, then when I merged back to the dev branch all hell broke loose, and I learned that I could revert a merge by checking out ORIG_HEAD:

https://icinga.com/blog/undo-git-reset-hard/

  #~/.config/git/config
  [rerere]
      enabled = true
      autoUpdate = true

while you're editing git config, consider these:

    [pull]
      rebase = true

  [rebase]
      autoSquash = true
      autoStash = true

  [merge]
      # zdiff3 adds original text markers and removes matching lines from conflict regions
      # https://git-scm.com/docs/git-config#Documentation/git-config.txt-mergeconflictStyle
      conflictStyle = zdiff3
      autoStash = true

  [push]
      autoSetupRemote = true
      default = simple

  [init]
      defaultBranch = main

Do people really merge left and right between branches? Tell me if I got this wrong, this is how I work.

You got 4 devs. Each branched off from master. And we never merge from each other. Suppose 3 other people merged to master I pull it from master and fix only those conflicts. I’m not bringing your code into my branch unless it’s already finished and on master. If I need something you’re working on and it’s not on master when I need it, it’s a much larger planning issue.

If you have multiple environments before a stable master, do it only in one direction: feature > dev > staging > master. Don’t merge branches straight into staging or master.

I thought this was how everyone worked.

I'm using these options https://blog.gitbutler.com/how-git-core-devs-configure-git and I'm happy with it

I use rerere when "forced" by team to use rebase. It not even work that much at the end (you can't control workflows outside yourself, that is why git ux is wrong: it desperately need total discipline).

THEN, I move to jujutsu. Only has a few problems at start trying to use it as git, but after get the idea, all fine.

BTW, this was with the same team and they never know, so JJ is in fact better: It survive OTHERS workflows.

Every time I think I am adept in git, something like this is shown to me. I really should read into it more lol.

I'm using these options https://blog.gitbutler.com/how-git-core-devs-configure-git and I'm happy with it

Every time I think I am adept in git, something like this is shown to me. I really should read into it more lol.

https://stackoverflow.com/a/28510260

I've never had a silent regression since, and never had a problematic conflict scenario.

https://icinga.com/blog/undo-git-reset-hard/

I follow this approach and still get the same merge conflicts coming repeatedly while doing rebase.

Let's say in my feature branch in the first commit I change a line in a file which also gets changed in the main branch. Then I have done 3-4 change commits doing changes in the same file. Now while doing rebase, I will have to resolve this conflict 3–4 times again and again while git re-applies commit one by one, during rebase.

I think I get this sometimes even if rerere is enabled, I am doing rebase using Intellij though, so maybe rerere doesn't get used here somehow or maybe diff context changes, so rerere is not applicable.

I'm also like this, rebasing feature branches onto main - I however have one suggestion when it comes to the push back up to origin

Instead of

`git push --force`

always use

`git push --force-with-lease`

https://git-scm.com/docs/git-push

This probably should be the default in git (as in there should be a `git push --force-without-lease` instead) and asks git to make sure the commits locally on your branch are up-to-date with those on remote/origin. It then fails if you try to overwrite commits that you haven't seen, and has saved me a few times when working between computers on the same project when i could have lost history on the remote that i failed to fetch.

If you squash merge PRs, this is equivalent to merging master back into your feature branch before merging to master.

I do that a lot to avoid commits mutating mid-review, so you avoid having to force push over reviewed commits (which is a sin)

rerere is still useful here to handle merge conflicts after repeated rebases.

I've never even seen someone suggest a rebase master onto feature workflow! TIL.

This is what I've been doing for years. It's remarkably stress-free!

  #~/.config/git/config
  [rerere]
      enabled = true
      autoUpdate = true

while you're editing git config, consider these:

    [pull]
      rebase = true

  [rebase]
      autoSquash = true
      autoStash = true

  [merge]
      # zdiff3 adds original text markers and removes matching lines from conflict regions
      # https://git-scm.com/docs/git-config#Documentation/git-config.txt-mergeconflictStyle
      conflictStyle = zdiff3
      autoStash = true

  [push]
      autoSetupRemote = true
      default = simple

  [init]
      defaultBranch = main

addendum

these are changes tacked to my git config over time. they're the result of working in a constant crunch state, where smooth forward move was more import than the state of the git history you left behind. these options remove small annoying road bumps. you should avoid some of them if you're working under different constraints, or the commit process differs such that these options don't apply.

it all starts with `pull.rebase=true` it makes `git pull --rebase` the default behavior. then comes `rebase.autoStash`, which just wraps the rebase with a stash push/pop envelop. if rebase is not your thing, and you prefer merge, `merge.autoStash=true` works the same. finally `push.autoSetupRemote=true` will skip asking you to set a remote tracking branch, it makes `git push` default to `git push --set-upstream`.

    [pull]
      rebase = true

Don't use this one unless you really know what you're doing. Multiple co-workers have gotten into really bizarre rebases because of it (like rebasing 70+ commits from master on top of their branch instead of the other way around), it seems to cause more problems than it solves.

The man page for "git pull" even has a warning about using this flag.

I believe push default simple is the default in git now and does not need to be explicitly set.

Do people really merge left and right between branches? Tell me if I got this wrong, this is how I work.

If you have multiple environments before a stable master, do it only in one direction: feature > dev > staging > master. Don’t merge branches straight into staging or master.

I thought this was how everyone worked.

I thought this was how everyone worked.

I wish it were :)

This is the right way to do it. Whether using trunk-based development, git-flow, etc -- you're controlling the flow of merges in a particular direction.

However, I think the "larger planning issue" is harder to easily avoid when you have more devs, more changes, or the AI-boosted output we have today. If feature B requires feature A, and feature A isn't up-to-date with main, I could rebase feature A to main, then feature B to feature A. When feature A is merged and we're ready to PR feature B, I can rebase to main again then make my PR.

Yeah, I absolutely never run into this problem.

Sometimes we will have a huge stack of changes one of us is "finished but not clear to merge".

either:

- We just swap ownership of the branches and eng 2 now commits directly in branch 1. We review the final content together and typically pull in a 3rd person to review our combined work. Eng 1 either pairs with eng 2 until its finished or starts on a task that is decoupled from those changes.

- We use an integration branch that gets threated like the temporary master branch until the feature is ready to merge.

when you're not just doing small patches (like Greenfield work), and you have multiple developers, it can be unreasonable and wasteful to do what you describe. say we're redoing the way memory management it being done, or changing a foundational api. A and B could go through master, but they'd have to redo a whole bunch of work. Or we could make an integration branch where A and B hash things out together and only push to master when their work is done. you can also see how this allows a lot more back and forth about the places where the designs interact rather than 'I got in master before you, so you get to eat a multi-day merge and I'm outta here'

this kind of interaction shows up more in a 'release' based model than CI environment, and one where the cost of testing is non-zero (because we're actually doing testing). I know that's not a very popular model, but not all software development is CI websites.

THEN, I move to jujutsu. Only has a few problems at start trying to use it as git, but after get the idea, all fine.

BTW, this was with the same team and they never know, so JJ is in fact better: It survive OTHERS workflows.

You should not be seeing the same conflicts over and over again in a situation in which you are on a single branch, just rebasing your unpublished commits.

You might get conflicts regularly, if people are touching the same area of the code that you are touching, but not the same ones.

rerere mainly comes into play when people have to backport commits into multiple similar branches, due to having a heavily forked git landscape.

No idea why this was downvoted. This is one of many ways in which jj is far better than raw dogging git

I follow this approach and still get the same merge conflicts coming repeatedly while doing rebase.

If you squash merge PRs, this is equivalent to merging master back into your feature branch before merging to master.

I do that a lot to avoid commits mutating mid-review, so you avoid having to force push over reviewed commits (which is a sin)

This is what I've been doing for years. It's remarkably stress-free!

addendum

I'm also like this, rebasing feature branches onto main - I however have one suggestion when it comes to the push back up to origin

Instead of

`git push --force`

always use

`git push --force-with-lease`

https://git-scm.com/docs/git-push

--force-with-lease serves no purpose.

If you are sure that the repo you are pushing to is a stable target (nobody else is accessing it), you just use --force.

If the repo you are pushing to is a moving target, you ... don't force push to it. Or else you warn all the repo users that you are about to rewrite history. Which means they not only should refrain from pushing, but have to be prepared for a second announcement which informs them that the rewrite is done; they must then fetch the rewritten head and fix up their unpublished work against the non-fast-forward change.

rerere is still useful here to handle merge conflicts after repeated rebases.

As someone who tried rerere and didn't see the point:

How? Usually I rebase the same branch multiple times onto different, but successive commits of the master branch. But after I solved a bunch of conflicts of the first rebase, I shouldn't have the same conflicts again in a second one, since the rebased branch contains the merged conflict. Rebasing again could only turn up new conflicts (with newer, other commits on the master branch).

How can I have the same conflict again for repeated rebases?

I've never even seen someone suggest a rebase master onto feature workflow! TIL.

I think the terminology would be the other way around, like you're rebasing the feature onto the main:

git checkout feature

git rebase main

git checkout main

git merge feature

that way you get all your conflicts on the feature branch during rebase and your merge is always clean.

> I get all my conflicts on branches because I rebase before merging

Pretty sure it's the other way around. You're on the branch and rebase it atop current master. If you merge after that, you won't have merge conflicts.

--force-with-lease serves no purpose.

If you are sure that the repo you are pushing to is a stable target (nobody else is accessing it), you just use --force.

    [pull]
      rebase = true

The man page for "git pull" even has a warning about using this flag.

Yeah, I absolutely never run into this problem.

Sometimes we will have a huge stack of changes one of us is "finished but not clear to merge".

either:

- We use an integration branch that gets threated like the temporary master branch until the feature is ready to merge.

You should not be seeing the same conflicts over and over again in a situation in which you are on a single branch, just rebasing your unpublished commits.

You might get conflicts regularly, if people are touching the same area of the code that you are touching, but not the same ones.

rerere mainly comes into play when people have to backport commits into multiple similar branches, due to having a heavily forked git landscape.

No idea why this was downvoted. This is one of many ways in which jj is far better than raw dogging git

I think the terminology would be the other way around, like you're rebasing the feature onto the main:

git checkout feature

git rebase main

git checkout main

git merge feature

that way you get all your conflicts on the feature branch during rebase and your merge is always clean.

> I get all my conflicts on branches because I rebase before merging

Pretty sure it's the other way around. You're on the branch and rebase it atop current master. If you merge after that, you won't have merge conflicts.

As someone who tried rerere and didn't see the point:

How can I have the same conflict again for repeated rebases?

The point is that some organizations have a chaotic development process consisting of numerous similar branches. Often there is a main trunk, and then branches that were made for particular product variants (like piece of hardware or whatever) and cut at a particular point in time, in order to isolate from the trunk.

What then happens is that when a bug is found that affects all branches, it must be cherry picked into all of them. If that cherry pick runs into conflicts, it is often the same conflicts, over and over again on each branch.

Of course, the fix is not to do that, but it's easier to say that than to get away from that kind of workflow once you are steeped in it up to the chin.

I know what you mean but doesn't that require squashing as well? If I have a branch with 5 commits, I think rerere helps me by only having to fix the conflict once, not potentially multiple times. I might be wrong here though.

I believe push default simple is the default in git now and does not need to be explicitly set.

I thought this was how everyone worked.

I wish it were :)

This is the right way to do it. Whether using trunk-based development, git-flow, etc -- you're controlling the flow of merges in a particular direction.

Of course, the fix is not to do that, but it's easier to say that than to get away from that kind of workflow once you are steeped in it up to the chin.

> This mode is the default since Git 2.0, and is the safest option suited for beginners.

the docs checkout, ty

That sounds like a lot of work. If you rebase A on main then B on main, you end up having to resolve A's conflicts on main twice. But if you stick with merge commits, A's conflicts are handled once in the merge from main.

Those big refactoring should be discussed with the whole teams so everyone understands how they’re going to be affected. And rework like this should probably be hidden behind an abstraction (to do it gradually).

> This mode is the default since Git 2.0, and is the safest option suited for beginners.

the docs checkout, ty

yes of course they should, that doesn't mean that integration branches aren't a good way to manage the tactical process. and often this kind of situation comes up when you're trying to do exactly that, introduce that abstraction that solves a whole bunch of issues, but requires cross-cutting changes.

the problem comes up the other way too. the person making the refactor can get stuck in a perpetual hell of constantly reapplying the changes to smaller deltas that are being committed daily to master.

I don’t remember which books exactly, but it was about legacy code. The author was talking about seams. If you’re trying to complete a refactor in one go, it will usually be arduous and maybe fail. What you want is to find seams in the codebase and maybe create some, where you can gradually decouple the architecture. After that you’ll have contracts and independent modules that can be reworked.

It’s not easy though and you need to have a very good understanding of the codebase. But the nice thing is that you can do it without having to maintain a long lived branch (which is an antipattern).

Have you ever tried to merge two branches only to end up in conflict hell? You fix a bunch of conflicts only to run git merge --continue and be presented with the same conflicts. Repeat this process and after a few iterations you give up because it just isn't worth the pain and effort.

Would you be surprised to know that there is a git feature specifically for this problem? It's called rerere and I'm going to enrich your life with it now. (I'm going to talk specifically about merging but I think it also helps rebasing)

rerere stands for Reuse Recorded Resolution. The TL;DR version is you ask git to remember how you've resolved hunks in the past, and if the same one comes up for a file in future just redo what you did last time.

To enable this feature just run this lovely command git config --global rerere.enabled true. You can also turn it on by creating this directory in your projects .git/rr-cache, although the global setting is much clearer.

I'll try to take you through an example of how this works, bear with me it might get long.

We have our (tiny) project with only one file in it, which looks like this

.
└── user.rb

0 directories, 1 file

I branch off master to create a branch called dev and I add a line to user.rb. Now I would like to stage this change so I pull down staging and try to merge my dev branch but uh oh, someone has merged a change to staging affecting the same line in user.rb that I am editing.

/tmp/example [staging] » git merge dev
Auto-merging user.rb
CONFLICT (content): Merge conflict in user.rb
Automatic merge failed; fix conflicts and then commit the result.

We've all seen this before, a run of the mill conflict message. However if you were to have rerere enabled you would get this output

/tmp/example [staging] » git merge dev
Auto-merging user.rb
CONFLICT (content): Merge conflict in user.rb
Recorded preimage for 'user.rb'
Automatic merge failed; fix conflicts and then commit the result.

You can now see a new line saying Recorded preimage for 'user.rb'. Running git rerere diff right now will give you the current state of the resolution file:

/tmp/example [c013552] » git rerere diff
--- a/user.rb
+++ b/user.rb
@@ -1,5 +1,5 @@
-<<<<<<<
-hello
-=======
+<<<<<<< HEAD
 hi
 ->>>>>>>
 +=======
 +hello
 +>>>>>>> commit from dev

You go about the usual conflict workflow, choose which changes to keep, and commit the result. If you run git rerere diff again, you see the recorded resolution:

/tmp/example [c013552] » git rerere diff
--- a/user.rb
+++ b/user.rb
@@ -1,5 +1 @@
-<<<<<<<
 hello
-=======
-hi
->>>>>>>

Running git merge --continue will apply your commit and tell you about the new resolution for our file:

/tmp/example [c013552] » git merge --continue
Recorded resolution for 'user.rb'.```
Let's undo that merge with `git reset --hard HEAD^` and merge again:

```/tmp/example [staging] » git merge dev
Auto-merging user.rb
CONFLICT (content): Merge conflict in user.rb
Resolved 'user.rb' using previous resolution.
Automatic merge failed; fix conflicts and then commit the result.
/tmp/example [staging] » git add .
/tmp/example [staging] » git merge --continue
[staging f4a7d36] Merge branch 'dev' into staging

The important line in this output is Resolved 'user.rb' using previous resolution.. I didn't need to even look at the file, just commit the result. This worked because git saw the conflict, looked in the rr-cache folder and recognised this hunk from this file from a previous merge and applied your decision from last time!

As always, I hope you found that useful.

Hacker Times

Hacker Times

Using Git's rerere feature to escape recurring conflict hell

Discussion

Discussion