#git cotree

[mentalisttraceur-software]

Does anyone have questions about git-cotree that they would like answered in a README, code comment, -‍-‍help output, or other documentation?

Obviously it needs a good big-picture "what even is this? / what does this do? / what's the benefit?" introduction, and until I provide that it won't be easy to know what you'd want to know.

So, to summarize:

git-cotree lets you have more than one branch/etc checked out into its own folder at once (in a nicer way than "git worktree");

git-cotree lets you turn your normal local repo copy into a folder where each branch/etc that you check out is its own subdirectory (same name as the branch) - it also lets you switch back to normal easily. It takes great care to preserve any uncommitted stuff in your working tree when making the change.

git-cotree is "now that I've seen it I can't unsee it: why didn't they always have each branch just be its own directory!?" (no reason that can't be solved by stuff like copy-on-write optimizations at the file system level).

[mentalisttraceur-incomplete]

There's this pattern I see in software which really bothers me.

People are far too willing or even eager to dismiss features/abstractions that empower doing something unless they have an example.

Most good functionality will have more uses than you could think of. You have to have a superpower before you can conceive of most ways it could be used.

Usually, at the moment that I recognize that something would be useful to implement, I can describe 0-2 examples for it, and those examples are almost always some niche thing which won't be compelling to most.

A few months ago, I implemented overwriting paste in my vi-like Emacs setup. I put the cursor on where I want the paste to start, I hit "gp", and it replaces the text starting from the cursor, until it runs out of characters to paste.

Vi-style editing in CLI/REPL prompts

My in-Emacs window management

Git cotree

I could probably never convince most people that this is useful to have. I could've never thought of, or remember, all the ways in which I use it. And yet I use it on average at least once a day.

[mentalisttraceur-software]

Today I improved "git-cotree --init" to handle several edge cases, primarily involving untracked and ignored files. Previously its support for untracked or ignored files was rather limited, covering only the most basic of cases.

But one big goal of git-cotree was to make switching from a normal clone to a nice multiple worktrees setup seamless, and this brings it a lot closer to that goal.

So I am now a lot more confident recommending git-cotree for others to use. I still don't know when I'll actually flesh out the README, but at least now the risk is a lot lower.

However, I'm not sure what happens if for example Git submodules or Git LFS are in the picture. But odds are now good that any remaining issues are due to limitations in how multiple Git worktrees interact with some other Git feature rather than the fault of git-cotree specifically.

[mentalisttraceur-software]

I don't exactly dislike writing documentation, and I don't at all mind the effort and thinking that it takes, but I just hate how much time it takes to write documentation even remotely well.

And it's frustrating because in a very important way, the person who wrote the code is the worst person to document it. So the time inefficiency is very high.

The best person to document software is someone eager to get an informal explanation from the person who wrote it.

When I have just written a piece of software, I am literally mentally as far possible from someone who needs the documentation. I know exactly what my software does. I don't have any questions. All features, quirks, and caveats of the interface and any side-effects are equally unsurprising.

To assume the perspective of a person who actually needs to be helped by the docs, to see what could be better with the docs, is far more expensive to the fresh author than anyone else.

I often have set my software aside long enough to start forgetting what it does, or go through several rounds of trying to explain it to others, before I can see what the docs need to be - any docs I previously forced out of myself are usually revealed to be bad once that happens.

I do seem to be getting better at that... the first pass of README for my Python macaddress library still holds up after a few years, I think.

But even so, it's exhausting how time-consuming it is. Does everyone have to take this long? Probably not, but it just rarely comes easy for me. I have to write stuff, re-read it a little bit later, a little fresher, while trying to put on a different mental angle, see flaws - the misinterpretation potentials, the omissions of why it matters, spots where detail or focus is unnecessary, engineer a different wording or examples, rinse and repeat.

And it feels frustrating to not spend that time now, to just put it aside and not force it until one day suddenly the explanation flows... Because documentation is the main step between done and Done - between having a thing that works and any actual benefits of a having thing that works which involve other people. Because if I don't force it, something really nice like git-cotree will just sit indefinitely - sharing things doesn't actually accessibly share it with anyone unless you also explain/present/demo/market it.

[mentalisttraceur-software]

Alright, so, to summarize all the Emacs Git/diff/pop-to-command/why-Magit-is-not-for-me posts: I am now a very happy user of one consistent interface for all diff-y text operations!

Before I headed down this path, I was looking at the prospect of having 3-5 different interfaces to deal with:

an Emacs-specific interface for Git (admittedly a very nice one for certain usage patterns and workflows - Magit brings some really really nice touches to the table),

a different Emacs-specific interface for viewing the differences between a file's on-disk and in-editor contents (Emac's built-in diff mode, which adds its own rather nice refinement to regular diff command output),

yet a third Emacs-specific interface for granularly applying/reverting changes between a file's on-disk and in-editor contents (Emac's built-in Ediff, which, again, is admittedly nice),

and possibly something else that I'd have to figure out, because after scouting a bit, it seemed that Ediff didn't really cover both partial save and partial reload out of the box.

And of course, I'd still occasionally find myself having to use the Git CLI outside of Emacs (and maybe even from inside Emacs, if I couldn't get Magit to properly wrap my git-cotree stuff).

And those Emacs solutions were going to demand specialized effort to learn, configure, integrate, work around, hack, and monkeypatch - much of that was not shared. There wasn't going to be any of the force-multiplication of just solving a thing in code/config once or mentally adapting once and getting the benefit everywhere. And I'd have to force them to behave in ways they weren't designed for where my UX preferences/needs were different.

Now, instead:

my Emacs interface for Git is just Git CLI commands, popping up in their own in-Emacs terminals when I call them;

my Emacs interface for getting differences between files on-disk and in-editor is just "git diff", using its built-in ability to work as a regular diff (which means I get niceties like Git's moved line detection and my git-delta setup);

my Emacs interface for granularly applying/discarding changes between a file and its buffer is just "git add -p" (the only touch I'm missing here is that I'd like to replace "Stage this hunk" in the prompt text with strings like "Save this hunk" and "Load this hunk", but near as I can tell that's hardcoded in one of the Perl files that make up the Git CLI);

to bring anything else into the same UI/UX, all I have to figure out is how to map those things into temporary files/directories for input and output (for example, grab the currently opened file or given file-name, and the first of its Syncthing sync-conflict files), and maybe more fundamentally how to enhance my diffing to handle them better (for example, bring in language-aware diffing like difftastic offers).

When I find myself outside of Emacs, the only thing I've lost is the efficient keybinds for the commands and the window splitting/management around them, not the core functionality and interface.

[mentalisttraceur-incomplete]

I have this weird habit of holding back on publishing a new code feature of there are improvements to related code which could easily be done either before or after.

For example, for basically a day, maybe closer to two now, I've had one of the most important remaining git-cotree features implemented:

git cotree --collapse <co-tree>

This "collapses" the given work tree's contents back into the repo root.

The typical case for me would be when I'm down to one work tree and I don't expect to have more than one for a while.

Another typical case would be getting people excited about git-cotree, both as a concept and to try it. Init is already really cool as a demo. Clone repo, make a bunch of changes, stage some

git cotree --init

and BAM, everything is now in the subfolder "main" (or "master" or whatever the repo's main branch is called); changes and index state and so on perfectly preserved. But if that wasn't cool enough? if you were hesitant to because what if you don't like the directory layout or workflow or whatever?

git cotree --collapse main

and BAAAM you're back to normal. You can delete the other co-trees first, or just let them hang around (but mind the collision risk - if you've got a file named "foo" in main and created a file/directory/cotree named "foo" in the repo root next to "main", the MVP of the collapse feature isn't protecting you against clobbering like that, and I'm not sure that in the general case it should ).

In practice, since so much test/build/etc code is written making dumb fragile hard-coded assumptions about relative paths, a third use-case would be to swap a worktree back into the "normal" position temporarily to run that stuff. git-cotree doesn't directly provide that complete swap experience, but cotree initialize and collapse handle the tricky and git-specific parts.

Anyway, despite having coded it, I've just been leaving it sitting in my local code, carefully working around it with `git add -p`, because I keep thinking of improvements that are relevant to both collapse and other stuff that's already published.

[mentalisttraceur-incomplete]

Significant rework of "git cotree --init" tonight, which:

Makes it categorically impossible to lose anything through corner-case interactions between Git and unstaged/untracked/ignored files and directories, or related to file system metadata that Git doesn't track. I've been wrong before and I might have missed something, but the surface area in the possibility space for even that should be much smaller with thew new approach. I think this marks the end of playing whack-a-mole/ping-pong with bugs in those edge cases.

Significantly optimizes performance. Basically zero file system copies (no more moving files through Git in the form of a checkout into the new worktree subdirectory and Git removing them in the base directory). No need to programmatically add anything to the Git index or stash just to get it out again. Never any need to recurse when moving things into the directory. The total cost is now basically just "mktemp -d", a directory listing in the base directory, a rename(2) call for each of those items (which stays within the same file system, so basically just updating pointers, and at worst something like a "strcpy" on the name of each moved thing, and twice as many writes of modified timestamps, but with modern file system in-memory caching that could lead to as little as writing out a couple pages to actual storage), a mkdir (+1 mkdir for each "/" in your current branch name), and some Git bookkeeping which from my understanding I would suspect is O(1).

This is the approach I first imagined doing when I came up with git-cotree. This is what I wanted to do all along, but I didn't know how to get Git to do this - a couple years later and my ability to... basically understand what the Git documentation for

[mentalisttraceur-incomplete]

I'm often surprised by how fancy some people make their shell prompts. I have no problem with it, and I think I get the appeal, but it's just... so not me, so not what I want, that it takes me by surprise.

Mine is

$

Not even the current directory, let alone git branch, virtualenv, or whatever. What am I gonna do, forget where I am? Okay fine, I'll run "pwd" the one time per week(s) that happens. Forget what branch I'm in? "git branch", which I could shorthand if I ever needed it enough (and with git-cotree, "pwd" covers this). I did that with virtualenvs back when I needed to know that often enough: "pwe" for print-working-env. I've got "git status" aliased to "s" (just "s", not "git s") for the rare times hitting " vv" in my Emacs isn't faster, and git-status-ware ls-clone (Exa/Eza) on "l". Will I miss a "cd" or "git checkout" in the scrollback? Hasn't happened to me yet.

But you know what happens fairly often? Being in some directory whose path is long enough to significantly push the input line over, or make me have to look and think around a line wrap. And the benefits of an uncomplicated, universal, O(1) visual parse are reaped every time I look at scrollback each and every line of scrollback.

(This, incidentally, is why I understand the multi-line colored prompts. They seem silly at first but they put all the variable-length information on their own line so the commands always start in the same place, and color-code it so you can find it or know to skip it with far less visual parsing effort. If I ever wanted to pay the cost of that extra information at every call just to have it the few times I need it, I'd go multi-line and color-coded.)

If I'm using a shell with vi mode (and which can actually redraw its own prompt every time I change vi mode), it's

I$

or

-$

and I cannot conceive of how people use vi-style line editing in a shell/REPL/whatever without needing an indicator.

And I think there's something of a useful parallel here - it's tempting to dismiss the super fancy shell prompts as superfluous. others want/need current directory/git/etc state the way I want/need vi mode.

[mentalisttraceur-software]

Words cannot describe how happy I am with `git-cotree`, and everything that went into `git-cotree --init` so that I can switch a repo from one normal working tree to cotrees with zero mental overhead and zero worry about uncommitted/unstaged/untracked work being lost in the process.

One of the best quality-of-life investments I've made for myself in the last year or two.

[mentalisttraceur-software]

As foretold (by me) in the prophesies a couple rambling posts, my efforts to achieve a great briefly exciting (for me) workflow with multiple Git work trees have "matured" (I spent a few says compulsively thinking about it every once in a while) into `git-cotree`.

`git-worktree-mode` became `git-cotree --init`. The new functionality is "just" ergonomically and robustly creating and deleting work trees co-trees, CLI argument parsing, and help text.

It is the same `git` history/repository (so you can still get your hands on the earlier work , I just renamed it in GitHub (so the old GitHub repo URL should redirect to the new one into the indefinite future).

[mentalisttraceur-software]

Git Co-Tree updates re: base branches:

So, the co-tree root technically continues to have the HEAD that it was on when you ran `git-cotree --init`. This is just a side-effect of how Git works, but I deliberately take advantage of it in the co-tree approach because this is how we get the base branch behavior "for free" (though I pay the cost in having to think about and code the stuff discussed here, so it's not entirely free - but we don't have to reimplement .

Everything works fine unless you delete the base branch. Then lots of Git commands start failing cryptic warnings or errors, especially when executed inside the co-tree root.

One indended use-case for Git co-tree is that you can switch to multiple work trees from a normal directory at any moment, in the middle of whatever you're doing, including while being on some branch which won't be around much longer. I ran into this this this weekend, when I was on a feature branch. Later, I was cleaning up branches, and suddenly things started giving errors about not being able to find HEAD or whatever.

Two days ago, I updated `git-cotree` to solve this problem in a way I quickly regretted. See, for a while I had been entertaining the idea of "detaching" the co-tree root from all branches - just park it on an empty orphan commit, the root of a new independent history. This seemed like a great way to solve this "deleting base branch" problem: no more base branch, no more problem. But then I as soon as I was actually using it, I found myself immediately facing all the ergonomics losses - new branches and work trees created while in the co-tree root now got based on this dummy commit; branch deletions always thought they were locally unmerged, and so on.

So then today I updated it again, in a much better way: the orphan commit behavior is reverted, and now `git-cotree --init` locks the first co-tree it creates (with `git worktree lock --reason 'co-tree base branch'`). This prevents us from deleting that work tree, and thus that branch, with a mere `git-cotree --delete` or `git branch --delete`. One unintentional but maybe reasonable thing it does not prevent is if you do a `git checkout` or `git switch` inside the locked work tree, so that the lock on the work tree no longer protects the base branch of the same name. But this should not normally come up in a co-tree workflow, because the normal way to switch branches in co-trees is to create or switch into another work tree, not create or switch into another branch in the same work tree.

Having made that change, I finally accepted that it was time to add a "git-cotree --base" to change the base branch of the co-tree, while taking care of updating those work tree locks, doing some checks, and so on. This could be done manually with just three Git commands, but it's just enough of a hassle that I want a shorthand, and requires a bit of knowledge of Git internals that I don't want to make users remember and type out on the fly. This also provides a purely Git co-tree way to handle the case when you really do want to delete the base branch: first switch to another base branch, then you're free to delete the original. No need to learn the lower-level details.

[mentalisttraceur-software]

Git Co-Tree Terminology

Couple names for things I've found myself needing to refer to when talking about the Git co-tree concept, workflows, and script:

co-tree root: The original work tree root after you run `git-cotree --init`, which empties it out and prepared it. This is where all the co-trees for that repo are located.

co-tree base branch: The branch which the whole repo is "on". When you run `git log` in the co-tree root, this is the branch whose log it shows. When you create a new work tree from within the co-tree root, this is the branch it uses by default as the base. Initially, this is the branch you had checked out when you ran `git-cotree --init`. Can be changed later with `git-cotree --base`.

[mentalisttraceur-software]

Actually, I was wrong in this statement from my post about home as a git-managed directory:

the next time that you try to do a “git init” in a folder like “~/code/my-new-project”, and git says “reinitializing .git directory in ~” instead of creating a new git directory.

Now that I've had some rest and recharged my testing-edge-cases patience, I can report that it actually works fine: it initializes a new git repo in ~/code/my-new-project as you'd expect, even without ceiling directories. That was a bad example.

What's funny is that I was very confident that I remembered it working just fine many times before over the years, but then when I last tested yesterday it it seemed not to. I must've done something wrong (probably literally just missed/forgot a directory change in the relevant shell, and didn't notice) - but I ended up just assuming that maybe I was misremembering or that it was a behavior that changed in newer git versions.

Having said that:

There are still good reasons for doing the ceiling directories workaround.

Without the ceiling directories setting, we don't get "not a git repository" detection in all subdirectories of home. Which means any code you might run could end up treating pretty much anything within your home as a git repo.

And it's easy to overlook how significant that can be - if I'm sloppy and forget a "git init" in some code folder, or just happen to be in the wrong directory, future git operations in the code folder might effect the home directory, and not just "oops I staged a file" but "oops I ran 'git cotree init' and now literally everything in my home directory has been moved into ~/main" (in the case of git cotree, I'm pretty sure I didn't even implement it as a literal move, but actually as a stash of staged changes followed by a stash of unstaged changes, and then two stash pops (separated by a stage) in the new worktree... and in the case of my home folder, I'm likely to have something like a catchall .gitignore and many things in home not tracked, so I'm not even sure what the interactions there would be off the top of my head - [edit: I tested, and it seems to at least move everything safely without problems]).

So sticking with ~ in my ceiling directories seems like the far safer bet, because I don't have to mentally scan the full possibility space of what could go wrong to be sure that it's fine, I just preclude all of that possibility space from applying.

[mentalisttraceur-software]

Aaaaahhhhh! (but in a good way)

cdexec ../some-other-branch git stash

git stash pop

!!!

So smooth and convenient, relative to the many little manual steps you need to do this normally.

(`cdexec` is another thing I made long ago, over here. The way cdexec and git cotrees regularly complement each other like this has been bringing me joy regularly for months.)

Words cannot describe how happy I am with `git-cotree`, and everything that went into `git-cotree --init` so that I can switch a repo from one normal working tree to cotrees with zero mental overhead and zero worry about uncommitted/unstaged/untracked work being lost in the process.

One of the best quality-of-life investments I've made for myself in the last year or two.