first impression of jj is a failure that makes no sense.

[hemna]

I have an existing git repo that I want to try jj against. I ran the jj git init --colocate and it gave me an error

I have no idea what jj is doing and one of the arguments about why jj is so much better than git is unknown weird operations on the backend that cause weird errors....just like what I'm getting with jj the very first time I'm trying to use it.

└─> jj git init --colocate
zsh: correct 'git' to '.git' [nyae]? n
Done importing changes from the underlying Git repo.
Error: Failed to snapshot the working copy
Caused by: New file /Users/I530566/devel/mine/hamradio/aprsd/aprsd/web/chat/static/css/upstream/fonts/bootstrap-icons.svg of size ~1.0MiB exceeds snapshot.max-new-file-size (1.0MiB)
Hint: Increase the value of the `snapshot.max-new-file-size` config option if you
want this file to be snapshotted. Otherwise add it to your `.gitignore` file.

why is this failing? this should just work. Increase the value of 'snapshot.max-new-file-size' config option.....where?
what config file? why do I even have to do this? smh.

[necauqua]

Have you read any of jj tutorials, intros or documentation before trying it?.

On almost every command, jj snapshots the files in the folder, meaning it automatically commits everything into a working copy commit, the @

If you know about this, the error is very clear, and has a hint.. I'm not in your head though, suggestions to make this better are welcome.

Usually git/jj repos don't contain large files, it's not the best idea, and since snapshotting (aka creating a commit) happens automatically, the max-new-file-size heuristic thing was added to prevent accidents - like accidentally committing some giant log file that you forgot to add to .gitignore or something.

I don't get the issue about config option - literally 99% of cli software that does more than one simple thing has config files - again you can look up where the config is and what's its format is in the docs, literally jj help config is good enough. But well ok I get how if you didn't know about the core feature of jj you could be confused as to why would you have to change some random config, that does make sense

[ilyagr]

Thanks for explaining things. I would argue with your tone a little: it makes perfect sense to me that a brand new user would try jj init before understanding much or any core features of jj, just to experiment with it. That what would motivate their interest; one has to be interested in a thing before they can learn about it, even the basics of it.

Moreover, hopefully one day we'll be getting users who aren't even familiar with Git, for whom what we now might call "basics" will not seem basic at all. They'd be learning about all the other basics of version control at the same time.

[necauqua]

Yeah closer to the end I realized that as well, you can almost see it

fwiw I'm sorry, @hemma, you were clearly just frustrated, I took some offence for no reason

[hemna]

No I haven't read any tutorials. I had seen a youtube video on a person touting how great jj was over git, mostly because it was easier to do some things. So, I installed jj with brew, and then tried to run a jj command in my repo. It then said I needed to run the jj init command, and then everything got confusing and fell apart from there.

One of the reasons that I figured I would try jj is that the youtube video mentioned it really was better for people that didn't want to have to understand the internals of how git works, which is a pain. So, my first (uneducated) impression of actually using jj showed me with an error that has to do with the internal implementation of jj. You can imagine my frustration...albeit...yes RTFM I get it. But I figured installing it and trying something like jj log would work out of the box.....when it didn't....because of internal implementation detail errors.

I remember feeling frustrated when switching from svn to git and then loving git, so I'm hoping that can be the case with jj as well. Maybe my approach is abnormal, but I didn't think so. I know git and how to use it, so why wouldn't a simple jj log command work?....oh..

[necauqua]

I would not call this an internal implementation detail, it's a very user-facing check, idk, @thoughtpolice described it way better than I did

The ssh issue you also had is definitely an internal implementation thing 😂, that's a common pain point actually (that I don't exactly know how people fix since it didn't happen for me), you're hitting like all of them

---

"the internals of how git works" - people keep and keep saying that, but the actual internals of git are like.. super-duper simple?.. The git just failed to have a ux that's not terrible

And if you understand those, then working with git (or with jj, whose main backend is git) is also a breeze (but jj has amazing ux). Like just reading the chapter on git objects of the git book years ago just made git click for me

[necauqua]

@ilyagr I just realised I was also biased because this:

brand new user would try jj init before understanding

never works that way for me, I always read (at least look through) tutorials/demos before actually trying something, and if there are no demos or easily readable tutorials it's unlikely I'm trying that. That's why git-branchless and jj were great for me, they had those things :)

So my first reaction was "what, why'd that person do this like that and then complain", although in hindsight you are completely right

[ilyagr]

Thanks you for the reminder, I do think this hint needs improvements. I had a few problems running into this error even though I know what the feature is about (see @necauqua's comment):

1. It's unclear what the valid values or units for snapshot.max-new-file-size are.
2. You might not want to enable this option globally, or even for this entire repo.
3. I'd also want to have a way to allow this large file (regardless of its size), but not to automatically add all large files (which might indeed be files one meant to .gitignore).

Perhaps, as a first stab, we could suggest jj --config-toml 'snapshot.max-new-file-size="5MiB"' log (which might look a little weird, but the user could copy-paste; hopefully it works on Windows too (?) ), but then we could keep track of a list of allowed large files, keep a list of them, and have a command to add to the list.

(Update:) One more confusing point: I am not entirely sure at the moment to what degree the command I gave above would grandfather the large files being snapshotted right now for future invocations of jj. IIRC, it will grandfather them in, even if they grow to even larger sizes later.

[necauqua]

The hint in the error is very technical, it assumes people know about snapshotting, which wasn't the case here

I guess if it could print a condensed version of my explanation (and less passive-agressive of course xd) it'd be better for beginners

[ilyagr]

Yeah, I think this is a main issue: this error/hint is currently not clear/actionable enough for an error that a brand-new user is relatively likely to see on one of the very first things they try with jj. There are technical reasons for this, but hopefully they are surmountable with some work.

[thoughtpolice]

When jj runs nearly any command, it takes a "snapshot", which among other things adds stuff to .git storage. If you have a really large file in your repository that has not yet been committed, for example a temporary log file, it is easy for it to get added to the snapshot, and that can cause really high space usage in the .git and .jj directories, quite fast.

You can garbage collect these things to reduce that impact, but GC is actually a relatively new feature for jj; the snapshot.max-new-file-size option is an anti-footgun measure that was added before it was available. Using GC also requires you to use the operation log, so it's kind of annoying for new users. Once you push something though, that file exists forever in .git, so you're stuck with it.

Note that I said "really large file", while the bootstrap-icons.svg file is only ~1MiB. Well, the reality is that Git just doesn't do very well even with lots of small binary files, much less actually large ones, so the 1MiB limit was chosen because most of the time you want to be explicit about files like that. It's a bit arbitrary. Naturally, Git repositories can basically only ever increase in size over time, never decrease, so it's very easy to get bloated repositories if you aren't careful. And most of the internal techniques that git uses to help reduce that growth (packfiles) simply do not work very well on actual binary files; taking a 1MiB file and updating it to a new version is effectively 1MiB+1MiB total added to your repository size, forever. This can add up very quickly.

(I was the person who proposed this max-file-size feature, for what it's worth, because I've generally had to deal with too many repositories where people just throw binary blobs inside of it and hope for the best, because most people just aren't familiar with Git's storage model; jj's snapshot design makes that issue more potent if you aren't careful. It's simply one of the natural trade offs in the design space)

The error message needs to be improved. We should probably recommend two options:

• Run jj config set --repo snapshot.max-new-file-size ... in order to immediately set the option in the per-repository configuration, so this does not happen again in this repository.
• Run jj --config-toml 'snapshot.max-new-file-size="..."' log to snapshot the file immediately, in a once-off way, so that you can get on with your work, while still refusing to add large files in the future.

A final note: I assume your Git repository has a dirty working copy; you have not made a commit that features the bootstrap-icons.svg file, so it shows up as untracked in your git status. Maybe we should refuse to initialize a repository unless the working copy is clean; I'm not sure if that would break anything, but it would definitely lead to a simpler error message in this case: "Please commit or stash your uncommitted changes before running jj git init"

[hemna]

╭─aprsd on  stats-rework [$!?⇡] via 🐪 v5.34.1 via 🐍 v3.10.13 ❯
└─> jj config set --repo snapshot.max-new-file-size 10000000
╭─aprsd on  stats-rework [$!?⇡] via 🐪 v5.34.1 via 🐍 v3.10.13 ❯
└─> jj st
Config error: invalid type: integer `10000000`, expected a size in bytes with an optional binary unit

[ilyagr]

Try

jj config set --repo snapshot.max-new-file-size 10MiB

This definitely needs to be fixed in some way.

[ilyagr]

Also, the behavior you see is buggy (as opposed to just very confusing) since the error says that the binary unit is "optional", but it clearly isn't.

[thoughtpolice]

Thanks for this. I've got a patch incoming to fix this and make the error message more clear.

[thoughtpolice]

#3444 should fix the error seen here, in particular; it isn't because the unit was mandatory — it was in fact optional — it's because the value provided was an integer, not a string. In other words, "10000000" would have worked, "10000000B" would have also worked, but not 10000000 as that example shows. The PR fixes this, among other things.

[hemna]

I guess I'm just getting lucky finding some things as a new user.
Thanks so much for the help. adding the 'optional' units to the integer seems to have fixed it.

It seems to not be complaining now about running commands now.

jj st

It would be nice if jj log automatically paged at the size of the terminal instead of dumping the entire repo's log commits. Maybe make it a switch to disable paging?

[thoughtpolice]

You can set ui.pager="none" in order to disable paging entirely. This actually works somewhat nicely if you use --reversed in order to get the log output in reverse, so the latest stuff all comes last.

By default, we use (in rough order):

• $PAGER, if it is set
• less -FRX if you are on Unix (and $PAGER is not set)
  • Note that less also respects the $LESS environment variable containing default options. The equivalent to the above is to set PAGER=less and LESS=FRX, which may be nicer.
• A built-in, simpler pager, if you are on Windows
  • You can also install less.exe on Windows, put it in $PATH, and then set $PAGER to less -FRX too, if you want

Note that the -F option to less is also called --quit-if-one-screen, which accomplishes the behavior you're asking for. When you run jj log, the default set of revisions is calculated by the option called revsets.log, which by default is something like "All of the commits not in the main branch on your origin." So, you may be seeing things that just haven't been merged, yet, if you have a lot of outstanding branches on your remote.

Try doing this instead, which will probably show you a small amount of commits that should fit within your terminal height: jj -r 'trunk()..@', and it should automatically exit the pager.

Be sure to check the tutorial; it has a simple introduction and has a section covering jj log and "revsets", which are heavily used throughout most commands.

BTW, you can always use jj config edit --user or jj config edit --repo to have the relevant configuration file opened in your chosen $EDITOR

[hemna]

and now trying to go through the tutorial and cloning one of my github repositories fails with an error I don't know how to fix

└─> jj git clone git@github.com:hemna/aprsd-irc-extension.git
Fetching into new repo in "/Users/I530566/devel/mine/hamradio/tmp/aprsd-irc-extension"
Error: authentication required but no callback set; class=Ssh (23); code=Auth (-16)
Hint: Jujutsu uses libssh2, which doesn't respect ~/.ssh/config. Does `ssh -F /dev/null` to the host work?

[hemna]

yes it works.

ssh -F /dev/null git@github.com responded with

└─> ssh -F /dev/null git@github.com
PTY allocation request failed on channel 0
Hi XXXXX! You've successfully authenticated, but GitHub does not provide shell access.
Connection to github.com closed.

[martinvonz]

And jj git clone still doesn't work? I think sometimes running the ssh commands results in the host getting added to known_hosts and then libbsh2 allows the connection after that.

There's also a pending PR to use OpenSSH instead.

[hemna]

yah it's still not working.

└─> ssh -F /dev/null git@github.com
PTY allocation request failed on channel 0
Hi hemna! You've successfully authenticated, but GitHub does not provide shell access.
Connection to github.com closed.
╭─mine/hamradio/tmp via 🐍 v3.10.13 ❯
└─> jj git clone git@github.com:hemna/aprsd-irc-extension.git
Fetching into new repo in "/Users/I530566/devel/mine/hamradio/tmp/aprsd-irc-extension"
Error: authentication required but no callback set; class=Ssh (23); code=Auth (-16)
Hint: Jujutsu uses libssh2, which doesn't respect ~/.ssh/config. Does `ssh -F /dev/null` to the host work?

[joyously]

There's also a pending PR to use OpenSSH instead

The link is messed up.

[martinvonz]

Thanks. Fixed

[martinvonz]

It haven't followed all of the discussion above, so I'm sorry if this has already been mentioned, but why don't we just skip large files and print a warning instead? We shouldn't have to fail, I think. Maybe it's just that the error message was added before we became more careful to not overwrite unexpected contents in the working copy (commit 44eb902)?

[ilyagr]

That's dangerous in its own way: people will assume the files are version-controlled (committed, pushed, etc.) when they aren't.

Perhaps this could be mitigated by every command printing a warning in red letters. But then, is printing a warning "Something is very wrong with file XXX" every time really better than failing until the user fixes the problem?

The best potential solution that came to my mind would be to print an error, but to also have a command jj allow-large-file XXX in addition to what we have now. The hint might suggest both.

[martinvonz]

If we print the message when snapshotting, it would happen on basically every command, so I don't think users will forget about these files. Do you?

Once we have the planned jj track, it might make sense to teach that to also track large files (in addition to ignored files).

Edit: But to answer your question, I'm not sure it's better to print a warning than to fail. It might be risky especially for scripts.

[hemna]

Anything wrong with increasing the size of the default to a large value and then print a warning?

[ilyagr]

But to answer your question, I'm not sure it's better to print a warning than to fail. It might be risky especially for scripts.

Yeah, I'd rather fail.

Once we have the planned jj track, it might make sense to teach that to also track large files (in addition to ignored files).

+1. In fact, we could implement it for large files first.

Anything wrong with increasing the size of the default to a large value and then print a warning?

Increasing the size of the default to a large value would defeat the point of the feature. However, maybe there are less disruptive ways to achieve the goals of the feature.

The reason for the feature is that without it, jj makes it very easy to accidentally put large files in the repo, since there's not git add (which is convenient 99% of the time, but not in this case). The error has two goals: make it harder to push large files to a remote by accident (where it could become a part of the git history forever), and to keep the size of local .git dir from ballooning.

For example, perhaps we could allow snapshotting without an error (maybe with a warning), and print the error on pushing large files and on paking them part of an immutable commit? We could suggest running jj util gc to get rid of the references to the large files in the local repo.