git-repository/gix towards 1.0

[Byron]

A probably incomplete collection of features we still want to see in git-repository before it can possibly see a first stable release.

initialization

• make it easy to change configuration during discovery into something restrictive, accommodating starship and helix-editor.

Diff

• high-level API with configuration for typical use-cases. (git-repository diff API #535)
• rename tracking

Submodules

• iteration and access (with active handling)
• integration with fetch (see also: submodule.recurse config flag)
• git-sec integration to avoid having to turn off file protocol support if applicable.

Lock

• a configurable option to emit traces in case somebody is blocked on a lock for too long.

Pathspecs

• pathspec globals that can be used for parsing pathspecs (i.e. GIT_GLOB_PATHSPECS)

Protocol

• remove duplicate parts of the fetch implementation, as well as Delegates as much as possible.

Remotes

An abstractions to list them and learn about which branches interact.

• list remotes
• anonymous remotes
• fetch refs and build ref-map
  • optimization: don't list refs if all refspecs denote objects by hash (git implements this)
• save to configuration in memory
• authentication function should be configured per passed URL, which needs rewriting too (on demand). This is relevant for redirects where a new URL is passed that currently doesn't trigger rewrites nor finding the correct credential helper.
• fast connectivity validation of objects in received pack, see c6807a4 in git for all the details
• save to configuration on disk
• respect fetch.unpackLimit - important as it helps keeping repos usable, probably good along with a way to build bigger packs from X loose objects.
• option to turn off fast-forward checks for forced refs (this is to speed up typical API usage where the information isn't used anyway)
• consecutive fetch negotiation algorithm (the default in git)
• FETCHHEAD updates
• an easy way to find the local tracking branch of any reference (also called upstream branch now)
• respect remote configuration
  • tagOpt (autotags or tag-following, "include-tag" as arg for 'fetch' command (and is the reason that git pulls tags by default if they are pointing to commits that are fetched).
    • This also affects clone as we should allow this option to be configured.
    • V1 'include-tag' support - needs another fetch round, or maybe rather not implement it or just fetch all tags? V1 doesn't have include-tag I think so one will have to fetch the refs and when commits are there, we traverse the commits received to match them with tags we know. Then we fetch these tag objects if they are annotated tags, and create tags in any way.
  • prune refs - be aware that there are side-effects between --tags and --prune, so probably this is more than just a boolean to make desired effects explicit.
  • Maintenance
    • ideally, make this available as object that can be executed by the user, potentially queuing other maintenance operations as well so users can execute them on a separate thread. It should just be a list of tasks, which can be executed once a repo is provided.
    • explode smaller packs into loose objects automatically (see fetch.unpackLimit). Having this is really important to avoid running out of ODB slots for packs in long-running applications that fetch a lot during their runtime.
  • support for anonymous remotes (to be (re-)configured by owner)
• http transport configuration (auto-configure based on git-config)
  • custom headers for http transports via extra.headers
  • http.followRedirect and override
  • http proxy settings
  • complex http..* based option overrides
• access to remote groups and a way to fetch them?
• merge configuration in bare repos - it probably should update the respective branch if checked out.

Questions

What about auto-tags? Maybe if enabled, we will automatically add a ref-prefix to receive tag information, and then auto-create annotated tags according to the typical refspec?

Should we generally only provide options for values that are not in the git-config? Probably… It's easy enough to use config_snapshot_mut() and can be a general directive that leverages git configuration docs.
However, it must be possible to set these settings in bulk in case of anonymous remotes, but separated by transport config and remote config.

Outscoped

Decide what's needed of these, if any, and keep track of them.

• proxy support
• mirror support
• All information related to pushing
• promisor objects
• branch.<name>.merge support, as we don't yet merge anything and fetching still fetches the entire refspec of as configured in the remote.

rev-walk

• add tips by iterator
• add rev-specs (basically includes and excludes in one)
• add commits for exclusion

git-revision

• a graph type to implement describe() with
• calculate the merge base of commits using a ref-spec maybe
• support the commit-graph (see this explanation for details)

Cloning

• recursive submodule checkout
• worktree checkout
  • apply standard filters
  • run filter programs
    • with both one-off and long-running protocols
• directory to clone into is specified so that it can be adjusted based on the final (potentially redirected) url if this is what git does (to be validated). In any case, implement it correctly and so that everybody can use it (probably the logic should go into a re-usable function or method) or use an enum for the directory instead.
• support --mirror option (see this comment for initial analysis)

Out of scope

• local-clone options (i.e. copy ODB files directly, hard-link, symlink, alternates)

rev-parse

• Support for main@{push} handling, which depends on properly resolving push.default
• Support for main@{upstream} which needs fiddling with ref-specs for remotes
• Support for reflog access by date, depends on gix-date towards 1.0 #471 .
• @: empty - paths should yield the tree, and not an error that says "" could not be found

Config

• add config() which assures the returned snapshot is up-to-date. Metadata allows to remove outdated files and replace them with their updated counterparts. Includes can be detected and re-resolved (but that needs configuration), and the cache needs to be updated afterwards too. There is definitely some details to consider, but it's possible.
• config_mut() to take a lock file at the repository local configuration, collecting changes in memory and writing only those back to disk on drop, or when commit() is called similar to SnapshotMut. Sister-method config_mut_at() to chose a particular file for writing given a path. Assume that config() will pick it up when called again, and it has to be as _mut() need a mutable Repository to use Rust semantics to protect against surprises.
• use configuration trust of returned value and warn on invalid value if untrusted, abort otherwise. That way, invalid configuration can't be preventing us from coming up. Make sure this is configurable so it's part of the trust map. (lenient configuration)
• review all gix progress environment variables and implement as many as feasible.
• Add open::Option parameter to allow parsing the GIT_CONFIG_PARAMETERS environment variable which is a way of storing -c flags in git. Useful when implementing remote-helpers and probably hooks as well (or any program that git runs. Consider reviewing the git-command as it could reasonably pass API and CLI overrides that way as well.

Build

• fine-grained control over dependencies, with switches for
  • worktree
  • only initialization, nothing else (for tokei tests)