[Pitch] Replacing the `update-checkout` script with `repo`

I would like to propose to replace the update-checkout script with the repo tool.

Motivation

From repo's Readme:

Repo is a tool built on top of Git. Repo helps manage many Git repositories, does the uploads to revision control systems, and automates parts of the development workflow. Repo is not meant to replace Git, only to make it easier to work with Git. The repo command is an executable Python script that you can put anywhere in your path.

Repo is actively maintained and addresses some of the issues which come up when using update-checkout, specifically on Windows:

  1. it works reliably on windows
  2. it can support exporting stable build points to XML.
  3. it can work in quiet mode (1/4th of the CI logs are git commands executed by update-checkout.

@compnerd has been using repo to build the toolchain on Windows for the past couple of years and is suggesting to use it on their instructions on how to build the toolchain on Windows.

Blockers

One thing I have not managed to replicate on repo is the --skip-repository argument. --github-comment is also missing.

2 Likes

I'm clearly biased, but I am strongly in favour of switching to repo :slight_smile:

--github-comment is unrelated to the repository management and is used by the CI system to track the dependent changes for a cross-repo change. This can be a simple curl request.

--skip-repository is not a part of repo as it is meant to synchronize the full set to a known point. The --skip-repository allows the user to bypass that and get into potentially an invalid state. This is more about following POLA.

1 Like

We use --skip-repository in build-windows-toolchain.bat, can we safely remove this argument there? Or replace it?

My primary use for --skip-repository is to prevent it from overwriting my local checkout when i've rebased my branch and i want to update dependencies. Since i have a Git alias set up to update a branch without checking it out, using --skip-repository means i also don't have to check out the base branch to update the dependencies.

4 Likes

We could remove most of it (I think that --skip-repository swift is needed for update-checkout to work?). The usage there is to avoid checking out macOS specific repositories to save time. This would be handled in repo with adding a group="notdefault,macOS" on the repositories.

1 Like

repo only seems to have a flag for repo sync to specify which repository you want to sync. This makes syncing all expect one repository very verbose.

The group arg looks great. It would allow to have an explicit list of repositories which are not required for Windows. I believe that this list only exists implicitly in the arguments of build-windows-toolchain.bat.

There is a palpable friction for swiftlang developers when working on patches that require corresponding changes across multiple repositories. Based on its command reference and sources, repo seems to have many great, optional features like cross-project branches, staging, rebasing, diffs, etc, that update-checkout lacks.

I'm definitely in favor of adding the needed manifests so people can choose to use repo's extra features, as that would provide a standard, streamlined way to juggle cross-project changes throughout the development cycle.

We don't have to immediately ditch update-checkout, as much of its existing functionality can probably be implemented by shelling out to repo. In a future where more components live in separate git repositories, like the standard library, tooling like repo may even become essential.

3 Likes

It would be easy to support both update-checkout and repo at the same time while we experiment/transition: comparing this repo manifest to our update-checkout-config.json it seems like we could easily add a flag in update-checkout to export the config to a repo manifest.

We could have a swift/utils/repo_manifests directory with the manifests that were exported from update-checkout-config.json.

2 Likes

repo sync will rebase your local changes on top of the remote. I think that should eliminate the use of your git alias in this specific case?