我将使用子模块(如Pat Notz)或两个不同的存储库。如果你太频繁地修改二进制文件，那么我会尽量减少巨大的存储库清理历史记录的影响:

几个月前我遇到了一个非常类似的问题:~21 GB的MP3文件，未分类(糟糕的名称，糟糕的id3，不知道我是否喜欢这个MP3文件……)，并在三台计算机上复制。

我使用带有主Git存储库的外部硬盘驱动器，并将其克隆到每台计算机中。然后，我开始用习惯的方式对它们进行分类(推、拉、合并……)多次删除和重命名)。

最后，我只有~ 6gb的MP3文件和~83 GB的.git目录。我使用git-write-tree和git-commit-tree创建了一个新的提交，没有提交祖先，并启动了一个指向该提交的新分支。该分支的“git日志”只显示了一次提交。

然后，我删除了旧的分支，只保留了新的分支，删除了ref-logs，并运行“git prune”:在那之后，我的.git文件夹只重约6gb…

你可以不时地用同样的方法“清除”这个巨大的存储库:你的“git克隆”会更快。

在我看来，如果您可能经常修改这些大文件，或者您打算进行大量的git克隆或git签出，那么您应该认真考虑使用另一个git存储库(或者可能是访问这些文件的另一种方法)。

但是如果您像我们一样工作，并且您的二进制文件不经常修改，那么第一次克隆/签出将会很长，但是在那之后它应该和您想要的一样快(考虑到您的用户一直使用他们拥有的第一个克隆存储库)。

我正在寻找如何处理我的源代码(web应用程序)依赖的大型二进制文件的意见。你对此有什么经验/想法?

当我的web应用程序二进制数据超过3gb时，我个人在我的一些云主机上就遇到过Git同步失败的情况。我当时考虑过BFT回购清洁，但感觉像一个黑客。从那时起，我开始将文件置于Git的权限之外，而是利用专门构建的工具(如Amazon S3)来管理文件、版本控制和备份。

有人有在一个项目中使用多个Git存储库并管理它们的经验吗?

是的。雨果主题主要是这样管理的。这有点滑稽，但它能完成任务。

我的建议是选择适合这项工作的工具。如果它是为一个公司，你在GitHub上管理你的代码线，付钱并使用Git-LFS。否则，您可以探索更有创意的选项，例如使用区块链进行分散加密文件存储。

需要考虑的其他选项包括Minio和s3cmd。

我最近发现了git-annex，我觉得很棒。它是为有效地管理大文件而设计的。我用它来收集我的照片/音乐(等)。git-annex的开发非常活跃。文件的内容可以从Git存储库中删除，Git只跟踪树的层次结构(通过符号链接)。然而，要获得文件的内容，在拉/推之后需要第二步，例如:

$ git annex add mybigfile
$ git commit -m'add mybigfile'
$ git push myremote
$ git annex copy --to myremote mybigfile ## This command copies the actual content to myremote
$ git annex drop mybigfile ## Remove content from local repo
...
$ git annex get mybigfile ## Retrieve the content
## or to specify the remote from which to get:
$ git annex copy --from myremote mybigfile

有很多可用的命令，网站上有很好的文档。Debian上有一个软件包。

我想提出的解决方案是基于孤立分支和对标记机制的轻微滥用，因此称为*孤立标记二进制存储(OTABS)

如果你可以使用github的LFS或其他第三方，无论如何你应该。如果你不能，那么继续读下去。请注意，这个解决方案是一个黑客，应该被这样对待。

OTABS的理想属性

it is a pure git and git only solution -- it gets the job done without any 3rd party software (like git-annex) or 3rd party infrastructure (like github's LFS). it stores the binary files efficiently, i.e. it doesn't bloat the history of your repository. git pull and git fetch, including git fetch --all are still bandwidth efficient, i.e. not all large binaries are pulled from the remote by default. it works on Windows. it stores everything in a single git repository. it allows for deletion of outdated binaries (unlike bup).

OTABS的不良属性

it makes git clone potentially inefficient (but not necessarily, depending on your usage). If you deploy this solution you might have to advice your colleagues to use git clone -b master --single-branch <url> instead of git clone. This is because git clone by default literally clones entire repository, including things you wouldn't normally want to waste your bandwidth on, like unreferenced commits. Taken from SO 4811434. it makes git fetch <remote> --tags bandwidth inefficient, but not necessarily storage inefficient. You can can always advise your colleagues not to use it. you'll have to periodically use a git gc trick to clean your repository from any files you don't want any more. it is not as efficient as bup or git-bigfiles. But it's respectively more suitable for what you're trying to do and more off-the-shelf. You are likely to run into trouble with hundreds of thousands of small files or with files in range of gigabytes, but read on for workarounds.

添加二进制文件

在开始之前，请确保您已经提交了所有的更改，您的工作树是最新的，并且您的索引不包含任何未提交的更改。这可能是一个好主意，把你所有的本地分支推到你的远程(github等)以防任何灾难发生。

Create a new orphan branch. git checkout --orphan binaryStuff will do the trick. This produces a branch that is entirely disconnected from any other branch, and the first commit you'll make in this branch will have no parent, which will make it a root commit. Clean your index using git rm --cached * .gitignore. Take a deep breath and delete entire working tree using rm -fr * .gitignore. Internal .git directory will stay untouched, because the * wildcard doesn't match it. Copy in your VeryBigBinary.exe, or your VeryHeavyDirectory/. Add it && commit it. Now it becomes tricky -- if you push it into the remote as a branch all your developers will download it the next time they invoke git fetch clogging their connection. You can avoid this by pushing a tag instead of a branch. This can still impact your colleague's bandwidth and filesystem storage if they have a habit of typing git fetch <remote> --tags, but read on for a workaround. Go ahead and git tag 1.0.0bin Push your orphan tag git push <remote> 1.0.0bin. Just so you never push your binary branch by accident, you can delete it git branch -D binaryStuff. Your commit will not be marked for garbage collection, because an orphan tag pointing on it 1.0.0bin is enough to keep it alive.

签出二进制文件

How do I (or my colleagues) get the VeryBigBinary.exe checked out into the current working tree? If your current working branch is for example master you can simply git checkout 1.0.0bin -- VeryBigBinary.exe. This will fail if you don't have the orphan tag 1.0.0bin downloaded, in which case you'll have to git fetch <remote> 1.0.0bin beforehand. You can add the VeryBigBinary.exe into your master's .gitignore, so that no-one on your team will pollute the main history of the project with the binary by accident.

完全删除二进制文件

如果你决定完全清除VeryBigBinary.exe从你的本地存储库，你的远程存储库和你的同事的存储库，你可以:

Delete the orphan tag on the remote git push <remote> :refs/tags/1.0.0bin Delete the orphan tag locally (deletes all other unreferenced tags) git tag -l | xargs git tag -d && git fetch --tags. Taken from SO 1841341 with slight modification. Use a git gc trick to delete your now unreferenced commit locally. git -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0 -c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now gc "$@". It will also delete all other unreferenced commits. Taken from SO 1904860 If possible, repeat the git gc trick on the remote. It is possible if you're self-hosting your repository and might not be possible with some git providers, like github or in some corporate environments. If you're hosting with a provider that doesn't give you ssh access to the remote just let it be. It is possible that your provider's infrastructure will clean your unreferenced commit in their own sweet time. If you're in a corporate environment you can advice your IT to run a cron job garbage collecting your remote once per week or so. Whether they do or don't will not have any impact on your team in terms of bandwidth and storage, as long as you advise your colleagues to always git clone -b master --single-branch <url> instead of git clone. All your colleagues who want to get rid of outdated orphan tags need only to apply steps 2-3. You can then repeat the steps 1-8 of Adding the Binary Files to create a new orphan tag 2.0.0bin. If you're worried about your colleagues typing git fetch <remote> --tags you can actually name it again 1.0.0bin. This will make sure that the next time they fetch all the tags the old 1.0.0bin will be unreferenced and marked for subsequent garbage collection (using step 3). When you try to overwrite a tag on the remote you have to use -f like this: git push -f <remote> <tagname>

后记

OTABS doesn't touch your master or any other source code/development branches. The commit hashes, all of the history, and small size of these branches is unaffected. If you've already bloated your source code history with binary files you'll have to clean it up as a separate piece of work. This script might be useful. Confirmed to work on Windows with git-bash. It is a good idea to apply a set of standard trics to make storage of binary files more efficient. Frequent running of git gc (without any additional arguments) makes git optimise underlying storage of your files by using binary deltas. However, if your files are unlikely to stay similar from commit to commit you can switch off binary deltas altogether. Additionally, because it makes no sense to compress already compressed or encrypted files, like .zip, .jpg or .crypt, git allows you to switch off compression of the underlying storage. Unfortunately it's an all-or-nothing setting affecting your source code as well. You might want to script up parts of OTABS to allow for quicker usage. In particular, scripting steps 2-3 from Completely Deleting Binary Files into an update git hook could give a compelling but perhaps dangerous semantics to git fetch ("fetch and delete everything that is out of date"). You might want to skip the step 4 of Completely Deleting Binary Files to keep a full history of all binary changes on the remote at the cost of the central repository bloat. Local repositories will stay lean over time. In Java world it is possible to combine this solution with maven --offline to create a reproducible offline build stored entirely in your version control (it's easier with maven than with gradle). In Golang world it is feasible to build on this solution to manage your GOPATH instead of go get. In python world it is possible to combine this with virtualenv to produce a self-contained development environment without relying on PyPi servers for every build from scratch. If your binary files change very often, like build artifacts, it might be a good idea to script a solution which stores 5 most recent versions of the artifacts in the orphan tags monday_bin, tuesday_bin, ..., friday_bin, and also an orphan tag for each release 1.7.8bin 2.0.0bin, etc. You can rotate the weekday_bin and delete old binaries daily. This way you get the best of two worlds: you keep the entire history of your source code but only the relevant history of your binary dependencies. It is also very easy to get the binary files for a given tag without getting entire source code with all its history: git init && git remote add <name> <url> && git fetch <name> <tag> should do it for you.

你也可以用git-fat。我喜欢它只依赖于stock Python和rsync。它还支持通常的Git工作流，使用以下自解释命令:

git fat init
git fat push
git fat pull

此外，您需要将.gitfat文件签入存储库，并修改.gitattributes以指定您希望gitfat管理的文件扩展名。

您可以使用普通的git add添加一个二进制文件，它会根据您的gitattributes规则调用git fat。

最后，它还有一个优点，即二进制文件实际存储的位置可以跨存储库和用户共享，并支持rsync所做的一切。

更新:如果你正在使用Git-SVN网桥，不要使用git-fat。它最终将从Subversion存储库中删除二进制文件。但是，如果您使用的是纯Git存储库，那么它的工作效果非常好。