Tag: git

Migrating from TFS to Git in Visual Studio Team Services

This post will explain how to migrate from a legacy Team Foundation Server version control into a Git-based source control. It is targets the case when both TFS and Git are hosted on Visual Studio Team Services (VSTS, former Visual Studio Online), but a great deal of it will also apply to on-premise TFS or other git hosting options.

The general migration strategy is pretty simple: use git-tfs bridge to create a local git repository which mirrors the existing TFS one, then create a remote git repository on your hosting of choice and push from the tfs mirror into it. However, you may encounter a plenty of unexpected issues, so I’ll discuss workarounds for them.

Let’s start from the general outline.
First, install git and git-tfs.

Then, create a git repository in VSTS and start a master branch by creating an initial commit.

After that, proceed with the following commands:

git tfs clone https://company.visualstudio.com/DefaultCollection $/branchPath . --export --branches=none

git remote add origin https://company.visualstudio.com/DefaultCollection/git-project-name/_git

git push -uf origin master~2400:master
git push -u origin master~2200:master
git push -u origin master~2000:master
...
git push -u origin master:master

Adjust TFS urls in git tfs clone and git remote add according to your setup. We will discuss the magic numbers below.

Why push is so complicated?

You might expect to send the whole repository to VSTS with a simple push. However, if your repository is large enough, things won’t be so simple. What you’ll probably get instead is this cryptic message:

RPC failed; result=22, HTTP code = 404

A lot of results on the Internet suggest increasing the post buffer:

git config http.postBuffer 524288000

It’s good to do so, but probably it won’t solve the problem completely.
There is no explanation given for such VSTS behaviour, but there is an official workaround: push in chunks.

Firstly, you’ll need to know the total number of commits in your branch, which can be retrieved using the following command:

git rev-list master --first-parent --count

Now choose a number somewhat smaller than that (2400 in my example), and push all commits which are farther than that amount of commits from the branch top. Then repeat the process by decreasing the number (pushing commits up to more and more recent points in time), until you can finish it by doing just git push master. The chunk size is determined empirically.

Make sure the first push is forced (-f) to erase the initial commit you might have created to bootstrap the master branch. If you don’t do it, you’ll end up with a messy history.

Why export flag?

Export flag for tfs clone is needed purely for work item tracking. By default, git-tfs retrieves work items related to your changesets, and stores this information in git commit notes, but VSTS is unable to show links between a commit and a work item based on that. Instead, use export flag to include related work item information directly into a commit message, which will be understood and properly displayed by VSTS.

In case you wonder, the commit message format will be as follows:

<Original commit message>
work-items: #<work item id>
git-tfs-id: [<TFS project collection url>][<TFS branch path>];C<changesetId>

Why no branches?

In my experience, cloning all the branches resulted in impossibility to push to VSTS even despite the chunking. Due to the merge changeset, one chunk was just to big to fit, and Microsoft support was not really willing to help solving this issue. Therefore I decided to clone only trunk, which also made a nice linear history.

How to port changesets appearing in TFS after the migration?

If there is a pending work based on TFS (e.g. some items are under review or testing when you perform the migration), chances are new changesets might appear on TFS after the migration. It is possible to migrate them to git as well.

To do that, first fetch the latest changesets using git-tfs. Then go to your local git repository which is already connected to the remote VSTS git, and add a remote pointing to your TFS mirror created by git-tfs. Fetch such a remote and then perform a cherry pick of the commits representing the newly added changesets. It will port those changes into your local branch, and you only need to push them into the VSTS remote.

# in your git-tfs clone of the TFS repository
git tfs pull

# in your git repository connected to the remote VSTS git repository
git remote add tfs-mirror </path/to/git-tfs/mirror/on/your/pc>
git fetch tfs-mirror
git cherry-pick <commit from the tfs-mirror>

# ...proceed as usual
git push

Gotcha when using git-tfs from Git Bash

If you get the following message when running git-tfs under bash:

TFS repository can not be root and must start with “$/”

prepend the command with MSYS_NO_PATHCONV=1 https://github.com/git-tfs/git-tfs/issues/845 (or, use cmd or Powershell).

A note about credentials

On some systems, git command line doesn’t work with VSTS out of the box (you may encounter permission errors). If that’s the case, install Git Credential Manager for Windows. You might also need to create a token as described here.

ccnet and git

CruiseControl.NET has a built-in git integration via Git Source Control Block. Still, when using msysgit on Windows connecting to a github repository, you may find out that ccnet will hang on getting the sources until timeout kills the build. For me the simplest solution that worked was to run ccnet service under the user account I used git before (I guess it has something to do with SSH keys). In a different situation you may want to create a dedicated account, setup the git, including keys and all, and then use this account for CruiseControl service.