Migrating From visualstudio.com TFS to an In-House TFS Server (With Full History)

Posted by Jarren Long at 2017-08-11 21:34:34

For the past two years, I have been actively developing a collection of proprietary software products, which I have maintained in one of Microsoft's free visualstudio.com TFS repositories. For a single developer, this has been a great way to keep a complete revision history on the code base. Recently, I ran into a few caveats that have put me in a situation where it is time for the repository to be moved to a newly-created in-house Team Foundation Server. Thinking to myself "ok, I'll just migrate the repository real quick", I set to work. After a few hours of poking around VSO, it became painfully clear that there is no viable way to do this (while keeping the full revision history), since I did not have access to the underlying database for the repository. This gave me two options:

  1. Checkout the latest copy of the repository and commit it to the new repository (keep the latest source but lose the history), or
  2. Write some script that will loop through the repository, checking out each and every revision (starting at 1), and committing them one-by-one to the new repository (keep the source and the history, but may take DAYS or WEEKS to run).

After thinking on the options, I realized that they both sucked. I then remembered that I had written an article awhile back about Migrating from TFS to Git, so I pulled it up, re-read it, and though "I wonder if I can use this same technique to do a TFS to TFS migration?" The answer was yes! Though there was one downside: I kept all of the revision history, but I lost the Changeset Timestamps (all of the migrated history has the same check-in date). In the same fashion of migrating from TFS to Git, I had to use the "git-tf" tool for this process. Here's how I did it:

1. Installing Git-TF

Download and install the Git-TF utility from the CodePlex page here, and extract it somewhere on your computer. Don't forget to install the Java Runtime Environment (JRE) if you don't already have it, it is required for the tool to run.

2. Cloning the TFS repository (with full history)

The next step was a bit trickier. The tool needs the latest copy of the TFS repository that is going to be migrated. However, to clone the repository, I needed to configure Alternate Credentials on my visualstudio.com account. It took a while to find a recent enough article on how to do this. After some trial and error, it's easier than it should be:

  • Log into your visualstudio.com account
  • In the top-right corner, click on your name and select Security
  • In the new window, click on "Alternate authentication credentials" on the left
  • Make sure the "Enable alternate authentication credentials" checkbox is checked
  • Enter a secondary username and password to use, and click save

Once this was done, it was just one command from a Command Prompt to clone my TFS repo:

git-tf.cmd clone https://jarrenlong.visualstudio.com/DefaultCollection $/RepoIAmMoving --deep

Note: If you didn't follow the Git-TF instructions and add the Git-TF root directory to your system path, just use the full path to the git-tf.cmd file when executing the command. Since I only planned on using this tool once, this is what I did.

This did take a while to clone, as it is pulling the entire TFS repo history with it. Just let it cook until it's done. The repository that I was migrating had just over 4900 Changesets, so it ended up taking about 24 hours to do the complete clone. While this is running, it will be a good time to go ahead and create an empty TFS repository on your new server, if you haven't already done so. For this example, we'll say that my new Team Foundation Server is accessible at https://tfs.mynewserver.com/DefaultCollection, and the repo I created is called "RepoIAmHosting".

3. Performing the TFS to TFS Migration

Before you commit the repository to the new server, you need to make a few minor changes:

  • Using Windows Explorer, you need to open the .git directory that was created inside of the cloned repo. There should be a file in there named "git-tf"; rename it to something else. This file tracks all of the Changesets for the repo, but is bound to the old server. If you tried to commit the repo to the new server now, you will most likely get a "Changeset XXX not found" error.
  • Use a text editor to modify the "config" file in the .git directory. This file tells git where the server for the repository is located. In here, you need to modify the [git-tf "server"] section to point to the new server/repository.

For this example, I would change

[git-tf "server"]
collection = https://jarrenlong.visualstudio.com/DefaultCollection
serverpath = $/RepoIAmMoving


[git-tf "server"]
collection = https://tfs.mynewserver.com/DefaultCollection
serverpath = $/RepoIAmHosting

Save and close the config file. You are now ready for the actual commit! From the root directory of the repository you cloned, you just need to issue a "git-tf.cmd checkin --deep" command, which will start committing the complete repository to the new server. Again, this is going to take a while, but when the check-in is finished, you will have your complete repository history visible in the new TFS portal. Note: If you need to retain commit usernames, use the --keep-authors flag with this command (see git-tf documentation for info on how this works). In my scenario, I was the only developer on the project, so there was no need for me to do this.

As I said at the beginning of this article, there is only one downfall to this process, which is that each and every Changeset will have the same timestamp (+/- a few minutes). Sadly, this appears to be unavoidable (at least, I have not found a way to preserve the commit timestamps). There is one way that you can (partially) retain the timestamps though. By using the --metadata flag with the checkin command, git-tf will attach the additional metadata for each commit from the old repository. This will preserve the timestamps, however it makes the display of each commit look a little funky in the Changeset list for the repo when viewed from the web portal. Instead of showing the Changeset # and the description attached to the commit, the web portal will just show "Commit xyz (Timestamp)", and the description of the commit will be embedded further inside of the Changeset's details.