I have been working on migrating a VSS repository to SVN and this post includes an updated version of the Perl migration script (migrate.pl) found at "Migrating from Visual Source Safe to Subversion (VSS to SVN)" by Magnus Hyllander which in turn is based on "Visual SourceSafe to Subversion Migration" by Brett Wooldridge (this site doesn't seem to be available anymore).
I have made the following changes:
3.00 - Changes by Neil Sleightholm:
- Improved generation of atoms to make resume more reliable.
- Handle pinned files.
- Print migration start/end date and time.
- Improve status ('sugar') feedback.
- Simplify extract status code.
- Supply username and password for admin functions.
- Code reformatting.
- Added support for cutoff date to allow only newer files to be migrated.
- Always dump user names during migration.
- Add support for usernames containing spaces.
- Exclude additional vss warning message.
- Change to UK date format. Add commented out code to allow for european and US date formats. TODO: Make this a command line option.
- Add version number.
3.01 - Don't import vss files e.g. files ending .vssscc, .vspscc etc.
3.02 - Allow vss project $/ e.g. all of vss.
3.03 - Don't import vss files - .vsscc
3.04 - Improve handling of cuttoff date if it is not set.
3.05 - Changed so that svn url is the root and the vss path is append to it e.g. $/MyPath is imported in to url svn://Import as svn://Import/MyPath
- Remove redundant commented out code.
- Add support for multiple vss projects on the command line.
This new version can be download from here: http://www.x2systems.com/files/migrate.pl.txt (rename to migrate.pl).
This version was modified to support a US install of VSS (thanks Rory Kingan): http://www.x2systems.com/files/migrate_us.pl.txt (rename to migrate.pl).
Edit 17-09-2008
Command line examples:
Assuming these environment variables have been set
set SS_HOME=C:\Program Files\Microsoft Visual SourceSafe
set SSDIR=D:\VssData
set SVNREPOS=file:///D:/svnrepos2/migrate
set VSSPROJ="$/X2 Systems"
set CUTOFF=20050701
This is a full migration:
perl migrate.pl --repos=%SVNREPOS% %VSSPROJ% --cutoff-date=%CUTOFF%
This will migrate the latest changes:
perl migrate.pl --migrate-latest --repos=%SVNREPOS% %VSSPROJ% --cutoff-date=%CUTOFF%
This will resume a migration:
perl migrate.pl --resume --repos=%SVNREPOS% %VSSPROJ% --cutoff-date=%CUTOFF%
This will generate a file containing all the VSS users:
perl migrate.pl --dumpusers --repos=%SVNREPOS% %VSSPROJ% --cutoff-date=%CUTOFF%
Force multipliers
3 months ago
Posts
34 comments:
Apprently there may be a problem with this script when using VSS 2005 on win2000. See here for a work around:
http://forums.microsoft.com/msdn/showpost.aspx?postid=124591&siteid=1&sb=0&d=1&at=7&ft=11&tf=0&pageid=1
Neil
It is also mentioned here: http://support.microsoft.com/kb/910793/en-us
Can't get this to work with Perl version 5.10 (ActivePerl) - get an error:
$* is no longer supported at migrate.pl line 380
...although that might have been because I stupidly used -sshome as the option, rather than --sshome
...but worked with Perl version 5.08 (ActivePerl) (on Windows XP)
This looks great. It would be useful to provide one or more example commands using this.
thanks,
Rory
As requested I have added some command line examples.
Thanks for the examples!
I get an error as below, any thoughts?
thanks,
- Rory
D:\vss_to_svn>perl migrate.pl --ssrepo=D:\SourceSafe_RM_V23 --sshome="D:\Program
Files\Microsoft Visual SourceSafe" --repos=file:///D:/Repositories/RMProject/trunk $/Code
Visual SourceSafe to Subversion Migration Tool - v3.05
Original by Brett Wooldridge (brettw@riseup.com)
Modified by Magnus Hyllander 2006-12-05
Modified by Neil Sleightholm v3.xx
SourceSafe repository: D:\SourceSafe_RM_V23
SourceSafe directory : D:\Program Files\Microsoft Visual SourceSafe
Subversion repository: file:///D:/Repositories/RMProject/trunk
SourceSafe project : $/Code
History cut off : not set
Migration started: 2008-09-17 18:22:46
Project: $/Code
Building directory hierarchy: done (117 dirs)
Building file list: done (2134 files)
Building file histories: done (2439 versions)
Building user list: done (2 users)
Building atoms: 100%ERROR: Files would be checked in in an unexpected order
D:\vss_to_svn>
According to the helpful folk here the warning about $* means the .pl script needs to be modified slightly to work with recent versions of perl.
I'm using ActivePerl version which is effectively perl 5.10.
I've commented out the lines that set $* to 1 or 0 and then added 'sm' (without quotes) to the end of the regular expressions between the lines that set $* = 1 and $* = 0.
That fixed the warning at least... I'm still having some other problems but if I solve them I'll post here.
- Rory
I don't think that is the problem as I was using the same version of Perl. The most likely problems are date format related, there are some comments in the code related to this (the code defaults to UK date format). The other issue is hilighted in an earlier comment and relates to the version of VSS. Feel free to email me directly if you still have issues with this.
A coworker and I were using this script, and we ran into the "ERROR: Files would be checked in in an unexpected order" issue. After looking around the code, we found that the script was not converting the sourcesafe time from 12-hour clock notation to 24-hour clock notation properly when the time is 12am.
We made a change that should fix this. I am putting a diff of the change below:
*** migrate_us.pl Thu Oct 02 00:02:57 2008
--- migrate_us-mod.pl Thu Oct 02 00:03:44 2008
***************
*** 553,574 ****
--- 553,579 ----
#($day,$month,$year) = split('/', $date); # UK date format
($hour,$minute) = split(':', $time);
#RJK: If time has trailing a or p then remove it and add 12 to the hour if appropriate
if ($minute =~ /(..)([aApP])/)
{
$minute = $1;
if ($2 =~ /[pP]/ && $hour != 12)
{
$hour += 12;
}
+ # TNS/RMB: If the hour is midnight, proper hour should be 0.
+ elsif ($2 =~ /[aA]/ && $hour == 12)
+ {
+ $hour = 0;
+ }
}
$year = ($year < 80) ? 2000 + $year : 1900 + $year;
$datetime = sprintf("%04d-%02d-%02d %02d:%02d",$year,$month,$day,$hour,$minute);
$timestamp = POSIX::mktime(0, $minute, $hour, $day, $month - 1, $year - 1900, -1, -1, -1);
if (!defined($timestamp)) {
print STDERR "$file:\n";
print STDERR "$line => $year-$month-$day $hour:$minute => $timestamp\n";
print "\$timestamp is undef!!!\n";
exit;
}
Thanks for the update, I have corrected the US download.
Thanks... I've used the previous version and now the latest 3.07 for the last rogue VSS repositories finally being sunset. For Canadian users (running their systems as Canada locale), change the following date format line to enable the migration:
#($month,$day,$year) = split('/', $date); # US date format
($day,$month,$year) = split('/', $date); # CDN date format
Other changes I have done do this script:
1. lc the $user used during migration to match the format used in SVN (VSS was inconsistent with case in my experience.)
2. Load a @passwords hash from my SVN database USERS.CONF and use the correct password for all svn commit commands.
I'm getting "ERROR: Files would be checked in in an unexpected order" and looking into the problem it looks like one of the user's had their clock months out-of-date, and consequently the script is attempting to create version 1 of a file after version 2 has been checked-in. Can anyone suggest a solution to this problem?
Thanks,
Andy
Update: I have fixed this problem by running through the histories (in build_histories()) and replacing the dates from any out-of-order entries with 01-01-1990.
Add this code before the histories are written to histories.txt:
# If one of the user's had their clock set wrong on their PC then an error will be detected later:
# ERROR: Files would be checked in in an unexpected order
# Fix this by adjusting the date to something way back in the past (the real date has been lost so
# this will not cause anymore damage to the histories).
my $last_file = "";
my $last_timestamp = 0;
for (my $i = 0; $i < $#histories; $i++)
{
my ($file, $version, $datetime, $timestamp, $user, $action, $comment) = split(',', $histories[$i], 7);
if ($file eq $last_file && $timestamp > $last_timestamp)
{
$timestamp = POSIX::mktime(0, 0, 0, 1, 0, 90, -1, -1, -1);
$histories[$i] = join(',', $file, $version, "1990-01-01 00:00", $timestamp, $user, $action, $comment);
print STDERR "Replaced date on $file v$version as it was out-of-order";
}
else
{
$last_file = $file;
$last_timestamp = $timestamp;
}
}
I had a similar but legitimate problem where VSS check-ins spanned the DST clock fallback period. As you mention, manually tweaking the histories.txt file and normalizing the dates/times and resuming the migration is an effective way of working around this.
Thanks for the feedback, it just makes me realise why moving away from VSS was such a good idea!
I get the same error:
Building atoms: 100%ERROR: Files would be checked in in an unexpected order
Checked by histories.txt files...it shows
DataAccessFactory.cs,2,2008-03-11 05:12,1205192520,Shinto.t,checkedin,
the date format is yyyy-dd-mm.
I did the changes to
Added
($year,$day,$month) = split('-', $date); # yyyy-dd-mm date format
Commented out
#($year,$month,$day) = split('-', $date); # yyyy-mm-dd date format
and resumed..
still i get the same error message.
Please help!
Thanks,
Rajesh
Take a look at Andy Duplain's comment above, I think he found a solution.
Please update me where to apply that code exactly in between lines. I am not that good at scripting.
Anywhere in between this will help
sub build_histories
{
my $last_file = "";
.
.
.
$last_file = $file;
$last_timestamp = $timestamp;
}
}
.
.
.
.
open(HIST, ">histories.txt");
foreach my $hist (@histories)
{
print HIST "$hist\n";
}
close(HIST);
$oldname =~ s/./\b/g;
$count = @histories;
print "$oldname\b\b\b\b\b\b\b\b\b: done ($count versions)" . substr($pad, 0, 20) . "\n";
$PHASE = 3;
}
)
Rajesh.
I think it goes just before line 484:
open(HIST, ">histories.txt");
Correct; it goes where Neil has mentioned. I have heavily modified the script to migrate directly to Mercurial and will be putting up the final version of the script shortly on my blog (http://chimpchess.blogspot.com) once it's been properly tested.
I have ripped out support for multiple VSS projects and added the ability to translate VSS labels into Mercurial tags.
I've added this for German-VSS users:
($day,$month,$year) = split('\.', $date); # German date format
Thanks for the script!
Testing the script i faced a few issues
- I had to remove all "2>&1". It did not work with this term. Is the removing "dangerous"? (I don't know perl at all.)
- We are using a german VSS. Therefor the output of History is in German and I have to "translate" the statements in proc_history. Has someone done this before? (Of course, it's manageable, but why doing it twice?)
- There seems to be something wrong with the directory-handling. I get a lot of empty directories, which do not exist in VSS.
- If a directory in VSS has the same name a upper directory, the script does not work. (e.g.: /folder/folder/...)
Any help is appreciated.
- Removing those will probably stop it working, they redirect stdout and stderr to the same file. They are standard DOS commands not perl so I am not sure why you would need to remove them.
- I don't know if anyone has translated the history commands. Have you considered using English VSS just to run the import.
- I am not aware of any folder issues, may be removing the "2>&1" has caused a problem.
If you could create a sample VSS database that demonstrates the problems I will see if I can debug it for you.
I found the solutions.
If you use a localized VSS, just change the language for the SourceSafe user Interface. Start VSS, and select Tools->Options->International Options and change the Application Language.
The problem with the folders can be solved, if you change the line (344 at the moment):
my $cmd = $SSCMD . " Dir \"$proj\" -I- -F-";
to:
my $cmd = $SSCMD . " Dir \"$proj\" -I- -F- -R-";
With the -R_ option, VSS does not list the folders recursively, which caused the problems.
And the "2>$1"? I have no idea, why it is not working. Maybe because I start the tool from a *.bat file. But it seems to work fine without it. I will not get the error messages, but that's fine for me.
That is odd -R- means turn off recursion (the final - negates the option). Are you using VSS 2005 or VSS 6.0 - I used 6.0 for my migrations so that might explain the differences.
I am using VSS 2005. And I have the option "Act on projects recursively" turned on :-(
Most likely this caused the problems. All the folders were listed recursively. After turning off this behavior with the option -R- everything works fine now.
Hi, looks like this is a great script!
I have only one big problem: I have written my VSS-Checkin Comments in german, which means that there are some special codes (äöü,ß).
In the script exists a section where these codes get translated:
"#Translate character codes from CP437/CP850 to UTF-8..."
The ü is missing, I tried to complement this character, but did not find any codepage where to get it: Either ASCII, CP437 or CP850, nothing worked, it mainly translated 'l' to 'u' or similar.
Has anyone run into this problem or can tell me the correct character codes for 'ü','Ü','ß'...?
You can use charmap to find the values to replace them with (I think you want 252,220,223) the problem is finding out what they current are. If you add:
if ($DEBUG)
{
print STDERR "$comment\n";
}
just after the # Translate comment you should see the comments in the migrate.log file. You can then open this in binary mode to see what the character codes are. I hope this helps.
Thank you very much!
I found the value with a brute-force: I tried to replace from 200 to 219 with a to t and looked which character is replaced into the ü. In my case, it was 201.
I had a similar problem (like some other users) with the date which I solved quickly: my vss (2005) gave me the date in dd.mm.yy not dd/mm/yy, so I had to change the split-command...
I'm trying to make a migration from VSS to SVN but I'm not able to make it work...and neither to get the VSS users to be saved in the users.txt file.
I show you my command. Premise: I have saved both the VSSRep folder in which it is saved the repository and the folder of the Microsoft Visual SourceSafe program into a folder "migr" savend in C (so the path is C:\migr\VSSRep for the repository). Moreover I have saved in "migr" also a folder "SVNtemp" in which I want to collect the migrated repository.
So, my commands are
set SS_HOME=C:\migr\Program Files\Microsoft Visual SourceSafe
set SSDIR=C:\VSSRep
set SVNREPOS=file:///C:/migr/SVNtemp
set VSSPROJ="$/ProgettoTest"
set CUTOFF=20050701
perl migrate.pl --dumpusers --repos=file:///C:/migr/SVNtemp --cutoff-date=20050701
and I have this result:
"
Useless use of sort in void context at migrate.pl line 327.
Name "main::DEBUG" used only once: possible typo at migrate.pl line 158.
Visual SourceSafe to Subversion Migration Tool - v3.05
Original by Brett Wooldridge (brettw@riseup.com)
Modified by Magnus Hyllander 2006-12-05
Modified by Neil Sleightholm v3.xx
SourceSafe repository: C:\VSSRep
SourceSafe directory : C:\migr\Program Files\Microsoft Visual SourceSafe
Subversion repository: file:///C:/migr/SVNtemp
History cut off : Mon Jul 11 00:00:00 2005
Migration started: 2016-01-11 15:40:34
Building file list: done (0 files)
Building file histories: done (0 versions)
Building user list: done (0 users)
Building user list: done (0 users)
Users.txt file has been created. Use the list of users in this
file to create matching user accounts in Subversion. Ensure that these
accounts initially have NO AUTHENTICATION, otherwise the migration will
likely fail. Alternatively, you can use the --force-user option to
create all files with the same username. Either way, you can resume
this migration, picking up from this point, by using the --resume
option on the command line.
"
Where am I wrong?
I thank gratefully whoever will answer.
MOREOVER:
I have tried to fix the error caused by $* in this way:
"
#$* = 1;
$/ = ':';
$cmd = $SSCMD . " Dir -I- \"$proj\"";
$_ = `$cmd`;
# what this next expression does is to merge wrapped lines like:
# $/DeviceAuthority/src/com/eclyptic/networkdevicedomain/deviceinterrogator/excep
# tion:
# into:
# $/DeviceAuthority/src/com/eclyptic/networkdevicedomain/deviceinterrogator/exception:
s/\n((\w*\-*\.*\w*\/*)+\:)/$1/gsm;
#$* = 0;
"
and later in the script
"
#$* = 1;
$out =~ s/\n?Project.*rebuilt\.//gsm;
$out =~ s/\n?File.*rebuilt\.//gsm;
$out =~ s/\n.*was moved out of this project.*rebuilt\.//gsm;
$out =~ s/\nContinue anyway.*Y//gsm;
#$* = 0;
"
It has been a very long time since I looked at this code. There were some comment earlier about using Perl 5.08, may be that will help. I also remember that it was very sensitive to to the date format, I did all my testing with UK format.
Post a Comment