New VSS Import Tool: Faster than Ever

SourceAnywhere

SQL server-based version control software. Supports integration with Visual Studio, Eclipse, Dreamweaver and other IDEs; cross-platform access; unique caching mechanism for remote performance.

It offers special optimizations for VSS users. Familiar UI and working style makes developers feel right at home, while Import Tool ensures effortless data migration from VSS.

Download

VSSImportTool New Design

During the past months, we received quite a few feedbacks regarding our VSS Import Tool. Knowing how important the history of source code in VSS and easy migration to SourceAnywhere means to our customers, we decided to improve the VSS Import Tool to better serve our users.

After several weeks’ effort, we finally worked out an impressive version. According to one of our users, “I’m very pleased by great acceleration of importing (from 10 days to 8 hours)”.

Main improvements of VSS Import Tool:

* Greatly fastened the import speed by introducing multi-threading technology
* Enabled users work normally while import is in the process. Users are able to access/edit the data once the latest versions of files/projects have been imported
* Added support of importing the data from the error point if the process fails
* Optimized the import of Label
* Optimized the project selection. Refreshing the projects is much faster now.

Background

Visual SourceSafe uses file system to store the source code, while Dynamsoft SourceAnywhere puts the data in SQL Server for enhanced security and integrity. With a VSS Import tool, users can easily migrate all their data histories from VSS to SourceAnywhere.

The pain is that, most users have been using VSS for many years and have quite large VSS DBs. Plus, VSS API doesn’t provide full interfaces to interact with VSS database. So migration away from Visual SourceSafe in an accurate and fast way is not easy.

The previous versions of SourceAnywhere offered a VSS Import Tool, but the old version has its glitches. Our team decided to re-design VSS Import tool for a better user experience.

Problems in the old version of VSS Import tool

1. The “label” import was not so smooth. Too much time is spent on importing the labels and this process has a high error probability. Usually there are many labels in a VSS database.

2. The tool is single threaded, so it cannot take advantage of multi-CPU systems.

3. The import can be time-consuming sometimes for a big VSS DB. And users can only start using/evaluating SourceAnywhere after the whole import process is done.

4. Once an error occurs during the import, the whole process needs to be started from scratch.

SourceAnywhere features:

1. Data is compressed before storing in SQL Server.

2. SourceAnywhere calculates the data delta and performs compression before checking-in data to the SQL Server.

3. In SQL Server, item histories (table) and file content (table) are in reference relationship. They are stored separately so the content of files can be updated separately.

4. When performing Check In or Check Out on the latest version of an item, only the content of the latest version and the nearest full version (Full version is in contrast to Delta. Every several versions, a full version is created as a base. Latest version = nearest full version + delta.) are needed.

5. Label, as a tag/collection on the existing histories, doesn’t modify any items. Thus it doesn’t affect user operations.

In view of above considerations, we redesign the import tool in the new version.

Import Steps

The new VSS import tool separates the whole import process into 5 modules. Each module uses a separate thread.

(modules in VSS import tool)

1. Import Monitor: Monitors the whole import process and lists the ongoing steps and their statuses. A dialog box prompts to notify the user a successful import.

2. Import Item: Imports anything about an item (name, location in a project tree, history version info, etc.) but the label/pin info and file content. A corresponding Import File Content (Module 3) thread will be fired once an item is imported. This way, items and file content can be imported simultaneously. A new import process is needed if crash, cancel or fatal error occurs during this step.

3. Import File Content: Imports the content of all files. This step runs in two concurrent threads.

First the module scans for items that have been imported and then starts importing the according file content.

The content import consists of the following 4 parts in sequence: import the latest version of an item; import the nearest full version; scan and see if there are any items that are ready (If yes, import their latest versions and according full versions. Otherwise, continue to the next part.); import the old versions of items.
four parts in Import File Content

(Steps in Import File Content)

Once an item’s nearest full version is imported successfully, you can perform version control operations, such as Check Out, Get, Check In, Undo Check out, on its latest version.

If your machine doesn’t have enough resources to run 2 concurrent threads, you can cancel/pause the content import. It’ll be triggered automatically when there is another item imported or you can manually start the tool. Also, you don’t need to worry if some errors occur during the content import. With break-point transmission applied to the new design, restarting the import tool will restore to the last interruption point.

4. Import Pin: Imports all files’ Pin status. This step will take place only after Import File Content completes successfully. So if an error or a cancel occurs in this step, it will start from this step next time rather than importing all the content from the beginning. However, the status of the current step won’t be kept. Thus, you may find error messages in the log file saying some Pin statuses have already been imported but it’s OK.

5. Import Label: Imports selected Labels. Arriving here, you can begin using SourceAnywhere at ease since Label won’t update existing items in the repository. You can select the labels to be imported or the tool will import all the labels by default. You can click Pause or Cancel during the process and re-select the labels. Interruptions during the Label import can only be restored to the beginning of this part. This case, you may find errors like the label already exists but it’s fine.

You can click “Complete” button to finish the whole import.

Import Selected SourceSafe Projects

With previous versions of VSS Import Tool, during the initialization of Select SourceSafe project, the whole directory tree will be displayed before you’re able to perform the next step. When the directory tree is large, the process can be really time-consuming. You have to wait for the whole directory tree to be listed even when you only want to import $/ or a subdirectory under $/. Obviously, the old design is inconvenient and not so user friendly.

In the new version, the operation of unfolding directory tree is performed in a separate thread. The directory tree is expanded layer by layer. First it expands the subfolders under $/ and then expands each subfolder recursively. This way, it’ll be much faster for users to select $/ or a subfolder under $/. Once a directory is selected, you can carry on importing without further waiting and the thread listing the directory tree will be ceased automatically.