Last updated 15 June 1998
This is the FAQ for CVSup. Please send comments and suggestions to cvsup-bugs@polstra.com.
CVSup is a software package for transferring and updating collections of files across a network. It consists of a server called cvsupd and a client called cvsup.
CVSup is faster (often by an order of magnitude) and more flexible than traditional network update packages.
CVSup can efficiently update any kind of file. It can even update Unix device nodes, symbolic links, and hard links. CVSup supports several different algorithms for updating various kinds of files. It tries to use the most efficient method for each file. For example, RCS files are updated using a specialized algorithm that takes advantage of their structure to greatly reduce the update time. Log files (which are changed only by appending new text at the end) are likewise updated by a special algorithm that transmits only the new text. Other text and binary files can be updated efficiently by the rsync algorithm, which is built into CVSup.
You can get CVSup in from ftp://ftp.FreeBSD.org/pub/FreeBSD/CVSup/, and from FreeBSD's FTP mirror sites all over the world.
CVSup is also available directly from the author's FTP server, ftp://ftp.polstra.com/pub/FreeBSD/CVSup/. But please avoid this server if possible. It has a relatively slow link to the Internet.
At these sites, you will find the following files:
| cvsup-x.y.tar.gz | Full sources (read this) |
| cvsup-bin-x.y.tar.gz | FreeBSD static binaries for the client (with GUI) |
| cvsup.nogui-bin-x.y.tar.gz | FreeBSD static binaries for the client (without GUI) |
| cvsupd-bin-x.y.tar.gz | FreeBSD static binaries for the server |
Here, "x.y" stands for the CVSup version number. If there are several versions, get the highest-numbered one.
Yes, there are. The PostgreSQL project has publicly available binaries for the following platforms:
You can find them at ftp://ftp.postgresql.org/pub/CVSup/.
Please address all mail regarding CVSup to cvsup-bugs@polstra.com.
An RCS file stores many versions of a single source file, all in one place. During the lifetime of a project, its source files evolve. Each source file starts with an initial version. Over time, changes are made to the source files to fix bugs and add features. At key moments, a programmer wish to save the current version of a source file. He can do this by checking it into its corresponding RCS file. From a given RCS file, one can later extract any desired version of the source file. This makes it easy to undo changes that turned out to be ill-advised, or to recreate an earlier release of the software.
A CVS repository is a collection of RCS files that are managed together. For example, the RCS files for all of the sources in a project would typically be stored together in a CVS repository.
Yes, try Jordan Hubbard's "cvsupit" package. You'll find it at ftp://ftp.freebsd.org/pub/FreeBSD/CVSup/cvsupit.tgz.
Make a cvsupfile that looks something like this:
*default host=cvsup3.freebsd.org
*default base=/usr
*default prefix=/usr
*default release=cvs
*default delete use-rel-suffix
*default tag=.
src-all
This cvsupfile will create a directory tree "/usr/src" and populate it with all of the main FreeBSD sources except the export-restricted cryptographic code. It will also create a tree "/usr/sup" containing the checkouts files that CVSup uses to maintain state between updates. If you want your source tree to be somewhere other than "/usr/src", change the "prefix=" clause. If you want the checkouts files to be located somewhere other than "/usr/sup", change the "base=" clause.
Be sure not to leave out the "tag=." clause. That tells CVSup to use checkout mode to send you the most recent version of each source file.
Simply add one more line to the end of the cvsupfile above:
cvs-crypto
Note: If you are located outside of North America, then you must also change the "host=" clause to specify a non-North American server. For example:
*default host=cvsup.internat.freebsd.org
See here
for a complete list of public CVSup servers carrying the FreeBSD
sources.
No, you don't. CVSup is capable of "adopting" your existing sources and bringing them up to date. If you think about it, your situation is the same as if you had been updating with CVSup all along but had lost your checkouts files. And CVSup uses the same technique to deal with both situations. It uses checksums to determine which revisions of the files you have currently, and then updates them in the appropriate ways to transform them into the latest versions.
Look here for the safest way to do this.
Yes, you do. The greater the distance between the revisions you have and the revisions you want to get, the greater the chance that you'll miss some important file deletions. Luckily, if you know approximately or exactly which versions of the files you are starting with, you can reduce or eliminate this risk. You do this by updating twice. First, you tell CVSup to "update" you to the versions that you already have. This won't change any of your files, but it will create a checkouts file that precisely reflects what you have currently. Second, you update again, this time telling CVSup which version you really want. On the second update, CVSup will have all the information it needs in order to know which files to delete.
This is a little tricky and there are a couple of important details we haven't mentioned yet. So we'd better give you an example. Suppose you installed FreeBSD-2.2.5, including sources, from the CD-ROM. Now you decide you want to use CVSup to track the 2.2-stable sources. For your first update only, use a cvsupfile like this:
*default host=cvsup2.freebsd.org
*default base=/usr
*default prefix=/usr
*default release=cvs
*default delete use-rel-suffix
src-all tag=RELENG_2_2_5_RELEASE list=cvs:RELENG_2_2
For subsequent updates, change the last line to:
src-all tag=RELENG_2_2
The unmentioned details are in this last line. First of all, how do you know what to use for the tag? The answer is, you have to consult a list. See the "Configuration" section of the CVSup chapter in the FreeBSD Handbook for a list of tags that are valid for the FreeBSD sources. The important point is that the tag for your first update should correspond to the version of the sources that you already have. And the tag for your second and subsequent updates should correspond to the version of the sources that you want to receive.
Second, what is this business with the "list" keyword? It is rarely used, but this is a situation where it is necessary. When CVSup creates or consults one of its checkouts files, it uses a filename which by default is based on the "release" and "tag" values for the collection. Specifically, the usual name of a checkouts file is "checkouts.RELEASE:TAG", where RELEASE and TAG are the "release" and "tag" settings in your cvsupfile.
In the rather special situation we are addressing here, the naming conventions for the checkouts files cause a problem. By default, our first update would produce a file named "checkouts.cvs:RELENG_2_2_5_RELEASE", while the second update would look for a file named "checkouts.cvs:RELENG_2_2". In order for the second update to benefit from the information garnered in the first, both updates must use the same checkouts file. The "list" specification in the first cvsupfile allows us to accomplish this. It overrides the default suffix in the name of the checkouts file, and forces it to have the same name as will be used in subsequent updates.
Admittedly, this is arcane. But you only have to do it once.
It doesn't make any difference. This cvsupfile:
*default host=cvsup3.freebsd.org
*default base=/usr
*default prefix=/usr
*default release=cvs
*default delete use-rel-suffix
*default tag=.
src-all
could just as well be expressed as a single line, like this:
src-all host=cvsup3.freebsd.org base=/usr prefix=/usr release=cvs delete use-rel-suffix tag=.
Using the "*default" lines can make your cvsupfile easier to read, by shortening the lines. They can also make your cvsupfile more concise if you are receiving several collections.
It gives CVSup permission to remove files on your machine. For example, suppose you have a file "foo" which you originally received using CVSup. Now the maintainer of the server host deletes "foo". When you next run CVSup, if "delete" is specified in your cvsupfile then CVSup will delete "foo" on your machine. Otherwise it will leave it alone.
Except for a few unusual applications, you should always specify "delete" in your cvsupfile.
Originally CVSup was designed to be a drop-in replacement for sup. Because of that, the defaults had to be the same whether they made sense or not.
Both you and your friendly server administrator need to upgrade to CVSup 15.4 or later. That will solve this problem.
CVSup upgrades an RCS file by deconstructing it on the server, sending only the pieces that have changed to the client, and reconstructing the file on the client. (It's roughly patterned after the transporter on the starship Enterprise.) CVSup then compares the MD5 checksum of the reconstructed file with that of the original on the server, to make sure that the process worked correctly.
Unfortunately, recent releases of CVS have introduced some gratuitous changes in the format of the RCS files that they write. These are simply changes in white space, which have no bearing on the logical meaning of the file. However, the result is that an RCS file constructed by the CVSup client no longer matches the original file byte-by-byte as before. Even though the reconstructed file is logically identical to the original, it does not have the same checksum. This causes older versions of CVSup to reject the updated file and use a fixup to re-transfer the entire file.
To solve this problem, CVSup 15.4 introduces a logical checksum which is used only for RCS files. Instead of blindly computing a byte-by-byte checksum over the entire file, the new checksum algorithm carefully canonicalizes the file so that irrelevant white space differences are ignored. The logical checksum should make CVSup immune to any future problems of this nature.
You are behind a firewall which is blocking attempts by the CVSup server to establish a second TCP connection to your client. Add the option "-P m" to your cvsup command line, and everything should work fine.
Yes, the statically linked FreeBSD binaries work fine under other BSD-derived operating systems. But for some of them, including BSD/OS, you have to add "@M3novm" to the command line.
CVSup is written in Modula-3, and its runtime system uses a sophisticated garbage collector which exploits hooks into the VM subsystem of the operating system to gain better interactive performance. This feature stumbles upon an incompatibility between BSD/OS and FreeBSD, causing the core dumps. The cryptic argument "@M3novm" disables the VM hooks and makes it possible to run FreeBSD binaries under other BSD-derived operating systems.
Those strange files are RCS files, and CVSup sent them to you because your cvsupfile told it to. When you ask CVSup to send you updates from a CVS repository, there are two different things you can ask for. First, you can ask for the source files to be extracted from the repository and sent to you. That is apparently what you wanted in this case. Second, you can ask for the raw RCS files (containing all versions of the sources) to be sent to you.
These two modes of operation are fundamentally different. Both work from RCS files in a CVS repository on the server host. But they use the RCS files in different ways. The first mode, called "checkout mode," extracts a particular version of the sources from the RCS files, and sends you that version. The second mode, called "CVS mode," sends you the RCS files themselves, in the same form as on the server. In the cvsupfile, the "tag" and "date" keywords control which mode is used. If either of these keywords is present, then CVSup uses checkout mode to send you a set of source files. If neither "tag" nor "date" is present, then CVSup uses CVS mode to send you the RCS files.
Assuming you want the most recent version of the sources, you simply need to add "tag=." to your supfile.
In order to update your files efficiently, CVSup needs to know what you've already got. It stores this information in files called "checkouts" files. Each time you run cvsup, it reads your checkouts files to see which files (and which revisions of them) you have. As it updates your files, it also updates the information in the checkouts files.
Confusingly, checkouts files are also sometimes referred to as "list" files.
It is not a big problem. If CVSup can't find a checkouts file that it needs, it falls back on other methods of determining which files you have. One such method is to compute checksums (MD5 file signatures) for each of your files, and use those to figure out which file revisions you have. This is perfectly safe, but it is inefficient. It slows down your update and also puts a heavier load on the server.
CVSup will detect the problem and quit, with a message suggesting that you delete the corrupted file and try again. If you follow the suggestion, it will complete your update using the fallback methods mentioned above, and recreate your checkouts file for next time.
There is just one thing that can go wrong, and it's not very serious. CVSup will only delete files that are listed in its checkouts file. Thus if you lose your checkouts file, and then a file "foo" is deleted on the server host, and then you run CVSup, your file "foo" will not be deleted.
Files are never supposed to be deleted from a CVS repository, so this isn't much of a problem in real life. But to be safe, you should run CVSup sooner rather than later, if you find that you've lost a checkouts file.
If you are careful and if you understand what is going on behind the scenes, you can make this work. CVSup was designed to allow extra locally checked-in revisions to coexist in the RCS files along with the revisions taken from the master repository on the server. Because CVSup understands the structure of RCS files, it is able to bring in new revisions from the server without disturbing your own revisions that you have checked in locally. Unfortunately, certain logistical issues make this capability awkward to use in practical situations.
To keep local revisions in your copy of the CVS repository, you have to:
We cover these topics separately below.
Simply remove the "delete" keyword from your cvsupfile. When present, this keyword gives CVSup permission to delete extra revisions from the RCS files. It also permits CVSup to delete entire RCS files, provided that it created them in the first place. By removing the "delete" keyword, you can prevent CVSup from removing extra revisions as well as entire RCS files. However, it is very important to understand the consequences of removing the "delete" keyword.
First, suppose that one or more RCS files are intentionally removed from the CVS repository at the master site. Without the "delete" keyword, CVSup will not propagate these file removals to your site. Thus you will have some RCS files in your repository that otherwise shouldn't be there.
In an ideal world, this would not be a problem. That is because, in an ideal world, one never deletes an RCS file from a CVS repository. Thus, the situation should never arise.
Unfortunately, most administrators don't manage their repositories quite so idealistically. Committers make mistakes and check files into the wrong locations. Most administrators will manually repair such errors by moving the files in the repository, rather than live with the mistake forever. As another example, in any large repository, some files eventually become completely obsolete. Eventually, developers will complain about the wasted disk space and the clutter in the repository, and the repository administrator will respond by deleting the files.
Usually, undeleted RCS files don't cause any problems, and they can be safely ignored. This is especially true if the repository administrator at the master site has taken care to mark the files "dead" on all branches with "cvs remove" a few weeks before completely removing the unwanted RCS files. Nevertheless, extra RCS files can cause problems in some cases, and you need to be aware of that.
It is difficult, because CVS chooses the revision numbers itself. One way around this is to modify CVS slightly. Before discussing that, though, the first rule of thumb is to create a new branch to hold your local changes. Don't try to check your local changes into the main branch or a branch that exists in the master repository. Doing so would make revision number collisions very likely, sooner or later.
If your local revisions are on their own branch, then the problem is reduced to ensuring that your branch has a unique revision number which will never be duplicated in the master repository. The easiest way to accomplish this is to modify CVS. The version of CVS released with FreeBSD includes such a modification. In that version, you can influence the revision numbers of branches by setting the environment variable CVS_LOCAL_BRANCH_NUM. This variable should be set to an integer value, and CVS will use that as the starting point when choosing revision numbers for new branches. By default, CVS allocates branch numbers starting with 1. So a high value such as 1000 makes a good choice. New branches at the master site will receive low revision numbers, while your own local branches will receive high revision numbers. Thus the two won't conflict.
When using this method, it is of the utmost importance that committers do not set CVS_LOCAL_BRANCH_NUM when creating branches in the master repository.
If you don't have a version of CVS that supports CVS_LOCAL_BRANCH_NUM, it is still possible to avoid conflicts with little cooperation from the master site. Simply create a branch in the master repository, declare it to be reserved for local modifications, and leave it unused in the master repository.
The basic problem is that CVSup is written in the programming language NotC. What is NotC? Well, to most people, it really doesn't matter. The important thing is that NotC is not C. Consequently, it takes a little extra work to use it.
In the case of CVSup, NotC means Modula-3. Modula-3 is an efficient compiled programming language with excellent built-in support for important features such as garbage collection and threads. For more information about Modula-3, see the Modula-3 Home Page.
Fortunately, Modula-3 ports exist for many platforms. If there is already a port for your platform of choice, then you've already cleared the biggest hurdle to building CVSup.
A second problem that many people run into is that building the Modula-3 compiler and runtime system requires a lot of virtual memory. You will need on the order of 64 MB of available virtual memory to build it. (Building CVSup itself requires less than that.) You will also need to ensure that your resource limits are set high enough. These are controlled by the "ulimit" command in sh-like shells, or by the "limit" command in csh.
Third, by default you have to have X11 on your system to build the Modula-3 runtime. It is possible to build Modula-3 without X11, but to do so you have to edit the file m3/src/m3makefile and comment out the packages that depend on X11.
No, CVSup does not currently work on Windows platforms. It relies on a number of features which are specific to POSIX-like operating systems.
No. All releases of CVSup that have ever been made public will inter-operate with each other. You can use an old client with a new server, or a new client with an old server. Of course, the feature set is limited to what is supported by the older component. So it is always a good idea to upgrade both the client and the server when a new version is released.
No. Although the CVSup client can use supfiles written for SUP, the two packages are otherwise incompatible.