[UPHPU] Source Control Questions

[Lamont Peterson]

I hate top posting, but wanted to ask a small question out of context.  I 
can't see any good reason for having taken my responses out of context 
against nearly all of Wade's and others' in this case, so why was it done?

On Friday 29 December 2006 04:28pm, Bryan Petty wrote:
> On 12/29/06, Lamont Peterson <peregrine at openbrainstem.net> wrote:
> > One of the things that Bazaar (baz) & Bazaar-ng (bzr) do for you that CVS
> > (cvs) and Subversion (svn) can not, is that each person can commit
> > whenever they need to.  With cvs or svn, there is one repository that
> > everyone shares. They have to access to that one repository in order to
> > run (almost) any cvs or svn commands (and definitely to commit anything)
> > at all times.  If that one copy of the repository becomes corrupted in
> > some way, that's it; you'd better have good, ultra-recent backups.
>
> You forget that both CVS and SVN have a long history of managing
> thousands of repositories that have never seen corruption.

I forget nothing.  I've used CVS for years and still have code in there.  
Those familiar with the history of Subversion (in particular, the reasons why 
it was developed) would not try to argue that CVS is a "good" tool.

> You also 
> forget that having one single repository to make your changes against
> and pull updates from is a good thing for projects that aren't open
> source, GPL, and work like the Cathedral, not the Bazaar.

What does that have to do with the price of tea in China?  It doesn't matter 
what kind of software you are developing, bits are bits.  This is nonsense.

Don't confuse the names of the Bazaar & Bazaar-ng projects/programs with the 
title of RMS's book.

But, just to be clear, I've used all 4 for both open and closed projects.

> To top this 
> all off, everyone knows that the main repository isn't something that
> goes without having an automated backup system in place no matter what
> system you're using;

You are (mostly) right.

Given that there are millions of programmers on the planet, and given the 
cross-section of those I've met, I would have to say that the vast majority 
do not "[know] that the main repository isn't something that goes without 
having an automated backup system in place no matter what system you're 
using".

Personally, I prefer not to gloss over important things that should be obvious 
when I've seen many people completely ignore it before.  Yes, I was stating 
the obvious, but it was also just one of two main components to a single 
major point.

> Bazaar is just as prone to corruption. 

No, it isn't.  Because each workstation has a complete copy of the repository, 
loss of the "only" central copy is trivial to fix, even in the event that the 
backups are not recoverable.  Developers can also continue to work and commit 
while such repairs are made.  Even if the backups restore perfectly without 
any extra effort, they probably don't contain up to the absolute last changes 
before the crash.

This isn't a central feature of either baz or bzr, but a happy side effect of 
the way that they operate.

Also, I wasn't suggesting that baz or bzr would obviate the need for good 
backups, only that it was more critical to pay closer attention to them for 
CVS and SVN as the loss of the only copy of the repo can more easily mean the 
loss of all the history at best or of key files at worst.

> > With baz or bzr, each person has their own repository (called an archive
> > in the parlance of their documentation).  You keep a "mirror" copy on a
> > central server, but I can work completely disconnected from the network
> > (like on a plane or at home w/o VPN).  You simple mirror your repos to
> > the central server when you can/are ready to.  Then, others can merge
> > from your branch. Also, there is so server-side code to run; you just
> > need ssh access to be able to write and read files from some central
> > storage (i.e. file server).
>
> Nothing in Dan's list implied any use of a distributed VCS.

I'm missing where I suggested that he did.

> In his 
> situation, doing so could lead to developers forgetting to pull code
> up to the appropriate repositories, or not bothering to do so because
> of some confusion with the review system.

I can't agree.  Most programmers are not so stupid.  The choice of any of the 
4 SCM systems that we've been talking about in this thread would not affect 
this.  If a developer forgets to commit or merge into the proper branch, CVS 
and SVN aren't going to magically take care of it.

> It also means that there 
> could be conflicting code other developers are working on that the
> code reviewer (the person doing the repository merges) would run into
> instead of the original developer meaning there's a much higher chance
> of introducing bugs into production code.

Currently, I'm working with 7 people on a system with over 2000 files in baz 
with a central trunk repo.  When I need to make changes, I create my own 
private branch from trunk.  When I mirror my changes to the central server, I 
then provide the branch ID in a tracking system ticket and one of two people 
who can commit it to truck take responsibility.  Part of the process requires 
me to make sure that I have merged the latest from trunk before submitting my 
branch for inclusion.  Each developer is still responsible for conflict 
resolution on their changes.

Did you catch that?  I do not have commit access to trunk.  Code must be 
reviewed before it gets merged into trunk.  If my branch is not acceptable, 
they kick it back to me and I have to fix the problem (whether it's because 
of merge conflicts or dumb coding on my part).  This system is *far* less 
likely to introduce bugs into production code than CVS or SVN when developers 
all use one shared central repo as their commit point.

None of this is complex or voodoo at all.  These things are described in the 
baz & bzr documentation that explains the basics of how it works.

BTW, we're moving it to bzr sometime soon (who knows when it will actually 
happen).

> You're trying to substitute the traditional method of development with
> some unstable form of code management using distributed repositories.

Absolutely not.

> What's wrong with the time-tested methods of everyone committing their
> changes, reviewers looking through logs, putting the project in
> feature-freeze, then code-freeze, doing testing, fixing bugs, then
> tagging/branching stable?

Nothing.  All 4 (cvs, svn, baz & bzr) can be used as a component of this 
process.

> It's generally a known fact that grabbing 
> code from CVS/SVN head is testing code, not production.

Yup.  The same holds true of *any* CM, SCM, VCS system, though the term "HEAD" 
might not be used by all of them.

> > I've used all 4.  I've used cvs going back 10 years, svn going back about
> > 2, baz for 2 and bzr for 1 (it's very new).  I will never put new code
> > into cvs or svn again, even if it's a just-me-by-myself project.  Yes,
> > people are very familiar with cvs & svn and there are several front-ends
> > already available. The paradigm of baz & bzr requires a shift in
> > thinking.  People don't like to leave their comfort zones, especially
> > with tools like cvs that required lots of effort to learn how to work
> > around all the problems.  cvs & svn are just too limiting.
>
> They aren't limiting at all, especially considering they both can do
> everything Dan is asking for and more.

I never said they didn't.  In fact, quite the opposite.  I did say that I 
*felt* that in general baz & bzr are the better way to go in the future and 
that cvs & svn are going to go away.  Not anytime soon (as in I won't be 
surprised when it's been a decade), but I think it will happen over time.

> In my view, bzr is more 
> limiting due to lack of good GUI front-ends.

I don't agree that it's more limiting.  I have, and sometimes still do, use 
GUI front ends to cvs & svn.

But I definitely agree with you that there are no good front ends (that I've 
seen) for either baz or bzr (though, there is a nice GTK+ front-end to baz & 
bzr that's rapidly approaching "production-quality" release).

> I know of one developer 
> that really doesn't like it:
> http://www.kdedevelopers.org/node/2024

An interresting read.  He makes some good points, but also got a couple of 
things wrong.  For example, baz & bzr both can "redo" an "undo". The long bit 
followed by, "If you lost me here, that's OK..." describes a situation that 
I've never seen, but I already know how to fix it.

Even though baz has 47 commands to svn's 30, I only use maybe 5 or 6 of the 
baz commands in a typical day.  I usually have to use about the same for 
either cvs or svn.

He is right about Bazaar having some more complexity than it should.  That's 
why the developers stopped working on it and are producing Bazaar-ng (bzr) 
instead. It already deals with almost all of those kinds of issues that were 
in Bazaar, plus it's really easy to learn bzr when coming from either cvs or 
svn.

I like that he compared it to darcs, because there are some good things in 
there.  I've not personally used it though, so I didn't list or comment on it 
earlier and will refrain from doing so.

> > > > * I'd like the ability to review code before it gets committed, to
> > > > prevent obvious stupidities from getting into production code
> > >
> > > Good plan. I am not aware of any automated services to facilitate
> > > this, but I find peer review to generally be sufficient for
> > > preventing obvious mistakes.
> >
> > I've already commented on this elsewhere in this thread, but, since I
> > spoke about cvs, svn, baz & bzr here ...
> >
> > This is harder to do with cvs or svn, but not impossible.  The real
> > problem is when you need to reject code.  The "easiest" solution if you
> > are going to use these is to manually copy code from separate,
> > per-developer central repositories, but that means that all your history
> > is lost, in addition to several other problems that make it much harder
> > for a team to work coherently.
> >
> > baz & bzr make this really, really easy.  You simply create a "trunk" (or
> > name it whatever you want to) archive that your merge code from
> > per-developer archives (actually the mirrors on the central server).
>
> Again, there's nothing stopping this from being done on the testing
> branch/trunk of code like so many projects have done in the past
> before branching/tagging stable. You don't lose any history.

How do you copy all the history from one cvs repo to another while committing 
a set of files into the later?  I'm not aware of any way to do this without 
massive manual hacking of the RCS files, but if there's a way that the cvs 
command can do it, I'd love to hear about it.

Yes, you can branch and merge like crazy in a single cvs repo.  My point was 
that the history usually doesn't survive when you try to do this with 
per-developer cvs repos and thus that approach might have another drawback 
that he might not want.  Or, perhaps Dan doesn't care about that much 
history.

> Just 
> because bzr is the newest, hottest VCS to hit the scene doesn't mean
> everyone should switch to using distributed repositories,

Did I say that?  I apologize if anyone felt that I was 
suggesting "bandwagonism" here.

> nor does it 
> mean that CVS and SVN aren't proven VC systems with thousands of
> successful projects.

Nor do CVS and Subversion mean that there weren't hundreds or even thousands 
of successful projects before either of them existed.

Man has always fought against the new and unknown.  Just because you have a 
rock that works well for driving nails doesn't mean that a hammer can't be 
useful.  It takes time for any new tool to break into mainstram use.

As it should.  We want to test the tools out before we use them.

> I also highly doubt that bzr has much of a 
> history with proprietary code repositories to have been proven a
> better system than CVS or SVN in those situations either.

As I said earlier, we're using baz here for a project with 7 people and over 
2000 files, and that we plan to move it to bzr soon.  There is a lot of churn 
in those files.  Typically, we're all working on files that are related to 
each other and don't have any problems with merging.

> > > > * I want to be able to keep some files "hidden" from some developers
> > > > - specifically the files that contain our DB password info, credit
> > > > card gateway info, etc.  They would need to stay in the live code
> > > > and/or the repository, but not get checked out along with everything
> > > > else when someone grabs the code.
> > >
> > > I am fairly certain that you can restrict access to files in the
> > > repository.
> >
> > Not easily.  The main problem with this is that developers (at least
> > those that are good enough to have on your staff) will know how to find
> > the code, if it's in the same tree.
>
> There's nothing hard about this with SVN, but with any VCS, it would
> be best to use the same method most projects do with configuration
> files: make a config.ini.dist file where basic changes are made, and
> put an ignore on config.ini. Then you only need to worry about whoever
> is doing administration of updating config.ini on the production
> server, and your passwords are _never_ committed to your VCS.

Yes, that works just fine.  I have also configured a single file in several 
projects that had the production info (we didn't care if the devs had those, 
as they needed to anyway) and the developer system info configured in a very 
lean if-else tree (put the production stuff first).  There are other ways to 
do this too.  Pick your favorite method.

But Dan asked about keeping those files for production use that have sensitive 
access control/authentication data isolated from individual developers.

> There's no reason Dan needs to setup multiple servers with separate
> repositories,

I didn't day he did.  I said, "Here are a couple of ways that you could go 
about it."

> or any of that ridiculous advice you've just given. 
>
> > > > * Expanding upon that, it'd be nice if I could just give someone
> > > > access to certain parts of the code and not others.
> > >
> > > This is also easy with a versioning control system. The code can
> > > exist in various branches and other chunks that you can specify who
> > > has access to.
> >
> > Again, this could be done pretty easily with baz & bzr, but it's not so
> > easy with either cvs or svn.  Basically, I would do the same thing as for
> > other "restricted" code (that appears just before this part in this
> > message) and then use FACLs (see the setfacl(1) and getfacl(1) commands)
> > to control access to the archive directories.
> >
> > Of course, his would work best if your apps are modular in design.
>
> SVN has ACLs

CVS does not.  And SVN's ACLs can't be enforced when developers have access to 
the repository files (as in via ssh, etc.).  Again, I didn't say it couldn't 
be done.  I said, "...it's not so easy with either cvs or svn."  You have to 
deal with filesystem permissions or FACLs.  This is lots easier to do on the 
whole repo level than on an individual file-by-file, directory-by-directory 
basis.  That's why it would be a bit easier with baz or bzr or git or [insert 
other distributed systems here].

> that can be set on specific directories with a repository 
> for read/write access that are very easy to setup.

Do they work on anything finer grained than directories?  If not, that might 
not be a good solution for him, as he might want to control files in a 
directory with other files that won't be restricted.

> Don't say "it's not 
> so easy" unless you really know it isn't.

I didn't.  You'll note that I specifically didn't mention or comment on parts 
of which I didn't have enough knowledge to do so.

> That tactic of yours has led 
> to a lot of bad advice this round.

There are no "tactics" here.  I'm just pointing out some (and there's a *ton* 
I'm not doing) of the pros and cons.
-- 
Lamont Peterson <peregrine at OpenBrainstem.net>
Founder [ http://blog.OpenBrainstem.net/peregrine/ ]
GPG Key fingerprint: 0E35 93C5 4249 49F0 EC7B  4DDD BE46 4732 6460 CCB5
  ___                   ____            _           _
 / _ \ _ __   ___ _ __ | __ ) _ __ __ _(_)_ __  ___| |_ ___ _ __ ___
| | | | '_ \ / _ \ '_ \|  _ \| '__/ _` | | '_ \/ __| __/ _ \ '_ ` _ \
| |_| | |_) |  __/ | | | |_) | | | (_| | | | | \__ \ ||  __/ | | | | |
 \___/| .__/ \___|_| |_|____/|_|  \__,_|_|_| |_|___/\__\___|_| |_| |_|
      |_|               Intelligent Open Source Software Engineering
                              [ http://www.OpenBrainstem.net/ ]