Issue 898
| Description: |
Opened: Wed Sep 11 02:42:00 -0700 2002 |
Sort by: Oldest first | Newest first |
------- Additional comments from Branko Cibej Wed Sep 11 02:44:52 -0700 2002 -------
This absolutely must be fixed by Beta.
------- Additional comments from Greg Stein Wed Dec 4 12:49:41 -0700 2002 -------
"Absolutely must" ? This sounds like a feature which can be postponed
until after 1.0.
------- Additional comments from Philip Martin Sun Dec 8 15:14:54 -0700 2002 -------
One common operation is to rename, edit to take account of the the new
names, then commit. Would atomic rename apply in this case as well? I
don't really want to do it in two commits, because neither the edit or
the rename will compile without the other.
------- Additional comments from Karl Fogel Mon Dec 9 07:22:20 -0700 2002 -------
Philip,
Sure, you can get the atomicity you need in that scenario -- just
include everything in one commit. The fact that rename is currently
broken down into copy+delete doesn't mean you can't commit both the
copy and the delete in the same commit.
------- Additional comments from Philip Martin Mon Dec 9 08:33:19 -0700 2002 -------
Perhaps I didn't make my question clear :)
As I understand the proposal an atomic rename would operate a bit like
a cheap copy, whichever directory referred to the file before the
rename would have a new version that didn't refer to the file. There
would also be a new reference to the original file either in the new
directory with a different filename or in a different directory.
My question is then how such a rename would work if the user both
renamed and modfied in a single commit. In this circumstance using a
new reference to the original file won't work. So I am wondering if
atomic rename makes any difference to this case.
If I compare to ClearCase, for instance, the atomic rename involves
committing directories, and the file modification involves a
committing the file. Essentially "rename and modify" is always two
separate commits, and the "rename" commit never modifies the file
itself, it just moves it. I was wondering how this would be achieved
with Subversion's whole repository commit.
------- Additional comments from Branko Cibej Tue Dec 17 17:29:23 -0700 2002 -------
I think the client would have to send directory modifications (i.e.,
the rename) before file modifications -- and if it doesn't do that
now, we're in trouble anyway.
So what happens in your case:
-- First, the rename is sent, which produces one or two directory
changes in the commit txn, essentially just a relinking of the
existing file node.
-- Later, the modification for that file coms in (in the new location,
mind!), which produces a new file node and again modifies directory
that's the target of the rename.
I don't *think* that would be a problem, the server should already
notice that the directory was modified before. In fact, it should work
right out of the box, because a commit can handle several distinct
directory changes already.
------- Additional comments from Kevin Pilch-Bisson Tue Jan 21 09:17:07 -0700 2003 -------
Adding a small description of a use case where the distinction between
rename and copy+delete is important.
svn cp http://server/trunk http://server/branch
svn co http://server/branch
cd branch
svn cp template.c foo.c
svn ci
edit foo.c
svn ci
cd ../trunk
edit template.c
svn ci
Now I want to merge the changes back and forth. I probably don't
want to merge both sets of changes together, because this was a copy.
However, if I do the exact same thing, but with a mv instead of cp, I
DO want to merge the changes in.
svn cp http://server/trunk http://server/branch
svn co http://server/branch
cd branch
svn mv template.c foo.c
svn ci
edit foo.c
svn ci
cd ../trunk
edit template.c
svn ci
Now I probably do want the changes to be applied. As Kazinator
pointed out on IRC, copy is to create a new, but related item in the fs,
whereas move/rename should not change the object, just it's location
as far as operations like merge are concerned.
------- Additional comments from Ben Collins-Sussman Wed May 26 12:56:53 -0700 2004 -------
*** Issue 1864 has been marked as a duplicate of this issue. ***
------- Additional comments from Karl Fogel Sat Mar 5 22:51:11 -0700 2005 -------
Goodness, I thought Mike and I had copied the important data below into this
issue already, but apparently not...
Here are some notes from discussions we had on 18 August 2004, regarding subtle
problems in a "naive" implementation of true renames:
---------------------------------------------------------------------------
We posited that renames would work the "natural way", that is, entries
would get shifted around, but the target's entity ID would not change.
Then Mike demonstrated the collision problem, whereby we could end up
with two different names for the exact same entity ID:
1. begin with /trunk/foo/bar.c
2. copy /trunk /branches
3. rename /branches/foo /trunk/bloo
4. modify /branches/foo/bar.c and /trunk/bloo/bar.c in the same txn.
Here's what things like like right before step 4:
r1 r2 r3
======== ========= =========
0.0.1 0.0.2 0.0.3
.------. .------. .------.
|trunk----. .--<----trunk | .----<-----trunk |
| | | | |branch----. | |branch------>------.
| | | | | | | | | | |
|______| | | |______| | | |______| |
v v v v v
1.0.1 | | 1.1.2 | | 1.0.3 1.1.3 |
.------. | | .------. | | .------. .------. |
| |<-+--<----' | |<--' `->| | | | |
| foo-----. .--<----foo | .--<-----foo | | |<--'
| | | | | | | .-<---bloo | | |
|______| | | |______| | | |______| |______|
v v v |
2.0.1 | | | v
.------. | | | |
| |<-+----<---+-------<--------'<-'
|bar.c---.
| | |
|______| |
v
3.0.1 |
(file)<--'
You can see how modifying bar.c in both places now will result in an
entity ID collision.
(Open question: can we also construct a cycle via normal Subversion
operations, with these new renames? That would be even worse.)
Some possible solutions to the collision situation:
* Schema-preserving Brute Force Fix:
Disallow the rename if the immediate parents of the merge source
and dest have different CopyIDs.
* Schema-semi-preserving fix:
Add an optional component(s) to entity IDs. In new filesystem
code, all new IDs get extended, but the code is still able to
handle the old 3-component kinds too. This needs to be fleshed
out to describe exactly what those components would be, of
course; the point here is merely that schema changes are not
always an all-or-nothing proposition.
* Schema-non-preserving fix:
Add another component to entity IDs. After several iterations of
discussion, Mike reduced it to just a UniqueID, constantly
incremented -- the UniqueID would be different for *every* Node
Revision in the database, in fact, it could maybe serve as a
unique identifier itself?
Some other random thoughts:
* 'svn log renamed-tgt' requires that renames be tracked in the copies
table. While we're at it, copies themselves should be tracked
directly there, Mike says.
* Since we have to implement copy-on-write to handle subnodes of
renamed trees anyway, why not go all the way and implement
copy-on-write completely? Something like: 'svn cp URL/foo URL/bar'
would create new entity IDs (because that is the correct semantic of
copy), but the two would share the same underlying string in the db.
* User Interface:
An important benefit of true rename support would be that users can
tell (from the client side) when two differently-named entites are
the same object. Whether we do this by simply exposing entity IDs,
or a uniquely-identifying portion of entity IDs, or something else,
doesn't matter so much, as long as the question is unambiguously
resolveable.
------- Additional comments from C. Michael Pilato Fri Apr 22 20:40:44 -0700 2005 -------
What do you mean by saying you think the ID collision could be caused by "normal
Subversion operations"? Had we svn_fs_rename(), that would be a normal
Subversion operation.
Or are you saying this bug exists today, sans-rename?
------- Additional comments from Philip Martin Fri Apr 22 20:57:56 -0700 2005 -------
I mean that if we were using the svn_fs_rename in my patch then a sequence of
svn commands could be sufficient to trigger the problem.
------- Additional comments from Karl Fogel Fri May 20 10:06:30 -0700 2005 -------
Changing summary because "atomic" is a bit misleading (or at least ambiguous) in
this context
------- Additional comments from Peter N. Lundblad Fri Sep 23 05:33:46 -0700 2005 -------
Moving to 1.4, since, while cmpilato has started this work on the fs-atomic-
renames branch, it won't be merged before 1.3.
------- Additional comments from Karl Fogel Thu Mar 30 11:41:59 -0700 2006 -------
Garrett Rooney is actively working on this, so assigning the issue to him (I'm
assuming he doesn't mind).
------- Additional comments from Karl Fogel Thu Mar 30 11:42:25 -0700 2006 -------
Mark as started.
------- Additional comments from Peter N. Lundblad Tue May 2 02:14:02 -0700 2006 -------
The 1.4 branching point is approaching and this is not happening before that,
so move into 1.5-consider.
------- Additional comments from Ringo De Smet Mon May 22 05:01:26 -0700 2006 -------
I think the OID story (see my comments in issue 1525) must be implemented first
before you can properly implement this one.
------- Additional comments from Garrett Rooney Wed Dec 13 14:58:24 -0700 2006 -------
Let's not kid ourselves, there's no way I'm ever going to finish this stuff.
Assigning back to issues@subversion on the off chance someone else wants to pick
it up.
------- Additional comments from Malcolm Rowe Sun Dec 31 02:25:38 -0700 2006 -------
According to sussman, any fix for this should also incorporate a fix for issue 2685 - when merging a
rename from (say) trunk to branch, we should merge by renaming an existing branch file, not overwriting
the branch file with one on trunk.
------- Additional comments from Daniel Rall Fri Mar 30 16:01:48 -0700 2007 -------
Issue 2685 is related, but may not be dependent upon this issue.
------- Additional comments from Erik Huelsmann Wed Apr 4 09:46:49 -0700 2007 -------
Signing away, I said.
------- Additional comments from Erik Huelsmann Wed Apr 4 09:47:56 -0700 2007 -------
Blundering with IZ again. Sorry for the last comment...
------- Additional comments from Erik Huelsmann Wed Apr 4 09:50:16 -0700 2007 -------
With resources fully concentrated on Merge Tracking, I don't think having this
in 1.5 is viable. Moving to 1.6-consider.
------- Additional comments from Hyrum K. Wright Tue Mar 10 07:45:10 -0700 2009 -------
Post-1.6 issue sweep. Since 1.7 is already shaping up to be a large release,
move to 1.8-consider.
------- Additional comments from Mark Haase Wed Apr 29 10:13:11 -0700 2009 -------
To my mind, the rename is issue is far more important that automatic merge
tracking.
Merge tracking is solving a problem I don't really have. I follow the best
practices in the 1.4 documentation and I include the merged revisions, source,
and destination in the commit message. When I want to merge again, I grep for
the previous merge and look at the message again.
Seems like this thing is going to be pushed out indefinitely.
------- Additional comments from Steve Brown Thu Sep 3 08:07:52 -0700 2009 -------
> Instead, a rename should only change the (old and new) parent directory, not
the file itself.
Clearcase has this concept, and it's not particularly easy for developers to
understand or use.
An easier to understand concept/implementation is to have a (versioned)
directory-path as a property of the file. When you move the file, you create a
new version (commit) of that file where the directory-path property is the only
change. The file should keep it's original unique id, and all internal
operations should use the unique-id instead of the pathname+filename.
------- Additional comments from Branko Cibej Mon Sep 7 08:26:09 -0700 2009 -------
> An easier to understand concept/implementation is to have a (versioned)
directory-path as a property of the file. When you move the file, you create a
new version (commit) of that file where the directory-path property is the only
change.
That would become *really* hairy IMHO because then we suddenly loose the concept
of a first-class versioned tree (and consequently versioned directories); the
whole object hierarchy in the repository becomes a guessing game based on
properties of the object, instead of the other way around, as it is now.
The idea that file moves/renames only affect directories, and that by
implication file names are properties of directories, not files, is how the way
the Unix virtual file system hierarchy is structured. It works very well on the
implementation level, although it may be slightly confusing since users would
typically expect the name to be a property of the fils -- that's how DOS/Windows
(and their conceptual predecessors) originally treated it, with implementation
problems and limitations arising from the fact that names were file properties
but paths (i.e., containing directories) were not. Notice how NTFS, for example,
sneakily adopted the Unix concept on the implementation level, which is what
makes hard-links possible on NTFS but not on FAT.
------- Additional comments from Julian Foad Fri May 28 04:30:54 -0700 2010 -------
See also:
Issue #3630 "Rename tracking"
Issue #3633 "Track renames as renames inside Subversion repository"
------- Additional comments from Stewart Gordon Tue Apr 19 06:46:58 -0700 2011 -------
Re Steve
Well said - you've taken the words out of my mouth. Think of it like page
moving on MediaWiki (Wikipedia et al). Current SVN renaming is a cut-and-paste
move, a common sin among WP users who don't know better. The file under its new
name is a new file, with a new revision history, split from the file under its
old name. When pages are moved correctly under MW, the revision history remains
intact. A file rename under SVN ought to do the same.
Re Branko
This would only be the case if we move _all_ path information from the directory
node to the file node. Here's an approach that, as far as I can see, would
achieve the best of both worlds.
Each file node contains:
- unique node ID (immutable)
- filename (versioned)
- ID of the directory node containing it (versioned)
- other properties (versioned)
- file contents (versioned)
All directory nodes are file nodes - but the "file contents" become a list of
the IDs of files in the directory.
File renaming is then straightforward - just change the filename in the file node.
Moving a file would entail these steps:
- change the directory node ID in the file node
- change the filename in the file node, if it is being renamed at the same time
- remove the ID of this node from the source directory node's list
- add the ID of this node to the destination directory node's list
The drawback is that it would be a major change to SVN repository structure, and
so a lot of SVN code would need to be rewritten. The question is: can the
changes be kept on the server side so that existing clients will still work on
repositories using the new system? Or will SVN clients need to change as well?
------- Additional comments from Stewart Gordon Wed Apr 27 12:03:55 -0700 2011 -------
A challenge is how to deal with file moves when the user updates, when either
(a) it has moved into or out of the subdirectory the user has updated
(b) the new name clashes with something
An example of (a): dir1\file1 has been renamed to dir2\file1
User updates whole working copy - just move - straightforward
User updates only dir1 or even file1 - does file1 just disappear, or is the file
moved across?
User updates only dir2 - is he left with two copies of file1, or is the file
moved across?
User has checked out only dir1 - does file1 just disappear?
It gets even more complicated when you consider that the local file may have
uncommitted changes.
Possible cases of (b):
- new name conflicts with an unversioned file that the user has put there
- file1 has been deleted or renamed, and file2 has been renamed file1, but user
updated only file2 and still has the old file1
I suppose the simplest is to continue to process renaming as delete and add when
updating. Of course, we would have to re-retrieve the full history of the
renamed file, and think about how access to a deleted file's history would work.
------- Additional comments from Stewart Gordon Wed Apr 27 12:07:18 -0700 2011 -------
A challenge is how to deal with file moves when the user updates, when either
(a) it has moved into or out of the subdirectory the user has updated
(b) the new name clashes with something
An example of (a): dir1\file1 has been renamed to dir2\file1
User updates whole working copy - just move - straightforward
User updates only dir1 or even file1 - does file1 just disappear, or is the file
moved across?
User updates only dir2 - is he left with two copies of file1, or is the file
moved across?
User has checked out only dir1 - does file1 just disappear?
It gets even more complicated when you consider that the local file may have
uncommitted changes.
Possible cases of (b):
- new name conflicts with an unversioned file that the user has put there
- file1 has been deleted or renamed, and file2 has been renamed file1, but user
updated only file2 and still has the old file1
I suppose the simplest is to continue to process renaming as delete and add when
updating. Of course, we would have to re-retrieve the full history of the
renamed file, and think about how access to a deleted file's history would work.
------- Additional comments from Daniel Shahaf Wed Apr 27 12:12:58 -0700 2011 -------
I don't see the problem. If you moved d1/f1 to d2/f2 and you update d1 to rN, then either d1@N
contains an 'f1' child or it doesn't. If you had no local changes, then *why* it does or doesn't have an
'f1' child is irrelevant; we just create or delete the wc's 'f1' child as appropriate.
------- Additional comments from Stewart Gordon Wed Apr 27 12:31:57 -0700 2011 -------
That's basically what I meant by my final paragraph. I was pondering over
whether that or something else is the best strategy for dealing with it.
------- Additional comments from Tim Bain Wed Apr 27 15:38:16 -0700 2011 -------
I think Stewart Gordon's comments from 4/19 make sense. However, the challenge
will be that many (most?) users interact with SVN via a client on a machine
whose file system doesn't use the scheme he described. A file in Windows (or
Unix) doesn't have a means to store his immutable unique node ID in a manner
that is invisible to the client yet inseparable from the file content, leaving
two possibilities:
1. Store the data within the file itself, which is a non-starter since that
would probably invalidate the file format for any applications using the file.
So we're left with...
2. Store the data outside of the file itself (e.g. in the .svn directory
somewhere). This will require that users performing a rename/move operation do
so in a manner that properly migrates the unique node ID to the new location for
the file (which might simply be a different filename within the same .svn
directory).
Option 2 would probably be possible within the major stand-alone SVN clients
(including the IDE plugins), though I imagine it would be a breaking change for
older versions of those clients. But I don't see how it would work for the
developer using Windows Explorer or the mv command to move/rename, unless the
developer explicitly picks some shell extension option ("Move in SVN" instead of
"Move") or runs a SVN-specific mv command.
But if we do manage to make that work, it eliminates the concerns about things
like unversioned copies existing at the time of an update (your merge resolution
process involves deciding which unique node ID to use, just like you decide
whether you'd rather keep your version or the server's version of line 43 when
both have changes.
------- Additional comments from Stewart Gordon Wed Apr 27 16:03:31 -0700 2011 -------
My idea was on the basis that node IDs would be just an SVN thing, not part of
the filesystem. Using filesystem node IDs probably wouldn't work even if they
were a feature of all filesystems, because they cannot be in sync across the
repository and all working copies.
SVN already has this move/rename command. The problem is that it's currently
just shorthand for a copy (which in turn is just shorthand for copying the file
manually and then adding it) followed by a delete. But current SVN clients
would probably still do this, whereas newer versions would have the proper
rename. Actually, ISTM the node ID doesn't need to be stored in working copies,
if we take Daniel's approach.
The problem I can see is that existing SVN clients might not correctly retrieve
the revision history of a file that's been renamed.
------- Additional comments from Mauro Molinari Wed Nov 23 04:44:57 -0700 2011 -------
Please fix this bug for 1.8! Isn't now the problem with the WC-NG a bit more easy
to fix?
Please, don't blame on me, consider this as a "vote". I'm not an SVN internals
expert, but I think that the problem depicted in http://svnbook.red-bean.com/
en/1.7/svn.branchmerge.advanced.html#svn.branchmerge.advanced.moves is quite
severe and it would be great if Subversion could correctly handle that at last.
------- Additional comments from Stefan Sperling Wed Nov 23 05:23:51 -0700 2011 -------
Mauro, work is being done for 1.8 in this area, but not in the context of this
issue. This issue is about changing the way renames are represented in the
Subversion filesystem. But there are other approaches to fix usability problems
present in the current implementation of moves (or, rather, lack of
implementation). See issue 3630 and the issues linked from it, in particular
issue 3631 which addresses what wc-ng can provide in this area.
It is not clear yet what the exact set of improvements in move support shipped
in 1.8 will look like. Discussion is still on-going, and some experimental work
is being done on branches which may or may not be merged back to the trunk (and
hence 1.8) eventually. Feel free to contribute to discussion and implementation.
------- Additional comments from Davide Cavestro Mon Dec 5 03:26:40 -0700 2011 -------
Just lost files added to my branch before merging from trunk, where the parent
folder was simply renamed...
Yes, it's a well known weak spot, but it's REALLY SEVERE, so it would be good if
at least people could vote for its resolution.
------- Additional comments from Stefan Sperling Mon Dec 5 04:00:30 -0700 2011 -------
There is no point in having people vote on this, or add even more "me too"
comments to this and related issues. The developers are well aware that this is
a severe limitation for many users. Work is being done, aiming towards 1.8.
If you want to get this fixed asap, please help out by sending patches or
funding additional developer time for new and/or existing developers. Else, be
patient, let those who are putting in the effort work at their own pace, and
wait for results in some future release. Thanks.
------- Additional comments from Ben Reser Mon May 19 16:50:33 -0700 2014 -------
Moving to 1.10-consider. Considerable work has gone into this but it won't be
done in time for 1.9.0
