Login | Register
My pages Projects Community openCollabNet

Discussions > dev [DISABLED] > Re: svnsync and large property changes

subversion
Discussion topic

There will be a regularly scheduled maintenance window every Friday at 17:00 Pacific time, beginning August 13. During this time, the system may be briefly unavailable.

Hide all messages in topic

All messages in topic

Re: svnsync and large property changes

Author Adriaan de Groot <groot at kde dot org>
Full name Adriaan de Groot <groot at kde dot org>
Date 2009-05-08 02:17:59 PDT
Message
Attachments
attachment-2437436

Re: svnsync and large property changes

Author cmpilato
Full name C. Michael Pilato
Date 2009-05-07 10:11:40 PDT
Message C. Michael Pilato wrote:
> C. Michael Pilato wrote:
>> Adriaan de Groot wrote:
>>> r.951943 changes a (long) property as part of a svk merge; it changes no files
>>> and seems to touch only one directory:
>>> r951943 | winterz | 2009-04-10 18:28:49 +0200 (Fri, 10 Apr 2009) | 8 lines
>>> Changed paths:
>>> M /branches/kdepim/ent​erprise4/kdepim
>>> Blocked revisions 951934 via svnmerge
>> [...]
>>
>>> The regular svn client has no issues updating from r.951942 to 951943; only
>>> svnsync hits a problem.
>>>
>>> Does this seem familiar to anyone? None of the svnsync bugs that have been
>>> files seem to apply, and searching for the specific error message didn't turn
>>> up anything (now it shows my blog entry describing how I worked around the
>>> problem). I can provide the revs/ and revprops/ files for some of the
>>> revisions around the problem one, if that would help anyone. It'll be a while
>>> before I've re-created the problem situation, though -- running svnsync for
>>> the whole repo and for only the problem branch, to see if that makes any
>>> difference.
>> Adriaan, this is the second time in a week that I've heard about this kind
>> of problem. So, thanks to you, I can't call the other report a fluke. :-)
>>
>> In the first report, the reason for the XML bogosity was due to the base64'd
>> property contents being truncated. The reporter seemed to think there was a
>> buffer overflow in an underlying APR routine. I found it very odd that
>> updates and checkouts with large property values seemed to work fine, but
>> not svnsync. I was never able to reproduce the problem myself, either.
>>
>> I did notice, though, a subtle difference in the way these operations stream
>> the property values across the wire. I'm attaching a patch in this mail
>> which makes the code that svnsync uses a bit more like what is used for
>> regular updates. But it's a server-side change -- you'd have to convince
>> the KDE folks to give this a shot. If you can do that, though, I would
>> *love* to hear if it helps things. If it does, we can not only patch
>> Subversion to avoid this problem, but we'll also know better now to talk to
>> the APR folks about what might be a real problem in their codebase.
>
> For more on this issue (including a commit to APR which fixes it), see
> http://subversion.ti​gris.org/ds/viewMess​age.do?dsForumId=462​&viewType=browse​All&dsMessageId=​1897250
>
> TODO: File a Subversion issue about the problem, noting the fix in APR, but
> devising a workaround for folks that can't easily get a new APR release. I
> suspect this will have a little something to do with not using
> apr_brigade_vprintf() at all for the large property values, but using
> apr_brigade_write() directly instead.

Okay, I never got around to filing the issue, but the APR fix is done, and
now trunk uses the workaround of _write() instead of _vprintf() for the
large base64 property block now (with a backport recommended for 1.6.x).

--
C. Michael Pilato <cmpilato@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Attachments
signature.asc

Re: svnsync and large property changes

Author cmpilato
Full name C. Michael Pilato
Date 2009-04-24 14:09:07 PDT
Message C. Michael Pilato wrote:
> Adriaan de Groot wrote:
>> r.951943 changes a (long) property as part of a svk merge; it changes no files
>> and seems to touch only one directory:
>> r951943 | winterz | 2009-04-10 18:28:49 +0200 (Fri, 10 Apr 2009) | 8 lines
>> Changed paths:
>> M /branches/kdepim/ent​erprise4/kdepim
>> Blocked revisions 951934 via svnmerge
>
> [...]
>
>> The regular svn client has no issues updating from r.951942 to 951943; only
>> svnsync hits a problem.
>>
>> Does this seem familiar to anyone? None of the svnsync bugs that have been
>> files seem to apply, and searching for the specific error message didn't turn
>> up anything (now it shows my blog entry describing how I worked around the
>> problem). I can provide the revs/ and revprops/ files for some of the
>> revisions around the problem one, if that would help anyone. It'll be a while
>> before I've re-created the problem situation, though -- running svnsync for
>> the whole repo and for only the problem branch, to see if that makes any
>> difference.
>
> Adriaan, this is the second time in a week that I've heard about this kind
> of problem. So, thanks to you, I can't call the other report a fluke. :-)
>
> In the first report, the reason for the XML bogosity was due to the base64'd
> property contents being truncated. The reporter seemed to think there was a
> buffer overflow in an underlying APR routine. I found it very odd that
> updates and checkouts with large property values seemed to work fine, but
> not svnsync. I was never able to reproduce the problem myself, either.
>
> I did notice, though, a subtle difference in the way these operations stream
> the property values across the wire. I'm attaching a patch in this mail
> which makes the code that svnsync uses a bit more like what is used for
> regular updates. But it's a server-side change -- you'd have to convince
> the KDE folks to give this a shot. If you can do that, though, I would
> *love* to hear if it helps things. If it does, we can not only patch
> Subversion to avoid this problem, but we'll also know better now to talk to
> the APR folks about what might be a real problem in their codebase.

For more on this issue (including a commit to APR which fixes it), see
http://subversion.ti​gris.org/ds/viewMess​age.do?dsForumId=462​&viewType=browse​All&dsMessageId=​1897250

TODO: File a Subversion issue about the problem, noting the fix in APR, but
devising a workaround for folks that can't easily get a new APR release. I
suspect this will have a little something to do with not using
apr_brigade_vprintf() at all for the large property values, but using
apr_brigade_write() directly instead.

--
C. Michael Pilato <cmpilato@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Attachments
signature.asc

grep -v 'traced call' (was: Re: svnsync and large property changes)

Author Daniel Shahaf <d dot s at daniel dot shahaf dot name>
Full name Daniel Shahaf <d dot s at daniel dot shahaf dot name>
Date 2009-04-16 17:46:38 PDT
Message Daniel Shahaf wrote on Thu, 16 Apr 2009 at 14:38 +0300:
> % svn log --snip-options 2>&1 | grep -v "traced call"
> ..\..\..\subversi​on\svn\log-cmd.c:5​93: (apr_err=170001)
> ..\..\..\subversi​on\libsvn_client\l​og.c:515: (apr_err=170001)
> ..\..\..\subversi​on\libsvn_client\r​a.c:449: (apr_err=170001)
> ..\..\..\subversi​on\libsvn_ra\ra_lo​ader.c:481: (apr_err=170001)
> ..\..\..\subversi​on\libsvn_ra_neon\​util.c:608: (apr_err=170001)
> svn: OPTIONS of 'https://svn.kde.org/​home/kde': authorization failed
>
> Daniel
> (looks like I'll add an alias to the "2>&1 | grep -v traced\ call" bit...)

Fixed in r37315.

Re: svnsync and large property changes

Author cmpilato
Full name C. Michael Pilato
Date 2009-04-16 07:50:40 PDT
Message Adriaan de Groot wrote:
> r.951943 changes a (long) property as part of a svk merge; it changes no files
> and seems to touch only one directory:
> r951943 | winterz | 2009-04-10 18:28:49 +0200 (Fri, 10 Apr 2009) | 8 lines
> Changed paths:
> M /branches/kdepim/ent​erprise4/kdepim
> Blocked revisions 951934 via svnmerge

[...]

> The regular svn client has no issues updating from r.951942 to 951943; only
> svnsync hits a problem.
>
> Does this seem familiar to anyone? None of the svnsync bugs that have been
> files seem to apply, and searching for the specific error message didn't turn
> up anything (now it shows my blog entry describing how I worked around the
> problem). I can provide the revs/ and revprops/ files for some of the
> revisions around the problem one, if that would help anyone. It'll be a while
> before I've re-created the problem situation, though -- running svnsync for
> the whole repo and for only the problem branch, to see if that makes any
> difference.

Adriaan, this is the second time in a week that I've heard about this kind
of problem. So, thanks to you, I can't call the other report a fluke. :-)

In the first report, the reason for the XML bogosity was due to the base64'd
property contents being truncated. The reporter seemed to think there was a
buffer overflow in an underlying APR routine. I found it very odd that
updates and checkouts with large property values seemed to work fine, but
not svnsync. I was never able to reproduce the problem myself, either.

I did notice, though, a subtle difference in the way these operations stream
the property values across the wire. I'm attaching a patch in this mail
which makes the code that svnsync uses a bit more like what is used for
regular updates. But it's a server-side change -- you'd have to convince
the KDE folks to give this a shot. If you can do that, though, I would
*love* to hear if it helps things. If it does, we can not only patch
Subversion to avoid this problem, but we'll also know better now to talk to
the APR folks about what might be a real problem in their codebase.

--
C. Michael Pilato <cmpilato@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Attachments
replay-propprint-patch.txt
signature.asc

Re: svnsync and large property changes

Author Daniel Shahaf <d dot s at daniel dot shahaf dot name>
Full name Daniel Shahaf <d dot s at daniel dot shahaf dot name>
Date 2009-04-16 04:39:13 PDT
Message What command can I use to access that repository with the svn client?

% svn log --non-interactive --trust-server-cert --username guest --password guest https://svn.kde.org/home/kde -r951943 2>&1 | grep -v "traced call"
..\..\..\subversi​on\svn\log-cmd.c:5​93: (apr_err=170001)
..\..\..\subversi​on\libsvn_client\l​og.c:515: (apr_err=170001)
..\..\..\subversi​on\libsvn_client\r​a.c:449: (apr_err=170001)
..\..\..\subversi​on\libsvn_ra\ra_lo​ader.c:481: (apr_err=170001)
..\..\..\subversi​on\libsvn_ra_neon\​util.c:608: (apr_err=170001)
svn: OPTIONS of 'https://svn.kde.org/​home/kde': authorization failed: Could not authenticate to server: rejected Basic challenge (https://svn.kde.org)

Daniel
(looks like I'll add an alias to the "2>&1 | grep -v traced\ call" bit...)


Adriaan de Groot wrote on Wed, 15 Apr 2009 at 23:32 +0200:
> I use svnsync -- 1.5.5 up until recently, now 1.6.0, both from FreeBSD ports
> which use gcc 3.4.6 -- on FreeBSD 6.2 to mirror the KDE SVN repository. Other
> KDE SVN mirrors use svnsync on Linux, I'm not sure what versions. I use https
> access to the main repository, I don't know what the other mirrors use. There
> is no direct anonymous or svn:// style access to the main repository. I don't
> know which version of svn the main server runs (is there a simple way to find
> out?).
>
> All of our svnsync mirroring fails on revision 951943. You can see the
> revision with websvn at
> http://websvn.kde.or​g/?pathrev=951943
> and get the log with
> http://websvn.kde.or​g/branches/?view=log​&pathrev=951943
> (well, assuming websvn likes that at all; i seems to take forever for me and
> then 500s).
>
> Unfortunately, getting the anonsvn service up and running again was of higher
> priority than keeping a good forensic copy. I should have thought of
> filesystem snapshots, I know. I worked around the problem by rsyncing a new
> copy of the repo up to the problem revision + 1, then adjusting the svnsync
> properties by hand, then letting svnsync go again. This seems to fix the
> problem.
>
> In order to re-create the problem (presumably, and for debugging purposes) I'm
> doing yet another svnsync from r.0, but you can understand that that takes a
> while with 900k+ revisions (as in nearly all day today for the first 30k, but
> that's on a loaded local disk).
>
> r.951943 changes a (long) property as part of a svk merge; it changes no files
> and seems to touch only one directory:
> r951943 | winterz | 2009-04-10 18:28:49 +0200 (Fri, 10 Apr 2009) | 8 lines
> Changed paths:
> M /branches/kdepim/ent​erprise4/kdepim
> Blocked revisions 951934 via svnmerge
>
> From my notes, the error message was:
> svnsync: Got unexpected element svn::close-directory
> This comes from libsvn_ra_neon/replay.c (I was looking at the svnsync program
> from subversion 1.5.5), in start_element(). There's a comment there in the
> source that says that entities are not nested. The KDE SVN server sends
> (again, notes):
> open-root
> target-revision
> open-directory
> open-directory
> open-directory
> change-dir-prop
> close-directory
> close-directory
> I tried futzing around with start_element(), for instance allowing
> parent_state == ELEM_change_dir_prop (which is what parent_state is after the
> change-dir-prop). That doesn't seem to help; after a bit of fiddling around I
> had svnsync ignoring the trailing close-directory elements, but it would then
> print REPORT of https://svn.kde.org/home/kde 200 OK -- and not sync. I didn't
> mess around with end_element() though, so I don't know what any of the
> functions called from there (e.g. change_dir_prop) were doing.
>
> The regular svn client has no issues updating from r.951942 to 951943; only
> svnsync hits a problem.
>
> Does this seem familiar to anyone? None of the svnsync bugs that have been
> files seem to apply, and searching for the specific error message didn't turn
> up anything (now it shows my blog entry describing how I worked around the
> problem). I can provide the revs/ and revprops/ files for some of the
> revisions around the problem one, if that would help anyone. It'll be a while
> before I've re-created the problem situation, though -- running svnsync for
> the whole repo and for only the problem branch, to see if that makes any
> difference.
>
> [ade] (please CC, as I'm not subscribed)
>
>
>
>

svnsync and large property changes

Author Adriaan de Groot <groot at kde dot org>
Full name Adriaan de Groot <groot at kde dot org>
Date 2009-04-15 14:33:07 PDT
Message
Attachments
attachment-2056836
Messages per page: