Login | Register
My pages Projects Community openCollabNet

Discussions > users [DISABLED] > RE: Using memcached with FSFS repositories

subversion
Discussion topic

There will be a brief maintenance window every Friday at 17:00 Pacific.
For further details, see CollabNet's maintenance and upgrade policy.

Back to topic list

RE: Using memcached with FSFS repositories

Author Johan Corveleyn <johan dot corveleyn at uz dot kuleuven dot ac dot be>
Full name Johan Corveleyn <johan dot corveleyn at uz dot kuleuven dot ac dot be>
Date 2009-05-22 07:24:18 PDT
Message > Anyone do any performance analysis of using memcached? I'm not
> seeing much (if any) benefit, but that could be due to a fairly
> high performance SAN storage solution for storage.
>
> Here are my numbers:
>
> no cache: 10 min 33 sec
> cold cache: 10 min 30 sec
> hot cache: 10 min 26 sec
> (The differences are statistically insignificant.)
>
> This is for a single repo with 600M of data in 30k files.
>
> Server cpu usage seemed consistent with and without
> the cache. I (unfortunately) wasn't able to monitor disk I/O.
>
> I was using a 2G memcached, of which it used around 600M for the
> cache. Oddly enough, monitoring memcached showed only a 70%
> cache hit when repeatedly downloading the same repository.
> (I would have expected a 100% hit ratio from the memcached side.)
>
> memcached was running on the same server as apache for the tests.
> (A quad core 64-bit AMD server running solaris 10 x86. 4G of RAM
> with storage on an enterprise SAN connected with fiber.)
>
> svn 1.6.2
> httpd 2.2.11
> apr 1.3.3
> apr-util 1.3.4
> neon 0.28.4
> memcached 1.2.8
>
> Test repository was created and populated with svn 1.6.2.
>
> I also noticed if no memcached servers were running, but a repository
> was configured to use it, a checkout would fail during the checkout
> of the first directory. It probably should continue to work as
> if no cache was configured instead of failing...
>
> Anyone else see some better results?
>

It's not really clear to me what you measured here, but deducing from your previous post, I'm guessing you timed a large checkout or something similar. What was your test setup (what kind of client, how did it access the repo (local/remote, svn(+ssh), http(s), file), ...)? And how did you measure it (e.g. running "time svn co ..." in a console, or via TortoiseSVN, or ...)?

If it's really a checkout, I think it's normal that you don't see an improvement (especially if it's with a windows client). In my experience, a checkout (or update) on Windows is mainly client-side cpu-bound (also heavy on IO, but mainly constrained by cpu). Could you verify this?

I don't know that much about SVN (client) internals, but I'm hoping this will be improved in 1.7 by the new WC-NG (see http://subversion.ti​gris.org/issues/show​_bug.cgi?id=3357). Another issue that might be related (and will probably go away with WC-NG) is http://subversion.ti​gris.org/issues/show​_bug.cgi?id=3369.

It would be interesting to know which svn actions are significantly improved by memcached (log, blame, ...). Any chance you could test this with your setup?

Regards,
Johan

« Previous message in topic | 8 of 11 | Next message in topic »

Messages

Show all messages in topic

Subject
Author
Full name
Date posted
+
Using memcached with FSFS repositories Chris Lieb <chris dot lieb+nospam at gmail dot com> Chris Lieb <chris dot lieb+nospam at gmail dot com> 2009-05-13 09:31:54 PDT
     Re: Using memcached with FSFS repositories markphip Mark Phippard 2009-05-13 09:37:58 PDT
         Re: Using memcached with FSFS repositories Chris Lieb <chris dot lieb+nospam at gmail dot com> Chris Lieb <chris dot lieb+nospam at gmail dot com> 2009-05-13 09:56:43 PDT
             Re: Using memcached with FSFS repositories hwright Hyrum K. Wright 2009-05-13 10:34:45 PDT
         Re: Using memcached with FSFS repositories kmradke at rockwellcollins dot com kmradke at rockwellcollins dot com 2009-05-18 10:45:37 PDT
             Re: Using memcached with FSFS repositories blair Blair Zajac 2009-05-18 12:45:32 PDT
                 Re: Using memcached with FSFS repositories kmradke at rockwellcollins dot com kmradke at rockwellcollins dot com 2009-05-18 12:51:01 PDT
             RE: Using memcached with FSFS repositories Johan Corveleyn <johan dot corveleyn at uz dot kuleuven dot ac dot be> Johan Corveleyn <johan dot corveleyn at uz dot kuleuven dot ac dot be> 2009-05-22 07:24:18 PDT
                 RE: Using memcached with FSFS repositories kmradke at rockwellcollins dot com kmradke at rockwellcollins dot com 2009-05-27 08:19:42 PDT
                     RE: Using memcached with FSFS repositories Johan Corveleyn <johan dot corveleyn at uz dot kuleuven dot ac dot be> Johan Corveleyn <johan dot corveleyn at uz dot kuleuven dot ac dot be> 2009-05-27 08:56:31 PDT
                         RE: Using memcached with FSFS repositories kmradke at rockwellcollins dot com kmradke at rockwellcollins dot com 2009-05-27 09:25:48 PDT
Messages per page: