Login | Register
My pages Projects Community openCollabNet

Discussions > dev [DISABLED] > [RFC/PATCH] Revamping ra_svn tunnel agents

subversion
Discussion topic

Back to topic list

[RFC/PATCH] Revamping ra_svn tunnel agents

Author ghudson
Full name Greg Hudson
Date 2003-04-26 17:19:32 PDT
Message I've written a patch to change how ra_svn decides on tunnel agents, so
that you write "svn+ssh://hostname/path" to use svn tunneled over ssh.
Other tunnel schemes can be defined in the [tunnels] section of
.subversion/config. First, the costs:

  * Everyone using tunneled ra_svn will have to switch to using
    svn+ssh: URLs; the current configuration mechanism will break.

  * This patch updates INSTALL and the default config files, but the
    book will need to be updated separately.

  * I've heard of a few people using the same working copies for
    tunneled and non-tunneled ra_svn access, using .subversion/servers
    to switch between them. The need may be real, but the mechanism
    is gross; it won't work any more. (A better long-term answer
    might be some kind of repository uuid -> URL mapping which
    overrides the URLs in .svn/entries. I don't know.)

The most obvious benefits of the new scheme:

  * This is much more GUI-client-friendly, I think. A checkout dialog
    could have a checkbox for tunneling the svn protocol over ssh, or
    perhaps a pull-down menu of defined tunnel schemes. Previously,
    you'd have to mess with user configuration in order to get that
    functionality.

  * There will no longer be any need to configure anything to get
    started with ra_svn over ssh. Just check out
    svn+ssh://hostname/path and go.

Some side benefits of this patch, which could be obtained easily
enough without revamping anything:

  * People who want to use an environment variable to specify the
    command can do so; SVN_SSH will override the default command for
    the built-in ssh tunnel scheme.

  * You can specify command arguments to the tunnel agent, using the
    apr_tokenize_to_argv() syntax (which is just standard shell
    quoting).

I've been unable to test this patch so far because of a preexisting
problem trying to get svn to work on Red Hat 9 with the native db
libraries:

  error-messages% svnadmin create /path/to/repos
  subversion/libsvn_fs​/bdb/bdb-err.c:61: (apr_err=160029)
  svn: Berkeley DB error
  svn: Berkeley DB error while creating environment for filesystem /path/to/repos/db:
  Function not implemented

Has anyone seen this before? I assume I can probably work around it
by grabbing a db library and dropping it into the svn build area.

Here are the log message and untested patch:

Change how tunnel agents are specified. Instead of specifying them by
hostname in ~/.subversion/servers, you now specify them abstractly in
the URL with "svn+scheme://hostname/path". By default, the only
defined scheme is "ssh", which runs the command $SVN_SSH if it is
define or "ssh" if not.

* Makefile.in: Add "svnsshcheck" target.
* INSTALL: Revise description of how to use ra_svn.
* subversion/libsvn_ra​/ra_loader.c
  (svn_ra_get_ra_library): Allow "+foo" suffix to URL scheme. Perform
    case-insensitive comparison per RFC 2396.
* subversion/include/svn_config.h
  (SVN_CONFIG_OPTION_S​VN_TUNNEL_AGENT): Out with the old...
  (SVN_CONFIG_SECTION_TUNNELS): ... and in with the new
* libsvn_subr/config_file.c
  (svn_config_ensure): Update default config files to document the new
    world order and not the old one.
* libsvn_ra_svn/client.c
  (parse_url): Understand and return a tunnel specifier.
  (find_tunnel_agent): Return an argv array instead of a command name,
    and use the tunnel specifier rather than the hostname to decide
    what command to run, if any.
  (ra_svn_open): Use new parse_url and find_tunnel_agent features to
    implement the new mechanism.

Index: Makefile.in
====================​====================​====================​=======
--- Makefile.in (revision 5739)
+++ Makefile.in (working copy)
@@ -247,6 +247,12 @@
 svncheck:
     @make check BASE_URL=svn://localhost
 
+# First make sure you can ssh to localhost and that "svnserve" is in
+# the path of the resulting shell.
+svnsshcheck:
+ @make check \
+ BASE_URL=svn+ssh://l​ocalhost/`pwd`/subve​rsion/tests/clients/​cmdline
+
 check-clean:
     rm -rf subversion/tests/cli​ents/cmdline/reposit​ories \
                subversion/tests/cli​ents/cmdline/working​_copies \
Index: subversion/libsvn_ra​/ra_loader.c
====================​====================​====================​=======
--- subversion/libsvn_ra​/ra_loader.c (revision 5739)
+++ subversion/libsvn_ra​/ra_loader.c (working copy)
@@ -183,7 +183,7 @@
   apr_hash_index_t *this;
   apr_hash_t *hash = ra_baton;
   
- /* Figure out which RA library key matches URL */
+ /* Figure out which RA library key matches URL. */
   for (this = apr_hash_first (pool, hash); this; this = apr_hash_next (this))
     {
       const void *key;
@@ -195,8 +195,10 @@
       apr_hash_this (this, &key, &keylen, &val);
       keystr = key;
 
- /* case-sensitive scheme comparison */
- if (memcmp (keystr, URL, keylen) == 0 && URL[keylen] == ':')
+ /* Case-insensitive comparison, per RFC 2396 section 3.1. Allow
+ URL to contain a trailing "+foo" section in the scheme. */
+ if (strncasecmp (keystr, URL, keylen) == 0 &&
+ (URL[keylen] == ':' || URL[keylen] == '+'))
         {
           *library = (svn_ra_plugin_t *) val;
           return SVN_NO_ERROR;
Index: subversion/include/svn_config.h
====================​====================​====================​=======
--- subversion/include/svn_config.h (revision 5740)
+++ subversion/include/svn_config.h (working copy)
@@ -61,7 +61,6 @@
 #define SVN_CONFIG_OPTION_HTTP_TIMEOUT "http-timeout"
 #define SVN_CONFIG_OPTION_HT​TP_COMPRESSION "http-compression"
 #define SVN_CONFIG_OPTION_NE​ON_DEBUG_MASK "neon-debug-mask"
-#define SVN_CONFIG_OPTION_SV​N_TUNNEL_AGENT "svn-tunnel-agent"
 #define SVN_CONFIG_OPTION_SS​L_AUTHORITIES_FILE "ssl-authorities-file"
 #define SVN_CONFIG_OPTION_SS​L_IGNORE_UNKNOWN_CA "ssl-ignore-unknown-ca"
 #define SVN_CONFIG_OPTION_SS​L_IGNORE_INVALID_DAT​E "ssl-ignore-invalid-date"
@@ -83,6 +82,7 @@
 #define SVN_CONFIG_OPTION_GL​OBAL_IGNORES "global-ignores"
 #define SVN_CONFIG_OPTION_LOG_ENCODING "log-encoding"
 #define SVN_CONFIG_OPTION_TEMPLATE_ROOT "template-root"
+#define SVN_CONFIG_SECTION_TUNNELS "tunnels"
 
 
 /** Read configuration information from the standard sources and merge
Index: subversion/libsvn_su​br/config_file.c
====================​====================​====================​=======
--- subversion/libsvn_su​br/config_file.c (revision 5739)
+++ subversion/libsvn_su​br/config_file.c (working copy)
@@ -776,9 +776,7 @@
       apr_file_t *f;
       const char *contents =
         "### This file specifies server-specific protocol parameters,\n"
- "### including HTTP proxy information, HTTP timeout settings, and\n"
- "### svn protocol tunnel agents (for running svn over ssh or similar\n"
- "### tools).\n"
+ "### including HTTP proxy information, and HTTP timeout settings.\n"
         "###\n"
         "### The currently defined server options are:\n"
         "### http-proxy-host Proxy host for HTTP connection\n"
@@ -789,7 +787,6 @@
         "### http-timeout Timeout for HTTP requests in seconds\n"
         "### http-compression Whether to compress HTTP requests\n"
         "### neon-debug-mask Debug mask for Neon HTTP library\n"
- "### svn-tunnel-agent Program to connect to svn server\n"
         "### ssl-authorities-file File listing known and trusted CAs\n"
         "### ssl-ignore-unknown-ca Allow untrusted server certificates\n"
         "### ssl-ignore-invalid-date Allow expired/postdated certificates\n"
@@ -838,10 +835,6 @@
         "# http-proxy-port = 9000\n"
         "# No username and password, so use the defaults below.\n"
         "\n"
- "### Information for the third group:\n"
- "# [thirdgroup]\n"
- "# svn-tunnel-agent = ssh\n"
- "\n"
         "### You can set default parameters in the 'global' section.\n"
         "### These parameters apply if no corresponding parameter is set in\n"
         "### a specifically matched group as shown above. Thus, if you go\n"
@@ -937,6 +930,26 @@
         "# diff3-has-program-arg = false\n"
 #endif
         "\n"
+ "### Section for configuring tunnel agents.\n"
+ "# [tunnels]\n"
+ "### Configure svn protocol tunnel schemes here. By default, only\n"
+ "### the `ssh' scheme is defined. You can define other schemes to\n"
+ "### be used with `svn+scheme://hostname/path' URLs. A scheme\n"
+ "### definition is simply a command, optionally prefixed by an\n"
+ "### environment variable name which can override the command if it\n"
+ "### is defined. The command (or environment variable) may contain\n"
+ "### arguments, using standard shell quoting for arguments with\n"
+ "### spaces. The command will be invoked as:\n"
+ "### <command> <hostname> svnserve -t\n"
+ "### If the built-in ssh scheme were not predefined, it could be\n"
+ "### defined as:\n"
+ "# ssh = $SVN_SSH ssh\n"
+ "### If you wanted to define a new `rsh' scheme, to be used with\n"
+ "### 'svn+rsh:' URLs, you could do so as follows:\n"
+ "# rsh = rsh\n"
+ "### Or, if you wanted to specify a full path and arguments:\n"
+ "# rsh = /path/to/rsh -l myusername\n"
+ "\n"
         "### Section for configuring miscelleneous Subversion options.\n"
         "# [miscellany]\n"
         "### Set global-ignores to a set of whitespace-delimited globs\n"
Index: subversion/libsvn_ra​_svn/client.c
====================​====================​====================​=======
--- subversion/libsvn_ra​_svn/client.c (revision 5739)
+++ subversion/libsvn_ra​_svn/client.c (working copy)
@@ -54,10 +54,12 @@
   void *edit_baton;
 } ra_svn_reporter_baton_t;
 
-/* Parse an svn URL's authority section into user, host, and port components.
- * Return 0 on success, -1 on failure. *user may be set to NULL. */
-static int parse_url(const char *url, const char **user, unsigned short *port,
- const char **hostname, apr_pool_t *pool)
+/* Parse an svn URL's authority section into tunnel, user, host, and
+ * port components. Return 0 on success, -1 on failure. *tunnel_spec
+ * and *user may be set to NULL. */
+static int parse_url(const char *url, const char **tunnel, const char **user,
+ unsigned short *port, const char **hostname,
+ apr_pool_t *pool)
 {
   const char *p;
 
@@ -65,9 +67,25 @@
   *port = SVN_RA_SVN_PORT;
   *hostname = NULL;
 
- if (strncmp(url, "svn://", 6) != 0)
+ if (strncasecmp(url, "svn", 3) != 0)
     return -1;
- url += 6;
+ url += 3;
+
+ /* Get the tunnel specification, if any. */
+ if (*url == '+')
+ {
+ url++;
+ p = strchr(url, ':');
+ if (!p)
+ return -1;
+ *tunnel = apr_pstrmemdup(pool, url, p - url);
+ url = p;
+ }
+
+ if (strncmp(url, "://", 3) != 0)
+ return -1;
+ url += 3;
+
   while (1)
     {
       p = url + strcspn(url, "@:/");
@@ -252,22 +270,67 @@
 
 /* --- RA LAYER IMPLEMENTATION --- */
 
-static svn_error_t *find_tunnel_agent(const char *hostname, const char **agent,
- apr_hash_t *config, apr_pool_t *pool)
+static svn_error_t *find_tunnel_agent(const char *tunnel, const char *hostname,
+ const char ***argv, apr_hash_t *config,
+ apr_pool_t *pool)
 {
- svn_config_t *cfg = config ? apr_hash_get (config,
- SVN_CONFIG_CATEGORY_SERVERS,
- APR_HASH_KEY_STRING) : NULL;
- const char *server_group;
+ svn_config_t *cfg;
+ const char *val, *var, *cmd;
+ char **cmd_argv;
+ apr_size_t len;
+ apr_status_t status;
+ int n;
 
- server_group = svn_config_find_group(cfg, hostname,
- SVN_CONFIG_SECTION_GROUPS, pool);
- if (! server_group)
- server_group = SVN_CONFIG_SECTION_GLOBAL;
+ /* Look up the tunnel specification in config. */
+ cfg = config ? apr_hash_get(config, SVN_CONFIG_CATEGORY_CONFIG,
+ APR_HASH_KEY_STRING) : NULL;
+ svn_config_get(cfg, &val, SVN_CONFIG_SECTION_TUNNELS, tunnel, NULL);
 
- svn_config_get(cfg, agent, server_group, SVN_CONFIG_OPTION_SV​N_TUNNEL_AGENT,
- NULL);
+ /* We have one predefined tunnel scheme, if it isn't overridden by config. */
+ if (!val && strcmp(tunnel, "ssh") == 0)
+ val = "$SVN_SSH ssh";
 
+ if (!val || !*val)
+ return svn_error_createf(SV​N_ERR_BAD_URL, NULL,
+ "Undefined tunnel scheme %s", tunnel);
+
+ /* If the scheme definition begins with "$varname", it means there
+ * is an environment variable which can override the command. */
+ if (*val == '$')
+ {
+ val++;
+ len = strcspn(val, " ");
+ var = apr_pstrmemdup(pool, val, len);
+ cmd = getenv(var);
+ if (!cmd)
+ {
+ cmd = val + len;
+ while (*cmd == ' ')
+ cmd++;
+ if (!*cmd)
+ return svn_error_createf(SV​N_ERR_BAD_URL, NULL,
+ "Tunnel scheme %s requires environment "
+ "variable %s to be defined", tunnel, var);
+ }
+ }
+ else
+ cmd = val;
+
+ /* Tokenize the command into a list of arguments. */
+ status = apr_tokenize_to_argv(cmd, &cmd_argv, pool);
+ if (status != APR_SUCCESS)
+ return svn_error_createf(status, NULL, "Can't tokenize command %s", cmd);
+
+ /* Append the fixed arguments to the result. */
+ for (n = 0; argv[n] != NULL; n++)
+ ;
+ *argv = apr_palloc(pool, (n + 4) * sizeof(char *));
+ memcpy(*argv, cmd_argv, n * sizeof(char *));
+ (*argv)[n++] = hostname;
+ (*argv)[n++] = "svnserve";
+ (*argv)[n++] = "-t";
+ (*argv)[n] = NULL;
+
   return SVN_NO_ERROR;
 }
 
@@ -307,31 +370,25 @@
 {
   svn_ra_svn_conn_t *conn;
   apr_socket_t *sock;
- const char *hostname, *user, *status, *tunnel_agent, *args[5];
+ const char *hostname, *user, *status, *tunnel, **args;
   unsigned short port;
   apr_uint64_t minver, maxver;
   apr_array_header_t *mechlist, *caplist, *status_param;
   apr_procattr_t *attr;
   apr_proc_t *proc;
 
- if (parse_url(url, &user, &port, &hostname, pool) != 0)
+ if (parse_url(url, &tunnel, &user, &port, &hostname, pool) != 0)
     return svn_error_createf(SV​N_ERR_RA_ILLEGAL_URL​, NULL,
                              "Illegal svn repository URL '%s'", url);
 
- SVN_ERR(find_tunnel_​agent(hostname, &tunnel_agent, config, pool));
- if (tunnel_agent)
+ if (tunnel)
     {
- /* ### It would be nice if tunnel_agent could contain flags. */
- args[0] = tunnel_agent;
- args[1] = hostname;
- args[2] = "svnserve";
- args[3] = "-t";
- args[4] = NULL;
+ SVN_ERR(find_tunnel_​agent(tunnel, hostname, &args, config, pool));
       apr_procattr_create(&attr, pool);
       apr_procattr_io_set(attr, 1, 1, 0);
       apr_procattr_cmdtype_set(attr, APR_PROGRAM_PATH);
       proc = apr_palloc(pool, sizeof(*proc));
- apr_proc_create(proc, tunnel_agent, args, NULL, attr, pool);
+ apr_proc_create(proc, *args, args, NULL, attr, pool);
       conn = svn_ra_svn_create_conn(NULL, proc->out, proc->in, pool);
       conn->proc = proc;
 
@@ -367,7 +424,7 @@
     return svn_error_createf(SV​N_ERR_RA_SVN_BAD_VER​SION, NULL,
                              "Server requires minimum version %d",
                              (int) minver);
- if (tunnel_agent && find_mech(mechlist, "EXTERNAL"))
+ if (tunnel && find_mech(mechlist, "EXTERNAL"))
     {
       /* Ask the server to use the ssh connection environment (on
        * Unix, that means uid) to determine the authentication name. */
Index: INSTALL
====================​====================​====================​=======
--- INSTALL (revision 5739)
+++ INSTALL (working copy)
@@ -942,28 +942,24 @@
          $ svnserve -d # becomes a background daemon
          $ svn checkout svn://localhost/usr/​local/svn/repository​
 
- This network layer passes 'make check', but is still relatively
- new and untested. You can use the "-r" option to svnserve to
- set a logical root for repositories, and the "-R" option to
- restrict connections to read-only access. ("Read-only" is a
- logical term here; svnserve still needs write access to the
- database in this mode.)
+ You can use the "-r" option to svnserve to set a logical root
+ for repositories, and the "-R" option to restrict connections to
+ read-only access. ("Read-only" is a logical term here; svnserve
+ still needs write access to the database in this mode, but will
+ not allow commits or revprop changes.)
 
+ (Expect the server invocation options to change incompatibly
+ before they are finalized.)
+
       For now, svnserve in daemon mode only supports anonymous access.
- But you can also tunnel ra_svn over ssh or any similar tool. To
- do this, create a server group for the repository server in
- ~/.subversion/servers, and set the variable "svn-tunnel-agent"
- to "ssh". An example:
+ But you can also tunnel ra_svn over ssh or any similar tool:
 
- [groups]
- mygroup = server.host.name
+ $ svn checkout svn+ssh://hostname/u​sr/local/svn/reposit​ory
 
- [mygroup]
- svn-tunnel-agent = ssh
-
- (Right now you can't put any arguments to ssh in that variable.
- That may change.) Make sure svnserve is in your path on the
- server.
-
- Expect the server invocation options to change incompatibly
- before they are finalized.
+ In this mode of operation, there is no svnserve daemon process;
+ the svn client will ssh to the server and run "svnserve -t" to
+ do the serving. Make sure svnserve is in your path on the
+ server. You can override the "ssh" command with the SVN_SSH
+ environment variable, or you can define new tunnel schemes in
+ the [tunnels] section of ~/.subversion/config. By default, only
+ the "ssh" scheme is allowed.

--------------------​--------------------​--------------------​---------
To unsubscribe, e-mail: dev-unsubscribe@subv​ersion.tigris.org
For additional commands, e-mail: dev-help at subversion dot tigris dot org

« Previous message in topic | 1 of 15 | Next message in topic »

Messages

Show all messages in topic

[RFC/PATCH] Revamping ra_svn tunnel agents ghudson Greg Hudson 2003-04-26 17:19:32 PDT
     Re: [RFC/PATCH] Revamping ra_svn tunnel agents rooneg Garrett Rooney 2003-04-26 17:42:17 PDT
         Re: [RFC/PATCH] Revamping ra_svn tunnel agents ghudson Greg Hudson 2003-04-26 18:01:22 PDT
             Re: [RFC/PATCH] Revamping ra_svn tunnel agents ghudson Greg Hudson 2003-04-26 22:31:31 PDT
                 Re: [RFC/PATCH] Revamping ra_svn tunnel agents ghudson Greg Hudson 2003-04-26 23:14:33 PDT
                 Re: [RFC/PATCH] Revamping ra_svn tunnel agents Colin Watson <cjwatson at flatline dot org dot uk> Colin Watson <cjwatson at flatline dot org dot uk> 2003-04-27 03:41:51 PDT
                     Re: [RFC/PATCH] Revamping ra_svn tunnel agents mjw Michael Wood 2003-04-27 06:00:09 PDT
                         Re: [RFC/PATCH] Revamping ra_svn tunnel agents Colin Watson <cjwatson at flatline dot org dot uk> Colin Watson <cjwatson at flatline dot org dot uk> 2003-04-27 06:20:48 PDT
                 Re: [RFC/PATCH] Revamping ra_svn tunnel agents Jani Averbach <jaa at cc dot jyu dot fi> Jani Averbach <jaa at cc dot jyu dot fi> 2003-04-27 03:55:25 PDT
             Re: [RFC/PATCH] Revamping ra_svn tunnel agents pretzelgod Eric Gillespie 2003-04-27 21:34:40 PDT
                 Re: [RFC/PATCH] Revamping ra_svn tunnel agents Ben Collins-Sussman <sussman at collab dot net> Ben Collins-Sussman <sussman at collab dot net> 2003-04-28 06:53:46 PDT
                     Re: [RFC/PATCH] Revamping ra_svn tunnel agents cmpilato C. Michael Pilato 2003-04-28 07:05:46 PDT
                         Re: [RFC/PATCH] Revamping ra_svn tunnel agents Karl Fogel <kfogel at newton dot ch dot collab dot net> Karl Fogel <kfogel at newton dot ch dot collab dot net> 2003-04-28 07:36:15 PDT
                 Re: [RFC/PATCH] Revamping ra_svn tunnel agents brane Branko Cibej 2003-04-28 09:04:07 PDT
     Re: [RFC/PATCH] Revamping ra_svn tunnel agents joe Joe Orton 2003-04-27 01:11:12 PDT
Messages per page: