SSL/HTTPS

The following urlcp settings control SSL/HTTPS-related behavior of <fetch>/<submit>.

• addentropy $entropy [$fraction] Uses random data in string $entropy to help seed random number generator for SSL/HTTPS plugin initialization. $fraction is the fraction of $random considered to be truly random; it is a floating-point value from 0.0 to 1.0 (default 1.0). Only some Unix platforms need entropy for SSL startup, and it is usually obtained from the prngd daemon; this setting is a last-ditch fallback and should not be needed in a production environment. Note that providing non-random data to addentropy can decrease the security of SSL/HTTPS connections. Added in version 4.01.1028146532 20020731.

• embedsecurity (string) Sets security level for embedded URLs (e.g. frames, scripts) in a page. This controls whether or not to fetch such embedded objects based on the relative security (https or not) of the main URL and the embedded object URL. Possible values are:

  • off Default: fetch any embedded object URL as requested.

  • nodecrease Do not fetch non-https embedded objects if the main page is https.

  • noincrease Do not fetch https embedded objects if the main page is non-https.

  • sameprotocol Do not fetch embedded objects unless they are the same protocol (http, https, ftp etc.) as the main page.
  Note that currently, this setting only treats HTTPS URLs as secure; it does not check if HTTP URLs were upgraded to SSL via the Upgrade header. Returns previous setting. Added in version 4.01.1031348302 20020906. See also secure setting, p here.

   

• entropypipe (string) Sets the Unix file pipe path used to access the entropy daemon, when initializing the SSL/HTTPS plugin on some platforms. (The entropy daemon is only used when needed on Unix platforms that do not have /dev/random.) The default is the [Texis] Entropy Pipe setting in conf/texis.ini (here), or if that is not set, "etc/egd-pool" in the install directory. The value "none" indicates that no pipe is set, i.e. the entropy daemon should not be accessed. Returns 1 on success, 0 if failure. Added in version 4.01.1031605926 20020909. See also the prngdpid value of the urlinfo function (here).

• secure (string)   Controls whether pages are fetched securely (via SSL) or not, and whether to attempt to upgrade security if needed. Possible values are:

  • off Security is not needed. Pages will be fetched using the security provided by the URL/proxy protocol, if any; i.e. only HTTPS fetches will be secure. This is the default.

  • preferred Security is preferred but not required. If a non-secure protocol is used and supports upgraded security, the upgrade attempt will be made, but if the upgrade fails, or only part of the transaction can be made secure (e.g. response but not request), the rest of the fetch will still be made normally (insecurely). Currently only HTTP fetches can be upgraded to SSL, via the RFC 2817 Upgrade method.

  • required Security is required. Insecure protocols will only be permitted if they can be fully upgraded to secure (before the main URL request is made), otherwise the fetch will fail. Only HTTPS fetches, and HTTP with a successful Upgrade, are possible.

  Note that for this setting, security applies to the over-the-wire protocol; i.e. an FTP URL can still be fetched securely using an HTTPS proxy. Security also applies to the first-hop connection, i.e. Vortex <fetch>: a proxy may or may not have a secure connection from itself to the requested resource.

  When upgrading an HTTP connection to SSL via the OPTIONS-Upgrade RFC 2817 method, the OPTIONS method is used as a "no-op" request first, to negotiate the connection secure before the main URL request is sent. Upgrading to SSL requires Upgrade header, OPTIONS method, and Keep-Alive connection support by both the client (Vortex) and the server: if any of these are disabled or unsupported, the transaction may be partially or wholly insecure. For example, if the OPTIONS method is disabled via <urlcp methods> (here), the SSL upgrade would have to be attempted on the main request, instead of the no-op OPTIONS request: if secure is required, this is not allowed and the fetch fails. But if secure is preferred, this is permitted, and although the request will be insecure, at least the server response is secure (if the upgrade succeeds). This can be a way of getting at least a partially-secure transaction from a server that does not support OPTIONS.

  If a transaction cannot be conducted wholly securely and secure is set to required, an error such as Will not fetch URL http://...: Secure transaction not possible: ... will occur. Reasons include:

  • TLS/SSL Upgrade was ignored, failed or insecure An Upgrade to SSL failed.

  • TLS/SSL Upgrade via OPTIONS not possible: OPTIONS disallowed The OPTIONS method is disallowed (via <urlcp methods>, here), and is needed to make the main request secure (by upgrading before the main request).

  • Main request will not be possible on same connection if TLS/SSL Upgraded via OPTIONS: maxconnrequests <= 1 The connection would have to be closed after the OPTIONS upgrade - before the main request can be made securely - since <urlcp maxconnrequests> is less than or equal to 1.

  The secure setting returns its previous value. See also the secure option to <urlinfo> (here). Added in version 6.

• sslcertificatefile $certFile  

  Sets SSL client (Vortex) certificate to PEM-encoded file $certFile; returns 1 on success, 0 on error. The certificate will be sent to HTTPS/SSL servers that specifically request it upon contact. (Not all servers do; client certificates are usually only requested if the server has SSL client authentication enabled.) An empty or unset $certFile value will clear the existing certificate (if any). Any pre-existing (cached/Keep-Alive) SSL connections will be terminated, so that the new certificate will be used.

  Note that if a certificate is set, the corresponding private key must also be set with the sslcertificatekey setting, or errors such as "SSL certificate set without certificate key: secure connections may fail" and "Cannot complete SSL handshake with host: ... alert handshake failure" may result during fetches. (The same PEM file may be used to contain both the certificate and key; however the sslcertificatekeyfile setting must still also be called.)

  If the server being connected to does authentication of client certificates, and the $certFile certificate is rejected, an error such as "Cannot complete SSL handshake with host: ... alert bad certificate" or "... alert certificate unknown" may result.

  The sslcertificate setting was added in version 6.00.1320460000 20111104. The default is unset (no client certificate). Note that setting a client certificate is not generally needed; it is usually only used when contacting an HTTPS server that does SSL client authentication.

• sslcertificate $certString Same as sslcertificatefile, but loads from a PEM string $certString instead of a file.

• sslcertificatekeyfile $keyFile [$password]

  Sets SSL client (Vortex) certificate private key to PEM-encoded file $keyFile: this key must correspond to the client certificate loaded with the sslcertificatefile setting. Returns 1 on success, 0 on error. An empty or unset $keyFile will clear the existing key (if any). The optional $password argument gives the password used to decrypt the key, if encrypted. Any pre-existing (cached/Keep-Alive) SSL connections will be terminated, so that the new key will be used.

  Note that if a key is set, a certificate must also be set with the sslcertificatefile setting, or errors such as "SSL certificate key set without certificate: secure connections may fail" and "Cannot complete SSL handshake with host: ... alert handshake failure" may result during fetches. (The same PEM file may be used to contain both the certificate and key; however the sslcertificatefile setting must still also be called.)

  If the key is encrypted and the $password argument is incorrect, an error such as "Cannot parse SSL certificate key: Bad password" may result. If the key is encrypted and the $password argument is missing, an error such as " Cannot obtain password to decrypt SSL certificate key: Missing or empty <urlcp sslcertificatekey> password argument" may result. If the key does not match the certificate set with sslcertificate, an error such as "Cannot use SSL certificate key: ... key values mismatch" may result during fetches.

  The sslcertificatekey setting was added in version 6.00.1320460000 20111104. The default is unset (no client certificate private key). Note that setting a client certificate and key is not generally needed; it is usually only used when contacting an HTTPS server that does SSL client authentication.

• sslcertificatekey $keyString [$password]

  Same as sslcertificatekeyfile, but loads key from PEM string $keyString instead of a file.

• sslcertificatechainfile $chainFile [skipfirst]   Sets the chain for the sslcertificatefile client certificate to the PEM-encoded CA certificate(s) in file $chainFile, in order from issuer to root. Returns the number of certificates loaded into the chain (i.e. 0 or more), or -1 on error (e.g. file not found). (Note that it is not an error for there to be no certificates in the chain file.)

  A certificate's chain is the list of zero or more CA (certificate authority) certificates that form its "pedigree", i.e. so its authenticity can be verified and it can be trusted by peers - in this case by servers, since this chain is for the client (Vortex) certificate. A chain starts with the certificate of the CA that issued (signed) the leaf certificate (sslcertificate in this case), followed by the certificate of the CA that issued that CA certificate, etc. up to a root (self-signed) CA certificate. Any certificate that was issued by another certificate (i.e. is not self-signed) needs a chain.

  HTTPS/SSL servers that do client authentication may need the client's chain to be able to follow it up to a root certificate that they trust: otherwise they cannot verify the client certificate and may terminate the connection with an SSL error (that may mention "alert bad certificate" or "alert certificate unknown").

  Note that if no chain is set with sslcertificatechainfile - but the sslcertificatefile certificate needs one - the Vortex fetch library will attempt to automatically complete the chain with certificates from the sslcacertificatefile list, if possible. Since sslcacertificatefile certificates are trusted but sslcertificatechainfile certificates are not, if the client chain certificates are not to be trusted (i.e. for verification of servers) they should be set with sslcertificatechainfile and not sslcacertificatefile.

  If the skipfirst option is given, the first certificate in $chainFile is skipped. This facilitates using an "all-in-one" PEM file that contains both the client certificate and its chain certificate(s): the same file can be given to both sslcertificatefile and sslcertificatechainfile, with the skipfirst option for the latter.

  The sslcertificatechainfile setting was added in version 6.00.1320460000 20111104. The default is unset (chain auto-completed from sslcacertificatefile certificates, or no chain). Note that a chain is only needed if a certificate is set.

• sslcertificatechain $chainString [skipfirst]

  Same as sslcertificatechainfile, except that the chain is loaded from the PEM string $chainString instead of a file.

• sslcacertificatefile $caCertFile   Sets the list of trusted CA certificates to the PEM-encoded certificates in file $caCertFile. Returns the number of certificates read (0 or more), or -1 on error. (Note that it is not an error for there to be no certificates in the file.)

  Trusted certificates are used when verifying peer certificates (server certificates in this case): the server certificate chain's root certificate must be trusted (i.e. included in this setting) for the server certificate to be verified by the Vortex fetch library. See sslverifyserver (here) for more details on verification of server certificates.

  Trusted certificates are also used to automatically complete certificate chains if needed (both server and client). See sslcertificatechainfile (here) for why it may not be best practice to implicitly set client (Vortex) chain certificates via sslcacertificatefile.

  The sslcacertificatefile setting was added in version 6.00.1320460000 20111104. The default is unset (no trusted certificates). Note that trusted certificates are only needed if sslverifyserver is not off.

• sslcacertificate $caCertString

  Same as sslcacertificatefile, except that the trusted certificate(s) are loaded from the PEM string $caCertString instead of a file.

• sslverifyserver $onOffOrFlags   Enables or disables verification of HTTPS/SSL server certificates during fetches. Verification is enabled if $onOffOrFlags is on, or disabled if off (the default, which can also be set with an empty or unset value). Returns 1 on success, 0 on error (i.e. bad setting value). This verification is similar to what browsers do when connecting to a secure server.

  If verification of server certificates is enabled, and a server certificate cannot be successfully verified upon connecting to the server, the connection will be terminated with the error " Cannot verify certificate from host:port: reason at depth N", and <urlinfo errtoken> will return CannotVerifyServerCertificate. The reason may vary (e.g. certificate has expired, unable to get issuer certificate, etc.); the full list is in the SSL Client/Server Certificate Verification appendix (here). The depth N given in the message is the chain depth, or number of chain certificates away from the server certificate that the error happened. Thus depth 0 is the server certificate itself; depth 1 is the server certificate's issuer certificate, etc.

  In addition to on (perform all verification checks) or off, the sslverifyserver setting may be set to the Apache-compatible values require (same as on), none (same as off), or optional (same as on -No_Peer_Certificate). Also, any number of individual verification errors may be enabled/disabled, by listing them, space-separated, after the initial on/off/etc. value, each with a plus or minus sign for enable or disable; see here for details. Note that disabling individual sslverifyserver errors like this should be done with caution, as it can weaken the security provided by the full-check on level.

  The sslverifyserver setting was added in version 6.00.1320460000 20111104. It defaults to off. See the SSL Client/Server Certificate Verification appendix (here) for more on certificate verification.

• sslverifydepth $depth

  Sets the maximum chain depth to allow when verifying an HTTPS/SSL server certificate. This is the maximum number of CA certificates (beyond the server certififcate itself) to allow in the chain. I.e. a depth of 0 only allows a self-signed server certificate, a depth of 1 only allows a single CA chain certificate, etc.; a depth of -1 indicates no limit. Exceeding the verify depth results in a "Cannot verify certificate from host:port: certificate chain too long at depth N" error message and the connection fails.

  The sslverifydepth setting was added in version 6.00.1320460000 20111104. It defaults to 1; however chain depth is only verified if sslverifyserver is on and the latter defaults to off. Note that a depth of -1, disabling the X509_V_ERR_CERT_CHAIN_TOO_LONG error in sslverifyserver, or turning off sslverifyserver completely, will disable server chain-depth checks.

• sslprotocols [add|del|set] [$sslprotocols ...]   Which SSL protocol(s) to allow. The $sslprotocols argument(s) are zero or more of the values SSLv2, SSLv3, TLSv1. If the first value of the first argument is add, the given protocol list will be added to the allowed list; if del, deleted from; if set, cleared and set (the default). Alternately, the default SSL protocols may be restored with set default. Returns previous setting. Added in version 5.01.1184873000 20070719. The default is to allow all but SSLv2.