SSL/HTTPS

The following urlcp settings control SSL/HTTPS-related behavior of <fetch>/<submit>.

• addentropy $entropy [$fraction] Uses random data in string $entropy to help seed random number generator for SSL/HTTPS plugin initialization. $fraction is the fraction of $random considered to be truly random; it is a floating-point value from 0.0 to 1.0 (default 1.0). Only some Unix platforms need entropy for SSL startup, and it is usually obtained from the prngd daemon; this setting is a last-ditch fallback and should not be needed in a production environment. Note that providing non-random data to addentropy can decrease the security of SSL/HTTPS connections. Added in version 4.01.1028146532 20020731.

• embedsecurity (string) Sets security level for embedded URLs (e.g. frames, scripts) in a page. This controls whether or not to fetch such embedded objects based on the relative security (HTTPS or not) of the main URL and the embedded object URL. Possible values are:

  • off Default: fetch any embedded object URL as requested.

  • nodecrease Do not fetch non-HTTPS embedded objects if the main page is HTTPS.

  • noincrease Do not fetch HTTPS embedded objects if the main page is non-HTTPS.

  • sameprotocol Do not fetch embedded objects unless they are the same protocol (http, https, ftp etc.) as the main page.
  Note that currently, this setting only treats HTTPS URLs as secure; it does not check if HTTP URLs were upgraded to SSL via the Upgrade header. Returns previous setting. Added in version 4.01.1031348302 20020906. See also secure setting, p here.

• entropypipe (string) Sets the Unix file pipe path used to access the entropy daemon, when initializing the SSL/HTTPS plugin on some platforms. (The entropy daemon is only used when needed on Unix platforms that do not have /dev/random.) The default is the [Texis] Entropy Pipe setting in texis.ini (here), or if that is not set, etc/egd-pool in the install directory. The value none indicates that no pipe is set, i.e. the entropy daemon should not be accessed. Returns 1 on success, 0 if failure. Added in version 4.01.1031605926 20020909. See also the prngdpid value of the urlinfo function (here).

• secure (string) Controls whether pages are fetched securely (via SSL) or not, and whether to attempt to upgrade security if needed. Possible values are:

  • off Security is not needed. Pages will be fetched using the security provided by the URL/proxy protocol, if any; i.e. only HTTPS fetches will be secure. This is the default.

  • preferred Security is preferred but not required. If a non-secure protocol is used and supports upgraded security, the upgrade attempt will be made, but if the upgrade fails, or only part of the transaction can be made secure (e.g. response but not request), the rest of the fetch will still be made normally (insecurely). Currently only HTTP fetches can be upgraded to SSL, via the RFC 2817 Upgrade method.

  • required Security is required. Insecure protocols will only be permitted if they can be fully upgraded to secure (before the main URL request is made), otherwise the fetch will fail. Only HTTPS fetches, and HTTP with a successful Upgrade, are possible.

  Note that for this setting, security applies to the over-the-wire protocol; i.e. an FTP URL can still be fetched securely using an HTTPS proxy. Security also applies to the first-hop connection, i.e. Vortex <fetch>: a proxy may or may not have a secure connection from itself to the requested resource.

  When upgrading an HTTP connection to SSL via the OPTIONS-Upgrade RFC 2817 method, the OPTIONS method is used as a "no-op" request first, to negotiate the connection secure before the main URL request is sent. Upgrading to SSL requires Upgrade header, OPTIONS method, and Keep-Alive connection support by both the client (Vortex) and the server: if any of these are disabled or unsupported, the transaction may be partially or wholly insecure. For example, if the OPTIONS method is disabled via <urlcp methods> (here), the SSL upgrade would have to be attempted on the main request, instead of the no-op OPTIONS request: if secure is required, this is not allowed and the fetch fails. But if secure is preferred, this is permitted, and although the request will be insecure, at least the server response is secure (if the upgrade succeeds). This can be a way of getting at least a partially-secure transaction from a server that does not support OPTIONS.

  If a transaction cannot be conducted wholly securely and secure is set to required, an error such as Will not fetch URL http://...: Secure transaction not possible: ... will occur. Reasons include:

  • TLS/SSL Upgrade was ignored, failed or insecure An Upgrade to SSL failed.

  • TLS/SSL Upgrade via OPTIONS not possible: OPTIONS disallowed The OPTIONS method is disallowed (via <urlcp methods>, here), and is needed to make the main request secure (by upgrading before the main request).

  • Main request will not be possible on same connection if TLS/SSL Upgraded via OPTIONS: maxconnrequests <= 1 The connection would have to be closed after the OPTIONS upgrade - before the main request can be made securely - since <urlcp maxconnrequests> is less than or equal to 1.

  The secure setting returns its previous value. See also the secure option to <urlinfo> (here). Added in version 6.

• sslcertificatefile $certFile

  Sets SSL client (Vortex) certificate to PEM-encoded file $certFile; returns 1 on success, 0 on error. The certificate will be sent to HTTPS/SSL servers that specifically request it upon contact. (Not all servers do; client certificates are usually only requested if the server has SSL client authentication enabled.) An empty or unset $certFile value will clear the existing certificate (if any). Any pre-existing (cached/Keep-Alive) SSL connections will be terminated, so that the new certificate will be used.

  Note that if a certificate is set, the corresponding private key must also be set with the sslcertificatekey setting, or errors such as "SSL certificate set without certificate key: secure connections may fail" and "Cannot complete SSL handshake with host: ... alert handshake failure" may result during fetches. (The same PEM file may be used to contain both the certificate and key; however the sslcertificatekeyfile setting must still also be called.)

  If the server being connected to does authentication of client certificates, and the $certFile certificate is rejected, an error such as "Cannot complete SSL handshake with host: ... alert bad certificate", "... alert certificate unknown" or "alert unknown ca" may result.

  The sslcertificate setting was added in version 6.00.1320460000 20111104. The default is unset (no client certificate). Note that setting a client certificate is not generally needed; it is usually only used when contacting an HTTPS server that does SSL client authentication.

• sslcertificate $certString Same as sslcertificatefile, but loads from a PEM string $certString instead of a file.

• sslcertificatekeyfile $keyFile [$password]

  Sets SSL client (Vortex) certificate private key to PEM-encoded file $keyFile: this key must correspond to the client certificate loaded with the sslcertificatefile setting. Returns 1 on success, 0 on error. An empty or unset $keyFile will clear the existing key (if any). The optional $password argument gives the password used to decrypt the key, if encrypted. Any pre-existing (cached/Keep-Alive) SSL connections will be terminated, so that the new key will be used.

  Note that if a key is set, a certificate must also be set with the sslcertificatefile setting, or errors such as "SSL certificate key set without certificate: secure connections may fail" and "Cannot complete SSL handshake with host: ... alert handshake failure" may result during fetches. (The same PEM file may be used to contain both the certificate and key; however the sslcertificatefile setting must still also be called.)

  If the key is encrypted and the $password argument is incorrect, an error such as "Cannot parse SSL certificate key: Bad password" may result. If the key is encrypted and the $password argument is missing, an error such as "Cannot obtain password to decrypt SSL certificate key: Missing or empty <urlcp sslcertificatekey> password argument" may result. If the key does not match the certificate set with sslcertificate, an error such as "Cannot use SSL certificate key: ... key values mismatch" may result during fetches.

  The sslcertificatekey setting was added in version 6.00.1320460000 20111104. The default is unset (no client certificate private key). Note that setting a client certificate and key is not generally needed; it is usually only used when contacting an HTTPS server that does SSL client authentication.

• sslcertificatekey $keyString [$password]

  Same as sslcertificatekeyfile, but loads key from PEM string $keyString instead of a file.

• sslcertificatechainfile $chainFile [skipfirst] Sets the chain for the sslcertificatefile client certificate to the PEM-encoded CA certificate(s) in file $chainFile, in order from issuer to root. Returns the number of certificates loaded into the chain (i.e. 0 or more), or -1 on error (e.g. file not found). (Note that it is not an error for there to be no certificates in the chain file.)

  A certificate's chain is the list of zero or more CA (certificate authority) certificates that form its "pedigree", i.e. so its authenticity can be verified and it can be trusted by peers - in this case by servers, since this chain is for the client (Vortex) certificate. A chain starts with the certificate of the CA that issued (signed) the leaf certificate (sslcertificate in this case), followed by the certificate of the CA that issued that CA certificate, etc. up to a root (self-signed) CA certificate. Any certificate that was issued by another certificate (i.e. is not self-signed) needs a chain.

  HTTPS/SSL servers that do client authentication may need the client's chain to be able to follow it up to a root certificate that they trust: otherwise they cannot verify the client certificate and may terminate the connection with an SSL error (that may mention "alert bad certificate", "alert certificate unknown", or "alert unknown ca").

  Note that if no chain is set with sslcertificatechainfile - but the sslcertificatefile certificate needs one - the Vortex fetch library will attempt to automatically complete the chain with certificates from the sslcacertificatefile list, if possible. Since sslcacertificatefile certificates are trusted but sslcertificatechainfile certificates are not, if the client chain certificates are not to be trusted (i.e. for verification of servers) they should be set with sslcertificatechainfile and not sslcacertificatefile.

  If the skipfirst option is given, the first certificate in $chainFile is skipped. This facilitates using an "all-in-one" PEM file that contains both the client certificate and its chain certificate(s): the same file can be given to both sslcertificatefile and sslcertificatechainfile, with the skipfirst option for the latter.

  The sslcertificatechainfile setting was added in version 6.00.1320460000 20111104. The default is unset (chain auto-completed from sslcacertificatefile certificates, or no chain). Note that a chain is only needed if a certificate is set.

• sslcertificatechain $chainString [skipfirst]

  Same as sslcertificatechainfile, except that the chain is loaded from the PEM string $chainString instead of a file.

• sslcacertificatefile $caCertFile Sets the list of trusted CA certificates to the PEM-encoded certificates in file $caCertFile. Returns the number of certificates read (0 or more), or -1 on error. (Note that it is not an error for there to be no certificates in the file.)

  Trusted certificates are used when verifying peer certificates (server certificates in this case): the server certificate chain's root certificate must be trusted (i.e. included in this setting) for the server certificate to be verified by the Vortex fetch library. See sslverifyserver (here) for more details on verification of server certificates.

  Trusted certificates are also used to automatically complete certificate chains if needed (both server and client). See sslcertificatechainfile (here) for why it may not be best practice to implicitly set client (Vortex) chain certificates via sslcacertificatefile.

  The sslcacertificatefile setting was added in version 6.00.1320460000 20111104. The default is unset (no trusted certificates). Note that trusted certificates are only needed if sslverifyserver is not off.

• sslcacertificate $caCertString

  Same as sslcacertificatefile, except that the trusted certificate(s) are loaded from the PEM string $caCertString instead of a file.

• sslverifyserver $onOffOrFlags Enables or disables verification of HTTPS/SSL server certificates during fetches. Verification is enabled if $onOffOrFlags is on, or disabled if off (the default, which can also be set with an empty or unset value). Returns 1 on success, 0 on error (i.e. bad setting value). This verification is similar to what browsers do when connecting to a secure server.

  If verification of server certificates is enabled, and a server certificate cannot be successfully verified upon connecting to the server, the connection will be terminated with the error "Cannot verify certificate from host:port: reason at depth N", and <urlinfo errtoken> will return CannotVerifyServerCertificate. The reason may vary (e.g. certificate has expired, unable to get issuer certificate, etc.); the full list is in the SSL Client/Server Certificate Verification appendix (here). The depth N given in the message is the chain depth, or number of chain certificates away from the server certificate that the error happened. Thus depth 0 is the server certificate itself; depth 1 is the server certificate's issuer certificate, etc.

  In addition to on (perform all verification checks) or off, the sslverifyserver setting may be set to the Apache-compatible values require (same as on), none (same as off), or optional (same as on -No_Peer_Certificate). Also, any number of individual verification errors may be enabled/disabled, by listing them, space-separated, after the initial on/off/etc. value, each with a plus or minus sign for enable or disable; see here for details. Note that disabling individual sslverifyserver errors like this should be done with caution, as it can weaken the security provided by the full-check on level.

  The sslverifyserver setting was added in version 6.00.1320460000 20111104. It defaults to off. See the SSL Client/Server Certificate Verification appendix (here) for more on certificate verification.

• sslverifydepth $depth

  Sets the maximum chain depth to allow when verifying an HTTPS/SSL server certificate. This is the maximum number of CA certificates (beyond the server certififcate itself) to allow in the chain. I.e. a depth of 0 only allows a self-signed server certificate, a depth of 1 only allows a single CA chain certificate, etc.; a depth of -1 indicates no limit. Exceeding the verify depth results in a "Cannot verify certificate from host:port: certificate chain too long at depth N" error message and the connection fails.

  The sslverifydepth setting was added in version 6.00.1320460000 20111104. It defaults to 1; however chain depth is only verified if sslverifyserver is on and the latter defaults to off. Note that a depth of -1, disabling the X509_V_ERR_CERT_CHAIN_TOO_LONG error in sslverifyserver, or turning off sslverifyserver completely, will disable server chain-depth checks.

• sslciphers [$group] $cipherList

  Which SSL ciphers to allow. The $cipherList argument is a string in OpenSSL cipher list format detailing the SSL ciphers to permit. It is a list of one or more cipher strings, separated by colons. Ciphers may be removed with "-" (and never added again with "!"), or added with "+". See https://www.openssl.org/docs/apps/ciphers.html-LIST-FORMAT for details on cipher strings and syntax. Returns nonzero on success, 0 on error.

  Changing the SSL cipher list may be used to remove weak or compromised ciphers, or to reduce the size of the SSL ClientHello message to avoid timeouts when connecting to long-handshake-intolerant servers. Added in version 7.03.1435875000 20150702. Note that some ciphers (e.g. considered weak or vulnerable) may be unavailable in later versions of Texis, depending on the OpenSSL version in use by that Texis release. Note also that setting an invalid cipher list may not result in an error until an HTTPS connection is actually attempted.

  In version 7.07 and later, an optional cipher $group may be given, to set the cipher list for that protocol group. The group may be SSL (the default) for protocols TLSv1.2 and below, or TLSv1.3 for TLSv1.3 ciphers; the two lists are independent.

  Weaker SSL protocols and ciphers are deprecated in later OpenSSL versions for security reasons, especially SSLv3, TLSv1 and TLSv1.1. To use such protocols in Texis releases that use OpenSSL 3 and later (e.g. Texis version 8.00.1633988159 20211011 and later), in addition to enabling the protocol (e.g. TLSv1.1) via <urlcp sslprotocols>, it may also be necessary to add to cipher group SSL the cipher list DEFAULT:@SECLEVEL=0 to lower the security level (e.g. set <urlcp sslciphers "DEFAULT:@SECLEVEL=0">. This will enable lower-security signature algorithms to be used that legacy protocols may need. Doing so, or using legacy protocols at all, is not recommended.

• ssllegacyserverconnect yes|no|yesandwarn

  Whether to allow legacy insecure renegotiation with https servers that do not support secure renegotiation (RFC 5746). The default is yesandwarn: allows insecure renegotiation so unpatched servers can be reached, but warns (with message Enabling unsafe legacy renegotiation for N.N.N.N (www.example.com): Host does not support secure renegotiation) because this is an insecure connection (see CVE-2009-3555). When set to yes, connections also proceed but no warning message is issued (and insecure servers do not require a TCP reconnection). With this set to no, attempting to connect to a server that does not support secure renegotiation may fail with the error Cannot complete SSL handshake with www.example.com:443: error:0A000152:SSL routines::unsafe legacy renegotiation disabled.

  Note that values yesandwarn (only when a warning is issued) and yes (silently any time) may allow insecure connections vulnerable to third-party compromise; the remote host should be upgraded if possible, as support for legacy insecure renegotiation may be removed in a future release of OpenSSL in Texis. Setting added in version 8.00.1633988159 20211011, when OpenSSL 3 support was added (which disabled insecure renegotiation by default). The value yesandwarn was added (and default changed to that) in version 8.01.1672953292 20230105.

  Returns previous value, as integer (2 for yesandwarn).

• sslprotocols [add|del|set] [$sslprotocols ...]

  Which SSL protocol(s) to allow. The $sslprotocols argument(s) are zero or more of the values SSLv2, SSLv3, TLSv1, TLSv1.1, TLSv1.2, TLSv1.3 or all for all protocols. If the first value of the first argument is add, the given protocol list will be added to the allowed list; if del, deleted from; if set, cleared and set (the default). Alternately, the default SSL protocols may be restored with set default. Returns previous setting. Added in version 5.01.1184873000 20070719. The default is TLSv1, TLSv1.1, TLSv1.2, TLSv1.3. (Prior to version 7.07 TLSv1.3 was unsupported. Prior to version 7.03, TLSv1.1 and TLSv1.2 were unsupported. Prior to version 7.02.1413403000 20141015, the default was SSLv3 and TLSv1.) See SSLv3 comment under <urlcp sslciphers> (here) for additional issues if using that protocol. Note also that SSLv3, TLSv1, and TLSv1.1 may require sslsecuritylevel (here) set to 0.

• sslsecuritylevel $n

  Set OpenSSL security level to $n (0-5). Increasing security levels require increasing bit lengths, and remove weaker protocols and ciphers from availability; see the OpenSSL documentation for details. The default level is 1. Note that some weaker/deprecated protocols such as SSLv3, TLSv1, and TLSv1.1 require (weaker) security level 0, as they are unavailable in level 1 and above. Note also that enabling a protocol/cipher that is unavailable in the current security level may not immediately produce an error at the time of enabling: the lack of the protocol/cipher may only be discernable indirectly later, e.g. as an SSL handshake failure when an attempt to use the unavailable protocol is made. Setting add in version 8.01.1686081586 20230606.

• sslusesni $onOff

  Whether to use SNI (Server Name Indication) when the TLS protocol (successor to SSL) is used. When on (the default), the server hostname is set at the TLS level, enabling the remote server to send the right certificate if it has multiple certificates for different hostnames. When off, the hostname is not set, and connecting to some HTTPS servers that require SNI may fail with the error ... alert handshake failure. Returns previous setting. Added in version 7.03.1426024000 20150310. In previous versions SNI was effectively off.