Tor 0.2.7.3-rc is released

Tor 0.2.7.3-rc is the first release candidate in the 0.2.7 series. It contains numerous usability fixes for Ed25519 keys, safeguards against several misconfiguration problems, significant simplifications to Tor's callgraph, and numerous bugfixes and small features.
This is the most tested release of Tor to date. The unit tests cover 39.40% of the code, and the integration tests (accessible with "make test-full-online", requiring stem and chutney and a network connection) raise the coverage to 64.49%.
NOTE: This is a release candidate. We think we've squashed most of the bugs, but there are probably a few more left over.

Changes in version 0.2.7.3-rc - 2015-09-25

• Major features (security, hidden services):
  • Hidden services, if using the EntryNodes option, are required to use more than one EntryNode, in order to avoid a guard discovery attack. (This would only affect people who had configured hidden services and manually specified the EntryNodes option with a single entry-node. The impact was that it would be easy to remotely identify the guard node used by such a hidden service. See ticket for more information.) Fixes ticket 14917.
• Major features (Ed25519 keys, keypinning):
  • The key-pinning option on directory authorities is now advisory- only by default. In a future version, or when the AuthDirPinKeys option is set, pins are enforced again. Disabling key-pinning seemed like a good idea so that we can survive the fallout of any usability problems associated with Ed25519 keys. Closes ticket 17135.

• Major features (Ed25519 performance):
  • Improve the speed of Ed25519 operations and Curve25519 keypair generation when built targeting 32 bit x86 platforms with SSE2 available. Implements ticket 16535.
  • Improve the runtime speed of Ed25519 signature verification by using Ed25519-donna's batch verification support. Implements ticket 16533.
• Major features (performance testing):
  • The test-network.sh script now supports performance testing. Requires corresponding chutney performance testing changes. Patch by "teor". Closes ticket 14175.
• Major features (relay, Ed25519):
  • Significant usability improvements for Ed25519 key management. Log messages are better, and the code can recover from far more failure conditions. Thanks to "s7r" for reporting and diagnosing so many of these!
  • Add a new OfflineMasterKey option to tell Tor never to try loading or generating a secret Ed25519 identity key. You can use this in combination with tor --keygen to manage offline and/or encrypted Ed25519 keys. Implements ticket 16944.
  • Add a --newpass option to allow changing or removing the passphrase of an encrypted key with tor --keygen. Implements part of ticket 16769.
  • On receiving a HUP signal, check to see whether the Ed25519 signing key has changed, and reload it if so. Closes ticket 16790.
• Major bugfixes (relay, Ed25519):
  • Avoid crashing on 'tor --keygen'. Fixes bug 16679; bugfix on 0.2.7.2-alpha. Reported by "s7r".
  • Improve handling of expired signing keys with offline master keys. Fixes bug 16685; bugfix on 0.2.7.2-alpha. Reported by "s7r".
• Minor features (client-side privacy):
  • New KeyAliveSOCKSAuth option to indefinitely extend circuit lifespan when IsolateSOCKSAuth and streams with SOCKS authentication are attached to the circuit. This allows applications like TorBrowser to manage circuit lifetime on their own. Implements feature 15482.
  • When logging malformed hostnames from SOCKS5 requests, respect SafeLogging configuration. Fixes bug 16891; bugfix on 0.1.1.16-rc.
• Minor features (compilation):
  • Give a warning as early as possible when trying to build with an unsupported OpenSSL version. Closes ticket 16901.
  • Fail during configure if we're trying to build against an OpenSSL built without ECC support. Fixes bug 17109, bugfix on 0.2.7.1-alpha which started requiring ECC.
• Minor features (geoip):
  • Update geoip and geoip6 to the September 3 2015 Maxmind GeoLite2 Country database.
• Minor features (hidden services):
  • Relays need to have the Fast flag to get the HSDir flag. As this is being written, we'll go from 2745 HSDirs down to 2342, a ~14% drop. This change should make some attacks against the hidden service directory system harder. Fixes ticket 15963.
  • Turn on hidden service statistics collection by setting the torrc option HiddenServiceStatistics to "1" by default. (This keeps track only of the fraction of traffic used by hidden services, and the total number of hidden services in existence.) Closes ticket 15254.
  • Client now uses an introduction point failure cache to know when to fetch or keep a descriptor in their cache. Previously, failures were recorded implicitly, but not explicitly remembered. Closes ticket 16389.
• Minor features (testing, authorities, documentation):
  • New TestingDirAuthVote{Exit,Guard,HSDir}IsStrict flags to explicitly manage consensus flags in testing networks. Patch by "robgjansen", modified by "teor". Implements part of ticket 14882.
• Minor bugfixes (security, exit policies):
  • ExitPolicyRejectPrivate now also rejects the relay's published IPv6 address (if any), and any publicly routable IPv4 or IPv6 addresses on any local interfaces. ticket 17027. Patch by "teor". Fixes bug 17027; bugfix on 0.2.0.11-alpha.
• Minor bug fixes (torrc exit policies):
  • In torrc, "accept6 *" and "reject6 *" ExitPolicy lines now only produce IPv6 wildcard addresses. Previously they would produce both IPv4 and IPv6 wildcard addresses. Patch by "teor". Fixes part of bug 16069; bugfix on 0.2.4.7-alpha.
  • When parsing torrc ExitPolicies, we now warn for a number of cases where the user's intent is likely to differ from Tor's actual behavior. These include: using an IPv4 address with an accept6 or reject6 line; using "private" on an accept6 or reject6 line; and including any ExitPolicy lines after accept *:* or reject *:*. Related to ticket 16069.
  • When parsing torrc ExitPolicies, we now issue an info-level message when expanding an "accept/reject *" line to include both IPv4 and IPv6 wildcard addresses. Related to ticket 16069.
  • In each instance above, usage advice is provided to avoid the message. Resolves ticket 16069. Patch by "teor". Fixes part of bug 16069; bugfix on 0.2.4.7-alpha.
• Minor bugfixes (authority):
  • Don't assign "HSDir" to a router if it isn't Valid and Running. Fixes bug 16524; bugfix on 0.2.7.2-alpha.
  • Downgrade log messages about Ed25519 key issues if they are in old cached router descriptors. Fixes part of bug 16286; bugfix on 0.2.7.2-alpha.
  • When we find an Ed25519 key issue in a cached descriptor, stop saying the descriptor was just "uploaded". Fixes another part of bug 16286; bugfix on 0.2.7.2-alpha.
• Minor bugfixes (control port):
  • Repair a warning and a spurious result when getting the maximum number of file descriptors from the controller. Fixes bug 16697; bugfix on 0.2.7.2-alpha.
• Minor bugfixes (correctness):
  • When calling channel_free_list(), avoid calling smartlist_remove() while inside a FOREACH loop. This partially reverts commit 17356fe7fd96af where the correct SMARTLIST_DEL_CURRENT was incorrectly removed. Fixes bug 16924; bugfix on 0.2.4.4-alpha.
• Minor bugfixes (documentation):
  • Advise users on how to configure separate IPv4 and IPv6 exit policies in the manpage and sample torrcs. Related to ticket 16069.
  • Fix the usage message of tor-resolve(1) so that it no longer lists the removed -F option. Fixes bug 16913; bugfix on 0.2.2.28-beta.
  • Fix an error in the manual page and comments for TestingDirAuthVoteHSDir[IsStrict], which suggested that a HSDir required "ORPort connectivity". While this is true, it is in no way unique to the HSDir flag. Of all the flags, only HSDirs need a DirPort configured in order for the authorities to assign that particular flag. Patch by "teor". Fixed as part of 14882; bugfix on 0.2.6.3-alpha.
• Minor bugfixes (Ed25519):
  • Fix a memory leak when reading router descriptors with expired Ed25519 certificates. Fixes bug 16539; bugfix on 0.2.7.2-alpha.
• Minor bugfixes (linux seccomp2 sandbox):
  • Allow bridge authorities to run correctly under the seccomp2 sandbox. Fixes bug 16964; bugfix on 0.2.5.1-alpha.
  • Allow routers with ed25519 keys to run correctly under the seccomp2 sandbox. Fixes bug 16965; bugfix on 0.2.7.2-alpha.
• Minor bugfixes (open file limit):
  • Fix set_max_file_descriptors() to set by default the max open file limit to the current limit when setrlimit() fails. Fixes bug 16274; bugfix on tor- 0.2.0.10-alpha. Patch by dgoulet.
• Minor bugfixes (portability):
  • Try harder to normalize the exit status of the Tor process to the standard-provided range. Fixes bug 16975; bugfix on every version of Tor ever.
  • Check correctly for Windows socket errors in the workqueue backend. Fixes bug 16741; bugfix on 0.2.6.3-alpha.
  • Fix the behavior of crypto_rand_time_range() when told to consider times before 1970. (These times were possible when running in a simulated network environment where time()'s output starts at zero.) Fixes bug 16980; bugfix on 0.2.7.1-alpha.
  • Restore correct operation of TLS client-cipher detection on OpenSSL 1.1. Fixes bug 14047; bugfix on 0.2.7.2-alpha.
• Minor bugfixes (relay):
  • Ensure that worker threads actually exit when a fatal error or shutdown is indicated. This fix doesn't currently affect the behavior of Tor, because Tor workers never indicates fatal error or shutdown except in the unit tests. Fixes bug 16868; bugfix on 0.2.6.3-alpha.
  • Unblock threads before releasing the work queue mutex to ensure predictable scheduling behavior. Fixes bug 16644; bugfix on 0.2.6.3-alpha.
• Code simplification and refactoring:
  • Change the function that's called when we need to retry all downloads so that it only reschedules the downloads to happen immediately, rather than launching them all at once itself. This further simplifies Tor's callgraph.
  • Move some format-parsing functions out of crypto.c and crypto_curve25519.c into crypto_format.c and/or util_format.c.
  • Move the client-only parts of init_keys() into a separate function. Closes ticket 16763.
  • Simplify the microdesc_free() implementation so that it no longer appears (to code analysis tools) to potentially invoke a huge suite of other microdesc functions.
  • Simply the control graph further by deferring the inner body of directory_all_unreachable() into a callback. Closes ticket 16762.
  • Treat the loss of an owning controller as equivalent to a SIGTERM signal. This removes a tiny amount of duplicated code, and simplifies our callgraph. Closes ticket 16788.
  • When generating an event to send to the controller, we no longer put the event over the network immediately. Instead, we queue these events, and use a Libevent callback to deliver them. This change simplifies Tor's callgraph by reducing the number of functions from which all other Tor functions are reachable. Closes ticket 16695.
  • Wrap Windows-only C files inside '#ifdef _WIN32' so that tools that try to scan or compile every file on Unix won't decide that they are broken.
  • Remove the unused "nulterminate" argument from buf_pullup().
• Documentation:
  • Recommend a 40 GB example AccountingMax in torrc.sample rather than a 4 GB max. Closes ticket 16742.
  • Include the TUNING document in our source tarball. It is referred to in the ChangeLog and an error message. Fixes bug 16929; bugfix on 0.2.6.1-alpha.
• Removed code:
  • The internal pure-C tor-fw-helper tool is now removed from the Tor distribution, in favor of the pure-Go clone available from https://gitweb.torproject.org/tor-fw-helper.git/ . The libraries used by the C tor-fw-helper are not, in our opinion, very confidence- inspiring in their secure-programming techniques. Closes ticket 13338.
  • Remove the code that would try to aggressively flush controller connections while writing to them. This code was introduced in 0.1.2.7-alpha, in order to keep output buffers from exceeding their limits. But there is no longer a maximum output buffer size, and flushing data in this way caused some undesirable recursions in our call graph. Closes ticket 16480.
• Testing:
  • Make "bridges+hs" the default test network. This tests almost all tor functionality during make test-network, while allowing tests to succeed on non-IPv6 systems. Requires chutney commit 396da92 in test-network-bridges-hs. Closes tickets 16945 (tor) and 16946 (chutney). Patches by "teor".
  • Autodetect CHUTNEY_PATH if the chutney and Tor sources are side- by-side in the same parent directory. Closes ticket 16903. Patch by "teor".
  • Use environment variables rather than autoconf substitutions to send variables from the build system to the test scripts. This change should be easier to maintain, and cause 'make distcheck' to work better than before. Fixes bug 17148.
  • Add a new set of callgraph analysis scripts that use clang to produce a list of which Tor functions are reachable from which other Tor functions. We're planning to use these to help simplify our code structure by identifying illogical dependencies.
  • Add new 'test-full' and 'test-full-online' targets to run all tests, including integration tests with stem and chutney.
  • Make the test-workqueue test work on Windows by initializing the network before we begin.
  • New make target (make test-network-all) to run multiple applicable chutney test cases. Patch from Teor; closes 16953.
  • Unit test dns_resolve(), dns_clip_ttl() and dns_get_expiry_ttl() functions in dns.c. Implements a portion of ticket 16831.
  • When building Tor with testing coverage enabled, run Chutney tests (if any) using the 'tor-cov' coverage binary.
  • When running test-network or test-stem, check for the absence of stem/chutney before doing any build operations.