New Release: Tor 0.4.0.1-alpha

There's a new alpha release available for download. If you build Tor from source, you can download the source code for 0.4.0.1-alpha from the usual place on the website. Packages should be available over the coming weeks, with a new alpha Tor Browser release likely by the end of the month.

Remember, this is an alpha release: you should only run this if you'd like to find and report more bugs than usual.

Tor 0.4.0.1-alpha is the first release in the new 0.4.0.x series. It introduces improved features for power and bandwidth conservation, more accurate reporting of bootstrap progress for user interfaces, and an experimental backend for an exciting new adaptive padding feature. There is also the usual assortment of bugfixes and minor features, all described below.

Changes in version 0.4.0.1-alpha - 2019-01-18

• Major features (battery management, client, dormant mode):
  • When Tor is running as a client, and it is unused for a long time, it can now enter a "dormant" state. When Tor is dormant, it avoids network and CPU activity until it is reawoken either by a user request or by a controller command. For more information, see the configuration options starting with "Dormant". Implements tickets 2149 and 28335.
  • The client's memory of whether it is "dormant", and how long it has spent idle, persists across invocations. Implements ticket 28624.
  • There is a DormantOnFirstStartup option that integrators can use if they expect that in many cases, Tor will be installed but not used.
• Major features (bootstrap reporting):
  • When reporting bootstrap progress, report the first connection uniformly, regardless of whether it's a connection for building application circuits. This allows finer-grained reporting of early progress than previously possible, with the improvements of ticket 27169. Closes tickets 27167 and 27103. Addresses ticket 27308.
  • When reporting bootstrap progress, treat connecting to a proxy or pluggable transport as separate from having successfully used that proxy or pluggable transport to connect to a relay. Closes tickets 27100 and 28884.

• Major features (circuit padding):
  • Implement preliminary support for the circuit padding portion of Proposal 254. The implementation supports Adaptive Padding (aka WTF-PAD) state machines for use between experimental clients and relays. Support is also provided for APE-style state machines that use probability distributions instead of histograms to specify inter-packet delay. At the moment, Tor does not provide any padding state machines that are used in normal operation: for now, this feature exists solely for experimentation. Closes ticket 28142.
• Major features (refactoring):
  • Tor now uses an explicit list of its own subsystems when initializing and shutting down. Previously, these systems were managed implicitly in various places throughout the codebase. (There may still be some subsystems using the old system.) Closes ticket 28330.
• Minor features (bootstrap reporting):
  • When reporting bootstrap progress, stop distinguishing between situations where only internal paths are available and situations where external paths are available. Previously, Tor would often erroneously report that it had only internal paths. Closes ticket 27402.
• Minor features (continuous integration):
  • Log Python version during each Travis CI job. Resolves issue 28551.
• Minor features (controller):
  • Add a DROPOWNERSHIP command to undo the effects of TAKEOWNERSHIP. Implements ticket 28843.
• Minor features (developer tooling):
  • Provide a git hook script to prevent "fixup!" and "squash!" commits from ending up in the master branch, as scripts/main/pre- push.git-hook. Closes ticket 27993.
• Minor features (directory authority):
  • Directory authorities support a new consensus algorithm, under which the family lines in microdescriptors are encoded in a canonical form. This change makes family lines more compressible in transit, and on the client. Closes ticket 28266; implements proposal 298.
• Minor features (directory authority, relay):
  • Authorities now vote on a "StaleDesc" flag to indicate that a relay's descriptor is so old that the relay should upload again soon. Relays treat this flag as a signal to upload a new descriptor. This flag will eventually let us remove the 'published' date from routerstatus entries, and make our consensus diffs much smaller. Closes ticket 26770; implements proposal 293.
• Minor features (fallback directory mirrors):
  • Update the fallback whitelist based on operator opt-ins and opt- outs. Closes ticket 24805, patch by Phoul.
• Minor features (FreeBSD):
  • On FreeBSD-based systems, warn relay operators if the "net.inet.ip.random_id" sysctl (IP ID randomization) is disabled. Closes ticket 28518.
• Minor features (HTTP standards compliance):
  • Stop sending the header "Content-type: application/octet-stream" along with transparently compressed documents: this confused browsers. Closes ticket 28100.
• Minor features (IPv6):
  • We add an option ClientAutoIPv6ORPort, to make clients randomly prefer a node's IPv4 or IPv6 ORPort. The random preference is set every time a node is loaded from a new consensus or bridge config. We expect that this option will enable clients to bootstrap more quickly without having to determine whether they support IPv4, IPv6, or both. Closes ticket 27490. Patch by Neel Chauhan.
  • When using addrs_in_same_network_family(), avoid choosing circuit paths that pass through the same IPv6 subnet more than once. Previously, we only checked IPv4 subnets. Closes ticket 24393. Patch by Neel Chauhan.
• Minor features (log messages):
  • Improve log message in v3 onion services that could print out negative revision counters. Closes ticket 27707. Patch by "ffmancera".
• Minor features (memory usage):
  • Save memory by storing microdescriptor family lists with a more compact representation. Closes ticket 27359.
  • Tor clients now use mmap() to read consensus files from disk, so that they no longer need keep the full text of a consensus in memory when parsing it or applying a diff. Closes ticket 27244.
• Minor features (parsing):
  • Directory authorities now validate that router descriptors and ExtraInfo documents are in a valid subset of UTF-8, and reject them if they are not. Closes ticket 27367.
• Minor features (performance):
  • Cache the results of summarize_protocol_flags(), so that we don't have to parse the same protocol-versions string over and over. This should save us a huge number of malloc calls on startup, and may reduce memory fragmentation with some allocators. Closes ticket 27225.
  • Remove a needless memset() call from get_token_arguments, thereby speeding up the tokenization of directory objects by about 20%. Closes ticket 28852.
  • Replace parse_short_policy() with a faster implementation, to improve microdescriptor parsing time. Closes ticket 28853.
  • Speed up directory parsing a little by avoiding use of the non- inlined strcmp_len() function. Closes ticket 28856.
  • Speed up microdescriptor parsing by about 30%, to help improve startup time. Closes ticket 28839.
• Minor features (pluggable transports):
  • Add support for emitting STATUS updates to Tor's control port from a pluggable transport process. Closes ticket 28846.
  • Add support for logging to Tor's logging subsystem from a pluggable transport process. Closes ticket 28180.
• Minor features (process management):
  • Add a new process API for handling child processes. This new API allows Tor to have bi-directional communication with child processes on both Unix and Windows. Closes ticket 28179.
  • Use the subsystem manager to initialize and shut down the process module. Closes ticket 28847.
• Minor features (relay):
  • When listing relay families, list them in canonical form including the relay's own identity, and try to give a more useful set of warnings. Part of ticket 28266 and proposal 298.
• Minor features (required protocols):
  • Before exiting because of a missing required protocol, Tor will now check the publication time of the consensus, and not exit unless the consensus is newer than the Tor program's own release date. Previously, Tor would not check the consensus publication time, and so might exit because of a missing protocol that might no longer be required in a current consensus. Implements proposal 297; closes ticket 27735.
• Minor features (testing):
  • Allow a HeartbeatPeriod of less than 30 minutes in testing Tor networks. Closes ticket 28840. Patch by Rob Jansen.
• Minor bugfixes (client, clock skew):
  • Bootstrap successfully even when Tor's clock is behind the clocks on the authorities. Fixes bug 28591; bugfix on 0.2.0.9-alpha.
  • Select guards even if the consensus has expired, as long as the consensus is still reasonably live. Fixes bug 24661; bugfix on 0.3.0.1-alpha.
• Minor bugfixes (compilation):
  • Compile correctly on OpenBSD; previously, we were missing some headers required in order to detect it properly. Fixes bug 28938; bugfix on 0.3.5.1-alpha. Patch from Kris Katterjohn.
• Minor bugfixes (directory clients):
  • Mark outdated dirservers when Tor only has a reasonably live consensus. Fixes bug 28569; bugfix on 0.3.2.5-alpha.
• Minor bugfixes (directory mirrors):
  • Even when a directory mirror's clock is behind the clocks on the authorities, we now allow the mirror to serve "future" consensuses. Fixes bug 28654; bugfix on 0.3.0.1-alpha.
• Minor bugfixes (DNS):
  • Gracefully handle an empty or absent resolve.conf file by falling back to using "localhost" as a DNS server (and hoping it works). Previously, we would just stop running as an exit. Fixes bug 21900; bugfix on 0.2.1.10-alpha.
• Minor bugfixes (guards):
  • In count_acceptable_nodes(), the minimum number is now one bridge or guard node, and two non-guard nodes for a circuit. Previously, we had added up the sum of all nodes with a descriptor, but that could cause us to build failing circuits when we had either too many bridges or not enough guard nodes. Fixes bug 25885; bugfix on 0.3.6.1-alpha. Patch by Neel Chauhan.
• Minor bugfixes (IPv6):
  • Fix tor_ersatz_socketpair on IPv6-only systems. Previously, the IPv6 socket was bound using an address family of AF_INET instead of AF_INET6. Fixes bug 28995; bugfix on 0.3.5.1-alpha. Patch from Kris Katterjohn.
• Minor bugfixes (logging):
  • Rework rep_hist_log_link_protocol_counts() to iterate through all link protocol versions when logging incoming/outgoing connection counts. Tor no longer skips version 5, and we won't have to remember to update this function when new link protocol version is developed. Fixes bug 28920; bugfix on 0.2.6.10.
• Minor bugfixes (networking):
  • Introduce additional checks into tor_addr_parse() to reject certain incorrect inputs that previously were not detected. Fixes bug 23082; bugfix on 0.2.0.10-alpha.
• Minor bugfixes (onion service v3, client):
  • Stop logging a "BUG()" warning and stacktrace when we find a SOCKS connection waiting for a descriptor that we actually have in the cache. It turns out that this can actually happen, though it is rare. Now, tor will recover and retry the descriptor. Fixes bug 28669; bugfix on 0.3.2.4-alpha.
• Minor bugfixes (periodic events):
  • Refrain from calling routerlist_remove_old_routers() from check_descriptor_callback(). Instead, create a new hourly periodic event. Fixes bug 27929; bugfix on 0.2.8.1-alpha.
• Minor bugfixes (pluggable transports):
  • Make sure that data is continously read from standard output and standard error pipes of a pluggable transport child-process, to avoid deadlocking when a pipe's buffer is full. Fixes bug 26360; bugfix on 0.2.3.6-alpha.
• Minor bugfixes (unit tests):
  • Instead of relying on hs_free_all() to clean up all onion service objects in test_build_descriptors(), we now deallocate them one by one. This lets Coverity know that we are not leaking memory there and fixes CID 1442277. Fixes bug 28989; bugfix on 0.3.5.1-alpha.
• Minor bugfixes (usability):
  • Stop saying "Your Guard ..." in pathbias_measure_{use,close}_rate(). Some users took this phrasing to mean that the mentioned guard was under their control or responsibility, which it is not. Fixes bug 28895; bugfix on Tor 0.3.0.1-alpha.
• Code simplification and refactoring:
  • Reimplement NETINFO cell parsing and generation to rely on trunnel-generated wire format handling code. Closes ticket 27325.
  • Remove unnecessary unsafe code from the Rust macro "cstr!". Closes ticket 28077.
  • Rework SOCKS wire format handling to rely on trunnel-generated parsing/generation code. Resolves ticket 27620.
  • Split out bootstrap progress reporting from control.c into a separate file. Part of ticket 27402.
  • The .may_include files that we use to describe our directory-by- directory dependency structure now describe a noncircular dependency graph over the directories that they cover. Our checkIncludes.py tool now enforces this noncircularity. Closes ticket 28362.
• Documentation:
  • Mention that you cannot add a new onion service if Tor is already running with Sandbox enabled. Closes ticket 28560.
  • Improve ControlPort documentation. Mention that it accepts address:port pairs, and can be used multiple times. Closes ticket 28805.
  • Document the exact output of "tor --version". Closes ticket 28889.
• Removed features:
  • Stop responding to the 'GETINFO status/version/num-concurring' and 'GETINFO status/version/num-versioning' control port commands, as those were deprecated back in 0.2.0.30. Also stop listing them in output of 'GETINFO info/names'. Resolves ticket 28757.
  • The scripts used to generate and maintain the list of fallback directories have been extracted into a new "fallback-scripts" repository. Closes ticket 27914.
• Testing:
  • Run shellcheck for scripts in the in scripts/ directory. Closes ticket 28058.
  • Add unit tests for tokenize_string() and get_next_token() functions. Resolves ticket 27625.
• Code simplification and refactoring (onion service v3):
  • Consolidate the authorized client descriptor cookie computation code from client and service into one function. Closes ticket 27549.
• Code simplification and refactoring (shell scripts):
  • Cleanup scan-build.sh to silence shellcheck warnings. Closes ticket 28007.
  • Fix issues that shellcheck found in chutney-git-bisect.sh. Resolves ticket 28006.
  • Fix issues that shellcheck found in updateRustDependencies.sh. Resolves ticket 28012.
  • Fix shellcheck warnings in cov-diff script. Resolves issue 28009.
  • Fix shellcheck warnings in run_calltool.sh. Resolves ticket 28011.
  • Fix shellcheck warnings in run_trunnel.sh. Resolves issue 28010.
  • Fix shellcheck warnings in scripts/test/coverage. Resolves issue 28008.