August 2010 Progress Report

by phobos | September 13, 2010

Releases

  • On August 21st we released Tor Browser Bundle 1.0.10 for GNU/Linux. See https://vbdvexcmqi.oedi.net/blog/tor-browser-bundle-1010-gnulinux-relea…
  • On August 18, we released Tor 0.2.2.15-alpha. It fixes a big bug in hidden service availability, fixes a variety of other bugs that were preventing performance experiments from moving forward, fixes several bothersome memory leaks, and generally closes a lot of smaller bugs that have been filling up trac lately. See https://vbdvexcmqi.oedi.net/blog/tor-02215-alpha-released
  • Released two new versions of Orbot, Tor for Android.

    Version 1.0.2
    - added "check" yes/no dialog prompt
    - debugged iptables/transprox settings on Android 1.6 and 2.2
    - added proxy settings help screen and fixed processSettings() NPE

    Version 1.0.1
    - found and fixed major bug in per-app trans proxying; list of apps was being cached and iptables rules were not properly updated as the user changed the selection in the list

  • On August 6, we released Libevent 2.0.6-rc, the first release
    candidate of the new Libevent 2.0 series. The main new feature that
    we want from Libevent 2.0 is its support for buffer-based (rather than
    socket-based) network abstractions, which will let us support Windows the
    way it wants to be supported. The new Libevent includes a wide variety of
    other features that will make our lives easier too, ranging from making
    it easier to support multi-threaded crypto operations to making it easier
    to modularly change Tor's transport from TLS-over-TCP to other options. See http://levent.git.sourceforge.net/git/gitweb.cgi?p=levent/levent;a=blob…

Funding
We finished the first year of our NSF grant, and wrote up the annual
(interim) report on what metrics work we've done and what research
projects we've helped with. We were then awarded the second year of the
two-year grant.

Design, develop, and implement enhancements that make Tor a better
tool for users in censored countries.

Continuing research into China's Great Firewall shows bridges are surviving for 1-2 weeks before being blocked. We're working to improve the bridge database such that it only gives out bridges that work in a requested country. Right now, we hand out a random selection of 3 bridges regardless of potential country of usage. In China, bridges and relays are still blocked by IP address and TCP port combinations.

Architecture and technical design docs for Tor enhancements
related to blocking-resistance.

We've brainstormed future tasks that need to be done for Tor, broken down
into two sets by upcoming priority:
https://trac.torproject.org/projects/tor/wiki/sponsors/SponsorD/March20…
and
https://trac.torproject.org/projects/tor/wiki/sponsors/SponsorD/June2011

We wrote up project sketches of how to tackle some of these tasks, such as:

Outreach and Advocacy

  • Roger met with the University of California - San Diego research group to help them better understand the research challenges in Tor. The full trip report is available at https://vbdvexcmqi.oedi.net/blog/trip-report-ucsd. Roger's presentation can be found at https://svn.torproject.org/svn/projects/presentations/slides-ucsd10.pdf.
  • Roger met with a researcher from The Cooperative Association for Internet Data Analysis (CAIDA) to discuss Tor, metrics, and analyzing our growing collection of data about the Tor network.
  • Roger attended the "Workshop on Cyber Security Data for Experimentation" organized by the National Science Foundation (NSF). The premise of the workshop was that many academics need real-world data sets to solve problems, whereas industry is the place with the real-world data sets and they don't have any real reason to share. By getting the academics and the industry people talking, with government funders nearby, they hoped to better understand the problems and maybe move things forward.
    Roger was there (and on the first panel) because of Tor's work on gathering Tor network snapshots, performance data, and user statistics. Tor's approach represents one way out of the trap where researchers never quite get the data they want, or if they do it isn't open enough (which hinders whether anybody else can reproduce their results). The full trip report is available at https://vbdvexcmqi.oedi.net/blog/trip-report-nsf-data-workshop.
  • Roger did a talk to a half dozen NSF program managers, to bring them up to speed on what Tor is up to and what sort of measurement and research projects we're working on and should work on next. The presentation can be found at https://svn.torproject.org/svn/projects/presentations/slides-nsf10.pdf.
  • Erinn discussed Tor and free software at DebConf 2010 in New York City. http://debconf10.debconf.org/.
  • Andrew and Paul presented at The International Conference on Cyber Security. http://www.iccs.fordham.edu/. Andrew's presentation can be found at https://svn.torproject.org/svn/projects/presentations/2010-iccs-tor-ove…. Paul presented the current attacks on Tor's design from a research perspective, as well as giving a briefing on current topics of research into trusted routing within Tor.
  • Ian, Roger, and others attended Usenix Security 2010, http://www.usenix.org/events/sec10/index.html.

Preconfigured privacy (circumvention) bundles for USB or LiveCD.

  • Torbrowser Bundle for GNU/Linux version 1.0.10 released. See above for the details.
  • Torbrowser Bundle for OSX works on OS X 10.4 and 10.5. 10.6 users continue to experience issues with the launching of the Firefox browser. Erinn and Steven are continuing to debug the 10.6 issues.

Bridge relay and bridge authority work.

We're developing designs to better handle bridge distribution. Part of this is to enable Tor clients to become bridges and then relays by default. The other part of this is for Tor clients to always have a set of working bridges, requesting new bridges in advance, or be able to track bridge address changes and update accordingly.

Scalability, load balancing, directory overhead, efficiency.

  • Accepted Proposal 174 for "Optimistic Data for Tor: Server Side" from Ian Goldberg. This change will save one OP-Exit round trip (down to one from two). There are still two SOCKS Client-OP round trips (negligible time) and two Exit-Server round trips. Depending on the ratio of the Exit-Server (Internet) RTT to the OP-Exit (Tor) RTT, this will decrease the latency by 25 to 50 percent. Experiments validate these predictions. [Goldberg, PETS 2010 rump session; see https://thunk.cs.uwaterloo.ca/optimistic-data-pets2010-rump.pdf] The full proposal can be read at https://gitweb.torproject.org/tor.git/blob_plain/HEAD:/doc/spec/proposa….
  • Mike Perry fixed a number of bugs in the bandwidth authority and exit scanner codebases. The exit scanner codebase is updated with the work of a Google Summer of Code student.
  • A Google Summer of Code student, Harry Bock, worked on improving the Tor DNSEL codebase. TorDNSEL is a DNSBL-style interface for querying information about Tor exit nodes, to be more thorough, more usable, and more maintainable. Out of this effort came TorBEL, a set of specifications and Python tools that try to address this problem. The full writeup on the new software can be found on our blog at https://vbdvexcmqi.oedi.net/blog/torbel-tor-bulk-exit-list-tools.

Translations
Runa implemented a change to the publishing of translated website pages. We now only publish a non-English page to the website if the text is 80% translated or more. This has cleared out hundreds of older, untranslated pages from the website that were misinforming and confusing readers.

Updated translations for all documents in Arabic, Persian, German, French, Polish, Romanian, Russian, Norwegian, Mandarin Chinese, and Turkish.

Comments

Please note that the comment area below has been archived.

Anonymous (not verified) said:

September 13, 2010

Permalink

Can you guys make a Tor Browser Bundle without Firefox? It would make LiveCD usage so much better in that closing the browser does not close Tor. I made this LiveCD (test it using VirtualBox) using Midori and XChat & different config directories so that one shortcut will use the default network, and the other will use Tor.

42.7MB http://www.mediafire.com/file/3uxbsu795ud3rbk/slitaz-tor-0.1.iso

Screenshot: http://imgur.com/pAbNf.png

Anonymous (not verified) said:

September 14, 2010

Permalink

I downloaded vidalia-0.2.9.tar.gz. I'm not a coder but removing more than 50 instances of the words "browser" and "firefox" from the source code seems like a bad idea.

However: I might be able to just remove the dialog error box that appears when the browser isn't found. :)

Anonymous (not verified) said:

September 14, 2010

Permalink

Yeah, I agree. The fact that closing Firefox also closes Tor is annoying especially for relay operators.

Anonymous (not verified) said:

September 14, 2010

Permalink

"Libevent ... support for buffer-based (rather than socket-based) network abstractions..."

I don't really understand what Libevent is, is it to be plumbed into a Tor package at some point in the future? I keep running into the "ENOBUFS - Out of memory" "WSAENOBUFS" "Error 10050" problem here which is getting a bit tiring. I gather this change should help cure that if I read it right?

Anonymous (not verified) said:

September 20, 2010

Permalink

Very good, so does 0.2.2.15-alpha contain the new Libevent method then, as I still encounter the same problem with it;

Sep 20 07:40:11.890 [warn] write() failed: WSAENOBUFS. Not enough ram?

I wouldn't have thought RAM was an issue here, unless it's some obscure system buffer tucked away somewhere defined (and limited) by Windows (probably).

These were the stats during the failed state;

Ram Total 2,096,620 KB
Available 1,380,376 KB
System Cache 248,356 KB
Kernel Memory Total 315,592 KB
Kernel Memory Paged 55,496 KB
Kernel Memory Nonpaged 260,088 KB
Tor.exe Mem Usage 39,856 KB
Tor.exe VM Size 24,692 KB

In tests prior a netstat hasn't revealed anything either; no masses of ports left open in a limbo state or anything, only what one would expect to see. So far I haven't been able to figure out just what to query where on the system to monitor whatever it is that's running out.

Whatever is going on it seems pretty clear that the Tor process isn't releasing some resource it's consuming, something I can't say I've experienced with other (similarly heavy) applications I've used in the past (which causes me to ask what it is that Tor does differently).

We have not built packages for windows which include libevent 2.0 yet. Libevent 2.0 is in release candidate. We're going to wait until libevent 2.0 is stable and start building the -alpha packages with it.

Bug 98 has the details of the problem, https://trac.torproject.org/projects/tor/ticket/98. The core problem is the way Windows handles sockets, which is not the proper way to do heavy network i/o on Windows. See http://msdn.microsoft.com/en-us/library/ms819739.

The real answer is buffer events, and this code is in libevent 2.0.

Anonymous (not verified) said:

September 14, 2010

Permalink

I was able to make TorBrowser standalone, to a degree. Just search for "unable to start" in /vidalia-0.2.9/src/vidalia/MainWindow.cpp and remove the 4 lines so that "MainWindow::onBrowserFailed(QString errmsg)" looks like this:

{
Q_UNUSED(errmsg);
}

and compile Vidalia from there. After that, you can simply replace the executable in ./App/.

A link to version 1.0.10 without Firefox & the modified Vidalia for those that can't compile: http://www.mediafire.com/file/zsx4mst75vdkj32/Torbundle-linux-i386-1.0…