{{Title| title=onion-grater, a Tor Control Port Filter Proxy }} {{Header}} {{#seo: |description=onion-grater - filtering dangerous Tor Control Port commands - Design Documentation }}
* [[onion-grater|onion-grater - User Documentation]] * [[Dev/onion-grater|onion-grater - Developer Documentation]]
{{intro| onion-grater, a Tor Control Port Filter Proxy - filtering dangerous Tor Control Port commands - Design Documentation }} = Introduction = Onion-grater is a filtering proxy for the [[Tor Controller]] listening on {{project_name_gateway_long}} internal network interface, which is the interface that the {{project_name_workstation_long}} connects when sending controller commands. It acts based on whitelists, denying everything by default and only allowing explicitly specified commands (and their arguments) and events. According to [https://gitlab.tails.boum.org/tails/tails/-/blob/master/config/chroot_local-includes/usr/local/lib/onion-grater#L3 upstream Tails OS documentation], onion-grater is defined as: ''This filter proxy allows fine-grained access whitelists of commands (and their arguments) and events on a per-application basis.'' Contrary to Tails, the fields user and per-application (AppArmor profile) are ignored with the wildcard '*' to include everything, that due to {{project_name_long}} design of user space and tor daemon separation, these methods are not possible. Instead of that, what provides greater security to further separate applications and their requests to Tor's Control Protocol is using [[Multiple_Whonix-Workstation]], and you can control per {{project_name_workstation_short}} hosts directive for the onion-grater profile to match the {{project_name_workstation_short}} LAN IP. As using Multiple {{project_name_workstation_short}} requires more work as the {{project_name_gateway_short}} does not know all the {{project_name_workstation_short}} LAN IPs, the hosts defaults to hosts: '*' and if you want to minimize the attack surface of {{project_name_workstation_short}} to the {{project_name_gateway_short}} Tor Controller, you could set the hosts directive to match your chosen {{project_name_workstation_short}} LAN IP. Tor's control port has in context of {{project_name_short}}, dangerous features. The answer to the Tor control command GETINFO address will be the real external IP of the Tor client. Other dangerous commands include SETCONF, LOADCONF, GETCONF, GETINFO ns/id/. The dangerous commands can only be limited with a proxy, since the feature request [https://gitlab.torproject.org/legacy/trac/-/issues/8369 Option to limit information Tor's control port discloses] against Tor has not been implemented. By the way, this Tor control port feature makes also a [https://gitlab.torproject.org/legacy/trac/-/wikis/doc/TorifyHOWTO/BridgeFirewall Bridge Firewall] impossible. Tor Browser [https://gitlab.torproject.org/legacy/trac/-/issues/6546#comment:33 by default performs its own control port verification]. It checks using Tor's control port(!), that the socks port Tor reports, is the same one as Tor Browser is configured to use. If it fails, and it would fail in {{project_name_short}} 0.5.6, Tor Button would look like disabled and show a failure message. (To see how this would look like, see [https://forums.whonix.org/uploads/default/original/1X/c2c9bb5dc7efee7a933dd00d3bf0c30c29c99daa.png this screenshot].) (To see how this generates user confusion, see [https://forums.whonix.org/t/something-went-wrong/2080 this forum thread].) Since {{project_name_short}} 6 this check is disabled using [https://gitlab.torproject.org/legacy/trac/-/issues/13079 environment variable to skip TorButton control port verification] (export TOR_SKIP_CONTROLPORTTEST=1) by package [https://github.com/{{project_name_short}}/anon-ws-disable-stacked-tor/blob/master/usr/libexec/anon-ws-disable-stacked-tor/torbrowser.sh anon-ws-disable-stacked-tor]. https://github.com/{{project_name_short}}/anon-ws-disable-stacked-tor/blob/master/usr/libexec/anon-ws-disable-stacked-tor/torbrowser.sh With {{project_name_workstation_short}} having no way of finding its own external IP address, joining Tor Browser's fingerprint for {{project_name_short}} users, having no access for {{project_name_workstation_short}} to Tor's control port with Tor Buttons new requirement to have access to Tor's control port contradicted itself. onion-grater has been implemented as a solution. It gives {{project_name_workstation_short}} access to a limited selection of Tor's control port commands, using whitelisting (not blacklisting). For example, it allows SIGNAL NEWNYM to make Tor Buttons New Identity feature available for users who use Tor Browser in {{project_name_workstation_short}}. onion-grater limits maximum accepted command string length to 128 (configurable) (credits: [https://web.archive.org/web/20170222094246/https://mailman.boum.org/pipermail/tails-dev/2014-February/005041.html As done by Tails.] Thanks to HulaHoop for [https://www.whonix.org/old-forum/index.php/topic,342.0.html suggesting this].) for better security. Advanced users who do not wish to use onion-grater can disable it, see [[Whonix-Gateway_Security_Hardening#Deactivate_onion-grater|deactivate onion-grater]]. = Directory of choice = * Using /usr/local/etc/onion-grater-merger.d/ because that onion-grater settings folder is persistent in [[Qubes|{{q_project_name_long}}]] TemplateBased ProxyVMs i.e. {{project_name_gateway_short}} (commonly called {{project_name_gateway_vm}}). * [[Non-Qubes-Whonix|{{non_q_project_name_short}}]] users could also utilize /etc/onion-grater-merger.d/. * [[Qubes|{{q_project_name_short}}]] users could also utilize /etc/onion-grater-merger.d/ but then /etc/onion-grater-merger.d/ must be made persistent, which means doing this procedure inside the {{project_name_gateway_short}} Template (commonly called {{project_name_gateway_template}}) and then restarting the {{project_name_gateway_short}} ProxyVM or using [https://www.qubes-os.org/doc/bind-dirs/ bind-dirs]. Both techniques are more complicated than simply using /usr/local/etc/onion-grater-merger.d/, since it is persistent either way. Further, it even allows multiple {{project_name_gateway_short}} ProxyVMs based on the same {{project_name_gateway_short}} Template; for example, one {{project_name_gateway_short}} ProxyVM extending and relaxing onion-grater's whitelist and the other {{project_name_gateway_short}} ProxyVM having the default onion-grater whitelist which is more restrictive. Onion-grater will only evaluate *.yml files. = {{project_name_short}} connection = == {{project_name_gateway_short}} == onion-grater listens on {{project_name_short}} internal network interface https://github.com/{{project_name_short}}/whonix-gw-network-conf/blob/master/usr/lib/systemd/system/onion-grater.service.d/30_cpfpy.conf , port 9051, filters (whitelist) incoming control port messages and forwards them to the real Tor Control Port listening on ControlSocket /run/tor/control. onion-grater itself uses cookies authentication to authenticate Tor's control port. The latter is not important, since {{project_name_gateway_short}} only purpose is running Tor, its not a multi user operating system, and if it were compromised, cookie authentication wouldn't be of help anymore either. == {{project_name_workstation_short}} == {{project_name_short}} sets appropriate environment variables for the controller via unix domain socket TOR_CONTROL_IPC_PATH=/run/anon-ws-disable-stacked-tor/127.0.0.1_9151.sock, which is the socket Tor Browser uses. This functionality is implemented by the [https://github.com/{{project_name_short}}/anon-ws-disable-stacked-tor anon-ws-disable-stacked-tor] package. See also [[Dev/anon-ws-disable-stacked-tor]]. = Attack Scenarios = Once {{project_name_workstation_short}} has been compromised, the adversary could continuously and/or using a pattern, send whitelisted commands to Tor. At the moment, only {{Code2|NEWNYM}} would be of interest. When the adversary is also an ISP level adversary, the adversary might be able to see the pattern being produced. However thinking that an attacker requires access to Tor's Controller to try to deanonymize you is wrong, there are numerous attacks that are possible without needing to extract information and manipulate tor configuration. The attacker can simply download code from any website including his own and begin to run his code. He could also make very unique patterns on the network and through traffic analysis to learn more about the paths the tor client is using. More worrying is the extended attack surface that allowing commands to be sent from a compromised {{project_name_workstation_short}} to a {{project_name_gateway_short}}, such as an unsafe parsing of commands, daemon C code memory leaks etc. Information from this kinda of attacks [https://gitlab.torproject.org/tpo/network-health/metrics/analysis/-/issues/5928 were asked upstream but had no manifestation of interest] ([https://web.archive.org/web/20200922223531/https://trac.torproject.org/projects/tor/ticket/5928 web archive]). = Imitating real Tor Control Connection = Should Tor close the connection, onion-grater will also close client connection. (Same for a real Tor control connection.) Both real Tor control connection and onion-grater close the connection if commands are sent when the client is not yet authenticated. (With the exception of authenticate, authchallenge and protocolinfo so there is parity.) Should Tor authentication fail, this is logged and onion-grater closes client connection. == Talking to the real Tor Controller == If instead of connecting to onion-grater, you want to talk directly with the Tor Controller, it should be done on the {{project_name_gateway_short}} and using the host 127.0.0.1, see [[Tor_Controller#Talking_to_the_real_Tor_Controller|Talking to the real Tor Controller]] for more information. = How to do onion-grater profiles = As an exercise, try building a profile for an application already supported by {{project_name_short}}, if you have trouble, you can always learn from the examples available at /usr/share/doc/onion-grater-merger/examples in your {{project_name_gateway_short}}. == For third-party projects with Tor friendly applications == Please document the controller commands, give hints to the scripts that send Tor commands, this can greatly increase the chance of your application being included to {{project_name_short}} and used by real users. Make your application handle unexpected replies such as 510 Command filtered without crashing, onion-grater is a whitelist filter, not a blacklist, this means that there is a great change the necessary commands your application requires may be filtered if not whitelisted. If your program crashes, that would lead to bad usability. = Profile format = There is no need to explain every directive in details as it is better to read the [https://gitlab.tails.boum.org/tails/tails/-/blob/master/config/chroot_local-includes/usr/local/lib/onion-grater#L3 code comments] for more up to date information. The following sections only mentions about differences in directive usage on {{project_name_short}} in comparison with Tails OS. A filter is matched if for each of the relevant qualifiers at least one of the elements match the client, the qualifiers being apparmor-profiles, users and hosts. It is very common for multiple qualifiers to match on {{project_name_short}}, as apparmor-profiles and users can't be set and setting hosts requires user intervention. == How onion-grater parses files == Parsing stops '''at the first match''', this means that it is not possible to override files by user a name with higher precedence such as 50_user.yml over 30_whonix-default.yml, what onion-grater does is sort the files in reverse lexical order, so if 50_user.yml matches, it will stop there, else continue to 30_whonix-default.yml. == name == We are not using the ''name'' directive because it is optional and the script names are already making the names unique. == apparmor-profiles == For local (loopback) clients. We can't use the ''apparmor-profiles'' directive because as per {{project_name_short}} design, the user programs are run on the {{project_name_workstation_short}} and the controller is on the {{project_name_gateway_short}}. Because of this, we default to: 
  apparmor-profiles:
    - '*'
== users == For local (loopback) clients. We can't use the ''users'' directive because as per {{project_name_short}} design, the user programs are run on the {{project_name_workstation_short}} and the controller is on the {{project_name_gateway_short}}. Because of this, we default to: 
  users:
    - '*'
== hosts == For remote (non-local) clients. We can use the ''hosts'' directive, but the {{project_name_gateway_short}} doesn't know which of your Multiple {{project_name_workstation_short}} requires the whitelist. Because of this, we default to: 
  hosts:
    - '*'
You can optionally change that directive to allow only the IP of your chosen {{project_name_workstation_short}}, therefore you can choose per host the rules you want to be whitelisted. = Application Support = Some applications that uses the commands and events whitelisted by default don't need to worry about setting profiles. Some applications might already have profiles for other applications that already cover their usage. == Tor Browser == * '''Page''': [[Tor Browser]] * '''Profile''': Mostly whitelisted by default except for [https://github.com/Whonix/onion-grater/blob/master/usr/share/doc/onion-grater-merger/examples/40_onion_authentication.yml /usr/share/doc/onion-grater-merger/examples/40_onion_authentication.yml] == OnionShare == * '''Page''': [[OnionShare]] * '''Profile''': [https://github.com/Whonix/onion-grater/blob/master/usr/share/doc/onion-grater-merger/examples/40_onionshare /usr/share/doc/onion-grater-merger/examples/40_onionshare.yml] == Bitcoin Core == * '''Page''': [[Bitcoin_Core]] * '''Profile''': [https://github.com/Whonix/onion-grater/blob/master/usr/share/doc/onion-grater-merger/examples/40_bitcoind.yml /usr/share/doc/onion-grater-merger/examples/40_bitcoind.yml] == Bisq == * '''Page''': [[Bisq]] * '''Profile''': [https://github.com/Whonix/onion-grater/blob/master/usr/share/doc/onion-grater-merger/examples/40_bisq.yml /usr/share/doc/onion-grater-merger/examples/40_bisq.yml] == Wahay == * '''Page''': [https://wahay.org/ wahay.org] * '''Profile''': [https://github.com/Whonix/onion-grater/blob/master/usr/share/doc/onion-grater-merger/examples/40_wahay.yml /usr/share/doc/onion-grater-merger/examples/40_wahay.yml] == ZeroNet == * '''Page''': [[ZeroNet]] * '''Profile''': [https://github.com/Whonix/onion-grater/blob/master/usr/share/doc/onion-grater-merger/examples/40_zeronet.yml /usr/share/doc/onion-grater-merger/examples/40_zeronet.yml] == Systemcheck == * '''Page''': [https://www.kicksecure.com/wiki/Systemcheck systemcheck] * '''Profile''': asks onion-grater status/bootstrap-phase Find out what status/bootstrap-phase answers yourself in {{project_name_gateway_short}}. {{CodeSelect|code= /usr/lib/helper-scripts/tor_bootstrap_check.py 127.0.0.1 9051 1 }} Example answer. {{CodeSelect|code= NOTICE BOOTSTRAP PROGRESS=100 TAG=done SUMMARY="Done" }} as well as status/circuit-established Find out what status/circuit-established answers yourself in {{project_name_gateway_short}}. {{CodeSelect|code= /usr/lib/helper-scripts/tor_circuit_established_check.py 127.0.0.1 9051 1 }} Example answer. {{CodeSelect|code= 1 }} in Tor Bootstrap Status Check https://github.com/Kicksecure/systemcheck/blob/master/usr/libexec/systemcheck/check_tor_bootstrap.bsh {{CodeSelect|code= whonixcheck --function check_tor_bootstrap }} {{CodeSelect|code= whonixcheck --function check_tor_bootstrap --verbose --debug }} (usability feature). == TorLauncher == To get a list of Tor ControlPort commands, that TorLauncher uses, get into TorLauncher source code, extract TorLauncher, then run. {{CodeSelect|code= grep -r -i getconf }} == TorButton == === TorButton commands === To get a list of Tor ControlPort commands, that TorButton uses, get into TorButton source code, extract TorButton, then run. {{CodeSelect|code= grep -r -i torbutton_send_ctrl_cmd * }} {{CodeSelect|code= grep -r -i sendCommand * }} === TorButton Network Settings === {{CodeSelect|code= TorButton → Open Network Settings... }} Is using. {{CodeSelect|code= "GETCONF Socks4Proxy" "GETCONF Socks5Proxy" "GETCONF HTTPSProxy" "GETCONF ReachableAddresses" "GETCONF ReachableAddresses" "GETCONF UseBridges" "GETCONF Bridge" }} As the next logical step by the way, they are likely going to add. {{CodeSelect|code= "GETCONF HTTPProxy" }} We are [https://github.com/{{project_name_short}}/anon-ws-disable-stacked-tor/blob/master/usr/libexec/anon-ws-disable-stacked-tor/torbrowser.sh setting] environment variable [https://gitlab.torproject.org/legacy/trac/-/issues/14100 export TOR_NO_DISPLAY_NETWORK_SETTINGS=1] to disable the "TorButton" → "Open Network Settings..." menu item. It is not useful and confusing to have on a workstation, because Tor must be configured on the gateway, which is for security reasons forbidden from the workstation. === TorButton issues === Looking for contributor! * Say hello in the [https://forums.whonix.org/c/development {{project_name_short}} Development Forum]. ==== Clock skew ==== In the future, Tor Button will likely also use something like [https://gitlab.torproject.org/legacy/trac/-/issues/3652 GETINFO clockskew]. Tor Browser developer [https://gitlab.torproject.org/legacy/trac/-/issues/8032 rejected] the idea of not adding the statement {{Code2|require no access to Tor's control port}} to Tor Browser's design. Therefore when onion-grater gets asked GETINFO net/listeners/socks, it lies, and answers 250-net/listeners/socks="127.0.0.1:9150". This makes Tor Button happy and therefore it shows a "Congratulations!" (success) welcome page on its default homepage about:tor and not the failure page, which would confuse users. Since bug [https://gitlab.torproject.org/legacy/trac/-/issues/9224 TorButton about:tor fails when using additional socks listeners] has been fixed by Tor Tor Project, that lie wouldn't be necessary anymore. We're keeping it, because it is not necessary for Tor Button get a full list of all ports Tor is listening on. If an attacker compromised {{project_name_workstation_short}}, hiding that list has an advantage. The attacker can probe what ports are available to that {{project_name_workstation_short}}, but if the user added extra ports not available to the compromised {{project_name_workstation_short}} (only available to another {{project_name_workstation_short}} listening on another IP), at least those remain secret. (This is a bit theoretical, because a compromised {{project_name_workstation_short}} can spoof its LAN IP and most likely very few users are using ARP spoofing defenses. Should we add ARP spoofing defenses by default at some point, we at least don't have to worry about this point.) For further eventual Tor control port access requirements by Tor Button, the configuration file of accepted commands has to be extended. If Tor Button would ever ask for anything which violates {{project_name_short}} design ({{project_name_workstation_short}} has no way of finding out its own external IP address in particular), such as GETINFO address, for example if Tor Button wanted to ensure, that the user is not using its own external IP address, a new lie would have to be added to the onion-grater script. In case many more lies are required, lies should go into the config file as well, for now, hard coding is sufficient. ==== Circuit View ==== TODO: Make Tor Circuit View [[Tor_Browser/Advanced_Users#Tor_Circuit_View|screenshot]] with redacted IPs, fingerprints, locations work with Tor Browser in {{project_name_short}} through filtered Tor control port access (onion-grater). Purpose: do not confuse Tor Browser to fix onion authentication through Tor Browser. '''Regular Expression RegEx''' Required for onion-grater. rewrite source 
    650 STREAM 547 SUCCEEDED 210 99.84.158.73:80 SOCKS_USERNAME="--unknown--" SOCKS_PASSWORD="f0358c6d18973a9dc667f5dcd75c131e" CLIENT_PROTOCOL=SOCKS5 NYM_EPOCH=20 SESSION_GROUP=-59 ISO_FIELDS=SOCKS_USERNAME,SOCKS_PASSWORD,CLIENTADDR,SESSION_GROUP,NYM_EPOCH
to 
    650 STREAM 547 SUCCEEDED 210 127.0.0.1:443 SOCKS_USERNAME="--unknown--" SOCKS_PASSWORD="redacted" CLIENT_PROTOCOL=SOCKS5 NYM_EPOCH=00 SESSION_GROUP=-00 ISO_FIELDS=SOCKS_USERNAME,SOCKS_PASSWORD,CLIENTADDR,SESSION_GROUP,NYM_EPOCH
---- rewrite source 
    650 STREAM 487 SENTCONNECT 174 abcdefghz2352sf.onion:80 CLIENT_PROTOCOL=SOCKS5 NYM_EPOCH=19 SESSION_GROUP=-41 ISO_FIELDS=DESTPORT,DESTADDR,SOCKS_USERNAME,SOCKS_PASSWORD,CLIENTADDR,SESSION_GROUP,NYM_EPOCH
to 
    650 STREAM 487 SENTCONNECT 000 redacted.onion:80 CLIENT_PROTOCOL=SOCKS5 NYM_EPOCH=19 SESSION_GROUP=-00 ISO_FIELDS=DESTPORT,DESTADDR,SOCKS_USERNAME,SOCKS_PASSWORD,CLIENTADDR,SESSION_GROUP,NYM_EPOCH
---- rewrite source 
    81 BUILT $89FD4FB2A5FD73B50F2E5D85C4707883F8CD5130~VisitFrance,$9EB3FD84065E5622A57EFEF14E41A01B5B99A022~whatconfig,$E7EF73B4722CFAF0E8AA2151D3AA78701DE5F19B~Silverwing,$9B1BE5D20FB9069523EF4889027325CE89B42460~scha1k BUILD_FLAGS=IS_INTERNAL,NEED_CAPACITY,NEED_UPTIME PURPOSE=HS_CLIENT_REND HS_STATE=HSCR_JOINED REND_QUERY=crypty22ijtotell TIME_CREATED=2020-06-05T18:00:41.170815
to 
    81 BUILT $redacted~redacted,$redacted~redacted,$redacted~redacted,$redacted~redacted BUILD_FLAGS=NEED_CAPACITY PURPOSE=HS_CLIENT_REND HS_STATE=HSCR_JOINED REND_QUERY=redacted TIME_CREATED=2020-00-00T00:00:00.000000
---- rewrite source 
250+ns/id/24A17A4C1FA8C2108ACDE3BE681B0731384BF3A5=
r dragonhoard JKF6TB+owhCKzeO+aBsHMThL86U hrPrav+cqzyxUd0uNuXEA1U56xk 2020-06-05 09:21:43 159.69.21.196 8080 8008
a [2a01:4f8:1c1c:2e49::1]:8080
s Fast Guard Running Stable V2Dir Valid
w Bandwidth=13000
.
250 OK
to 
250+ns/id/24A17A4C1FA8C2108ACDE3BE681B0731384BF3A5=
r redacted redacted redacted 2020-00-00 00:00:00 00.00.00.00 8080 8008
a [0000:0000:0000:0000::1]:8080
s Fast Guard Running Stable V2Dir Valid
w Bandwidth=00
.
250 OK
---- Syntax example 
    HS_DESC:
      response:
        - pattern:     '650 HS_DESC CREATED (\S+) (\S+) (\S+) \S+ (.+)'
          replacement: '650 HS_DESC CREATED {} {} {} redacted {}'
* Curent state: For the Tor Circuit View to appear, it requires valid credentials replied from the SENTCONNECT [https://gitlab.torproject.org/tpo/applications/torbutton/-/blob/main/chrome/content/tor-circuit-display.js#L167 event] and if they are not the same credentials the Tor Browser sent by that tab, [https://gitlab.torproject.org/tpo/applications/torbutton/-/blob/main/chrome/content/tor-circuit-display.js#L410 the circuits will not be shown]. Tor Browser wants to get the same SOCKS_USERNAME and SOCKS_PASSWORD it send for that tab in the first place, if it doesn,t the whole tab is not shown, you can check the reply with the ''Browser Console'' (Ctrl+Shift+J): ''Torbutton NOTE: no SOCKS credentials found for current document''. The interest is not showing the circuits per se, but at least showing literally Hop1, Hop2 and Hop2 to not confuse users why the tab is not shown. Another side effect is that it also hides the button New circuit for this site, but still available in the supermenu, but less convenient and not expected by the user to be this way. If someone with JavaScript knowledge could patch upstream, at least showing the New circuit for this site even if the credentials don't match would be a start, but it currently is not shown because either the Tor Button shows completely or doesn't show at all. A more difficult task would be to instead of not showing the circuit view, if the browser fails to get the correct credentials, it could show a helpful text that it can't show the circuits because it could not determine the stream. There is a lower chance of rewriting the countries and IPs of the circuits to fake data if it can't establish the link because upstream could consider this a workaround. Anyway, a better user experience is expected from the Tor Circuit View, even if an error message, so at least it is understood by non-developers or for people not familiarized with the Tor Button and Tor Browser source code. Trials with onion-grater: 
---
- hosts:
    - '*'
  commands:
    SETEVENTS:
      - 'STREAM'
    SIGNAL:
      - 'NEWNYM'
    GETCONF:
      - pattern: 'bridge'
        response:
        - pattern: '250 Bridge.*'
          replacement: '250 Bridge'
    GETINFO:
      - 'status/circuit-established'
      - 'version'
      - 'consensus/valid-after'
      - 'consensus/valid-until'
      - 'consensus/fresh-until'
      - pattern: 'net/listeners/socks'
        response:
        - pattern:     '250-net/listeners/socks=".*"'
          replacement: '250-net/listeners/socks="127.0.0.1:9150"'
      - pattern: 'circuit-status'
        response:
        - pattern: '.*,.*'
          replacement: '10 BUILT $1~a,$2~b,$3~c BUILD_FLAGS=NEED_CAPACITY PURPOSE=redacted REND_QUERY=redacted TIME_CREATED=2020-00-00T00:00:00.000000'
      - pattern: 'ns/id/1'
        response:
        - pattern:     '551 .*'
          replacement: "250+ns/id/1=

r redacted redacted redacted 2020-00-00 00:00:00 00.00.00.00 8080 8008

a 127.0.0.1:8080

s Fast Guard Running Stable V2Dir Valid

w Bandwidth=00

."
      - pattern: 'ns/id/2'
        response:
        - pattern:     '551 .*'
          replacement: "250+ns/id/2=

r redacted redacted redacted 2020-00-00 00:00:00 00.00.00.00 8080 8008

a 127.0.0.2:8080

s Fast Guard Running Stable V2Dir Valid

w Bandwidth=00

."
      - pattern: 'ns/id/3'
        response:
        - pattern:     '551 .*'
          replacement: "250+ns/id/3=

r redacted redacted redacted 2020-00-00 00:00:00 00.00.00.00 8080 8008

a 127.0.0.3:8080

s Fast Guard Running Stable V2Dir Valid

w Bandwidth=00

."
      - pattern: 'ip-to-country/127.0.0.1'
        response:
        - pattern: '250-ip-to-country/.*'
          replacement: '250-ip-to-country/127.0.0.1=us'
      - pattern: 'ip-to-country/127.0.0.2'
        response:
        - pattern: '250-ip-to-country/.*'
          replacement: '250-ip-to-country/127.0.0.2=us'
      - pattern: 'ip-to-country/127.0.0.3'
        response:
        - pattern: '250-ip-to-country/.*'
          replacement: '250-ip-to-country/127.0.0.3=us'
  confs:
    __owningcontrollerprocess:
  events:
    STREAM:
      response:
        - pattern: '650 STREAM (\S+) SENTCONNECT .*'
          replacement: '650 STREAM 10 SENTCONNECT 10 127.0.0.1:443 SOCKS_USERNAME="unknown.org" SOCKS_PASSWORD="unknownhash" CLIENT_PROTOCOL=SOCKS5 NYM_EPOCH=0 SESSION_GROUP=-13 ISO_FIELDS=DESTPORT,DESTADDR,SOCKS_USERNAME,SOCKS_PASSWORD,CLIENTADDR,SESSION_GROUP,NYM_EPOCH'
        - pattern: '650 STREAM .*'
          replacement: '650 STREAM 10 REDACTED 10 127.0.0.1:443 SOCKS_USERNAME="unknown.org" SOCKS_PASSWORD="unknownhash" CLIENT_PROTOCOL=SOCKS5 NYM_EPOCH=0 SESSION_GROUP=-13 ISO_FIELDS=DESTPORT,DESTADDR,SOCKS_USERNAME,SOCKS_PASSWORD,CLIENTADDR,SESSION_GROUP,NYM_EPOCH'
    SIGNAL:
      suppress: true
    CONF_CHANGED:
      suppress: true
    STATUS_SERVER:
      suppress: true
    restrict-stream-events: true
Onion-grater logs: 
(filter: /usr/local/etc/onion-grater-merger.d/50_user): rewrote received event:
    650 STREAM 97825 SENTCONNECT 82470 content-signature-2.cdn.mozilla.net:443 SOCKS_USERNAME="--unknown--" SOCKS_PASSWORD="fa8348e1f2c300cad85b6052285edbe4" CLIENT_PROTOCOL=SOCKS5 NYM_EPOCH=1 SESSION_GROUP=-47 ISO_FIELDS=SOCKS_USERNAME,SOCKS_PASSWORD,CLIENTADDR,SESSION_GROUP,NYM_EPOCH
to:
    650 STREAM 10 SENTCONNECT 10 127.0.0.1:443 SOCKS_USERNAME="unknown.org" SOCKS_PASSWORD="unknownhash" CLIENT_PROTOCOL=SOCKS5 NYM_EPOCH=0 SESSION_GROUP=-13 ISO_FIELDS=DESTPORT,DESTADDR,SOCKS_USERNAME,SOCKS_PASSWORD,CLIENTADDR,SESSION_GROUP,NYM_EPOCH
(filter: /usr/local/etc/onion-grater-merger.d/50_user): -> getconf bridge
(filter: /usr/local/etc/onion-grater-merger.d/50_user): <- 250 Bridge
(filter: /usr/local/etc/onion-grater-merger.d/50_user): -> getconf bridge
(filter: /usr/local/etc/onion-grater-merger.d/50_user): <- 250 Bridge
(filter: /usr/local/etc/onion-grater-merger.d/50_user): -> getconf bridge
(filter: /usr/local/etc/onion-grater-merger.d/50_user): <- 250 Bridge
(filter: /usr/local/etc/onion-grater-merger.d/50_user): -> getinfo ns/id/1
(filter: /usr/local/etc/onion-grater-merger.d/50_user): rewrote response:
    551 Data not decodeable as hex
to:
    250+ns/id/1=
    r redacted redacted redacted 2020-00-00 00:00:00 00.00.00.00 8080 8008
    a 127.0.0.1:8080
    s Fast Guard Running Stable V2Dir Valid
    w Bandwidth=00
    .
10.138.31.44:57912 (filter: /usr/local/etc/onion-grater-merger.d/50_user): <- (multi-line)
    250+ns/id/1=
    r redacted redacted redacted 2020-00-00 00:00:00 00.00.00.00 8080 8008
    a 127.0.0.1:8080
    s Fast Guard Running Stable V2Dir Valid
    w Bandwidth=00
    .
After the first query of the relay info, it doesn't query the other nodes for unknown reasons. Note that the command getinfo ip-to-country was never issued, unknown reason. After that, only redacted streams were replied. Folder with other examples already using this type of regex: https://github.com/{{project_name_short}}/onion-grater/blob/master/usr/share/doc/onion-grater-merger/examples Direct links to profiles using similar regex: * https://github.com/Whonix/onion-grater/blob/master/usr/share/doc/onion-grater-merger/examples/40_bisq.yml * https://github.com/Whonix/onion-grater/blob/master/usr/share/doc/onion-grater-merger/examples/40_onionshare.yml = Functionality support = == Indicator for current Circuit Status and Exit IP == * The Tor Project Ticket: [https://gitlab.torproject.org/legacy/trac/-/issues/8641 Create Browser UI indication for current circuit status and exit IP ] * Screenshots: ** https://trac.torproject.org/projects/tor/raw-attachment/ticket/8641/tor_circuit_diagram_screenshot.png ** https://trac.torproject.org/projects/tor/raw-attachment/ticket/8641/mockup_torCircuitStatus.png * The Tor Project source code: https://gitlab.torproject.org/tpo/applications/torbutton/-/blob/main/chrome/content/tor-circuit-display.js * Requires SETEVENTS STREAM which we do not want because we do not wish the workstation to know its Tor entry and/or Tor middle relay. The type and variation of messages makes a catch all solution too difficult to implement. It allows way too much information about what Tor is doing to be safe https://git-tails.immerda.ch/onion-grater/commit/?id=33ae20850329930bd1a3686cca2a9ac4dc766aea +* '''EVENTS''': Tor Browser is subscribed to all '''STREAM''' events for its + implementation of the circuit view. This means it will know + basically everything Tor is doing which is pretty bad in case Tor + Browser is compromised. * [https://forums.whonix.org/t/tbb-sanitized-circuit-path-diagram-support/10284 Stalled] onion-grater profile for enabling the circuit path diagram support. 
---
- apparmor-profiles:
    - '*'
  users:
    - '*'
  hosts:
    - '*'
  commands:
    SIGNAL:
      - 'NEWNYM'
    GETINFO:
      - pattern: 'circuit-status'
        response:
        - pattern: '250(.+)circuit-status=(\S+) (\S+) (.+) (\S+) (\S+)'
        - replacement: '250+circuit-status='
    GETCONF:
      - pattern: 'bridge'
        response:
        - pattern:     '250-Bridge=(.+) (\S+) (\S+) (\S+) (\S+)'
          replacement: '250-Bridge='
        - pattern:     '250 Bridge=(.+) (\S+) (\S+) (\S+) (\S+)'
          replacement: '250 Bridge='
  events:
    STREAM:
  restrict-stream-events: true
== onion_client_auth_add == * '''Profile''': [https://github.com/Whonix/onion-grater/blob/master/usr/share/doc/onion-grater-merger/examples/40_onion_authentication.yml /usr/share/doc/onion-grater-merger/examples/40_onion_authentication.yml] Example command for Tor control protocol: {{CodeSelect|code= onion_client_auth_add m5bmcnsk64naezc26scz2xb3l3n2nd5xobsljljrpvf77tclmykn7wid x25519:uBKh6DGrkcFxB1adYuyKQltUDDUT9IZrOsne3nfHbHI= }} Test key that can be entered into browser. {{CodeSelect|code= XAJKD2BRVOI4C4IHK2OWF3EKIJNVIDBVCP2IM2Z2ZHPN456HNRZA }} == UI for ExitNode country selection in tor-launcher == https://gitlab.torproject.org/legacy/trac/-/issues/11406 == Feedback mechanism for clock-skew and other bad problems == https://gitlab.torproject.org/legacy/trac/-/issues/9675 = Debugging Inspiration = == onion-grater debug mode == On {{project_name_gateway_short}}. {{Open with root rights|filename= /lib/systemd/system/onion-grater.service.d/50_user.conf }} Add. {{CodeSelect|code= [Service] ## Clear onion-grater default file '/lib/systemd/system/onion-grater.service'. ExecStart= ## Same as above but enable debug mode. ExecStart=/usr/lib/onion-grater --listen-interface eth1 --listen-port 9051 --debug ## Same as above but enable complain mode which permits everything (insecure in production). #ExecStart=/usr/lib/onion-grater --listen-interface eth1 --listen-port 9051 --complain }} Save. Reload systemd daemon: {{CodeSelect|code= sudo systemctl daemon-reload }} Restart onion-grater service: {{CodeSelect|code= sudo systemctl restart onion-grater }} Done, debug mode is now enabled. Watch onion-grater's logs while using it in a separate terminal tab: {{CodeSelect|code= sudo journalctl -f -u onion-grater }} == onion-grater passthrough mode == This disables filtering (allows all commands) and should be used during development only. Because this mode allows everything, we will not be running as a service, but as a command on the terminal, this means that if the program receives and interrupt signal of the machine reboots, it will enable the standard onion-grater service. On {{project_name_gateway_short}}. {{CodeSelect|code= sudo systemctl stop onion-grater }} {{CodeSelect|code= sudo /usr/lib/onion-grater-merger }} {{CodeSelect|code= sudo -u onion-grater /usr/lib/onion-grater --listen-interface eth1 --listen-port 9051 --complain }} On {{project_name_workstation_short}}. {{CodeSelect|code= tor-ctrl GETINFO status/bootstrap-phase }} Press Enter. == AppArmor issues == Watch kern.log for eventual iptables log messages, run this on {{project_name_gateway_short}}. {{CodeSelect|code= sudo apparmor-info }} == Connect to onion-grater from {{project_name_gateway_short}} == To connect to onion-grater from a [[Non-Qubes-Whonix|{{non_q_project_name_short}}]]-Gateway: {{CodeSelect|code= tor-ctrl -s 10.152.152.10:9051 GETINFO config-file }} To connect to onion-grater from a [[Qubes|{{q_project_name_short}}]]-Gateway: {{CodeSelect|code= tor-ctrl -s $(qubesdb-read /qubes-ip):9051 GETINFO config-file }} == Connect to onion-grater from {{project_name_workstation_short}} == On the {{project_name_workstation_short}} you should be able to run without setting the interface: {{CodeSelect|code= tor-ctrl GETINFO address }} Type GETINFO address. Enter. Should reply 510 Command filtered. == tcpdump - Less Important == Only in case of suspected fundamental issues with low level networking. To see what's being send to onion-grater's port, run this on {{project_name_gateway_short}}. [[Non-Qubes-Whonix|{{non_q_project_name_short}}]]-Gateway: {{CodeSelect|code= sudo tcpdump -i eth1 -l -s0 -w - tcp dst port 9051 | strings }} [[Qubes|{{q_project_name_short}}]]-Gateway: (Note: replace vif18.0 with the device found with the command ip route.) {{CodeSelect|code= sudo tcpdump -i vif18.0 -l -s0 -w - tcp dst port 9051 | strings }} = Comparison of Control Port Filters = == onion-grater by Tails == [https://gitlab.tails.boum.org/tails/tails/-/blob/master/config/chroot_local-includes/usr/local/lib/onion-grater onion-grater] by Tails ([https://gitlab.tails.boum.org/tails/tails/-/tree/master/config/chroot_local-includes/etc/onion-grater.d config folder]) * Written in python3. * All dependencies already inside Debian. * Wildcard / regex support. * Supports rewriting client requests. * Supports rewriting Tor replies. * Parallel connections support. * Can reply to {{Code2|getinfo net/listeners/socks}} with the lie '250-net/listeners/socks="127.0.0.1:9150"'. * Logs to journal. * Honors signals sigterm, sigint. * Supports subscribing Tor ControlPort events ({{Code2|setevent}}).
== onion-grater by Tails forked by {{project_name_short}} == [https://github.com/{{project_name_short}}/onion-grater onion-grater] ([https://github.com/{{project_name_short}}/onion-grater/blob/master/usr/lib/onion-grater-merger config folder]) ([https://github.com/{{project_name_short}}/onion-grater/tree/master/usr/share/doc/onion-grater-merger/examples additional onion-grater-merger example profiles]) ([https://phabricator.whonix.org/project/view/72/ issue tracker]) * mostly same as [[#onion-grater by Tails]] * different config parsing mechanism ** [[Dev/onion-grater/tor-controlport-filter/config|description]] ** [[Dev/onion-grater/tor-controlport-filter/config#{{project_name_short}}_fork_commits|commits]] * Debian packaging * systemd unit file * systemd-notify support - https://mailman.boum.org/pipermail/tails-dev/2017-March/011327.html submitted patch upstream * systemd seccomp hardening https://phabricator.whonix.org/T631 * AppArmor profile
== roflcoptor by subgraph os == [https://github.com/subgraph/roflcoptor roflcoptor] by subgraph os * Many golang dependencies that are not packaged for Debian. * Event support. * Wildcard support. * TODO: expand
== surrogate.go by Yawning Angel == [https://gitweb.torproject.org/tor-browser/sandboxed-tor-browser.git/tree/src/cmd/sandboxed-tor-browser/internal/tor/surrogate.go surrogate.go (tor project gitweb)] by Yawning Angel * golang (build dep, not run dep) * AGPL (Achtung!) * Part of new/upcoming [https://gitlab.torproject.org/legacy/trac/-/wikis/doc/TorBrowser/Sandbox/Linux Sandboxed Tor Browser] * "Tor control/socks port surrogates." (quoting top comment) Intended to proxy/filter both ControlPort and SOCKSPort of tor process in one bubblewrapped sandbox, while actual Tor Browser runs in another bubblewrapped sandbox. * Observed version: [https://gitweb.torproject.org/tor-browser/sandboxed-tor-browser.git/commit/src/cmd/sandboxed-tor-browser/internal/tor/surrogate.go?id=6ff4802a9f4b76c791c2bf864b5446170474215f 6ff4802] * So, official TPO Tor Browser variant also has this problem. Oops!
== legacy control-port-filter-python by {{project_name_short}} == legacy [https://github.com/{{project_name_short}}/control-port-filter-python control-port-filter-python] ([https://github.com/{{project_name_short}}/control-port-filter-python/blob/master/usr/sbin/cpfpd main script]) [{{project_name_short}} 10 to {{project_name_short}} 13 ([https://github.com/{{project_name_short}}/control-port-filter-python/releases/tag/1.7-1 version 1.7-1]] (last git commit [https://github.com/{{project_name_short}}/control-port-filter-python/commit/99e8395fbafcb7789c7d1b5cbf7410ed65192e9a up to 99e8395fbafcb7789c7d1b5cbf7410ed65192e9a]) ([https://phabricator.whonix.org/tag/control-port-filter-python/ issue tracker]) [deprecated] * A fork of (very early) [https://gitlab.tails.boum.org/tails/tails/-/blob/master/config/chroot_local-includes/usr/local/sbin/tor-controlport-filter tor-controlport-filter] by Tails. (See above.) * Written in python2.7. * Configurable by dropping .d-style{{kicksecure_wiki |wikipage=Configuration_Files#Configuration_Drop-In_Folders |text=Style Configuration Folders }} configuration snippets into {{Code2|/etc/cpfpy.d}}. * Support to answer {{Code2|getinfo net/listeners/socks}} with the lie '250-net/listeners/socks="127.0.0.1:9150"'. * Supports logging. * Honors signals sigterm, sigint, keyboard interrupt. * Supports parallel connections. * Supports wildcards, which is for tools such as [[Next#onionshare|onionshare]] that are using add_onion *. * Injects workstation IP into add_onion. For example will transform from / to: ** add_onion new:best port=80,17600 ** add_onion new:best port=80,10.137.6.41:17600 * Support subscribing Tor ControlPort events ({{Code2|setevent}}). ({{project_name_short}} 14 and above.) https://phabricator.whonix.org/T448 * Complete systemd unit file. * Lintian clean {{Code2|/debian}} packaging folder. * Will be deprecated when {{project_name_short}} 14 get released.
== legacy2 control-port-filter by {{project_name_short}} == legacy^2 control-port-filter [up to {{project_name_short}} 9.x] [deprecated] * Supports parallel connections. * Written in bash. ([https://github.com/adrelanos/control-port-filter github]) (archive only) * [https://github.com/adrelanos/control-port-filter/blob/master/etc/controlportfilt.d/30_controlportfilt_default White lists multiple useful Tor ControlPort commands]. * Configurable using /etc/controlportfilt.d drop-in files. * Support to answer {{Code2|getinfo net/listeners/socks}} with the lie '250-net/listeners/socks="127.0.0.1:9150"'. * Supports logging. * Honors signals sigterm, sigint. * Does not support wildcards. * Does not support subscribing Tor ControlPort events ({{Code2|setevent}}). * Complete sysvinit script. * Lintian clean {{Code2|/debian}} packaging folder. * Deprecated since the release of {{project_name_short}} 10. = Old method to manage profiles = Previously manual instructions. No longer needed. onion-grater-add automates that. == Add profile == '''1.''' Create folder /usr/local/etc/onion-grater-merger.d. {{CodeSelect|code= sudo mkdir -p /usr/local/etc/onion-grater-merger.d }} '''2.''' Symlink the onion-grater profile to the onion-grater settings folder. {{CodeSelect|code= sudo ln -s 40_onion_authentication.yml /usr/local/etc/onion-grater-merger.d/ }} '''3.''' Restart onion-grater. {{CodeSelect|code= sudo service onion-grater restart }} == Remove profile == '''1.''' Remove symlink from the onion-grater profile {{CodeSelect|code= sudo unlink /usr/local/etc/onion-grater-merger.d/40_onion_authentication.yml }} '''2.''' Restart onion-grater. {{CodeSelect|code= sudo service onion-grater restart }} = {{project_name_short}} Forum Discussion = * [https://forums.whonix.org/t/new-tor-controlport-commands-wanted-by-tbb-4-5-and-above new Tor ControlPort commands wanted by TBB 4.5 and above] * https://forums.whonix.org/t/onion-grater-development/15845 = Source Code = Original upstream by Tails: * [https://gitlab.tails.boum.org/tails/tails/-/blob/master/config/chroot_local-includes/usr/local/lib/onion-grater /config/chroot_local-includes/usr/local/lib/onion-grater] * [https://gitlab.tails.boum.org/tails/tails/-/blob/master/config/chroot_local-includes/etc/onion-grater.d/onionshare.yml /config/chroot_local-includes/etc/onion-grater.d/onionshare.yml] Fork by {{project_name_short}}: * [https://github.com/Whonix/onion-grater onion-grater] * [https://github.com/Whonix/onion-grater/blob/master/usr/lib/onion-grater /usr/lib/onion-grater] * [https://github.com/Whonix/onion-grater/blob/master/usr/lib/onion-grater-merger /usr/lib/onion-grater-merger] * [https://github.com/Whonix/onion-grater/blob/master/usr/lib/systemd/system/onion-grater.service /lib/systemd/system/onion-grater.service] * [https://github.com/{{project_name_short}}/onion-grater/tree/master/usr/share/doc/onion-grater-merger/examples /usr/share/doc/onion-grater-merger/examples] {{project_name_short}} supplementary: * [https://github.com/Whonix/whonix-gw-network-conf/blob/master/usr/lib/systemd/system/onion-grater.service.d/30_cpfpy.conf /lib/systemd/system/onion-grater.service.d/30_cpfpy.conf] * [https://github.com/Whonix/qubes-whonix/blob/master/usr/lib/systemd/system/onion-grater.service.d/40_qubes.conf /lib/systemd/system/onion-grater.service.d/40_qubes.conf] * [https://github.com/Whonix/anon-gw-anonymizer-config/blob/master/etc/onion-grater-merger.d/30_whonix-default.yml /etc/onion-grater-merger.d/30_whonix-default.yml] * [https://github.com/Whonix/anon-gw-anonymizer-config/blob/master/usr/bin/onion-grater-add /usr/bin/onion-grater-add] * [https://github.com/Whonix/anon-gw-anonymizer-config/blob/master/usr/bin/onion-grater-remove /usr/bin/onion-grater-remove] * [https://github.com/Whonix/anon-gw-anonymizer-config/blob/master/usr/bin/onion-grater-list /usr/bin/onion-grater-list] = See Also = * [[Onion-grater]] * [[Tor_Controller]] * [[Dev/anon-ws-disable-stacked-tor]] * [https://gitweb.torproject.org/torspec.git?a=blob_plain;hb=HEAD;f=control-spec.txt Tor Control Protocol] = Footnotes = {{reflist|close=1}} {{Footer}} [[Category:Design]]