Consensus list contains nodes not accessible via port 443

[cgrigis]

Using the [latest files released], I still occasionally experience errors where a circuit cannot be built:

Tried to build a circuit 3 times, but all attempts failed.

A likely cause is that some nodes are accessed on a port other than 443, as some previous logs look like:

tor_chanmgr::builder: Negotiating TLS with 45.153.160.140:9001

even though the generated consensus file should filter for those.

Examining the latest consensus file, there is an entry as follows:

r Lux1 tLrugDtut1dQ1lhKJPs3vlP0510 2021-10-05 08:21:19 104.244.75.132 9001 9030
a [2605:6400:30:fa81:6547:73f4:7b51:7e2]:443
m FDzZJLSas74TnMkLPVU4K/iH2gSeICkmzCcP61otHMA
s Fast Guard Running Stable V2Dir Valid
v Tor 0.4.5.10
pr Cons=1-2 Desc=1-2 DirCache=2 FlowCtrl=1 HSDir=1-2 HSIntro=3-5 HSRend=1-2 Link=1-5 LinkAuth=1,3 Microdesc=1-2 Padding=2 Relay=1-3
w Bandwidth=51000

which uses port 9001 on its IPv4 address, and port 443 on its IPv6 address, which would explain why it went through the filtering.

We also find entries such as:

r Digitalcourage4iphb Q4Hkcek1iuTYFQITAsfy4WUCvuE 2021-10-05 16:06:13 185.220.102.247 993 8080
a [2a0b:f4c1:2::247]:993
m WD97leLNxr3KhkLxBmem5YM45+styD9zUh8FTUp8jJI
s Fast HSDir Running Stable V2Dir Valid
v Tor 0.4.5.10
pr Cons=1-2 Desc=1-2 DirCache=2 FlowCtrl=1 HSDir=1-2 HSIntro=3-5 HSRend=1-2 Link=1-5 LinkAuth=1,3 Microdesc=1-2 Padding=2 Relay=1-3
w Bandwidth=84000

and

r Digitalcourage4ipha 68VTkTH+6gBMQZhsC9A7XIW769U 2021-10-05 16:06:12 185.220.102.247 443 80
a [2a0b:f4c1:2::247]:443
m u5wzm/8CZHyPlkXuZ/Boi3yyx8jBeYarsSDk7GMrYrA
s Fast Guard HSDir Running Stable V2Dir Valid
v Tor 0.4.5.10
pr Cons=1-2 Desc=1-2 DirCache=2 FlowCtrl=1 HSDir=1-2 HSIntro=3-5 HSRend=1-2 Link=1-5 LinkAuth=1,3 Microdesc=1-2 Padding=2 Relay=1-3
w Bandwidth=91000

with the same IP but different ports. Could that confuse the circuit building process?

[laurent-girod]

The circuit building works fine, but the issue comes from the routers selection.
When selecting which routers to include in the custom consensus, I am just checking that the ones to be used as guards are reachable through port 443 to pass through strict firewalls, but I have not checked if they are using this port for an IPv4 address or an IPv6 one.
So we can still encounter some problems in a network with a strict firewall and no IPv6 support.

[ineiti]

@cgrigis - can you please check that all nodes are now OK and then close this issue?

[cgrigis]

Examining the last few files generated by lightarti-directory, the IPv4 ports used by the nodes flagged Guard are:

2021-10-01:
     39 443
      1 9001
2021-10-02:
     39 443
      1 9001
2021-10-03:
     39 443
      1 9001
2021-10-04:
     39 443
      1 9001
2021-10-05:
     40 443
2021-10-06:
     39 443
      1 9001
2021-10-19:
     40 443
2021-10-20:
     40 443
2021-10-21:
     40 443
2021-10-22:
     40 443

but that does not prove the issue is gone ;-)
Perhaps we could add a unit test to ensure those nodes (the IPv6:443) are not picked up?

[ineiti]

You mean adding a test to the generation of the arti-directory releases?

[cgrigis]

What I had in mind was a unit test of the gen_fresh_dirinfo.py tool somewhere in .../tools/tests, as there seems to be other tests in there?

[ineiti]

Yes, but if we're looking for Heisenbugs the arti-directory seems a better place, as it's called regularly.

[cgrigis]

I am not talking about heisenbugs, but about verifying that the filtering feature works on test data. Checking in arti-directory does not guarantee the code path is triggered.

[ineiti]

So we should do both ;) Test that the IPv6:443 is correctly refused, and that new arti-directorys don't create new bugs...

Volunteers?

[laurent-girod]

I can create a dummy consensus to test if the filtering works on the different cases.

[cgrigis]

@laurent-girod Did you already have a chance to work on this?

[laurent-girod]

I haven't created the dummy yet.