Redis nodes become unreachable.
Issue: Redis nodes are showing UNREACHABLE.
Root Cause: The main reason the nodes became unreachable between each of them is because of blocking ICMP traffic over the ports.
Resolution:
- Enable ICMP traffic
Recommendation of Port on REDIS for ICMP:
According to the Network port configurations page in the documentation, the port for ICMP is *.
Unlike the Internet Protocol (IP), ICMP is not associated with a transport layer protocol such as TCP or UDP. This makes ICMP a connectionless protocol: one device does not need to open a connection with another device before sending an ICMP message.
Normal IP traffic is sent using TCP, which means any two devices that exchange data will first carry out a TCP handshake to ensure both devices are ready to receive data. ICMP does not open a connection in this way. The ICMP protocol also does not allow for targeting a specific port on a device.
Source: What is the Internet Control Message Protocol (ICMP)?
Also, enable TCP connectivity between the nodes for 8002, 8004, 8006 ports in case the rule is missing or if it has been removed when ICMP is enabled.