Redis HAA install for Multinode setup

Help Orchestrator
, , , ,
1

Hi All,

We have a Redis HAA installed on Azure VM which runs on RHEL platform. The install was successful and the slave nodes where able to join the master. But when running the rladmin status command we found this error on the slave nodes.

image
image975×542 133 KB

We also checked if there is any issues with the port and found nothing by running the rlcheck command.

image
image1053×585 14.5 KB

image
image1053×585 13 KB

Any inputs/help to point us to the right direction would really help.
Please let me know if you have any question or clarifications on this setup or install.

Thanks
Madhan

2

Thanks for making a public post @dmadhan1980!

Could you share the details of your Installation / Steps you went through, plus anything you might have done to prepare the system? If you have the specific commands and any output that would be helpful (redacting any sensitive information)

  • Did you use the get-haa.sh script provided by UiPath? If so, which version of HAA did it download?
  • Did you run through the basic setup, or did you use advanced installation, or your own setup?

During my own time in setting up HAA about a year or so ago… biggest hurdle typically came back to Network Connectivity both on the local system and across the vlan/network between hosts. I found that if I correcting any ACLs / Firewall Rules / Routing after the fact, it already caused an issue during initial setup/install that was not obvious and typically was easier to blow Redis off the system and redo the steps.

That would be my first thoughts with this

  • Do you have any local system Firewalls (Firewalld, iptables, etc.) in place? (In my case I disabled IPTables, and enabled FirewallD, which one is enabled by default will depend on your OS and OS Version, any any policies you might have in place or configured on any images/blueprints/bootstraps)
  • Network ACLs in place for Azure between hosts? (I’m not familiar with Azure, but assume they have similar capabilities in place by default like AWS).
3

Hi All,

we have followed the steps mentioned in the document for installing the HAA Nodes.

Created a Folder names “haa” and downloaded the script into this folder using “wget https://download.uipath.com/haa/get-haa.sh” you can see the version we have used in the below image.

image
image1053×585 20.7 KB

and then made the script executable “chmod a+x get-haa.sh”
and ran the following command to install the master node "sudo ./get-haa.sh -u mduraisa@c.gc.ca -p abcd --verbose --accept-license-agreement

and then subsequently run the similar steps to install and join the slave nodes to the master.

I have attached the install steps and the results along with this.
HAA Install Results_TobeShared.docx (2.7 MB)

Thanks
Madhan

4

If you have those steps that you did that would be ideally to validate it. Many a times, someone has said the steps of instructions were followed only to discovery a step was missed and/or included a type, etc. (Myself included).

It’s preferable to provide text where possible, instead of images. Images of text (especially when combing through logs) makes things more difficult for others.

I would remove / replace the docx you provided, whether it is the password you ended up using or not, you have a unblurred out password, but have blurred out passwords and keys in other parts of the document, so I thought I would mention it. A simple Text file should suffice and/or small snippets of the commands/logs inserted inline of the post in code blocks like I have done further down in this post.

  • Not sure if this is relevant to your challenge, but you have one error right off the bat with “Port 25224 is occupied.” Which looks to fall in the range of the Shards Port Range
  • Have an error about Firewalld on your secondary nodes

This leads me to my same suggestion that you should check your local firewalls (ensure you don’t have both IPTables and FirewallD running) and that you have the appropriate Network ACLs setup in Azure for the nodes to communicate with each other. You may want to run a port scan or similar to validate.

When I went through the motions originally it was with HAA 2.0.1 I still had troubles with the script, and end up manually managing the local firewalls ahead of running the installation script. Similar steps to what I took can be seen here in one of my older posts, if you choose to try it, validate that the ports are still accurate and the configuration is right for you.

Reviewing the install.sh script that is downloaded for 2022.4.0 it is mostly the same with a few additions such as Sentinel and Redis TLS by default

MIN_ENDPOINTS_PORT_RANGE=10000  # Used for exposing databases externally (10000-19999)
MAX_SHARDS_PORT_RANGE=29999     # Used for internal communications with database shards (20000-29999)
# See the user manual for a list of ports used by High Availability Add-on.
PORT_CNM_HTTPS=9443
PORT_SENTINEL=8001
PORT_CCS_REDIS_TLS=3342
PORT_CCS_REDIS_PLAINTEXT=36379
PORT_CLUSTER_WD=3334
PORT_NODE_WD=3335
PORT_RESOURCE_MGR=3336
PORT_INFO_PROVIDER=3339
PORT_CM=8444
PORT_METRICS_EXPORTER=8071
REQUIRED_PORTS=(
    53
    3333
    $PORT_CLUSTER_WD
    $PORT_NODE_WD
    $PORT_RESOURCE_MGR
    $PORT_INFO_PROVIDER
    3340
    3341
    3343
    3344
    $PORT_SENTINEL
    8070
    $PORT_METRICS_EXPORTER
    8080
    8443
    $PORT_CM
    9080
    9081
    $PORT_CNM_HTTPS
    $PORT_CCS_REDIS_TLS
    $PORT_CCS_REDIS_PLAINTEXT
    36380
)
...

setup_firewall() {
    if [ -x /bin/firewall-cmd ]; then
        firewall=firewalld
    elif [ -x /usr/sbin/lokkit ]; then
        firewall=system-config-firewall
        # Make sure lokkit does not start iptables if it's not loaded
        /sbin/iptables -L --line-numbers | grep -q '^[0-9]'
        if [ $? != 0 ]; then
            lokkit_args="--nostart"
        fi
    fi

    if [ -n "$firewall" ]; then
        if confirm "We detected that firewalld is installed. To use High Availability Add-on with a local\nfirewall we must open required network ports. Do you want to use the firewall and\nopen the ports used by High Availability Add-on in the default firewall zone?" "firewall"; then
            if [ "$firewall" = "firewalld" ]; then
                execute /bin/firewall-cmd --add-service=redislabs || print warning "Firewall configuration failed"
                execute /bin/firewall-cmd --add-service=redislabs --permanent || print warning "Firewall configuration failed"
            elif [ "$firewall" = "system-config-firewall" ]; then
                execute /usr/sbin/lokkit ${lokkit_args} \
                    --port=3333-3340:tcp \
                    --port=3344:tcp \
                    --port=8080:tcp \
                    --port=8443:tcp \
                    --port=${PORT_CNM_HTTPS}:tcp \
                    --port=5353:udp \
                    --port=53:udp \
                    --port=10000-19999:tcp \
                    --port=20000-29999:tcp \
                    --port=${PORT_CCS_REDIS_TLS}:tcp \
                    --port=${PORT_CCS_REDIS_PLAINTEXT}:tcp \
                    --port=${PORT_SENTINEL}:tcp
            fi
        fi
    fi
}

I’m not familiar with lokkit command used for iptables. but looking at the firewalld commands they are using, I don’t see a reload issued for the changes to take affect (firewall-cmd --reload).

If you are indeed using Firewalld, but haven’t explicitly disabled iptables (if it was enabled) the two would conflict, and by turning on Firewalld iptables would be stopped, but not disabled and end up turning on again after a reboot.