I am using an Odyssey mini PC with Ubuntu desktop 20.04 for the master node, and four Raspberry Pi 4 Model Bs with Ubuntu server 20.04 for the worker nodes. Please see my previous post Pi cluster (hardware) for more details.

SSH is enabled by default on the Ubuntu 20.04 server, so by default SSH is simply:

$ ssh ubuntu@<IP address>

0. IP configurations

To check internal IP address of each machine, use:

$ ip a

To check for currently used DNS server IP address

$ systemd-resolve --status | grep Current

To display default gateway IP address:

$ ip r

1. Change hostname

Default hostname of every Pi is ubuntu, which will be confusing if left unchanged. To change hostname of each Pi, use:

$ sudo hostnamectl set-hostname <new hostname>

Run $ sudo nano /etc/hosts and add in the following:

10.42.0.1 odyssey
10.42.0.138 pi0
10.42.0.139 pi1
10.42.0.140 pi2
10.42.0.141 pi3

I also had to comment out other references to odyssey.

2. Configuring static IP

Make sure that the network interface is not managed by CloudInit. Open the configuration file with $ sudo nano /etc/cloud/cloud.cfg.d/subiquity-disable-cloudinit-networking.cfg and append network: {config: disabled}.

Open the Netplan configuration file with $ sudo nano /etc/netplan/00-installer-config.yaml and replace everything with:

network:
  version: 2
  ethernets:
    eth0:
      addresses: [10.42.0.138/24]
      gateway4: 10.42.0.1
      nameservers:
        addresses: [10.42.0.1, 255.255.255.0]

Run $ sudo netplan try to check for syntax errors, by pressing ENTER to keep settings. Network configuration should be accepted if nothing goes wrong.

To make changes permanent run $ sudo netplan apply and reboot with $ sudo reboot. SSH into the Pi to confirm that a static IP has been set.

3. SSH alias

Run $ sudo nano .ssh/config on every node and insert the following to simplify the SSH process to just $ ssh <Host>:

Host odyssey
User zyf0717
Hostname 10.42.0.1

Host pi0
User ubuntu
Hostname 10.42.0.138

Host pi1
User ubuntu
Hostname 10.42.0.139

Host pi2
User ubuntu
Hostname 10.42.0.140

Host pi3
User ubuntu
Hostname 10.42.0.141

4. Setting up public / private key pairs

On every node run the following command to generate a public and private key pair (default location for is in ~/.ssh/ folder):

$ ssh-keygen -t ed25519

Each public key will need to be concatenated to the ~/.ssh/authorized_keys file on every other device. Starting from the master node, ssh into each Pi and run the following command:

$ cat ~/.ssh/id_ed25519.pub | ssh zyf0717@odyssey 'cat >> .ssh/authorized_keys'

Also insert public key of the master node into ~/.ssh/authorized_keys with:

$ cat .ssh/id_ed25519.pub >> .ssh/authorized_keys

Copy ~/.ssh/authorized_keys into every worker node with:

$ scp ~/.ssh/authorized_keys <Host>:~/.ssh/authorized_keys

SSH connections can now be established between any of the master or worker nodes with just ssh <Host>.

5. Cluster command

In the master node, define some shortcuts by adding the following lines to ~/.bashrc:

# To get hostname of all worker nodes.
function workernodes {
  grep "pi" /etc/hosts | awk '{print $2}' | grep -v $(hostname)
}

# To send command to all worker nodes.
function clustercmd {
  for pi in $(workernodes); do ssh $pi "$@"; done
}

Commands can now be run across all worker nodes, e.g., to reboot cluster, simply run:

$ clustercmd sudo reboot

To continue with setting up Hadoop, Spark and HDFS, please proceed to my next post Pi cluster (master node Hadoop and Spark).