configure networking explicitly on each node

Due to a bug in kubernetes-nmstate/nmstate/NM, we get a random MAC
address on our bridge after reboot. Because of that we get a different
IP address and lose the host.

With this patch, we explicitly request MAC of the NIC on the bridge.

Signed-off-by: Petr Horacek <[email protected]>

Author: phoracek

CNV/configure-network.sh

@@ -7,30 +7,38 @@ BRIDGE_NAME=brext
 
 export KUBECONFIG=${KUBECONFIG:-../OpenShift/ocp/auth/kubeconfig}
 
+nodes=$(oc get nodes -o jsonpath='{range .items[*]}{.metadata.name} {end}')
+
 echo "Configuring networks on nodes"
+for node in $nodes; do
+    echo "Detecting the default interface"
+    while ! default_iface=$(oc get nodenetworkstate ${node} -o jsonpath="{.status.currentState.routes.running[?(@.destination==\"${MACHINE_CIDR}\")].next-hop-interface}" | cut -d " " -f 1); do
+        sleep 10
+    done
 
-echo "Detecting the default interface"
-while ! default_iface=$(oc get nodenetworkstate ${node} -o jsonpath="{.items[0].status.currentState.routes.running[?(@.destination==\"${MACHINE_CIDR}\")].next-hop-interface}" | cut -d " " -f 1); do
-    sleep 10
-done
+    if [ "${default_iface}" == "${BRIDGE_NAME}" ]; then
+        echo "Bridge ${BRIDGE_NAME} seems to be already configured as the default interface on node ${node}, skipping the rest of network setup"
+        continue
+    fi
 
-if [ "${default_iface}" == "${BRIDGE_NAME}" ]; then
-    echo "Bridge ${BRIDGE_NAME} seems to be already configured as the default interface, skipping the rest of network setup"
-    exit 0
-fi
+    echo "Detecting MAC address of the default interface"
+    default_iface_mac=$(oc get nodenetworkstate ${node} -o jsonpath="{.status.currentState.interfaces[?(@.name==\"${default_iface}\")].mac-address}")
 
-echo "Applying node network configuration policy"
-cat <<EOF | oc apply -f -
+    echo "Applying node network configuration policy"
+    cat <<EOF | oc apply -f -
 apiVersion: nmstate.io/v1alpha1
 kind: NodeNetworkConfigurationPolicy
 metadata:
-  name: kni-policy
+  name: kni-${node}
 spec:
+  nodeSelector:
+    kubernetes.io/hostname: ${node}
   desiredState:
     interfaces:
     - name: ${BRIDGE_NAME}
       type: linux-bridge
       state: up
+      mac-address: ${default_iface_mac}
       ipv4:
         dhcp: true
         enabled: true
@@ -44,10 +52,10 @@ spec:
         port:
         - name: ${default_iface}
 EOF
+done
 
 echo "Waiting until the configuration is done, it may take up to 5 minutes until keepalived gets reconfigured"
-nodes=$(oc get nodes -o jsonpath='{range .items[*]}{.metadata.name} {end}')
 for node in $nodes; do
     until [ "$(oc get nodenetworkstate ${node} -o jsonpath="{.status.currentState.routes.running[?(@.destination==\"${MACHINE_CIDR}\")].next-hop-interface}")" == "${BRIDGE_NAME}" ]; do sleep 10; done
+    oc wait node ${node} --for condition=Ready --timeout=10m
 done
-oc wait node --all --for condition=Ready --timeout=10m

README.md

@@ -83,10 +83,9 @@ The deployment process will use scripts to perform the following on the configur
    (#5)](https://github.com/openshift-kni/install-scripts/issues/5)
    for image storage.
 1. [Deploy
-   CNV](https://github.com/openshift-kni/install-scripts/blob/master/CNV/deploy-cnv.sh). [Configure
-   a bridge on the `External` interface on OpenShift nodes
-   (#18)](https://github.com/openshift-kni/install-scripts/issues/18)
-   to allow VMs access this network.
+   CNV](https://github.com/openshift-kni/install-scripts/blob/master/CNV/deploy-cnv.sh).
+1. [Configure bridge as the default interface](https://github.com/openshift-kni/install-scripts/blob/master/CNV/configure-network.sh).
+   Please note that this configuration has to be retriggered when new nodes are added.
 1. Temporarily install Ripsaw, carry out some performance tests, and
    capture the results.