Weka mount checker

Find pods where /mnt/<shared-fs> is hung, stale, or missing.

When a Weka cluster has a hiccup (org change, drivers-loader restart, NIC flap) you usually end up with a subset of pods that still mount fine and another subset where the mount silently hangs. The fix is the same — restart the Weka client / re-mount — but you need to find them first.

The pattern is: run ls on the mount in every pod with a 5-second timeout, prefix each line with OK: or FAIL/HANG:, paste the result here, and the tool splits it into hangs (timeout / NFS errors) vs hard failures (auth, missing dir).

Step 1 — probe every pod

Replace <pod-prefix> and <mount-path> for your workload.

for node in $(kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}'); do
  pod=$(kubectl get pods -A --field-selector spec.nodeName=$node \
        --no-headers -o custom-columns=":metadata.namespace,:metadata.name" 2>/dev/null \
        | grep "<pod-prefix>" | head -1)
  if [ -z "$pod" ]; then continue; fi
  ns=$(echo "$pod" | awk '{print $1}'); name=$(echo "$pod" | awk '{print $2}')
  out=$(timeout 5 kubectl exec -n "$ns" "$name" -- ls <mount-path> 2>&1)
  rc=$?
  if [ $rc -eq 0 ]; then echo "OK: $name => $out"
  else echo "FAIL/HANG: $name => $out"
  fi
done

Step 2 — paste output

Paste output above. Format: OK: pod => … or FAIL/HANG: pod => ….