Skip to content

Linux Block Device Operations

Experimental

Day-2 operations specific to clusters in the Linux block device mode (lblk) are described below. Concepts and deployment are described under Linux Block Devices (lblk) and Deploy with Linux Block Devices.

Node Restarts

No lblk-specific handling is required for a node restart. On restart, the node's configured devices are re-resolved serial-first against the live host inventory. Kernel device names may have changed across a reboot (for example, /dev/sdb and /dev/sdc swapping), but the devices are still matched correctly by their persisted serial numbers, and the AIO bdevs and the storage stack above them are rebuilt exactly as recorded in the cluster database.

A configured device that is missing from the host at restart is marked removed (the same semantics as a missing NVMe controller), and the standard failed-device data migration is triggered.

Info

The --ssd-pcie option of storage-node restart, by which new devices are added during a restart, is not supported on lblk-mode clusters and is rejected.

Adding Devices to a Storage Node

Growing a node's device set is performed by reconfiguring and re-adding the node, not at restart time:

  1. Attach the new block devices to the host.
  2. Remove the storage node from the cluster. Its data is migrated to the remaining nodes, as with any node replacement.

  3. Re-run the configuration with a selection that includes the new devices:

    Reconfiguring the node with an extended device selection
    sudo sbctl storage-node configure \
   --lblk --blk-names sdb,sdc,sdd --max-lvol 50

  4. Re-add the node to the cluster. The node joins with the extended device set, and data is redistributed onto it by the automatic rebalancing.

Cluster capacity can alternatively be extended by adding a new storage node with its own devices.

Failed Devices

Device failures are handled by the same machinery as in NVMe mode. A device producing I/O errors is marked unavailable and, after the retry budget is exhausted, marked failed. The cluster map is updated, and the affected data is rebuilt from redundancy by a data migration.

A device whose I/O hangs without erroring is caught by the lblk hung-IO watchdog (roughly 30 seconds of zero progress with outstanding I/O) and driven through the same unavailable, restart, and failed path. A device that disappears from the host, through hot removal or a cloud volume detach, is detected and treated like an NVMe hot-remove.

A failed device is replaced by attaching a replacement device to the host and then following the Adding Devices procedure. Alternatively, the whole node is replaced, following Replacing a Storage Node.

Info

SMART health information is not available for AIO-backed devices. Device health checks (storage-node check-device) are limited to liveness and I/O statistics.