From 6afb78f9535e646c3d7b1b59f22935e91c57ca39 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Tue, 7 Dec 2021 17:43:48 +0100 Subject: [PATCH 01/48] multipathd.service: remove LimitCORE=infinity directive This would normally be set by general system policy. According to systemd-system.conf(5), it is set to infinity by default anyway in systemd. If a user changes this in systemd-system.conf, the setting should also apply to multipathd. Users who don't wont this can still use drop-ins to change the specific settings. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- multipathd/multipathd.service | 1 - 1 file changed, 1 deletion(-) diff --git a/multipathd/multipathd.service b/multipathd/multipathd.service index 1919b380c..6d09768a8 100644 --- a/multipathd/multipathd.service +++ b/multipathd/multipathd.service @@ -14,7 +14,6 @@ ConditionVirtualization=!container [Service] Type=notify NotifyAccess=main -LimitCORE=infinity ExecStartPre=-/sbin/modprobe -a scsi_dh_alua scsi_dh_emc scsi_dh_rdac dm-multipath ExecStart=/sbin/multipathd -d -s ExecReload=/sbin/multipathd reconfigure From e90c387e044c264bef3ee9b438f4343e34e14973 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Tue, 7 Dec 2021 17:55:19 +0100 Subject: [PATCH 02/48] multipathd.service: don't load SCSI device handler modules The kernel has had an autoloading mechanism for SCSI device handlers since v4.3. No need to force loading them here again. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- multipathd/multipathd.service | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/multipathd/multipathd.service b/multipathd/multipathd.service index 6d09768a8..5f2d637e8 100644 --- a/multipathd/multipathd.service +++ b/multipathd/multipathd.service @@ -14,7 +14,7 @@ ConditionVirtualization=!container [Service] Type=notify NotifyAccess=main -ExecStartPre=-/sbin/modprobe -a scsi_dh_alua scsi_dh_emc scsi_dh_rdac dm-multipath +ExecStartPre=-/sbin/modprobe dm-multipath ExecStart=/sbin/multipathd -d -s ExecReload=/sbin/multipathd reconfigure TasksMax=infinity From 11f044067974ae12c097aa48d27a46cbad6ce934 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Tue, 7 Dec 2021 17:56:14 +0100 Subject: [PATCH 03/48] multipathd.service: add dependency on initrd-cleanup.service This is necessary in the initramfs. With this added, almost no differences between our shipped multipathd.service and the one shipped in dracut remain. The rest should be fixed in dracut. It would be nice to be able to use the original service file from multipath-tools in dracut, in particular for future changes. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- multipathd/multipathd.service | 2 ++ 1 file changed, 2 insertions(+) diff --git a/multipathd/multipathd.service b/multipathd/multipathd.service index 5f2d637e8..530f012ab 100644 --- a/multipathd/multipathd.service +++ b/multipathd/multipathd.service @@ -5,8 +5,10 @@ Before=local-fs-pre.target blk-availability.service shutdown.target Wants=systemd-udevd-kernel.socket After=systemd-udevd-kernel.socket After=multipathd.socket systemd-remount-fs.service +Before=initrd-cleanup.service DefaultDependencies=no Conflicts=shutdown.target +Conflicts=initrd-cleanup.service ConditionKernelCommandLine=!nompath ConditionKernelCommandLine=!multipath=off ConditionVirtualization=!container From 7b251a0b791045a88f12d79c8e528bf8cb598f70 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Tue, 7 Dec 2021 20:49:30 +0100 Subject: [PATCH 04/48] multipathd.service: drop dependencies on iscsi and iscsid We've had "Before=" dependencies on iscsi.service and iscsid.service since this unit was first created. I'm not sure if we ever needed them. Since c9689b6 ("multipathd: Remove dependency on systemd-udev-settle.service"), we definitely don't. iscsi and iscsid sort themselves after network.target, and multipathd does not, thus these Before= dependencies are redundant. Note: Before c9689b6, iSCSI was actually treated differently than other transports - we forced multipathd to be started after FC device detection, but before iSCSI device detection. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- multipathd/multipathd.service | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/multipathd/multipathd.service b/multipathd/multipathd.service index 530f012ab..ae98034c6 100644 --- a/multipathd/multipathd.service +++ b/multipathd/multipathd.service @@ -1,6 +1,6 @@ [Unit] Description=Device-Mapper Multipath Device Controller -Before=iscsi.service iscsid.service lvm2-activation-early.service +Before=lvm2-activation-early.service Before=local-fs-pre.target blk-availability.service shutdown.target Wants=systemd-udevd-kernel.socket After=systemd-udevd-kernel.socket From cc18bab3453532adcfe0eebeef8b044404ea93a1 Mon Sep 17 00:00:00 2001 From: Benjamin Marzinski Date: Wed, 15 Dec 2021 11:44:08 -0600 Subject: [PATCH 05/48] libmultipath: embed dm_info in multipath structure Instead of having multipath allocate its dm_info structure, it should just embed it to save on the extra allocations. This also lets us have only one function that all callers use to fill a dm_info structure. Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- libmultipath/configure.c | 7 +++-- libmultipath/devmapper.c | 51 +++++++++++-------------------- libmultipath/devmapper.h | 3 +- libmultipath/libmultipath.version | 1 + libmultipath/print.c | 12 +++++--- libmultipath/structs.c | 10 ++---- libmultipath/structs.h | 3 +- multipathd/main.c | 2 +- 8 files changed, 37 insertions(+), 52 deletions(-) diff --git a/libmultipath/configure.c b/libmultipath/configure.c index 67459769c..ec38f325d 100644 --- a/libmultipath/configure.c +++ b/libmultipath/configure.c @@ -450,12 +450,12 @@ get_udev_for_mpp(const struct multipath *mpp) dev_t devnum; struct udev_device *udd; - if (!mpp || !mpp->dmi) { + if (!mpp || !has_dm_info(mpp)) { condlog(1, "%s called with empty mpp", __func__); return NULL; } - devnum = makedev(mpp->dmi->major, mpp->dmi->minor); + devnum = makedev(mpp->dmi.major, mpp->dmi.minor); udd = udev_device_new_from_devnum(udev, 'b', devnum); if (!udd) { condlog(1, "failed to get udev device for %s", mpp->alias); @@ -574,7 +574,8 @@ sysfs_set_max_sectors_kb(struct multipath *mpp, int is_reload) return 0; max_sectors_kb = mpp->max_sectors_kb; if (is_reload) { - if (!mpp->dmi && dm_get_info(mpp->alias, &mpp->dmi) != 0) { + if (!has_dm_info(mpp) && + dm_get_info(mpp->alias, &mpp->dmi) != 0) { condlog(1, "failed to get dm info for %s", mpp->alias); return 1; } diff --git a/libmultipath/devmapper.c b/libmultipath/devmapper.c index c0eb33510..36038150e 100644 --- a/libmultipath/devmapper.c +++ b/libmultipath/devmapper.c @@ -611,12 +611,21 @@ int dm_addmap_reload(struct multipath *mpp, char *params, int flush) return 0; } -static int -do_get_info(const char *name, struct dm_info *info) +bool +has_dm_info(const struct multipath *mpp) +{ + return (mpp && mpp->dmi.exists != 0); +} + +int +dm_get_info(const char *name, struct dm_info *info) { int r = -1; struct dm_task *dmt; + if (!name || !info) + return r; + if (!(dmt = libmp_dm_task_create(DM_DEVICE_INFO))) return r; @@ -646,7 +655,7 @@ int dm_map_present(const char * str) { struct dm_info info; - return (do_get_info(str, &info) == 0); + return (dm_get_info(str, &info) == 0); } int dm_get_map(const char *name, unsigned long long *size, char **outparams) @@ -969,7 +978,7 @@ dm_dev_t (const char * mapname, char * dev_t, int len) { struct dm_info info; - if (do_get_info(mapname, &info) != 0) + if (dm_get_info(mapname, &info) != 0) return 1; if (snprintf(dev_t, len, "%i:%i", info.major, info.minor) > len) @@ -1013,7 +1022,7 @@ dm_get_major_minor(const char *name, int *major, int *minor) { struct dm_info info; - if (do_get_info(name, &info) != 0) + if (dm_get_info(name, &info) != 0) return -1; *major = info.major; @@ -1367,7 +1376,7 @@ dm_geteventnr (const char *name) { struct dm_info info; - if (do_get_info(name, &info) != 0) + if (dm_get_info(name, &info) != 0) return -1; return info.event_nr; @@ -1378,7 +1387,7 @@ dm_is_suspended(const char *name) { struct dm_info info; - if (do_get_info(name, &info) != 0) + if (dm_get_info(name, &info) != 0) return -1; return info.suspended; @@ -1542,7 +1551,7 @@ dm_get_deferred_remove (const char * mapname) { struct dm_info info; - if (do_get_info(mapname, &info) != 0) + if (dm_get_info(mapname, &info) != 0) return -1; return info.deferred_remove; @@ -1583,32 +1592,6 @@ dm_cancel_deferred_remove (struct multipath *mpp __attribute__((unused))) #endif -static struct dm_info * -alloc_dminfo (void) -{ - return calloc(1, sizeof(struct dm_info)); -} - -int -dm_get_info (const char * mapname, struct dm_info ** dmi) -{ - if (!mapname) - return 1; - - if (!*dmi) - *dmi = alloc_dminfo(); - - if (!*dmi) - return 1; - - if (do_get_info(mapname, *dmi) != 0) { - free(*dmi); - *dmi = NULL; - return 1; - } - return 0; -} - struct rename_data { const char *old; char *new; diff --git a/libmultipath/devmapper.h b/libmultipath/devmapper.h index 45a676de2..703f3bf85 100644 --- a/libmultipath/devmapper.h +++ b/libmultipath/devmapper.h @@ -70,7 +70,8 @@ char * dm_mapname(int major, int minor); int dm_remove_partmaps (const char * mapname, int need_sync, int deferred_remove); int dm_get_uuid(const char *name, char *uuid, int uuid_len); -int dm_get_info (const char * mapname, struct dm_info ** dmi); +bool has_dm_info(const struct multipath *mpp); +int dm_get_info (const char * mapname, struct dm_info *dmi); int dm_rename (const char * old, char * new, char * delim, int skip_kpartx); int dm_reassign(const char * mapname); int dm_reassign_table(const char *name, char *old, char *new); diff --git a/libmultipath/libmultipath.version b/libmultipath/libmultipath.version index 9c7ffa718..7cdce9fc0 100644 --- a/libmultipath/libmultipath.version +++ b/libmultipath/libmultipath.version @@ -121,6 +121,7 @@ global: get_used_hwes; get_vpd_sgio; group_by_prio; + has_dm_info; init_checkers; init_config; init_foreign; diff --git a/libmultipath/print.c b/libmultipath/print.c index 221b515f2..c93fffd4d 100644 --- a/libmultipath/print.c +++ b/libmultipath/print.c @@ -171,8 +171,8 @@ snprint_name (struct strbuf *buff, const struct multipath * mpp) static int snprint_sysfs (struct strbuf *buff, const struct multipath * mpp) { - if (mpp->dmi) - return print_strbuf(buff, "dm-%i", mpp->dmi->minor); + if (has_dm_info(mpp)) + return print_strbuf(buff, "dm-%i", mpp->dmi.minor); else return append_strbuf_str(buff, "undef"); } @@ -180,9 +180,9 @@ snprint_sysfs (struct strbuf *buff, const struct multipath * mpp) static int snprint_ro (struct strbuf *buff, const struct multipath * mpp) { - if (!mpp->dmi) + if (!has_dm_info(mpp)) return append_strbuf_str(buff, "undef"); - if (mpp->dmi->read_only) + if (mpp->dmi.read_only) return append_strbuf_str(buff, "ro"); else return append_strbuf_str(buff, "rw"); @@ -256,7 +256,9 @@ snprint_nb_paths (struct strbuf *buff, const struct multipath * mpp) static int snprint_dm_map_state (struct strbuf *buff, const struct multipath * mpp) { - if (mpp->dmi && mpp->dmi->suspended) + if (!has_dm_info(mpp)) + return append_strbuf_str(buff, "undef"); + else if (mpp->dmi.suspended) return append_strbuf_str(buff, "suspend"); else return append_strbuf_str(buff, "active"); diff --git a/libmultipath/structs.c b/libmultipath/structs.c index d1b8aa33d..17f4baf6d 100644 --- a/libmultipath/structs.c +++ b/libmultipath/structs.c @@ -18,6 +18,7 @@ #include "prio.h" #include "prioritizers/alua_spc3.h" #include "dm-generic.h" +#include "devmapper.h" struct adapter_group * alloc_adaptergroup(void) @@ -278,11 +279,6 @@ free_multipath (struct multipath * mpp, enum free_path_mode free_paths) mpp->alias = NULL; } - if (mpp->dmi) { - free(mpp->dmi); - mpp->dmi = NULL; - } - if (!free_paths && mpp->pg) { struct pathgroup *pgp; struct path *pp; @@ -407,10 +403,10 @@ find_mp_by_minor (const struct _vector *mpvec, unsigned int minor) return NULL; vector_foreach_slot (mpvec, mpp, i) { - if (!mpp->dmi) + if (!has_dm_info(mpp)) continue; - if (mpp->dmi->minor == minor) + if (mpp->dmi.minor == minor) return mpp; } return NULL; diff --git a/libmultipath/structs.h b/libmultipath/structs.h index c0f8929c2..7f621941c 100644 --- a/libmultipath/structs.h +++ b/libmultipath/structs.h @@ -4,6 +4,7 @@ #include #include #include +#include #include "prio.h" #include "byteorder.h" @@ -386,7 +387,7 @@ struct multipath { unsigned long long size; vector paths; vector pg; - struct dm_info * dmi; + struct dm_info dmi; /* configlet pointers */ char * alias; diff --git a/multipathd/main.c b/multipathd/main.c index 7a57a7980..fe40f6e35 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -908,7 +908,7 @@ ev_remove_map (char * devname, char * alias, int minor, struct vectors * vecs) } if (strcmp(mpp->alias, alias)) { condlog(2, "%s: minor number mismatch (map %d, event %d)", - mpp->alias, mpp->dmi->minor, minor); + mpp->alias, mpp->dmi.minor, minor); return 1; } return flush_map(mpp, vecs, 0); From 67c86adbcb043d1abaee9b290444075f13fda320 Mon Sep 17 00:00:00 2001 From: Benjamin Marzinski Date: Wed, 15 Dec 2021 11:44:09 -0600 Subject: [PATCH 06/48] multipathd: update dm_info on multipath change events When multipathd gets a change event for a multipath device, the dm info may have changed, so update it. Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- multipathd/main.c | 1 + 1 file changed, 1 insertion(+) diff --git a/multipathd/main.c b/multipathd/main.c index fe40f6e35..2e99a2667 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -818,6 +818,7 @@ ev_add_map (char * dev, const char * alias, struct vectors * vecs) conf = get_multipath_config(); reassign_maps = conf->reassign_maps; put_multipath_config(conf); + dm_get_info(mpp->alias, &mpp->dmi); if (mpp->wait_for_udev) { mpp->wait_for_udev = 0; if (get_delayed_reconfig() && From 6e40bf92486f82b4c58c1bfecf815e3410892d06 Mon Sep 17 00:00:00 2001 From: Benjamin Marzinski Date: Wed, 15 Dec 2021 11:44:10 -0600 Subject: [PATCH 07/48] multipathd: avoid unnecessary path read-only reloads A mulitpath device can only be reloaded read/write when all paths are read/write. Also, whenever a read-only scsi device is rescanned, the scsi subsystem will first unconditionally issue a uevent with DISK_RO=0 before checking the read-only status, and if it the device is still read-only, issuing another uevent with DISK_RO=1. These uevents cause pointless reloads when read-only paths are rescanned. To avoid this, first check if the path is being changed to the existing multipath read-only state. If the state is the same, do nothing. If it's different, check to see if all paths are read/write before changing a multipath device from read-only to read/write. If the multipath device read-only state is unknown, assume that it needs to be reloaded. Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- multipathd/main.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 47 insertions(+), 1 deletion(-) diff --git a/multipathd/main.c b/multipathd/main.c index 2e99a2667..009a99d9d 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -1440,6 +1440,52 @@ finish_path_init(struct path *pp, struct vectors * vecs) return -1; } +static int +sysfs_get_ro (struct path *pp) +{ + int ro; + char buff[3]; /* Either "0\n\0" or "1\n\0" */ + + if (!pp->udev) + return -1; + + if (sysfs_attr_get_value(pp->udev, "ro", buff, sizeof(buff)) <= 0) { + condlog(3, "%s: Cannot read ro attribute in sysfs", pp->dev); + return -1; + } + + if (sscanf(buff, "%d\n", &ro) != 1 || ro < 0 || ro > 1) { + condlog(3, "%s: Cannot parse ro attribute", pp->dev); + return -1; + } + + return ro; +} + +static bool +needs_ro_update(struct multipath *mpp, int ro) +{ + struct pathgroup * pgp; + struct path * pp; + unsigned int i, j; + + if (!mpp || ro < 0) + return false; + if (!has_dm_info(mpp)) + return true; + if (mpp->dmi.read_only == ro) + return false; + if (ro == 1) + return true; + vector_foreach_slot (mpp->pg, pgp, i) { + vector_foreach_slot (pgp->paths, pp, j) { + if (sysfs_get_ro(pp) == 1) + return false; + } + } + return true; +} + static int uev_update_path (struct uevent *uev, struct vectors * vecs) { @@ -1512,7 +1558,7 @@ uev_update_path (struct uevent *uev, struct vectors * vecs) } ro = uevent_get_disk_ro(uev); - if (mpp && ro >= 0) { + if (needs_ro_update(mpp, ro)) { condlog(2, "%s: update path write_protect to '%d' (uevent)", uev->kernel, ro); if (mpp->wait_for_udev) From 54e7ed696b47b847c449b90f4c78f10e4f749b41 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Wed, 15 Dec 2021 23:16:38 +0100 Subject: [PATCH 08/48] libmpathpersist: split public and internal API The libmpathpersist library exports symbols that are not part of the public API. Move the respective code into a separate source file. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- libmpathpersist/Makefile | 2 +- libmpathpersist/libmpathpersist.version | 27 +- libmpathpersist/mpath_persist.c | 776 +---------------------- libmpathpersist/mpath_persist_int.c | 779 ++++++++++++++++++++++++ libmpathpersist/mpath_persist_int.h | 17 + libmpathpersist/mpathpr.h | 12 - mpathpersist/main.c | 1 + multipathd/main.c | 1 + 8 files changed, 818 insertions(+), 797 deletions(-) create mode 100644 libmpathpersist/mpath_persist_int.c create mode 100644 libmpathpersist/mpath_persist_int.h diff --git a/libmpathpersist/Makefile b/libmpathpersist/Makefile index 1e6399d61..fbc401e39 100644 --- a/libmpathpersist/Makefile +++ b/libmpathpersist/Makefile @@ -10,7 +10,7 @@ LDFLAGS += -L$(multipathdir) -L$(mpathcmddir) LIBDEPS += -lmultipath -lmpathcmd -ldevmapper -lpthread -ldl -OBJS = mpath_persist.o mpath_updatepr.o mpath_pr_ioctl.o +OBJS = mpath_persist.o mpath_updatepr.o mpath_pr_ioctl.o mpath_persist_int.o all: $(DEVLIB) man diff --git a/libmpathpersist/libmpathpersist.version b/libmpathpersist/libmpathpersist.version index fa312f6b2..a8c6aae77 100644 --- a/libmpathpersist/libmpathpersist.version +++ b/libmpathpersist/libmpathpersist.version @@ -10,27 +10,28 @@ * * See libmultipath.version for general policy about version numbers. */ -LIBMPATHPERSIST_2.0.0 { +LIBMPATHPERSIST_2.1.0 { global: - - __mpath_persistent_reserve_in; - __mpath_persistent_reserve_out; - dumpHex; - mpath_alloc_prin_response; + /* public API as defined in mpath_persist.h */ + libmpathpersist_exit; + libmpathpersist_init; mpath_lib_exit; mpath_lib_init; mpath_mx_alloc_len; + mpath_persistent_reserve_free_vecs; + __mpath_persistent_reserve_in; mpath_persistent_reserve_in; mpath_persistent_reserve_init_vecs; + __mpath_persistent_reserve_out; mpath_persistent_reserve_out; - mpath_persistent_reserve_free_vecs; +local: *; +}; + +__LIBMPATHPERSIST_INT_1.0.0 { + /* Internal use by multipath-tools */ + dumpHex; + mpath_alloc_prin_response; prin_do_scsi_ioctl; prout_do_scsi_ioctl; update_map_pr; - - /* added in 1.1.0 */ - libmpathpersist_init; - libmpathpersist_exit; - -local: *; }; diff --git a/libmpathpersist/mpath_persist.c b/libmpathpersist/mpath_persist.c index 3097c8139..6cfcddee5 100644 --- a/libmpathpersist/mpath_persist.c +++ b/libmpathpersist/mpath_persist.c @@ -1,39 +1,13 @@ #include -#include "defaults.h" -#include -#include -#include -#include "vector.h" -#include "checkers.h" -#include "structs.h" -#include "structs_vec.h" -#include -#include "prio.h" -#include -#include "devmapper.h" -#include "debug.h" -#include "config.h" -#include "switchgroup.h" -#include "discovery.h" -#include "configure.h" -#include "dmparser.h" -#include -#include "propsel.h" #include "util.h" -#include "unaligned.h" +#include "vector.h" +#include "config.h" +#include "debug.h" +#include "devmapper.h" #include "mpath_persist.h" -#include "mpathpr.h" -#include "mpath_pr_ioctl.h" - -#include -#include -#include -#include -#include - -#define __STDC_FORMAT_MACROS 1 +#include "mpath_persist_int.h" extern struct udev *udev; @@ -97,42 +71,6 @@ int libmpathpersist_exit(void) return 0; } -int -mpath_prin_activepath (struct multipath *mpp, int rq_servact, - struct prin_resp * resp, int noisy) -{ - int i,j, ret = MPATH_PR_DMMP_ERROR; - struct pathgroup *pgp = NULL; - struct path *pp = NULL; - - vector_foreach_slot (mpp->pg, pgp, j){ - vector_foreach_slot (pgp->paths, pp, i){ - if (!((pp->state == PATH_UP) || - (pp->state == PATH_GHOST))){ - condlog(2, "%s: %s not available. Skip.", - mpp->wwid, pp->dev); - condlog(3, "%s: status = %d.", - mpp->wwid, pp->state); - continue; - } - - condlog(3, "%s: sending pr in command to %s ", - mpp->wwid, pp->dev); - ret = mpath_send_prin_activepath(pp->dev, rq_servact, - resp, noisy); - switch(ret) - { - case MPATH_PR_SUCCESS: - case MPATH_PR_SENSE_INVALID_OP: - return ret; - default: - continue; - } - } - } - return ret; -} - static vector curmp; static vector pathvec; @@ -182,83 +120,6 @@ int mpath_persistent_reserve_init_vecs(int verbose) return __mpath_persistent_reserve_init_vecs(&curmp, &pathvec, verbose); } -static int mpath_get_map(vector curmp, vector pathvec, int fd, char **palias, - struct multipath **pmpp) -{ - int ret = MPATH_PR_DMMP_ERROR; - struct stat info; - int major, minor; - char *alias; - struct multipath *mpp; - - if (fstat(fd, &info) != 0){ - condlog(0, "stat error fd=%d", fd); - return MPATH_PR_FILE_ERROR; - } - if(!S_ISBLK(info.st_mode)){ - condlog(3, "Failed to get major:minor. fd=%d", fd); - return MPATH_PR_FILE_ERROR; - } - - major = major(info.st_rdev); - minor = minor(info.st_rdev); - condlog(4, "Device %d:%d", major, minor); - - /* get alias from major:minor*/ - alias = dm_mapname(major, minor); - if (!alias){ - condlog(0, "%d:%d failed to get device alias.", major, minor); - return MPATH_PR_DMMP_ERROR; - } - - condlog(3, "alias = %s", alias); - - if (dm_map_present(alias) && dm_is_mpath(alias) != 1){ - condlog(3, "%s: not a multipath device.", alias); - goto out; - } - - /* get info of all paths from the dm device */ - if (get_mpvec(curmp, pathvec, alias)){ - condlog(0, "%s: failed to get device info.", alias); - goto out; - } - - mpp = find_mp_by_alias(curmp, alias); - - if (!mpp) { - condlog(0, "%s: devmap not registered.", alias); - goto out; - } - - ret = MPATH_PR_SUCCESS; - if (pmpp) - *pmpp = mpp; - if (palias) { - *palias = alias; - alias = NULL; - } -out: - free(alias); - return ret; -} - -static int do_mpath_persistent_reserve_in (vector curmp, vector pathvec, - int fd, int rq_servact, struct prin_resp *resp, int noisy) -{ - struct multipath *mpp; - int ret; - - ret = mpath_get_map(curmp, pathvec, fd, NULL, &mpp); - if (ret != MPATH_PR_SUCCESS) - return ret; - - ret = mpath_prin_activepath(mpp, rq_servact, resp, noisy); - - return ret; -} - - int __mpath_persistent_reserve_in (int fd, int rq_servact, struct prin_resp *resp, int noisy) { @@ -266,86 +127,6 @@ int __mpath_persistent_reserve_in (int fd, int rq_servact, resp, noisy); } -static int do_mpath_persistent_reserve_out(vector curmp, vector pathvec, int fd, - int rq_servact, int rq_scope, unsigned int rq_type, - struct prout_param_descriptor *paramp, int noisy) -{ - struct multipath *mpp; - char *alias; - int ret; - uint64_t prkey; - struct config *conf; - - ret = mpath_get_map(curmp, pathvec, fd, &alias, &mpp); - if (ret != MPATH_PR_SUCCESS) - return ret; - - conf = get_multipath_config(); - select_reservation_key(conf, mpp); - select_all_tg_pt(conf, mpp); - put_multipath_config(conf); - - memcpy(&prkey, paramp->sa_key, 8); - if (mpp->prkey_source == PRKEY_SOURCE_FILE && prkey && - (rq_servact == MPATH_PROUT_REG_IGN_SA || - (rq_servact == MPATH_PROUT_REG_SA && - (!get_be64(mpp->reservation_key) || - memcmp(paramp->key, &mpp->reservation_key, 8) == 0)))) { - memcpy(&mpp->reservation_key, paramp->sa_key, 8); - if (update_prkey_flags(alias, get_be64(mpp->reservation_key), - paramp->sa_flags)) { - condlog(0, "%s: failed to set prkey for multipathd.", - alias); - ret = MPATH_PR_DMMP_ERROR; - goto out1; - } - } - - if (memcmp(paramp->key, &mpp->reservation_key, 8) && - memcmp(paramp->sa_key, &mpp->reservation_key, 8) && - (prkey || rq_servact != MPATH_PROUT_REG_IGN_SA)) { - condlog(0, "%s: configured reservation key doesn't match: 0x%" PRIx64, alias, get_be64(mpp->reservation_key)); - ret = MPATH_PR_SYNTAX_ERROR; - goto out1; - } - - switch(rq_servact) - { - case MPATH_PROUT_REG_SA: - case MPATH_PROUT_REG_IGN_SA: - ret= mpath_prout_reg(mpp, rq_servact, rq_scope, rq_type, paramp, noisy); - break; - case MPATH_PROUT_RES_SA : - case MPATH_PROUT_PREE_SA : - case MPATH_PROUT_PREE_AB_SA : - case MPATH_PROUT_CLEAR_SA: - ret = mpath_prout_common(mpp, rq_servact, rq_scope, rq_type, paramp, noisy); - break; - case MPATH_PROUT_REL_SA: - ret = mpath_prout_rel(mpp, rq_servact, rq_scope, rq_type, paramp, noisy); - break; - default: - ret = MPATH_PR_OTHER; - goto out1; - } - - if ((ret == MPATH_PR_SUCCESS) && ((rq_servact == MPATH_PROUT_REG_SA) || - (rq_servact == MPATH_PROUT_REG_IGN_SA))) - { - if (prkey == 0) { - update_prflag(alias, 0); - update_prkey(alias, 0); - } else - update_prflag(alias, 1); - } else if ((ret == MPATH_PR_SUCCESS) && (rq_servact == MPATH_PROUT_CLEAR_SA)) { - update_prflag(alias, 0); - update_prkey(alias, 0); - } -out1: - free(alias); - return ret; -} - int __mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope, unsigned int rq_type, struct prout_param_descriptor *paramp, int noisy) @@ -384,550 +165,3 @@ int mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope, __mpath_persistent_reserve_free_vecs(curmp, pathvec); return ret; } - -int -get_mpvec (vector curmp, vector pathvec, char * refwwid) -{ - int i; - struct multipath *mpp; - - vector_foreach_slot (curmp, mpp, i){ - /* - * discard out of scope maps - */ - if (!mpp->alias) { - condlog(0, "%s: map with empty alias!", __func__); - continue; - } - - if (mpp->pg != NULL) - /* Already seen this one */ - continue; - - if (refwwid && strncmp (mpp->alias, refwwid, WWID_SIZE - 1)) - continue; - - if (update_multipath_table(mpp, pathvec, DI_CHECKER) != DMP_OK || - update_mpp_paths(mpp, pathvec)) { - condlog(1, "error parsing map %s", mpp->wwid); - remove_map(mpp, pathvec, curmp); - i--; - } else - extract_hwe_from_path(mpp); - } - return MPATH_PR_SUCCESS ; -} - -int mpath_send_prin_activepath (char * dev, int rq_servact, - struct prin_resp * resp, int noisy) -{ - - int rc; - - rc = prin_do_scsi_ioctl(dev, rq_servact, resp, noisy); - - return (rc); -} - -int mpath_prout_reg(struct multipath *mpp,int rq_servact, int rq_scope, - unsigned int rq_type, struct prout_param_descriptor * paramp, int noisy) -{ - - int i, j, k; - struct pathgroup *pgp = NULL; - struct path *pp = NULL; - int rollback = 0; - int active_pathcount=0; - int rc; - int count=0; - int status = MPATH_PR_SUCCESS; - int all_tg_pt; - uint64_t sa_key = 0; - - if (!mpp) - return MPATH_PR_DMMP_ERROR; - - all_tg_pt = (mpp->all_tg_pt == ALL_TG_PT_ON || - paramp->sa_flags & MPATH_F_ALL_TG_PT_MASK); - active_pathcount = count_active_paths(mpp); - - if (active_pathcount == 0) { - condlog (0, "%s: no path available", mpp->wwid); - return MPATH_PR_DMMP_ERROR; - } - - struct threadinfo thread[active_pathcount]; - int hosts[active_pathcount]; - - memset(thread, 0, sizeof(thread)); - - /* init thread parameter */ - for (i =0; i< active_pathcount; i++){ - hosts[i] = -1; - thread[i].param.rq_servact = rq_servact; - thread[i].param.rq_scope = rq_scope; - thread[i].param.rq_type = rq_type; - thread[i].param.paramp = paramp; - thread[i].param.noisy = noisy; - thread[i].param.status = MPATH_PR_SKIP; - - condlog (3, "THREAD ID [%d] INFO]", i); - condlog (3, "rq_servact=%d ", thread[i].param.rq_servact); - condlog (3, "rq_scope=%d ", thread[i].param.rq_scope); - condlog (3, "rq_type=%d ", thread[i].param.rq_type); - condlog (3, "rkey="); - condlog (3, "paramp->sa_flags =%02x ", - thread[i].param.paramp->sa_flags); - condlog (3, "noisy=%d ", thread[i].param.noisy); - condlog (3, "status=%d ", thread[i].param.status); - } - - pthread_attr_t attr; - pthread_attr_init(&attr); - pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE); - - vector_foreach_slot (mpp->pg, pgp, j){ - vector_foreach_slot (pgp->paths, pp, i){ - if (!((pp->state == PATH_UP) || (pp->state == PATH_GHOST))){ - condlog (1, "%s: %s path not up. Skip.", mpp->wwid, pp->dev); - continue; - } - if (all_tg_pt && pp->sg_id.host_no != -1) { - for (k = 0; k < count; k++) { - if (pp->sg_id.host_no == hosts[k]) { - condlog(3, "%s: %s host %d matches skip.", pp->wwid, pp->dev, pp->sg_id.host_no); - break; - } - } - if (k < count) - continue; - } - strlcpy(thread[count].param.dev, pp->dev, - FILE_NAME_SIZE); - - if (count && (thread[count].param.paramp->sa_flags & MPATH_F_SPEC_I_PT_MASK)){ - /* - * Clearing SPEC_I_PT as transportids are already registered by now. - */ - thread[count].param.paramp->sa_flags &= (~MPATH_F_SPEC_I_PT_MASK); - } - - condlog (3, "%s: sending pr out command to %s", mpp->wwid, pp->dev); - - rc = pthread_create(&thread[count].id, &attr, mpath_prout_pthread_fn, (void *)(&thread[count].param)); - if (rc){ - condlog (0, "%s: failed to create thread %d", mpp->wwid, rc); - thread[count].param.status = MPATH_PR_THREAD_ERROR; - } - else - hosts[count] = pp->sg_id.host_no; - count = count + 1; - } - } - for( i=0; i < count ; i++){ - if (thread[i].param.status != MPATH_PR_THREAD_ERROR) { - rc = pthread_join(thread[i].id, NULL); - if (rc){ - condlog (0, "%s: Thread[%d] failed to join thread %d", mpp->wwid, i, rc); - } - } - if (!rollback && (thread[i].param.status == MPATH_PR_RESERV_CONFLICT)){ - rollback = 1; - sa_key = get_unaligned_be64(¶mp->sa_key[0]); - status = MPATH_PR_RESERV_CONFLICT ; - } - if (!rollback && (status == MPATH_PR_SUCCESS)){ - status = thread[i].param.status; - } - } - if (rollback && ((rq_servact == MPATH_PROUT_REG_SA) && sa_key != 0 )){ - condlog (3, "%s: ERROR: initiating pr out rollback", mpp->wwid); - memcpy(¶mp->key, ¶mp->sa_key, 8); - memset(¶mp->sa_key, 0, 8); - for( i=0 ; i < count ; i++){ - if(thread[i].param.status == MPATH_PR_SUCCESS) { - rc = pthread_create(&thread[i].id, &attr, mpath_prout_pthread_fn, - (void *)(&thread[i].param)); - if (rc){ - condlog (0, "%s: failed to create thread for rollback. %d", mpp->wwid, rc); - thread[i].param.status = MPATH_PR_THREAD_ERROR; - } - } else - thread[i].param.status = MPATH_PR_SKIP; - } - for(i=0; i < count ; i++){ - if (thread[i].param.status != MPATH_PR_SKIP && - thread[i].param.status != MPATH_PR_THREAD_ERROR) { - rc = pthread_join(thread[i].id, NULL); - if (rc){ - condlog (3, "%s: failed to join thread while rolling back %d", - mpp->wwid, i); - } - } - } - } - - pthread_attr_destroy(&attr); - return (status); -} - -void * mpath_prout_pthread_fn(void *p) -{ - int ret; - struct prout_param * param = (struct prout_param *)p; - - ret = prout_do_scsi_ioctl( param->dev,param->rq_servact, param->rq_scope, - param->rq_type, param->paramp, param->noisy); - param->status = ret; - pthread_exit(NULL); -} - -int mpath_prout_common(struct multipath *mpp,int rq_servact, int rq_scope, - unsigned int rq_type, struct prout_param_descriptor* paramp, int noisy) -{ - int i,j, ret; - struct pathgroup *pgp = NULL; - struct path *pp = NULL; - - vector_foreach_slot (mpp->pg, pgp, j){ - vector_foreach_slot (pgp->paths, pp, i){ - if (!((pp->state == PATH_UP) || (pp->state == PATH_GHOST))){ - condlog (1, "%s: %s path not up. Skip", - mpp->wwid, pp->dev); - continue; - } - - condlog (3, "%s: sending pr out command to %s", mpp->wwid, pp->dev); - ret = send_prout_activepath(pp->dev, rq_servact, - rq_scope, rq_type, - paramp, noisy); - return ret ; - } - } - condlog (0, "%s: no path available", mpp->wwid); - return MPATH_PR_DMMP_ERROR; -} - -int send_prout_activepath(char * dev, int rq_servact, int rq_scope, - unsigned int rq_type, struct prout_param_descriptor * paramp, int noisy) -{ - struct prout_param param; - param.rq_servact = rq_servact; - param.rq_scope = rq_scope; - param.rq_type = rq_type; - param.paramp = paramp; - param.noisy = noisy; - param.status = -1; - - pthread_t thread; - pthread_attr_t attr; - int rc; - - memset(&thread, 0, sizeof(thread)); - strlcpy(param.dev, dev, FILE_NAME_SIZE); - /* Initialize and set thread joinable attribute */ - pthread_attr_init(&attr); - pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE); - - rc = pthread_create(&thread, &attr, mpath_prout_pthread_fn, (void *)(¶m)); - if (rc){ - condlog (3, "%s: failed to create thread %d", dev, rc); - return MPATH_PR_THREAD_ERROR; - } - /* Free attribute and wait for the other threads */ - pthread_attr_destroy(&attr); - rc = pthread_join(thread, NULL); - - return (param.status); -} - -int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope, - unsigned int rq_type, struct prout_param_descriptor * paramp, int noisy) -{ - int i, j; - int num = 0; - struct pathgroup *pgp = NULL; - struct path *pp = NULL; - int active_pathcount = 0; - pthread_attr_t attr; - int rc, found = 0; - int count = 0; - int status = MPATH_PR_SUCCESS; - struct prin_resp resp; - struct prout_param_descriptor *pamp; - struct prin_resp *pr_buff; - int length; - struct transportid *pptr; - - if (!mpp) - return MPATH_PR_DMMP_ERROR; - - active_pathcount = count_active_paths(mpp); - - if (active_pathcount == 0) { - condlog (0, "%s: no path available", mpp->wwid); - return MPATH_PR_DMMP_ERROR; - } - - struct threadinfo thread[active_pathcount]; - memset(thread, 0, sizeof(thread)); - for (i = 0; i < active_pathcount; i++){ - thread[i].param.rq_servact = rq_servact; - thread[i].param.rq_scope = rq_scope; - thread[i].param.rq_type = rq_type; - thread[i].param.paramp = paramp; - thread[i].param.noisy = noisy; - thread[i].param.status = MPATH_PR_SKIP; - - condlog (3, " path count = %d", i); - condlog (3, "rq_servact=%d ", thread[i].param.rq_servact); - condlog (3, "rq_scope=%d ", thread[i].param.rq_scope); - condlog (3, "rq_type=%d ", thread[i].param.rq_type); - condlog (3, "noisy=%d ", thread[i].param.noisy); - condlog (3, "status=%d ", thread[i].param.status); - } - - pthread_attr_init (&attr); - pthread_attr_setdetachstate (&attr, PTHREAD_CREATE_JOINABLE); - - vector_foreach_slot (mpp->pg, pgp, j){ - vector_foreach_slot (pgp->paths, pp, i){ - if (!((pp->state == PATH_UP) || (pp->state == PATH_GHOST))){ - condlog (1, "%s: %s path not up.", mpp->wwid, pp->dev); - continue; - } - - strlcpy(thread[count].param.dev, pp->dev, - FILE_NAME_SIZE); - condlog (3, "%s: sending pr out command to %s", mpp->wwid, pp->dev); - rc = pthread_create (&thread[count].id, &attr, mpath_prout_pthread_fn, - (void *) (&thread[count].param)); - if (rc) { - condlog (0, "%s: failed to create thread. %d", mpp->wwid, rc); - thread[count].param.status = MPATH_PR_THREAD_ERROR; - } - count = count + 1; - } - } - pthread_attr_destroy (&attr); - for (i = 0; i < count; i++){ - if (thread[i].param.status != MPATH_PR_THREAD_ERROR) { - rc = pthread_join (thread[i].id, NULL); - if (rc){ - condlog (1, "%s: failed to join thread. %d", mpp->wwid, rc); - } - } - } - - for (i = 0; i < count; i++){ - /* check thread status here and return the status */ - - if (thread[i].param.status == MPATH_PR_RESERV_CONFLICT) - status = MPATH_PR_RESERV_CONFLICT; - else if (status == MPATH_PR_SUCCESS - && thread[i].param.status != MPATH_PR_RESERV_CONFLICT) - status = thread[i].param.status; - } - - status = mpath_prin_activepath (mpp, MPATH_PRIN_RRES_SA, &resp, noisy); - if (status != MPATH_PR_SUCCESS){ - condlog (0, "%s: pr in read reservation command failed.", mpp->wwid); - return MPATH_PR_OTHER; - } - - num = resp.prin_descriptor.prin_readresv.additional_length / 8; - if (num == 0){ - condlog (2, "%s: Path holding reservation is released.", mpp->wwid); - return MPATH_PR_SUCCESS; - } - condlog (2, "%s: Path holding reservation is not avialable.", mpp->wwid); - - pr_buff = mpath_alloc_prin_response(MPATH_PRIN_RFSTAT_SA); - if (!pr_buff){ - condlog (0, "%s: failed to alloc pr in response buffer.", mpp->wwid); - return MPATH_PR_OTHER; - } - - status = mpath_prin_activepath (mpp, MPATH_PRIN_RFSTAT_SA, pr_buff, noisy); - - if (status != MPATH_PR_SUCCESS){ - condlog (0, "%s: pr in read full status command failed.", mpp->wwid); - goto out; - } - - num = pr_buff->prin_descriptor.prin_readfd.number_of_descriptor; - if (0 == num){ - goto out; - } - length = sizeof (struct prout_param_descriptor) + (sizeof (struct transportid *)); - - pamp = (struct prout_param_descriptor *)malloc (length); - if (!pamp){ - condlog (0, "%s: failed to alloc pr out parameter.", mpp->wwid); - goto out1; - } - - memset(pamp, 0, length); - - pamp->trnptid_list[0] = (struct transportid *) malloc (sizeof (struct transportid)); - if (!pamp->trnptid_list[0]){ - condlog (0, "%s: failed to alloc pr out transportid.", mpp->wwid); - goto out1; - } - - if (get_be64(mpp->reservation_key)){ - memcpy (pamp->key, &mpp->reservation_key, 8); - condlog (3, "%s: reservation key set.", mpp->wwid); - } - - status = mpath_prout_common (mpp, MPATH_PROUT_CLEAR_SA, - rq_scope, rq_type, pamp, noisy); - - if (status) { - condlog(0, "%s: failed to send CLEAR_SA", mpp->wwid); - goto out1; - } - - pamp->num_transportid = 1; - pptr=pamp->trnptid_list[0]; - - for (i = 0; i < num; i++){ - if (get_be64(mpp->reservation_key) && - memcmp(pr_buff->prin_descriptor.prin_readfd.descriptors[i]->key, - &mpp->reservation_key, 8)){ - /*register with tarnsport id*/ - memset(pamp, 0, length); - pamp->trnptid_list[0] = pptr; - memset (pamp->trnptid_list[0], 0, sizeof (struct transportid)); - memcpy (pamp->sa_key, - pr_buff->prin_descriptor.prin_readfd.descriptors[i]->key, 8); - pamp->sa_flags = MPATH_F_SPEC_I_PT_MASK; - pamp->num_transportid = 1; - - memcpy (pamp->trnptid_list[0], - &pr_buff->prin_descriptor.prin_readfd.descriptors[i]->trnptid, - sizeof (struct transportid)); - status = mpath_prout_common (mpp, MPATH_PROUT_REG_SA, 0, rq_type, - pamp, noisy); - - pamp->sa_flags = 0; - memcpy (pamp->key, pr_buff->prin_descriptor.prin_readfd.descriptors[i]->key, 8); - memset (pamp->sa_key, 0, 8); - pamp->num_transportid = 0; - status = mpath_prout_common (mpp, MPATH_PROUT_REG_SA, 0, rq_type, - pamp, noisy); - } - else - { - if (get_be64(mpp->reservation_key)) - found = 1; - } - - - } - - if (found){ - memset (pamp, 0, length); - memcpy (pamp->sa_key, &mpp->reservation_key, 8); - memset (pamp->key, 0, 8); - status = mpath_prout_reg(mpp, MPATH_PROUT_REG_SA, rq_scope, rq_type, pamp, noisy); - } - - - free(pptr); -out1: - free (pamp); -out: - free (pr_buff); - return (status); -} - -void * mpath_alloc_prin_response(int prin_sa) -{ - void * ptr = NULL; - int size=0; - switch (prin_sa) - { - case MPATH_PRIN_RKEY_SA: - size = sizeof(struct prin_readdescr); - break; - case MPATH_PRIN_RRES_SA: - size = sizeof(struct prin_resvdescr); - break; - case MPATH_PRIN_RCAP_SA: - size=sizeof(struct prin_capdescr); - break; - case MPATH_PRIN_RFSTAT_SA: - size = sizeof(struct print_fulldescr_list) + - sizeof(struct prin_fulldescr *)*MPATH_MX_TIDS; - break; - } - if (size > 0) - { - ptr = calloc(size, 1); - } - return ptr; -} - -int update_map_pr(struct multipath *mpp) -{ - int noisy=0; - struct prin_resp *resp; - unsigned int i; - int ret, isFound; - - if (!get_be64(mpp->reservation_key)) - { - /* Nothing to do. Assuming pr mgmt feature is disabled*/ - condlog(4, "%s: reservation_key not set in multipath.conf", - mpp->alias); - return MPATH_PR_SUCCESS; - } - - resp = mpath_alloc_prin_response(MPATH_PRIN_RKEY_SA); - if (!resp) - { - condlog(0,"%s : failed to alloc resp in update_map_pr", mpp->alias); - return MPATH_PR_OTHER; - } - ret = mpath_prin_activepath(mpp, MPATH_PRIN_RKEY_SA, resp, noisy); - - if (ret != MPATH_PR_SUCCESS ) - { - condlog(0,"%s : pr in read keys service action failed Error=%d", mpp->alias, ret); - free(resp); - return ret; - } - - if (resp->prin_descriptor.prin_readkeys.additional_length == 0 ) - { - condlog(3,"%s: No key found. Device may not be registered. ", mpp->alias); - free(resp); - return MPATH_PR_SUCCESS; - } - - condlog(2, "%s: Multipath reservation_key: 0x%" PRIx64 " ", mpp->alias, - get_be64(mpp->reservation_key)); - - isFound =0; - for (i = 0; i < resp->prin_descriptor.prin_readkeys.additional_length/8; i++ ) - { - condlog(2, "%s: PR IN READKEYS[%d] reservation key:", mpp->alias, i); - dumpHex((char *)&resp->prin_descriptor.prin_readkeys.key_list[i*8], 8 , 1); - - if (!memcmp(&mpp->reservation_key, &resp->prin_descriptor.prin_readkeys.key_list[i*8], 8)) - { - condlog(2, "%s: reservation key found in pr in readkeys response", mpp->alias); - isFound =1; - } - } - - if (isFound) - { - mpp->prflag = 1; - condlog(2, "%s: prflag flag set.", mpp->alias ); - } - - free(resp); - return MPATH_PR_SUCCESS; -} diff --git a/libmpathpersist/mpath_persist_int.c b/libmpathpersist/mpath_persist_int.c new file mode 100644 index 000000000..7f6ac0413 --- /dev/null +++ b/libmpathpersist/mpath_persist_int.c @@ -0,0 +1,779 @@ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "vector.h" +#include "defaults.h" +#include "checkers.h" +#include "structs.h" +#include "structs_vec.h" +#include "prio.h" +#include "devmapper.h" +#include "debug.h" +#include "config.h" +#include "switchgroup.h" +#include "discovery.h" +#include "configure.h" +#include "dmparser.h" +#include "propsel.h" +#include "util.h" +#include "unaligned.h" + +#include "mpath_persist.h" +#include "mpath_persist_int.h" +#include "mpathpr.h" +#include "mpath_pr_ioctl.h" + + +#define __STDC_FORMAT_MACROS 1 + +static int mpath_send_prin_activepath (char * dev, int rq_servact, + struct prin_resp * resp, int noisy) +{ + + int rc; + + rc = prin_do_scsi_ioctl(dev, rq_servact, resp, noisy); + + return (rc); +} + +static int mpath_prin_activepath (struct multipath *mpp, int rq_servact, + struct prin_resp * resp, int noisy) +{ + int i,j, ret = MPATH_PR_DMMP_ERROR; + struct pathgroup *pgp = NULL; + struct path *pp = NULL; + + vector_foreach_slot (mpp->pg, pgp, j){ + vector_foreach_slot (pgp->paths, pp, i){ + if (!((pp->state == PATH_UP) || + (pp->state == PATH_GHOST))){ + condlog(2, "%s: %s not available. Skip.", + mpp->wwid, pp->dev); + condlog(3, "%s: status = %d.", + mpp->wwid, pp->state); + continue; + } + + condlog(3, "%s: sending pr in command to %s ", + mpp->wwid, pp->dev); + ret = mpath_send_prin_activepath(pp->dev, rq_servact, + resp, noisy); + switch(ret) + { + case MPATH_PR_SUCCESS: + case MPATH_PR_SENSE_INVALID_OP: + return ret; + default: + continue; + } + } + } + return ret; +} + +void *mpath_alloc_prin_response(int prin_sa) +{ + void * ptr = NULL; + int size=0; + switch (prin_sa) + { + case MPATH_PRIN_RKEY_SA: + size = sizeof(struct prin_readdescr); + break; + case MPATH_PRIN_RRES_SA: + size = sizeof(struct prin_resvdescr); + break; + case MPATH_PRIN_RCAP_SA: + size=sizeof(struct prin_capdescr); + break; + case MPATH_PRIN_RFSTAT_SA: + size = sizeof(struct print_fulldescr_list) + + sizeof(struct prin_fulldescr *)*MPATH_MX_TIDS; + break; + } + if (size > 0) + { + ptr = calloc(size, 1); + } + return ptr; +} + +static int get_mpvec(vector curmp, vector pathvec, char *refwwid) +{ + int i; + struct multipath *mpp; + + vector_foreach_slot (curmp, mpp, i){ + /* + * discard out of scope maps + */ + if (!mpp->alias) { + condlog(0, "%s: map with empty alias!", __func__); + continue; + } + + if (mpp->pg != NULL) + /* Already seen this one */ + continue; + + if (refwwid && strncmp (mpp->alias, refwwid, WWID_SIZE - 1)) + continue; + + if (update_multipath_table(mpp, pathvec, DI_CHECKER) != DMP_OK || + update_mpp_paths(mpp, pathvec)) { + condlog(1, "error parsing map %s", mpp->wwid); + remove_map(mpp, pathvec, curmp); + i--; + } else + extract_hwe_from_path(mpp); + } + return MPATH_PR_SUCCESS ; +} + +static int mpath_get_map(vector curmp, vector pathvec, int fd, char **palias, + struct multipath **pmpp) +{ + int ret = MPATH_PR_DMMP_ERROR; + struct stat info; + int major, minor; + char *alias; + struct multipath *mpp; + + if (fstat(fd, &info) != 0){ + condlog(0, "stat error fd=%d", fd); + return MPATH_PR_FILE_ERROR; + } + if(!S_ISBLK(info.st_mode)){ + condlog(3, "Failed to get major:minor. fd=%d", fd); + return MPATH_PR_FILE_ERROR; + } + + major = major(info.st_rdev); + minor = minor(info.st_rdev); + condlog(4, "Device %d:%d", major, minor); + + /* get alias from major:minor*/ + alias = dm_mapname(major, minor); + if (!alias){ + condlog(0, "%d:%d failed to get device alias.", major, minor); + return MPATH_PR_DMMP_ERROR; + } + + condlog(3, "alias = %s", alias); + + if (dm_map_present(alias) && dm_is_mpath(alias) != 1){ + condlog(3, "%s: not a multipath device.", alias); + goto out; + } + + /* get info of all paths from the dm device */ + if (get_mpvec(curmp, pathvec, alias)){ + condlog(0, "%s: failed to get device info.", alias); + goto out; + } + + mpp = find_mp_by_alias(curmp, alias); + + if (!mpp) { + condlog(0, "%s: devmap not registered.", alias); + goto out; + } + + ret = MPATH_PR_SUCCESS; + if (pmpp) + *pmpp = mpp; + if (palias) { + *palias = alias; + alias = NULL; + } +out: + free(alias); + return ret; +} + +int do_mpath_persistent_reserve_in(vector curmp, vector pathvec, + int fd, int rq_servact, + struct prin_resp *resp, int noisy) +{ + struct multipath *mpp; + int ret; + + ret = mpath_get_map(curmp, pathvec, fd, NULL, &mpp); + if (ret != MPATH_PR_SUCCESS) + return ret; + + ret = mpath_prin_activepath(mpp, rq_servact, resp, noisy); + + return ret; +} + +static void *mpath_prout_pthread_fn(void *p) +{ + int ret; + struct prout_param * param = (struct prout_param *)p; + + ret = prout_do_scsi_ioctl( param->dev,param->rq_servact, param->rq_scope, + param->rq_type, param->paramp, param->noisy); + param->status = ret; + pthread_exit(NULL); +} + +static int mpath_prout_reg(struct multipath *mpp,int rq_servact, int rq_scope, + unsigned int rq_type, + struct prout_param_descriptor * paramp, int noisy) +{ + + int i, j, k; + struct pathgroup *pgp = NULL; + struct path *pp = NULL; + int rollback = 0; + int active_pathcount=0; + int rc; + int count=0; + int status = MPATH_PR_SUCCESS; + int all_tg_pt; + uint64_t sa_key = 0; + + if (!mpp) + return MPATH_PR_DMMP_ERROR; + + all_tg_pt = (mpp->all_tg_pt == ALL_TG_PT_ON || + paramp->sa_flags & MPATH_F_ALL_TG_PT_MASK); + active_pathcount = count_active_paths(mpp); + + if (active_pathcount == 0) { + condlog (0, "%s: no path available", mpp->wwid); + return MPATH_PR_DMMP_ERROR; + } + + struct threadinfo thread[active_pathcount]; + int hosts[active_pathcount]; + + memset(thread, 0, sizeof(thread)); + + /* init thread parameter */ + for (i =0; i< active_pathcount; i++){ + hosts[i] = -1; + thread[i].param.rq_servact = rq_servact; + thread[i].param.rq_scope = rq_scope; + thread[i].param.rq_type = rq_type; + thread[i].param.paramp = paramp; + thread[i].param.noisy = noisy; + thread[i].param.status = MPATH_PR_SKIP; + + condlog (3, "THREAD ID [%d] INFO]", i); + condlog (3, "rq_servact=%d ", thread[i].param.rq_servact); + condlog (3, "rq_scope=%d ", thread[i].param.rq_scope); + condlog (3, "rq_type=%d ", thread[i].param.rq_type); + condlog (3, "rkey="); + condlog (3, "paramp->sa_flags =%02x ", + thread[i].param.paramp->sa_flags); + condlog (3, "noisy=%d ", thread[i].param.noisy); + condlog (3, "status=%d ", thread[i].param.status); + } + + pthread_attr_t attr; + pthread_attr_init(&attr); + pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE); + + vector_foreach_slot (mpp->pg, pgp, j){ + vector_foreach_slot (pgp->paths, pp, i){ + if (!((pp->state == PATH_UP) || (pp->state == PATH_GHOST))){ + condlog (1, "%s: %s path not up. Skip.", mpp->wwid, pp->dev); + continue; + } + if (all_tg_pt && pp->sg_id.host_no != -1) { + for (k = 0; k < count; k++) { + if (pp->sg_id.host_no == hosts[k]) { + condlog(3, "%s: %s host %d matches skip.", pp->wwid, pp->dev, pp->sg_id.host_no); + break; + } + } + if (k < count) + continue; + } + strlcpy(thread[count].param.dev, pp->dev, + FILE_NAME_SIZE); + + if (count && (thread[count].param.paramp->sa_flags & MPATH_F_SPEC_I_PT_MASK)){ + /* + * Clearing SPEC_I_PT as transportids are already registered by now. + */ + thread[count].param.paramp->sa_flags &= (~MPATH_F_SPEC_I_PT_MASK); + } + + condlog (3, "%s: sending pr out command to %s", mpp->wwid, pp->dev); + + rc = pthread_create(&thread[count].id, &attr, mpath_prout_pthread_fn, (void *)(&thread[count].param)); + if (rc){ + condlog (0, "%s: failed to create thread %d", mpp->wwid, rc); + thread[count].param.status = MPATH_PR_THREAD_ERROR; + } + else + hosts[count] = pp->sg_id.host_no; + count = count + 1; + } + } + for( i=0; i < count ; i++){ + if (thread[i].param.status != MPATH_PR_THREAD_ERROR) { + rc = pthread_join(thread[i].id, NULL); + if (rc){ + condlog (0, "%s: Thread[%d] failed to join thread %d", mpp->wwid, i, rc); + } + } + if (!rollback && (thread[i].param.status == MPATH_PR_RESERV_CONFLICT)){ + rollback = 1; + sa_key = get_unaligned_be64(¶mp->sa_key[0]); + status = MPATH_PR_RESERV_CONFLICT ; + } + if (!rollback && (status == MPATH_PR_SUCCESS)){ + status = thread[i].param.status; + } + } + if (rollback && ((rq_servact == MPATH_PROUT_REG_SA) && sa_key != 0 )){ + condlog (3, "%s: ERROR: initiating pr out rollback", mpp->wwid); + memcpy(¶mp->key, ¶mp->sa_key, 8); + memset(¶mp->sa_key, 0, 8); + for( i=0 ; i < count ; i++){ + if(thread[i].param.status == MPATH_PR_SUCCESS) { + rc = pthread_create(&thread[i].id, &attr, mpath_prout_pthread_fn, + (void *)(&thread[i].param)); + if (rc){ + condlog (0, "%s: failed to create thread for rollback. %d", mpp->wwid, rc); + thread[i].param.status = MPATH_PR_THREAD_ERROR; + } + } else + thread[i].param.status = MPATH_PR_SKIP; + } + for(i=0; i < count ; i++){ + if (thread[i].param.status != MPATH_PR_SKIP && + thread[i].param.status != MPATH_PR_THREAD_ERROR) { + rc = pthread_join(thread[i].id, NULL); + if (rc){ + condlog (3, "%s: failed to join thread while rolling back %d", + mpp->wwid, i); + } + } + } + } + + pthread_attr_destroy(&attr); + return (status); +} + +static int send_prout_activepath(char *dev, int rq_servact, int rq_scope, + unsigned int rq_type, + struct prout_param_descriptor * paramp, int noisy) +{ + struct prout_param param; + param.rq_servact = rq_servact; + param.rq_scope = rq_scope; + param.rq_type = rq_type; + param.paramp = paramp; + param.noisy = noisy; + param.status = -1; + + pthread_t thread; + pthread_attr_t attr; + int rc; + + memset(&thread, 0, sizeof(thread)); + strlcpy(param.dev, dev, FILE_NAME_SIZE); + /* Initialize and set thread joinable attribute */ + pthread_attr_init(&attr); + pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE); + + rc = pthread_create(&thread, &attr, mpath_prout_pthread_fn, (void *)(¶m)); + if (rc){ + condlog (3, "%s: failed to create thread %d", dev, rc); + return MPATH_PR_THREAD_ERROR; + } + /* Free attribute and wait for the other threads */ + pthread_attr_destroy(&attr); + rc = pthread_join(thread, NULL); + + return (param.status); +} + +static int mpath_prout_common(struct multipath *mpp,int rq_servact, int rq_scope, + unsigned int rq_type, + struct prout_param_descriptor* paramp, int noisy) +{ + int i,j, ret; + struct pathgroup *pgp = NULL; + struct path *pp = NULL; + + vector_foreach_slot (mpp->pg, pgp, j){ + vector_foreach_slot (pgp->paths, pp, i){ + if (!((pp->state == PATH_UP) || (pp->state == PATH_GHOST))){ + condlog (1, "%s: %s path not up. Skip", + mpp->wwid, pp->dev); + continue; + } + + condlog (3, "%s: sending pr out command to %s", mpp->wwid, pp->dev); + ret = send_prout_activepath(pp->dev, rq_servact, + rq_scope, rq_type, + paramp, noisy); + return ret ; + } + } + condlog (0, "%s: no path available", mpp->wwid); + return MPATH_PR_DMMP_ERROR; +} + +static int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope, + unsigned int rq_type, + struct prout_param_descriptor * paramp, int noisy) +{ + int i, j; + int num = 0; + struct pathgroup *pgp = NULL; + struct path *pp = NULL; + int active_pathcount = 0; + pthread_attr_t attr; + int rc, found = 0; + int count = 0; + int status = MPATH_PR_SUCCESS; + struct prin_resp resp; + struct prout_param_descriptor *pamp; + struct prin_resp *pr_buff; + int length; + struct transportid *pptr; + + if (!mpp) + return MPATH_PR_DMMP_ERROR; + + active_pathcount = count_active_paths(mpp); + + if (active_pathcount == 0) { + condlog (0, "%s: no path available", mpp->wwid); + return MPATH_PR_DMMP_ERROR; + } + + struct threadinfo thread[active_pathcount]; + memset(thread, 0, sizeof(thread)); + for (i = 0; i < active_pathcount; i++){ + thread[i].param.rq_servact = rq_servact; + thread[i].param.rq_scope = rq_scope; + thread[i].param.rq_type = rq_type; + thread[i].param.paramp = paramp; + thread[i].param.noisy = noisy; + thread[i].param.status = MPATH_PR_SKIP; + + condlog (3, " path count = %d", i); + condlog (3, "rq_servact=%d ", thread[i].param.rq_servact); + condlog (3, "rq_scope=%d ", thread[i].param.rq_scope); + condlog (3, "rq_type=%d ", thread[i].param.rq_type); + condlog (3, "noisy=%d ", thread[i].param.noisy); + condlog (3, "status=%d ", thread[i].param.status); + } + + pthread_attr_init (&attr); + pthread_attr_setdetachstate (&attr, PTHREAD_CREATE_JOINABLE); + + vector_foreach_slot (mpp->pg, pgp, j){ + vector_foreach_slot (pgp->paths, pp, i){ + if (!((pp->state == PATH_UP) || (pp->state == PATH_GHOST))){ + condlog (1, "%s: %s path not up.", mpp->wwid, pp->dev); + continue; + } + + strlcpy(thread[count].param.dev, pp->dev, + FILE_NAME_SIZE); + condlog (3, "%s: sending pr out command to %s", mpp->wwid, pp->dev); + rc = pthread_create (&thread[count].id, &attr, mpath_prout_pthread_fn, + (void *) (&thread[count].param)); + if (rc) { + condlog (0, "%s: failed to create thread. %d", mpp->wwid, rc); + thread[count].param.status = MPATH_PR_THREAD_ERROR; + } + count = count + 1; + } + } + pthread_attr_destroy (&attr); + for (i = 0; i < count; i++){ + if (thread[i].param.status != MPATH_PR_THREAD_ERROR) { + rc = pthread_join (thread[i].id, NULL); + if (rc){ + condlog (1, "%s: failed to join thread. %d", mpp->wwid, rc); + } + } + } + + for (i = 0; i < count; i++){ + /* check thread status here and return the status */ + + if (thread[i].param.status == MPATH_PR_RESERV_CONFLICT) + status = MPATH_PR_RESERV_CONFLICT; + else if (status == MPATH_PR_SUCCESS + && thread[i].param.status != MPATH_PR_RESERV_CONFLICT) + status = thread[i].param.status; + } + + status = mpath_prin_activepath (mpp, MPATH_PRIN_RRES_SA, &resp, noisy); + if (status != MPATH_PR_SUCCESS){ + condlog (0, "%s: pr in read reservation command failed.", mpp->wwid); + return MPATH_PR_OTHER; + } + + num = resp.prin_descriptor.prin_readresv.additional_length / 8; + if (num == 0){ + condlog (2, "%s: Path holding reservation is released.", mpp->wwid); + return MPATH_PR_SUCCESS; + } + condlog (2, "%s: Path holding reservation is not avialable.", mpp->wwid); + + pr_buff = mpath_alloc_prin_response(MPATH_PRIN_RFSTAT_SA); + if (!pr_buff){ + condlog (0, "%s: failed to alloc pr in response buffer.", mpp->wwid); + return MPATH_PR_OTHER; + } + + status = mpath_prin_activepath (mpp, MPATH_PRIN_RFSTAT_SA, pr_buff, noisy); + + if (status != MPATH_PR_SUCCESS){ + condlog (0, "%s: pr in read full status command failed.", mpp->wwid); + goto out; + } + + num = pr_buff->prin_descriptor.prin_readfd.number_of_descriptor; + if (0 == num){ + goto out; + } + length = sizeof (struct prout_param_descriptor) + (sizeof (struct transportid *)); + + pamp = (struct prout_param_descriptor *)malloc (length); + if (!pamp){ + condlog (0, "%s: failed to alloc pr out parameter.", mpp->wwid); + goto out1; + } + + memset(pamp, 0, length); + + pamp->trnptid_list[0] = (struct transportid *) malloc (sizeof (struct transportid)); + if (!pamp->trnptid_list[0]){ + condlog (0, "%s: failed to alloc pr out transportid.", mpp->wwid); + goto out1; + } + + if (get_be64(mpp->reservation_key)){ + memcpy (pamp->key, &mpp->reservation_key, 8); + condlog (3, "%s: reservation key set.", mpp->wwid); + } + + status = mpath_prout_common (mpp, MPATH_PROUT_CLEAR_SA, + rq_scope, rq_type, pamp, noisy); + + if (status) { + condlog(0, "%s: failed to send CLEAR_SA", mpp->wwid); + goto out1; + } + + pamp->num_transportid = 1; + pptr=pamp->trnptid_list[0]; + + for (i = 0; i < num; i++){ + if (get_be64(mpp->reservation_key) && + memcmp(pr_buff->prin_descriptor.prin_readfd.descriptors[i]->key, + &mpp->reservation_key, 8)){ + /*register with tarnsport id*/ + memset(pamp, 0, length); + pamp->trnptid_list[0] = pptr; + memset (pamp->trnptid_list[0], 0, sizeof (struct transportid)); + memcpy (pamp->sa_key, + pr_buff->prin_descriptor.prin_readfd.descriptors[i]->key, 8); + pamp->sa_flags = MPATH_F_SPEC_I_PT_MASK; + pamp->num_transportid = 1; + + memcpy (pamp->trnptid_list[0], + &pr_buff->prin_descriptor.prin_readfd.descriptors[i]->trnptid, + sizeof (struct transportid)); + status = mpath_prout_common (mpp, MPATH_PROUT_REG_SA, 0, rq_type, + pamp, noisy); + + pamp->sa_flags = 0; + memcpy (pamp->key, pr_buff->prin_descriptor.prin_readfd.descriptors[i]->key, 8); + memset (pamp->sa_key, 0, 8); + pamp->num_transportid = 0; + status = mpath_prout_common (mpp, MPATH_PROUT_REG_SA, 0, rq_type, + pamp, noisy); + } + else + { + if (get_be64(mpp->reservation_key)) + found = 1; + } + + + } + + if (found){ + memset (pamp, 0, length); + memcpy (pamp->sa_key, &mpp->reservation_key, 8); + memset (pamp->key, 0, 8); + status = mpath_prout_reg(mpp, MPATH_PROUT_REG_SA, rq_scope, rq_type, pamp, noisy); + } + + + free(pptr); +out1: + free (pamp); +out: + free (pr_buff); + return (status); +} + +int do_mpath_persistent_reserve_out(vector curmp, vector pathvec, int fd, + int rq_servact, int rq_scope, unsigned int rq_type, + struct prout_param_descriptor *paramp, int noisy) +{ + struct multipath *mpp; + char *alias; + int ret; + uint64_t prkey; + struct config *conf; + + ret = mpath_get_map(curmp, pathvec, fd, &alias, &mpp); + if (ret != MPATH_PR_SUCCESS) + return ret; + + conf = get_multipath_config(); + select_reservation_key(conf, mpp); + select_all_tg_pt(conf, mpp); + put_multipath_config(conf); + + memcpy(&prkey, paramp->sa_key, 8); + if (mpp->prkey_source == PRKEY_SOURCE_FILE && prkey && + (rq_servact == MPATH_PROUT_REG_IGN_SA || + (rq_servact == MPATH_PROUT_REG_SA && + (!get_be64(mpp->reservation_key) || + memcmp(paramp->key, &mpp->reservation_key, 8) == 0)))) { + memcpy(&mpp->reservation_key, paramp->sa_key, 8); + if (update_prkey_flags(alias, get_be64(mpp->reservation_key), + paramp->sa_flags)) { + condlog(0, "%s: failed to set prkey for multipathd.", + alias); + ret = MPATH_PR_DMMP_ERROR; + goto out1; + } + } + + if (memcmp(paramp->key, &mpp->reservation_key, 8) && + memcmp(paramp->sa_key, &mpp->reservation_key, 8) && + (prkey || rq_servact != MPATH_PROUT_REG_IGN_SA)) { + condlog(0, "%s: configured reservation key doesn't match: 0x%" PRIx64, alias, get_be64(mpp->reservation_key)); + ret = MPATH_PR_SYNTAX_ERROR; + goto out1; + } + + switch(rq_servact) + { + case MPATH_PROUT_REG_SA: + case MPATH_PROUT_REG_IGN_SA: + ret= mpath_prout_reg(mpp, rq_servact, rq_scope, rq_type, paramp, noisy); + break; + case MPATH_PROUT_RES_SA : + case MPATH_PROUT_PREE_SA : + case MPATH_PROUT_PREE_AB_SA : + case MPATH_PROUT_CLEAR_SA: + ret = mpath_prout_common(mpp, rq_servact, rq_scope, rq_type, paramp, noisy); + break; + case MPATH_PROUT_REL_SA: + ret = mpath_prout_rel(mpp, rq_servact, rq_scope, rq_type, paramp, noisy); + break; + default: + ret = MPATH_PR_OTHER; + goto out1; + } + + if ((ret == MPATH_PR_SUCCESS) && ((rq_servact == MPATH_PROUT_REG_SA) || + (rq_servact == MPATH_PROUT_REG_IGN_SA))) + { + if (prkey == 0) { + update_prflag(alias, 0); + update_prkey(alias, 0); + } else + update_prflag(alias, 1); + } else if ((ret == MPATH_PR_SUCCESS) && (rq_servact == MPATH_PROUT_CLEAR_SA)) { + update_prflag(alias, 0); + update_prkey(alias, 0); + } +out1: + free(alias); + return ret; +} + +int update_map_pr(struct multipath *mpp) +{ + int noisy=0; + struct prin_resp *resp; + unsigned int i; + int ret, isFound; + + if (!get_be64(mpp->reservation_key)) + { + /* Nothing to do. Assuming pr mgmt feature is disabled*/ + condlog(4, "%s: reservation_key not set in multipath.conf", + mpp->alias); + return MPATH_PR_SUCCESS; + } + + resp = mpath_alloc_prin_response(MPATH_PRIN_RKEY_SA); + if (!resp) + { + condlog(0,"%s : failed to alloc resp in update_map_pr", mpp->alias); + return MPATH_PR_OTHER; + } + ret = mpath_prin_activepath(mpp, MPATH_PRIN_RKEY_SA, resp, noisy); + + if (ret != MPATH_PR_SUCCESS ) + { + condlog(0,"%s : pr in read keys service action failed Error=%d", mpp->alias, ret); + free(resp); + return ret; + } + + if (resp->prin_descriptor.prin_readkeys.additional_length == 0 ) + { + condlog(3,"%s: No key found. Device may not be registered. ", mpp->alias); + free(resp); + return MPATH_PR_SUCCESS; + } + + condlog(2, "%s: Multipath reservation_key: 0x%" PRIx64 " ", mpp->alias, + get_be64(mpp->reservation_key)); + + isFound =0; + for (i = 0; i < resp->prin_descriptor.prin_readkeys.additional_length/8; i++ ) + { + condlog(2, "%s: PR IN READKEYS[%d] reservation key:", mpp->alias, i); + dumpHex((char *)&resp->prin_descriptor.prin_readkeys.key_list[i*8], 8 , 1); + + if (!memcmp(&mpp->reservation_key, &resp->prin_descriptor.prin_readkeys.key_list[i*8], 8)) + { + condlog(2, "%s: reservation key found in pr in readkeys response", mpp->alias); + isFound =1; + } + } + + if (isFound) + { + mpp->prflag = 1; + condlog(2, "%s: prflag flag set.", mpp->alias ); + } + + free(resp); + return MPATH_PR_SUCCESS; +} diff --git a/libmpathpersist/mpath_persist_int.h b/libmpathpersist/mpath_persist_int.h new file mode 100644 index 000000000..58d9c33ce --- /dev/null +++ b/libmpathpersist/mpath_persist_int.h @@ -0,0 +1,17 @@ +#ifndef _MPATH_PERSIST_INT_H +#define _MPATH_PERSIST_INT_H + +struct multipath; + +void * mpath_alloc_prin_response(int prin_sa); +int do_mpath_persistent_reserve_in(vector curmp, vector pathvec, + int fd, int rq_servact, + struct prin_resp *resp, int noisy); +void *mpath_alloc_prin_response(int prin_sa); +int do_mpath_persistent_reserve_out(vector curmp, vector pathvec, int fd, + int rq_servact, int rq_scope, + unsigned int rq_type, + struct prout_param_descriptor *paramp, + int noisy); + +#endif /* _MPATH_PERSIST_INT_H */ diff --git a/libmpathpersist/mpathpr.h b/libmpathpersist/mpathpr.h index 5ea8cd6fa..59411e6fe 100644 --- a/libmpathpersist/mpathpr.h +++ b/libmpathpersist/mpathpr.h @@ -31,20 +31,8 @@ int prin_do_scsi_ioctl(char * dev, int rq_servact, struct prin_resp * resp, int int prout_do_scsi_ioctl( char * dev, int rq_servact, int rq_scope, unsigned int rq_type, struct prout_param_descriptor *paramp, int noisy); void * _mpath_pr_update (void *arg); -int mpath_send_prin_activepath (char * dev, int rq_servact, struct prin_resp * resp, int noisy); -int get_mpvec (vector curmp, vector pathvec, char * refwwid); -void * mpath_prout_pthread_fn(void *p); void dumpHex(const char* , int len, int no_ascii); -int mpath_prout_reg(struct multipath *mpp,int rq_servact, int rq_scope, - unsigned int rq_type, struct prout_param_descriptor * paramp, int noisy); -int mpath_prout_common(struct multipath *mpp,int rq_servact, int rq_scope, - unsigned int rq_type, struct prout_param_descriptor * paramp, int noisy); -int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope, - unsigned int rq_type, struct prout_param_descriptor * paramp, int noisy); -int send_prout_activepath(char * dev, int rq_servact, int rq_scope, - unsigned int rq_type, struct prout_param_descriptor * paramp, int noisy); - int update_prflag(char *mapname, int set); int update_prkey_flags(char *mapname, uint64_t prkey, uint8_t sa_flags); #define update_prkey(mapname, prkey) update_prkey_flags(mapname, prkey, 0) diff --git a/mpathpersist/main.c b/mpathpersist/main.c index 14245cc3f..4bdd55c2e 100644 --- a/mpathpersist/main.c +++ b/mpathpersist/main.c @@ -10,6 +10,7 @@ #include #include #include "mpath_persist.h" +#include "mpath_persist_int.h" #include "main.h" #include "debug.h" #include diff --git a/multipathd/main.c b/multipathd/main.c index 009a99d9d..861fad34f 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -66,6 +66,7 @@ #include "mpath_cmd.h" #include "mpath_persist.h" +#include "mpath_persist_int.h" #include "prioritizers/alua_rtpg.h" From 2f7fc4f70d1de5c2a6a00d76b4e17b1084f81c8f Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Thu, 16 Dec 2021 13:04:14 +0100 Subject: [PATCH 09/48] multipathd: remove duplicate definitions from main.h We shouldn't define function prototypes in more than a single place. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- libmpathpersist/mpath_persist_int.h | 10 +++++++++- multipathd/main.c | 3 +++ multipathd/main.h | 15 +-------------- 3 files changed, 13 insertions(+), 15 deletions(-) diff --git a/libmpathpersist/mpath_persist_int.h b/libmpathpersist/mpath_persist_int.h index 58d9c33ce..31457535a 100644 --- a/libmpathpersist/mpath_persist_int.h +++ b/libmpathpersist/mpath_persist_int.h @@ -1,7 +1,10 @@ #ifndef _MPATH_PERSIST_INT_H #define _MPATH_PERSIST_INT_H -struct multipath; +/* + * This header file contains symbols that are used by multipath-tools + * but aren't part of the public libmpathpersist API. + */ void * mpath_alloc_prin_response(int prin_sa); int do_mpath_persistent_reserve_in(vector curmp, vector pathvec, @@ -13,5 +16,10 @@ int do_mpath_persistent_reserve_out(vector curmp, vector pathvec, int fd, unsigned int rq_type, struct prout_param_descriptor *paramp, int noisy); +int prin_do_scsi_ioctl(char * dev, int rq_servact, struct prin_resp * resp, int noisy); +int prout_do_scsi_ioctl( char * dev, int rq_servact, int rq_scope, + unsigned int rq_type, struct prout_param_descriptor *paramp, int noisy); +void dumpHex(const char* , int len, int no_ascii); +int update_map_pr(struct multipath *mpp); #endif /* _MPATH_PERSIST_INT_H */ diff --git a/multipathd/main.c b/multipathd/main.c index 861fad34f..09ccf0c14 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -89,6 +89,9 @@ #define CMDSIZE 160 #define MSG_SIZE 32 +int mpath_pr_event_handle(struct path *pp); +void * mpath_pr_event_handler_fn (void * ); + #define LOG_MSG(lvl, pp) \ do { \ if (pp->mpp && checker_selected(&pp->checker) && \ diff --git a/multipathd/main.h b/multipathd/main.h index 8356b250f..459c7342e 100644 --- a/multipathd/main.h +++ b/multipathd/main.h @@ -27,9 +27,6 @@ enum remove_path_result { * because the path was also removed */ }; -struct prout_param_descriptor; -struct prin_resp; - extern pid_t daemon_pid; extern int uxsock_timeout; @@ -44,17 +41,7 @@ int ev_remove_path (struct path *, struct vectors *, int); int ev_add_map (char *, const char *, struct vectors *); int ev_remove_map (char *, char *, int, struct vectors *); int flush_map(struct multipath *, struct vectors *, int); -void * mpath_alloc_prin_response(int prin_sa); -int prin_do_scsi_ioctl(char *, int rq_servact, struct prin_resp * resp, - int noisy); -void dumpHex(const char * , int len, int no_ascii); -int prout_do_scsi_ioctl(char * , int rq_servact, int rq_scope, - unsigned int rq_type, - struct prout_param_descriptor *param, int noisy); -int mpath_pr_event_handle(struct path *pp); -void * mpath_pr_event_handler_fn (void * ); -int update_map_pr(struct multipath *mpp); -void * mpath_pr_event_handler_fn (void * pathp ); + void handle_signals(bool); int __setup_multipath (struct vectors * vecs, struct multipath * mpp, int reset); From ac8da6075bf68f1fdb6cf2108388b52e9472166e Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Thu, 16 Dec 2021 13:10:32 +0100 Subject: [PATCH 10/48] libmpathpersist/multipathd: remove duplicate definition ... of FILE_NAME_SIZE. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- libmpathpersist/mpath_pr_ioctl.c | 3 +-- multipathd/main.c | 1 - 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/libmpathpersist/mpath_pr_ioctl.c b/libmpathpersist/mpath_pr_ioctl.c index 126601c34..17c528482 100644 --- a/libmpathpersist/mpath_pr_ioctl.c +++ b/libmpathpersist/mpath_pr_ioctl.c @@ -17,8 +17,7 @@ #include "unaligned.h" #include "debug.h" - -#define FILE_NAME_SIZE 256 +#include "structs.h" /* FILE_NAME_SIZE */ #define TIMEOUT 2000 #define MAXRETRY 5 diff --git a/multipathd/main.c b/multipathd/main.c index 09ccf0c14..6087b5f4f 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -85,7 +85,6 @@ #include "../third-party/valgrind/drd.h" #include "init_unwinder.h" -#define FILE_NAME_SIZE 256 #define CMDSIZE 160 #define MSG_SIZE 32 From e179cb84ec80f8c7303d91c9042b80e338ef18fc Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Thu, 16 Dec 2021 13:06:57 +0100 Subject: [PATCH 11/48] libmpathpersist: remove __STDC_FORMAT_MACROS This has no effect. If at all, it should have been included before inttypes.h. See https://stackoverflow.com/questions/8132399/how-to-printf-uint64-t-fails-with-spurious-trailing-in-format/8132440#8132440 Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- libmpathpersist/mpath_persist_int.c | 1 - 1 file changed, 1 deletion(-) diff --git a/libmpathpersist/mpath_persist_int.c b/libmpathpersist/mpath_persist_int.c index 7f6ac0413..91bc35ae0 100644 --- a/libmpathpersist/mpath_persist_int.c +++ b/libmpathpersist/mpath_persist_int.c @@ -34,7 +34,6 @@ #include "mpath_pr_ioctl.h" -#define __STDC_FORMAT_MACROS 1 static int mpath_send_prin_activepath (char * dev, int rq_servact, struct prin_resp * resp, int noisy) From e608144bfcaef293c11da1bf83eb91e2dc96d324 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Thu, 16 Dec 2021 13:08:09 +0100 Subject: [PATCH 12/48] libmpathpersist: cleanup mpathpr.h From now on this header contains only symbols that need to be accessed from multiple sources of libmpathpersist, but not from libmultipath or other source files (those go into libmpathpersist_int.h). Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- libmpathpersist/mpath_persist_int.c | 16 ++++++++++++- libmpathpersist/mpath_updatepr.c | 1 + libmpathpersist/mpathpr.h | 37 ++++------------------------- 3 files changed, 20 insertions(+), 34 deletions(-) diff --git a/libmpathpersist/mpath_persist_int.c b/libmpathpersist/mpath_persist_int.c index 91bc35ae0..c6fc28719 100644 --- a/libmpathpersist/mpath_persist_int.c +++ b/libmpathpersist/mpath_persist_int.c @@ -33,7 +33,21 @@ #include "mpathpr.h" #include "mpath_pr_ioctl.h" - +struct prout_param { + char dev[FILE_NAME_SIZE]; + int rq_servact; + int rq_scope; + unsigned int rq_type; + struct prout_param_descriptor *paramp; + int noisy; + int status; +}; + +struct threadinfo { + int status; + pthread_t id; + struct prout_param param; +}; static int mpath_send_prin_activepath (char * dev, int rq_servact, struct prin_resp * resp, int noisy) diff --git a/libmpathpersist/mpath_updatepr.c b/libmpathpersist/mpath_updatepr.c index bdecaa05d..5824c1698 100644 --- a/libmpathpersist/mpath_updatepr.c +++ b/libmpathpersist/mpath_updatepr.c @@ -1,6 +1,7 @@ #include #include #include +#include #include #include #include diff --git a/libmpathpersist/mpathpr.h b/libmpathpersist/mpathpr.h index 59411e6fe..39a7d8ed0 100644 --- a/libmpathpersist/mpathpr.h +++ b/libmpathpersist/mpathpr.h @@ -1,42 +1,13 @@ #ifndef MPATHPR_H #define MPATHPR_H -#include "structs.h" /* FILE_NAME_SIZE */ - -struct prin_param { - char dev[FILE_NAME_SIZE]; - int rq_servact; - struct prin_resp *resp; - int noisy; - int status; -}; - -struct prout_param { - char dev[FILE_NAME_SIZE]; - int rq_servact; - int rq_scope; - unsigned int rq_type; - struct prout_param_descriptor *paramp; - int noisy; - int status; -}; - -struct threadinfo { - int status; - pthread_t id; - struct prout_param param; -}; - -int prin_do_scsi_ioctl(char * dev, int rq_servact, struct prin_resp * resp, int noisy); -int prout_do_scsi_ioctl( char * dev, int rq_servact, int rq_scope, - unsigned int rq_type, struct prout_param_descriptor *paramp, int noisy); -void * _mpath_pr_update (void *arg); -void dumpHex(const char* , int len, int no_ascii); +/* + * This header file contains symbols that are only used by + * libmpathpersist internally. + */ int update_prflag(char *mapname, int set); int update_prkey_flags(char *mapname, uint64_t prkey, uint8_t sa_flags); #define update_prkey(mapname, prkey) update_prkey_flags(mapname, prkey, 0) -void * mpath_alloc_prin_response(int prin_sa); -int update_map_pr(struct multipath *mpp); #endif From 69d5e814022eff674e6fe7fe020bf8dc29e25242 Mon Sep 17 00:00:00 2001 From: Xose Vazquez Perez Date: Sat, 18 Dec 2021 02:17:39 +0100 Subject: [PATCH 13/48] multipath-tools: fix misspellings Cc: Martin Wilck Cc: Benjamin Marzinski Cc: Christophe Varoqui Cc: DM-DEVEL ML Signed-off-by: Xose Vazquez Perez Signed-off-by: Martin Wilck Reviewed-by: Martin Wilck --- kpartx/crc32.c | 2 +- kpartx/gpt.c | 2 +- libdmmp/libdmmp.c | 2 +- libmpathcmd/mpath_cmd.h | 2 +- libmpathpersist/mpath_persist.h | 2 +- libmpathpersist/mpath_persist_int.c | 2 +- libmpathvalid/mpath_valid.h | 6 +++--- libmultipath/Makefile | 2 +- libmultipath/checkers.h | 2 +- libmultipath/checkers/directio.c | 2 +- libmultipath/dmparser.c | 2 +- libmultipath/foreign.h | 2 +- libmultipath/prioritizers/alua_spc3.h | 2 +- libmultipath/prioritizers/path_latency.c | 2 +- libmultipath/uevent.c | 2 +- multipath/multipath.conf.5 | 18 +++++++++--------- multipathd/dmevents.c | 2 +- multipathd/main.c | 8 ++++---- multipathd/main.h | 2 +- multipathd/multipathd.8 | 2 +- multipathd/uxlsnr.c | 2 +- tests/README.md | 2 +- tests/hwtable.c | 12 ++++++------ 23 files changed, 41 insertions(+), 41 deletions(-) diff --git a/kpartx/crc32.c b/kpartx/crc32.c index b23a0835a..e688f8e94 100644 --- a/kpartx/crc32.c +++ b/kpartx/crc32.c @@ -342,7 +342,7 @@ uint32_t attribute((pure)) crc32_be(uint32_t crc, unsigned char const *p, size_t * but again the multiple of the polynomial to subtract depends only on * the high bits, the high 8 bits in this case. * - * The multile we need in that case is the low 32 bits of a 40-bit + * The multiple we need in that case is the low 32 bits of a 40-bit * value whose high 8 bits are given, and which is a multiple of the * generator polynomial. This is simply the CRC-32 of the given * one-byte message. diff --git a/kpartx/gpt.c b/kpartx/gpt.c index f7fefb70e..34a910cf3 100644 --- a/kpartx/gpt.c +++ b/kpartx/gpt.c @@ -94,7 +94,7 @@ efi_crc32(const void *buf, unsigned long len) * * Description: Returns 1 if PMBR is valid, 0 otherwise. * Validity depends on two things: - * 1) MSDOS signature is in the last two bytes of the MBR + * 1) MS-DOS signature is in the last two bytes of the MBR * 2) One partition of type 0xEE is found */ static int diff --git a/libdmmp/libdmmp.c b/libdmmp/libdmmp.c index aafd5099b..51f1d8fc1 100644 --- a/libdmmp/libdmmp.c +++ b/libdmmp/libdmmp.c @@ -189,7 +189,7 @@ int dmmp_mpath_array_get(struct dmmp_context *ctx, j_token = json_tokener_new(); if (j_token == NULL) { rc = DMMP_ERR_BUG; - _error(ctx, "BUG: json_tokener_new() retuned NULL"); + _error(ctx, "BUG: json_tokener_new() returned NULL"); goto out; } j_obj = json_tokener_parse_ex(j_token, j_str, strlen(j_str) + 1); diff --git a/libmpathcmd/mpath_cmd.h b/libmpathcmd/mpath_cmd.h index ccfd35f26..30838b02d 100644 --- a/libmpathcmd/mpath_cmd.h +++ b/libmpathcmd/mpath_cmd.h @@ -80,7 +80,7 @@ int mpath_disconnect(int fd); * mpath_recv_reply() * * RETURNS: - * 0 on successs, and reply will either be NULL (if there was no + * 0 on success, and reply will either be NULL (if there was no * reply data), or point to the reply string, which must be freed by * the caller. -1 on failure (with errno set). */ diff --git a/libmpathpersist/mpath_persist.h b/libmpathpersist/mpath_persist.h index 9e9c0a82b..0046f1206 100644 --- a/libmpathpersist/mpath_persist.h +++ b/libmpathpersist/mpath_persist.h @@ -77,7 +77,7 @@ extern "C" { #define MPATH_PROTOCOL_ID_SAS 0x06 -/*Transport ID FORMATE CODE */ +/*Transport ID FORMAT CODE */ #define MPATH_WWUI_DEVICE_NAME 0x00 /* World wide unique initiator device name */ #define MPATH_WWUI_PORT_IDENTIFIER 0x40 /* World wide unique initiator port identifier */ diff --git a/libmpathpersist/mpath_persist_int.c b/libmpathpersist/mpath_persist_int.c index c6fc28719..e34fc32dc 100644 --- a/libmpathpersist/mpath_persist_int.c +++ b/libmpathpersist/mpath_persist_int.c @@ -546,7 +546,7 @@ static int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope, condlog (2, "%s: Path holding reservation is released.", mpp->wwid); return MPATH_PR_SUCCESS; } - condlog (2, "%s: Path holding reservation is not avialable.", mpp->wwid); + condlog (2, "%s: Path holding reservation is not available.", mpp->wwid); pr_buff = mpath_alloc_prin_response(MPATH_PRIN_RFSTAT_SA); if (!pr_buff){ diff --git a/libmpathvalid/mpath_valid.h b/libmpathvalid/mpath_valid.h index 63de4e1ce..ed06196ed 100644 --- a/libmpathvalid/mpath_valid.h +++ b/libmpathvalid/mpath_valid.h @@ -83,7 +83,7 @@ int mpathvalid_init(int verbosity, int log_style); /* * DESCRIPTION: - * Reread the multipath configuration files and reinitalize + * Reread the multipath configuration files and reinitialize * the device mapper multipath configuration. This function can * be called as many times as necessary. * @@ -95,8 +95,8 @@ int mpathvalid_reload_config(void); /* * DESCRIPTION: * Release the device mapper multipath configuration. This - * function must be called to cleanup resoures allocated by - * mpathvalid_init(). After calling this function, no futher + * function must be called to cleanup resources allocated by + * mpathvalid_init(). After calling this function, no further * libmpathvalid functions may be called. * * RETURNS: 0 = Success, -1 = Failure diff --git a/libmultipath/Makefile b/libmultipath/Makefile index d4af1a542..2e889dd83 100644 --- a/libmultipath/Makefile +++ b/libmultipath/Makefile @@ -61,7 +61,7 @@ nvme-lib.o: nvme-lib.c nvme-ioctl.c nvme-ioctl.h $(CC) $(CFLAGS) -Wno-unused-function -c -o $@ $< # there are lots of "unused parameters" in dict.c -# because not all handler / snprint methods nees all parameters +# because not all handler / snprint methods need all parameters dict.o: dict.c $(CC) $(CFLAGS) -Wno-unused-parameter -c -o $@ $< diff --git a/libmultipath/checkers.h b/libmultipath/checkers.h index f4600ed36..5d25a4219 100644 --- a/libmultipath/checkers.h +++ b/libmultipath/checkers.h @@ -156,7 +156,7 @@ void checker_disable (struct checker *); * * Rationale: * Path checkers that do I/O may hang forever. To avoid blocking, some - * checkers therefore use asyncronous, detached threads for checking + * checkers therefore use asynchronous, detached threads for checking * the paths. These threads may continue hanging if multipathd is stopped. * In this case, we can't unload the checker DSO at exit. In order to * avoid race conditions and crashes, the entry point of the thread diff --git a/libmultipath/checkers/directio.c b/libmultipath/checkers/directio.c index f73cbe38e..28b0cf515 100644 --- a/libmultipath/checkers/directio.c +++ b/libmultipath/checkers/directio.c @@ -23,7 +23,7 @@ /* Note: This checker type relies on the fact that only one checker can be run * at a time, since multiple checkers share the same aio_group, and must be - * able to modify other checker's async_reqs. If multple checkers become able + * able to modify other checker's async_reqs. If multiple checkers become able * to be run at the same time, this checker will need to add locking, and * probably polling on event fds, to deal with that */ diff --git a/libmultipath/dmparser.c b/libmultipath/dmparser.c index 74c921550..24ba4ac3d 100644 --- a/libmultipath/dmparser.c +++ b/libmultipath/dmparser.c @@ -102,7 +102,7 @@ int assemble_map(struct multipath *mp, char **params) } /* - * Caution callers: If this function encounters yet unkown path devices, it + * Caution callers: If this function encounters yet unknown path devices, it * adds them uninitialized to the mpp. * Call update_pathvec_from_dm() after this function to make sure * all data structures are in a sane state. diff --git a/libmultipath/foreign.h b/libmultipath/foreign.h index cf8f570d7..f547c14b2 100644 --- a/libmultipath/foreign.h +++ b/libmultipath/foreign.h @@ -42,7 +42,7 @@ struct foreign { /** * method: init(api, name) * Initialize foreign library, and check API compatibility - * return pointer to opaque internal data strucure if successful, + * return pointer to opaque internal data structure if successful, * NULL otherwise. * * @param[in] api: API version diff --git a/libmultipath/prioritizers/alua_spc3.h b/libmultipath/prioritizers/alua_spc3.h index f0a4bc441..08e992bd1 100644 --- a/libmultipath/prioritizers/alua_spc3.h +++ b/libmultipath/prioritizers/alua_spc3.h @@ -286,7 +286,7 @@ struct rtpg_tp_dscr { struct rtpg_tpg_dscr { unsigned char b0; /* x....... = pref(ered) port */ /* .xxx.... = reserved */ - /* ....xxxx = asymetric access state */ + /* ....xxxx = asymmetric access state*/ unsigned char b1; /* xxx..... = reserved */ /* ...x.... = LBA dependent support */ /* ....x... = unavailable support */ diff --git a/libmultipath/prioritizers/path_latency.c b/libmultipath/prioritizers/path_latency.c index 078226df5..e155f6dc5 100644 --- a/libmultipath/prioritizers/path_latency.c +++ b/libmultipath/prioritizers/path_latency.c @@ -284,7 +284,7 @@ int getprio(struct path *pp, char *args, unsigned int timeout) if (lg_avglatency > lg_maxavglatency) { pp_pl_log(2, - "%s: average latency (%lld us) is outside the thresold (%lld us)", + "%s: average latency (%lld us) is outside the threshold (%lld us)", pp->dev, (long long)pow(base_num, lg_avglatency), (long long)MAX_AVG_LATENCY); return DEFAULT_PRIORITY; diff --git a/libmultipath/uevent.c b/libmultipath/uevent.c index 70ad21759..b1f00b3e0 100644 --- a/libmultipath/uevent.c +++ b/libmultipath/uevent.c @@ -292,7 +292,7 @@ static bool uevent_can_merge(struct uevent *earlier, struct uevent *later) { /* merge paths uevents - * whose wwids exsit and are same + * whose wwids exist and are same * and actions are same, * and actions are addition or deletion */ diff --git a/multipath/multipath.conf.5 b/multipath/multipath.conf.5 index 88d2a1df9..80fa29201 100644 --- a/multipath/multipath.conf.5 +++ b/multipath/multipath.conf.5 @@ -251,7 +251,7 @@ The default is: \fBfailover\fR .B uid_attrs . Setting this option activates \fBmerging uevents\fR by WWID, which may improve -uevent processing effiency. Moreover, it's an alternative method to configure +uevent processing efficiency. Moreover, it's an alternative method to configure the udev properties to use for determining unique path identifiers (WWIDs). .RS .PP @@ -587,7 +587,7 @@ and multipathd. This is equivalent to ulimit \-n. A value of \fImax\fR will set this to the system limit from \fI/proc/sys/fs/nr_open\fR. If this is not set, the maximum number of open fds is taken from the calling process. It is usually 1024. To be safe, this should be set to the maximum number of paths plus 32, -if that number is greated than 1024. +if that number is greater than 1024. .RS .TP The default is: \fBmax\fR @@ -830,7 +830,7 @@ The default is: \fBno\fR .I yes and the SCSI layer has already attached a hardware_handler to the device, multipath will not force the device to use the hardware_handler specified by -mutipath.conf. If the SCSI layer has not attached a hardware handler, +multipath.conf. If the SCSI layer has not attached a hardware handler, multipath will continue to use its configured hardware handler. .RS .PP @@ -1097,8 +1097,8 @@ device is not configured to queue IO in this case, it can cause IO errors to occur, even though there are marginal paths available. However, if this option is set to \fIyes\fR, when one of the marginal path detecting methods determines that a path is marginal, it will be reinstated and placed in a -seperate pathgroup that will only be used after all the non-marginal pathgroups -have been tried first. This prevents the possibility of IO errors occuring +separate pathgroup that will only be used after all the non-marginal pathgroups +have been tried first. This prevents the possibility of IO errors occurring while marginal paths are still usable. After the path has been monitored for the configured time, and is declared healthy, it will be returned to its normal pathgroup. See "Shaky paths detection" below for more information. @@ -1140,7 +1140,7 @@ device path. .I smart This differs from \fIfind_multipaths yes\fR only in the way it treats new devices for which only one path has been -detected yet. When such a device is first encounted in udev rules, it is +detected yet. When such a device is first encountered in udev rules, it is treated as a multipath device. multipathd waits whether additional paths with the same WWID appears. If that happens, it sets up a multipath map. If it doesn\'t happen until a @@ -1420,7 +1420,7 @@ from later entries take precedence. The \fImultipath\fR subsection recognizes the following attributes: .TP 17 .B wwid -(Mandatory) World Wide Identifier. Detected multipath maps are matched agains this attribute. +(Mandatory) World Wide Identifier. Detected multipath maps are matched against this attribute. Note that, unlike the \fIwwid\fR attribute in the \fIblacklist\fR section, this is \fBnot\fR a regular expression or a substring; WWIDs must match exactly inside the multipaths section. @@ -1547,7 +1547,7 @@ device type, instead of the default "mpath". The vendor specific vpd page information, using the vpd page abbreviation. The vpd page abbreviation can be found by running \fIsg_vpd -e\fR. multipathd will use this information to gather device specific information that can be -displayed with the \fI%g\fR wilcard for the \fImultipathd show maps format\fR +displayed with the \fI%g\fR wildcard for the \fImultipathd show maps format\fR and \fImultipathd show paths format\fR commands. Currently only the \fBhp3par\fR vpd page is supported. .TP @@ -1795,7 +1795,7 @@ in most scenarios. Users who want to enable uevent merging must set .SH "Shaky paths detection" .\" ---------------------------------------------------------------------------- . -A common problem in SAN setups is the occurence of intermittent errors: a +A common problem in SAN setups is the occurrence of intermittent errors: a path is unreachable, then reachable again for a short time, disappears again, and so forth. This happens typically on unstable interconnects. It is undesirable to switch pathgroups unnecessarily on such frequent, unreliable diff --git a/multipathd/dmevents.c b/multipathd/dmevents.c index f035ee7f6..3a8596916 100644 --- a/multipathd/dmevents.c +++ b/multipathd/dmevents.c @@ -385,7 +385,7 @@ void *wait_dmevents (__attribute__((unused)) void *unused) if (!waiter) { - condlog(0, "dmevents waiter not intialized"); + condlog(0, "dmevents waiter not initialized"); return NULL; } diff --git a/multipathd/main.c b/multipathd/main.c index 6087b5f4f..4d611647d 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -1599,7 +1599,7 @@ uev_update_path (struct uevent *uev, struct vectors * vecs) condlog(0, "%s: spurious uevent, path not found", uev->kernel); } - /* pp->initalized must not be INIT_PARTIAL if needs_reinit is set */ + /* pp->initialized must not be INIT_PARTIAL if needs_reinit is set */ if (needs_reinit) retval = uev_add_path(uev, vecs, 1); return retval; @@ -1791,7 +1791,7 @@ uxlsnrloop (void * ap) /* * Wait for initial reconfiguration to finish, while - * hadling signals + * handling signals */ while (wait_for_state_change_if(DAEMON_CONFIGURE, 50) == DAEMON_CONFIGURE) @@ -2107,7 +2107,7 @@ static int check_path_reinstate_state(struct path * pp) { /* * This function is only called when the path state changes * from "bad" to "good". pp->state reflects the *previous* state. - * If this was "bad", we know that a failure must have occured + * If this was "bad", we know that a failure must have occurred * beforehand, and count that. * Note that we count path state _changes_ this way. If a path * remains in "bad" state, failure count is not increased. @@ -2277,7 +2277,7 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks) /* * provision a next check soonest, - * in case we exit abnormaly from here + * in case we exit abnormally from here */ pp->tick = checkint; diff --git a/multipathd/main.h b/multipathd/main.h index 459c7342e..e8bee8e6a 100644 --- a/multipathd/main.h +++ b/multipathd/main.h @@ -17,7 +17,7 @@ enum remove_path_result { REMOVE_PATH_FAILURE = 0x0, /* path could not be removed. It is still * part of the kernel map, but its state * is set to INIT_REMOVED, and it will be - * removed at the next possible occassion */ + * removed at the next possible occasion */ REMOVE_PATH_SUCCESS = 0x1, /* path was removed */ REMOVE_PATH_DELAY = 0x2, /* path is set to be removed later. it * currently still exists and is part of the diff --git a/multipathd/multipathd.8 b/multipathd/multipathd.8 index 293e7f224..1e318bdc7 100644 --- a/multipathd/multipathd.8 +++ b/multipathd/multipathd.8 @@ -82,7 +82,7 @@ multipathd. See .B \-w Since kernel 4.14 a new device-mapper event polling interface is used for updating multipath devices on dmevents. Use this flag to force it to use the old event -waiting method, based on creating a seperate thread for each device. +waiting method, based on creating a separate thread for each device. . . . diff --git a/multipathd/uxlsnr.c b/multipathd/uxlsnr.c index 912ac3c5f..c07367f3d 100644 --- a/multipathd/uxlsnr.c +++ b/multipathd/uxlsnr.c @@ -75,7 +75,7 @@ enum { }; #define POLLFD_CHUNK (4096 / sizeof(struct pollfd)) -/* Minimum mumber of pollfds to reserve for clients */ +/* Minimum number of pollfds to reserve for clients */ #define MIN_POLLS (POLLFD_CHUNK - POLLFDS_BASE) /* * Max number of client connections allowed diff --git a/tests/README.md b/tests/README.md index 47c0f0b22..f0d9be3e6 100644 --- a/tests/README.md +++ b/tests/README.md @@ -38,7 +38,7 @@ device will be opened in read-only mode; you don't need to worry about data loss. However, the user needs to specify a device to be used. Set the environment variable `DIO_TEST_DEV` to the path of the device. Alternatively, create a file `directio_test_dev` under -the `tests` directory containting a single line that sets this environment +the `tests` directory containing a single line that sets this environment variable in Bourne Shell syntax, like this: DIO_TEST_DEV=/dev/sdc3 diff --git a/tests/hwtable.c b/tests/hwtable.c index 6f5766f71..79bfa5f4f 100644 --- a/tests/hwtable.c +++ b/tests/hwtable.c @@ -421,7 +421,7 @@ static const struct key_value vnd_boo = { _vendor, "boo" }; static const struct key_value prd_baz = { _product, "baz" }; static const struct key_value wwid_test = { _wwid, default_wwid }; -/* Regular expresssions */ +/* Regular expressions */ static const struct key_value vnd__oo = { _vendor, ".oo" }; static const struct key_value vnd_t_oo = { _vendor, "^.oo" }; static const struct key_value prd_ba_ = { _product, "ba." }; @@ -603,7 +603,7 @@ static int setup_internal_nvme(void **state) } /* - * Device section with a simple entry qith double quotes ('foo:"bar"') + * Device section with a simple entry with double quotes ('foo:"bar"') */ static void test_quoted_hwe(const struct hwt_state *hwt) { @@ -967,7 +967,7 @@ static int setup_string_regex_hwe_dir(void **state) } /* - * Two identical device entries kv1 and kv2, trival regex ("string"). + * Two identical device entries kv1 and kv2, trivial regex ("string"). * Both are added to the main config file. * These entries are NOT merged. * This could happen in a large multipath.conf file. @@ -1003,7 +1003,7 @@ static int setup_2_ident_strings_hwe(void **state) } /* - * Two identical device entries kv1 and kv2, trival regex ("string"). + * Two identical device entries kv1 and kv2, trivial regex ("string"). * Both are added to an extra config file. * This could happen in a large multipath.conf file. * @@ -1043,7 +1043,7 @@ static int setup_2_ident_strings_both_dir(void **state) } /* - * Two identical device entries kv1 and kv2, trival regex ("string"). + * Two identical device entries kv1 and kv2, trivial regex ("string"). * Both are added to an extra config file. * An empty entry kv0 with the same string exists in the main config file. * @@ -1086,7 +1086,7 @@ static int setup_2_ident_strings_both_dir_w_prev(void **state) } /* - * Two identical device entries kv1 and kv2, trival regex ("string"). + * Two identical device entries kv1 and kv2, trivial regex ("string"). * kv1 is added to the main config file, kv2 to a config_dir file. * These entries are merged. * This case is more important as you may think, because it's equivalent From 1834dc46e7ab9b08b0919d189f7ae1642443c06d Mon Sep 17 00:00:00 2001 From: Benjamin Marzinski Date: Wed, 22 Dec 2021 19:32:33 -0600 Subject: [PATCH 14/48] libmultipath: fix DI_NOIO pathinfo PATH_PENDING handling pp->state should only be set to PATH_PENDING if it is currently in the PATH_UNCHECKED or PATH_WILD states. Otherwise, it should retain its current state. Fix this in pathinfo() calls with DI_NOIO, to match with the rest of the code. Signed-off-by: Benjamin Marzinski Reviewed-by: Martinn Wilck --- libmultipath/discovery.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c index 7d939ae08..6e5d1005f 100644 --- a/libmultipath/discovery.c +++ b/libmultipath/discovery.c @@ -2345,7 +2345,10 @@ int pathinfo(struct path *pp, struct config *conf, int mask) * Avoid any IO on the device itself. * simply use the path_offline() return as its state */ - pp->chkrstate = pp->state = path_state; + if (path_state != PATH_PENDING || + pp->state == PATH_UNCHECKED || + pp->state == PATH_WILD) + pp->chkrstate = pp->state = path_state; return PATHINFO_OK; } From 5c88969edb7744f1718b64eb11774feda4a87c37 Mon Sep 17 00:00:00 2001 From: Benjamin Marzinski Date: Wed, 22 Dec 2021 19:32:34 -0600 Subject: [PATCH 15/48] libmultipath: simplify out __set_no_path_retry() The only caller of __set_no_path_retry() is set_no_path_retry(), so remove the define and the unneeded arguments from the function. Signed-off-by: Benjamin Marzinski Reviewed-by: Martinn Wilck --- libmultipath/libmultipath.version | 2 +- libmultipath/structs_vec.c | 11 +++++------ libmultipath/structs_vec.h | 3 +-- 3 files changed, 7 insertions(+), 9 deletions(-) diff --git a/libmultipath/libmultipath.version b/libmultipath/libmultipath.version index 7cdce9fc0..2d0c9f26b 100644 --- a/libmultipath/libmultipath.version +++ b/libmultipath/libmultipath.version @@ -176,7 +176,7 @@ global: select_reservation_key; send_packet; set_max_fds; - __set_no_path_retry; + set_no_path_retry; set_path_removed; set_prkey; setup_map; diff --git a/libmultipath/structs_vec.c b/libmultipath/structs_vec.c index cb0ebae69..f1031f616 100644 --- a/libmultipath/structs_vec.c +++ b/libmultipath/structs_vec.c @@ -588,23 +588,22 @@ static void leave_recovery_mode(struct multipath *mpp) } } -void __set_no_path_retry(struct multipath *mpp, bool check_features) +void set_no_path_retry(struct multipath *mpp) { bool is_queueing = false; /* assign a value to make gcc happy */ - check_features = check_features && mpp->features != NULL; - if (check_features) + if (mpp->features) is_queueing = strstr(mpp->features, "queue_if_no_path"); switch (mpp->no_path_retry) { case NO_PATH_RETRY_UNDEF: break; case NO_PATH_RETRY_FAIL: - if (!check_features || is_queueing) + if (!mpp->features || is_queueing) dm_queue_if_no_path(mpp->alias, 0); break; case NO_PATH_RETRY_QUEUE: - if (!check_features || !is_queueing) + if (!mpp->features || !is_queueing) dm_queue_if_no_path(mpp->alias, 1); break; default: @@ -613,7 +612,7 @@ void __set_no_path_retry(struct multipath *mpp, bool check_features) * If in_recovery is set, leave_recovery_mode() takes * care of dm_queue_if_no_path. Otherwise, do it here. */ - if ((!check_features || !is_queueing) && + if ((!mpp->features || !is_queueing) && !mpp->in_recovery) dm_queue_if_no_path(mpp->alias, 1); leave_recovery_mode(mpp); diff --git a/libmultipath/structs_vec.h b/libmultipath/structs_vec.h index d33fe9848..3253f1bb1 100644 --- a/libmultipath/structs_vec.h +++ b/libmultipath/structs_vec.h @@ -11,8 +11,7 @@ struct vectors { struct mutex_lock lock; /* defined in lock.h */ }; -void __set_no_path_retry(struct multipath *mpp, bool check_features); -#define set_no_path_retry(mpp) __set_no_path_retry(mpp, true) +void set_no_path_retry(struct multipath *mpp); int adopt_paths (vector pathvec, struct multipath * mpp); void orphan_path (struct path * pp, const char *reason); From 727810804a94097456639e979a6a573d24111d14 Mon Sep 17 00:00:00 2001 From: Benjamin Marzinski Date: Wed, 22 Dec 2021 19:32:35 -0600 Subject: [PATCH 16/48] libmultipath: don't enter recovery mode with pending paths set_no_path_retry() could make a multipath device enter recovery mode simply because its paths were still in the PATH_PENDING state. This is possible when a multipath device is first created. After commit 2e61b8fb [libmultipath (coverity): Revert "setup_map: wait for pending path checkers to finish"] it has become much more likely. To avoid this, don't enter recovery mode as long as there are some paths in PATH_PENDING. Since path's can only be in this state if they've never had their state checker finish, this change will only effect recently created devices, whose checker hasn't completed yet. Signed-off-by: Benjamin Marzinski Reviewed-by: Martinn Wilck --- libmultipath/structs_vec.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libmultipath/structs_vec.c b/libmultipath/structs_vec.c index f1031f616..6c23df860 100644 --- a/libmultipath/structs_vec.c +++ b/libmultipath/structs_vec.c @@ -616,7 +616,7 @@ void set_no_path_retry(struct multipath *mpp) !mpp->in_recovery) dm_queue_if_no_path(mpp->alias, 1); leave_recovery_mode(mpp); - } else + } else if (pathcount(mpp, PATH_PENDING) == 0) enter_recovery_mode(mpp); break; } From dd6338ddc17c2315a188fcbf29ec5cfef1231c8e Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Thu, 23 Dec 2021 09:55:42 +0100 Subject: [PATCH 17/48] libmultipath.version: bump major version cc18bab ("libmultipath: embed dm_info in multipath structure") requires a major version bump. Signed-off-by: Martin Wilck --- libmultipath/libmultipath.version | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libmultipath/libmultipath.version b/libmultipath/libmultipath.version index 2d0c9f26b..d8cee0c3b 100644 --- a/libmultipath/libmultipath.version +++ b/libmultipath/libmultipath.version @@ -31,7 +31,7 @@ * The new version inherits the previous ones. */ -LIBMULTIPATH_13.0.0 { +LIBMULTIPATH_14.0.0 { global: /* symbols referenced by multipath and multipathd */ add_foreign; From a220dd114343680854aa53235550ee6b96e1910b Mon Sep 17 00:00:00 2001 From: Sam James Date: Mon, 3 Jan 2022 11:14:42 +0000 Subject: [PATCH 18/48] build: don't compress man pages This tends to complicate packaging for downstreams as we may choose to not compress man pages at all, use a different tool, or use different options. Most projects don't therefore bother compressing, so this change brings multipath-tool in line with others. Signed-off-by: Sam James Reviewed-by: Martin Wilck --- kpartx/Makefile | 7 +++---- libdmmp/Makefile | 14 ++++---------- libmpathcmd/Makefile | 2 +- libmpathpersist/Makefile | 16 ++++++---------- libmpathvalid/Makefile | 2 +- libmultipath/Makefile | 2 +- mpathpersist/Makefile | 7 +++---- multipath/Makefile | 12 +++++------- multipathd/Makefile | 7 +++---- 9 files changed, 27 insertions(+), 42 deletions(-) diff --git a/kpartx/Makefile b/kpartx/Makefile index 2906a984d..dadf11b96 100644 --- a/kpartx/Makefile +++ b/kpartx/Makefile @@ -21,7 +21,6 @@ all: $(EXEC) $(EXEC): $(OBJS) $(CC) $(CFLAGS) $(OBJS) -o $(EXEC) $(LDFLAGS) $(LIBDEPS) - $(GZIP) $(EXEC).8 > $(EXEC).8.gz install: $(EXEC) $(EXEC).8 $(INSTALL_PROGRAM) -d $(DESTDIR)$(bindir) @@ -33,11 +32,11 @@ install: $(EXEC) $(EXEC).8 $(INSTALL_PROGRAM) -m 644 kpartx.rules $(DESTDIR)$(libudevdir)/rules.d/66-kpartx.rules $(INSTALL_PROGRAM) -m 644 del-part-nodes.rules $(DESTDIR)$(libudevdir)/rules.d/68-del-part-nodes.rules $(INSTALL_PROGRAM) -d $(DESTDIR)$(man8dir) - $(INSTALL_PROGRAM) -m 644 $(EXEC).8.gz $(DESTDIR)$(man8dir) + $(INSTALL_PROGRAM) -m 644 $(EXEC).8 $(DESTDIR)$(man8dir) uninstall: $(RM) $(DESTDIR)$(bindir)/$(EXEC) - $(RM) $(DESTDIR)$(man8dir)/$(EXEC).8.gz + $(RM) $(DESTDIR)$(man8dir)/$(EXEC).8 $(RM) $(DESTDIR)$(libudevdir)/kpartx_id $(RM) $(DESTDIR)$(libudevdir)/rules.d/11-dm-parts.rules $(RM) $(DESTDIR)$(libudevdir)/rules.d/66-kpartx.rules @@ -45,7 +44,7 @@ uninstall: $(RM) $(DESTDIR)$(libudevdir)/rules.d/68-del-part-nodes.rules clean: dep_clean - $(RM) core *.o $(EXEC) *.gz + $(RM) core *.o $(EXEC) include $(wildcard $(OBJS:.o=.d)) diff --git a/libdmmp/Makefile b/libdmmp/Makefile index de6166894..00fc85296 100644 --- a/libdmmp/Makefile +++ b/libdmmp/Makefile @@ -21,7 +21,7 @@ CFLAGS += $(LIB_CFLAGS) -fvisibility=hidden -I$(libdmmpdir) -I$(mpathcmddir) \ LIBDEPS += $(shell $(PKGCONFIG) --libs json-c) -L$(mpathcmddir) -lmpathcmd -lpthread all: $(LIBS) doc -.PHONY: doc doc.gz clean install uninstall check speed_test dep_clean +.PHONY: doc clean install uninstall check speed_test dep_clean $(LIBS): $(OBJS) $(CC) $(LDFLAGS) $(SHARED_FLAGS) -Wl,-soname=$@ -o $@ $(OBJS) $(LIBDEPS) @@ -31,7 +31,7 @@ $(DEVLIB): $(LIBS) abi: $(DEVLIB:%.so=%.abi) -install: doc.gz +install: mkdir -p $(DESTDIR)$(usrlibdir) $(INSTALL_PROGRAM) -m 755 $(LIBS) $(DESTDIR)$(usrlibdir)/$(LIBS) $(INSTALL_PROGRAM) -m 644 -D \ @@ -45,7 +45,7 @@ install: doc.gz $(DESTDIR)$(pkgconfdir)/$(PKGFILE) perl -i -pe 's|__INCLUDEDIR__|$(includedir)|g' \ $(DESTDIR)$(pkgconfdir)/$(PKGFILE) - $(INSTALL_PROGRAM) -m 644 -t $(DESTDIR)$(man3dir) docs/man/*.3.gz + $(INSTALL_PROGRAM) -m 644 -t $(DESTDIR)$(man3dir) docs/man/*.3 uninstall: $(RM) $(DESTDIR)$(usrlibdir)/$(LIBS) @@ -58,8 +58,7 @@ uninstall: $(RM) $(DESTDIR)$(pkgconfdir)/$(PKGFILE) clean: dep_clean - $(RM) core *.a *.o *.gz *.so *.so.* *.abi $(NV_VERSION_SCRIPT) - $(RM) docs/man/*.gz + $(RM) core *.a *.o *.so *.so.* *.abi $(NV_VERSION_SCRIPT) $(MAKE) -C test clean include $(wildcard $(OBJS:.o=.d)) @@ -70,13 +69,8 @@ check: all speed_test: all $(MAKE) -C test speed_test -doc.gz: doc $(patsubst %,%.gz,$(wildcard docs/man/*.3)) - doc: docs/man/dmmp_strerror.3 -docs/man/%.3.gz: docs/man/%.3 - gzip -c $< >$@ - docs/man/dmmp_strerror.3: $(HEADERS) TEMPFILE=$(shell mktemp); \ cat $^ | perl docs/doc-preclean.pl >$$TEMPFILE; \ diff --git a/libmpathcmd/Makefile b/libmpathcmd/Makefile index 72cab1e32..0f83fe7b3 100644 --- a/libmpathcmd/Makefile +++ b/libmpathcmd/Makefile @@ -42,7 +42,7 @@ uninstall: $(RM) $(DESTDIR)$(includedir)/mpath_cmd.h clean: dep_clean - $(RM) core *.a *.o *.so *.so.* *.gz *.abi $(NV_VERSION_SCRIPT) + $(RM) core *.a *.o *.so *.so.* *.abi $(NV_VERSION_SCRIPT) include $(wildcard $(OBJS:.o=.d)) diff --git a/libmpathpersist/Makefile b/libmpathpersist/Makefile index fbc401e39..9843d96ec 100644 --- a/libmpathpersist/Makefile +++ b/libmpathpersist/Makefile @@ -12,7 +12,7 @@ LIBDEPS += -lmultipath -lmpathcmd -ldevmapper -lpthread -ldl OBJS = mpath_persist.o mpath_updatepr.o mpath_pr_ioctl.o mpath_persist_int.o -all: $(DEVLIB) man +all: $(DEVLIB) $(LIBS): $(OBJS) $(VERSION_SCRIPT) $(CC) $(LDFLAGS) $(SHARED_FLAGS) -Wl,-soname=$@ \ @@ -32,10 +32,6 @@ abi: $(LIBS:%.so.$(SONAME)=%-nv.abi) $(DEVLIB): $(LIBS) $(LN) $(LIBS) $@ -man: - $(GZIP) mpath_persistent_reserve_in.3 > mpath_persistent_reserve_in.3.gz - $(GZIP) mpath_persistent_reserve_out.3 > mpath_persistent_reserve_out.3.gz - install: all $(INSTALL_PROGRAM) -d $(DESTDIR)$(syslibdir) $(INSTALL_PROGRAM) -m 755 $(LIBS) $(DESTDIR)$(syslibdir)/$(LIBS) @@ -43,19 +39,19 @@ install: all $(INSTALL_PROGRAM) -m 755 -d $(DESTDIR)$(man3dir) $(INSTALL_PROGRAM) -m 755 -d $(DESTDIR)$(includedir) $(LN) $(LIBS) $(DESTDIR)$(syslibdir)/$(DEVLIB) - $(INSTALL_PROGRAM) -m 644 mpath_persistent_reserve_in.3.gz $(DESTDIR)$(man3dir) - $(INSTALL_PROGRAM) -m 644 mpath_persistent_reserve_out.3.gz $(DESTDIR)$(man3dir) + $(INSTALL_PROGRAM) -m 644 mpath_persistent_reserve_in.3 $(DESTDIR)$(man3dir) + $(INSTALL_PROGRAM) -m 644 mpath_persistent_reserve_out.3 $(DESTDIR)$(man3dir) $(INSTALL_PROGRAM) -m 644 mpath_persist.h $(DESTDIR)$(includedir) uninstall: $(RM) $(DESTDIR)$(syslibdir)/$(LIBS) - $(RM) $(DESTDIR)$(man3dir)/mpath_persistent_reserve_in.3.gz - $(RM) $(DESTDIR)$(man3dir)/mpath_persistent_reserve_out.3.gz + $(RM) $(DESTDIR)$(man3dir)/mpath_persistent_reserve_in.3 + $(RM) $(DESTDIR)$(man3dir)/mpath_persistent_reserve_out.3 $(RM) $(DESTDIR)$(includedir)/mpath_persist.h $(RM) $(DESTDIR)$(syslibdir)/$(DEVLIB) clean: dep_clean - $(RM) core *.a *.o *.so *.so.* *.gz *.abi $(NV_VERSION_SCRIPT) + $(RM) core *.a *.o *.so *.so.* *.abi $(NV_VERSION_SCRIPT) include $(wildcard $(OBJS:.o=.d)) diff --git a/libmpathvalid/Makefile b/libmpathvalid/Makefile index dce261033..fefeb2a3d 100644 --- a/libmpathvalid/Makefile +++ b/libmpathvalid/Makefile @@ -43,7 +43,7 @@ uninstall: $(RM) $(DESTDIR)$(includedir)/mpath_valid.h clean: dep_clean - $(RM) core *.a *.o *.so *.so.* *.gz *.abi $(NV_VERSION_SCRIPT) + $(RM) core *.a *.o *.so *.so.* *.abi $(NV_VERSION_SCRIPT) include $(wildcard $(OBJS:.o=.d)) diff --git a/libmultipath/Makefile b/libmultipath/Makefile index 2e889dd83..c7318cebe 100644 --- a/libmultipath/Makefile +++ b/libmultipath/Makefile @@ -110,7 +110,7 @@ uninstall: $(RM) $(DESTDIR)$(syslibdir)/$(DEVLIB) clean: dep_clean - $(RM) core *.a *.o *.so *.so.* *.gz *.abi nvme-ioctl.c nvme-ioctl.h $(NV_VERSION_SCRIPT) + $(RM) core *.a *.o *.so *.so.* *.abi nvme-ioctl.c nvme-ioctl.h $(NV_VERSION_SCRIPT) include $(wildcard $(OBJS:.o=.d)) diff --git a/mpathpersist/Makefile b/mpathpersist/Makefile index 51268010b..eb26970fa 100644 --- a/mpathpersist/Makefile +++ b/mpathpersist/Makefile @@ -14,22 +14,21 @@ all: $(EXEC) $(EXEC): $(OBJS) $(CC) $(OBJS) -o $(EXEC) $(LDFLAGS) $(CFLAGS) $(LIBDEPS) - $(GZIP) $(EXEC).8 > $(EXEC).8.gz install: $(INSTALL_PROGRAM) -d $(DESTDIR)$(bindir) $(INSTALL_PROGRAM) -m 755 $(EXEC) $(DESTDIR)$(bindir)/ $(INSTALL_PROGRAM) -d $(DESTDIR)$(man8dir) - $(INSTALL_PROGRAM) -m 644 $(EXEC).8.gz $(DESTDIR)$(man8dir) + $(INSTALL_PROGRAM) -m 644 $(EXEC).8 $(DESTDIR)$(man8dir) clean: dep_clean - $(RM) core *.o $(EXEC) *.gz + $(RM) core *.o $(EXEC) include $(wildcard $(OBJS:.o=.d)) uninstall: $(RM) $(DESTDIR)$(bindir)/$(EXEC) - $(RM) $(DESTDIR)$(man8dir)/$(EXEC).8.gz + $(RM) $(DESTDIR)$(man8dir)/$(EXEC).8 dep_clean: $(RM) $(OBJS:.o=.d) diff --git a/multipath/Makefile b/multipath/Makefile index 0828a8f72..76735537d 100644 --- a/multipath/Makefile +++ b/multipath/Makefile @@ -16,8 +16,6 @@ all: $(EXEC) $(EXEC): $(OBJS) $(multipathdir)/libmultipath.so $(mpathcmddir)/libmpathcmd.so $(CC) $(CFLAGS) $(OBJS) -o $(EXEC) $(LDFLAGS) $(LIBDEPS) - $(GZIP) $(EXEC).8 > $(EXEC).8.gz - $(GZIP) $(EXEC).conf.5 > $(EXEC).conf.5.gz install: $(INSTALL_PROGRAM) -d $(DESTDIR)$(bindir) @@ -26,19 +24,19 @@ install: $(INSTALL_PROGRAM) -m 644 11-dm-mpath.rules $(DESTDIR)$(udevrulesdir) $(INSTALL_PROGRAM) -m 644 $(EXEC).rules $(DESTDIR)$(libudevdir)/rules.d/56-multipath.rules $(INSTALL_PROGRAM) -d $(DESTDIR)$(man8dir) - $(INSTALL_PROGRAM) -m 644 $(EXEC).8.gz $(DESTDIR)$(man8dir) + $(INSTALL_PROGRAM) -m 644 $(EXEC).8 $(DESTDIR)$(man8dir) $(INSTALL_PROGRAM) -d $(DESTDIR)$(man5dir) - $(INSTALL_PROGRAM) -m 644 $(EXEC).conf.5.gz $(DESTDIR)$(man5dir) + $(INSTALL_PROGRAM) -m 644 $(EXEC).conf.5 $(DESTDIR)$(man5dir) uninstall: $(RM) $(DESTDIR)$(bindir)/$(EXEC) $(RM) $(DESTDIR)$(udevrulesdir)/11-dm-mpath.rules $(RM) $(DESTDIR)$(libudevdir)/rules.d/56-multipath.rules - $(RM) $(DESTDIR)$(man8dir)/$(EXEC).8.gz - $(RM) $(DESTDIR)$(man5dir)/$(EXEC).conf.5.gz + $(RM) $(DESTDIR)$(man8dir)/$(EXEC).8 + $(RM) $(DESTDIR)$(man5dir)/$(EXEC).conf.5 clean: dep_clean - $(RM) core *.o $(EXEC) *.gz + $(RM) core *.o $(EXEC) include $(wildcard $(OBJS:.o=.d)) diff --git a/multipathd/Makefile b/multipathd/Makefile index 393b6cbb2..8ea581807 100644 --- a/multipathd/Makefile +++ b/multipathd/Makefile @@ -40,7 +40,6 @@ all : $(EXEC) $(EXEC): $(OBJS) $(multipathdir)/libmultipath.so $(mpathcmddir)/libmpathcmd.so $(CC) $(CFLAGS) $(OBJS) $(LDFLAGS) -o $(EXEC) $(LIBDEPS) - $(GZIP) $(EXEC).8 > $(EXEC).8.gz cli_handlers.o: cli_handlers.c $(CC) $(CFLAGS) -Wno-unused-parameter -c -o $@ $< @@ -54,16 +53,16 @@ ifdef SYSTEMD $(INSTALL_PROGRAM) -m 644 $(EXEC).socket $(DESTDIR)$(unitdir) endif $(INSTALL_PROGRAM) -d $(DESTDIR)$(man8dir) - $(INSTALL_PROGRAM) -m 644 $(EXEC).8.gz $(DESTDIR)$(man8dir) + $(INSTALL_PROGRAM) -m 644 $(EXEC).8 $(DESTDIR)$(man8dir) uninstall: $(RM) $(DESTDIR)$(bindir)/$(EXEC) - $(RM) $(DESTDIR)$(man8dir)/$(EXEC).8.gz + $(RM) $(DESTDIR)$(man8dir)/$(EXEC).8 $(RM) $(DESTDIR)$(unitdir)/$(EXEC).service $(RM) $(DESTDIR)$(unitdir)/$(EXEC).socket clean: dep_clean - $(RM) core *.o $(EXEC) *.gz + $(RM) core *.o $(EXEC) include $(wildcard $(OBJS:.o=.d)) From c54d933264bc8053dacd4574fb38c58064cf5ac8 Mon Sep 17 00:00:00 2001 From: Xose Vazquez Perez Date: Tue, 4 Jan 2022 19:34:28 +0100 Subject: [PATCH 19/48] multipath-tools: remove duplicate headers Minimal clean up done with scripts/checkincludes.pl Cc: Martin Wilck Cc: Benjamin Marzinski Cc: Christophe Varoqui Cc: DM-DEVEL ML Signed-off-by: Xose Vazquez Perez Reviewed-by: Martin Wilck --- libdmmp/libdmmp.c | 1 - libmpathpersist/mpath_pr_ioctl.c | 1 - libmultipath/alias.c | 3 --- libmultipath/checkers/directio.c | 1 - libmultipath/checkers/tur.c | 1 - libmultipath/print.c | 1 - libmultipath/uevent.c | 1 - multipath/main.c | 1 - multipathd/main.c | 2 -- tests/strbuf.c | 1 - 10 files changed, 13 deletions(-) diff --git a/libdmmp/libdmmp.c b/libdmmp/libdmmp.c index 51f1d8fc1..0025e66d4 100644 --- a/libdmmp/libdmmp.c +++ b/libdmmp/libdmmp.c @@ -26,7 +26,6 @@ #include #include #include -#include #include #include #include diff --git a/libmpathpersist/mpath_pr_ioctl.c b/libmpathpersist/mpath_pr_ioctl.c index 17c528482..093ec71bb 100644 --- a/libmpathpersist/mpath_pr_ioctl.c +++ b/libmpathpersist/mpath_pr_ioctl.c @@ -10,7 +10,6 @@ #include #include #include -#include #include #include "mpath_pr_ioctl.h" #include "mpath_persist.h" diff --git a/libmultipath/alias.c b/libmultipath/alias.c index 87c33afcf..548a1184d 100644 --- a/libmultipath/alias.c +++ b/libmultipath/alias.c @@ -4,7 +4,6 @@ */ #include #include -#include #include #include #include @@ -19,8 +18,6 @@ #include "checkers.h" #include "structs.h" #include "config.h" -#include "util.h" -#include "errno.h" #include "devmapper.h" #include "strbuf.h" diff --git a/libmultipath/checkers/directio.c b/libmultipath/checkers/directio.c index 28b0cf515..bc7b7be52 100644 --- a/libmultipath/checkers/directio.c +++ b/libmultipath/checkers/directio.c @@ -12,7 +12,6 @@ #include #include #include -#include #include #include "checkers.h" diff --git a/libmultipath/checkers/tur.c b/libmultipath/checkers/tur.c index a4b4a2130..c93e46251 100644 --- a/libmultipath/checkers/tur.c +++ b/libmultipath/checkers/tur.c @@ -23,7 +23,6 @@ #include "../libmultipath/sg_include.h" #include "../libmultipath/util.h" #include "../libmultipath/time-util.h" -#include "../libmultipath/util.h" #define TUR_CMD_LEN 6 #define HEAVY_CHECK_COUNT 10 diff --git a/libmultipath/print.c b/libmultipath/print.c index c93fffd4d..ef4e05d22 100644 --- a/libmultipath/print.c +++ b/libmultipath/print.c @@ -8,7 +8,6 @@ #include #include #include -#include #include #include #include diff --git a/libmultipath/uevent.c b/libmultipath/uevent.c index b1f00b3e0..29e255723 100644 --- a/libmultipath/uevent.c +++ b/libmultipath/uevent.c @@ -41,7 +41,6 @@ #include #include #include -#include #include "debug.h" #include "list.h" diff --git a/multipath/main.c b/multipath/main.c index 0a9377e5f..d09f62db4 100644 --- a/multipath/main.c +++ b/multipath/main.c @@ -63,7 +63,6 @@ #include "time-util.h" #include "file.h" #include "valid.h" -#include "alias.h" /* * Return values of configure(), check_path_valid(), and main(). diff --git a/multipathd/main.c b/multipathd/main.c index 4d611647d..c241dad59 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -59,7 +59,6 @@ #include "prio.h" #include "wwids.h" #include "pgpolicies.h" -#include "uevent.h" #include "log.h" #include "uxsock.h" #include "alias.h" @@ -80,7 +79,6 @@ #include "waiter.h" #include "dmevents.h" #include "io_err_stat.h" -#include "wwids.h" #include "foreign.h" #include "../third-party/valgrind/drd.h" #include "init_unwinder.h" diff --git a/tests/strbuf.c b/tests/strbuf.c index 43a477db5..f8554da56 100644 --- a/tests/strbuf.c +++ b/tests/strbuf.c @@ -9,7 +9,6 @@ #include #include #include -#include #include #include #include "strbuf.h" From 23165664d3ed90c4dcae0a7c4b165706f40ea588 Mon Sep 17 00:00:00 2001 From: Sergei Trofimovich Date: Sun, 9 Jan 2022 09:28:13 +0000 Subject: [PATCH 20/48] kpartx/devmapper.c: fix unused-but-set variable error On gcc-12 build failed as: devmapper.c: In function 'dm_simplecmd': devmapper.c:61:13: error: unused variable 'udev_wait_flag' [-Werror=unused-variable] 61 | int udev_wait_flag = (task == DM_DEVICE_RESUME || | ^~~~~~~~~~~~~~ Fix error by hiding it's declaration under #ifdef that uses it. CC: Martin Wilck CC: Benjamin Marzinski Signed-off-by: Sergei Trofimovich Reviewed-by: Martin Wilck --- kpartx/devmapper.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kpartx/devmapper.c b/kpartx/devmapper.c index 3efd6dfcc..511c090f3 100644 --- a/kpartx/devmapper.c +++ b/kpartx/devmapper.c @@ -58,9 +58,9 @@ int dm_prereq(char * str, uint32_t x, uint32_t y, uint32_t z) int dm_simplecmd(int task, const char *name, int no_flush, uint16_t udev_flags) { int r = 0; +#ifdef LIBDM_API_COOKIE int udev_wait_flag = (task == DM_DEVICE_RESUME || task == DM_DEVICE_REMOVE); -#ifdef LIBDM_API_COOKIE uint32_t cookie = 0; #endif struct dm_task *dmt; From be16f42c608e58ac973fc1d43a101a7334ca9b21 Mon Sep 17 00:00:00 2001 From: Sergei Trofimovich Date: Sun, 9 Jan 2022 09:28:14 +0000 Subject: [PATCH 21/48] kpartx/devmapper.c: fix unused variable error Without the change the build fails as: devmapper.c:58:69: error: unused parameter 'udev_flags' [-Werror=unused-parameter] 58 | int dm_simplecmd(int task, const char *name, int no_flush, uint16_t udev_flags) | ~~~~~~~~~^~~~~~~~~~ The change adds __attribute__((used)) annotation to unused function parameter. CC: Martin Wilck CC: Benjamin Marzinski Signed-off-by: Sergei Trofimovich Reviewed-by: Martin Wilck Reviewed-by: Martin Wilck --- kpartx/devmapper.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/kpartx/devmapper.c b/kpartx/devmapper.c index 511c090f3..49ffd3103 100644 --- a/kpartx/devmapper.c +++ b/kpartx/devmapper.c @@ -18,6 +18,12 @@ #define MAX_PREFIX_LEN (_UUID_PREFIX_LEN + 4) #define PARAMS_SIZE 1024 +#ifdef LIBDM_API_COOKIE +# define __DM_API_COOKIE_UNUSED__ /* empty */ +#else +# define __DM_API_COOKIE_UNUSED__ __attribute__((unused)) +#endif + int dm_prereq(char * str, uint32_t x, uint32_t y, uint32_t z) { int r = 1; @@ -55,7 +61,7 @@ int dm_prereq(char * str, uint32_t x, uint32_t y, uint32_t z) return r; } -int dm_simplecmd(int task, const char *name, int no_flush, uint16_t udev_flags) +int dm_simplecmd(int task, const char *name, int no_flush, __DM_API_COOKIE_UNUSED__ uint16_t udev_flags) { int r = 0; #ifdef LIBDM_API_COOKIE From 921fabd48963e84e1c441327f8d8d542a3511f79 Mon Sep 17 00:00:00 2001 From: Sergei Trofimovich Date: Sun, 9 Jan 2022 10:29:06 +0000 Subject: [PATCH 22/48] multipath-tools: autodiscover libdevmapper.h headers On NixOS nothing is installed in /usr/include and instead lives in it's own prefix. pkg-config variables are expected to be used for installation discovery: $ pkg-config --variable=includedir devmapper /nix/store/c30fr0ahpa285sjkjgiinc2rr68ysmid-lvm2-2.03.14-dev/include The change switches libdevmapper.h discovery to pkg-config provided path. CC: Martin Wilck CC: Benjamin Marzinski Signed-off-by: Sergei Trofimovich Reviewed-by: Martin Wilck --- Makefile.inc | 8 ++++++++ kpartx/Makefile | 2 +- libmultipath/Makefile | 10 +++++----- multipathd/Makefile | 2 +- 4 files changed, 15 insertions(+), 7 deletions(-) diff --git a/Makefile.inc b/Makefile.inc index b340f2ae6..e4bf744d9 100644 --- a/Makefile.inc +++ b/Makefile.inc @@ -52,6 +52,14 @@ ifndef SYSTEMDPATH SYSTEMDPATH=usr/lib endif +ifndef DEVMAPPER_INCDIR + ifeq ($(shell $(PKGCONFIG) --modversion devmapper >/dev/null 2>&1 && echo 1), 1) + DEVMAPPER_INCDIR = $(shell $(PKGCONFIG) --variable=includedir devmapper) + else + DEVMAPPER_INCDIR = /usr/include + endif +endif + prefix = exec_prefix = $(prefix) usr_prefix = $(prefix) diff --git a/kpartx/Makefile b/kpartx/Makefile index dadf11b96..e9900fbcf 100644 --- a/kpartx/Makefile +++ b/kpartx/Makefile @@ -8,7 +8,7 @@ LDFLAGS += $(BIN_LDFLAGS) LIBDEPS += -ldevmapper -ifneq ($(call check_func,dm_task_set_cookie,/usr/include/libdevmapper.h),0) +ifneq ($(call check_func,dm_task_set_cookie,$(DEVMAPPER_INCDIR)/libdevmapper.h),0) CFLAGS += -DLIBDM_API_COOKIE endif diff --git a/libmultipath/Makefile b/libmultipath/Makefile index c7318cebe..d4f7d846f 100644 --- a/libmultipath/Makefile +++ b/libmultipath/Makefile @@ -21,15 +21,15 @@ ifdef SYSTEMD endif endif -ifneq ($(call check_func,dm_task_no_flush,/usr/include/libdevmapper.h),0) +ifneq ($(call check_func,dm_task_no_flush,$(DEVMAPPER_INCDIR)/libdevmapper.h),0) CFLAGS += -DLIBDM_API_FLUSH -D_GNU_SOURCE endif -ifneq ($(call check_func,dm_task_get_errno,/usr/include/libdevmapper.h),0) +ifneq ($(call check_func,dm_task_get_errno,$(DEVMAPPER_INCDIR)/libdevmapper.h),0) CFLAGS += -DLIBDM_API_GET_ERRNO endif -ifneq ($(call check_func,dm_task_set_cookie,/usr/include/libdevmapper.h),0) +ifneq ($(call check_func,dm_task_set_cookie,$(DEVMAPPER_INCDIR)/libdevmapper.h),0) CFLAGS += -DLIBDM_API_COOKIE endif @@ -37,11 +37,11 @@ ifneq ($(call check_func,udev_monitor_set_receive_buffer_size,/usr/include/libud CFLAGS += -DLIBUDEV_API_RECVBUF endif -ifneq ($(call check_func,dm_task_deferred_remove,/usr/include/libdevmapper.h),0) +ifneq ($(call check_func,dm_task_deferred_remove,$(DEVMAPPER_INCDIR)/libdevmapper.h),0) CFLAGS += -DLIBDM_API_DEFERRED endif -ifneq ($(call check_func,dm_hold_control_dev,/usr/include/libdevmapper.h),0) +ifneq ($(call check_func,dm_hold_control_dev,$(DEVMAPPER_INCDIR)/libdevmapper.h),0) CFLAGS += -DLIBDM_API_HOLD_CONTROL endif diff --git a/multipathd/Makefile b/multipathd/Makefile index 8ea581807..9f61b4ac9 100644 --- a/multipathd/Makefile +++ b/multipathd/Makefile @@ -1,6 +1,6 @@ include ../Makefile.inc -ifneq ($(call check_func,dm_task_get_errno,/usr/include/libdevmapper.h),0) +ifneq ($(call check_func,dm_task_get_errno,$(DEVMAPPER_INCDIR)/libdevmapper.h),0) CFLAGS += -DLIBDM_API_GET_ERRNO endif From 1017d59d9fa6f4cdcfb24947f0a260d05906dc22 Mon Sep 17 00:00:00 2001 From: Sergei Trofimovich Date: Sun, 9 Jan 2022 10:29:07 +0000 Subject: [PATCH 23/48] multipath-tools: autodiscover libudev.h headers On NixOS nothing is installed in /usr/include and instead lives in it's own prefix. pkg-config variables are expected to be used for installation discovery: $ pkg-config --variable=includedir libudev /nix/store/27mwkz5zhzw0gip8y7pvjyma5r0hzzaw-systemd-249.7-dev/include The change switches libudev.h discovery to pkg-config provided path. CC: Martin Wilck CC: Benjamin Marzinski Signed-off-by: Sergei Trofimovich Reviewed-by: Martin Wilck --- Makefile.inc | 8 ++++++++ libmultipath/Makefile | 2 +- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/Makefile.inc b/Makefile.inc index e4bf744d9..ebaff3f33 100644 --- a/Makefile.inc +++ b/Makefile.inc @@ -60,6 +60,14 @@ ifndef DEVMAPPER_INCDIR endif endif +ifndef LIBUDEV_INCDIR + ifeq ($(shell $(PKGCONFIG) --modversion libudev >/dev/null 2>&1 && echo 1), 1) + LIBUDEV_INCDIR = $(shell $(PKGCONFIG) --variable=includedir libudev) + else + LIBUDEV_INCDIR = /usr/include + endif +endif + prefix = exec_prefix = $(prefix) usr_prefix = $(prefix) diff --git a/libmultipath/Makefile b/libmultipath/Makefile index d4f7d846f..58e80e95c 100644 --- a/libmultipath/Makefile +++ b/libmultipath/Makefile @@ -33,7 +33,7 @@ ifneq ($(call check_func,dm_task_set_cookie,$(DEVMAPPER_INCDIR)/libdevmapper.h), CFLAGS += -DLIBDM_API_COOKIE endif -ifneq ($(call check_func,udev_monitor_set_receive_buffer_size,/usr/include/libudev.h),0) +ifneq ($(call check_func,udev_monitor_set_receive_buffer_size,$(LIBUDEV_INCDIR)/libudev.h),0) CFLAGS += -DLIBUDEV_API_RECVBUF endif From 193455b2c32eb781c47039d0f6bb251235c61e36 Mon Sep 17 00:00:00 2001 From: Sergei Trofimovich Date: Sun, 9 Jan 2022 10:29:08 +0000 Subject: [PATCH 24/48] multipath-tools: allow passing non-standard linux-headers location On NixOS nothing is installed in /usr/include and instead lives in it's own prefix. The change switches linux/nvme_ioctl.h discovery to user provided path. CC: Martin Wilck CC: Benjamin Marzinski Signed-off-by: Sergei Trofimovich Reviewed-by: Martin Wilck --- Makefile.inc | 5 +++++ libmultipath/prioritizers/Makefile | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/Makefile.inc b/Makefile.inc index ebaff3f33..f08f0a348 100644 --- a/Makefile.inc +++ b/Makefile.inc @@ -68,6 +68,11 @@ ifndef LIBUDEV_INCDIR endif endif +# Allow user to override default location. +ifndef LINUX_HEADERS_INCDIR + LINUX_HEADERS_INCDIR = /usr/include +endif + prefix = exec_prefix = $(prefix) usr_prefix = $(prefix) diff --git a/libmultipath/prioritizers/Makefile b/libmultipath/prioritizers/Makefile index 8d34ae322..16c639776 100644 --- a/libmultipath/prioritizers/Makefile +++ b/libmultipath/prioritizers/Makefile @@ -23,7 +23,7 @@ LIBS = \ libpriopath_latency.so \ libpriosysfs.so -ifneq ($(call check_file,/usr/include/linux/nvme_ioctl.h),0) +ifneq ($(call check_file,$(LINUX_HEADERS_INCDIR)/linux/nvme_ioctl.h),0) LIBS += libprioana.so CFLAGS += -I../nvme endif From 02bc8893f839ecc16f88920f286a7e756be6f70b Mon Sep 17 00:00:00 2001 From: Sergei Trofimovich Date: Sun, 9 Jan 2022 10:29:05 +0000 Subject: [PATCH 25/48] multipath-tools: avoid using GZIP Makefile variable `gzip` supports (deprecated) `GZIP` environment variable. If it's already present Makefile would override it and pass it through causing thre breakage: $ dev>GZIP=-n make gzip -9 -c mpath_persistent_reserve_in.3 > mpath_persistent_reserve_in.3.gz gzip: -c: option not valid in GZIP environment variable Try `gzip --help' for more information. Fix build by renaming GZIP variable to GZIP_PROG to avoid collision. mwilck: most of this is obsoleted by a220dd1 ("build: don't compress man pages") CC: Martin Wilck CC: Benjamin Marzinski Signed-off-by: Sergei Trofimovich Signed-off-by: Martin Wilck Reviewed-by: Martin Wilck --- Makefile.inc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.inc b/Makefile.inc index f08f0a348..a7d16dfdd 100644 --- a/Makefile.inc +++ b/Makefile.inc @@ -96,7 +96,7 @@ nvmedir = $(TOPDIR)/libmultipath/nvme includedir = $(prefix)/usr/include pkgconfdir = $(usrlibdir)/pkgconfig -GZIP = gzip -9 -c +GZIP_PROG = gzip -9 -c RM = rm -f LN = ln -sf INSTALL_PROGRAM = install From 353419e81529c52ec01088ede1c841a7f60c5e0b Mon Sep 17 00:00:00 2001 From: Benjamin Marzinski Date: Mon, 17 Jan 2022 21:27:31 -0600 Subject: [PATCH 26/48] libmultipath: fix disassemble status for historical-service-time PS The historical-service-time path selector prints out 2 path group status arguments. This is the only path selector that uses the group status arguments. All the others only have path status arguments. disassemble_status() was expecting the number of group status arguments to always be zero, causing it to fail at disassembling the status of devices that use historical-service-time path selector. Now multipath actually checks the number of group arguments, and skips them. Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- libmultipath/dmparser.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/libmultipath/dmparser.c b/libmultipath/dmparser.c index 24ba4ac3d..50d13c08d 100644 --- a/libmultipath/dmparser.c +++ b/libmultipath/dmparser.c @@ -436,9 +436,19 @@ int disassemble_status(const char *params, struct multipath *mpp) free(word); /* - * PG Status (discarded, would be '0' anyway) + * Path Selector Group Arguments */ - p += get_word(p, NULL); + p += get_word(p, &word); + + if (!word) + return 1; + + num_pg_args = atoi(word); + free(word); + + /* Ignore ps group arguments */ + for (j = 0; j < num_pg_args; j++) + p += get_word(p, NULL); p += get_word(p, &word); From ad16d0b88991370f1b504a7ba6c35087fcb48abe Mon Sep 17 00:00:00 2001 From: Benjamin Marzinski Date: Mon, 17 Jan 2022 21:27:32 -0600 Subject: [PATCH 27/48] libmultipath: make helper function to trigger path uevents Pull the code that checks if a path needs to trigger a uevent, and triggers, out of trigger_paths_udev_change() and into a new function, trigger_path_udev_change(). This function will be used separately by a future patch. No functional changes. Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- libmultipath/configure.c | 79 +++++++++++++++++++++------------------- libmultipath/configure.h | 1 + 2 files changed, 43 insertions(+), 37 deletions(-) diff --git a/libmultipath/configure.c b/libmultipath/configure.c index ec38f325d..eca11ba0b 100644 --- a/libmultipath/configure.c +++ b/libmultipath/configure.c @@ -503,11 +503,8 @@ void trigger_partitions_udev_change(struct udev_device *dev, } void -trigger_paths_udev_change(struct multipath *mpp, bool is_mpath) +trigger_path_udev_change(struct path *pp, bool is_mpath) { - struct pathgroup *pgp; - struct path *pp; - int i, j; /* * If a path changes from multipath to non-multipath, we must * synthesize an artificial "add" event, otherwise the LVM2 rules @@ -515,6 +512,45 @@ trigger_paths_udev_change(struct multipath *mpp, bool is_mpath) * irritate ourselves with an "add", so use "change". */ const char *action = is_mpath ? "change" : "add"; + const char *env; + + if (!pp->udev) + return; + /* + * Paths that are already classified as multipath + * members don't need another uevent. + */ + env = udev_device_get_property_value( + pp->udev, "DM_MULTIPATH_DEVICE_PATH"); + + if (is_mpath && env != NULL && !strcmp(env, "1")) { + /* + * If FIND_MULTIPATHS_WAIT_UNTIL is not "0", + * path is in "maybe" state and timer is running + * Send uevent now (see multipath.rules). + */ + env = udev_device_get_property_value( + pp->udev, "FIND_MULTIPATHS_WAIT_UNTIL"); + if (env == NULL || !strcmp(env, "0")) + return; + } else if (!is_mpath && + (env == NULL || !strcmp(env, "0"))) + return; + + condlog(3, "triggering %s uevent for %s (is %smultipath member)", + action, pp->dev, is_mpath ? "" : "no "); + sysfs_attr_set_value(pp->udev, "uevent", + action, strlen(action)); + trigger_partitions_udev_change(pp->udev, action, + strlen(action)); +} + +void +trigger_paths_udev_change(struct multipath *mpp, bool is_mpath) +{ + struct pathgroup *pgp; + struct path *pp; + int i, j; if (!mpp || !mpp->pg) return; @@ -522,39 +558,8 @@ trigger_paths_udev_change(struct multipath *mpp, bool is_mpath) vector_foreach_slot (mpp->pg, pgp, i) { if (!pgp->paths) continue; - vector_foreach_slot(pgp->paths, pp, j) { - const char *env; - - if (!pp->udev) - continue; - /* - * Paths that are already classified as multipath - * members don't need another uevent. - */ - env = udev_device_get_property_value( - pp->udev, "DM_MULTIPATH_DEVICE_PATH"); - - if (is_mpath && env != NULL && !strcmp(env, "1")) { - /* - * If FIND_MULTIPATHS_WAIT_UNTIL is not "0", - * path is in "maybe" state and timer is running - * Send uevent now (see multipath.rules). - */ - env = udev_device_get_property_value( - pp->udev, "FIND_MULTIPATHS_WAIT_UNTIL"); - if (env == NULL || !strcmp(env, "0")) - continue; - } else if (!is_mpath && - (env == NULL || !strcmp(env, "0"))) - continue; - - condlog(3, "triggering %s uevent for %s (is %smultipath member)", - action, pp->dev, is_mpath ? "" : "no "); - sysfs_attr_set_value(pp->udev, "uevent", - action, strlen(action)); - trigger_partitions_udev_change(pp->udev, action, - strlen(action)); - } + vector_foreach_slot(pgp->paths, pp, j) + trigger_path_udev_change(pp, is_mpath); } mpp->needs_paths_uevent = 0; diff --git a/libmultipath/configure.h b/libmultipath/configure.h index efe18b7d4..2bf73e651 100644 --- a/libmultipath/configure.h +++ b/libmultipath/configure.h @@ -56,6 +56,7 @@ int coalesce_paths (struct vectors *vecs, vector curmp, char * refwwid, int forc int get_refwwid (enum mpath_cmds cmd, const char *dev, enum devtypes dev_type, vector pathvec, char **wwid); struct udev_device *get_udev_device(const char *dev, enum devtypes dev_type); +void trigger_path_udev_change(struct path *pp, bool is_mpath); void trigger_paths_udev_change(struct multipath *mpp, bool is_mpath); void trigger_partitions_udev_change(struct udev_device *dev, const char *action, int len); From 9b634e88d37a3c340683ac1a3a0d358aa60d67c7 Mon Sep 17 00:00:00 2001 From: Benjamin Marzinski Date: Mon, 17 Jan 2022 21:27:33 -0600 Subject: [PATCH 28/48] multipathd: trigger udev change on path addition When a multipath device is created for the first time, there is a window where some path devices way be added to the multipath device, but never claimed in udev. This can allow other device owners, like lvm, to think they can use the device. When a multipath device is first created, all the existing paths that are not claimed by multipath have a uevent triggered so that they can get claimed. After that, multipath assumes all future paths added to the multipath device will have been claimed by multipath, since the device's WWID is now in the wwids file. This doesn't work for any paths that have already been processed by the multipath.rules udev rules before the multipath device was created. To close this window, when path device is added, and a matching multipath device already exists, multipathd now checks if the device is claimed by multipath, and if not, triggers a uevent to claim it. Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- libmultipath/libmultipath.version | 1 + multipathd/main.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/libmultipath/libmultipath.version b/libmultipath/libmultipath.version index d8cee0c3b..b663cb29a 100644 --- a/libmultipath/libmultipath.version +++ b/libmultipath/libmultipath.version @@ -210,6 +210,7 @@ global: sysfs_is_multipathed; timespeccmp; timespecsub; + trigger_path_udev_change; trigger_paths_udev_change; truncate_strbuf; udev; diff --git a/multipathd/main.c b/multipathd/main.c index c241dad59..44ca5b12a 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -1157,6 +1157,8 @@ ev_add_path (struct path * pp, struct vectors * vecs, int need_do_map) free_path(pp); return 1; } + if (mpp) + trigger_path_udev_change(pp, true); if (mpp && mpp->wait_for_udev && (pathcount(mpp, PATH_UP) > 0 || (pathcount(mpp, PATH_GHOST) > 0 && From 378932f296c153c634eaaaf1d6bf6e93ee8e7fc0 Mon Sep 17 00:00:00 2001 From: Benjamin Marzinski Date: Tue, 25 Jan 2022 23:35:25 -0600 Subject: [PATCH 29/48] libmultipath: use asprintf() to allocate prefixed_uuid gcc 12.0.1 failed building libmultipath due to a format-overflow false positive on 32-bit architectures. This isn't so surprising as format-overflow=2 is very aggressive in the assumptions it makes about the arguments. Here, it assumes that mpp->wwid could take up all the space that a pointer could point to, even if I add code to this function to explicitly null terminate mpp->wwid to fit in WWID_SIZE. To avoid this and simplify the function, switch from using calloc() and sprintf() to just using asprintf(). For reference, the gcc build error that this fixes is: devmapper.c: In function 'dm_addmap.constprop.0': devmapper.h:27:21: error: '%s' directive writing up to 2147483644 bytes into a region of size 2147483641 [-Werror=format-overflow=] 27 | #define UUID_PREFIX "mpath-" | ^~~~~~~~ devmapper.c:484:53: note: format string is defined here 484 | sprintf(prefixed_uuid, UUID_PREFIX "%s", mpp->wwid); | ^~ Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- libmultipath/devmapper.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/libmultipath/devmapper.c b/libmultipath/devmapper.c index 36038150e..2507f77fd 100644 --- a/libmultipath/devmapper.c +++ b/libmultipath/devmapper.c @@ -473,14 +473,11 @@ dm_addmap (int task, const char *target, struct multipath *mpp, dm_task_set_ro(dmt); if (task == DM_DEVICE_CREATE) { - prefixed_uuid = calloc(1, UUID_PREFIX_LEN + - strlen(mpp->wwid) + 1); - if (!prefixed_uuid) { + if (asprintf(&prefixed_uuid, UUID_PREFIX "%s", mpp->wwid) < 0) { condlog(0, "cannot create prefixed uuid : %s", strerror(errno)); goto addout; } - sprintf(prefixed_uuid, UUID_PREFIX "%s", mpp->wwid); if (!dm_task_set_uuid(dmt, prefixed_uuid)) goto freeout; dm_task_skip_lockfs(dmt); From 1ef4541ca0ceab4cdec43548c61542e532e0f08e Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Tue, 1 Feb 2022 16:59:49 +0100 Subject: [PATCH 30/48] multipath/Makefile: use $(udevrulesdir) Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- multipath/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/multipath/Makefile b/multipath/Makefile index 76735537d..c6903688c 100644 --- a/multipath/Makefile +++ b/multipath/Makefile @@ -22,7 +22,7 @@ install: $(INSTALL_PROGRAM) -m 755 $(EXEC) $(DESTDIR)$(bindir)/ $(INSTALL_PROGRAM) -d $(DESTDIR)$(udevrulesdir) $(INSTALL_PROGRAM) -m 644 11-dm-mpath.rules $(DESTDIR)$(udevrulesdir) - $(INSTALL_PROGRAM) -m 644 $(EXEC).rules $(DESTDIR)$(libudevdir)/rules.d/56-multipath.rules + $(INSTALL_PROGRAM) -m 644 $(EXEC).rules $(DESTDIR)$(udevrulesdir)/56-multipath.rules $(INSTALL_PROGRAM) -d $(DESTDIR)$(man8dir) $(INSTALL_PROGRAM) -m 644 $(EXEC).8 $(DESTDIR)$(man8dir) $(INSTALL_PROGRAM) -d $(DESTDIR)$(man5dir) From 41d0b66f6ca6cbdc18c95d209b85d96fdd9f6299 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Tue, 1 Feb 2022 17:01:40 +0100 Subject: [PATCH 31/48] multipath-tools: Makefile.inc: delete obsolete comment We haven't used the options since 0.4.5. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- Makefile.inc | 6 ------ 1 file changed, 6 deletions(-) diff --git a/Makefile.inc b/Makefile.inc index a7d16dfdd..c5f1fa55b 100644 --- a/Makefile.inc +++ b/Makefile.inc @@ -2,12 +2,6 @@ # Copyright (C) 2004 Christophe Varoqui, # -# -# Allow to force some libraries to be used statically. (Uncomment one of the -# following lines or define the values when calling make.) -# -# WITH_LOCAL_LIBDM = 1 -# WITH_LOCAL_LIBSYSFS = 1 # # Uncomment to disable libdmmp support # ENABLE_LIBDMMP = 0 From 92f0893ac1f0c9daf7eef8753e6bd9730bcce4c6 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Tue, 1 Feb 2022 17:04:19 +0100 Subject: [PATCH 32/48] multipath-tools: install modules-load.d/multipath.conf If multipath-tools is installed, we want to load the dm-multipath module early on, because multipath -u might spit out irritating error messages in syslog otherwise. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- Makefile.inc | 1 + multipath/Makefile | 3 +++ multipath/modules-load.conf | 3 +++ 3 files changed, 7 insertions(+) create mode 100644 multipath/modules-load.conf diff --git a/Makefile.inc b/Makefile.inc index c5f1fa55b..5223c96fe 100644 --- a/Makefile.inc +++ b/Makefile.inc @@ -73,6 +73,7 @@ usr_prefix = $(prefix) bindir = $(exec_prefix)/sbin libudevdir = $(prefix)/$(SYSTEMDPATH)/udev udevrulesdir = $(libudevdir)/rules.d +modulesloaddir = $(prefix)/$(SYSTEMDPATH)/modules-load.d multipathdir = $(TOPDIR)/libmultipath man8dir = $(prefix)/usr/share/man/man8 man5dir = $(prefix)/usr/share/man/man5 diff --git a/multipath/Makefile b/multipath/Makefile index c6903688c..015f73c3b 100644 --- a/multipath/Makefile +++ b/multipath/Makefile @@ -23,6 +23,8 @@ install: $(INSTALL_PROGRAM) -d $(DESTDIR)$(udevrulesdir) $(INSTALL_PROGRAM) -m 644 11-dm-mpath.rules $(DESTDIR)$(udevrulesdir) $(INSTALL_PROGRAM) -m 644 $(EXEC).rules $(DESTDIR)$(udevrulesdir)/56-multipath.rules + $(INSTALL_PROGRAM) -d $(DESTDIR)$(modulesloaddir) + $(INSTALL_PROGRAM) -m 644 modules-load.conf $(DESTDIR)$(modulesloaddir)/multipath.conf $(INSTALL_PROGRAM) -d $(DESTDIR)$(man8dir) $(INSTALL_PROGRAM) -m 644 $(EXEC).8 $(DESTDIR)$(man8dir) $(INSTALL_PROGRAM) -d $(DESTDIR)$(man5dir) @@ -31,6 +33,7 @@ install: uninstall: $(RM) $(DESTDIR)$(bindir)/$(EXEC) $(RM) $(DESTDIR)$(udevrulesdir)/11-dm-mpath.rules + $(RM) $(DESTDIR)$(modulesloaddir)/multipath.conf $(RM) $(DESTDIR)$(libudevdir)/rules.d/56-multipath.rules $(RM) $(DESTDIR)$(man8dir)/$(EXEC).8 $(RM) $(DESTDIR)$(man5dir)/$(EXEC).conf.5 diff --git a/multipath/modules-load.conf b/multipath/modules-load.conf new file mode 100644 index 000000000..b517d32ae --- /dev/null +++ b/multipath/modules-load.conf @@ -0,0 +1,3 @@ +# load dm-multipath early, both multipathd and multipath depend on it +# (note that multipath may be called from udev rules!) +dm-multipath From a1eabea75e8e0f6072f2b655cae25ec473b006c5 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Tue, 1 Feb 2022 17:06:28 +0100 Subject: [PATCH 33/48] multipathd.service: drop ExecStartPre for loading dm-multipath As we install a modules-load.d file now, no need to call modprobe in ExecStartPre any more. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- multipathd/multipathd.service | 1 - 1 file changed, 1 deletion(-) diff --git a/multipathd/multipathd.service b/multipathd/multipathd.service index ae98034c6..aec62dbb6 100644 --- a/multipathd/multipathd.service +++ b/multipathd/multipathd.service @@ -16,7 +16,6 @@ ConditionVirtualization=!container [Service] Type=notify NotifyAccess=main -ExecStartPre=-/sbin/modprobe dm-multipath ExecStart=/sbin/multipathd -d -s ExecReload=/sbin/multipathd reconfigure TasksMax=infinity From 2159f761f83a0f253c7bcf439fd319eab7ac81a9 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Tue, 1 Feb 2022 17:59:07 +0100 Subject: [PATCH 34/48] multipath: Makefile: modules-load.d file for SCSI device handlers The kernel's autoload mechanism for SCSI device handlers doesn't work during SCSI device probing. While it's possible to load and attach device handlers after probing, it has disadvantages: the handlers are useful for error handling even before multipathd has started, and at least up to kernel 5.17, the sysfs "access_state" attribute will remain invisible for already probed devices. Distributions will therefore want to make sure the handlers are either built-in in the kernel, or loaded early. Add functionality to create and install a modules-load.d file with a list of handlers to load. By default, the list is empty, and no file will be generated. The list can be specified at install-time like this: make SCSI_DH_MODULES_PRELOAD="scsi_dh_rdac scsi_dh_emc" install dracut automatically adds modules-load.d files and the modules they reference to the initramfs. Note: distributions that compile scsi_mod as a module may rather want to use a modprobe.d file with a statement like this: softdep scsi_mod post: scsi_dh_alua scsi_dh_rdac Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- Makefile.inc | 5 +++++ multipath/Makefile | 6 ++++++ multipath/scsi_dh.conf | 2 ++ 3 files changed, 13 insertions(+) create mode 100644 multipath/scsi_dh.conf diff --git a/Makefile.inc b/Makefile.inc index 5223c96fe..3342af638 100644 --- a/Makefile.inc +++ b/Makefile.inc @@ -9,6 +9,11 @@ # Uncomment to disable dmevents polling support # ENABLE_DMEVENTS_POLL = 0 +# List of scsi device handler modules to load on boot, e.g. +# SCSI_DH_MODULES_PRELOAD := scsi_dh_alua scsi_dh_rdac +SCSI_DH_MODULES_PRELOAD := + + PKGCONFIG ?= pkg-config ifeq ($(TOPDIR),) diff --git a/multipath/Makefile b/multipath/Makefile index 015f73c3b..c930499dd 100644 --- a/multipath/Makefile +++ b/multipath/Makefile @@ -29,11 +29,17 @@ install: $(INSTALL_PROGRAM) -m 644 $(EXEC).8 $(DESTDIR)$(man8dir) $(INSTALL_PROGRAM) -d $(DESTDIR)$(man5dir) $(INSTALL_PROGRAM) -m 644 $(EXEC).conf.5 $(DESTDIR)$(man5dir) +ifneq ($(SCSI_DH_MODULES_PRELOAD),) + $(INSTALL_PROGRAM) -m 644 scsi_dh.conf $(DESTDIR)$(modulesloaddir)/scsi_dh.conf + for _x in $(SCSI_DH_MODULES_PRELOAD); do echo "$$_x"; done \ + >>$(DESTDIR)$(modulesloaddir)/scsi_dh.conf +endif uninstall: $(RM) $(DESTDIR)$(bindir)/$(EXEC) $(RM) $(DESTDIR)$(udevrulesdir)/11-dm-mpath.rules $(RM) $(DESTDIR)$(modulesloaddir)/multipath.conf + $(RM) $(DESTDIR)$(modulesloaddir)/scsi_dh.conf $(RM) $(DESTDIR)$(libudevdir)/rules.d/56-multipath.rules $(RM) $(DESTDIR)$(man8dir)/$(EXEC).8 $(RM) $(DESTDIR)$(man5dir)/$(EXEC).conf.5 diff --git a/multipath/scsi_dh.conf b/multipath/scsi_dh.conf new file mode 100644 index 000000000..a13dd82e0 --- /dev/null +++ b/multipath/scsi_dh.conf @@ -0,0 +1,2 @@ +# Load SCSI device handler modules for multipath early +# This file may be empty From 5e5e51c71f7e8277d128683871c7021acc58cc55 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Tue, 1 Feb 2022 20:47:23 +0100 Subject: [PATCH 35/48] README.md: add basic information about building multipath-tools Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- README.md | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/README.md b/README.md index b15c265fd..154786258 100644 --- a/README.md +++ b/README.md @@ -52,6 +52,62 @@ To get latest devel code: Github page: https://github.com/opensvc/multipath-tools +Building multipath-tools +======================== + +Prerequisites: development packages of for `libdevmapper`, `libreadline`, +`libaio`, `libudev`, `libjson-c`, `liburcu`, and `libsystemd`. + +To build multipath-tools, type: + + make + make DESTDIR="/my/target/dir" install + +To uninstall, type: + + make uninstall + +Customizing the build +--------------------- + +The following variables can be passed to the `make` command line: + + * `ENABLE_LIBDMMP=0`: disable building libdmmp + * `ENABLE_DMEVENTS_POLL=0`: disable support for the device-mapper event + polling API. For use with pre-5.0 kernels that don't supprt dmevent polling + (but even if you don't use this option, multipath-tools will work with + these kernels). + * `SCSI_DH_MODULES_PRELOAD="(list)"`: specify a space-separated list of SCSI + device handler kernel modules to load early during boot. Some + multipath-tools functionality depends on these modules being loaded + early. This option causes a *modules-load.d(5)* configuration file to be + created, thus it depends on functionality provided by *systemd*. + This variable only matters for `make install`. + +Note: The usefulness of the preload list depends on the kernel configuration. +It's especially useful if `scsi_mod` is builtin but `scsi_dh_alua` and +other device handler modules are built as modules. If `scsi_mod` itself is compiled +as a module, it might make more sense to use a module softdep for the same +purpose. + +See `Makefile.inc` for additional variables to customize paths and compiler +flags. + +Special Makefile targets +------------------------ + +The following targets are intended for developers only. + + * `make test` to build and run the unit tests + * `make valgrind-test` to run the unit tests under valgrind + * `make abi` to create an XML representation of the ABI of the libraries in + the `abi/` subdirectory + * `make abi-test` to compare the ABI of a different multipath-tools version, + which must be stored in the `reference-abi/` subdirectory. If this test + fails, the ABI has changed wrt the reference. + * `make compile-commands.json` to create input for [clangd](https://clangd.llvm.org/). + + Add storage devices =================== From 544c4f4824de6132817015908cedf86f706c4d77 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Wed, 2 Feb 2022 12:24:03 +0100 Subject: [PATCH 36/48] libmultipath: hwtable: use ALUA for all LIO targets LIO supports ALUA for all backends. Use it by default. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- libmultipath/hwtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libmultipath/hwtable.c b/libmultipath/hwtable.c index c65e5e10d..bd1571032 100644 --- a/libmultipath/hwtable.c +++ b/libmultipath/hwtable.c @@ -1039,7 +1039,7 @@ static struct hwentry default_hw[] = { { /* Linux-IO Target */ .vendor = "(LIO-ORG|SUSE)", - .product = "RBD", + .product = ".", .hwhandler = "1 alua", .pgpolicy = GROUP_BY_PRIO, .pgfailback = -FAILBACK_IMMEDIATE, From 4f50e0e2a0dc9e216d56476ceb8a1fa7ac4d0d5b Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Mon, 7 Feb 2022 18:10:04 +0100 Subject: [PATCH 37/48] kpartx.rules: skip MD devices With the mdadm metadata format v1.0 (and DDF), the MD superblock is at the end of the device, keeping the partition table at the beginning. This may cause wrong partition mappings to be created by kpartx on RAID component devices. So far I've only seen ugly error messages, but at least in principle it can happen that kpartx wins a race with MD, and prevents correct setup of the MD device. Sample error messages: [ 4.029490] systemd-udevd[445]: dm-0: '/sbin/kpartx -un -p -part /dev/dm-0'(err) 'device-mapper: reload ioctl on 3600140508dbcf02acb448188d73ec97d-part1 failed: Device or resource busy' [ 4.075666] kernel: device-mapper: table: 254:1: linear: Device lookup failed [ 4.075945] kernel: device-mapper: ioctl: error adding target to table Skip creating partition mappings on MD components. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- kpartx/kpartx.rules | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kpartx/kpartx.rules b/kpartx/kpartx.rules index d7527d7d2..1969dee05 100644 --- a/kpartx/kpartx.rules +++ b/kpartx/kpartx.rules @@ -12,6 +12,9 @@ ENV{DM_UDEV_DISABLE_OTHER_RULES_FLAG}=="1", GOTO="kpartx_end" # Create dm tables for partitions on multipath devices. ENV{DM_UUID}!="mpath-?*", GOTO="mpath_kpartx_end" +# Ignore RAID members +ENV{ID_FS_TYPE}=="linux_raid_member|isw_raid_member|ddf_raid_member", GOTO="mpath_kpartx_end" + # DM_SUBSYSTEM_UDEV_FLAG1 is the "skip_kpartx" flag. # For events not generated by libdevmapper, we need to fetch it from db: # - "change" events with DM_ACTIVATION!="1" (e.g. partition table changes) From cfff03efbca753ef485ad717087464dced9c721a Mon Sep 17 00:00:00 2001 From: Muneendra Kumar Date: Wed, 9 Feb 2022 19:28:10 -0800 Subject: [PATCH 38/48] multipathd: handle fpin events This patch incorporates the functionality to handle FPIN ELS events present as part of FCTransport daemon (available in EPEL8) into the multipathd. This helps us to reduce the response time to react and take the necessary actions on receiving the FPIN events. This patch currently support FPIN-Li Events. It adds a new thread to listen for ELS frames from driver and on receiving the frame payload, push the payload to a list and notify the fpin_els_li_consumer thread to process it.Once consumer thread is notified, it returns to listen for more ELS frames from driver. The consumer thread process the ELS frames and moves the devices paths which are affected due to link integrity to marginal path groups. This also sets the associated portstate to marginal. The paths which are set to marginal path group will be unset on receiving the RSCN events [ MW: minor fixup for 32bit compilation ] Signed-off-by: Muneendra Kumar Signed-off-by: Benjamin Marzinski Signed-off-by: Martin Wilck Reviewed-by: Martin Wilck --- Makefile.inc | 13 + libmultipath/Makefile | 5 + libmultipath/dict.c | 56 +++- libmultipath/libmultipath.version | 1 + libmultipath/propsel.c | 47 ++- libmultipath/structs.h | 7 + multipath/multipath.conf.5 | 19 +- multipathd/Makefile | 10 + multipathd/fpin.h | 20 ++ multipathd/fpin_handlers.c | 540 ++++++++++++++++++++++++++++++ multipathd/main.c | 43 ++- 11 files changed, 745 insertions(+), 16 deletions(-) create mode 100644 multipathd/fpin.h create mode 100644 multipathd/fpin_handlers.c diff --git a/Makefile.inc b/Makefile.inc index 3342af638..d24da43e7 100644 --- a/Makefile.inc +++ b/Makefile.inc @@ -159,6 +159,19 @@ check_file = $(shell \ echo "$$found" \ ) +# Check whether a file contains a variable with name $1 in header file $2 +check_var = $(shell \ + if grep -Eq "(^|[[:blank:]])$1([[:blank:]]|=|$$)" "$2"; then \ + found=1; \ + status="yes"; \ + else \ + found=0; \ + status="no"; \ + fi; \ + echo 1>&2 "Checking for .. $1 in $2 ... $$status"; \ + echo "$$found" \ + ) + %.o: %.c @echo building $@ because of $? $(CC) $(CFLAGS) $(CPPFLAGS) -c -o $@ $< diff --git a/libmultipath/Makefile b/libmultipath/Makefile index 58e80e95c..77e954ae0 100644 --- a/libmultipath/Makefile +++ b/libmultipath/Makefile @@ -45,6 +45,11 @@ ifneq ($(call check_func,dm_hold_control_dev,$(DEVMAPPER_INCDIR)/libdevmapper.h) CFLAGS += -DLIBDM_API_HOLD_CONTROL endif +ifneq ($(call check_var,ELS_DTAG_LNK_INTEGRITY,$(LINUX_HEADERS_INCDIR)/scsi/fc/fc_els.h),0) + CFLAGS += -DFPIN_EVENT_HANDLER +endif + + OBJS = parser.o vector.o devmapper.o callout.o \ hwtable.o blacklist.o util.o dmparser.o config.o \ structs.o discovery.o propsel.o dict.o \ diff --git a/libmultipath/dict.c b/libmultipath/dict.c index 7ad9f6e26..2af9764c6 100644 --- a/libmultipath/dict.c +++ b/libmultipath/dict.c @@ -512,6 +512,59 @@ snprint_def_find_multipaths(struct config *conf, struct strbuf *buff, find_multipaths_optvals[conf->find_multipaths]); } +static const char * const marginal_pathgroups_optvals[] = { + [MARGINAL_PATHGROUP_OFF] = "off", + [MARGINAL_PATHGROUP_ON] = "on", +#ifdef FPIN_EVENT_HANDLER + [MARGINAL_PATHGROUP_FPIN] = "fpin", +#endif +}; + +static int +def_marginal_pathgroups_handler(struct config *conf, vector strvec, + const char *file, int line_nr) +{ + char *buff; + unsigned int i; + + buff = set_value(strvec); + if (!buff) + return 1; + for (i = MARGINAL_PATHGROUP_OFF; + i < ARRAY_SIZE(marginal_pathgroups_optvals); i++) { + if (marginal_pathgroups_optvals[i] != NULL && + !strcmp(buff, marginal_pathgroups_optvals[i])) { + conf->marginal_pathgroups = i; + break; + } + } + + if (i >= ARRAY_SIZE(marginal_pathgroups_optvals)) { + if (strcmp(buff, "no") == 0 || strcmp(buff, "0") == 0) + conf->marginal_pathgroups = MARGINAL_PATHGROUP_OFF; + else if (strcmp(buff, "yes") == 0 || strcmp(buff, "1") == 0) + conf->marginal_pathgroups = MARGINAL_PATHGROUP_ON; + /* This can only be true if FPIN_EVENT_HANDLER isn't defined, + * otherwise this check will have already happened above */ + else if (strcmp(buff, "fpin") == 0) + condlog(1, "%s line %d, support for \"fpin\" is not compiled in for marginal_pathgroups", file, line_nr); + else + condlog(1, "%s line %d, invalid value for marginal_pathgroups: \"%s\"", + file, line_nr, buff); + } + free(buff); + return 0; +} + +static int +snprint_def_marginal_pathgroups(struct config *conf, struct strbuf *buff, + const void *data) +{ + return append_strbuf_quoted(buff, + marginal_pathgroups_optvals[conf->marginal_pathgroups]); +} + + declare_def_handler(selector, set_str) declare_def_snprint_defstr(selector, print_str, DEFAULT_SELECTOR) declare_hw_handler(selector, set_str) @@ -1526,9 +1579,6 @@ declare_ovr_snprint(all_tg_pt, print_yes_no_undef) declare_hw_handler(all_tg_pt, set_yes_no_undef) declare_hw_snprint(all_tg_pt, print_yes_no_undef) -declare_def_handler(marginal_pathgroups, set_yes_no) -declare_def_snprint(marginal_pathgroups, print_yes_no) - declare_def_handler(recheck_wwid, set_yes_no_undef) declare_def_snprint_defint(recheck_wwid, print_yes_no_undef, DEFAULT_RECHECK_WWID) declare_ovr_handler(recheck_wwid, set_yes_no_undef) diff --git a/libmultipath/libmultipath.version b/libmultipath/libmultipath.version index b663cb29a..216f0eeff 100644 --- a/libmultipath/libmultipath.version +++ b/libmultipath/libmultipath.version @@ -56,6 +56,7 @@ global: check_foreign; cleanup_charp; cleanup_lock; + cleanup_mutex; cleanup_ucharp; close_fd; coalesce_paths; diff --git a/libmultipath/propsel.c b/libmultipath/propsel.c index a842fc39b..1419ec6f8 100644 --- a/libmultipath/propsel.c +++ b/libmultipath/propsel.c @@ -83,6 +83,8 @@ static const char cmdline_origin[] = "(setting: multipath command line [-p] flag)"; static const char autodetect_origin[] = "(setting: storage device autodetected)"; +static const char fpin_marginal_path_origin[] = + "(setting: overridden by marginal_path_fpin)"; static const char marginal_path_origin[] = "(setting: implied by marginal_path check)"; static const char delay_watch_origin[] = @@ -1035,9 +1037,12 @@ int select_san_path_err_threshold(struct config *conf, struct multipath *mp) const char *origin; STRBUF_ON_STACK(buff); - if (marginal_path_check_enabled(mp)) { + if (marginal_path_check_enabled(mp) || (conf->marginal_pathgroups == MARGINAL_PATHGROUP_FPIN)) { mp->san_path_err_threshold = NU_NO; - origin = marginal_path_origin; + if (conf->marginal_pathgroups == MARGINAL_PATHGROUP_FPIN) + origin = fpin_marginal_path_origin; + else + origin = marginal_path_origin; goto out; } mp_set_mpe(san_path_err_threshold); @@ -1058,9 +1063,12 @@ int select_san_path_err_forget_rate(struct config *conf, struct multipath *mp) const char *origin; STRBUF_ON_STACK(buff); - if (marginal_path_check_enabled(mp)) { + if (marginal_path_check_enabled(mp) || (conf->marginal_pathgroups == MARGINAL_PATHGROUP_FPIN)) { mp->san_path_err_forget_rate = NU_NO; - origin = marginal_path_origin; + if (conf->marginal_pathgroups == MARGINAL_PATHGROUP_FPIN) + origin = fpin_marginal_path_origin; + else + origin = marginal_path_origin; goto out; } mp_set_mpe(san_path_err_forget_rate); @@ -1082,9 +1090,12 @@ int select_san_path_err_recovery_time(struct config *conf, struct multipath *mp) const char *origin; STRBUF_ON_STACK(buff); - if (marginal_path_check_enabled(mp)) { + if (marginal_path_check_enabled(mp) || (conf->marginal_pathgroups == MARGINAL_PATHGROUP_FPIN)) { mp->san_path_err_recovery_time = NU_NO; - origin = marginal_path_origin; + if (conf->marginal_pathgroups == MARGINAL_PATHGROUP_FPIN) + origin = fpin_marginal_path_origin; + else + origin = marginal_path_origin; goto out; } mp_set_mpe(san_path_err_recovery_time); @@ -1106,6 +1117,12 @@ int select_marginal_path_err_sample_time(struct config *conf, struct multipath * const char *origin; STRBUF_ON_STACK(buff); + if (conf->marginal_pathgroups == MARGINAL_PATHGROUP_FPIN) { + mp->marginal_path_err_sample_time = NU_NO; + origin = fpin_marginal_path_origin; + goto out; + } + mp_set_mpe(marginal_path_err_sample_time); mp_set_ovr(marginal_path_err_sample_time); mp_set_hwe(marginal_path_err_sample_time); @@ -1129,6 +1146,12 @@ int select_marginal_path_err_rate_threshold(struct config *conf, struct multipat const char *origin; STRBUF_ON_STACK(buff); + if (conf->marginal_pathgroups == MARGINAL_PATHGROUP_FPIN) { + mp->marginal_path_err_rate_threshold = NU_NO; + origin = fpin_marginal_path_origin; + goto out; + } + mp_set_mpe(marginal_path_err_rate_threshold); mp_set_ovr(marginal_path_err_rate_threshold); mp_set_hwe(marginal_path_err_rate_threshold); @@ -1146,6 +1169,12 @@ int select_marginal_path_err_recheck_gap_time(struct config *conf, struct multip const char *origin; STRBUF_ON_STACK(buff); + if (conf->marginal_pathgroups == MARGINAL_PATHGROUP_FPIN) { + mp->marginal_path_err_recheck_gap_time = NU_NO; + origin = fpin_marginal_path_origin; + goto out; + } + mp_set_mpe(marginal_path_err_recheck_gap_time); mp_set_ovr(marginal_path_err_recheck_gap_time); mp_set_hwe(marginal_path_err_recheck_gap_time); @@ -1164,6 +1193,12 @@ int select_marginal_path_double_failed_time(struct config *conf, struct multipat const char *origin; STRBUF_ON_STACK(buff); + if (conf->marginal_pathgroups == MARGINAL_PATHGROUP_FPIN) { + mp->marginal_path_double_failed_time = NU_NO; + origin = fpin_marginal_path_origin; + goto out; + } + mp_set_mpe(marginal_path_double_failed_time); mp_set_ovr(marginal_path_double_failed_time); mp_set_hwe(marginal_path_double_failed_time); diff --git a/libmultipath/structs.h b/libmultipath/structs.h index 7f621941c..205cf64e9 100644 --- a/libmultipath/structs.h +++ b/libmultipath/structs.h @@ -111,6 +111,12 @@ enum find_multipaths_states { __FIND_MULTIPATHS_LAST, }; +enum marginal_pathgroups_mode { + MARGINAL_PATHGROUP_OFF = YN_NO, + MARGINAL_PATHGROUP_ON = YN_YES, + MARGINAL_PATHGROUP_FPIN, +}; + enum flush_states { FLUSH_UNDEF = YNU_UNDEF, FLUSH_DISABLED = YNU_NO, @@ -419,6 +425,7 @@ struct multipath { unsigned char prflag; int all_tg_pt; struct gen_multipath generic_mp; + bool fpin_must_reload; }; static inline int marginal_path_check_enabled(const struct multipath *mpp) diff --git a/multipath/multipath.conf.5 b/multipath/multipath.conf.5 index 80fa29201..746bb60c3 100644 --- a/multipath/multipath.conf.5 +++ b/multipath/multipath.conf.5 @@ -1088,20 +1088,26 @@ The default is: \fBno\fR . .TP .B marginal_pathgroups -If set to \fIno\fR, the \fIdelay_*_checks\fR, \fImarginal_path_*\fR, and +If set to \fIoff\fR, the \fIdelay_*_checks\fR, \fImarginal_path_*\fR, and \fIsan_path_err_*\fR options will keep marginal, or \(dqshaky\(dq, paths from being reinstated until they have been monitored for some time. This can cause situations where all non-marginal paths are down, and no paths are usable until multipathd detects this and reinstates a marginal path. If the multipath device is not configured to queue IO in this case, it can cause IO errors to occur, even though there are marginal paths available. However, if this -option is set to \fIyes\fR, when one of the marginal path detecting methods +option is set to \fIon\fR, when one of the marginal path detecting methods determines that a path is marginal, it will be reinstated and placed in a separate pathgroup that will only be used after all the non-marginal pathgroups have been tried first. This prevents the possibility of IO errors occurring while marginal paths are still usable. After the path has been monitored for the configured time, and is declared healthy, it will be returned to its -normal pathgroup. See "Shaky paths detection" below for more information. +normal pathgroup. +However if this option is set to \fIfpin\fR multipathd will receive fpin +notifications, set path states to "marginal" accordingly, and regroup paths +as described for "marginal_pathgroups yes". This option can't be used in combination +with other options for "Shaky path detection" (see below).If it is set to fpin, +marginal_path_xyz and san_path_err_xyz parameters are implicitly set to 0. +See "Shaky paths detection" below for more information. .RS .TP The default is: \fBno\fR @@ -1841,6 +1847,13 @@ increase and the threshold is never reached. Ticks are the time between path checks by multipathd, which is variable and controlled by the \fIpolling_interval\fR and \fImax_polling_interval\fR parameters. . +.TP +.B \(dqFPIN \(dq failure tracking +Fibre channel fabrics can notify hosts about fabric-level issues such +as integrity failures or congestion with so-called Fabric Performance +Impact Notifications (FPINs).On receiving the fpin notifications through ELS +multipathd will move the affected path and port states to marginal. +. .RS 8 .LP This method is \fBdeprecated\fR in favor of the \(dqmarginal_path\(dq failure diff --git a/multipathd/Makefile b/multipathd/Makefile index 9f61b4ac9..9a491445a 100644 --- a/multipathd/Makefile +++ b/multipathd/Makefile @@ -4,6 +4,10 @@ ifneq ($(call check_func,dm_task_get_errno,$(DEVMAPPER_INCDIR)/libdevmapper.h),0 CFLAGS += -DLIBDM_API_GET_ERRNO endif +ifneq ($(call check_var,ELS_DTAG_LNK_INTEGRITY,$(LINUX_HEADERS_INCDIR)/scsi/fc/fc_els.h),0) + CFLAGS += -DFPIN_EVENT_HANDLER + FPIN_SUPPORT = 1 +endif # # debugging stuff # @@ -34,6 +38,12 @@ endif OBJS = main.o pidfile.o uxlsnr.o uxclnt.o cli.o cli_handlers.o waiter.o \ dmevents.o init_unwinder.o +ifeq ($(FPIN_SUPPORT),1) +OBJS += fpin_handlers.o +endif + + + EXEC = multipathd all : $(EXEC) diff --git a/multipathd/fpin.h b/multipathd/fpin.h new file mode 100644 index 000000000..bfcc1ce2c --- /dev/null +++ b/multipathd/fpin.h @@ -0,0 +1,20 @@ +#ifndef __FPIN_H__ +#define __FPIN_H__ + +#ifdef FPIN_EVENT_HANDLER +void *fpin_fabric_notification_receiver(void *unused); +void *fpin_els_li_consumer(void *data); +void fpin_clean_marginal_dev_list(__attribute__((unused)) void *arg); +#else +static void *fpin_fabric_notification_receiver(__attribute__((unused))void *unused) +{ + return NULL; +} +static void *fpin_els_li_consumer(__attribute__((unused))void *data) +{ + return NULL; +} +/* fpin_clean_marginal_dev_list() is never called */ +#endif + +#endif diff --git a/multipathd/fpin_handlers.c b/multipathd/fpin_handlers.c new file mode 100644 index 000000000..aaf5655d3 --- /dev/null +++ b/multipathd/fpin_handlers.c @@ -0,0 +1,540 @@ +#include +#include +#include +#include +#include +#include +#include + +#include "parser.h" +#include "vector.h" +#include "structs.h" +#include "structs_vec.h" +#include "main.h" +#include "debug.h" +#include "util.h" +#include "sysfs.h" + +#include "fpin.h" +#include "devmapper.h" + +static pthread_cond_t fpin_li_cond = PTHREAD_COND_INITIALIZER; +static pthread_mutex_t fpin_li_mutex = PTHREAD_MUTEX_INITIALIZER; +static pthread_mutex_t fpin_li_marginal_dev_mutex = PTHREAD_MUTEX_INITIALIZER; + +static LIST_HEAD(els_marginal_list_head); +static LIST_HEAD(fpin_li_marginal_dev_list_head); + + +#define DEF_RX_BUF_SIZE 4096 +#define DEV_NAME_LEN 128 +#define FCH_EVT_LINKUP 0x2 +#define FCH_EVT_LINK_FPIN 0x501 +#define FCH_EVT_RSCN 0x5 + +#define list_first_entry(ptr, type, member) \ + list_entry((ptr)->next, type, member) + +/* max ELS frame Size */ +#define FC_PAYLOAD_MAXLEN 2048 + +struct els_marginal_list { + uint32_t event_code; + uint16_t host_num; + uint16_t length; + char payload[FC_PAYLOAD_MAXLEN]; + struct list_head node; +}; +/* Structure to store the marginal devices info */ +struct marginal_dev_list { + char dev_t[BLK_DEV_SIZE]; + uint32_t host_num; + struct list_head node; +}; + +static void _udev_device_unref(void *p) +{ + udev_device_unref(p); +} + + +/*set/unset the path state to marginal*/ +static int fpin_set_pathstate(struct path *pp, bool set) +{ + const char *action = set ? "set" : "unset"; + + if (!pp || !pp->mpp || !pp->mpp->alias) + return -1; + + condlog(3, "\n%s: %s marginal path %s (fpin)", + action, pp->mpp->alias, pp->dev_t); + pp->marginal = set; + pp->mpp->fpin_must_reload = true; + return 0; +} + +/* This will unset marginal state of a device*/ +static void fpin_path_unsetmarginal(char *devname, struct vectors *vecs) +{ + struct path *pp; + + pp = find_path_by_dev(vecs->pathvec, devname); + if (!pp) + pp = find_path_by_devt(vecs->pathvec, devname); + + fpin_set_pathstate(pp, false); +} + +/*This will set the marginal state of a device*/ +static int fpin_path_setmarginal(struct path *pp) +{ + return fpin_set_pathstate(pp, true); +} + +/* Unsets all the devices in the list from marginal state */ +static void +fpin_unset_marginal_dev(uint32_t host_num, struct vectors *vecs) +{ + struct marginal_dev_list *tmp_marg = NULL; + struct marginal_dev_list *marg = NULL; + struct multipath *mpp; + int ret = 0; + int i; + + pthread_cleanup_push(cleanup_lock, &vecs->lock); + lock(&vecs->lock); + pthread_testcancel(); + + pthread_mutex_lock(&fpin_li_marginal_dev_mutex); + pthread_cleanup_push(cleanup_mutex, &fpin_li_marginal_dev_mutex); + pthread_testcancel(); + if (list_empty(&fpin_li_marginal_dev_list_head)) { + condlog(4, "Marginal List is empty\n"); + goto empty; + } + list_for_each_entry_safe(marg, tmp_marg, &fpin_li_marginal_dev_list_head, node) { + if (marg->host_num != host_num) + continue; + condlog(4, " unsetting marginal dev: is %s %d\n", + tmp_marg->dev_t, tmp_marg->host_num); + fpin_path_unsetmarginal(marg->dev_t, vecs); + list_del(&marg->node); + free(marg); + } +empty: + pthread_cleanup_pop(1); + /* walk backwards because reload_and_sync_map() can remove mpp */ + vector_foreach_slot_backwards(vecs->mpvec, mpp, i) { + if (mpp->fpin_must_reload) { + ret = reload_and_sync_map(mpp, vecs, 0); + if (ret == 2) + condlog(2, "map removed during reload"); + else + mpp->fpin_must_reload = false; + } + } + pthread_cleanup_pop(1); +} + +/* + * On Receiving the frame from HBA driver, insert the frame into link + * integrity frame list which will be picked up later by consumer thread for + * processing. + */ +static int +fpin_els_add_li_frame(struct fc_nl_event *fc_event) +{ + struct els_marginal_list *els_mrg = NULL; + int ret = 0; + + if (fc_event->event_datalen > FC_PAYLOAD_MAXLEN) + return -EINVAL; + + pthread_mutex_lock(&fpin_li_mutex); + pthread_cleanup_push(cleanup_mutex, &fpin_li_mutex); + pthread_testcancel(); + els_mrg = calloc(1, sizeof(struct els_marginal_list)); + if (els_mrg != NULL) { + els_mrg->host_num = fc_event->host_no; + els_mrg->event_code = fc_event->event_code; + els_mrg->length = fc_event->event_datalen; + memcpy(els_mrg->payload, &(fc_event->event_data), fc_event->event_datalen); + list_add_tail(&els_mrg->node, &els_marginal_list_head); + pthread_cond_signal(&fpin_li_cond); + } else + ret = -ENOMEM; + pthread_cleanup_pop(1); + return ret; + +} + +/*Sets the rport port_state to marginal*/ +static void fpin_set_rport_marginal(struct udev_device *rport_dev) +{ + sysfs_attr_set_value(rport_dev, "port_state", + "Marginal", strlen("Marginal")); +} + +/*Add the marginal devices info into the list*/ +static void +fpin_add_marginal_dev_info(uint32_t host_num, char *devname) +{ + struct marginal_dev_list *newdev = NULL; + + newdev = calloc(1, sizeof(struct marginal_dev_list)); + if (newdev != NULL) { + newdev->host_num = host_num; + strlcpy(newdev->dev_t, devname, BLK_DEV_SIZE); + condlog(4, "\n%s hostno %d devname %s\n", __func__, + host_num, newdev->dev_t); + pthread_mutex_lock(&fpin_li_marginal_dev_mutex); + list_add_tail(&(newdev->node), + &fpin_li_marginal_dev_list_head); + pthread_mutex_unlock(&fpin_li_marginal_dev_mutex); + } +} + +/* + * This function goes through the vecs->pathvec, and for + * each path, check that the host number, + * the target WWPN associated with the path matches + * with the els wwpn and sets the path and port state to + * Marginal + */ +static int fpin_chk_wwn_setpath_marginal(uint16_t host_num, struct vectors *vecs, + uint64_t els_wwpn) +{ + struct path *pp; + struct multipath *mpp; + int i, k; + char rport_id[42]; + const char *value = NULL; + struct udev_device *rport_dev = NULL; + uint64_t wwpn; + int ret = 0; + + pthread_cleanup_push(cleanup_lock, &vecs->lock); + lock(&vecs->lock); + pthread_testcancel(); + + vector_foreach_slot(vecs->pathvec, pp, k) { + /* Checks the host number and also for the SCSI FCP */ + if (pp->sg_id.proto_id != SCSI_PROTOCOL_FCP || host_num != pp->sg_id.host_no) + continue; + sprintf(rport_id, "rport-%d:%d-%d", + pp->sg_id.host_no, pp->sg_id.channel, pp->sg_id.transport_id); + rport_dev = udev_device_new_from_subsystem_sysname(udev, + "fc_remote_ports", rport_id); + if (!rport_dev) { + condlog(2, "%s: No fc_remote_port device for '%s'", pp->dev, + rport_id); + continue; + } + pthread_cleanup_push(_udev_device_unref, rport_dev); + value = udev_device_get_sysattr_value(rport_dev, "port_name"); + if (!value) + goto unref; + + if (value) + wwpn = strtol(value, NULL, 16); + /* + * If the port wwpn matches sets the path and port state + * to marginal + */ + if (wwpn == els_wwpn) { + ret = fpin_path_setmarginal(pp); + if (ret < 0) + goto unref; + fpin_set_rport_marginal(rport_dev); + fpin_add_marginal_dev_info(host_num, pp->dev); + } +unref: + pthread_cleanup_pop(1); + } + /* walk backwards because reload_and_sync_map() can remove mpp */ + vector_foreach_slot_backwards(vecs->mpvec, mpp, i) { + if (mpp->fpin_must_reload) { + ret = reload_and_sync_map(mpp, vecs, 0); + if (ret == 2) + condlog(2, "map removed during reload"); + else + mpp->fpin_must_reload = false; + } + } + pthread_cleanup_pop(1); + return ret; +} + +/* + * This function loops around all the impacted wwns received as part of els + * frame and sets the associated path and port states to marginal. + */ +static int +fpin_parse_li_els_setpath_marginal(uint16_t host_num, struct fc_tlv_desc *tlv, + struct vectors *vecs) +{ + uint32_t wwn_count = 0, iter = 0; + uint64_t wwpn; + struct fc_fn_li_desc *li_desc = (struct fc_fn_li_desc *)tlv; + int count = 0; + int ret = 0; + + /* Update the wwn to list */ + wwn_count = be32_to_cpu(li_desc->pname_count); + condlog(4, "Got wwn count as %d\n", wwn_count); + + for (iter = 0; iter < wwn_count; iter++) { + wwpn = be64_to_cpu(li_desc->pname_list[iter]); + ret = fpin_chk_wwn_setpath_marginal(host_num, vecs, wwpn); + if (ret < 0) + condlog(2, "failed to set the path marginal associated with wwpn: 0x%" PRIx64 "\n", wwpn); + + count++; + } + return count; +} + +/* + * This function process the ELS frame received from HBA driver, + * and sets the path associated with the port wwn to marginal + * and also set the port state to marginal. + */ +static int +fpin_process_els_frame(uint16_t host_num, char *fc_payload, struct vectors *vecs) +{ + + int count = -1; + struct fc_els_fpin *fpin = (struct fc_els_fpin *)fc_payload; + struct fc_tlv_desc *tlv; + + tlv = (struct fc_tlv_desc *)&fpin->fpin_desc[0]; + + /* + * Parse the els frame and set the affected paths and port + * state to marginal + */ + count = fpin_parse_li_els_setpath_marginal(host_num, tlv, vecs); + if (count <= 0) + condlog(4, "Could not find any WWNs, ret = %d\n", + count); + return count; +} + +/* + * This function process the FPIN ELS frame received from HBA driver, + * and push the frame to appropriate frame list. Currently we have only FPIN + * LI frame list. + */ +static int +fpin_handle_els_frame(struct fc_nl_event *fc_event) +{ + int ret = -1; + uint32_t els_cmd; + struct fc_els_fpin *fpin = (struct fc_els_fpin *)&fc_event->event_data; + struct fc_tlv_desc *tlv; + uint32_t dtag; + + els_cmd = (uint32_t)fc_event->event_data; + tlv = (struct fc_tlv_desc *)&fpin->fpin_desc[0]; + dtag = be32_to_cpu(tlv->desc_tag); + condlog(4, "Got CMD in add as 0x%x fpin_cmd 0x%x dtag 0x%x\n", + els_cmd, fpin->fpin_cmd, dtag); + + if ((fc_event->event_code == FCH_EVT_LINK_FPIN) || + (fc_event->event_code == FCH_EVT_LINKUP) || + (fc_event->event_code == FCH_EVT_RSCN)) { + + if (els_cmd == ELS_FPIN) { + /* + * Check the type of fpin by checking the tag info + * At present we are supporting only LI events + */ + if (dtag == ELS_DTAG_LNK_INTEGRITY) { + /*Push the Payload to FPIN frame queue. */ + ret = fpin_els_add_li_frame(fc_event); + if (ret != 0) + condlog(0, "Failed to process LI frame with error %d\n", + ret); + } else { + condlog(4, "Unsupported FPIN received 0x%x\n", dtag); + return ret; + } + } else { + /*Push the Payload to FPIN frame queue. */ + ret = fpin_els_add_li_frame(fc_event); + if (ret != 0) + condlog(0, "Failed to process Linkup/RSCN event with error %d evnt %d\n", + ret, fc_event->event_code); + } + } else + condlog(4, "Invalid command received: 0x%x\n", els_cmd); + return ret; +} + +/*cleans the global marginal dev list*/ +void fpin_clean_marginal_dev_list(__attribute__((unused)) void *arg) +{ + struct marginal_dev_list *tmp_marg = NULL; + + pthread_mutex_lock(&fpin_li_marginal_dev_mutex); + while (!list_empty(&fpin_li_marginal_dev_list_head)) { + tmp_marg = list_first_entry(&fpin_li_marginal_dev_list_head, + struct marginal_dev_list, node); + list_del(&tmp_marg->node); + free(tmp_marg); + } + pthread_mutex_unlock(&fpin_li_marginal_dev_mutex); +} + +/* Cleans the global els marginal list */ +static void fpin_clean_els_marginal_list(void *arg) +{ + struct list_head *head = (struct list_head *)arg; + struct els_marginal_list *els_marg; + + while (!list_empty(head)) { + els_marg = list_first_entry(head, struct els_marginal_list, + node); + list_del(&els_marg->node); + free(els_marg); + } +} + +static void rcu_unregister(__attribute__((unused)) void *param) +{ + rcu_unregister_thread(); +} +/* + * This is the FPIN ELS consumer thread. The thread sleeps on pthread cond + * variable unless notified by fpin_fabric_notification_receiver thread. + * This thread is only to process FPIN-LI ELS frames. A new thread and frame + * list will be added if any more ELS frames types are to be supported. + */ +void *fpin_els_li_consumer(void *data) +{ + struct list_head marginal_list_head; + int ret = 0; + uint16_t host_num; + struct els_marginal_list *els_marg; + uint32_t event_code; + struct vectors *vecs = (struct vectors *)data; + + pthread_cleanup_push(rcu_unregister, NULL); + rcu_register_thread(); + pthread_cleanup_push(fpin_clean_marginal_dev_list, NULL); + INIT_LIST_HEAD(&marginal_list_head); + pthread_cleanup_push(fpin_clean_els_marginal_list, + (void *)&marginal_list_head); + for ( ; ; ) { + pthread_mutex_lock(&fpin_li_mutex); + pthread_cleanup_push(cleanup_mutex, &fpin_li_mutex); + pthread_testcancel(); + while (list_empty(&els_marginal_list_head)) + pthread_cond_wait(&fpin_li_cond, &fpin_li_mutex); + + if (!list_empty(&els_marginal_list_head)) { + condlog(4, "Invoke List splice tail\n"); + list_splice_tail_init(&els_marginal_list_head, &marginal_list_head); + } + pthread_cleanup_pop(1); + + while (!list_empty(&marginal_list_head)) { + els_marg = list_first_entry(&marginal_list_head, + struct els_marginal_list, node); + host_num = els_marg->host_num; + event_code = els_marg->event_code; + /* Now finally process FPIN LI ELS Frame */ + condlog(4, "Got a new Payload buffer, processing it\n"); + if ((event_code == FCH_EVT_LINKUP) || (event_code == FCH_EVT_RSCN)) + fpin_unset_marginal_dev(host_num, vecs); + else { + ret = fpin_process_els_frame(host_num, els_marg->payload, vecs); + if (ret <= 0) + condlog(0, "ELS frame processing failed with ret %d\n", ret); + } + list_del(&els_marg->node); + free(els_marg); + + } + } + + pthread_cleanup_pop(1); + pthread_cleanup_pop(1); + pthread_cleanup_pop(1); + return NULL; +} + +static void receiver_cleanup_list(__attribute__((unused)) void *arg) +{ + pthread_mutex_lock(&fpin_li_mutex); + fpin_clean_els_marginal_list(&els_marginal_list_head); + pthread_mutex_unlock(&fpin_li_mutex); +} + +/* + * Listen for ELS frames from driver. on receiving the frame payload, + * push the payload to a list, and notify the fpin_els_li_consumer thread to + * process it. Once consumer thread is notified, return to listen for more ELS + * frames from driver. + */ +void *fpin_fabric_notification_receiver(__attribute__((unused))void *unused) +{ + int ret; + long fd; + uint32_t els_cmd; + struct fc_nl_event *fc_event = NULL; + struct sockaddr_nl fc_local; + unsigned char buf[DEF_RX_BUF_SIZE] __attribute__((aligned(sizeof(uint64_t)))); + size_t plen = 0; + + pthread_cleanup_push(rcu_unregister, NULL); + rcu_register_thread(); + + pthread_cleanup_push(receiver_cleanup_list, NULL); + fd = socket(PF_NETLINK, SOCK_DGRAM, NETLINK_SCSITRANSPORT); + if (fd < 0) { + condlog(0, "fc socket error %ld", fd); + return NULL; + } + + pthread_cleanup_push(close_fd, (void *)fd); + memset(&fc_local, 0, sizeof(fc_local)); + fc_local.nl_family = AF_NETLINK; + fc_local.nl_groups = ~0; + fc_local.nl_pid = getpid(); + ret = bind(fd, (struct sockaddr *)&fc_local, sizeof(fc_local)); + if (ret == -1) { + condlog(0, "fc socket bind error %d\n", ret); + goto out; + } + for ( ; ; ) { + condlog(4, "Waiting for ELS...\n"); + ret = read(fd, buf, DEF_RX_BUF_SIZE); + if (ret < 0) { + condlog(0, "failed to read the els frame (%d)", ret); + continue; + } + condlog(4, "Got a new request %d\n", ret); + if (!NLMSG_OK((struct nlmsghdr *)buf, (unsigned int)ret)) { + condlog(0, "bad els frame read (%d)", ret); + continue; + } + /* Push the frame to appropriate frame list */ + plen = NLMSG_PAYLOAD((struct nlmsghdr *)buf, 0); + fc_event = (struct fc_nl_event *)NLMSG_DATA(buf); + if (plen < sizeof(*fc_event)) { + condlog(0, "too short (%d) to be an FC event", ret); + continue; + } + els_cmd = (uint32_t)fc_event->event_data; + condlog(4, "Got host no as %d, event 0x%x, len %d evntnum %d evntcode %d\n", + fc_event->host_no, els_cmd, fc_event->event_datalen, + fc_event->event_num, fc_event->event_code); + fpin_handle_els_frame(fc_event); + } +out: + pthread_cleanup_pop(1); + pthread_cleanup_pop(1); + pthread_cleanup_pop(1); + return NULL; +} diff --git a/multipathd/main.c b/multipathd/main.c index 44ca5b12a..6bc5178dc 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -16,6 +16,7 @@ #include #include #include +#include "fpin.h" #ifdef USE_SYSTEMD #include #endif @@ -132,9 +133,11 @@ static bool __delayed_reconfig; pid_t daemon_pid; static pthread_mutex_t config_lock = PTHREAD_MUTEX_INITIALIZER; static pthread_cond_t config_cond; -static pthread_t check_thr, uevent_thr, uxlsnr_thr, uevq_thr, dmevent_thr; +static pthread_t check_thr, uevent_thr, uxlsnr_thr, uevq_thr, dmevent_thr, + fpin_thr, fpin_consumer_thr; static bool check_thr_started, uevent_thr_started, uxlsnr_thr_started, - uevq_thr_started, dmevent_thr_started; + uevq_thr_started, dmevent_thr_started, fpin_thr_started, + fpin_consumer_thr_started; static int pid_fd = -1; static inline enum daemon_status get_running_state(void) @@ -2879,7 +2882,9 @@ reconfigure (struct vectors * vecs) conf->sequence_nr = old->sequence_nr + 1; rcu_assign_pointer(multipath_conf, conf); call_rcu(&old->rcu, rcu_free_config); - +#ifdef FPIN_EVENT_HANDLER + fpin_clean_marginal_dev_list(NULL); +#endif configure(vecs); @@ -3098,6 +3103,11 @@ static void cleanup_threads(void) pthread_cancel(uevq_thr); if (dmevent_thr_started) pthread_cancel(dmevent_thr); + if (fpin_thr_started) + pthread_cancel(fpin_thr); + if (fpin_consumer_thr_started) + pthread_cancel(fpin_consumer_thr); + if (check_thr_started) pthread_join(check_thr, NULL); @@ -3109,6 +3119,11 @@ static void cleanup_threads(void) pthread_join(uevq_thr, NULL); if (dmevent_thr_started) pthread_join(dmevent_thr, NULL); + if (fpin_thr_started) + pthread_join(fpin_thr, NULL); + if (fpin_consumer_thr_started) + pthread_join(fpin_consumer_thr, NULL); + /* * As all threads are joined now, and we're in DAEMON_SHUTDOWN @@ -3202,6 +3217,7 @@ child (__attribute__((unused)) void *param) char *envp; enum daemon_status state; int exit_code = 1; + int fpin_marginal_paths = 0; init_unwinder(); mlockall(MCL_CURRENT | MCL_FUTURE); @@ -3280,7 +3296,10 @@ child (__attribute__((unused)) void *param) setscheduler(); set_oom_adj(); - +#ifdef FPIN_EVENT_HANDLER + if (conf->marginal_pathgroups == MARGINAL_PATHGROUP_FPIN) + fpin_marginal_paths = 1; +#endif /* * Startup done, invalidate configuration */ @@ -3348,6 +3367,22 @@ child (__attribute__((unused)) void *param) goto failed; } else uevq_thr_started = true; + + if (fpin_marginal_paths) { + if ((rc = pthread_create(&fpin_thr, &misc_attr, + fpin_fabric_notification_receiver, NULL))) { + condlog(0, "failed to create the fpin receiver thread: %d", rc); + goto failed; + } else + fpin_thr_started = true; + + if ((rc = pthread_create(&fpin_consumer_thr, + &misc_attr, fpin_els_li_consumer, vecs))) { + condlog(0, "failed to create the fpin consumer thread thread: %d", rc); + goto failed; + } else + fpin_consumer_thr_started = true; + } pthread_attr_destroy(&misc_attr); while (1) { From 3af023b501b6453e1ad0dad6be1c4920a6809bdd Mon Sep 17 00:00:00 2001 From: Benjamin Marzinski Date: Mon, 14 Feb 2022 10:24:36 -0600 Subject: [PATCH 39/48] multipathd: disallow changing to/from fpin marginal paths on reconfig Setting marginal_pathgroups to fpin causes two new threads to be created when multipathd starts. Turning it on after multipathd starts up won't cause the theads to start, and turing it off won't keep the threads from working. So disallow changing marginal_pathgroups to/from "fpin" on reconfigure. Signed-off-by: Benjamin Marzinski Cc: Muneendra Kumar Reviewed-by: Martin Wilck --- multipath/multipath.conf.5 | 13 ++++++++----- multipathd/main.c | 9 +++++++++ 2 files changed, 17 insertions(+), 5 deletions(-) diff --git a/multipath/multipath.conf.5 b/multipath/multipath.conf.5 index 746bb60c3..605b46e0c 100644 --- a/multipath/multipath.conf.5 +++ b/multipath/multipath.conf.5 @@ -1102,15 +1102,18 @@ have been tried first. This prevents the possibility of IO errors occurring while marginal paths are still usable. After the path has been monitored for the configured time, and is declared healthy, it will be returned to its normal pathgroup. -However if this option is set to \fIfpin\fR multipathd will receive fpin +If this option is set to \fIfpin\fR, multipathd will receive fpin notifications, set path states to "marginal" accordingly, and regroup paths -as described for "marginal_pathgroups yes". This option can't be used in combination -with other options for "Shaky path detection" (see below).If it is set to fpin, -marginal_path_xyz and san_path_err_xyz parameters are implicitly set to 0. +as described for \fIon\fR. This option can't be used in combination +with other options for "Shaky path detection" (see below). \fBNote:\fR If this +is set to \fIfpin\fR, the \fImarginal_path_*\fR and \fIsan_path_err_*\fR +options are implicitly set to \fIno\fP. Also, this option cannot be switched +either to or from \fIfpin\fR on a multipathd reconfigure. multipathd must be +restarted for the change to take effect. See "Shaky paths detection" below for more information. .RS .TP -The default is: \fBno\fR +The default is: \fBoff\fR .RE . . diff --git a/multipathd/main.c b/multipathd/main.c index 6bc5178dc..f2c0b2807 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2850,6 +2850,7 @@ int reconfigure (struct vectors * vecs) { struct config * old, *conf; + int old_marginal_pathgroups; conf = load_config(DEFAULT_CONFIGFILE); if (!conf) @@ -2879,6 +2880,14 @@ reconfigure (struct vectors * vecs) uxsock_timeout = conf->uxsock_timeout; old = rcu_dereference(multipath_conf); + old_marginal_pathgroups = old->marginal_pathgroups; + if ((old_marginal_pathgroups == MARGINAL_PATHGROUP_FPIN) != + (conf->marginal_pathgroups == MARGINAL_PATHGROUP_FPIN)) { + condlog(1, "multipathd must be restarted to turn %s fpin marginal paths", + (old_marginal_pathgroups == MARGINAL_PATHGROUP_FPIN)? + "off" : "on"); + conf->marginal_pathgroups = old_marginal_pathgroups; + } conf->sequence_nr = old->sequence_nr + 1; rcu_assign_pointer(multipath_conf, conf); call_rcu(&old->rcu, rcu_free_config); From 852a1dfd9488395e9a907ff882a630f11d10935c Mon Sep 17 00:00:00 2001 From: Steffen Maier Date: Tue, 15 Feb 2022 20:45:46 +0100 Subject: [PATCH 40/48] libmultipath: support host adapter name lookup for s390x ccw bus There are also (FCP) HBAs that appear on a bus different from PCI. Complements v0.6.0 commit 01ab2a468ea2 ("libmultipath: Add additional path wildcards"). With that we can easily get the full FCP addressing triplet (HBA, WWPN, LUN) from multipath tools without additional tools and correlation: $ multipathd -k'show paths format "%w|%i|%a|%r"' uuid |hcil |host adapter|target WWPN 36005076400820293e8000000000000a0|1:0:3:160 |0.0.5080 |0x500507680b25c449 36005076400820293e8000000000000a0|1:0:4:160 |0.0.5080 |0x500507680b25c448 36005076400820293e8000000000000a0|58:0:3:160 |0.0.50c0 |0x500507680b26c449 36005076400820293e8000000000000a0|58:0:4:160 |0.0.50c0 |0x500507680b26c448 ^^^^^^^^ instead of [undef] Make helper function static and generalize it to also cover CCW bus in addition to the already supported PCI bus. Had to move now static helper in front of its only user. While at it, use string functions that are safe against buffer overflows. As a side effect this patch theoretically also enables group by host adapter for s390x based on v0.6.0 commit a28e61e5cc9a ("Crafted ordering of child paths for round robin path selector"). Reviewed-by: Benjamin Block Reviewed-by: Martin Wilck Signed-off-by: Steffen Maier --- libmultipath/discovery.c | 79 ++++++++++++++++++++-------------------- libmultipath/discovery.h | 1 - 2 files changed, 39 insertions(+), 41 deletions(-) diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c index 6e5d1005f..c165d9b6b 100644 --- a/libmultipath/discovery.c +++ b/libmultipath/discovery.c @@ -475,60 +475,33 @@ sysfs_get_tgt_nodename(struct path *pp, char *node) return 0; } -int sysfs_get_host_adapter_name(const struct path *pp, char *adapter_name) -{ - int proto_id; - - if (!pp || !adapter_name) - return 1; - - proto_id = pp->sg_id.proto_id; - - if (proto_id != SCSI_PROTOCOL_FCP && - proto_id != SCSI_PROTOCOL_SAS && - proto_id != SCSI_PROTOCOL_ISCSI && - proto_id != SCSI_PROTOCOL_SRP) { - return 1; - } - /* iscsi doesn't have adapter info in sysfs - * get ip_address for grouping paths - */ - if (pp->sg_id.proto_id == SCSI_PROTOCOL_ISCSI) - return sysfs_get_iscsi_ip_address(pp, adapter_name); - - /* fetch adapter pci name for other protocols - */ - return sysfs_get_host_pci_name(pp, adapter_name); -} - -int sysfs_get_host_pci_name(const struct path *pp, char *pci_name) +static int sysfs_get_host_bus_id(const struct path *pp, char *bus_id) { struct udev_device *hostdev, *parent; char host_name[HOST_NAME_LEN]; - const char *driver_name, *value; + const char *driver_name, *subsystem_name, *value; - if (!pp || !pci_name) + if (!pp || !bus_id) return 1; - sprintf(host_name, "host%d", pp->sg_id.host_no); + snprintf(host_name, sizeof(host_name), "host%d", pp->sg_id.host_no); hostdev = udev_device_new_from_subsystem_sysname(udev, "scsi_host", host_name); if (!hostdev) return 1; - parent = udev_device_get_parent(hostdev); - while (parent) { + for (parent = udev_device_get_parent(hostdev); + parent; + parent = udev_device_get_parent(parent)) { driver_name = udev_device_get_driver(parent); - if (!driver_name) { - parent = udev_device_get_parent(parent); - continue; - } - if (!strcmp(driver_name, "pcieport")) + subsystem_name = udev_device_get_subsystem(parent); + if (driver_name && !strcmp(driver_name, "pcieport")) + break; + if (subsystem_name && !strcmp(subsystem_name, "ccw")) break; - parent = udev_device_get_parent(parent); } if (parent) { - /* pci_device found + /* pci_device or ccw fcp device found */ value = udev_device_get_sysname(parent); @@ -537,7 +510,7 @@ int sysfs_get_host_pci_name(const struct path *pp, char *pci_name) return 1; } - strncpy(pci_name, value, SLOT_NAME_SIZE); + strlcpy(bus_id, value, SLOT_NAME_SIZE); udev_device_unref(hostdev); return 0; } @@ -545,6 +518,32 @@ int sysfs_get_host_pci_name(const struct path *pp, char *pci_name) return 1; } +int sysfs_get_host_adapter_name(const struct path *pp, char *adapter_name) +{ + int proto_id; + + if (!pp || !adapter_name) + return 1; + + proto_id = pp->sg_id.proto_id; + + if (proto_id != SCSI_PROTOCOL_FCP && + proto_id != SCSI_PROTOCOL_SAS && + proto_id != SCSI_PROTOCOL_ISCSI && + proto_id != SCSI_PROTOCOL_SRP) { + return 1; + } + /* iscsi doesn't have adapter info in sysfs + * get ip_address for grouping paths + */ + if (pp->sg_id.proto_id == SCSI_PROTOCOL_ISCSI) + return sysfs_get_iscsi_ip_address(pp, adapter_name); + + /* fetch adapter bus-ID for other protocols + */ + return sysfs_get_host_bus_id(pp, adapter_name); +} + int sysfs_get_iscsi_ip_address(const struct path *pp, char *ip_address) { struct udev_device *hostdev; diff --git a/libmultipath/discovery.h b/libmultipath/discovery.h index 095657bb9..466af3450 100644 --- a/libmultipath/discovery.h +++ b/libmultipath/discovery.h @@ -44,7 +44,6 @@ int store_pathinfo (vector pathvec, struct config *conf, struct path **pp_ptr); int sysfs_set_scsi_tmo (struct multipath *mpp, unsigned int checkint); int sysfs_get_timeout(const struct path *pp, unsigned int *timeout); -int sysfs_get_host_pci_name(const struct path *pp, char *pci_name); int sysfs_get_iscsi_ip_address(const struct path *pp, char *ip_address); int sysfs_get_host_adapter_name(const struct path *pp, char *adapter_name); From 260d7cb411fedf49167646ccc8834fd4b88f8005 Mon Sep 17 00:00:00 2001 From: Steffen Maier Date: Tue, 15 Feb 2022 20:45:47 +0100 Subject: [PATCH 41/48] libmultipath: add %L path wildcard for 64-bit hex LUN Complements v0.6.0 commit 01ab2a468ea2 ("libmultipath: Add additional path wildcards") as well as ("libmultipath: support host adapter name lookup for s390x ccw bus"). With that we can easily get the full FCP addressing triplet (HBA, WWPN, FCPLUN) from multipath tools without additional tools and correlation: $ multipathd -k'show paths format "%w|%a|%r|%L"' uuid |host adapter|target WWPN |LUN hex 36005076400820293e8000000000000a0|0.0.5080 |0x500507680b25c449|0x00a0000000000000 36005076400820293e8000000000000a0|0.0.5080 |0x500507680b25c448|0x00a0000000000000 36005076400820293e8000000000000a0|0.0.50c0 |0x500507680b26c449|0x00a0000000000000 36005076400820293e8000000000000a0|0.0.50c0 |0x500507680b26c448|0x00a0000000000000 Likewise, add a field lun_hex for JSON path output. Reviewed-by: Benjamin Block Reviewed-by: Benjamin Marzinski Reviewed-by: Martin Wilck Signed-off-by: Steffen Maier --- libmultipath/print.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/libmultipath/print.c b/libmultipath/print.c index ef4e05d22..8ea855770 100644 --- a/libmultipath/print.c +++ b/libmultipath/print.c @@ -95,6 +95,7 @@ " \"host_wwpn\" : \"%R\",\n" \ " \"target_wwpn\" : \"%r\",\n" \ " \"host_adapter\" : \"%a\",\n" \ + " \"lun_hex\" : \"%L\",\n" \ " \"marginal_st\" : \"%M\"" #define PROGRESS_LEN 10 @@ -452,6 +453,24 @@ snprint_hcil (struct strbuf *buff, const struct path * pp) pp->sg_id.lun); } + +static int +snprint_path_lunhex (struct strbuf *buff, const struct path * pp) +{ + uint64_t lunhex = SCSI_INVALID_LUN, scsilun; + + if (!pp || pp->sg_id.host_no < 0) + return print_strbuf(buff, "0x%016" PRIx64, lunhex); + + scsilun = pp->sg_id.lun; + /* cf. Linux kernel function int_to_scsilun() */ + lunhex = ((scsilun & 0x000000000000ffffULL) << 48) | + ((scsilun & 0x00000000ffff0000ULL) << 16) | + ((scsilun & 0x0000ffff00000000ULL) >> 16) | + ((scsilun & 0xffff000000000000ULL) >> 48); + return print_strbuf(buff, "0x%016" PRIx64, lunhex); +} + static int snprint_dev (struct strbuf *buff, const struct path * pp) { @@ -843,6 +862,7 @@ static const struct path_data pd[] = { {'0', "failures", snprint_path_failures}, {'P', "protocol", snprint_path_protocol}, {'I', "init_st", snprint_initialized}, + {'L', "LUN hex", snprint_path_lunhex}, }; static const struct pathgroup_data pgd[] = { From 1a2f32924d8ad025f2d0837f509d4e7fd501f1d1 Mon Sep 17 00:00:00 2001 From: Benjamin Marzinski Date: Wed, 16 Feb 2022 00:12:29 -0600 Subject: [PATCH 42/48] libmultipath: fix printing native nvme multipath topology. Since commit 2f05df4 ("libmultipath: use strbuf in print.c"), when multipath prints the topology of native nvme devices, instead of printing the multipath device information, it prints "w [G]:d s". This is because nvme_style() switched from calling snprintf(), which supports format specifiers, to append_strbuf_str(), which doesn't, while still keeping the same string, "%%w [%%G]:%%d %%s". Remove the extra percent signs, since they don't need to be escaped in append_strbuf_str(). Fixes: 2f05df4 ("libmultipath: use strbuf in print.c") Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- libmultipath/foreign/nvme.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libmultipath/foreign/nvme.c b/libmultipath/foreign/nvme.c index 499b881d7..838e11640 100644 --- a/libmultipath/foreign/nvme.c +++ b/libmultipath/foreign/nvme.c @@ -337,7 +337,7 @@ static int snprint_nvme_pg(const struct gen_pathgroup *gmp, static int nvme_style(__attribute__((unused)) const struct gen_multipath* gm, struct strbuf *buf, __attribute__((unused)) int verbosity) { - return append_strbuf_str(buf, "%%w [%%G]:%%d %%s"); + return append_strbuf_str(buf, "%w [%G]:%d %s"); } static const struct gen_multipath_ops nvme_map_ops = { From d11793a9386cee36da58391128d17f3a70fa83ba Mon Sep 17 00:00:00 2001 From: Uglymotha Date: Sat, 8 Jan 2022 08:05:46 +0100 Subject: [PATCH 43/48] Add missing include checkers.c: In function 'start_checker_thread': checkers.c:392:9: error: implicit declaration of function 'assert' [-Werror=implicit-function-declaration] 392 | assert(ctx && ctx->cls && ctx->cls->thread); | ^~~~~~ checkers.c:13:1: note: 'assert' is defined in header ''; did you forget to '#include '? Signed-off-by: Sietse van Zanen Reviewed-by: Martin Wilck --- libmultipath/checkers.c | 1 + 1 file changed, 1 insertion(+) diff --git a/libmultipath/checkers.c b/libmultipath/checkers.c index ef346d795..00554d652 100644 --- a/libmultipath/checkers.c +++ b/libmultipath/checkers.c @@ -5,6 +5,7 @@ #include #include #include +#include #include "debug.h" #include "checkers.h" From 571f28bffa95eac49a5436ab9d34c6afff434757 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Thu, 17 Feb 2022 17:22:32 +0100 Subject: [PATCH 44/48] libmultipath: enable linear ordering of bus/proto tuple We categorized protocols by bus/proto_id, while we only differentiate protocol IDs for SCSI. Allow transforming this into a linear sequence of bus/protocol IDs by having non-SCSI first, and follwing up with the different SCSI protocols. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- libmultipath/structs.c | 10 ++++++++++ libmultipath/structs.h | 13 +++++++++++-- 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/libmultipath/structs.c b/libmultipath/structs.c index 17f4baf6d..4b62da54e 100644 --- a/libmultipath/structs.c +++ b/libmultipath/structs.c @@ -753,3 +753,13 @@ int remove_feature(char **f, const char *o) return 0; } + +unsigned int bus_protocol_id(const struct path *pp) { + if (!pp || pp->bus < 0 || pp->bus > SYSFS_BUS_SCSI) + return SYSFS_BUS_UNDEF; + if (pp->bus != SYSFS_BUS_SCSI) + return pp->bus; + if ((int)pp->sg_id.proto_id < 0 || pp->sg_id.proto_id > SCSI_PROTOCOL_UNSPEC) + return SYSFS_BUS_UNDEF; + return SYSFS_BUS_SCSI + pp->sg_id.proto_id; +} diff --git a/libmultipath/structs.h b/libmultipath/structs.h index 205cf64e9..d94f93a0f 100644 --- a/libmultipath/structs.h +++ b/libmultipath/structs.h @@ -57,12 +57,13 @@ enum failback_mode { FAILBACK_FOLLOWOVER }; +/* SYSFS_BUS_SCSI should be last, see bus_protocol_id() */ enum sysfs_buses { SYSFS_BUS_UNDEF, - SYSFS_BUS_SCSI, SYSFS_BUS_CCW, SYSFS_BUS_CCISS, SYSFS_BUS_NVME, + SYSFS_BUS_SCSI, }; enum pathstates { @@ -181,9 +182,17 @@ enum scsi_protocol { SCSI_PROTOCOL_ADT = 7, /* Media Changers */ SCSI_PROTOCOL_ATA = 8, SCSI_PROTOCOL_USB = 9, /* USB Attached SCSI (UAS), and others */ - SCSI_PROTOCOL_UNSPEC = 0xf, /* No specific protocol */ + SCSI_PROTOCOL_UNSPEC = 0xa, /* No specific protocol */ }; +/* + * Linear ordering of bus/protocol + * This assumes that SYSFS_BUS_SCSI is last in enum sysfs_buses + * SCSI is the only bus type for which we distinguish protocols. + */ +#define LAST_BUS_PROTOCOL_ID (SYSFS_BUS_SCSI + SCSI_PROTOCOL_UNSPEC) +unsigned int bus_protocol_id(const struct path *pp); + #define SCSI_INVALID_LUN ~0ULL enum no_undef_states { From 28fcf23cc30b48ddc82cb2024a455b124baa3257 Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Thu, 17 Feb 2022 17:24:25 +0100 Subject: [PATCH 45/48] libmultipath: use bus_protocol_id() in snprint_path_protocol() Simplify bus_protocol_id() by using the linear ordering. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- libmultipath/print.c | 58 ++++++++++++++++---------------------------- 1 file changed, 21 insertions(+), 37 deletions(-) diff --git a/libmultipath/print.c b/libmultipath/print.c index 8ea855770..bf88f301c 100644 --- a/libmultipath/print.c +++ b/libmultipath/print.c @@ -754,43 +754,27 @@ snprint_path_failures(struct strbuf *buff, const struct path * pp) int snprint_path_protocol(struct strbuf *buff, const struct path * pp) { - switch (pp->bus) { - case SYSFS_BUS_SCSI: - switch (pp->sg_id.proto_id) { - case SCSI_PROTOCOL_FCP: - return append_strbuf_str(buff, "scsi:fcp"); - case SCSI_PROTOCOL_SPI: - return append_strbuf_str(buff, "scsi:spi"); - case SCSI_PROTOCOL_SSA: - return append_strbuf_str(buff, "scsi:ssa"); - case SCSI_PROTOCOL_SBP: - return append_strbuf_str(buff, "scsi:sbp"); - case SCSI_PROTOCOL_SRP: - return append_strbuf_str(buff, "scsi:srp"); - case SCSI_PROTOCOL_ISCSI: - return append_strbuf_str(buff, "scsi:iscsi"); - case SCSI_PROTOCOL_SAS: - return append_strbuf_str(buff, "scsi:sas"); - case SCSI_PROTOCOL_ADT: - return append_strbuf_str(buff, "scsi:adt"); - case SCSI_PROTOCOL_ATA: - return append_strbuf_str(buff, "scsi:ata"); - case SCSI_PROTOCOL_USB: - return append_strbuf_str(buff, "scsi:usb"); - case SCSI_PROTOCOL_UNSPEC: - default: - return append_strbuf_str(buff, "scsi:unspec"); - } - case SYSFS_BUS_CCW: - return append_strbuf_str(buff, "ccw"); - case SYSFS_BUS_CCISS: - return append_strbuf_str(buff, "cciss"); - case SYSFS_BUS_NVME: - return append_strbuf_str(buff, "nvme"); - case SYSFS_BUS_UNDEF: - default: - return append_strbuf_str(buff, "undef"); - } + static const char * const protocol_name[LAST_BUS_PROTOCOL_ID + 1] = { + [SYSFS_BUS_UNDEF] = "undef", + [SYSFS_BUS_CCW] = "ccw", + [SYSFS_BUS_CCISS] = "cciss", + [SYSFS_BUS_NVME] = "nvme", + [SYSFS_BUS_SCSI + SCSI_PROTOCOL_FCP] = "scsi:fcp", + [SYSFS_BUS_SCSI + SCSI_PROTOCOL_SPI] = "scsi:spi", + [SYSFS_BUS_SCSI + SCSI_PROTOCOL_SSA] = "scsi:ssa", + [SYSFS_BUS_SCSI + SCSI_PROTOCOL_SBP] = "scsi:sbp", + [SYSFS_BUS_SCSI + SCSI_PROTOCOL_SRP] = "scsi:srp", + [SYSFS_BUS_SCSI + SCSI_PROTOCOL_ISCSI] = "scsi:iscsi", + [SYSFS_BUS_SCSI + SCSI_PROTOCOL_SAS] = "scsi:sas", + [SYSFS_BUS_SCSI + SCSI_PROTOCOL_ADT] = "scsi:adt", + [SYSFS_BUS_SCSI + SCSI_PROTOCOL_ATA] = "scsi:ata", + [SYSFS_BUS_SCSI + SCSI_PROTOCOL_USB] = "scsi:usb", + [SYSFS_BUS_SCSI + SCSI_PROTOCOL_UNSPEC] = "scsi:unspec", + }; + const char *pn = protocol_name[bus_protocol_id(pp)]; + + assert(pn != NULL); + return append_strbuf_str(buff, pn); } static int From 6ae1b0c8e401e9f6cf137a7c78dbf2801e80858b Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Thu, 17 Feb 2022 17:25:09 +0100 Subject: [PATCH 46/48] libmultipath: enable defining a static bitfield The STATIC_BITFIELD macro allows defining a static variable holding a bitfield of constant length. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- libmultipath/util.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/libmultipath/util.h b/libmultipath/util.h index 79d9f32ae..5a44018b9 100644 --- a/libmultipath/util.h +++ b/libmultipath/util.h @@ -82,6 +82,16 @@ struct bitfield { bitfield_t bits[]; }; +#define STATIC_BITFIELD(name, length) \ + static struct { \ + unsigned int len; \ + bitfield_t bits[((length) - 1) / bits_per_slot + 1]; \ + } __static__ ## name = { \ + .len = (length), \ + .bits = { 0, }, \ + }; \ + struct bitfield *name = (struct bitfield *)& __static__ ## name + struct bitfield *alloc_bitfield(unsigned int maxbit); void _log_bitfield_overflow(const char *f, unsigned int bit, unsigned int len); From ac61ccc950456acfe7a4dacdadef26e9546707fb Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Thu, 17 Feb 2022 17:25:58 +0100 Subject: [PATCH 47/48] libmultipath: only warn once about unsupported dev_loss_tmo On systems with many devices, the warning that setting dev_loss_tmo is unsupported for certain protocols is printed many times, cluttering the logs. Only print this message once per unsupported protocol. Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski --- libmultipath/discovery.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c index c165d9b6b..b969fba1a 100644 --- a/libmultipath/discovery.c +++ b/libmultipath/discovery.c @@ -843,6 +843,7 @@ sysfs_set_scsi_tmo (struct multipath *mpp, unsigned int checkint) int i; unsigned int dev_loss_tmo = mpp->dev_loss; struct path *err_path = NULL; + STATIC_BITFIELD(bf, LAST_BUS_PROTOCOL_ID + 1); if (mpp->no_path_retry > 0) { uint64_t no_path_retry_tmo = @@ -897,12 +898,13 @@ sysfs_set_scsi_tmo (struct multipath *mpp, unsigned int checkint) sysfs_set_eh_deadline(mpp, pp); } - if (err_path) { + if (err_path && !is_bit_set_in_bitfield(bus_protocol_id(pp), bf)) { STRBUF_ON_STACK(proto_buf); snprint_path_protocol(&proto_buf, err_path); condlog(2, "%s: setting dev_loss_tmo is unsupported for protocol %s", mpp->alias, get_strbuf_str(&proto_buf)); + set_bit_in_bitfield(bus_protocol_id(pp), bf); } return 0; } From fc87ba4fe63b3eda51325209b86e159db493975d Mon Sep 17 00:00:00 2001 From: Martin Wilck Date: Wed, 16 Feb 2022 22:16:15 +0100 Subject: [PATCH 48/48] Bump version to 0.8.9 Signed-off-by: Martin Wilck --- libmultipath/version.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libmultipath/version.h b/libmultipath/version.h index e86a3cc86..66c6cf325 100644 --- a/libmultipath/version.h +++ b/libmultipath/version.h @@ -20,8 +20,8 @@ #ifndef _VERSION_H #define _VERSION_H -#define VERSION_CODE 0x000808 -#define DATE_CODE 0x030c15 +#define VERSION_CODE 0x000809 +#define DATE_CODE 0x100216 #define PROG "multipath-tools"