implement a gross hack to fix "boot -a" on systems with usb keyboards on systems with ehci handover to uhci (and maybe ohci), and fix a similar problem for "boot -s". there is effort to ensure that all devices attached via USB are probed before RB_ASKNAME or RB_SINGLE attempts to ask any questions on the console, and largely this works, often by chance, today, for USB disks and root. i've pushed this more into uhub and general USB device attachment as well, and kept a config_pending reference across the first explore of a bus. however, on systems where devices connected to ehci ports are handed over to a companion uhci or ohci port, it may not be the first, or even second, bus explore that finds the device finally before attachment, and at this point all config_pending references are dropped. there is no direct communication between drivers, the potentials are looked up but their device_t is only used for generic things like the name, so informing the correct companion to expect a device and deal with the config_pending references is not possible without some fairly ugly layer violations or multi-level callbacks (eg, we have "ehci0", and usually an the relevant companion, eg, "uhci2", but it is the uhub that uhci2 has attached that will deal with the device attachment.) with the above fixes to generic USB code, the disown happens during the first explore. the hack works by, at this point, checking if (a) root is not mounted, (b) single user or ask name are set, and (c) if the hack as not been triggered already. if all 3 conditions are true, then a config_pending_incr() is called and a callback is triggered for (default) 5 seconds to call config_pending_decr(). ehci detach pauses waiting for this callback if scheduled. this allows enough time for the uhub and the ukbd/wskbd to attach before the RK_ASKROOT prompts appear. testing shows it takes between 1.5 and 2 seconds for the keyboard to appear after the disown occurs. Index: dev/usb/ehcivar.c - new sc_compcallout, sc_compcallout, sc_complock, and a state for th handover hack. Index: dev/usb/ehci.c ehci_init(): - use aprint_normal_dev() instead of manual device_xname(). - initialise sc_compcallout, sc_compcallout, sc_complock, and sc_comp_state. ehci_detach(): - if there are companion controllers, tear own the above, including waiting if there is a callback scheduled. ehci_disown_callback(): - new callout to call config_pending_decr() in the the future. schedule this ca ehci_disown_sched_callback(): - if booting to single user or asking names, call config_pending_incr() and schedule the callout above, default 5 second delay. ehci_disown(): - if disowning a port call ehci_disown_sched_callback(). Index: dev/usb/uhub.c - convert sc_explorepending and sc_running to bool. add new sc_first_explore. uhub_attach(), uhub_explore(): - call config_pending_incr() at the start of uhub_attach(). dropped in uhub_explore(), if this is the first explore. Index: dev/usb/usb_subr.c - call config_pending_incr() and config_pending_decr() around attaching devices against "usbdevif" attribute. Index: kern/kern_drvctl.c - put export for devmon_insert_vec into sys/device.h Index: kern/subr_autoconf.c - make a few things static - export root_is_mounted Index: sys/device.h - add externs for devmon_insert_vec and root_is_mounted Index: sys/systm.h - move device_printf() prototype out from the middle of two sets of aprint_*() functions. Index: dev/usb/ehci.c =================================================================== RCS file: /cvsroot/src/sys/dev/usb/ehci.c,v retrieving revision 1.264 diff -p -u -u -r1.264 ehci.c --- dev/usb/ehci.c 16 Sep 2018 20:21:56 -0000 1.264 +++ dev/usb/ehci.c 17 Sep 2018 09:26:00 -0000 @@ -75,6 +75,7 @@ __KERNEL_RCSID(0, "$NetBSD: ehci.c,v 1.2 #include #include #include +#include #include @@ -446,8 +447,9 @@ ehci_init(ehci_softc_t *sc) } if (sc->sc_ncomp > 0) { KASSERT(!(sc->sc_flags & EHCIF_ETTF)); - aprint_normal("%s: %d companion controller%s, %d port%s%s", - device_xname(sc->sc_dev), sc->sc_ncomp, + aprint_normal_dev(sc->sc_dev, + "%d companion controller%s, %d port%s%s", + sc->sc_ncomp, sc->sc_ncomp!=1 ? "s" : "", EHCI_HCS_N_PCC(sparams), EHCI_HCS_N_PCC(sparams)!=1 ? "s" : "", @@ -459,6 +461,11 @@ ehci_init(ehci_softc_t *sc) device_xname(sc->sc_comps[i])); } aprint_normal("\n"); + + mutex_init(&sc->sc_complock, MUTEX_DEFAULT, IPL_USB); + callout_init(&sc->sc_compcallout, CALLOUT_MPSAFE); + cv_init(&sc->sc_compcv, "ehciccv"); + sc->sc_comp_state = CO_EARLY; } sc->sc_noport = EHCI_HCS_N_PORTS(sparams); sc->sc_hasppc = EHCI_HCS_PPC(sparams); @@ -1337,6 +1344,17 @@ ehci_detach(struct ehci_softc *sc, int f if (rv != 0) return rv; + if (sc->sc_ncomp > 0) { + mutex_enter(&sc->sc_complock); + while (sc->sc_comp_state == CO_SCHED) + cv_wait(&sc->sc_compcv, &sc->sc_complock); + mutex_exit(&sc->sc_complock); + + callout_destroy(&sc->sc_compcallout); + cv_destroy(&sc->sc_compcv); + mutex_destroy(&sc->sc_complock); + } + callout_halt(&sc->sc_tmo_intrlist, NULL); callout_destroy(&sc->sc_tmo_intrlist); @@ -2584,6 +2602,72 @@ ehci_roothub_ctrl(struct usbd_bus *bus, return totlen; } +/* + * Handle ehci hand-off in early boot vs RB_ASKNAME/RB_SINGLE. + * + * This pile of garbage below works around the following problem without + * holding boots with no hand-over devices present, while penalising + * boots where the first ehci probe hands off devices with a 5 second + * delay, if RB_ASKNAME/RB_SINGLE is set. This is typically not a problem + * for RB_SINGLE, but the same basic issue exists. + * + * The way ehci hand-off works, the companion controller does not get the + * device until after its' initial bus explore, so the reference dropped + * after the first explore is not enough. 5 seconds should be enough, + * and EHCI_DISOWN_DELAY_SECONDS can be set to another value. + * + * There are 3 states. CO_EARLY is set during attach. CO_SCHED is set + * if the callback is scheduled. CO_DONE is set when the callout has + * called config_pending_decr(). + * + * There's a mutex, a cv and a callout here, and we delay detach if the + * callout has been set. + */ +#ifndef EHCI_DISOWN_DELAY_SECONDS +#define EHCI_DISOWN_DELAY_SECONDS 5 +#endif +static int ehci_disown_delay_seconds = EHCI_DISOWN_DELAY_SECONDS; + +static void +ehci_disown_callback(void *arg) +{ + ehci_softc_t *sc = arg; + + config_pending_decr(sc->sc_dev); + + mutex_enter(&sc->sc_complock); + KASSERT(sc->sc_comp_state == CO_SCHED); + sc->sc_comp_state = CO_DONE; + cv_signal(&sc->sc_compcv); + mutex_exit(&sc->sc_complock); +} + +static void +ehci_disown_sched_callback(ehci_softc_t *sc) +{ + extern bool root_is_mounted; + + mutex_enter(&sc->sc_complock); + + if (root_is_mounted || + (boothowto & (RB_ASKNAME|RB_SINGLE)) == 0 || + sc->sc_comp_state != CO_EARLY) { + mutex_exit(&sc->sc_complock); + return; + } + + callout_reset(&sc->sc_compcallout, ehci_disown_delay_seconds * hz, + ehci_disown_callback, &sc->sc_dev); + sc->sc_comp_state = CO_SCHED; + + mutex_exit(&sc->sc_complock); + + config_pending_incr(sc->sc_dev); + aprint_normal("delaying %s by %u seconds due to USB owner change.", + (boothowto & RB_ASKNAME) == 0 ? "ask root" : "single user", + ehci_disown_delay_seconds); +} + Static void ehci_disown(ehci_softc_t *sc, int index, int lowspeed) { @@ -2593,13 +2677,11 @@ ehci_disown(ehci_softc_t *sc, int index, EHCIHIST_FUNC(); EHCIHIST_CALLED(); DPRINTF("index=%jd lowspeed=%jd", index, lowspeed, 0, 0); -#ifdef DIAGNOSTIC if (sc->sc_npcomp != 0) { int i = (index-1) / sc->sc_npcomp; - if (i >= sc->sc_ncomp) - printf("%s: strange port\n", - device_xname(sc->sc_dev)); - else + if (i < sc->sc_ncomp) { + ehci_disown_sched_callback(sc); +#ifdef DIAGNOSTIC printf("%s: handing over %s speed device on " "port %d to %s\n", device_xname(sc->sc_dev), @@ -2607,10 +2689,16 @@ ehci_disown(ehci_softc_t *sc, int index, index, sc->sc_comps[i] ? device_xname(sc->sc_comps[i]) : "companion controller"); + } else { + printf("%s: strange port\n", + device_xname(sc->sc_dev)); +#endif + } } else { +#ifdef DIAGNOSTIC printf("%s: npcomp == 0\n", device_xname(sc->sc_dev)); - } #endif + } port = EHCI_PORTSC(index); v = EOREAD4(sc, port) &~ EHCI_PS_CLEAR; EOWRITE4(sc, port, v | EHCI_PS_PO); Index: dev/usb/ehcivar.h =================================================================== RCS file: /cvsroot/src/sys/dev/usb/ehcivar.h,v retrieving revision 1.45 diff -p -u -u -r1.45 ehcivar.h --- dev/usb/ehcivar.h 9 Aug 2018 06:26:47 -0000 1.45 +++ dev/usb/ehcivar.h 17 Sep 2018 09:26:00 -0000 @@ -182,6 +182,16 @@ typedef struct ehci_softc { u_int sc_npcomp; device_t sc_comps[EHCI_COMPANION_MAX]; + /* This chunk to handle early RB_ASKNAME hand over. */ + callout_t sc_compcallout; + kmutex_t sc_complock; + kcondvar_t sc_compcv; + enum { + CO_EARLY, + CO_SCHED, + CO_DONE, + } sc_comp_state; + usb_dma_t sc_fldma; ehci_link_t *sc_flist; u_int sc_flsize; Index: dev/usb/uhub.c =================================================================== RCS file: /cvsroot/src/sys/dev/usb/uhub.c,v retrieving revision 1.138 diff -p -u -u -r1.138 uhub.c --- dev/usb/uhub.c 1 Feb 2018 09:50:48 -0000 1.138 +++ dev/usb/uhub.c 17 Sep 2018 09:26:00 -0000 @@ -113,9 +113,9 @@ struct uhub_softc { uint8_t *sc_statuspend; uint8_t *sc_status; size_t sc_statuslen; - int sc_explorepending; - - u_char sc_running; + bool sc_explorepending; + bool sc_first_explore; + bool sc_running; }; #define UHUB_IS_HIGH_SPEED(sc) \ @@ -263,6 +263,8 @@ uhub_attach(device_t parent, device_t se usb_endpoint_descriptor_t *ed; struct usbd_tt *tts = NULL; + config_pending_incr(self); + UHUBHIST_FUNC(); UHUBHIST_CALLED(); sc->sc_dev = self; @@ -284,14 +286,14 @@ uhub_attach(device_t parent, device_t se if (err) { DPRINTF("configuration failed, sc %#jx error %jd", (uintptr_t)sc, err, 0, 0); - return; + goto bad2; } if (dev->ud_depth > USB_HUB_MAX_DEPTH) { aprint_error_dev(self, "hub depth (%d) exceeded, hub ignored\n", USB_HUB_MAX_DEPTH); - return; + goto bad2; } /* Get hub descriptor. */ @@ -301,7 +303,7 @@ uhub_attach(device_t parent, device_t se if (err) { DPRINTF("getting hub descriptor failed, uhub%jd error %jd", device_unit(self), err, 0, 0); - return; + goto bad2; } for (nremov = 0, port = 1; port <= nports; port++) @@ -365,7 +367,7 @@ uhub_attach(device_t parent, device_t se /* force initial scan */ memset(sc->sc_status, 0xff, sc->sc_statuslen); - sc->sc_explorepending = 1; + sc->sc_explorepending = true; err = usbd_open_pipe_intr(iface, ed->bEndpointAddress, USBD_SHORT_XFER_OK|USBD_MPSAFE, &sc->sc_ipipe, sc, @@ -450,8 +452,8 @@ uhub_attach(device_t parent, device_t se usbd_delay_ms(dev, pwrdly); /* The usual exploration will finish the setup. */ - - sc->sc_running = 1; + sc->sc_running = true; + sc->sc_first_explore = true; if (!pmf_device_register(self, NULL, NULL)) aprint_error_dev(self, "couldn't establish power handler\n"); @@ -469,7 +471,8 @@ uhub_attach(device_t parent, device_t se kmem_free(hub, sizeof(*hub) + (nports-1) * sizeof(struct usbd_port)); dev->ud_hub = NULL; - return; + bad2: + config_pending_decr(self); } usbd_status @@ -778,7 +781,7 @@ uhub_explore(struct usbd_device *dev) } } mutex_enter(&sc->sc_lock); - sc->sc_explorepending = 0; + sc->sc_explorepending = false; for (int i = 0; i < sc->sc_statuslen; i++) { if (sc->sc_statuspend[i] != 0) { memcpy(sc->sc_status, sc->sc_statuspend, @@ -789,6 +792,10 @@ uhub_explore(struct usbd_device *dev) } } mutex_exit(&sc->sc_lock); + if (sc->sc_first_explore) { + config_pending_decr(sc->sc_dev); + sc->sc_first_explore = false; + } return USBD_NORMAL_COMPLETION; } @@ -943,7 +950,7 @@ uhub_intr(struct usbd_xfer *xfer, void * } if (!sc->sc_explorepending) { - sc->sc_explorepending = 1; + sc->sc_explorepending = true; memcpy(sc->sc_status, sc->sc_statuspend, sc->sc_statuslen); Index: dev/usb/usb_subr.c =================================================================== RCS file: /cvsroot/src/sys/dev/usb/usb_subr.c,v retrieving revision 1.226 diff -p -u -u -r1.226 usb_subr.c --- dev/usb/usb_subr.c 2 Aug 2018 06:09:04 -0000 1.226 +++ dev/usb/usb_subr.c 17 Sep 2018 09:26:00 -0000 @@ -907,6 +907,7 @@ usbd_attachwholedevice(device_t parent, dlocs[USBDEVIFCF_INTERFACE] = -1; KERNEL_LOCK(1, curlwp); + config_pending_incr(parent); dv = config_found_sm_loc(parent, "usbdevif", dlocs, &uaa, usbd_print, config_stdsubmatch); KERNEL_UNLOCK_ONE(curlwp); @@ -917,6 +918,7 @@ usbd_attachwholedevice(device_t parent, dev->ud_nifaces_claimed = 1; /* XXX */ usbd_serialnumber(dv, dev); } + config_pending_decr(parent); return USBD_NORMAL_COMPLETION; } Index: kern/kern_drvctl.c =================================================================== RCS file: /cvsroot/src/sys/kern/kern_drvctl.c,v retrieving revision 1.43 diff -p -u -u -r1.43 kern_drvctl.c --- kern/kern_drvctl.c 30 Nov 2017 20:25:55 -0000 1.43 +++ kern/kern_drvctl.c 17 Sep 2018 09:26:00 -0000 @@ -107,7 +107,6 @@ static const struct fileops drvctl_fileo #define MAXLOCATORS 100 -extern int (*devmon_insert_vec)(const char *, prop_dictionary_t); static int (*saved_insert_vec)(const char *, prop_dictionary_t) = NULL; static int drvctl_command(struct lwp *, struct plistref *, u_long, int); Index: kern/subr_autoconf.c =================================================================== RCS file: /cvsroot/src/sys/kern/subr_autoconf.c,v retrieving revision 1.262 diff -p -u -u -r1.262 subr_autoconf.c --- kern/subr_autoconf.c 26 Jun 2018 06:03:57 -0000 1.262 +++ kern/subr_autoconf.c 17 Sep 2018 09:26:00 -0000 @@ -192,17 +192,17 @@ struct deferred_config { TAILQ_HEAD(deferred_config_head, deferred_config); -struct deferred_config_head deferred_config_queue = +static struct deferred_config_head deferred_config_queue = TAILQ_HEAD_INITIALIZER(deferred_config_queue); -struct deferred_config_head interrupt_config_queue = +static struct deferred_config_head interrupt_config_queue = TAILQ_HEAD_INITIALIZER(interrupt_config_queue); -int interrupt_config_threads = 8; -struct deferred_config_head mountroot_config_queue = +static int interrupt_config_threads = 8; +static struct deferred_config_head mountroot_config_queue = TAILQ_HEAD_INITIALIZER(mountroot_config_queue); -int mountroot_config_threads = 2; +static int mountroot_config_threads = 2; static lwp_t **mountroot_config_lwpids; static size_t mountroot_config_lwpids_size; -static bool root_is_mounted = false; +bool root_is_mounted = false; static void config_process_deferred(struct deferred_config_head *, device_t); Index: sys/device.h =================================================================== RCS file: /cvsroot/src/sys/sys/device.h,v retrieving revision 1.155 diff -p -u -u -r1.155 device.h --- sys/device.h 26 Jun 2018 06:03:57 -0000 1.155 +++ sys/device.h 17 Sep 2018 09:26:01 -0000 @@ -427,6 +427,7 @@ extern int booted_partition; /* the par extern daddr_t booted_startblk; /* or the start of a wedge */ extern uint64_t booted_nblks; /* and the size of that wedge */ extern char *bootspec; /* and the device/wedge name */ +extern bool root_is_mounted; /* true if root is mounted */ struct vnode *opendisk(device_t); int getdisksize(struct vnode *, uint64_t *, unsigned int *); @@ -441,6 +442,7 @@ int config_fini_component(struct cfdrive void config_init_mi(void); void drvctl_init(void); void drvctl_fini(void); +extern int (*devmon_insert_vec)(const char *, prop_dictionary_t); int config_cfdriver_attach(struct cfdriver *); int config_cfdriver_detach(struct cfdriver *); Index: sys/systm.h =================================================================== RCS file: /cvsroot/src/sys/sys/systm.h,v retrieving revision 1.277 diff -p -u -u -r1.277 systm.h --- sys/systm.h 10 Aug 2018 21:44:59 -0000 1.277 +++ sys/systm.h 17 Sep 2018 09:26:01 -0000 @@ -196,14 +196,14 @@ void aprint_naive(const char *, ...) __p void aprint_verbose(const char *, ...) __printflike(1, 2); void aprint_debug(const char *, ...) __printflike(1, 2); -void device_printf(device_t, const char *fmt, ...) __printflike(2, 3); - void aprint_normal_dev(device_t, const char *, ...) __printflike(2, 3); void aprint_error_dev(device_t, const char *, ...) __printflike(2, 3); void aprint_naive_dev(device_t, const char *, ...) __printflike(2, 3); void aprint_verbose_dev(device_t, const char *, ...) __printflike(2, 3); void aprint_debug_dev(device_t, const char *, ...) __printflike(2, 3); +void device_printf(device_t, const char *fmt, ...) __printflike(2, 3); + struct ifnet; void aprint_normal_ifnet(struct ifnet *, const char *, ...)