* XXX TODO

- Consider renaming delayed_task to task_timeout or something.
- Consider switching from container_of-style to cookie-style.

* Tasks

A /task/ is an action to be executed once asynchronously.  Tasks are
cheap: each one is four pointers long, initializing them is quick,
scheduling them requires constant time and writing only to per-CPU
memory, and they will run on the current CPU as soon as they can.

Once you have allocated storage for a struct task, you can ask the
system to execute your task by initializing and scheduling it:

	task_init(task, &my_task_action);
	task_schedule(task);

static void
my_task_action(struct task *task)
{
	printf("my task ran!\n");
}

task_init and task_schedule never fail, and you can use them in any
context, including hard interrupt context.  If you schedule a task
multiple times before it runs, it will be executed only once.  If you
schedule a task again after it has begun executing, it will not begin
executing again until its action has either returned or chosen to call
task_done.

Once you are done with a task, and you have made sure it is finished
executing, you must destroy it with task_destroy:

	task_destroy(task);

** Cancellation

If you're not sure whether a task is scheduled or not, or whether it
has finished executing or not, you can cancel it and wait for it to
complete with task_cancel.  There are two tricky details about task
cancellation:

1. The task might need a lock that the caller of task_cancel holds.  In
   that case, you must pass the lock to task_cancel so that it can drop
   the lock before waiting for the task, but after it has taken locks
   internal to the task abstraction to avoid races.

2. The task might be responsible for releasing a resource, even a
   resource such as the memory containing its struct task.  In that
   case, if the task was about to run but is cancelled, task_cancel
   returns true to indicate the caller must take responsibility for the
   resource.  Otherwise, task_cancel returns false.

The following contrived example illustrates a pattern that might arise
in a device driver using tasks.  Since the task action frees the struct
task, it must first call task_done before returning; otherwise the
thread running the task will continue to use the newly freed memory.

	/* Set up a task, if we need one.  */
	struct task *tmp = kmem_alloc(sizeof(*task), KM_SLEEP);
	mutex_enter(&sc->sc_lock);
	if (sc->sc_curtask == NULL) {
		sc->sc_curtask = tmp;
		tmp = NULL;
		task_init(sc->sc_curtask, &my_task_action);
		task_schedule(sc->sc_curtask);
	}
	mutex_exit(&sc->sc_lock);
	if (tmp != NULL)
		kmem_free(tmp, sizeof(*tmp));

	...

	/* Cancel the task, if there is one.  */
	struct task *task = NULL;
	mutex_enter(&sc->sc_lock);
	if (sc->sc_curtask != NULL) {
		if (task_cancel(sc->sc_curtask, &sc->sc_lock)) {
			/* We cancelled it, so we have to clean it up.  */
			task = sc->sc_curtask;
			sc->sc_curtask = NULL;
		}
	}
	mutex_exit(&sc->sc_lock);
	if (task != NULL) {
		task_destroy(task);
		kmem_free(task, sizeof(*task));
	}

static void
my_driver_action(struct task *task)
{
	printf("my driver's task ran!\n");
	mutex_enter(&sc->sc_lock);
	KASSERT(sc->sc_curtask == task);
	sc->sc_curtask = NULL;
	mutex_exit(&sc->sc_lock);
	task_done(task);
	task_destroy(task);
	kmem_free(task, sizeof(*task));
}

** Draining

If you haven't kept track of all your driver's tasks, but your device
is detaching, instead of cancelling them all you can wait for them to
complete with task_drain.  This won't prevent new tasks from being
scheduled -- it will only wait for the ones that were already scheduled
to complete.

static void
mydev_detach(device_t self, int flags)
{
	struct mydev_softc *sc = device_private(self);
	...
	task_drain();
	...
}

Draining means waiting for all currently scheduled tasks to complete on
all CPUs, so tasks scheduled with task_schedule are not allowed long
sleeps!  Nevertheless, task_drain may take a long time simply because
it must wait for all scheduled tasks on every CPU to complete.

** Delayed tasks

task_schedule schedules a task to be executed as soon as possible.  If
you want to execute a task after a delay, you can use a delayed task.
You must allocate storage for a struct delayed_task, initialize it with
delayed_task_init, and then schedule it with delayed_task_schedule, say
for one second in the future with ten milliseconds jitter (XXX clarify
the precision concept and perhaps replace nsec by struct timespec):

	delayed_task_init(dt, &my_delayed_action);
	delayed_task_schedule(dt, mstons(1000), mstons(10));

static void
my_delayed_action(struct delayed_task *dt)
{
	printf("my delayed action ran!");
}

Delayed tasks can be cancelled with delayed_task_cancel, and must be
destroyed with delayed_task_destroy.  If you schedule a delayed task
again before it has run, then, like task_schedule, it has no effect.
You can reschedule a delayed tasks for a different time, if it hasn't
already run, with delayed_task_reschedule, which returns true if it
rescheduled the task or false if either it wasn't running or it was too
late:

	/* Try to make it run 50 ms from now.  */
	if (!delayed_task_reschedule(dt, mstons(50), mstons(10))) {
		/* Too late.  Schedule it again after 1 s.  */
		delayed_task_schedule(dt, mstons(1000), mstons(10));
	}

If a delayed task is scheduled or rescheduled with zero delay, then,
like task_schedule, it will run as soon as possible.  The action can
distinguish whether it was executed after a nonzero delay by asking
delayed_task_timedout.  You might use this in a driver to set up a task
to handle either an interrupt or a timeout:

static void
mydev_start(struct mydev_softc *sc)
{
	...
	mydev_submit_request(sc);
	delayed_task_init(&sc->sc_dt, mydev_task);
	delayed_task_schedule(&sc->sc_dt, mstons(1000), mstons(10));
	...
}

static void
mydev_intr(struct mydev_softc *sc)
{
	...
	if (!delayed_task_reschedule(&sc->sc_dt, 0, 0)) {
		/* We already timed out, so say never mind.  */
		mydev_nack_request(sc);
	}
	...
}

static void
mydev_task(struct delayed_task *dt)
{
	struct mydev_softc *sc = container_of(dt, struct mydev_softc, sc_dt);

	if (delayed_task_timedout(dt)) {
		/* Deliver a timeout to userland.  */
		mydev_user_timeout(sc);
	} else {
		mydev_read_answer_to_user(sc);
	}
}

Note that task_drain does *not* wait for delayed tasks to complete --
that may require it to wait arbitrarily long in the future for delayed
tasks scheduled a year from now.  You must cancel them explicitly with
delayed_task_cancel.

** Task queues

Tasks are different from threads in that they are much lighter-weight
-- a struct lwp is dozens of pointers long, versus four for a task --
but are not as flexible.  Tasks are designed for short actions that
high-priority code needs to run at lower priority.  Sometimes, however,
you may want to run at higher priority than PRI_NONE, as task_schedule
runs them at, or take longer actions that may sleep, including memory
allocation.

If you want to run at a different priority, you can choose a different
/task queue/ to execute your tasks on with taskqueue_get.  For example,
if you want to run tasks at soft interrupt priority, you might use:

	struct taskqueue *taskqueue;

	error = taskqueue_get(&taskqueue, PRI_SOFTNET);
	if (error)
		goto fail;
	...
	taskqueue_put(taskqueue);

Then to schedule a task on that task queue instead of the shared system
low-priority task queue, use taskqueue_schedule:

	taskqueue_schedule(taskqueue, task);

For a delayed task, use taskqueue_schedule_delayed:

	taskqueue_schedule_delayed(taskqueue, dt, mstons(1234), mstons(10));

Tasks on task queues at different priority levels are run in separate
threads (or in softints if at soft interrupt priority levels).  The
task queues managed by taskqueue_get/taskqueue_put are still shared
system task queues and therefore do not allow long sleeps, however.  If
you also want the option of sleeping in your task queue, you must
create your own with taskqueue_create:

	struct taskqueue *taskqueue;

	error = taskqueue_create(&taskqueue, "mytaskq", PRI_NONE, IPL_BIO,
	    TASKQUEUE_PERCPU);
	if (error)
		goto fail;
	...
	taskqueue_destroy(taskqueue);

This task queue can have tasks scheduled from any interrupt priority
level up to and including IPL_BIO, and will execute tasks one at a time
per CPU in a thread at PRI_NONE.  Tasks on different task queues may be
executed concurrently on the same CPU by different threads, so a long-
running task on your task queue will not prevent tasks on shared system
ones from running.  (Nor will long-running tasks on the shared system
task queues prevent tasks on yours from running, but there should not
be any long-running tasks on the shared system ones!)

Just like task_drain will drain the shared system low-priority task
queue, taskqueue_drain will drain a task queue of your choice -- but
while taskqueue_drain is still an expensive global synchronization
operation, you won't have to wait for tasks on other task queues to
complete too.

	taskqueue_drain(taskqueue);

* Appendix

Tasks are meant to replace most applications of workqueues,
callouts, and softints.  However, that doesn't mean that those
abstractions will go away.  This code itself is an application of
callouts and softints, and while scheduling a task is more flexible
than queueing work, the latter is even lighter-weight, so
applications not needing the extra flexibility may reasonably stay
with workqueues.

The task abstraction was influenced by Linux workqueues, FreeBSD
taskqueues, NetBSD workqueues/callouts/softints, and design notes
for NetBSD kconts (`kernel continuations').