My bad. DeleteExpiredTaskAfter requires the EndBoundary to be set. StartBoundary is set automatically in the TimeTrigger constructor to DateTime.Now.
I don't know of anyway to delete the task immediately upon completion other than have the task's action delete the task or wait for the DeleteExpiredTaskAfter to kick in.