Andrew Vasquez wrote:
> fc-transport: Close state transition-window during rport deletion.
>
> After an rport's state has transitioned to FC_PORTSTATE_BLOCKED,
> but, prior to making the upcall to 'block' the scsi-target
> associated with an rport, queued commands can recycle and
> ultimately run out of retries causing failures to propagate to
> upper-level drivers. Close this transition-window by returning
> the non-'retries' modifying DID_IMM_RETRY status for submitted
> I/Os.
The same can happen for iscsi when transitioning from logged in
to failed and blocking the sdevs.
This patch converts iscsi and fc's transitions back to use DID_IMM_RETRY
instead of DID_TRANSPORT_DISRUPTED which has a limited number of retries
that we do not want to use for handling this race.
Signed-off-by: Andrew Vasquez <andrew.vasquez@qlogic.com>
[Addition of iscsi and fc port online devloss case conversion by Mike Christie]
Signed-off-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
When we reworked the transport for the rport lifetimes, in cases where the
rport was reused as a container for tgt id bindings, we inadvertantly
removed the callback to the driver indicating that dev_loss_tmo had fired.
This patch restores that functionality.
Signed-off-by: James Smart <james.smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
When the fastfail flag was added, it did not account for the flags
being bit fields. Correct the definition so there is no longer a
conflict.
Signed-off-by: James Smart <james.smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
If the target is blocked and fast io fail tmo has not fired
then we requeue with DID_TRANSPORT_DISRUPTED. Once that
tmo fires we fail with DID_TRANSPORT_FAILFAST.
v2
- seperate from
"fc class: unblock target after calling terminate callback"
to make it easier to review.
- Add JamesS's ack from list.
v2
- initial patch
Signed-off-by: Mike Christie <michaelc@cs.wisc.edu>
Acked-by: James Smart <James.Smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
When we block a rport and the driver implements the terminate
callback we will fail IO that was running quickly. However
IO that was in the scsi_device/block queue sits there until
the dev_loss_tmo fires, and this can make it look like IO is
lost because new IO will get executed but that IO stuck in
the blocked queue sits there for some time longer.
With this patch when the fast io fail tmo fires, we will
fail the blocked IO and any new IO. This patch also allows
all drivers to partially support the fast io fail tmo. If the
terminate io callback is not implemented, we will still fail blocked
IO and any new IO, so multipath can handle that.
This patch also allows the fc and iscsi classes to implement the
same behavior. The timers are just unfornately named differently.
This patch also fixes the problem where drivers were unblocking
the target in their terminate callback, which was needed for
rport removal, but for fast io fail timeout it would cause
IO to bounce arround the scsi/block layer and the LLD queuecommand.
And it for drivers that could have IO stuck but did not have
a terminate callback the unblock calls in the class will fix
them.
v2.
- fix up bit setting style to meet JamesS's pref.
- Broke out new host byte error changes to make it easier to read.
- added JamesS's ack from list.
v1
- initial patch
Signed-off-by: Mike Christie <michaelc@cs.wisc.edu>
Acked-by: James Smart <James.Smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
There's already a fc_vport_termintate() call exported by
the transport. This patch adds a symmetric call to the API to allow
an NPIV-capable LLD to instantiate vports sans user intervention.
Additional comments/updates:
Re: scsi_fc_transport.txt
Add a function prototype for fc_vport_terminate similar to what's
done for fc_vport_create
Re: fc_vport_create
I recommend we pass the channel number in fc_vport_create rather
than fixing it at zero.
Also, ids->vport_type should be set to FC_PORTTYPE_NPIV prior to
calling fc_vport_create. The comment is also meaningless.
Added-by and
Signed-off-by: James Smart <james.smart@emulex.com>
Signed-off-by: Andrew Vasquez <andrew.vasquez@qlogic.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
Kobjects do not have a limit in name size since a while, so stop
pretending that they do.
Signed-off-by: Kay Sievers <kay.sievers@vrfy.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
It's big, but there doesn't seem to be a way to split it up smaller...
Signed-off-by: Tony Jones <tonyj@suse.de>
Signed-off-by: Kay Sievers <kay.sievers@vrfy.org>
Cc: Roland Dreier <rolandd@cisco.com>
Cc: Sean Hefty <sean.hefty@intel.com>
Cc: Hal Rosenstock <hal.rosenstock@gmail.com>
Cc: James Bottomley <James.Bottomley@HansenPartnership.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This adds minimum target driver support like the srp transport does:
- fc_remote_port_{rolechg,delete} calls
scsi_tgt_it_nexus_{create,destroy} for target drivers.
- add callbacks to notify target drivers of the nexus and tmf
operation results to fc_function_template.
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
This change has already been discussed on linux-scsi:
http://marc.info/?t=118771096400003http://marc.info/?t=118760913100005
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Acked-by: James Smart <James.Smart@Emulex.Com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
When the vport attribute "delete" is used to delete the vport, sysfs
deadlocks waiting for the write to complete, which is waiting for the
sysfs teardown to complete. Moved this effort to a work_q element.
Took the opportunity to make some other cosmetic changes:
- removed tabs in Doc file - replaced with expanded spaces
- minor copyright text and author text updates
- removed a bunch of trailing whitespace
Signed-off-by: James Smart <James.Smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
This patch provides support for FC virtual ports based on NPIV.
For information on the interfaces and design, please read the
Documentation/scsi/scsi_fc_transport.txt file enclosed within
the patch.
The RFC was originally posted here:
http://marc.info/?l=linux-scsi&m=117226959918393&w=2
Changes from the initial RFC:
- Bug fix: needed a transport_class_unregister() for the vport class
- Create a symlink to the vport in the shost device if it is not the
parent of the vport.
- Made symbolic name writable so it can be set after creation
- Made the temporary fc_vport_identifiers struct private to the
transport.
- Deleted the vport_id field from the vport. I couldn't find any good
use for it (and symname is a good replacement).
- Made the vport_state and vport_last_state "private" attributes.
Added the fc_vport_set_state() helper function to manage state
transitions
- Updated vport_create() to allow a vport to be created in a disabled
state.
- Added INITIALIZING and FAILED vport states
- Added VPCERR_xxx defines for errors to be returned from vport_create()
- Created a Documentation/scsi/scsi_fc_transport.txt file that describes
the interfaces and expected LLDD behaviors.
Signed-off-by: James Smart <James.Smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
This patch updates the FC transport for all speeds identified in
SM-HBA. Note: it does not sync the "bit" definitions, as that is
actually insulated from user-space via the sysfs text string. (I could
do it, but it does introduce a potential binary-incompatibility).
Signed-off-by: James Smart <James.Smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
This patch adds the following functionality to the FC transport:
- dev_loss_tmo LLDD callback :
Called to essentially confirm the deletion of an rport. Thus, it is
called whenever the dev_loss_tmo fires, or when the rport is deleted
due to other circumstances (module unload, etc). It is expected that
the callback will initiate the termination of any outstanding i/o on
the rport.
- fast_io_fail_tmo and LLD callback:
There are some cases where it may take a long while to truly determine
device loss, but the system is in a multipathing configuration that if
the i/o was failed quickly (faster than dev_loss_tmo), it could be
redirected to a different path and completed sooner.
Many thanks to Mike Reed who cleaned up the initial RFC in support
of this post.
The original RFC is at:
http://marc.theaimsgroup.com/?l=linux-scsi&m=115505981027246&w=2
Signed-off-by: James Smart <James.Smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
During discussions with Mike Christie, I became convinced that we needed
a larger vendor id. This patch extends the id from 32 to 64 bits.
This applies on top of the prior patches that add SCSI transport events
via netlink.
Signed-off-by: James Smart <James.Smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
This patch formally adds support for the posting of FC events via netlink.
It is a followup to the original RFC at:
http://marc.theaimsgroup.com/?l=linux-scsi&m=114530667923464&w=2
and the initial posting at:
http://marc.theaimsgroup.com/?l=linux-scsi&m=115507374832500&w=2
The patch has been updated to optimize the send path, per the discussions
in the initial posting.
Per discussions at the Storage Summit and at OLS, we are to use netlink for
async events from transports. Also per discussions, to avoid a netlink
protocol per transport, I've create a single NETLINK_SCSITRANSPORT protocol,
which can then be used by all transports.
This patch:
- Creates new files scsi_netlink.c and scsi_netlink.h, which contains the
single and shared definitions for the SCSI Transport. It is tied into the
base SCSI subsystem intialization.
Contains a single interface routine, scsi_send_transport_event(), for a
transport to send an event (via multicast to a protocol specific group).
- Creates a new scsi_netlink_fc.h file, which contains the FC netlink event
messages
- Adds 3 new routines to the fc transport:
fc_get_event_number() - to get a FC event #
fc_host_post_event() - to send a simple FC event (32 bits of data)
fc_host_post_vendor_event() - to send a Vendor unique event, with
arbitrary amounts of data.
Note: the separation of event number allows for a LLD to send a standard
event, followed by vendor-specific data for the event.
Note: This patch assumes 2 prior fc transport patches have been installed:
http://marc.theaimsgroup.com/?l=linux-scsi&m=115555807316329&w=2http://marc.theaimsgroup.com/?l=linux-scsi&m=115581614930261&w=2
Sorry - next time I'll do something like making these individual
patches of the same posting when I know they'll be posted closely
together.
Signed-off-by: James Smart <James.Smart@emulex.com>
Tidy up configuration not to make SCSI always select NET
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
This patch updates the fc transport for the following:
- Addition of a new attribute "system_hostname" which can be
used to set the fully qualified hostname that the fc_host
is attached to. The fc_host can then register this string
as the FDMI-based host name attribute.
Note: for NPIV, a fc_host could be associated with a system which
is not the local system.
- Add the inline function u64_to_wwn(), which is the inverse of the
existing wwn_to_u64() function.
- Slight reorg, just to keep dynamic attributes with each other, etc
Signed-off-by: James Smart <James.Smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
As previously reported via Michael Reed, the FC transport took a hit
in 2.6.15 (perhaps a little earlier) when we solved a recursion error.
There are 2 deadlocks occurring:
- With scan and the delete items sharing the same workq, flushing the
workq for the delete code was getting it stalled behind a very long
running scan code path.
- There's a deadlock where scsi_remove_target() has to sit behind
scsi_scan_target() due to contention over the scan_lock().
This patch resolves the 1st deadlock and significantly reduces the
odds of the second. So far, we have only replicated the 2nd deadlock
on a highly-parallel SMP system. More on the 2nd deadlock in a following
email.
This patch reworks the transport to:
- Only use the scsi host workq for scanning
- Use 2 other workq's internally. One for deletions, the other for
scheduled deletions. Originally, we tried this with a single workq,
but the occassional flushes of the scheduled queues was hitting the
second deadlock with a slightly higher frequency. In the future, we'll
look at the LLDD's and the transport to see if we can get rid of this
extra overhead.
- When moving to the other workq's we tightened up some object states
and some lock handling.
- Properly syncs adds/deletes
- minor code cleanups
- directly reference fc_host_attrs, rather than through attribute
macros
- flush the right workq on delayed work cancel failures.
Large kudos to Michael Reed who has been working this issue for the last
month.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
Add fc_host attribute permanent_port_name which is
used to show the port name of the primary port -
the port that initially logged into the fabric.
For a virtual port (registered via the primary port with
FDISC command) it is useful to know not only its (virtual)
port name but also the permanent port name.
Signed-off-by: Andreas Herrmann <aherrman@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
In the scenario that a link was broken, the devloss timer for each
rport was expire at roughly the same time, causing lots of "delete"
workqueue items being queued. Depth is dependent upon the number of
rports that were on the link.
The rport target remove calls were calling flush_scheduled_work(),
which would interrupt the stream, and start the next workqueue item,
which did the same thing, and so on until recursion depth was large.
This fix stops the recursion in the initial delete path, and pushes it
off to a host-level work item that reaps the dead rports.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
I recently picked up my older work to remove unnecessary #includes of
sched.h, starting from a patch by Dave Jones to not include sched.h
from module.h. This reduces the number of indirect includes of sched.h
by ~300. Another ~400 pointless direct includes can be removed after
this disentangling (patch to follow later).
However, quite a few indirect includes need to be fixed up for this.
In order to feed the patches through -mm with as little disturbance as
possible, I've split out the fixes I accumulated up to now (complete for
i386 and x86_64, more archs to follow later) and post them before the real
patch. This way this large part of the patch is kept simple with only
adding #includes, and all hunks are independent of each other. So if any
hunk rejects or gets in the way of other patches, just drop it. My scripts
will pick it up again in the next round.
Signed-off-by: Tim Schmielau <tim@physik3.uni-rostock.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
We recently went back to implement a board reset. When we perform the
reset, we wanted to tear down the internal data structures and rebuild
them. Unfortunately, when it came to the rport structure, things were
odd. If we deleted them, the scsi targets and sdevs would be
torn down. Not a good thing for a temporary reset. We could block the
rports, but we either maintain the internal structures to keep the
rport reference (perhaps even replicating what's in the transport),
or we have to fatten the fc transport with new search routines to find
the rport (and deal with a case of a dangling rport that the driver
forgets).
It dawned on me that we had actually reached this state incorrectly.
When the fc transport first started, we did the block/unblock first, then
added the rport interface. The purpose of block/unblock is to hide the
temporary disappearance of the rport (e.g. being deleted, then readded).
Why are we making the driver do the block/unblock ? We should be making
the transport have only an rport add/delete, and the let the transport
handle the block/unblock.
So... This patch removes the existing fc_remote_port_block/unblock
functions. It moves the block/unblock functionality into the
fc_remote_port_add/delete functions. Updates for the lpfc driver are
included. Qlogic driver updates are also enclosed, thanks to the
contributions of Andrew Vasquez. [Note: the qla2xxx changes are
relative to the scsi-misc-2.6 tree as of this morning - which does
not include the recent patches sent by Andrew]. The zfcp driver does
not use the block/unblock functions.
One last comment: The resulting behavior feels very clean. The LLDD is
concerned only with add/delete, which corresponds to the physical
disappearance. However, the fact that the scsi target and sdevs are
not immediately torn down after the LLDD calls delete causes an
interesting scenario... the midlayer can call the xxx_slave_alloc and
xxx_queuecommand functions with a sdev that is at the location the
rport used to be. The driver must validate the device exists when it
first enters these functions. In thinking about it, this has always
been the case for the LLDD and these routines. The existing drivers
already check for existence. However, this highlights that simple
validation via data structure dereferencing needs to be watched.
To deal with this, a new transport function, fc_remote_port_chkready()
was created that LLDDs should call when they first enter these two
routines. It validates the rport state, and returns a scsi result
which could be returned. In addition to solving the above, it also
creates consistent behavior from the LLDD's when the block and deletes
are occuring.
Rejections fixed up and
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
Ok, here's a patch to add such a common API for fc transport users.
Relevant LLD changes (lpfc and qla2xxx) also present.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
obviously FC Port Speeds in scsi_transport_fc.h are defined according
to FC-HBA:
#define FC_PORTSPEED_1GBIT 1
#define FC_PORTSPEED_2GBIT 2
#define FC_PORTSPEED_10GBIT 4
#define FC_PORTSPEED_4GBIT 8
Problem is, whoever invented FC-HBA did not care about FC-FS or
FC-GS-x. Following FC-FS/FC-GS-x defintions of port speeds would look
like:
1 GBit: 0x0001
2 GBit: 0x0002
4 GBit: 0x0004
10GBit: 0x0008
(and new in FC-LS:
8 Gbit: 0x0010
16GBit: 0x0020)
I really appreciate if scsi_transport_fc.h would define port speeds
according to FC-GS-x/FC-FS. Thus mapping of port speed capabilities to
values defined in scsi_transport_fc.h can be avoided in the LLDD.
Attached is a patch to change the definitions.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
On some platforms the hard-casting of 8 byte node_name and
port_name arrays to an u64 would cause unaligned-access
warnings. Generalize the conversions with a transport
helper function which performs consistent shifting of WWN
bytes.
Signed-off-by: Andrew Vasquez <andrew.vasquez@qlogic.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.
Let it rip!
