A Storage Area Network (SAN) is a high-speed network of shared storage devices, servers, and clients that help in transferring data. There are various tools, such as NetWisdom and McDATA SANavigator, Xgig Analyzer Suite, and Generators that help in monitoring, analyzing, and testing SAN performance. These tools allow you to maximize SAN performance by reducing the time required for transferring data.

This ReferencePoint describes the features, components, and applications of a SAN. It explains the methods of maximizing SAN performance and discusses the tools available for monitoring, analyzing, and testing SAN performance.

Overview of SANs

SANs provides cost effective data transfer for organizations that want to transfer data over large geographical areas. SANs use Fibre Channel (FC) technology for high data transfer rate on the network. In addition, they provide effective management and utilization of storage devices. You can extend the storage capacity in a SAN based on the amount of data that needs to be stored. Storage devices on SANs are consolidated to form a single large storage subsystem. This storage subsystem is accessible to all connected servers and storage devices on a SAN. Figure 3-2-1

shows the structure of a SAN:

Figure 3-2-1: Structure of a SAN

SANs support high-speed data transfer among connected devices, such as servers and storage devices. You can transfer data between servers and storage devices by various ways:

• Server to Storage: Transfers the data between the servers and storage devices on a SAN.

• Server to Server: Transfers the data in large volumes between connected servers on a SAN.

• Storage to Storage: Transfers the data between connected storage devices on a SAN without involvement of the server.

Note

Multiple servers can access the same storage device on a SAN using parallel data paths.

Features of SANs

SANs provide various features, such as increased performance, improved and cost effective storage management, and high availability and security of data. A SAN increases the performance of a network by providing the following features:

• High data bandwidth: Provides high data bandwidth by using devices that can transfer data at the speed of 2 gigabytes per second.

• Unlimited scalable storage capacity: Helps increase storage capacity by adding unlimited number of storage devices.

• Improved storage utilization: Helps utilize the storage capacity of storage devices because in a SAN, you can flexibly allocate storage area to different types of data.

• Globally available storage: Helps all devices to have access to the storage area in the storage devices.

• Data sharing: Helps multiple devices to access a single file because a SAN provides concurrent reading and writing of data with high-bandwidth data access.

SANs also provide increased and cost effective storage management by using virtual storage management system. In virtual storage management system, servers with different operating systems, such as UNIX, Windows 2000, Netware, and Linux can use the virtual storage pool, which is a collection of different storage devices, such as Integrated Drive Electronics (IDE), Small Computer System Interface (SCSI), Self Storage Association (SSA) and Fibre Channel (FC), to store data

Figure 3-2-2 shows how the virtual storage management system works:

Figure 3-2-2: The Virtual Storage Management System

The virtual storage management system allows effective utilization of storage area in a SAN. It also allows you to backup data for storing without connecting to a Local Area Network (LAN). The process of data backup without connecting to LAN increases the speed of data backup and bandwidth available to the client computers.

The storage devices, servers, and client computers in a SAN are configured with multiple data paths, which provide uninterrupted transfer of data in case of hardware failure. SANs also provide integrated real-time backup and recovery to ensure high availability and security of data. In a SAN, you can use FC for greater bandwidth to speed up data backup operations.

Components of a SAN

In a SAN, there are various components, such as SAN servers and SAN storage, which are used for data communication. The components used in SAN are based on FC

Figure 3-2-3 shows the various components of SAN:

Figure 3-2-3: SAN Components

The components used in a SAN are:

• SAN servers: Include servers with different operating systems, such as UNIX, Windows 2000, and Netware.

• SAN storage: Includes storage devices that store data.

• SAN interconnects: Represent components that connect SAN servers and storage devices. The SAN interconnects components are:
  
  

  • Cable and connector: Includes Multi-Mode Fiber (MMF) and Single-Mode Fiber (SMF) fiber optic cables and connectors, which help to connect fiber-optic-based adapters in a SAN. The MMF fiber optic cable helps in connecting devices with a distance of 1.2 miles between them. The SMF fiber optic cable allows you to connect devices with a distance of 61.3 miles between them.

  • Gigabit interface converter: Specifies a laser-based, hot-pluggable transceiver that helps in data communication.

  • Host bus adapter: Specifies a device that connects to a server or a storage device and controls the electrical protocol and communications.

  • Extender: Specifies a device that connects servers and storage devices, which are located at long distances.

  • Hub: Specifies a device that connects storage devices, servers, and client computers.

  • Gateway: Specifies a device that helps interconnect one or more storage devices with different protocol support, such as SCSI to FC or FC to SCSI devices.

  • Switch: Specifies a device that helps connect a number of storage devices and servers.

  • Director: Specifies a device that connects Fibre Connection (FICON) or FC devices.

  
  

SAN Applications

There are various applications in which you can use a SAN to enhance the performance of the network in an organization. They are:

• Shared repository and data sharing

• Network architecture

• Data vaulting and data backup

• Data interchange

• Clustering

• Data protection and disaster recovery

Shared Repository and Data Sharing

SANs allow you to store data in a shared repository, which is a collection of storage devices shared by various servers. SAN also allows data sharing in which the data stored in the shared repository can be shared between servers for processing.

Figure 3-2-4 shows how data is shared between different servers:

Figure 3-2-4: Data Sharing

In a SAN, data sharing between servers can be of three types, which are:

• Storage sharing: Allows two or more heterogeneous or homogeneous servers to share data from a single shared repository. The storage area in the repository is partitioned and each partition is allocated to a single server. The server can only access the data stored in the storage area partition, which is allocated to it.

• Data-copy sharing: Allows one or more homogenous or heterogeneous servers to share data by sending a copy of the data from one heterogeneous or homogeneous server to another heterogeneous or homogeneous server.

• True data sharing: Allows one or more homogeneous or heterogeneous servers to access a single copy of the data stored in a shared repository.

Network Architecture

SANs allow you to have a network architecture, which is a combination of two networks, messaging network, such as Ethernet, and a SAN as the secondary network. In the network architecture, you can have a centralized storage system by using interconnected devices, such as hubs, switches, and gateways. The network architecture also allows a SAN to be a local or remote network and a shared or dedicated network. The network architecture of a SAN enhances the availability and performance of the storage devices.

Data Vaulting and Data Backup

In a SAN, you can perform data vaulting and data backup processes without the storage devices and servers being connected to each other in a LAN. In a SAN, the data backup process is fast because you need not connect storage devices and servers to each other in LAN to perform data backup process. Data vaulting is a process in which data is stored in storage devices to avoid loss of data due to failures, such as hardware failures. Data backup is a process in which you back up the data stored on a server to a storage device.

Data Interchange

Data interchange refers to transfer of data from one storage device to another. A SAN allows you to interchange data between different storage devices, which help to store and access data using methods, such as data encoding and file structure.

Clustering

The clustering process provides fault tolerance by connecting multiple servers to each other. In clustering, you can use another server if one server fails because clustering allows multiple servers to be connected to each other. SANs allow sharing of storage devices if you are using clustering in a network.

Data Protection and Disaster Recovery

SANs allow you to create duplicate copies of the data that are stored in storage devices to protect and recover data in case of a failure. You can use various methods, such as storage mirroring, remote cluster storage, and Peer-to-Peer Remote Copy (PPRC) to create duplicate copies for protecting data.

Managing SANs

SAN management involves management of servers, storage devices, and network system. A SAN is managed at three levels: storage, network, and enterprise systems. These levels help to simplify management of components used in a SAN, as shown in Figure 3-2-5:

Figure 3-2-5: Levels of SAN Management

SAN Storage Level

The SAN storage level contains storage devices, such as disks, tapes, and storage subsystems. SAN storage level management involves the management of storage devices and connected servers.

At the storage level, you can use three types of protocols: Small Computer System Interface (SCSI) protocol, SCSI-3 serial protocol, and American National Standard Institute (ANSI) SCSI-3 serial protocol, to manage storage devices. SCSI is a high-level protocol that supports various types of storage devices in a SAN. SCSI-3 serial protocol is higher version of SCSI that supports higher speed of data transfer on a SAN. The ANSI SCSI-3 serial protocol is compatible with FC technology.

The ANSI SCSI-3 serial protocol provides a set of services known as SCSI Enclosure Services (SES) that indicate the status of storage devices on a SAN. SES includes a set of commands, such as DIAGONOSTICS and RECEIVE DIAGONOSTICS RESULTS, for finding the temperature, fan speed, power supply status, and operational parameters from connected storage devices. You can configure the operational parameters of connected storage devices to obtain maximum output from storage devices.

Storage Management Tools

At the storage level, you can use various SAN storage management tools, which help monitor and manage storage devices. These SAN storage management tools are:

• IBM TotalStorage Enterprise Tape 3494 Library Specialist: Allows you to monitor and centrally manage the IBM Enterprise Storage Server (ESS), including the TotalStorage Virtual Tape Server.

• IBM TotalStorage LTO Tape Library Specialist: Allows you to monitor and centrally manage all the tape libraries in the LTO Tape Library Family.

• IBM TotalStorage ETL Expert: Allows you to monitor the status and performance of Tape Library and Virtual Tape Server.

• FAStT Storage Manager: Allows you to centrally manage the performance of storage devices. It also helps in storage partitioning and error reporting of storage devices.

At the storage level, you can also use IBM StorWatch storage management tool to manage storage devices on a SAN. The IBM StorWatch management tool helps to install, monitor, and configure storage devices. In addition, IBM StorWatch also provides storage management capabilities to storage devices and SAN fabric components, such as logical and physical disk devices, tape drives, FC switch, and FC hub.

IBM Storwatch Storage Management Tool

The IBM StorWatch management tool contains a set of software products and configuration tools for managing storage devices on a SAN. The various software products of IBM StorWatch are:

• StorWatch Serial Storage Expert: Monitors the Serial Storage Architecture (SSA) network devices for errors or faults. The StorWatch serial storage expert also allows you to organize, configure, and centrally manage SSA networks.

• StorWatch Versatile Storage Specialist: Allows you to configure, monitor, and centrally manage the IBM versatile storage servers.

• StorWatch Enterprise Storage Server Specialist: Allows you to configure, monitor, and centrally manage the IBM Enterprise Storage Server (ESS).

• StorWatch Enterprise Storage Server Expert: Allows you to manage the performance of ESS, keeps track of total capacity, assigned capacity, and free capacity on ESS anywhere in the enterprise, and identify the other servers accessing data from ESS.

• StorWatch Reporter: Provides capacity and utilization management for server platforms, such as Windows NT, OS/390, OS/400, AIX, and SOLARIS.

• StorWatch Fibre Channel Redundant Array of Inexpensive Disks (RAID) Specialist: Provides configuration, performance, error handling, and management solutions for FC RAID storage servers.

• StorWatch Data Path Optimizer: Provides dynamic allocation of paths between various connected storage devices on a SAN. It also provides load-balancing algorithm among connected host devices on a SAN.

SAN Network Level

SAN network level includes the management of various network devices, such as cables, FC switches, FC hubs, routers, and gateways. You can use the features of Simple Network Management Protocol (SNMP) to manage network devices on a SAN. SNMP is an IP-based protocol, which includes a set of instructions for receiving the status and operational parameters of network devices on a SAN. In SNMP based management, an SNMP manager manages all the network devices. All managed network devices are loaded with SNMP based applications. The managed data is arranged in a format called Management Information Base (MIB). The MIB contains the operational parameters of a particular networking device, which you want to measure.

Storage Networking Industry Association (SNIA) has defined standard MIBs for the network devices, such as FC switches, FC hubs, and gateways used in a SAN. SNMP manager obtains the status information of these network devices using the standard MIBs defined by SNIA. The network administrator observes and configures the operational parameters of network devices to manage the network devices in a SAN.

At the network level, you can use various SAN network management tools to monitor and manage network devices in a SAN. These SAN network management tools are:

• IBM TotalStorage SAN Data Gateway Specialist: Provides configuration, disk copy management, and device monitoring for IBM SAN data gateways.

• IBM TotalStorage SAN Switch Specialist: Provides configuration, network traffic management, alerts, and device monitoring for IBM SAN 2109 family switches.

• Fabric Manager: Provides monitoring and management of network devices in a SAN. For example, you can use the Fabric Manager tool to obtain the status of all switches. It also helps create error logs of network devices.

• Cisco Manager: Allows you to manage multiple switches. It also helps perform various functions, such as topology discovery, monitoring, and fault resolution of network devices.

• McDATA Enterprise Fabric Connectivity Management: Provides centralized management of multiple, distributed switches and directors. It also allows you to add network devices in a SAN.

• Emulex HBAnyware Management Software: Provides centralized management of HBAs. It uses driver-based technology for complete management of Emulex HBAs.

• EZ Fibre: Provides graphical user interface for easy installation and configuration of HBAs in Windows, Macintosh, or Solaris platforms.

• FINISAR SAN Performance Tool: Provides intelligent testing and monitoring of network device performance in a SAN.

Enterprise System Level

Enterprise system level includes management of the complete network system and host devices in an organization. Enterprise system management applications, such as Common Information Model (CIM), collect the management information of various enterprise network infrastructures, such as host devices, servers, LAN, WAN, and storage devices. Enterprise system management applications integrate the management data of the complete network infrastructure and present consolidated management information.

At the Enterprise System level, you can use various enterprise management tools to monitor and manage the enterprise network infrastructures. These enterprise management tools are:

• Web Based Enterprise Management: Provides management of enterprise network infrastructures using a Web interface.

• Common Interface Model: Integrates the management information of host devices and networks distributed over an enterprise.

• Application Program Interface: Helps write application program for the management of host and network devices in the enterprise.

• Java Management Application Program Interface: Provides management of Java computing platforms in the enterprise.

• Desktop Management Interface: Provides management of operating systems, and host and network devices, such as workstations, routers, and hubs.

SAN Topologies

A topology is the physical or logical arrangement of connected host devices on a network. In a SAN, you can use three types of topologies, which are:

• Point-to-Point topology

• Arbitrated Loop topology

• Switched topology

Point-to-Point Topology

Point-to-Point topology is a direct high-speed link between two connected host devices in a SAN.

Figure 3-2-6 shows the connected host devices using point-to-point topology in a SAN:

Figure 3-2-6: SAN Point-to-Point Topology

You use point-to-point topology when you want to transfer data between connected host devices in a SAN at high speed. For example, you can use point-to-point topology to transfer data between central processing units or from a host device to a specialized graphics processor.

Arbitrated Loop Topology

In Arbitrated Loop topology, you can use up to 126 host devices connected to each other in a single loop in a SAN. This Arbitrated Loop topology helps in data transfer at high bandwidth at a low cost. In Arbitrated Loop topology, the hosts establish a virtual connection with each other. When two hosts establish a virtual connection with each other, the complete bandwidth of the loop is dedicated to that virtual connection. The bandwidth of the loop is dedicated until the data transfer between the connected host devices is complete. You can connect loops to each other through connectors.

Figure 3-2-7 shows the SAN Arbitrated Loop topology:

Figure 3-2-7: SAN Arbitrated Loop Topology

Switched Topology

Switched topology allows you to increase the bandwidth of a particular network path, distribute the network traffic, and interconnect various host devices using FC switches in a SAN. The Switched topology helps manage networks that have large number of hosts. In a Switched topology, you can use up to sixteen million nodes and data rate of 100 MegaBytes/second between any two connected devices.

You can cascade switched topology by using multiple FC switches. Multiple FC switches help connect servers and storage devices. In cascaded Switched topology, you can interconnect and configure multiple FC switches to form one integrated switch. You need to properly manage the links between the switches so that if any of the switches fail, the functioning of the other switches should not be disrupted.

Optimizing SAN Performance

You can optimize SAN performance by planning, monitoring, analyzing, testing, and tuning a SAN. The first step in optimizing SAN performance is to plan what components of a SAN will be used, such as appropriate selection of connectors. In addition, you need to select SAN topologies, which will be used to connect host devices in a SAN. You can also monitor the performance of a SAN using monitoring tools, such as NetWisdom. For example, you monitor data traffic in a SAN and the working of SAN components using the monitoring tools. Analyzing and testing SAN performance helps find the errors that degrade the performance of a SAN. You can also tune the performance of a SAN to remove the errors, which degrade the performance of a SAN.

Planning SAN Performance

In the planning stage, you need to install servers in a SAN and plan for the clustering of servers to reduce the downtime caused due to the unavailability of servers. During planning, you also select connectors, such as hubs, routers, and FC switches that help in maintaining high data transfer rate in a SAN. For example, you can insert 500Mbps switch between two 1Gbps switches to reduce the data transfer rate in a SAN. In case of failure of a data path in a SAN, you need to allocate alternate data paths to FC packets so that the FC packets reach the destination host devices.

If you are implementing low capacity connectors in a SAN, you can use these connectors over less critical data paths and redundant fail over data paths in a SAN. You can also use a number of low capacity components instead of a single high capacity component to safely transmit data in a SAN. Using a number of low capacity components is advantageous because if one component fails, another component can take over the function of the failed component. For example, in RAID application, several low capacity storage devices are used instead of one large storage device for backup operation. In addition, you need to select SAN topologies based on your requirements while planning SAN performance.

After you have implemented a SAN, new storage devices, connected servers, and connectors in the network can change the behavior or performance of network. In addition, there can be a number of end user applications in a SAN that are accessing a single network resource. As a result, you need to provide multiple data paths to reach a single connected device so that multiple end user applications can access the network resources in a SAN at the same time.

Monitoring SAN Performance

Monitoring SAN performance helps in obtaining maximum performance from servers, storage devices, and connectors connected to each other in a SAN. There are various monitoring tools, such as NetWisdom and McDATA SANavigator, which allow you to obtain the status and value of operational parameters, such as data traffic in a SAN. You can also use the monitoring tools to monitor the functions of each connected device and SAN components.

Net Wisdom

NetWisdom is a tool for monitoring SAN performance. It helps obtain information about the SAN operational parameters, which you can use to increase the performance of a SAN. NetWisdom provides various operational parameters for monitoring SAN performance, which are:

• Comprehensive Monitoring: Compiles and analyzes detailed SAN operational parameters on FC technology.

• SAN Bandwidth Utilization: Shows an aggregate view of network traffic across the fabric of a SAN. It also shows the data bandwidth over a SAN and its utilization.

Note

Fabric specifies the network connection in a SAN using various network devices, such as hubs, switches, and gateways.

• Measurement of Applications Response Times: Measures the response time of applications between the source and destination hosts.

• Draw Performance Baselines: Constructs the performance baselines of a SAN and compares performance baselines to the real operating performance of a SAN.

• Problem Detection: Detects the main source of problem, so that it can be eliminated in real time.

• Trend Analysis: Prepares a log of performances over a period of time, and compares the performances with the current working of SAN to identify the deviation and root cause of problems.

• Custom Alarm: Sets user-defined alarms and informs the administrator when performance baselines are not achieved.

• Event Recording: Records each event that occurs in a SAN. This helps troubleshoot the problems that might occur in a SAN.

• Workload Balancing: Distributes the workload uniformly across the SAN.

• Centralized Control: Allows centralized management and monitoring of a SAN.

NetWisdom monitoring tool provides various advantages, such as accurate measurement of SAN performance. NetWisdom also helps maximize SAN performance, which provides reliability and minimizes downtime of data transfer. NetWisdom creates performance baselines to measure the real working performance of a SAN. NetWisdom contains three components, which are:

• Probes: Collects data through SAN components and calculates the data traffic and the transaction statistics over the SAN. Probes are individual hardware monitoring devices that are connected to the SAN fabric. These hardware devices extract the data rate locally between connected hosts in a SAN.

• Portal: Collects and stores the data from probes. This auto managed database integrates the transaction statistics over time, stores user-defined instructions, and sets custom alarm in probes.

• Views: Provides centralized monitoring of how the entire SAN works. You can view colored graphs and cycles through viewer software to analyze and interpret the data. NetWisdom Views help view the data paths and working of networking components. In addition, administrator can allow multiple authorized end users to access this software simultaneously.

McDATA SANavigator Tool

McDATA SANavigator tool helps monitor and manage the storage devices, networking components, and servers in a SAN. The functions of McDATA SANavigator are:

• Performance Monitoring: Helps measure the performance of each switch port and locate the bandwidth bottlenecks in a SAN. It also helps provide data path throughput and errors generated in a SAN.

• Event Management: Helps automate the routine tasks, such as error generation and device failure response in a SAN so that you can respond instantly when an error occurs in a SAN.

• Planning: Helps in planning how to increase storage capacity of a SAN.

• Improves Efficiency: Identifies the components of a SAN that are under utilized and over utilized so that you can balance the workload in a SAN.

• Securing Data: Prevents unauthorized access of data stored in the storage devices in a SAN. The zoning feature manages and configures zones for switches in a SAN.

Note

Zoning allows segmentation of the switched network in a SAN. Only members of the same zone can communicate within that zone, and all other attempts from outside are rejected.

Analyzing and Testing Tools for SAN Performance

Analyzing and testing tools help to analyze, diagnose, and display SAN performance. These tools identify and resolve the errors, which occur in a SAN. The various analyzing tools for analyzing SAN performance are:

• Taps: Allows monitoring, capturing, and analyzing of physical errors in a SAN. Fault tolerant and single port full duplex taps maximize the visibility of SAN performance.

• Xgig Analyzer Suite: Consists of software and hardware package, which analyzes the problem that degrades the performance of a SAN. This suite also helps test and evaluate the performance of a SAN.

• Expert: Analyzes and explains the network problem in a SAN. Expert also provides an overview of the problem with corrective action recommendations to resolve the problem.

You can use the following testing tools to test the performance of a SAN:

• Generators: Generate test data that help to test the data traffic of 1 GB/second and 2 GB/second in SAN.

• Jammers: Insert controlled, repeatable, and system level errors into a SAN. After inserting the errors, jammer checks for the recovery process to recover data lost due to the errors in a SAN.

• Bit Error Rate Testers: Provide low level testing, such as testing bit error rate of network devices in a SAN. This testing tool also helps verify data integrity by sending worst-case data patterns through network devices.

• Traffic Check: Monitors the status parameters, error conditions, and performance measures of network devices in a SAN.

• Bus Doctor: Connects with multiple buses, such as iSCSI and FC of 1GB/s and 2GB/s. Bus doctor can capture up to 256 million events, which occur in a SAN in their dedicated trace buffers.

• PacketMaker: Generates data traffic for high-speed serial storage buses, such as serial attached SCSI. PacketMaker also allows you to send illegal packet structures, field values, and other errors to test your serial storage device error handling.

Tuning SAN Performance

When tuning SAN performance, you eliminate the errors in the connected device or the single data path in a SAN. In addition, you can set a threshold performance of a SAN. If the threshold performance is not achieved, SAN performance monitoring tools provide alerts for the error in a SAN.

Visual indicators in SAN performance monitoring tools help isolate the degraded performance data path or component of a SAN. If the error is related to the data path and this data path contains various storage devices, servers, and connectors, the error can be related to any connected device in this path. You need to analyze all the connected devices in the degraded performance data path to troubleshoot the error and tune the SAN performance.

You should monitor the performance of all connected devices in the data path and identify whether any of the devices has degraded the performance of the data path. If the data path uses full data bandwidth, you need to provide additional bandwidth capacity in the data path. If the data bandwidth is within limit in the data path and the source device performs well but the destination host device is not responding, the connected device or connection contains an error. You also need to replace the cable or connector, if the performance degrades.

Venu Poddar

A Storage Area Network (SAN) is a high-speed storage network within a LAN. It enables groups of servers to access data in the form of blocks from storage devices. All servers in a SAN have permissions to access the storage resources. A SAN provides concurrent server access on LANs or WANs to all the storage devices. A SAN ensures scalability, reliability, security, low network traffic, improved data availability, and transparent backups and restores.

This ReferencePoint describes the SAN components and the design considerations in implementing a SAN. It also describes the SAN topology and protocols. Finally, it explains the SAN implementation process.

SAN Components

The following devices and components constitute a SAN:

• Servers

• Storage Devices

• Interfaces

• Interconnects

• Applications

The SAN architecture depends on the User Interfaces (UIs), interconnects, and fabrics used on the network. This architecture governs the performance of the SAN.

SAN Servers

A SAN supports a heterogeneous server environment and operates on several server platforms, such as UNIX and its versions, Windows NT, Windows 2000, various versions of Novell NetWare, Linux, HP-UX, and Sun Solaris. One of the limitations of the heterogeneous environment is the inability of the servers to share data among themselves because the file systems are different. This limitation is overcome by using data conversion applications that convert data from one file system to another.

SAN Storage Devices

A typical storage device in a SAN stores large amounts of data to avoid the data scattering over multiple storage devices. SAN storage devices are stored outside the primary network, enabling even distribution of storage devices across networks. The following storage devices are used in a SAN:

• Just a Bunch of Disks (JBOD)

• Disk arrays

• Optical storage libraries

• Tape libraries

• Disk storage systems

SAN Interfaces

SAN interfaces connect shared storage devices to SAN servers using technologies, such as Fibre Channel. These interfaces facilitate the data storage outside the SAN server. Table 3-1-1

describes several common SAN interfaces:

Table 3-1-1: SAN Interfaces

Interface	Description
SCSI	A high-speed parallel storage device interface that connects a large number of devices, such as JBODs, disk arrays, and optical storage libraries to a SAN across long distances.
High Performance Parallel Interface (HIPPI)	Provides links to transfer data at high speeds, such as 100 and 200 Mbps.
Bus and Tag Interface	Connects a SAN server directly to Mainframes at high speed using Peripheral Component Interconnect (PCI).
Fibre Channel Arbitrated Loop (FC-AL)	Connects a large number of storage devices with SAN servers that are SCSI-compatible, fault-tolerant, and transfer data at high speeds.
Enterprise System Connection (ESCON)	The protocol that connects SAN components with IBM switches at low speed in half duplex mode.
Fibre Connection (FICON)	The protocol that connects SAN components with Fibre Channel IBM switches in full duplex mode.

SAN Interconnects

SAN interconnects connect the interface topologies with the Fibre Channel fabrics. Several of the SAN interconnects, such as cables, adapters, connectors, hubs, routers, multiplexors, and bridges, together form a Fibre Channel fabric.

Fibre and copper cables are the two types of cables that connect SAN components with storage devices. Copper cables are used for distances of up to 30 meters. The two types of fiber cables follow:

• Multimode fiber cables: Sets the connection over distances ranging up to 2.56 kilometers.

• Singlemode fiber cables: Sets the connection over distances ranging up to 68.8 kilometers.

Connectors are used to connect fiber-optic devices with copper devices in a SAN network for data transmission. Media Interface Adapters (MIAs) convert fiber-based to copper-based connections.

Adapters act as an interface between SAN interfaces and host buses to facilitate proper communication between them. Host Bus Adapters (HBAs) and Network Interface Cards (NICs) are examples of adapters, which communicate using TCP/IP, FICON, and ESCON protocols.

A hub is a centralized device in a SAN network that connects various SAN devices. A fiber hub can connect up to 120 nodes. Every port on a hub has a Port Bypass Circuit (PBC) that prevents a failed device from affecting other working devices.

Bridges provide communication across networks using various protocols. They help in connecting SCSI and Fibre Channel interfaces. Multiplexors are bridges that enable the transmission of signals from multiple devices through a single transmission media.

Redundant Array of Inexpensive Disks (RAID) provides an array of storage devices that are fault-tolerant, creating a large virtual drive that reduces the need for multiple drives. Routers transfer data across networks using the addressing schemes and transmission media. For example, storage routers use the Fibre Channel Protocol (FCP) to transfer data across networks. Gateways connect individual LANs over WANs. Switches connect SAN devices with each other but unlike hubs, they do not share the network bandwidth. They pass signals to specific ports and reduce the network traffic. Figure 3-1-1 shows the relationship between various SAN interconnects:

Figure 3-1-1: Components of a SAN

SAN Application Components

SAN application components include the following:

• Applications to configure and manage the SAN fabric using management software such as:
  
  

  • Tivoli SANergy from IBM: Allows simultaneous sharing of the same storage, file systems, and the same files among multiple computers connected to a SAN.

  • Tivoli Storage Network Manager from IBM: Discovers, displays, allocates, monitors, automates, and manages various components of SAN Fabric and disk storage resources.

  • SmartClient from BakBone: Allows network administrators to centrally control the attached media devices.

  • SANworks Enterprise Network Storage Manager and SANworks Storage Resource Manager from Hewlett Packard: Reduces the load on the network server and creates procedures and functions to perform specific network storage and management-related tasks.

  
  

• Applications to make regular backups and perform restore operations, volume managers for remote mirroring, disk striping, data replication, and other network management software. Examples of these management applications are the Tivoli Storage Manager from IBM and NetVault Dynamically Shared Drives (DSDs).

• Applications to share files, folders, and file systems. These applications use zoning and Logical Unit Number (LUN) for masking. For example, the SV Zone Manager from Viacom and SANPoint from Veritas provide centralized management and storage network administration.

SAN Design Methodology

Efficiency of any network depends on the design of the existing network, the components, and the storage systems used. To design and optimize a successful SAN setup, you need to perform the following steps:

1. Assess

2. Plan

3. Implement

4. Manage

In the Assessing step, information is gathered about the existing network setup and the requirements for designing a SAN. Site auditing collects all the required information to set up a SAN. It is possible to either change the SAN design to adapt to the existing LAN structure, or the existing LAN structure can be changed to accommodate a SAN.

Site audit also helps in sorting out various business and technical issues and helps check SANs compatibility with other systems on the network. It helps in planning decisions and future tasks for designing a SAN. For example, when choosing storage devices for a SAN, their compatibility with the existing network systems is considered. These phases help decide the topology of the SAN setup.

In the Planning step, a decision is made about the devices, network topologies, and protocols to be used for the SAN design and implementation. This step also includes specifying business and technical goals, the quality of service, and the design infrastructure for successful SAN setup. A complete SAN document is created that specifies all the information pertaining to design.

The Implementing step includes the actual setting up of a SAN by going through the detailed document prepared during the planning step.

The Managing step administers a successful SAN implementation. The step performs all the SAN management and administration tasks that help a SAN function.

SAN Design Considerations

A SAN design is customized based on the needs, requirements, and limitations of an organization. While performing SAN customization, you need to evaluate the following:

• Business requirements

• Storage pooling

• Data availability

• Heterogeneity

• Network performance

The priority for implementing any of these considerations varies with different organizations.

SAN Business Requirements

A SAN can enhance data storage and a retrieval solutions for many enterprise activities:

• Enterprise Resource Planning (ERP): Operates on the enterprise resources, such as finance, manufacturing, and product distribution in an organization. ERP requires high transaction speed because it works on critical organizational data.

• Online Transaction Processing (OLTP): Responds immediately to client queries. Data-processing speed is important for the success of OLTP.

• E-business and e-commerce: Occurs between business organizations or between an organization and a customer.

• Data Warehousing: Separates the static data of an organization from the operational data and serves as a data repository for data mining tools. Extracting and storing data that is specific to a business is called data mining.

Storage Pooling

Storage pooling is an important factor when designing a SAN. There are three types of storage pools - Local, Centralized, and Distributed. A local pool provides one-to-one data connection between a storage device and a server and uses the same fabric switch. Figure 3-1-2 shows a local storage pool:

Figure 3-1-2: Local Storage Pool

A centralized storage pool provides a one-to-many data connection between multiple servers and the storage system. Figure 3-1-3 shows a centralized storage pool:

Figure 3-1-3: Centralize Storage Pool

A distributed storage pool provides a many-to-many connection between multiple servers and storage systems. Figure 3-1-4 shows a distributed storage pool:

Figure 3-1-4: Distributed Storage Pool

Note

It is useful to have a local or a centralized storage pool when designing a local or a remote SAN.

Heterogeneity

SAN supports multi-vendor platforms and operating systems, such as Windows NT, Windows 2000, Linux, Solaris, and Macintosh. The performance of a SAN is judged on how easily SAN integrates with these heterogeneous environments. Before building a SAN, the number of fabric switches and the total number of ports is calculated. When designing a SAN structure, you should keep the future requirements of the organization in mind. The structure should have the flexibility to support new devices without affecting the existing network design and performance.

Network Performance

Performance is an important aspect when designing a SAN. Several factors contribute to the performance of a SAN follow:

• Data accessibility

• Device connectivity

• Network manageability

• Fault tolerance

The physical structure of an existing network is an important consideration when designing a SAN. Proper placement of storage systems with respect to the SAN servers helps in designing a performance-oriented SAN. The physical layout of the network, such as the distance between the storage device and the servers, helps determine the actual cost for setting up a SAN.

Storage Environment Selection

You need to select an appropriate storage device before designing a SAN. For example, Redundant Array of Inexpensive Disks (RAID) provides an array of drives and storage devices that are fault-tolerant and can be used in a SAN design. In RAID, data is stored in several drives. All the levels of RAID provide various performance and redundancy levels. The basic RAID level is RAID 0 and does not offer any redundancy.

Redundancy in RAID is provided through virtual disks, ensuring data protection from disk failures. Virtual disks provide high-level fault tolerance and allow damaged disk swapping without affecting network performance. These disks are suitable for heavy database transactions, which are generally levels 3 and 5. Table 3-1-2 lists RAID levels, their descriptions, and use:

Table 3-1-2: RAID Levels

Level	Description	Use
0	Is the simplest RAID level and stores data on all the drives.	Provides maximum throughput at reduced cost.
1	Uses the mirroring concept of storing data and creates a copy of the data on multiple drives.	Provides high-level security to data. It is expensive to have multiple drives in this level.
3	Stores data using the parity concept and stores the parity values on a separate drive.	Provides a high data transfer rate and is less expensive than other RAID levels.
5	Stores data using the parity concept and stores parity values across multiple drives.	Provides high-level security, can be used with multiple applications, and can withstand single drive failure.
6	Stores data using the parity concept and stores parity-related information on stripped drives along with data.	Provides considerable security and high read speed.

Data Access Using SAN

In a SAN setup, multiple heterogeneous server platforms can access a storage device simultaneously. Several of the ways to access data from storage devices follow:

• Logical Disk Volume Partitioning

• Physical Disk Volume Partitioning

• File Pooling

• Sharing Data

Logical Disk Volume Partitioning

In logical disk volume partitioning, logical disk volumes are created and assigned to various servers to store data. A logical disk can be stored on a single storage resource or across multiple devices. The storage controller manages the logical volumes and restricts other servers from accessing the disk.

Physical Disk Volume Partitioning

The simplest and most commonly used method to access data from a storage resource in a heterogeneous environment is through the physically partitioned storage disk volumes. Every server is assigned an exclusive set of disk volumes using this method. The assigned disk volumes become inaccessible to other servers after they are assigned.

File Pooling

In the file pooling data access method, a mountable namespace, instead of a disk space, is assigned to a server. Namespace enables a partition set to add or remove file sets even if the server is running. The size of the namespace varies depending on the size of the data file being accessed.

The mountable space is released after the server accesses the files. All other servers can then use this disk space to access files. The namespace has the advantage of enabling various file systems to share the same storage space and add a large number of hard drives.

Sharing Data

The sharing data method enables data access in two ways:

• Sharing data copy: Allows the server to copy the data. After a file is copied to the server, it is updated at regular intervals whenever the file changes.

• True data sharing: Allows simultaneous data access to multiple servers. The three ways by which a server can access data are:
  
  

  • One-at-a-time access method: Allows sequential data access to servers on a first come-first served basis.

  • Multiple read access method: Allows the requesting servers to read data simultaneously. Updates are performed one at a time using this access method.

  • Multiple read/write method: Allows simultaneous reading and data updates by the servers. The limitation of this access method is that it can lead to data conflicts due to simultaneous updates.

SAN Topology and SAN Protocols

A topology depicts the logical arrangement of nodes on a network. A node can be any device attached to the network. When selecting a topology for SAN implementation, consider the present network type, the amount of network span, and the required storage space.

Simple Topology

A simple point-to-point topology establishes a direct connection between a server and a disk array. This is the simplest of all the SAN topologies with a minimal set of options. It provides easy connection and fast access to the disk array. This topology is unreliable because it contains various single points of failure, such as the server, cable, HBA, and disk array controller. Figure 3-1-5 shows a simple topology with a point-to-point connection between a server and a disk array:

Figure 3-1-5: Simple Topology with a Point-to-Point Connection

A simple point-to-point connection can be extended because disk arrays support two redundant controllers. A new HBA can be added to the server and then connected to the disk array. In this topology, the single point of failure is the server because it contains two HBAs. The server works even when the cable, the disk controller array, or the HBA fails. Figure 3-1-6 shows a two-path HBA point-to-point connection:

Figure 3-1-6: Two-Path Point-to-Point Connection

Adding another server can extend the point-to-point topology further. In the figure, both the servers are connected individually to the disk controller. Adding another server increases the single point failure, which includes servers, cables, disk arrays, and HBAs. Figure 3-1-7 shows the point-to-point connection between two servers:

Figure 3-1-7: Two-Server Point-to-Point Connection

Cluster the servers to eliminate any single point failure. Clustering provides access to the disk arrays even if one server fails. Figure 3-1-8 shows a clustered point-to-point connection:

Figure 3-1-8: Clustered Point-to-Point Connection

FC-AL Topology

Adding a SAN interconnect to a simple point-to-point connection forms a FC-AL topology. This topology is helpful in a SAN setup where storage devices are placed at long distances from the servers. A hub is used to interconnect servers and disk arrays. A hub contains two connections from each server, while two links from the hub are connected to the disk array controller.

In the FC-AL topology, the hub is the single point of failure. An entire network ceases to function when a hub fails. This topology is ideal in a small organization with only a few servers that can tolerate the downtime. Figure 3-1-9 shows a simple FC-AL topology:

Figure 3-1-9: Simple FC-AL Topology

You can extend the simple FC-AL topology by adding another hub to create an error-free structure. This structure can be used in a SAN setup that does not require immediate scalability. Organizations use this topology to build individual SAN structures. You can easily expand the SAN setup using this topology. Figure 3-1-10 shows the fail-proof Fibre Channel topology:

Figure 3-1-10: FC-AL Topology with Two Hubs

Fabric Topology

Fibre Channel fabric is a switch-based Fibre Channel connection network. Switches transfer information across various ports. Fabric-based topology enables end users to access the data storage devices simultaneously and also connects long distance networks. A switch acts as a SAN interconnect for different systems across long distances.

The advantage of using fabrics over the FC-AL topology is that they provide quicker data access. Fabric-based topology contains a switch or higher SAN interconnect instead of a hub. Figure 3-1-11 shows the simple fabric-based topology:

Figure 3-1-11: Simple Fabric-Based Topology

SAN supports a large number of protocols, such as Fibre Channel over Internet Protocol (FCIP), Internet Fibre Channel Protocol (iFCP), Transmission Control Protocol/Internet Protocol (TCP/IP), and Internetwork Packet Exchange (IPX). It also supports storage protocols, such as Storage IP and Infiniband. FCIP and iFCP are the most commonly used SAN protocols.

Fibre Channel Over Internet Protocol

FCIP enables Fibre Channel data transfer by tunneling data between SAN- and IP-based networks. Tunneling is the technique of transmitting private data of an enterprise network through a public network. FCIP transmits the data in a way that the routing nodes in the public network are unaware that the data of a private network is transmitted. This feature enables data sharing across the enterprise network.

FCIP enables quick data transmission in SAN and depends on IP-based network resources to interconnect SAN over LANs, MANs, and WANs. It encapsulates Fibre Channel frames within TCP/IP to link Fibre Channel SANs over WANs. The primary purpose of an FCIP protocol is to create and administer multiple FCIP links to transfer data from one SAN to another. The basic functions of an FCIP entity are to forward Fibre Channel frames and perform Fibre Channel frame encapsulation.

Internet Fibre Channel Protocol

The iFCP deals with the backup and storage data operations in a SAN. It is a gateway-to-gateway protocol, which provides the services of Fibre Channel fabric to Fibre Channel end devices within a TCP/IP network. iFCP helps extend Fibre Channel SANs over the Internet. It helps pass data to and from Fibre Channel storage devices in SAN using TCP/IP. The TCP component manages congestion in data flow, error checking, and data retrieval services for the Fibre Channel SANs. iFCP supports connectivity over long distances and also supports various data rates, media types, and connectors.

iFCP supports various types of communication within a network environment. The communication devices can vary from various Fibre Channel end-devices to an IP-network to SAN. The types of iFCP communication taking place within a Fibre Channel frame follow:

• Device-to-device: iFCP communication does not need a supporting Fibre Channel fabric because the data transaction occurs between similar type of devices.

• Device-to-SAN: iFCP communication needs a supporting Fibre Channel fabric to perform the data transaction. This fabric performs data mapping between the Fibre Channel devices and SAN components.

• SAN-to-SAN: iFCP communication uses an existing Fibre Channel fabric to perform data transaction by incorporating a TCP/IP network within a SAN.

Implementing SAN

To implement a feasible SAN network, the following are certain organization-specific issues that need to be resolved:

• Data transfer rate for transferring and accessing data across the network.

• Number of servers sharing and accessing information across the network.

• Distance at which the network needs to be spanned.

Constructing a SAN

The construction phase of SAN implementation follows the design consideration, component selection, and the fabric design phases. For this phase, an experienced and skilled team is required with an extensive working knowledge of heterogeneous platforms. After the implementation phase, all the design schemes and implementation considerations are documented for future reference. Some of the key points that the documentation should contain are the following:

• Requirements: Include all business requirements for implementing SAN as a solution.

• Design considerations: Include a list of possible limitations that may be encountered during SAN implementation.

• Logical and physical layout: Include recorded information related to the storage structure design of a SAN. The documentation should contain clearly drawn diagrams that depict the entire SAN setup.

• Positioning of storage devices: Include recorded information pertaining to the physical location of various storage devices that are used in the SAN network.

• Infrastructure information: Include the details of the existing infrastructure and information about future additions and updates should be recorded in the documentation.

• Topology details: Include details of the topology and the cabling system to decrease future debugging efforts.

• Applications and management tools: Include details of the applications running on various SAN components.

Implementing Backup Solutions

Backup is the process of duplicating data from the primary storage device on to the secondary storage. Organizations back up important data as part of their risk management process and store the data for future reference. The size of the secondary storage device depends on the size of the organization. It is a time-consuming process and generally requires network downtime.

Some of the important features of the SAN backup architecture that help in implementing a robust backup mechanism are the following:

• Tape mirroring: Provides safe and error-free data recovery. A mirroring controller supports multiple tape units to provide a reliable backup solution.

• Clustered server: Provides faster recovery of backed up data, using SCSI multiplexers shared by servers for daily backups. Multiplexers perform backup operations for the entire cluster without affecting the network performance.

• Remote backups: Enables SCSI multiplexers and Fibre Channel bridges to be placed at a remote site to back up data on a tape library. SCSI and Fibre Channel bridges are linked using Fibre Channel. Figure 3-1-12 shows local and remote backups using the mirroring technique:

  
  Figure 3-1-12: Backup from Local and Remote Sites Using the Mirroring Technique

The backup retrieval process involves restoring data from secondary storage devices to the primary storage device. There are two types of backups - Full and Incremental.

A full backup copies all the files to the secondary storage device, which can be done at the end of the day. In an office environment, every day new files are created and existing files are updated. In the full backup process, newly created files in addition to the updated files are backed up to secondary storage devices. In incremental backup, only modifications and new files are copied to the secondary storage file. Full backup is safer, but consumes more time and space.

The Distributed Backup Method

The distributed backup method enables data to be backed up when the backup devices are connected directly to the server. This process works well for organizations with a smaller network because the number of servers on a network determines the speed of the backup operation. In large organizations with multiple servers, the use of distributed technique results in slower backup. To manage backup operations efficiently, the backup administrator needs to store backed up data on disk drives. Figure 3-1-13 shows the architecture for distributed backups:

Figure 3-1-13: Distributed Backup Architecture

The Centralized Backup Method

The centralized backup method uses IP networks with a centralized backup repository. This method uses the CPU time of the server and slows down server performance. It also extends the backup time to go beyond the scheduled duration of the server.

An Ethernet LAN is used to transport data to the tape library. There are two servers in this setup and data is passed through the server twice before being copied to the tape library, which increases the network downtime. The centralized backup technique is cost-effective because backups can be scheduled using management tools and would not require the interference of the network administrator. Figure 3-1-14 shows the centralized backup architecture:

Figure 3-1-14: Centralized Backup Architecture

The Non-Centralized Backup Methods

In the non-centralized methods, a dedicated storage network is used to carry out the backup operation. Dissimilar to centralized storage, this method provides a higher data transfer rate. It provides all the advantages of the centralized backup method. In addition to providing reliable and efficient backup operations, SAN also provides data storage without using the server, which allows SAN to manage storage devices more easily. Figure 3-1-15 shows the backup method without a server:

Figure 3-1-15: Backup Architecture without a Server

In a backup method without a LAN, the LAN is free from data backup and data retrieval processes. To administer data flow from storage and backup devices, an administrator server is used. In server-less backup operations, a server is needed to supervise backup operations. The non-centralized backup methods provide higher bandwidth during the backup process. Figure 3-1-16 shows a Fibre Channel SAN architecture performing LAN-less backups:

Figure 3-1-16: Fibre Channel SAN Architecture Performing non-LAN Backups

In Fibre Channel SAN, the tape drives in a tape library increases the backup speed. As a result, SAN is a cost-effective and time saving solution because it eliminates the load on the server and the LAN.

SAN Virtualization

SAN uses storage devices that contain multiple disks and drives on a network. You may run out of drive letters when you attempt to map these drives. Proper planning is required to use the various drives for proper sharing of the available storage space.

SAN virtualization is the process of effectively using storage devices and drives. This virtualization process resolves all the storage space-related conflicts. The SAN virtualization process integrates the physical disks into a pool. The pooled physical disks are accessible to application servers without being mapped. The techniques for sharing the virtual disk capacity of the SAN are the following:

• Multiple host arrays

• Dedicated storage domain servers

• File system redirectors

• In-band virtualization engines

• Logical Unit Number (LUN) masking filters

These techniques vary from each other in terms of cost and application. Selecting a technique depends on the platform being used and the location of SAN.

The multihost storage array scheme provides high-level performance settings. Multiple platform connectivity support is also provided by the multihost storage array. Pooling is done at the storage level along with RAID. At times, the addition of extra pools results in the loss of centralized storage allocation. Figure 3-1-17 shows multiple host arrays with RAID:

Figure 3-1-17: Multiple Host Arrays with RAID

Dedicated storage domain servers allocate disk space to multiple hosts and deals with SAN virtualization. The virtualization process is applied on the network storage control layer above the operating system. This process controls the device operations, system networking, and SAN security features. It also enables the centralized administration of SAN and distributes management tasks to a large storage pool.

The number of domain servers required on the network and I/O performance can be optimized based on the design of the network. Optimized network enables server-less and LAN-less backups and data recovery across the storage pool. Figure 3-1-18 shows the dedicated storage domain server with multiple hosts:

Figure 3-1-18: Dedicated Storage Domain Server

The file system redirectors enable files to be accessed across the LAN for input/output across a SAN. Each SAN host requires software to map the addresses with the file names. The interface for the mapping process can be a File System Manager or an external metadata controller. File system redirectors help control disk I/O traffic. Figure 3-1-19 shows file system redirection operation:

Figure 3-1-19: File System Redirection using a Metadata Controller

Securing a SAN

Security can be implemented in SAN at either the hardware or software levels. Securing physical components of a SAN ensures hardware level security. Software level security is implemented with the help of SAN management applications. Two important SAN components that should be provided maximum securities are the SAN fabric and the storage file system.

The security of the SAN fabric depends on the Fibre Channel switches because all data passes through them. Fabric zoning is the best technique that can be used to secure data in the various storage systems in a SAN. This technique creates logical subsets in storage networks that can be accessed both at the device and port level. Creating logical subsets enables the network administrator to restrict devices or ports of one zone from accessing the devices or ports of other zones.

Port level zoning assigns devices or ports to various zones. Ports of the same zone can always communicate with each other. In device level zoning, an entire device is assigned to a single zone. Zoning enables a device or a port to be a member of more than one zone. This feature also allows specific devices or ports to access data that is stored in other zones. An example of device level zoning is switch-based zoning. Figure 3-1-20 shows switch-based zoning:

Figure 3-1-20: Switch-based Zoning

Fabric zoning enables the creation of a virtual SAN called SANlets. It is a logical storage network of SAN storage devices. SANlets have limited access to the resources that they share among themselves. The ncontrollers that have appropriate access and administration privileges. Otherwise, data is easily accessible to end users. Physical access to the storage controllers should also be controlled.