Contents

Dell Fluid Cache SAN 2.0.10V2.0.0 Storage Deployment Guide For VMware ESXi Systems PDF

1 of 54
1 of 54

Summary of Content for Dell Fluid Cache SAN 2.0.10V2.0.0 Storage Deployment Guide For VMware ESXi Systems PDF

Dell Fluid Cache for SAN Version 2.0.10 Deployment Guide for VMware ESXi Systems

Notes, cautions, and warnings NOTE: A NOTE indicates important information that helps you make better use of your computer.

CAUTION: A CAUTION indicates either potential damage to hardware or loss of data and tells you how to avoid the problem.

WARNING: A WARNING indicates a potential for property damage, personal injury, or death.

Copyright 2016 Dell Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. Dell and the Dell logo are trademarks of Dell Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies.

2015 - 05

Rev. A03

Contents

1 Dell Fluid Cache for SAN Product Overview....................................................6 Key Features...........................................................................................................................................6

Connectivity...........................................................................................................................................6

2 Requirements for Fluid Cache............................................................................8

3 Fluid Cache Deployment Overview................................................................. 11

4 Downloading Firmware, Drivers, and Software............................................ 12

5 Preparing the Fluid Cache Components........................................................ 13 Checking Security Settings..................................................................................................................13

Checking Network Connections........................................................................................................ 13

Checking Guest Virtual Machine Settings.......................................................................................... 14

Preparing the Servers.......................................................................................................................... 14

Preparing the Cache Devices..............................................................................................................14

Non-NVMe Cache Devices........................................................................................................... 15

Preparing the Cache Network Adapters............................................................................................. 15

Preparing the Cache Network Switch................................................................................................ 15

6 Installing and Setting up Fluid Cache............................................................. 17 Enabling SSH on the ESXi Host........................................................................................................... 17

Marking Devices for Passthrough....................................................................................................... 17

Configuring the Network Adapters.....................................................................................................18

Configuring Access to Compellent Storage.......................................................................................18

Configuring the iSCSI Network...........................................................................................................19

Two vSwitches on Separate Subnets with a Single VMNIC......................................................... 19

One vSwitch with Multiple VMNICS..............................................................................................19

Configuring Fibre Channel................................................................................................................. 20

Installing the Virtual Storage Appliance (VSA)....................................................................................20

Mapping Hardware.............................................................................................................................. 21

Configuring the VSA for Two iSCSI Subnets...................................................................................... 22

Configuring the VSA............................................................................................................................22

Fluid Cache Dependencies.................................................................................................................24

Fluid Cache Dependencies for Creating a Fluid Cache Cluster..................................................24

Fluid Cache Dependencies for Fluid Cache Configuration.........................................................24

Fluid Cache Dependencies for Starting Fluid Cache VSAs..........................................................24

3

7 Configuring and Managing Fluid Cache Clusters.........................................26 Creating a Fluid Cache Cluster...........................................................................................................26

Mapping Volumes in Fluid Cache....................................................................................................... 27

Mapping Volume to Servers.......................................................................................................... 27

Mapping Volumes to a Subcluster................................................................................................28

Adding Servers to a Fluid Cache Cluster............................................................................................29

Adding Cache Devices to a Fluid Cache Cluster............................................................................... 29

Adding Hardware to a VSA..................................................................................................................30

Adding a Storage Center to a Fluid Cache Cluster............................................................................30

A Maintaining Fluid Cache Installations.............................................................31 Removing Volume Mappings.............................................................................................................. 31

Removing Volume Mappings from a Server................................................................................. 31

Removing Volume Mappings from a Subcluster..........................................................................32

Removing a Server from a Fluid Cache Cluster................................................................................. 32

Removing a Cache Device from a Fluid Cache Cluster.................................................................... 33

Deleting or Removing a Fluid Cache Cluster.....................................................................................33

Deleting a Cluster..........................................................................................................................34

Removing a Cluster.......................................................................................................................34

Recreating a Fluid Cache Cluster....................................................................................................... 35

Upgrading Fluid Cache Cluster...........................................................................................................35

B Troubleshooting Fluid Cache Installations....................................................37 Basic Troubleshooting Steps.............................................................................................................. 37

The Mellanox Virtual Function is not Available for Passthrough in ESXi 6.0.....................................37

Server Does Not Appear in List of Servers......................................................................................... 38

Cache Device Does Not Appear in List of Cache Devices................................................................ 38

Cache Device Cannot Be Added to a Cluster....................................................................................38

Cannot Select a Specific Cache Mode ..............................................................................................38

Cached LUNs Are Unavailable............................................................................................................39

Enterprise Manager Freezes While Adding or Removing a Volume Mapping.................................. 39

Unable to Map Volumes to Fluid Cache using Enterprise Manager................................................. 39

Cannot Configure the Cache Network..............................................................................................40

Cannot Create a Fluid Cache Cluster.................................................................................................41

Cannot Map Volumes to a Fluid Cache Cluster.................................................................................41

Cannot Connect to Cluster Nodes.....................................................................................................41

Fluid Cache License Is Expired........................................................................................................... 42

Fluid Cache License Is Invalid.............................................................................................................42

Cannot Assign or Remove a Storage Center..................................................................................... 42

Cannot Determine Which Cache Device Failed................................................................................ 43

Events for Fluid Cache Are Not Shown in Enterprise Manager.........................................................43

Unable to PCI Passthrough a Samsung NVMe Flash Drive to Fluid Cache VSA through

VMware vCenter Web Client.............................................................................................................. 43

4

Fluid Cache VSA node not being added to Fluid Cache Cluster Server Object in Enterprise

Manager...............................................................................................................................................43

Cluster or Application Has Performance Issues................................................................................ 43

Unable to add a Fluid Cache ESXi 6.0 host to an existing server cluster..........................................45

Unable to install Dell Fluid Cache VSA properly on the following Fluid Cache Cluster

platforms............................................................................................................................................. 45

C Example Cabling Diagram................................................................................ 47 D Configuring a Dell Networking Switch..........................................................49 E Checking the System Configuration............................................................... 51

Checking the Cache Device Firmware and Driver............................................................................. 51

Non-NVMe Cache Devices........................................................................................................... 51

Checking the Network Adapter Firmware and Driver........................................................................51

F Related Documentation.................................................................................... 53 Accessing Enterprise Manager and Storage Center Documentation............................................... 54

5

1 Dell Fluid Cache for SAN Product Overview Dell Fluid Cache for SAN is a server-side caching accelerator software. Fluid Cache makes high-speed PCI Express (PCIe) SSDs a shared, distributed cache resource. Fluid Cache is deployed on clusters of Dell PowerEdge systems connected using RoCE-enabled Ethernet adapters and operates within a SAN environment employing a Dell Compellent backing store.

Key Features

Fluid Cache supports the following features:

Write-back caching.

Accelerates reads, writes, and read-after writes.

Up to nine nodes in the cache cluster.

Dynamic cache expansion.

Centralized configuration, management, and reporting.

Connectivity

Fluid Cache operates on the following networks and interfaces:

Cache Network A high speed, low latency private network that Fluid Cache uses to connect the servers in the Fluid Cache cluster.

Management Interface

The connection to Dell Compellent Enterprise Manager, which manages Fluid Cache and the SAN.

Storage Area Network

The network that Dell Compellent Storage Center uses to handle data connectivity within the SAN.

6

Figure 1. Fluid Cache Connectivity

1. Fluid Cache nodes communicate with each other over a private network using RDMA.

2. Enterprise Manager creates, manages, and monitors the Fluid Cache clusters.

3. Fluid Cache nodes communicate with the Management IP (VIP) of the Storage Controllers.

4. Fluid Cache communicates with vCenter to configure the VSA.

5. Enterprise Manager manages the Dell Compellent array.

The interfaces illustrated in the figure are associated with specific ports. See Checking Network Connections. To understand how the architecture is set up, see Example Racking and Cabling Diagram.

7

2 Requirements for Fluid Cache

The following are prerequisites for deploying Fluid Cache for SAN. Refer to the Release Notes for the most recent requirements.

The supported versions of firmware, drivers, and software are required to run Fluid Cache. These are listed in the Release Notes and are available at dell.com/support. For download instructions, see Downloading Firmware, Drivers, and Software.

NOTE: For the current list of validated components, see the Dell Fluid Cache for SAN Compatibility Matrix available at http://www.dell.com/CacheSolutions.

Table 1. System Requirements

Servers The cluster on which you deploy Fluid Cache must contain a minimum of three server per each cache cluster and a maximum of nine servers. All servers must support Single Root Input/Output Virtualization (SR-IOV).

Cache Devices There must be a cache device installed on at least two servers in the cluster.

Network Adapters A network adapter that supports Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) must be installed on each server in the cluster.

Network Switch A compatible switch must be available for the cache network. A redundant switch configuration is recommended.

SAN Management Software

Enterprise Manager must be started and configured to manage the Storage Center in use with Fluid Cache.

SAN Connectivity All servers in the cluster must be connected to the SAN and be displayed on the Dell Compellent array.

Table 2. Hardware and Software Requirements

Element Requirement

Servers Dell PowerEdge servers that support Dell Express Flash PCIe SSDs and SR-IOV:

Dell PowerEdge FC630 (FX2S chassis)

Dell PowerEdge M620

Dell PowerEdge M630

Dell PowerEdge M820

Dell PowerEdge R620

Dell PowerEdge R630

Dell PowerEdge T630

Dell PowerEdge R720

Dell PowerEdge R730

8

Element Requirement

NOTE: Dell PowerEdge R730 servers support only SR-IOV and does not support Dell Express Flash PCIe SSDs.

Dell PowerEdge R730XD

Dell PowerEdge R820

Dell PowerEdge R920

Dell PowerEdge T620

Operating Systems VMware ESXi 6.0 Update 1 (64-bit), VMware vCenter Server 6.0 Update 1

VMware ESXi 5.5 Update 2 (64-bit), VMware vCenter Server 5.5 Update 2

VMware ESXi 5.5 Update 3 (64-bit), VMware vCenter Server 5.5 Update 3

VMware ESXi 6.0 (64-bit), VMware vCenter Server 6.0

NOTE: vSphere Enterprise Plus license required for SR-IOV support (for Mellanox ConnectX-3 Adapters).

RAM and hard disk drive space

A datastore with 30GB of available disk space

Minimum 64GB RAM (the VSA uses 20GB)

Cache Devices Dell Express Flash PCIe SSDs (175GB, 350GB, 400GB, 800GB, or 1.6TB)

Micron HHHL PCIe SSD cards (700GB or 1.4TB)

Network Adapters Mellanox ConnectX-3 Dual Port 10 GbE SFP+ Adapter

Mellanox ConnectX-3 Dual Port 40 GbE QSFP+ Adapter

Mellanox ConnectX-3 Dual Port 10 GbE KR Mezzanine Adapter

Fibre Channel HBAs QLogic 2560, single-port 8GB, Optical Fibre Channel HBA

QLogic 2562, dual-port 8GB, Optical Fibre Channel HBA

QLogic 2660, single-port 16GB, Fibre Channel HBA, full height

QLogic 2662, dual-port 16GB, Fibre Channel HBA, full height

Emulex LPE 12000, single-port 8GB, Fibre Channel HBA

Emulex LPE 12002, dual-port 8GB, Fibre Channel HBA

Emulex LPe16000B, single-port 16GB, Fibre Channel HBA

Emulex LPe16002B, dual-port 16GB, Fibre Channel HBA

Cache Network Switches

Supported switches include:

Dell Networking N4032F

Dell Networking N4064F

Dell Networking S4048-ON

Dell Networking S4810

Dell Networking S5000

Dell Networking S6000

Dell Networking Z9500

Dell Networking MXL Blade

Cisco Nexus 5548UP

SAN Software Dell Compellent Enterprise Manager 2014 R2

9

Element Requirement

Dell Compellent Enterprise Manager 2014 R2

Dell Compellent Enterprise Manager 2015 R1

Dell Compellent Storage Center SCOS 6.5.2

Dell Compellent Storage Center SCOS 6.6.5

SAN Hardware Dell Compellent SC8000 controller

WARNING: Dell Fluid Cache for SAN requires Avahi traffic to run only on cache network interfaces running RoCE. Therefore, you must enable only the cache network running RoCE in the avahi-damon.conf file.

WARNING: Dell requires RoCE network dedicated only for Fluid Cache to avoid potential impact to traffic running on other interfaces including Oracle RAC traffic.

Optional: Fluid Cache Client Servers (do not have SSDs but participate in a cluster)

All Dell PowerEdge servers (or non-Dell servers with a supported operating system that can install a supported network adapter).

10

3 Fluid Cache Deployment Overview Complete the following tasks to deploy Fluid Cache for SAN. The sections referenced in each task give detailed instructions.

1. Download the required software, firmware, and drivers.

See Downloading Firmware, Drivers, and Software.

2. Identify and prepare the Fluid Cache servers and cache network switch.

See Preparing the Fluid Cache Servers and Cache Network Switch.

3. Install the Fluid Cache software on each server in the cluster.

See Installing and Setting Up Fluid Cache on ESXi.

4. Create and configure the Fluid Cache cluster.

See Configuring and Managing Fluid Cache Clusters.

For more information about managing Compellent arrays, see the Dell Compellent Enterprise Manager Administrators Guide available at https://portal.compellent.com/. Also, see Accessing Dell Compellent Documentation.

11

4 Downloading Firmware, Drivers, and Software

NOTE: It is recommended that you update servers using iDRAC. For more information, go to en.community.dell.com/techcenter/b/techcenter/archive/2013/04/17/idrac7-now-supports- updating-server-components-using-racadm-and-web-gui.aspx.

1. Determine which firmware, drivers, and BIOS are installed for your system components by following the procedures in Checking System Configuration.

2. Go to dell.com/support.

3. Enter the Service Tag for a supported PowerEdge server and click Submit.

NOTE: Entering the Service Tag for any supported server allows you to download the correct firmware, drivers, and software for all supported servers.

4. Click the Get Drivers & Downloads tab.

5. Select an appropriate operating system.

Make sure that you select the correct operating system.

6. Download the necessary firmware, drivers, and software:

NOTE: More than one download may be available for the same component. Identify the latest software or firmware based on the release date.

a. To download BIOS, select BIOS.

If you have different models of servers in your Fluid Cache cluster, download the BIOS versions for each model.

b. To download cache device firmware and drivers, select Solid State Storage.

c. To download network adapter firmware and drivers, select Network. Search for the Mellanox ConnectX-3 driver and firmware.

NOTE: Do not use a network adapter driver downloaded from the manufacturers site. Fluid Cache uses a custom driver provided only by Dell.

d. To download the Fluid Cache for SAN software, click Cache Solutions.

7. Download the files to an appropriate directory.

12

5 Preparing the Fluid Cache Components Before installing Fluid Cache, you must prepare the components of the Fluid Cache network: the servers, cache devices, network cards, and switches.

The instructions that follow assume that you have an existing SAN configured and managed. All nodes in the cache cluster must be connected to the SAN and visible on the Dell Compellent array. Instructions for racking and cabling a SAN solution are beyond the scope of this document. For more information about installing, upgrading, and configuring specific Dell components, see Related Documentation.

Checking Security Settings

Before you start the installation, make sure that firewall settings do not block traffic or devices on the cache network, or on networks used by Fluid Cache (refer to the table of port specifications here).

NOTE: If firewall settings are incorrect, the Fluid Cache cluster cannot be configured.

Checking Network Connections

For Fluid Cache to function correctly, each Fluid Cache server must be able to communicate with other network components. Make sure that the following ports are available:

Table 3. Ports Specifications

Interface Port number Port type

Fluid Cache cluster Configuration Manager 7449 TCP

Node Auto-Discovery functionality 5353 UDP

Web-based server status tool 6773 and 6774 TCP (https)

(optional) Web-based server status tool 8080 and 8082 TCP (http)

Fluid Cache to Enterprise Manager 3033 TCP

Fluid Cache to Storage Center 3033 TCP

Fluid Cache to vCenter 5480 TCP

NOTE: Fluid Cache cannot be configured and does not function properly if any of these ports are unavailable. To diagnose network connection problems , see the troubleshooting topic Cannot Configure Fluid Cache.

13

Checking Guest Virtual Machine Settings All guest VMs that use Fluid Cache volumes mapped to the ESXi hosts must be configured to start after the Fluid Cache VSA and stop before the Fluid Cache VSA. Make sure that any non-Fluid Cache VMs are not configured for a boot order priority of 1.

Preparing the Servers

Make sure that each server in the ESXi cluster has the latest supported BIOS version, Lifecycle Controller firmware, and iDRAC firmware.

For updating Dell Lifecycle Controller and BIOS firmware on 13th generation of PowerEdge servers, see the Dell Lifecycle Controller Graphical User Interface Version 2.05.05.05 For 13th Generation Dell PowerEdge Servers User's Guide, available at http://topics-cdn.dell.com/pdf/idrac8-with-lc- v2.05.05.05_User's%20Guide2_en-us.pdf

For updating Dell iDRAC firmware on 13th generation of PowerEdge servers, see the Integrated Dell Remote Access Controller 8 (iDRAC8) Version 2.05.05.05 User's Guide, available at http://topics- cdn.dell.com/pdf/idrac8-with-lc-v2.05.05.05_User's%20Guide_en-us.pdf

For updating Dell Lifecycle Controller and BIOS firmware on 12th generation of PowerEdge servers, see the Dell Lifecycle Controller 2 Release 1.4.0 User's Guide, available at http://downloads.dell.com/ Manuals/all-products/esuprt_software/esuprt_remote_ent_sys_mgmt/ esuprt_rmte_ent_sys_lifecycle_cntrllr/dell-lifecycle-cntrler-2-rel-v1.4.0_User%27s%20Guide_en- us.pdf

For updating Dell iDRAC firmware on 12th generation of PowerEdge servers, see the Integrated Dell Remote Access Controller 7 (iDRAC7) Version 1.50.50 User's Guide, available at http://topics- cdn.dell.com/pdf/integrated-dell-remote-access-cntrllr-7-v1.50.50_User's%20Guide_en-us.pdf

For other latest information related to Dell iDRAC and Dell Lifecycle Controller documentation, go to http://www.dell.com/support/home/us/en/19/product-support/product/idrac8-with-lc- v2.05.05.05/research

Modify your BIOS settings as follows:

1. Enter the system BIOS setup mode and click System BIOS Integrated Devices.

2. Select SR-IOV Global Enable.

NOTE: SR-IOV is required to create the virtual functions used by the Fluid Cache VSA.

3. Set System Profile Settings to Performance.

4. Click Back, and then click OK to return to the main BIOS page.

5. Restart the server to activate the BIOS changes.

6. Repeat this process for each server in the Fluid Cache cluster.

Preparing the Cache Devices

Make sure that all cache devices used in the Fluid Cache cluster have the minimum firmware and driver versions specified in Requirements for Fluid Cache. To download updated firmware or drivers, see Downloading Firmware, Drivers, and Software.

Fluid Cache supports clusters containing cache devices of different sizes, but for the most predictable performance, it is recommended that the cache devices be of similar capacity.

14

Non-NVMe Cache Devices

Firmware The firmware version for the Dell PCIe Express Flash SSDs must be B1490908 or later. Otherwise, even though they are visible in ESXi and in Enterprise Manager, the cache devices do not appear in the list of available devices.

NOTE: The installed firmware version cannot be verified in ESXi. If the cache devices do not appear in the list of available devices and you are not sure that your firmware version is B1490908 or later, update your firmware. Also, see Troubleshooting VMware Installations. You may also contact Dell Technical Support.

Driver The cache device driver used by Fluid Cache is native to the VSA and does not require updating.

CAUTION: If the cache devices contain data, back up this data before adding the cache devices to the Fluid Cache cluster. All data on the cache devices are lost when they are added to the cluster.

NOTE: If your server already has cache devices installed, make sure that the cache devices you intend to use in the Fluid Cache cluster are not used by any other application.

Preparing the Cache Network Adapters 1. Install the network adapters in PCIe slots on the servers (x16 slots, if available).

2. Check the network adapter driver and firmware versions, and then update, if necessary (see Checking the Network Adapter Firmware and Driver).

NOTE: It is recommended to use an MTU of 1500 for the cache network.

NOTE: Do not use a network adapter driver downloaded from the manufacturers site. Fluid Cache uses a custom driver available only through Dell.

3. If a blade enclosure is used, disable FlexAddress in the blade enclosure.

4. Repeat this process for each network adapter in the cache network.

5. Make sure that all servers in the cache network can ping every other server, and also the management interface.

NOTE: Make sure that all RoCE network adapters used by the Fluid Cache network are dedicated to the cache network and are not configured for any other network traffic.

Preparing the Cache Network Switch For a schematic of an example Fluid Cache installation, see Example Cabling Diagram.

For configuration tasks for a supported Dell Networking switch, see Configuring a Dell Networking Switch. For all other switches, see the manufacturers documentation.

For a list of supported switches, see Requirements for Fluid Cache.

1. Connect the ports on the network adapter to properly configured ports on a network switch.

NOTE: A redundant switch configuration is recommended.

If you have two switches for redundancy, on all of the cache network adapters, you must connect port 1 to one switch and port 2 to the other switch.

2. Make sure that all ports used in the cache network have the following settings:

The ports are in layer 2 mode.

15

The ports are in an untagged state.

The switch firmware is up to date.

Flow control (transmit and receive) is enabled and Data Center Bridging (DCB) is disabled.

NOTE: Enabling flow control is a requirement for Fluid Cache.

3. Save the running configuration.

4. To implement the changes, restart the switch.

16

6 Installing and Setting up Fluid Cache

NOTE: When installing and setting up Fluid Cache, you must restart the server multiple times.

Enabling SSH on the ESXi Host To enable SSH on the ESXi host, perform the following steps:

1. In the vSphere Windows client, select one of the servers in the Fluid Cache cluster.

2. In the Configuration tab, go to Software Security Profile.

3. In the Security Profile Services box, select Properties.

The Services Properties window is displayed.

4. In the Services Properties window, go to the SSH label, and under Service Properties, select Options.

The SSH (TSM-SSH) Options window is displayed.

5. Make sure that the Start tab is grayed out. If not, click Start, and then click OK.

6. Repeat this process for all servers in the Fluid Cache cluster.

Marking Devices for Passthrough Fluid Cache requires a direct (passthrough) connection between the VSA and the cache devices, bypassing ESXi itself.

1. In the vSphere Windows client, select one of the hosts in the Fluid Cache cluster.

2. In the Hardware area of the Configuration tab, click Advanced Settings.

The DirectPath I/O Configuration window is displayed.

3. In the DirectPath I/O Configuration window, select Edit.

The Mark Devices for Passthrough window is displayed.

4. In the Mark Devices for Passthrough window, select the first cache device.

Cache devices compatible with Fluid Cache contain either Micron RealSSD or Samsung Electronics in their name.

5. Read and acknowledge the warning message, and then click Yes.

6. Repeat steps 4 and 5 for each cache device in the cluster.

7. Click OK.

8. Restart the host.

NOTE: You do not need to restart the host at this time if you are continuing with the following procedure, Configuring the Network Adapters. The passthrough connection is enabled when the host is restarted at the conclusion of that procedure.

9. Repeat this procedure for all hosts in the Fluid Cache cluster.

17

Configuring the Network Adapters Before configuring network adapters, make sure that the network adapters are set up correctly. See Preparing the Cache Network Adapters. To configure the network adapters:

1. Download the network adapter driver for VMware.

See Downloading Firmware, Drivers, and Software

2. In the vSphere Windows client, select one of the host servers in the Fluid Cache cluster.

3. In the Hardware area of the Configuration tab, click Storage and select a datastore.

4. Right-click the datastore and select Browse Datastore.

The Datastore Browser dialog box is displayed.

5. In the Datastore Browser window, click the Upload symbol and select Upload File.

The Upload Items dialog box is displayed.

6. In the Upload Items dialog box, navigate to the directory where you downloaded the driver .zip file, select the file, and then click Open.

An upload/download operation message may be displayed.

7. Click Yes to acknowledge the upload/download operation warning message.

The file is uploaded and displayed in the Datastore Browser window.

8. Log in to the host either from the console or using SSH.

9. From the Tech Support Mode console on the host, or from the CLI, change to the directory where you uploaded the driver file.

For example, if the datastore name is datastore1, run the command: cd /vmfs/volumes/ datastore1 For more information about Tech Support Mode, see the Knowledge Base article 1017910 at kb.vmware.com.

10. Install the driver by running the command: esxcli software vib install -d /vmfs/ volumes/<datastore name>/<driver file name> --no-sig-check For example, if the driver filename is MLNX-OFED-ESX-1.9.10.2.zip, run the command: esxcli software vib install -d /vmfs/volumes/datastore1/MLNX-OFED-ESX-1.9.10.2.zip --no-sig-check

11. Enable SR-IOV virtual functions on the network adapter by running this command: esxcli system module parameters set --module mlx4_core --parameter-string max_vfs=1

NOTE: If you upgrade the firmware for the network adapter, this setting reverts to the default and you must re-enable SR-IOV virtual functions by running this command again.

12. Restart the host.

13. Repeat this procedure for all hosts in the Fluid Cache cluster.

Configuring Access to Compellent Storage

Fluid Cache requires access to a Compellent array on which you have storage that you want to cache. Fluid Cache can make use of an iSCSI or Fibre Channel connection to the Compellent array. If you want to access the Compellent storage using iSCSI, see Configuring the iSCSI Network. If you want to access your Compellent storage using Fibre Channel, see Configuring Fibre Channel.

18

Configuring the iSCSI Network

iSCSI networks with a single connection: You must add a virtual port group to the iSCSI vSwitch before installing the VSA. For instructions on creating a port group, refer to the topic Add a Virtual Machine Port Group available at the VMware Support website.

iSCSI networks with two subnets: Follow the procedure below.

In a configuration with two subnets, typically one subnet is used on port 1 of each Compellent controller and the second subnet is used on port 2 of each controller.

In ESXi 5.5, there are two recommended methods of using multiple network adapters between the host and the iSCSI environment:

Two vSwitches on separate subnets, each with one vmnic and one vmk port.

One vSwitch with two vmnics and two vmk ports, with the default switch failover order configured so that the vmk port on subnet A uses only the vmnic connected to subnet A, and the vmk port on subnet B uses only the vmnic connected to subnet B.

These configurations are described in detail in the vSphere Storage Guide, Chapter 11, available at the VMware Support website.

Fluid Cache needs to connect to the iSCSI network through a port group configured to match whichever method you are using. For instructions on creating a port group, refer to the topic Add a Virtual Machine Port Group available at the VMware Support website.

Two vSwitches on Separate Subnets with a Single VMNIC

Add a virtual machine port group to each vSwitch, labeling them to identify which subnet they correspond to.

One vSwitch with Multiple VMNICS

1. Add two virtual machine port groups to the same vSwitch, labeling them to identify which subnet they correspond to.

2. Edit each port group:

a. In the Hardware area of the Configuration tab, select Networking.

b. Locate the vSwitch and select Properties.

c. Select the first port group, and click Edit. d. In the NIC Teaming tab, select Override Switch Failover Order. e. Select one of the vmnics and use the Move Up and Move Down buttons to move it to the

Unused Adapters area, leaving only the other vmnic active for the port group.

f. Click OK.

g. Select the second port group, click Edit, and repeat tasks d through f, making the vmnic that was made inactive previously the only active vmnic for this port group.

Whatever your configuration, make sure the network card configuration you are using on the vmk ports is replicated on the virtual machine port groups that the Fluid Cache VSAs use. For more information, refer to the VMware Document Multipathing Configuration for Software iSCSI Using Port Binding available at the VMware Support website.

19

After installing the VSAs, you must edit the network adapter settings for the VSA (see Configuring the VSA).

For more information on setting up and administering an iSCSI network with two subnets, refer to the following documents:

Dell Compellent Storage Center Best Practices with VMware vSphere 5.x

Best Practices for Running VMware vSphere on iSCSI

Configuring Fibre Channel

If you plan to cache-enable volumes that are attached to the Fibre Channel network, one or more Fibre Channel HBAs must be enabled for the PCI pass-through feature, and then passed through to the VSA.

NOTE: If there is more than one Fibre Channel HBA configured on the server, cached LUNs are accessed using the HBA that is passed through to the VSA, and uncached volumes are accessed through other HBAs. If there is only one Fibre Channel HBA configured on the server, then all volumes presented to the cluster must be enabled for caching.

To access the Compellent Storage using Fibre Channel:

1. Install the Fibre Channel HBAs on the ESXi hosts that host the Fluid Cache VSAs.

2. Configure the Fibre Channel zoning on the Compellent array that has the storage you want to cache.

Installing the Virtual Storage Appliance (VSA) The following are the prerequisites for installing the Fluid Cache VSA:

You must have a licensed copy of ESXi 5.5 Update 2 and vCenter Server 5.5 Update 2.

You must have the Open Virtual Appliance (OVA) file containing the Fluid Cache software. See Downloading Firmware, Drivers, and Software.

To install the VSA:

1. In the vSphere Windows client, select one of the ESXi hosts in the Fluid Cache cluster.

2. Go to File Deploy OVF Template.

The Deploy OVF Template window is displayed.

3. In the Deploy OVF Template window, browse to the location of the OVA file and click Next.

The OVF Template Details window is displayed.

4. In the OVF Template Details window, click Next.

The Name and Location window is displayed.

5. In the Name and Location window, set a name for the VSA and click Next.

NOTE: Each Fluid Cache VSA must have a unique name. Consider using a name that references the name of the ESXi host in order to facilitate configuration of the VSA.

If you have multiple datastores, the Storage window is displayed.

6. Select a datastore and click Next.

The Disk Format window is displayed.

7. In the Disk Format window, select a provisioning method and click Next.

The Network Mapping window is displayed.

20

8. In the Network Mapping window, the following source networks must be configured to an appropriate destination network.

Management Network Configure with a destination network that can communicate to all the VSAs that participate in the Fluid Cache cluster.

iSCSI Network 1, ISCSI Network 2 Configure with a destination network that is connected to the iSCSI network. Using the network, the VSAs communicate to the Compellents that have the storage you want to cache.

NOTE: If you have configured Fluid Cache to access the Compellent storage through Fibre Channel, then do not change the default settings for the iSCSI destination networks.

9. Click Next.

The Ready to Complete window is displayed.

10. In the Ready to Complete window, review your deployment settings, and click Finish to start the deployment.

NOTE: Do not select the Power on after deployment option after adding the VSAs.

A dialog box displays the progress, which may take several minutes. When processing is complete, the VSA appears in the list of VSAs in vCenter.

11. Repeat this procedure for all servers in the Fluid Cache cluster.

NOTE: Do not attempt to install more than one VSA on an ESXi host. There must be only one VSA configured for each ESXi host.

NOTE: Make sure that the VSA is not migrated to another ESXi host using vMotion.

NOTE: If you need to set a custom MAC address or choose a different network for the management connection other than the default selection, right-click the VM, choose Edit Settings, and then edit the Network Adapter settings.

Mapping Hardware 1. In the vSphere Windows client, right-click a Fluid Cache VM and select Edit Settings.

The Virtual Machine Properties window is displayed.

2. In the Hardware tab, click Add.

The Add Hardware window is displayed.

3. In the Add Hardware window, select PCI Device from the list of device types, and then click Next.

4. Select the first cache device from the drop-down menu. Cache devices compatible with Fluid Cache contain either Micron RealSSD or Samsung Electronics in their name.

5. Click Next.

The Ready to Complete window is displayed.

6. In the Ready to Complete window, click Finish.

You may see a vCenter message indicating that data are no longer accessible to the host server. Accept the message, and then continue.

7. Repeat steps 36 for all remaining cache devices.

8. Repeat steps 36 for the network adapter virtual function. This is the item whose name contains Virtual Function in its description.

NOTE: Do not add the same device twice.

9. If Fibre Channel is used to access your Compellent Arrays, repeat steps 3 to 6 for the Fibre Channel HBAs that are connected to the Compellent Arrays that have the storage you want to cache.

21

10. When you have finished adding hardware components, click OK in the Virtual Machine Properties window.

11. Repeat this process for each Fluid Cache VM.

Configuring the VSA for Two iSCSI Subnets If your iSCSI connection consists of two subnets (a single iSCSI connection is also supported), follow this procedure to change the settings on each VSA in the Fluid Cache cluster:

1. In the vSphere Windows client, right-click one of the VSAs in the Fluid Cache cluster and select Edit Settings.

2. Change Network adapter 2 to use one of the iSCSI port groups.

3. Change Network adapter 3 to use the other iSCSI port group.

4. Repeat this process for each VSA in the Fluid Cache cluster.

NOTE: Make sure the addresses you assign to the vmnics are on the same subnets as their respective network adapters in the VSA.

When you follow the procedure in Configuring the VSA below, the second network adapter is the eth1 adapter in the Configure/Manage iSCSI Interfaces screen of the Fluid Cache configuration interface, and the third adapter is the eth2 adapter.

Configuring the VSA Before configuring the Fluid Cache VSA, make sure you have met the following prerequisites:

You have correctly cabled and configured all network adapters and network switches.

The vCenter user configuring Fluid Cache has the following privileges:

In the vSphere web client, for the Virtual machine privilege, under Configuration, Raw device must be selected.

For the Datastore privilege, the following must be selected: Allocate space, Browse datastore, Configure datastore, Low level file operations, and Remove datastore.

In Host Privileges, under Configuration, the following must be selected: Advanced settings, Change PciPassthru settings, Change settings, Connection, Maintenance, Memory configuration, Network configuration, Power, Storage partition configuration, System Management, System resources, and Virtual machine autostart configuration.

1. In the vSphere Windows client, right-click the VSA and click Power Power on.

2. Right-click the VSA and select Open Console.

After the VSA powers on, a login page is displayed. You are now in the text-based user interface (TUI) which configures the ESXi hosts and network interfaces of the Fluid Cache network. Use the arrow keys to navigate, press to select, and press to exit the current screen. Press < Alt> to exit the TUI.

3. Enter the login for the VSA and press .

The default login is fldc.

4. Enter the password for the VSA and press .

The default password is calvin.

22

NOTE: If you change the default login or password on any VSA in the Fluid Cache cluster, you must change them on every VSA. The login and password must be the same on all VSAs in the Fluid Cache cluster.

After pressing , the configuration home page is displayed.

5. Select Change vCenter and press to register the ESXi host with vCenter:

a. Enter the hostname or IP address for your vCenter and press .

b. Enter a vCenter username with the necessary privileges (See the and press .

NOTE: If this username or password changes, the cache cluster can no longer perform routine administrative functions. Follow this procedure again to enter the new username and password.

c. Enter the password and press .

You are returned to the configuration home page. The fields in the upper area now show information for the ESXi host. The entry for vCenter Connection should read OK.

6. Select View/Manage Appliance IP and press .

7. Select static IP addressing and press to begin configuring a static IP address:

a. Enter the static IP address and press .

NOTE: Static addressing is preferred. If you choose DHCP, the allocation method used by the DHCP server must prevent any changes to the VSAs IP address. The VSA is disabled if the IP address changes.

NOTE: If you need to change the IP addresses used for the management interface, these changes should be made from the vCenter console.

b. Enter the subnet mask and press .

c. Enter the gateway IP and press .

d. Enter the IP address of the primary DNS server and press .

e. Enter the IP address of the secondary DNS server and press .

You are returned to the configuration home page.

8. Select Configure This Appliance and press .

9. Configure your Backend Storage connection.

10. If you are accessing your Compellent Storage using iSCSI, perform the following steps:

a. Select Configure iSCSI and press .

b. Select Configure/Manage iSCSI Interfaces and press .

1. Enter the IP address or subnet mask for one or both of the iSCSI interfaces.

2. Press to get back up to the next level on the menu.

c. Select Configure/Manage Compellents and press .

1. Select Add Compellent and enter the IP address for the iSCSI interfaces on the Compellent.

2. Press three times to get back up to next level on the menu.

11. If you are accessing your Compellent Storage using Fibre Channel, perform the following steps:

a. Select View Fibre Channel Configuration and press .

The Fibre Channel HBAs page displays the list of available Fibre Channel HBAs.

b. Select each of the listed Fibre Channel HBAs and verify that the link status is reported as online.

12. Select Configure Cache Network and press .

23

The two iSER physical functions of the network adapter are displayed, and also the bonded interface. The bonded interface is within the VSA, while the two iSER interfaces are on the ESXi host.

NOTE: The bonded and two iSER interfaces must be on the same subnet.

13. Select one of the iSER interfaces and press to associate this interface with a virtual function:

a. Enter the IP address for the interface and press .

b. Enter the subnet mask for the interface and press .

14. Select the second iSER interface and press to configure the interface, then enter an IP address for the second interface and press again.

The subnet mask does not need to be re-entered. The second interface automatically uses the subnet mask entered for the first interface.

NOTE: If you choose not configure the second iSER interface, failover is not available for the system.

15. Select the bonded interface and press to configure the interface, then enter an IP address for the bonded interface and press again.

16. Select Finalize Setup and press .

NOTE: To check VSA connectivity, all of the cache network interfaces (iSER and bonded) should be pingable from each ESXi host.

Fluid Cache Dependencies

Fluid Cache Dependencies for Creating a Fluid Cache Cluster

Fluid Cache for VMware is delivered as an OVA. Therefore, to install Fluid Cache on ESXi host, make sure to create a local data store and install the OVA on the local data store.

After the Virtual Storage Appliances (VSAs) are installed and configured, use Dell Compellent Enterprise Manager to create and configure a Fluid Cache Cluster.

It is not mandatory for you to configure DNS for the Fluid Cache to work. However, if you have DNS configured, make sure that the DNS server works correctly.

Before creating a cluster, make sure a VMware vCenter server is properly functioning. Fluid Cache uses VMware vCenter to verify the ESXi server on which the Fluid Cache is running.

Fluid Cache Dependencies for Fluid Cache Configuration

Use Dell Compellent Enterprise Manager to do the following:

Configure the Fluid Cache cluster

Add or remove nodes

Add or remove cache devices

Map or unmap volumes

Use the VMware vCenter server to enable adding or removing nodes to the cluster.

NOTE: If Dell Compellent Enterprise Manager or VMware vCenter server runs on virtual machines, then make sure it does not resides on cache volumes.

Fluid Cache Dependencies for Starting Fluid Cache VSAs

Make sure the VMware vCenter server is functioning, because if VSA has stopped functioning or is manually turned off, the VMware vCenter server enables VSA to join the Fluid Cache cluster.

24

If the VSAs are configured to use DNS, then make sure the name resolution is available for the VSA to start.

25

7 Configuring and Managing Fluid Cache Clusters Make sure that your system meets the following prerequisites:

The Fluid Cache appliance must be installed and set up on all ESXi hosts to be used in the Fluid Cache cluster.

All ESXi hosts in the Fluid Cache cluster must be connected to the SAN and be visible on the Compellent array.

Enterprise Manager must be running and configured to manage the Storage Center to be used with Fluid Cache.

Your Enterprise Manager user account must have Administrator or Volume Manager privileges.

The Fluid Cache license file must be stored on the host running Enterprise Manager that is used to create the Fluid Cache cluster or on a shared folder available to it.

Creating a Fluid Cache Cluster 1. Log in to Enterprise Manager as a user with Administrator or Volume Manager privileges.

2. In Enterprise Managers Storage view, expand Storage Centers if necessary, and then select a Storage Center.

3. In the Storage pane, select Dell Compellent.

4. In the Summary tab, click Configure Fluid Cache Cluster to launch the configuration wizard. (Do not click Add FluidFS Cluster.)

The Discover Fluid Cache Servers page of the configuration wizard is displayed.

5. Type or select appropriate data in the Discover Fluid Cache Servers window.

a. In the Host or IP Address box, type the host name or IP address associated with the management network of any available Fluid Cache VSA.

NOTE: Do not reference a physical server here.

NOTE: Except for the entries in this Discover Fluid Cache Servers window, all other references to Fluid Cache servers in this document and in Enterprise Manager refer to the ESXi host and not the VSA. There is no further use of the VSA name or IP address when managing Fluid Cache.

b. The Port box is autopopulated. Change only if necessary.

c. In the User Name box, type the username for the VSA, which is fldc.

d. In the User Password box, type the password for the VSA. The default value is calvin.

NOTE: If you change these default properties on any node in a Fluid Cache cluster, you must change them on all nodes. The login and password must be the same on all nodes in the Fluid Cache cluster.

e. Click Next.

26

The Select Servers window is displayed.

6. By default, all available servers are selected. Clear the check box next to unwanted servers or select the Unselect All option, and then select three or more servers to be included. (Click Select All to use all available servers again.)

7. Click Next.

The Cluster Settings page of the configuration wizard is displayed.

8. Type or select appropriate data in the Cluster Settings window.

a. In the Name box, enter a name for the cluster.

b. Click Browse next to the License File box.

The Select Fluid Cache License File dialog box is displayed.

c. Browse to the location of the license file, select the file, and then click Save.

d. Verify that the license file and path displayed are correct and click Next.

The system processes for a few minutes while the cluster is created and the system automatically identifies compatible servers and devices on the network. After this process is completed, the Select Devices window is displayed.

9. By default, all available Fluid Cache devices are selected. Clear the check box next to unwanted devices or select the Unselect All option, and then select the required devices. (Select the Select All option to use all available devices again.)

CAUTION: You will lose any existing data on the cache devices when they are added to the Fluid Cache cluster. Back up this data before proceeding.

10. Click Next.

The Select Storage Centers page of the configuration wizard is displayed.

11. In the Select Storage Centers page, select one or more Storage Centers to include in the Fluid Cache cluster, and then click Finish.

After a delay while the system processes, the Storage pane contains a new top-level folder named Fluid Cache Clusters. Inside this folder is the Fluid Cache cluster that was just created. Fluid Cache clusters are denoted by a blue circle with the letters FC.

Select the Fluid Cache cluster. In the Summary tab, different areas of the window show the cluster's status, servers, devices, and other information.

Note that the servers listed are not the physical servers you added earlier, in the Select Servers window, but the ESXi hosts in the SAN. Also, the IP addresses listed next to these hosts are not the IP addresses of the ESXi hosts, but the management network addresses of the VSAs.

After creating a Fluid Cache cluster, map volumes to the cluster. See Mapping Volumes.

Mapping Volumes in Fluid Cache

In Enterprise Manager, volume mappings created for a server in a Fluid Cache cluster behave similar to any other volume mappings. Volumes can be mapped either to an individual server, or to a server cluster (a subcluster) within the Fluid Cache cluster.

If Fluid Cache is used with ESXi clusters, it is quicker to map to a subcluster than to each individual server.

Mapping Volume to Servers

Before mapping a volume to a server node, create a Fluid Cache cluster (see Creating a Fluid Cache Cluster).

27

To map a volume to a server:

1. In the Storage view, expand Storage Centers if necessary, and then select the Storage Center that contains the appropriate volume.

2. In the Storage tab, expand Volumes if necessary, and then locate the volume you want to map.

3. Right-click the volume and select Map Volume to Server.

4. In the Map Volume to Server window, select the server.

5. Click Next.

6. Select Enable Fluid Cache.

7. From the Host Cache Policy drop-down menu, select a cache mode:

Write-back (default): In addition to caching reads, write-back mode allows the caching of written data without waiting for the Compellent Array to acknowledge the write operation. Write-back caching requires a cache device on two or more servers in the cluster.

Write-through: Write-through mode forces writes to both the cache and the Compellent Array simultaneously. Warm reads and read-after writes are accelerated but write operations are not. Write-through caching requires only one cache device on one server in the cluster.

NOTE: The cache mode selected for a volume mapping cannot be changed. To select a different cache mode, you must remove the mapping and create a new mapping. When you create the new mapping, you can then select a different cache mode.

8. (Optional) Select the Keep cached data on the node that accessed the data option. Selecting this option gives the best performance for warm reads from the cache for datasets that fit on the local nodes cache devices, because all data is local to the client. If this option is not selected, cached data is evenly distributed among all cache devices.

NOTE: Use of this option with vMotioned guest VMs may result in reduced performance for that VM.

9. Click Finish.

At this stage, there is a delay while the system makes a number of configuration changes. When processing is complete, the volume is available for use in ESXi (as a datastore, RDM, and so on).

Mapping Volumes to a Subcluster

Before mapping volumes to a subcluster, create a Fluid Cache cluster (see Creating a Fluid Cache Cluster) and a server cluster (subcluster) within it. To map a volume to a subcluster, follow the procedure for mapping a volume to a server (see previous section), but instead of selecting a server in the Map Volume to Server window (step 4), select a subcluster within a Fluid Cache cluster.

Although not specific to Fluid Cache, be aware of these aspects of volume mappings created at the subcluster level in Enterprise Manager:

When servers are added to or removed from the subcluster, they automatically inherit or disinherit the subclusters volume mappings.

You can promote a volume mapping from a server to the subcluster, and demote it from the cluster to the subcluster.

If a server has existing mappings, the server keeps those mappings when it becomes part of a subcluster.

28

Adding Servers to a Fluid Cache Cluster To add servers to a Fluid Cache cluster:

1. Log in to Enterprise Manager as a user with Administrator or Volume Manager privileges.

2. In the Storage view, expand Fluid Cache Clusters if necessary.

3. Right-click the Fluid Cache cluster (not Fluid Cache Clusters itself) and click Tasks Add Servers to Cluster.

The Add Servers to Cluster dialog box is displayed. By default, all servers are selected that have Fluid Cache installed.

4. Clear the check boxes next to unwanted servers or select the Unselect All option, and then select the servers to be added. (To use all available servers again, select the Select All option.)

5. Click OK.

The system processes for some time, and then the server appears inside Fluid Cache Clusters in the Storage tab.

The Add Devices to Cluster dialog box is displayed. By default, all devices compatible with Fluid Cache are selected.

6. Clear the option next to unwanted cache devices or click the Unselect All option, and then select the cache devices to be added. (Select the Select All option to use all available cache devices again.)

CAUTION: Any existing data on a cache device is lost when the device is added to the Fluid Cache cluster. Back up this data before proceeding.

7. Click OK.

The device now appears in the list in the Devices section.

If you have a shared data application such as a cluster file system or clustered application, you may want to add the server to a server cluster (a subcluster) inside the Fluid Cache cluster. See Mapping Volumes. To create a subcluster, refer to the Enterprise Manager Administrators Guide.

Adding Cache Devices to a Fluid Cache Cluster 1. Log in to Enterprise Manager as a user with Administrator or Volume Manager privileges.

2. In the Storage view, expand Fluid Cache Clusters if necessary, and then right-click the Fluid Cache cluster. (Do not right-click Fluid Cache Clusters itself.)

3. Click Tasks Add Devices to Cluster.

The Add Devices to Cluster window is displayed. By default, all available devices compatible with Fluid Cache are selected.

4. Clear the option next to unwanted cache devices or click the Unselect All option, and then select the cache devices to be added. (Click the Select All option to use all available cache devices again.)

CAUTION: Any existing data on a cache device is lost when the device is added to the Fluid Cache cluster. Back up this data before proceeding.

5. Click OK.

The devices now appear in the list in the Devices section.

29

Adding Hardware to a VSA Before adding cache devices or network adapters to a Fluid Cache cluster, do the following:

1. Log in to the vSphere Windows client.

2. Right-click a VSA under the ESXi host, and click Power Power Off.

3. Confirm that you are turning off the VSA by clicking OK in the Confirm Shutdown dialog box.

4. Select the ESXi host.

5. In the Hardware area of the Configuration tab, click Advanced Settings Edit.

The Mark Devices for Passthrough window is displayed.

6. Select the cache devices for passthrough and click OK.

The DirectPath I/O Configuration window is displayed.

7. In the DirectPath I/O Configuration window, click Refresh to display the SSDs you just entered.

8. Restart the ESXi host to make the SSDs available.

9. In the Virtual Machines tab, right-click the VM, and then select Edit Settings.

The Virtual Machine Properties window is displayed.

10. In the Hardware tab, click Add.

The Add Hardware window is displayed.

11. In the Add Hardware window, select PCI Device from the list of device types, and then click Next.

12. Select the first cache device from the drop-down menu. Cache devices compatible with Fluid Cache contain either Micron RealSSD or Samsung Electronics in their name.

13. Click Next.

The Ready to Complete window is displayed.

14. In the Ready to Complete window, click Finish.

15. Repeat tasks 1215 for all remaining cache devices.

16. Repeat tasks 1215 for the network adapter virtual function (the adapters whose description contains the words Virtual Function).

NOTE: Do not add the same device twice.

17. When you have finished adding components, click OK in the Virtual Machine Properties window.

Complete the tasks in Adding Cache Devices to a Fluid Cache Cluster.

Adding a Storage Center to a Fluid Cache Cluster 1. Log in to Enterprise Manager as a user with Administrator or Volume Manager privileges.

2. Click the Storage view.

3. In the Storage pane, expand Fluid Cache Clusters if necessary, and then select the Fluid Cache cluster.

4. In the Cache tab, select Tasks and click Assign Storage Centers.

The Assign Storage Centers window is displayed.

5. In the Assign Storage Centers window, select one or more Storage Centers to be added.

6. Click OK.

30

A Maintaining Fluid Cache Installations This chapter describes tasks that must occasionally be performed after initial Fluid Cache deployment.

WARNING: When performing any operation to a server platform that hosts Fluid Cache software, you must perform either one of the following methods:

Shut down a single Fluid Cache Cluster node (VSA) at a time and perform the host-based server updates.

After the updates to host server are complete, reboot the host and bring the Fluid Cache Cluster node (VSA) online. Before shutting down and updating the next Fluid Cache cluster node, verify that the Fluid Cache Cluster node is added to Fluid Cache Cluster appropriately.

Place the entire Fluid Cache Cluster into maintenance mode, and then shut down the entire Fluid Cache Cluster by shutting down the Fluid Cache Cluster nodes (VSAs) one after the other.

After the updates to the host servers are complete, reboot the server and bring the Fluid Cache Cluster nodes (VSAs) online.

WARNING: To avoid potential performance issues and data loss, you must NOT shutdown multiple Fluid Cache Cluster (VSA) nodes at the same time to perform Fluid Cache host server based updates.

Removing Volume Mappings

To see a list of Fluid Cache mappings, select the appropriate Storage Center in the Storage view, and in the in the Storage tab, select the Fluid Cache cluster. The Fluid Cache mappings for the cluster are listed in the Volumes area at the bottom of the page.

Removing Volume Mappings from a Server

1. Make sure the volume is no longer in use and unmounted in vCenter.

2. In Enterprise Managers Storage view, expand Storage Centers if necessary and select the appropriate Storage Center. (Do not select Fluid Cache Clusters or its contents.)

3. In the Storage tab, expand Servers if necessary and locate the server whose Fluid Cache mappings you want to remove.

4. Right-click the server and select Remove Mappings.

The Remove Mappings window is displayed.

5. In the Remove Mappings window, select the volume and click OK.

There may be some delay while the mapping is removed and dirty data in the cache is flushed to main storage. When the mapping removal is complete, the volumes icon in the Storage tab turns from blue to gray color. You may need to manually refresh the display.

31

NOTE: The Fluid Cache mappings for the server are listed in the Volumes area at the bottom of the page.

Removing Volume Mappings from a Subcluster

1. Make sure the volume is no longer in use and unmounted in vCenter.

2. In Enterprise Managers Storage view, expand Storage Centers if necessary and select the appropriate Storage Center. (Do not select Fluid Cache Clusters or its contents.)

3. In the Storage tab, expand Servers, and then the Fluid Cache clusters if necessary and select the subcluster whose mappings you want to remove.

4. In the right pane, below the list of servers, select the volume and click Remove Mappings.

There may be some delay while the mapping is removed and dirty data in the cache is flushed to main storage. When the mapping removal is complete, the volumes icon in the Storage tab turns from blue to gray color. You may need to manually refresh the display.

NOTE: The Fluid Cache mappings for the subcluster are listed in the Volumes area at the bottom of the page.

NOTE: All Fluid Cache volume mappings must be removed before an ESXi host can be removed from a Fluid Cache cluster.

Removing a Server from a Fluid Cache Cluster 1. Make sure that no cached LUNs are in use by the ESXi server. Unmount VMFS datastores if present,

remove any RDMs or VMDKs, and so on.

2. If the server belongs to a server cluster (a subcluster) within a Fluid Cache cluster, remove the server from the subcluster:

a. Prior to removing the server from a Fluid Cache cluster, you must shutdown the host or stop the Fluid Cache service.

b. In Enterprise Managers Storage view, select the appropriate Storage Center. (Do not select Fluid Cache Clusters or its contents.)

c. In the Storage tab, expand Servers if necessary and locate the server.

d. Right-click the server and select Remove Server from Cluster. e. When asked to confirm the action, click OK.

In the Storage tab, the server now appears outside of the subcluster, but is still inside the Fluid Cache cluster.

NOTE: Removing the server from the subcluster removes all mappings created for the subcluster.

3. Power off the VSA:

a. Log in to the Windows vCenter client and in the Hosts and Clusters view, navigate to the VSAs ESXi host.

b. Right-click the VSA and select Power Power Off. There is a delay while the system writes cached data to disk. For a large quantity of data, this could take a considerable amount of time. Periodically refresh the display until the VSAs status shows that powering off is complete.

c. If you want to delete the VSA entirely, right-click the VSA again, select Delete from Disk, and when prompted to confirm the action, click OK.

32

NOTE: There are important differences between powering off a VSA and deleting it entirely. Power off the VSA if it may rejoin the same cluster but you want to free up RAM (the VSA is configured to use 20 GB of RAM). Delete the VSA if you do not intend to use it in the same cluster again and you want to free up disk space (the VSA uses approximately 10GB), or if you intend to move the server to a different Fluid Cache cluster (changing clusters requires reinstallation of the VSA).

4. Remove all Fluid Cache mappings from all volumes mapped to that server (see Removing Volume Mappings). Note that you do not have to remove non-Fluid Cache mappings.

5. Remove the server from the Fluid Cache cluster:

a. In the Storage view, select the Fluid Cache cluster. (Do not select Storage Center or its contents.)

b. In the Summary tab, locate the server in the Servers section.

c. Right-click the server and select Remove Server from Cluster. d. When asked to confirm the action, click OK.

After the system processes for some time, the server reappears outside of the cluster in the Servers area. You may need to manually refresh the display.

NOTE: After powering down the VSA on an ESXi host and then removing the ESXi host from a cluster, you must reinstall the VSA to add the server to any cluster, including the cluster from which it was removed.

Removing a Cache Device from a Fluid Cache Cluster If you are removing the cache device from the server, perform a graceful removal by first ensuring that the cache device is not in active use. See Removing a Server from a Fluid Cache Cluster for the steps on gracefully removing a server from a cluster.

1. In Enterprise Managers Storage view, expand Fluid Cache Clusters if necessary, and then select the Fluid Cache cluster. (Do not select Storage Centers or its contents.)

2. In the Summary tab, locate the cache device in the Devices section.

3. Right-click the device and select Remove Device from Cluster.

4. When asked to confirm the action, click OK.

As part of the deletion process, dirty data in the cache is flushed to main storage. This could take a considerable duration of time for a large quantity of data.

When processing is complete, the cache device no longer appears in the list of devices. You may need to manually refresh the data on the page.

Deleting or Removing a Fluid Cache Cluster There are important differences between deleting and removing a Fluid Cache cluster:

Deleting a Fluid Cache cluster deletes the cluster and all of its configuration information from Enterprise Manager, deletes the Storage Center object, and resets the configuration data on the Fluid Cache nodes so that they can be added to another cluster.

Removing a cluster removes it from Enterprise Manager but the nodes still contain configuration data for the cluster, which must be removed before the nodes can be added to another cluster.

NOTE: Deleting a cluster is the preferred action. Remove a cluster only if deleting it is not possible because the cluster is not functioning normally.

33

Deleting a Cluster

1. Stop all I/O on cached volumes.

2. Unmount cached volumes.

a. Remove the cached disks from the guest machine definition of any machine that uses a cached disk.

b. Unmount the volume in the ESXi client.

3. Remove all Fluid Cache mappings from all volumes in the cluster (see Removing Volume Mappings). Note that you do not have to remove non-Fluid Cache mappings.

4. In Enterprise Managers Storage view, right-click the Fluid Cache cluster and click Delete.

5. When prompted to confirm the deletion, click OK.

There may be a delay while the deletion is processed.

6. Shut down the Fluid Cache guest machines.

7. After deleting the cluster, remove the clusters static and dynamic iSER initiator targets:

NOTE: If you do not delete the old targets when removing or deleting clusters, the list of iSER targets can gradually become cluttered with obsolete addresses. This adversely affects performance and may eventually cause time out issues.

a. On the Configuration tab of the VMware client, choose Storage Adapters.

b. Right-click the first VMHBA below Mellanox iSCSI over RDMA (iSER) Adapters and click Properties.

c. In the iSCSI Initiator Properties window, select the Dynamic Discovery tab.

d. Choose the first IP address listed under iSCSI Server Location, then click Remove.

e. Repeat this process until the list is empty except for addresses used by non-Fluid Cache iSCSI servers.

f. In the iSCSI Initiator Properties window, select the Static Discovery tab.

g. Choose the first IP address listed under iSCSI Server Location, then click Remove.

h. Repeat this process until the list is empty except for addresses used by non-Fluid Cache iSCSI servers.

i. Close the Properties window, and rescan when prompted to do so.

Removing a Cluster

1. Stop all I/O on cached volumes.

2. Unmount cached volumes.

a. Remove the cached disks from the guest machine definition of any machine that uses a cached disk.

b. Unmount the volume in the ESXi client.

3. In Enterprise Managers Storage view, right-click the Fluid Cache cluster and click Remove.

4. In Enterprise Managers left pane, select the Storage Center for the cached volume.

5. In the Storage tab, expand Servers.

6. If the cluster still has the blue circle with the letters FC to indicate that it is a Fluid Cache cluster, wait a minute or two and refresh the view.

7. If the blue FC circle persists, power down the guest machines and refresh the view.

8. After the blue circle is replaced with a red X, right-click the cluster and click Delete.

9. Shut down the Fluid Cache guest machines.

34

10. Contact Dell Support for instructions on the removal of claim rules. This must be done before you can redeploy Fluid Cache on the same guest machines. An alternative to removing claim rules is to completely format and reinstall the hosts (not just an upgrade install) of ESXi.

11. After deleting the cluster, remove the clusters static and dynamic iSER initiator targets:

NOTE: If you do not delete the old targets when removing or deleting clusters, the list of iSER targets can gradually become cluttered with obsolete addresses. This adversely affects performance and may eventually cause time out issues.

a. On the Configuration tab of the VMware client, choose Storage Adapters.

b. Right-click the first VMHBA below Mellanox iSCSI over RDMA (iSER) Adapters and click Properties.

c. In the iSCSI Initiator Properties window, select the Dynamic Discovery tab.

d. Choose the first IP address listed under iSCSI Server Location, then click Remove.

e. Repeat this process until the list is empty except for addresses used by non-Fluid Cache iSCSI servers.

f. In the iSCSI Initiator Properties window, select the Static Discovery tab.

g. Choose the first IP address listed under iSCSI Server Location, then click Remove.

h. Repeat this process until the list is empty except for addresses used by non-Fluid Cache iSCSI servers.

i. Close the Properties window, and rescan when prompted to do so.

Recreating a Fluid Cache Cluster Although it is not part of standard operations or maintenance, in some instances you may want to recreate a Fluid Cache cluster.

1. Make sure that no cached LUNs on the servers are in use.

NOTE: All cached data on a server is lost when you perform the following procedure. Flush this data before proceeding.

2. Reinstall the Fluid Cache VSA. See Installing the Virtual Storage Appliance and the procedures that follow for mapping hardware and configuring a server.

3. Create and configure a new cluster. See Configuring and Managing Fluid Cache Clusters.

Upgrading Fluid Cache Cluster There is no direct method to upgrade Fluid Cache cluster to new releases. To deploy a new release of Fluid Cache, perform the following steps:

1. Remove the Fluid Cache cluster. See Deleting or Removing a Fluid Cache Cluster

2. Turn off and delete all the VSAs.

3. Create and configure a new cluster. See Configuring and Managing Fluid Cache Clusters

35

36

B Troubleshooting Fluid Cache Installations If you have unresolved issues running Fluid Cache after a successful completion of the installation procedure, contact your Compellent Copilot.

Troubleshooting the Compellent array and SAN architecture is beyond the scope of this document.

For additional troubleshooting information, refer to the Enterprise Manager Administrators Guide and the documentation for other hardware and software components. See Related Documentation.

After the initial list of Basic Troubleshooting Steps, the troubleshooting topics are presented in the order in which the issues are likely to appear as you deploy, configure, administer, and maintain a Fluid Cache installation.

Basic Troubleshooting Steps

Make sure the following conditions are fulfilled:

You have downloaded and installed the most recent firmware, drivers, and software required to support Fluid Cache for SAN (see Requirements for Fluid Cache).

The hardware is racked and cabled according to your hardware documentation.

The network security settings match those specified in Checking Network Connections.

The ports available to Fluid Cache match those specified in Connectivity.

All servers are in the same management interface subnet and in the same cache network subnet.

Each server in the Fluid Cache cluster appears in Enterprise Managers list of servers.

The Mellanox Virtual Function is not Available for Passthrough in ESXi 6.0

Possible Cause The ESXi 6.0 installation may include Mellanox drivers that do not support the virtual function.

Solution Disable or uninstall the following inbox drivers and make sure to install and configure the correct versions of the Mellanox drivers. For more information about the supported Mellanox drivers, see the Release Notes.

nmlx4_core

nmlx4-en

nmlx4-rdma

37

Server Does Not Appear in List of Servers

Possible Cause A configuration issue is preventing the server from appearing in the list.

Solution Make sure the procedure for configuring the server was completed successfully. See Configuring a Fluid Cache Server.

Possible Cause Firewall settings are preventing network communication.

Solution Check your firewall settings. See Checking Security Settings.

Possible Cause The network switch is not correctly cabled or configured.

Solution Review the settings for the network switch and consult your switch documentation. See Cluster or Application Has Performance Issues.

Cache Device Does Not Appear in List of Cache Devices

Possible Cause The cache device you are trying to add is not supported by Fluid Cache.

Solution Refer to the list of supported devices in Requirements for Fluid Cache.

Possible Cause The cache device is not functioning properly.

Solution To check device function, select the device in Enterprise Manager and in the Event tab, look for a device failure message. Replace the cache device if necessary, using instructions in the Dell Compellent Enterprise Manager Users Guide.

Cache Device Cannot Be Added to a Cluster

Possible Cause The cache device is not functioning properly. Under some conditions, the process of adding a device completes normally even though the device being added is not functioning properly.

Solution To check device function, select the device in Enterprise Manager and in the Event tab, look for a device failure message.

If necessary, replace the cache device by following the instructions provided in the Dell Compellent Enterprise Manager Users Guide.

Cannot Select a Specific Cache Mode

Possible Cause There is an existing cache mode configured for the volume mapping. The cache mode chosen for a volume mapping cannot be changed. The existing mapping must be deleted, and a new mapping created. A new cache mode can be selected while creating the new mapping.

Solution Remove the volume mapping and create a new mapping. When you create the new mapping, you can then select a different cache mode (cache policy). See Removing Volume Mappings and Mapping Volumes.

38

Possible Cause The cluster is in maintenance mode.

Solution Take the Fluid Cache cluster out of maintenance mode by selecting the cluster in Enterprise Manager, clicking Edit Settings in the Summary tab, and clearing the Maintenance Mode option. Note that certain system failures or an invalid license may prevent the cluster from being taken out of maintenance mode. Refer to the Enterprise Manager Status and Events tabs, and see Fluid Cache License Is Expired.

Cached LUNs Are Unavailable

Possible Cause After restarting Fluid Cache or restarting a server, cached LUNs may not be immediately available.

Solution Wait for the cached LUNs to reappear. It may take a significant amount of time to write a large amount of data. The cached volumes reappear after the cache completes recovery.

Possible Cause A fault in the cluster is preventing the volume from being recognized by Enterprise Manager.

Solution Check the Enterprise Manager status page and event log for error messages.

Enterprise Manager Freezes While Adding or Removing a Volume Mapping

Possible Cause You are using the VMFS filesystem and did not unmount the filesystem before removing volume mappings.

Solution Unmount the filesystem before removing the volume mappings (refer to the document VMware Knowledge Base article 2004605.

Possible Cause Under some conditions, when removing multiple volume mappings, the first volume may take an unusually long time (more than 20 minutes) to unmap.

Solution Wait for the volumes to unmap. Unmapping speed should increase as more volumes are unmapped.

Unable to Map Volumes to Fluid Cache using Enterprise Manager

Possible Cause ESXi hosts were added to vCenter by using their IP Address and not their hostname when DNS is being used on your network.

Solution Turn off the Fluid Cache VSA virtual machines on the ESXi hosts. Remove the ESXi hosts from vCenter. Add the ESXi hosts back to vCenter by using their registered DNS name.

Possible Cause DNS assigned address for ESXi host and the VSA virtual machine that is used for Fluid Cache has changed from its initial setting during Fluid Cache cluster creation.

39

Solution Configure your DNS server to use static addresses for your ESXi hosts and Fluid Cache VSA VMs.

Cannot Configure the Cache Network

Possible Cause Firewall settings are preventing access to one or more ports required by Fluid Cache.

Solution Change your firewall settings to allow access by Fluid Cache. For a list of required ports, see Checking Network Connections.

Possible Cause One of the ports required by Fluid Cache is in use by another process.

Solution Refer to the required ports listed in Checking Network Connections and reassign ports as needed.

NOTE: Port 3033 is used to communicate with Enterprise Manager and Storage Center. To configure those applications to use a different port, refer to the corresponding Administrators Guide. After a new port is configured in Enterprise Manager or Storage Center, Fluid Cache automatically uses the new port.

NOTE: Port 7449 and 5480 are required by Fluid Cache. If other applications are using these ports, configure those applications to use different ports.

Possible Cause If an incorrect IP address is entered during configuration of the VSA (see Configuring the VSA), an error message indicates that Fluid Cache cannot log in due to an incorrect vCenter address.

Solution Select Change vCenter and re-enter the IP address, or check network connections and settings.

Possible Cause If one of the iSER interfaces required by the VSA cannot be created, the VSA configuration fails and an error message indicates that the network is not configured properly. Note that an existing iSER interface configured in vCenter can prevent creation of the iSER interface required by the VSA.

Solution Check your network connections and verify functionality of the iSER interfaces

using vCenter (Configuration Networking). Do not use vCenter to delete or modify the iSER interfaces used by Fluid Cache, or to create new iSER interfaces. Remove any iSER interfaces that were created prior to configuration of the Fluid Cache VSA.

Possible Cause The correct version of the Mellanox driver is not installed, which results in an error message when attempting to power on the VSA. The error message may specify Cannot set MAC address or default VLAN for PCI Virtual Function or error 0x195887107.

Solution Make sure you have the correct Mellanox driver version installed (see Checking the Network Adapter Firmware and Driver).

Possible Cause One or more paths to the Compellent are not operational. During configuration of the VSA, all paths to the Compellent must be operational or the configuration is not successful.

40

Solution Make sure that all paths to the Compellent are operational.

Possible Cause The TUI configuration tool could not obtain the iSCSI IQN from the ESXi host. When attempting to configure the iSCSI network, the following error message is displayed: Cannot configure iSCSI. ESXi host <host_name> is missing configured iSCSI software adapter.

Solution Manually configure the ESXi host by logging in to the ESXi host that the VSA is deployed on and configure the iSCSI software initiator.

Cannot Create a Fluid Cache Cluster

Possible Cause A minimum of three PowerEdge servers have not been configured for use with Fluid Cache. Until at least three servers have Fluid Cache installed and configured, the option to create a Fluid Cache cluster is unavailable.

Solution Add three or more nodes and configure the nodes before attempting to create a Fluid Cache cluster in Enterprise Manager.

Possible Cause Network connectivity issues are preventing the creation of a Fluid Cache cluster.

Solution Check that the security settings are correct and the required ports and network connections are open. See Checking Network Connections and Checking Security Settings.

Cannot Map Volumes to a Fluid Cache Cluster

Possible Cause The iSCSI software adapter on the ESXi host is not logged in to the Storage Center that contains the LUN you are mapping.

Solution Make sure the iSCSI adapter on the ESXi host is logged in to the appropriate Storage Center.

Possible Cause The iSCSI VMkernel port on the ESXi host is not bound properly to the iSCSI initiator.

Solution Check the binding in vSphere by opening the iSCSI Initiator Properties window. If no VMkernels are shown, click Add, select the VMkernel that corresponds to your iSCSI (uncached) network, and click OK. Click Close on the Properties window and when prompted to rescan the adapter, click Yes.

Cannot Connect to Cluster Nodes

Possible Cause Time synch issues are preventing Enterprise Manager from connecting to the cluster nodes. The cluster appears to be configured correctly, but in the Overview page, no Fluid Cache connections are active.

Solution Make sure that you have enabled Network Time Protocol and that in the vTools control panel, Synchronize guest time with host is not selected. (For more information, refer to the VMware document Timekeeping in VMware Virtual Machines).

41

Fluid Cache License Is Expired

Possible Cause System settings such as changes to the system date cause the current Fluid Cache license to expire. You can still access data on cached volumes, but performance is degraded because the Fluid Cache cluster has been placed in maintenance mode and caching is no longer active.

Solution Check the status of the license file by selecting the Fluid Cache cluster in Enterprise Manager and referring to the status shown on the Events or Cache tabs. If the license is expired, make sure that your system settings for date and time are correct. If it is not correct, set the appropriate date and time. Take the Fluid Cache cluster out of maintenance mode by selecting the cluster in Enterprise Manager, clicking Edit Settings in the Summary tab, and then clearing the Maintenance Mode option.

Possible Cause Fluid Cache for SAN is running on an evaluation license (typically 90 days) and that time period has been exceeded. You can still access data on cached volumes, but performance is degraded because the Fluid Cache cluster has been placed in maintenance mode and caching is no longer active.

Solution Check the status of the license file by selecting the Fluid Cache cluster in Enterprise Manager and referring to the license type shown on the Events or Cache tabs. If the number of days remaining is zero, contact your Dell representative to purchase a Fluid Cache for SAN license. After activating the new license, take the Fluid Cache cluster out of maintenance mode by selecting the cluster in Enterprise Manager, clicking Edit Settings in the Summary tab, and deselecting Maintenance Mode.

Fluid Cache License Is Invalid

Possible Cause The license file is invalid if it is in any way modified. This causes unsuccessful digital signature validation.

Solution Contact Dell Customer Support.

Cannot Assign or Remove a Storage Center

Possible Cause The Storage Center is already assigned to another Fluid Cache cluster.

Solution In Enterprise Manager, see whether or not a Storage Center is listed for the Fluid Cache cluster.

Possible Cause Network connectivity issues are preventing Enterprise Manager from communicating with Storage Center.

Solution Make sure the network is functioning properly. Refer to Checking Network Connections and Checking Security Settings..

42

Cannot Determine Which Cache Device Failed

Possible Cause One of the cache devices has failed on a server with multiple cache devices installed, and it is not clear which of them failed.

Solution In Enterprise Manager, in the Cache tab, the failed cache device is identified by a red X through it. Note the last digits of the number for this cache device. This number matches the serial number printed on the label of the failed cache device.

Events for Fluid Cache Are Not Shown in Enterprise Manager

Possible Cause When Enterprise Manager was installed, the Data Collector was not configured for the correct IP address.

Solution Make sure the IP address that the Data Collector uses is accessible by the Fluid Cache nodes. To view the IP address, start the Data Collector Manager, select General Information, and note the IP address within the URL in the Web Site field. Change the address if necessary and restart the Data Collector.

Unable to PCI Passthrough a Samsung NVMe Flash Drive to Fluid Cache VSA through VMware vCenter Web Client

Solution Use the thick client of VMware to mark the Samsung NVMe flash drive for PCI Passthrough.

Fluid Cache VSA node not being added to Fluid Cache Cluster Server Object in Enterprise Manager

Possible Cause Duplicate hardware iSCSI initiator WWN configured on an ESXi host.

Solution Verify that all hardware iSCSI initiator WWNs configured on your ESXi hosts are unique across all ESXi nodes in your Fluid Cache Cluster.

Cluster or Application Has Performance Issues

Possible Cause One or more cache devices are uninstalled, have failed, or do not have the correct firmware or drivers.

Solution Use Enterprise Manager to check the functionality of the cache devices. See also Checking the Cache Device Firmware and Driver.

Possible Cause The Compellent storage array is overloaded.

Solution In Enterprise Manager, check the storage latencies and throughput on the cached volumes. Add more capacity to the Compellent array if necessary.

43

Possible Cause The application is not making use of the cache.

Solution Make sure the application has an I/O profile that can leverage Fluid Cache, which accelerates reads, writes, and read-after writes. If the application does, check the running cache mode for that volume in Enterprise Manager. If necessary, change cache modes by deleting the volume mapping and creating a new one with a different cache mode. See Mapping Volumes.

Possible Cause The active data set greatly exceeds the size of the cache pool.

Solution Add more cache devices to increase the size of the cache pool.

Possible Cause Network traffic from outside Fluid Cache is interfering with performance.

Solution The cache network switch is not configured correctly. Configure the ports on the switch used by Fluid Cache so that they are used solely by Fluid Cache.

Possible Cause The cache network switch is not configured correctly.

Solution Make sure that the switch has the following settings:

The ports are in Layer 2 mode.

The ports are in an untagged state.

The switch firmware is up to date.

Multicast is enabled.

Verify that flow control is enabled and DCB is disabled. For Dell Networking switches, see Configuring the Cache Network Switch. For all other switches, consult the manufacturers documentation.

NOTE: Enabling flow control is a requirement for Fluid Cache.

For blade enclosure, disable FlexAddress.

You can check network functionality by checking rx_over_errors using the

ethtool -S command.

Possible Cause Fluid Cache is running on an evaluation license (typically 90 days) and that time period has been exceeded. You can still access data on cached volumes, but performance is degraded because the Fluid Cache cluster has been placed in maintenance mode and caching is no longer active.

Solution Check the status of the license file by selecting the Fluid Cache cluster in Enterprise Manager and referring to the status shown on the Events or Cache tabs. Contact your Dell representative to purchase a Fluid Cache license. After activating the new license, take the Fluid Cache cluster out of maintenance mode by selecting the cluster in Enterprise Manager, clicking Edit Settings in the Summary tab, and then clearing the Maintenance Mode option.

44

Unable to add a Fluid Cache ESXi 6.0 host to an existing server cluster

Description Using Enterprise Manager GUI, if you try to add an ESXi 6.0 Fluid Cache host to an existing server cluster within a Storage Center, the following error message is displayed:

SC servers must all have the same operating system.

Solution When adding a new Fluid Cache ESXi6.0 server host to an existing sub server cluster, in the Edit Server Settings dialog box, select VMware ESXi 5.5 for the Operating System type instead of VMware ESXi 6.0. After adding the server to the sub server cluster, when Enterprise Manager prompts you to change the version, change the operating system type to VMware ESXi 6.0.

This method enables you to add an ESXi 6.0 host to an existing sub server cluster on the SC8000 without any issue.

Unable to install Dell Fluid Cache VSA properly on the following Fluid Cache Cluster platforms

Description When configuring Fluid Cache VSA iSCSI interface on the Fuid Cache Cluster ESXi5.5U3 and ESXi6.0U1 platforms during VSA installation, the following error message is displayed.

"Cannot configure ISCSI initiator already configured for a different ESX host".

Solution 1(Preferred)

Perform the following steps, and then configure the iSCSI section in the VSA setup:

1. In the Web Client, open the VSA console, and enable SSH.

2. Log in to VSA through SSH.

3. Run the following command to navigate to the iscsi directory:

#cd /etc/iscsi 4. Open the initiatorname.iscsi file and verify if the value for the InitiatorName is

displayed in asterix (*). For example:

root@localhost iscsi]# cat initiatorname.iscsi InitiatorName=***************************************

5. Run the following command to edit theinitiatorname.iscsi file.

#vi initiatorname.iscsi 6. Change the characters with asterisk (*) with the proper ESXi Hosts vmhba##

software IQN that the VSA is associated. For example:

root@localhost iscsi]# cat initiatorname.iscsi InitiatorName=iqn.1998-01.com.vmware:FLDC-QAESXi55U1- Srv1-48a43810

45

7. Run the following command to restart iscsid service.

#service iscsid restart

Solution 2 Perform the steps provided in the section Configuring the VSA of the deployment guide in the following sequence:

1. Perform the steps 1 through 5 as described in the deployment guide.

2. Select Configure This Appliance and press .

3. Configure your Backend Storage connection.

4. Perform the steps 6 and 7 and continue with step 10 as provided in the deployment guide.

46

C Example Cabling Diagram

Figure 2. Example Cabling Diagram

1. Enterprise Manager console 2. Management interface switch

3. Servers (4) 4. Cache network switches (2)

5. SAN switch 6. Storage Center controllers (2)

7. Storage Center expansion enclosures (2)

The example above has the following features:

47

Fluid Cache is installed on four servers, which conforms to the minimum of three servers per each cache cluster and maximum of nine servers required.

Each of the servers has a network adapter connected to a port on each of the two cache network switches. The switches are uplinked to each other.

The servers access the Storage Center using SAN connectivity.

Enterprise Manager is used to configure and monitor Fluid Cache.

NOTE: For best practices on racking and cabling each hardware component, refer to the rack placement for the component at dell.com/support/manuals.

48

D Configuring a Dell Networking Switch The following procedure is for one of the supported Dell Networking switches listed in the table in Requirements For Fluid Cache. For all other switches, see the manufacturers documentation. To configure a switch containing physical ports 0, 1, 2, and 3 for a cache network with four nodes connected to the switchs 10-gigabit Ethernet ports:

1. Telnet into the switch and enter the login name and password to enter Exec mode.

While you are in Exec mode, the > prompt is displayed following the host name prompt, which is FTOS by default.

telnet 172.31.1.53 Trying 172.31.1.53... Connected to 172.31.1.53. Escape character is '^]'. Login: username Password: FTOS>

2. Enter Exec Privilege mode by running the command: enable 3. Enter Configuration mode by running the command: configure 4. Select either a single physical port or range of physical ports:

To configure a single port (in this example, port 0), run the command: interface TenGigabitEthernet 0/0

To configure a range of ports (in this example, ports 0-3), run the command: interface range TenGigabitEthernet 0/0 - 3

5. Define static IP addressing by running the command: no ip address 6. Enter Layer 2 mode by running the command: switchport 7. Enable the port by running the command: no shutdown 8. Enable flow control and disable DCB by running the command: flowcontrol rx on tx on

NOTE: Enabling flow control is a requirement for Fluid Cache.

9. Close the configuration page by running the command: exit 10. Repeat tasks 49 for the remaining physical ports in the cache network.

11. Exit Exec Privilege mode by running the command: exit 12. Save all the changes by running the command: write

NOTE: To create a VLAN (for example, to use available ports on an existing network switch to create the cache network), make sure that the cache network ports are in Layer 2 mode and in the untagged state.

49

50

E Checking the System Configuration If any of the system checks below show that the firmware or drivers are earlier than the required versions, update the components. See Requirements for Fluid Cache and Downloading Firmware, Drivers, and Software.

Checking the Cache Device Firmware and Driver

Non-NVMe Cache Devices

Firmware The firmware version for the Dell PCIe Express Flash SSDs must be B1490908 or later. Otherwise, even though they are visible in ESXi and in Enterprise Manager, the cache devices do not appear in the list of available devices.

NOTE: The installed firmware version cannot be verified in ESXi. If the cache devices do not appear in the list of available devices and you are not sure that your firmware version is B1490908 or later, update your firmware. Also, see Troubleshooting VMware Installations. You may also contact Dell Technical Support.

Driver The cache device driver used by Fluid Cache is native to the VSA and does not require updating.

Checking the Network Adapter Firmware and Driver To make sure that the correct firmware and drivers are installed on each of the Mellanox network adapters used by Fluid Cache:

1. Get a list of interfaces on your server by running the command: esxcfg-nics -l 2. In the results, locate and write down the names of the network interfaces.These are in the format

vmnicX (for example, vmnic1).

The following example illustrates potential Fluid Cache interfaces on Mellanox network adapters:

Name PCI Driver Link Speed Duplex MAC Address MTU Description vmnic0 0000:01:00.00 bnx2x Up 1000Mbps Full e0:db: 55:11:92:24 1500 Broadcom Corporation NetXtreme II BCM57810 10 Gigabit Ethernet vmnic1 0000:01:00.01 bnx2x Up 10000Mbps Full e0:db: 55:11:92:26 1500 Broadcom Corporation NetXtreme II BCM57810 10 Gigabit Ethernet vmnic1000202 0000:07:00.00 mlx4_en Up 10000Mbps Full 00:02:c9:e5:f7:31 1500 Mellanox Technologies MT27500 Family [ConnectX-3] vmnic2 0000:07:00.00 mlx4_en Up 10000Mbps Full 00:02:c9:e5:f7:30 1500 Mellanox Technologies MT27500 Family [ConnectX-3]D

3. Make sure the firmware and driver versions on each of these interfaces by running the command:

ethtool -i vmnicX The firmware and driver must use supported versions as documented in the Release Notes.

51

4. Configure all interfaces for the management interface, the cache network, and potentially the iSCSI network.

5. Ping the management interface and cache network addresses from all nodes. To install Fluid Cache, you must be able to ping the management interface and cache network of every node in the cluster from each node in the cluster.

To download updated firmware or drivers, see Downloading Firmware, Drivers, and Software.

52

F Related Documentation For PowerEdge server documentation, go to dell.com/support/manuals and enter your Service Tag.

For cache device documentation, go to dell.com/storagecontrollermanuals and click Dell Power Edge Express Flash PCIeSSD.

The following table lists documents you may want to refer to while installing Fluid Cache.

Component Document Content

Server PowerEdge Owners Manual Describes how to install, remove, configure, and troubleshoot server components.

Rack Placement Describes how to rack the server.

Updating BIOS on Dell 12G PowerEdge Servers (iDRAC7 - 12G only)

Describes how to upgrade the BIOS on Dell 12G PowerEdge Servers using different Dell utilities.

Updating BIOS on Dell 13G PowerEdge Servers (iDRAC8 - 13G only) (New for 13G)

Describes how to upgrade the BIOS on Dell 13G PowerEdge Servers using different Dell utilities.

Lifecycle Controller Platform Update in Dell PowerEdge 12th Generation Servers

Describes how to upgrade the Lifecycle Controller using different Dell utilities.

Updating DRAC Firmware Describes how to upgrade iDRAC using different Dell utilities.

Cache Device PowerEdge Express Flash PCIeSSD Users Guide

Describes how to install, remove, configure, and troubleshoot PCIeSSDs.

Deploying the Dell PowerEdge Express Flash PCIeSSD

Describes the procedures for setting up, installing, and removing a PCIeSSD.

Network Switch Dell Networking Owners Manual/ Administrators Guide/ Rack Placement

Describes how to set up the switch.

Network Adapter ConnectX-3 VPI Adapter Card User Manual

Describes how to install, remove, and configure, the Ethernet adapter.

Dell Compellent Enterprise Manager

See Accessing Dell Compellent Documentation.

Release Notes Describes new features, known issues, and upgrade steps for Enterprise Manager.

Administrators Guide Describes how to monitor and run Dell Compellent Enterprise Manager.

53

Accessing Enterprise Manager and Storage Center Documentation Documentation for Dell Compellent produc

Manualsnet FAQs

If you want to find out how the Fluid Cache Dell works, you can view and download the Dell Fluid Cache SAN 2.0.10V2.0.0 Storage Deployment Guide For VMware ESXi Systems on the Manualsnet website.

Yes, we have the Deployment Guide For VMware ESXi Systems for Dell Fluid Cache as well as other Dell manuals. All you need to do is to use our search bar and find the user manual that you are looking for.

The Deployment Guide For VMware ESXi Systems should include all the details that are needed to use a Dell Fluid Cache. Full manuals and user guide PDFs can be downloaded from Manualsnet.com.

The best way to navigate the Dell Fluid Cache SAN 2.0.10V2.0.0 Storage Deployment Guide For VMware ESXi Systems is by checking the Table of Contents at the top of the page where available. This allows you to navigate a manual by jumping to the section you are looking for.

This Dell Fluid Cache SAN 2.0.10V2.0.0 Storage Deployment Guide For VMware ESXi Systems consists of sections like Table of Contents, to name a few. For easier navigation, use the Table of Contents in the upper left corner.

You can download Dell Fluid Cache SAN 2.0.10V2.0.0 Storage Deployment Guide For VMware ESXi Systems free of charge simply by clicking the “download” button in the upper right corner of any manuals page. This feature allows you to download any manual in a couple of seconds and is generally in PDF format. You can also save a manual for later by adding it to your saved documents in the user profile.

To be able to print Dell Fluid Cache SAN 2.0.10V2.0.0 Storage Deployment Guide For VMware ESXi Systems, simply download the document to your computer. Once downloaded, open the PDF file and print the Dell Fluid Cache SAN 2.0.10V2.0.0 Storage Deployment Guide For VMware ESXi Systems as you would any other document. This can usually be achieved by clicking on “File” and then “Print” from the menu bar.