Mellanox Technologies ConnectX-3 VPI Manuale Utente

www.mellanox.comConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User ManualP/N:MCX353A-FCBT, MCX353A-TCBT, MCX353A-QCBT, MCX354A-FCBT, MCX354A

OverviewRev 1.4Mellanox Technologies101 OverviewThis document is a User Manual for Mellanox Technologies VPI adapter based on the Con-nectX®-3 VPI int

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies111.1.3 InfiniBand QDRQDR is a standard InfiniBand dat

OverviewRev 1.4Mellanox Technologies121.2 Adapter Cards Covered in this ManualTable 3 lists the single and dual-port QDR adapter cards.Table 4 lists t

Table 5 - Single and Dual-port FDR Adapter CardsOrdering Part Number (OPN)PCI Express SERDES SpeedData Transmis-sion Rate/# of portsRoHS Adapter IC P

OverviewRev 1.4Mellanox Technologies141. Installation InstructionsRead all installation instructions before connecting the equipment to the power sour

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies158. Hazardous Radiation Exposure Caution – Use of

Adapter Card InterfacesRev 1.4Mellanox Technologies162 Adapter Card Interfaces2.1 I/O InterfacesEach adapter card includes the following interfaces:•

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies17Figure 3: Dual-port BracketFigure 4: Single-port Bra

Adapter Card InterfacesRev 1.4Mellanox Technologies182.1.1 InfiniBand InterfaceThe network ports of the ConnectX®-3 adapter cards are compliant with t

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies192.1.5 LED AssignmentThere are two I/O LEDs per port

Mellanox Technologies350 Oakmead Parkway Suite 100 Sunnyvale, CA 94085 U.S.A.www.mellanox.comTel: (408) 970-3400Fax: (408) 970-3403Mellanox Technologi

The short bracket has the same port and LED footprints as the tall bracket.Adapter Card InterfacesRev 1.4Mellanox Technologies202.1.6 I2C-compatible I

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies21When a jumper is inserted into the drill holes in th

Adapter Card InterfacesRev 1.4Mellanox Technologies2254 VPD Keyword EC STR Engineering Change Level of the card (rev)56 Length 0x257 Revision RV %STR

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies231539 Data STR_ZERO Reserved (0x00)1792 VPD Keyword R

Adapter Card InstallationRev 1.4Mellanox Technologies243 Adapter Card Installation3.1 Hardware RequirementsA system with a PCI Express x8 slot is requ

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies25Figure 7: PCI Device 5. In the Value display box, ch

Adapter Card InstallationRev 1.4Mellanox Technologies263.4 Cables and ModulesPlease refer to “Mellanox Products Approved Cable Lists” at http://www.me

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies273. Insert the connector into the adapter card. Be ca

Driver Software and FirmwareRev 1.4Mellanox Technologies284 Driver Software and Firmware4.1 Driver Software4.1.1 LinuxFor Linux, download and install

Advertised auto-negotiation: N Speed: 1000Mb/s Duplex: Full Port: Twisted Pair PHYAD: 0 Transceiver: intern

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies3Table of ContentsTable of Contents . . . . . . . . .

Physical state: LinkUp Rate: 56 Base lid: 8 LMC: 0 SM lid: 12

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies31The “Usage: openibd {start|stop|restart|status}” com

Driver Software and FirmwareRev 1.4Mellanox Technologies32Drivers =>InfiniBand & VPI Software/Drivers=> Mellanox WinOF VPI => Download. F

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies33Port configuration is saved in the file: /etc/infini

Driver Software and FirmwareRev 1.4Mellanox Technologies34Step 2. Right-click on the Mellanox ConnectX VPI network adapter and left-click Properties.

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies35project (see www.ipxe.org). For more information go

Driver Software and FirmwareRev 1.4Mellanox Technologies36The number of shared pages by GPUDirect is reported under:4.4.2.3 NVIDIA Driver Installation

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies37ConnectX®-3 EN with RoCE based network management is

Driver Software and FirmwareRev 1.4Mellanox Technologies382. Compare the programmed firmware version with the latest available. a. Go to Mellanox’s w

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies39 d. Reboot the computer. e. Enter: mst start. f. Ver

Rev 1.4Mellanox Technologies44.2.1 Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Rev 1.4Mellanox Technologies40Appendix A: SpecificationsA.1 MCX353A-QCBT SpecificationsTable 11 - MCX353A-QCBT Specifications TablePhysicalSize: 2.12i

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies41A.2 MCX354A-QCBT SpecificationsTable 12 - MCX354A-QC

Rev 1.4Mellanox Technologies42A.3 MCX353A-TCBT SpecificationsTable 13 - MCX353A-TCBT Specifications TablePhysicalSize: 2.12in. x5.6 in. (53.59mm x 142

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies43A.4 MCX354A-TCBT SpecificationsTable 14 - MCX354A-TC

Rev 1.4Mellanox Technologies44A.5 MCX353A-FCBT SpecificationsTable 15 - MCX353A-FCBT Specifications TablePhysicalSize: 2.12in. x5.6 in. (53.59mm x 142

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies45A.6 MCX354A-FCBT SpecificationsTable 16 - MCX354A-FC

Rev 1.4Mellanox Technologies46A.7 Board Mechanical Drawing and DimensionsAll dimensions are in millimeters.All the mechanical tolerances are +/- 0.1mm

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies47Figure 12: Mechanical Drawing of the Dual-port MCX35

Rev 1.4Mellanox Technologies48Appendix B: Interface Connectors PinoutB.1 I2C-compatible Connector PinoutFigure 13: Compatible Connector Plug and Pinou

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies49Figure 14: PCIe Connector Pinout

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies5List of TablesTable 1: Revision History Table . . . .

Rev 1.4Mellanox Technologies50B.3 QSFP+ Connector PinoutFigure 15: Connector and Cage Views

Table 18 - Connector Pin Number and Name to Signal Name MapConnector Pin Number Connector Pin Name Port A Signal Name1 GND GND2 TXN_2 Tx2n3 TXP_2 Tx2p

Rev 1.4Mellanox Technologies52Appendix C: Replacing a Tall Bracket With a Short Bracket This section provides instructions on how to remove the tall b

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies53C.3 Installing the New Bracket 1. Remove the paper t

Rev 1.4Mellanox Technologies54Figure 19: Placing the Bracket on the Card3. Make sure that the LEDs are aligned onto the bracket holes. 4. Use a torque

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies55Appendix D: Avertissements de sécurité d’installatio

Rev 1.4Mellanox Technologies567. Codes électriques locaux et nationaux Ce matériel doit être installé dans le respect des codes électriques locaux et

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies57Appendix E: Sicherheitshinweise (Warnings in German)

This equipment should be installed in compliance with local and national elec-trical codes.Rev 1.4Mellanox Technologies588. StrahlenkontakAchtung – Nu

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies59Appendix F: Advertencias de seguridad para la instal

Rev 1.4Mellanox Technologies6List of FiguresFigure 1: Card Product Label (Example) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Rev 1.4Mellanox Technologies607. Códigos eléctricos locales y nacionalesEste equipo se debe instalar conforme a los códigos eléctricos locales y nacio

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies7Revision HistoryThis document was printed on August 2

Rev 1.4Mellanox Technologies8About this ManualThis User Manual describes Mellanox Technologies ConnectX®-3 VPI Single and Dual QSFP+port PCI Express x

ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 1.4Mellanox Technologies9Customers who purchased Mellanox M-1 Global Support S