The Design and Implementation of the FreeBSD Operating System, Second Edition
Now available: The Design and Implementation of the FreeBSD Operating System (Second Edition)


[ source navigation ] [ identifier search ] [ freetext search ] [ file search ] [ list types ] [ track identifier ]

FreeBSD/Linux Kernel Cross Reference
sys/dev/vxge/

Version: -  FREEBSD  -  FREEBSD-13-STABLE  -  FREEBSD-13-0  -  FREEBSD-12-STABLE  -  FREEBSD-12-0  -  FREEBSD-11-STABLE  -  FREEBSD-11-0  -  FREEBSD-10-STABLE  -  FREEBSD-10-0  -  FREEBSD-9-STABLE  -  FREEBSD-9-0  -  FREEBSD-8-STABLE  -  FREEBSD-8-0  -  FREEBSD-7-STABLE  -  FREEBSD-7-0  -  FREEBSD-6-STABLE  -  FREEBSD-6-0  -  FREEBSD-5-STABLE  -  FREEBSD-5-0  -  FREEBSD-4-STABLE  -  FREEBSD-3-STABLE  -  FREEBSD22  -  l41  -  OPENBSD  -  linux-2.6  -  MK84  -  PLAN9  -  xnu-8792 
SearchContext: -  none  -  3  -  10 

Name Size Last modified (GMT) Description
Back Parent directory 2023-01-29 19:57:16
Folder include/ 2023-01-29 19:57:16
Folder vxgehal/ 2023-01-29 19:57:16
File LICENSE 1632 bytes 2023-01-29 19:57:16
File README 12704 bytes 2023-01-29 19:57:16
C file vxge-firmware.h 2185881 bytes 2023-01-29 19:57:16
C file vxge-osdep.h 18969 bytes 2023-01-29 19:57:16
C file vxge.c 104526 bytes 2023-01-29 19:57:16
C file vxge.h 15108 bytes 2023-01-29 19:57:16
C file vxgell-version.h 2830 bytes 2023-01-29 19:57:16

    1 $FreeBSD$
    2 ''_Readme for FreeBSD X3100 Series 10GbE PCIe I/O Virtualized Server Adapter Drivers_'''
    3 
    4 === Introduction ===
    5 FreeBSD Driver for X3100 10GbE Server/Storage adapters 
    6 * Drivers support all X3100 10GbE adapters with FreeBSD version 7.x, 8.x and 9.x
    7 * Supports both i386 and amd64 architectures
    8 * Features:     Jumbo frames (up to 9600),
    9                 LRO (Large Receive Offload),
   10                 TSO (TCP segmentation offload),
   11                 RTH (Receive Traffic Hash).
   12    Also, Extended Message Signaled Interrupts (MSI-X).
   13 
   14 
   15 ''Features''
   16 
   17     a.  Jumbo frames:
   18         X3110 and X3120 supports MTU up to 9600 bytes, modifiable using ifconfig command.
   19 
   20     b.  LRO (Large Receive Offload):
   21         LRO can be enabled/disabled before loading driver.
   22         Set lro_enable in vxge.conf to 1 before loading driver.
   23 
   24     c.  TSO (TCP Segmentation Offload)
   25         TSO can be enabled/disabled before loading driver.
   26         Set tso_enable in vxge.conf to 1 before loading driver.
   27 
   28     d.  RTH (Receive Traffic Hash)
   29         Receive side steering for better scaling.
   30         Set rth_enable in vxge.conf to 1 before loading driver.
   31 
   32     e.  MSI-X
   33         Can be enabled on platforms which support it, resulting in noticeable
   34         performance improvement. 
   35 
   36     f.  Multi-VPaths
   37         Up to 17 hardware based transmit and receive data channels, with
   38         multiple steering options.
   39 
   40 
   41 ''X3100 & Driver configuration: vxge.conf''
   42 
   43 The vxge.conf contains following attributes.
   44 
   45 
   46 ''msix_enable''
   47 
   48 Enable MSI (Message Signaled Interrupts) feature in driver.
   49 0 - INTA
   50 1 - MSI-X
   51 Default: 1
   52 
   53 
   54 ''rth_enable''
   55 
   56 Enables Receive side steering for better scaling (RTH - Receive Traffic Hash)
   57 Range: 0 - 1
   58 Default: 1
   59 
   60 
   61 ''lro_enable''
   62 
   63 Enables LRO (Large Receive Offload) feature in driver.
   64 Range: 0 - 1
   65 Default: 1
   66 
   67 
   68 ''tso_enable''
   69 
   70 Enables TSO (TCP Segmentaton Offload) feature in driver.
   71 Range: 0 - 1
   72 Default: 1
   73 
   74 
   75 ''no_of_vpath''
   76 
   77 Specifies maximum VPATH(s) configured for each device function.
   78 Valid range: 1-17
   79 Default: Optimized by driver
   80 
   81 
   82 ''func_mode''
   83 
   84 Change PCI function mode
   85     0  - SF1_VP17 (1 function with 17 VPATHs)
   86     1  - MF8_VP2  (8 functions with 2 VPATHs per function)
   87     8  - MF2_VP8  (2 functions, 8 Paths/Function)
   88     9  - MF4_VP4  (4 Functions, 4 Paths/Function)
   89     11 - MF8P_VP2 (8 functions with 2 VPATHS per function required for DirectIO)
   90     Default: -1
   91 
   92 
   93 ''port_mode''
   94 
   95 Change the default dual port mode
   96     2 - Active Passive
   97     3 - Single Port
   98     4 - Dual Port
   99 
  100 
  101 ''l2_switch''
  102 
  103 Turn on/off the inter function traffic through l2 switch
  104     0 - Disallow inter function traffic
  105     1 - Allow inter function traffic
  106     Default: -1
  107 
  108 
  109 ''bandwidth_0 - bandwidth_7''
  110 
  111     Desired max receive/transmit bandwidth,in Mbps for function 0 to function 7
  112     Minimum value is 100 Mbps, for 1 Gbps specify a value of 1024.
  113 
  114 
  115 ''priority_0 - priority_7''
  116 
  117     Desired receive/transmit priority for function 0 to function 7 
  118 
  119 
  120 ''intr_coalesce''
  121 
  122     Adaptive interrupt coalescing
  123     0 - Disable
  124     1 - Enable
  125 
  126 
  127 ''Low Latency''
  128 
  129    0 - Disable
  130    1 - Enable
  131 
  132 === Installation Instructions ===
  133 
  134 ''Identifying the Adapter''
  135 
  136     The X3100 adapter is identified by the board ID number on the adapter.
  137 
  138     Look for a label that has a barcode and a number, for example,
  139     SXT0425072. The factory-burned MAC address (hardware address)
  140     shows up on the board above the serial number, 
  141     (similar to 000CFC000449 -- 00:0C:FC:00:04:49).
  142 
  143 
  144 ''Kernel Driver Source Package''
  145 
  146 This package contains kernel_update.sh script which is to be used to copy driver sources to kernel path.
  147 It creates vxge folder with source code in /usr/src/sys/dev and Makefile in /usr/src/sys/modules.
  148 
  149 Loadable parameters can be changed by putting below lines in /boot/device.hints and set values as desired.
  150         hint.vxge.0.msix_enable="1"
  151         hint.vxge.0.rth_enable="1"
  152         hint.vxge.0.lro_enable="1"
  153         hint.vxge.0.tso_enable="1"
  154         hint.vxge.0.tx_steering="1"
  155         hint.vxge.0.no_of_vpath="-1"
  156         hint.vxge.0.func_mode="-1"
  157         hint.vxge.0.port_mode="-1"
  158         hint.vxge.0.fw_upgrade="1"
  159         hint.vxge.0.bandwidth_0="-1"
  160         hint.vxge.0.bandwidth_1="-1"
  161         hint.vxge.0.bandwidth_2="-1"
  162         hint.vxge.0.bandwidth_3="-1"
  163         hint.vxge.0.bandwidth_4="-1"
  164         hint.vxge.0.bandwidth_5="-1"
  165         hint.vxge.0.bandwidth_6="-1"
  166         hint.vxge.0.bandwidth_7="-1"
  167         hint.vxge.0.priority_0="-1"
  168         hint.vxge.0.priority_1="-1"
  169         hint.vxge.0.priority_2="-1"
  170         hint.vxge.0.priority_3="-1"
  171         hint.vxge.0.priority_4="-1"
  172         hint.vxge.0.priority_5="-1"
  173         hint.vxge.0.priority_6="-1"
  174         hint.vxge.0.priority_7="-1"
  175         hint.vxge.0.intr_coalesce="0"
  176         hint.vxge.0.low_latency="0"
  177 
  178         e.g., set hint.vxge.0.msix_enable to 0 to load driver in INTA mode.
  179         Then reboot the system to add loadable parameters to kenv.
  180 
  181 
  182 ''Standalone Driver Source Package''
  183 
  184 vxge (FreeBSD package)
  185     This directory contains FreeBSD driver sources for X3100 device(s),
  186     Makefile, and X3100 Hardware Abstraction headers and sources
  187     (vxgehal and include folders)
  188 
  189 vxgehal
  190     This directory contains the X3100 HAL sources.
  191     The driver uses HAL to perform operations on the X3100 hardware.
  192 
  193 include
  194     The include subdirectory contains HAL header files.
  195  
  196 Source code for vxge-manage tool (used to get statistics, pciconfig and
  197 register dump) are included in the freebsd directory.
  198 
  199 
  200 ''Building the driver''
  201 
  202     The driver is distributed in the source form. Driver and installation
  203     utility executables need to be built for the target platform. 
  204 
  205     In the directory containing Makefile for building Exar driver for FreeBSD, 
  206         #make clean
  207         #make
  208 
  209     Please unload previously installed Exar drivers before proceeding with following steps.
  210         #make uninstall
  211 
  212 
  213 ''Loading the driver''
  214 
  215      Use "kldload" to load driver module vxge.ko.
  216         #kldload ./vxge.ko
  217 
  218     Run "kldstat" and find an entry for vxge kernel module to ensure driver installation
  219     was successful.
  220         #kldstat | grep vxge
  221         3 1 0xc22cc000 26000   vxge.ko
  222 
  223 
  224 ''Enabling interface and assigning IP address''
  225 
  226     #ifconfig <INTERFACE> <IP_ADDRESS> up
  227     <INTERFACE> will be similar to vxge0, vxge1 etc. and can be
  228     found by executing "ifconfig -a".
  229     Neterion adapters typically have MAC addresses starting with
  230     "00:0C:FC" or "00:11:25".
  231 
  232     Example:
  233         #ifconfig vxge0 10.2.2.40 up
  234         Enables vxge0 interface and assigns to it the IP address 10.2.2.40.
  235 
  236         vxge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  237         options=53b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4,LRO>
  238         ether 00:0c:fc:00:da:47
  239         inet6 fe80::20c:fcff:fe00:da47%vxge0 prefixlen 64 scopeid 0x3
  240         inet 10.2.2.40 netmask 0xff000000 broadcast 17.255.255.255
  241         media: Ethernet autoselect (10Gbase-SR <full-duplex>)
  242         status: active
  243 
  244 
  245 ''Disabling the interface''
  246 
  247     #ifconfig <INTERFACE> down
  248     Example:
  249         #ifconfig vxge0 down
  250 
  251 
  252 ''Unloading the Driver''
  253         #kldunload vxge.ko
  254 
  255 
  256 === Performance Suggestions ===
  257 
  258 Sysctl Tuning Parameters
  259         #sysctl net.inet.tcp.sendspace=786432
  260         #sysctl net.inet.tcp.recvspace=786432
  261         #sysctl net.inet.tcp.recvbuf_max=16777216
  262         #sysctl net.inet.tcp.sendbuf_max=16777216
  263         #sysctl net.inet.tcp.blackhole=1
  264         #sysctl net.inet.tcp.rfc1323=1
  265         #sysctl net.inet.tcp.path_mtu_discovery=1
  266         #sysctl net.inet.tcp.inflight.enable=0
  267         #sysctl net.inet.ip.maxfragsperpacket=2147483647
  268         #sysctl kern.ipc.maxsockbuf=8388608
  269         #sysctl kern.ipc.nmbclusters=2147483647
  270         #sysctl kern.ipc.nmbjumbop=262144
  271         #sysctl kern.ipc.maxsockets=81920
  272         #sysctl hw.intr_storm_threshold=9000
  273 
  274 
  275 ''Usage & Troubleshooting''
  276 
  277 For general information and support, please visit Neterion support website at 
  278 http://www.neterion.com/support/support.html
  279 
  280 Make sure that the operating system identifies the X3100 adapter. Note that
  281 Neterion vendor ID is 0x17D5 and X3110 and X3120 adapters can be fixed to both PCIe slots.
  282 
  283 The rest of this section details troubleshooting tips and information. Some of
  284 them are general and some are more specific. For online Troubleshooting tips 
  285 and faqs, please visit
  286 http://trac.neterion.com/cgi-bin/trac.cgi/wiki/TitleIndex?anonymous
  287 
  288 
  289 ''Loading the driver and initializing the device''
  290 
  291 The first time FreeBSD identifies the device it stores the corresponding
  292 device/vendor IDs in an enumerated tree of PCI devices. Note that Neterion
  293 vendor id is 0x17d5.
  294 
  295 After cold reboot FreeBSD finds the device and tries to load the corresponding
  296 driver. If it fails, try to switch the card and/or cable. And, in parallel, send
  297 us the "tail" of the "/var/log/messages". We also need traces, register dump and
  298 statistics(Use vxge-manage tool).
  299 
  300 
  301 ''Collect information''
  302 
  303 If you're reporting a problem to Neterion, please describe:
  304 host(s) and adapter(s), switch (if used), software version.
  305 
  306 
  307 ''ARP''
  308 
  309 If this is a basic connectivity issue (e.g., cannot connect, cannot ping),
  310 make sure first that ARP works
  311 Do you see ARPs coming through a switch (in case switch is used)?
  312 Do you see frame drops at the switch when pinging?
  313 Do you see frame counts increasing via statistics?
  314 
  315 
  316 ''Have you tried''
  317 
  318     a. A previous driver release;
  319     b. A different adapter in the same PCI slot;
  320     c. A different PCI slot;
  321     d. Back-to-back setup so that the switch is excluded from the equation.
  322     e. To replace the cables?
  323     f. To use a different PCI slot?
  324 
  325 
  326 ''Start clean''
  327 
  328     a. Have you tried to reboot the switch? Cold-reboot the host?
  329     b. Make sure that the latest released driver gets loaded after the host
  330     reboot, and that the 10GE interface shows up via ifconfig.
  331 
  332 
  333 ''LEDs''
  334 
  335 Do you see LED going green after everything is connected and drivers loaded?
  336 How/when does the color changes?
  337 
  338 
  339 ''ifconfig''
  340 
  341 Run "ifconfig -a" at the command prompt and check whether the output Looks as
  342 expected. Include the output in your problem report.
  343 
  344 Note for instance that "all-foxes" i.e, FF:FF:FF:FF:FF:FF MAC address could
  345 explain a general connectivity issue, if that's what you see.
  346 
  347 
  348 ''Log''
  349 
  350 Please attach the generated log, with traces enabled.
  351 
  352 Note that the driver's logging facility is configurable at compile-time. Errors
  353 and traces can be compiled out on a per-component basis. The components are: HAL
  354 fifos and rings, device, etc., see VXGE_COMPONENT_HAL_??? in the Makefile.
  355 
  356 Here's how you enable all except data path traces:
  357 CFLAGS_VXGE = -DVXGE_DEBUG_MODULE_MASK=0xffffffbf  \
  358 -DVXGE_DEBUG_ERR_MASK=0xffffffbf
  359 Recompile with traces and include the log in the report.
  360 
  361 
  362 === Utilities ===
  363 
  364 ''Statistics''
  365 
  366  To print hardware and software statistics for interface instance 0
  367        (i.e., vxge0), run:
  368         #vxge-manage vxgeX stats common
  369         #vxge-manage vxgeX stats mrpcim
  370         #vxge-manage vxgeX stats driver
  371         #vxge-manage vxgeX pciconfig
  372         #vxge-manage vxgeX hwinfo
  373         #vxge-manage vxgeX bw_pri_get
  374         #vxge-manage vxgeX bw_pri_get vf_id
  375         #vxge-manage vxgeX port_mode_get
  376 
  377 The vxge-manage tool generates log file in the working directory. Once done, ping a
  378 few times, and collect the statistics again (Ping both from this and the remote
  379 machines).
  380 
  381 Many counters could be of interest. For example, "rx_vld_frms" counts all
  382 valid incoming Ethernet frames seen by the adapter. Information could be derived
  383 from the fact that (for instance) counter stay constant during ping, if that is
  384 what happening. For detailed description of the X3100 counters, please refer
  385 to the "X3100 User Guide".
  386 
  387 Please include the statistics into your problem report.
  388 
  389 
  390 ''X3100 registers''
  391 
  392        Use vxge-manage to dump all X3100 BAR0 space registers. Include this register dump
  393        into your problem report.
  394        #vxge-manage vxgeX regs
  395 
  396 
  397 ''PCI configuration space''
  398 
  399 Use vxge-manage to retrieve PCI vendor, device, etc. Include the PCI configutation
  400 space in your problem report.
  401         #vxge-manage vxgeX pciconfig
  402 
  403 
  404 ''Hardware Info''
  405 
  406 To retrieve hardware info of device, e.g, serial / part number and function mode etc.
  407 use vxge-manage 
  408         #vxge-manage vxgeX hwinfo
  409 
  410 
  411 ''Bandwidth and Priority''
  412 
  413 Use vxge-manage to display Bandwidth and Priority information.
  414         #vxge-manage vxgeX bw_pri_get
  415         or
  416         #vxge-manage vxgeX bw_pri_get vf_id
  417 
  418 vxge-manage can also be used to set bandwidth and priority for individual VF.
  419         #vxge-manage vxgeX bw_pri_set vf_id bandwidth 
  420         or
  421         #vxge-manage vxgeX bw_pri_set vf_id bandwidth priority
  422 
  423         Example:
  424         #vxge-manage vxge0 bw_pri_set 0 1500
  425         or
  426         #vxge-manage vxge0 bw_pri_set 0 1500 1
  427 
  428 ''Port mode''
  429 
  430 Use vxge-manage to display Port mode setting
  431         #vxge-manage vxgeX port_mode_get
  432 
  433 vxge-manage can also be used to set Port mode.
  434         #vxge-manage vxgeX port_mode_set port_mode_value
  435 
  436         Example:
  437         #vxge-manage vxge0 port_mode_set 2
  438 
  439 === Known Issues ===
  440 
  441 
  442 === Available Downloads ===
  443 
  444    For latest available drivers or further support please contact your network
  445    adapter provider or neterionsupport@exar.com.
  446 
  447 
  448 ===================================================================
  449  Exar Corp., Proprietary
  450  COPYRIGHT (c) 2002-2011 Exar corp., ALL RIGHTS RESERVED

[ source navigation ] [ identifier search ] [ freetext search ] [ file search ] [ list types ] [ track identifier ]


This page is part of the FreeBSD/Linux Linux Kernel Cross-Reference, and was automatically generated using a modified version of the LXR engine.