The Design and Implementation of the FreeBSD Operating System, Second Edition
Now available: The Design and Implementation of the FreeBSD Operating System (Second Edition)


[ source navigation ] [ diff markup ] [ identifier search ] [ freetext search ] [ file search ] [ list types ] [ track identifier ]

FreeBSD/Linux Kernel Cross Reference
sys/netinet/tcp_output.c

Version: -  FREEBSD  -  FREEBSD-13-STABLE  -  FREEBSD-13-0  -  FREEBSD-12-STABLE  -  FREEBSD-12-0  -  FREEBSD-11-STABLE  -  FREEBSD-11-0  -  FREEBSD-10-STABLE  -  FREEBSD-10-0  -  FREEBSD-9-STABLE  -  FREEBSD-9-0  -  FREEBSD-8-STABLE  -  FREEBSD-8-0  -  FREEBSD-7-STABLE  -  FREEBSD-7-0  -  FREEBSD-6-STABLE  -  FREEBSD-6-0  -  FREEBSD-5-STABLE  -  FREEBSD-5-0  -  FREEBSD-4-STABLE  -  FREEBSD-3-STABLE  -  FREEBSD22  -  l41  -  OPENBSD  -  linux-2.6  -  MK84  -  PLAN9  -  xnu-8792 
SearchContext: -  none  -  3  -  10 

    1 /*      $NetBSD: tcp_output.c,v 1.167.10.1 2011/03/29 20:12:14 riz Exp $        */
    2 
    3 /*
    4  * Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project.
    5  * All rights reserved.
    6  *
    7  * Redistribution and use in source and binary forms, with or without
    8  * modification, are permitted provided that the following conditions
    9  * are met:
   10  * 1. Redistributions of source code must retain the above copyright
   11  *    notice, this list of conditions and the following disclaimer.
   12  * 2. Redistributions in binary form must reproduce the above copyright
   13  *    notice, this list of conditions and the following disclaimer in the
   14  *    documentation and/or other materials provided with the distribution.
   15  * 3. Neither the name of the project nor the names of its contributors
   16  *    may be used to endorse or promote products derived from this software
   17  *    without specific prior written permission.
   18  *
   19  * THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND
   20  * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
   21  * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
   22  * ARE DISCLAIMED.  IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE
   23  * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
   24  * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
   25  * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
   26  * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
   27  * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
   28  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
   29  * SUCH DAMAGE.
   30  */
   31 
   32 /*
   33  *      @(#)COPYRIGHT   1.1 (NRL) 17 January 1995
   34  *
   35  * NRL grants permission for redistribution and use in source and binary
   36  * forms, with or without modification, of the software and documentation
   37  * created at NRL provided that the following conditions are met:
   38  *
   39  * 1. Redistributions of source code must retain the above copyright
   40  *    notice, this list of conditions and the following disclaimer.
   41  * 2. Redistributions in binary form must reproduce the above copyright
   42  *    notice, this list of conditions and the following disclaimer in the
   43  *    documentation and/or other materials provided with the distribution.
   44  * 3. All advertising materials mentioning features or use of this software
   45  *    must display the following acknowledgements:
   46  *      This product includes software developed by the University of
   47  *      California, Berkeley and its contributors.
   48  *      This product includes software developed at the Information
   49  *      Technology Division, US Naval Research Laboratory.
   50  * 4. Neither the name of the NRL nor the names of its contributors
   51  *    may be used to endorse or promote products derived from this software
   52  *    without specific prior written permission.
   53  *
   54  * THE SOFTWARE PROVIDED BY NRL IS PROVIDED BY NRL AND CONTRIBUTORS ``AS
   55  * IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
   56  * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
   57  * PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL NRL OR
   58  * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
   59  * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
   60  * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
   61  * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
   62  * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
   63  * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
   64  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   65  *
   66  * The views and conclusions contained in the software and documentation
   67  * are those of the authors and should not be interpreted as representing
   68  * official policies, either expressed or implied, of the US Naval
   69  * Research Laboratory (NRL).
   70  */
   71 
   72 /*-
   73  * Copyright (c) 1997, 1998, 2001, 2005, 2006 The NetBSD Foundation, Inc.
   74  * All rights reserved.
   75  *
   76  * This code is derived from software contributed to The NetBSD Foundation
   77  * by Jason R. Thorpe and Kevin M. Lahey of the Numerical Aerospace Simulation
   78  * Facility, NASA Ames Research Center.
   79  * This code is derived from software contributed to The NetBSD Foundation
   80  * by Charles M. Hannum.
   81  * This code is derived from software contributed to The NetBSD Foundation
   82  * by Rui Paulo.
   83  *
   84  * Redistribution and use in source and binary forms, with or without
   85  * modification, are permitted provided that the following conditions
   86  * are met:
   87  * 1. Redistributions of source code must retain the above copyright
   88  *    notice, this list of conditions and the following disclaimer.
   89  * 2. Redistributions in binary form must reproduce the above copyright
   90  *    notice, this list of conditions and the following disclaimer in the
   91  *    documentation and/or other materials provided with the distribution.
   92  *
   93  * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
   94  * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
   95  * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
   96  * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
   97  * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
   98  * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
   99  * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
  100  * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
  101  * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
  102  * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
  103  * POSSIBILITY OF SUCH DAMAGE.
  104  */
  105 
  106 /*
  107  * Copyright (c) 1982, 1986, 1988, 1990, 1993, 1995
  108  *      The Regents of the University of California.  All rights reserved.
  109  *
  110  * Redistribution and use in source and binary forms, with or without
  111  * modification, are permitted provided that the following conditions
  112  * are met:
  113  * 1. Redistributions of source code must retain the above copyright
  114  *    notice, this list of conditions and the following disclaimer.
  115  * 2. Redistributions in binary form must reproduce the above copyright
  116  *    notice, this list of conditions and the following disclaimer in the
  117  *    documentation and/or other materials provided with the distribution.
  118  * 3. Neither the name of the University nor the names of its contributors
  119  *    may be used to endorse or promote products derived from this software
  120  *    without specific prior written permission.
  121  *
  122  * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
  123  * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
  124  * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
  125  * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
  126  * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
  127  * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
  128  * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
  129  * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
  130  * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
  131  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  132  * SUCH DAMAGE.
  133  *
  134  *      @(#)tcp_output.c        8.4 (Berkeley) 5/24/95
  135  */
  136 
  137 #include <sys/cdefs.h>
  138 __KERNEL_RCSID(0, "$NetBSD: tcp_output.c,v 1.167.10.1 2011/03/29 20:12:14 riz Exp $");
  139 
  140 #include "opt_inet.h"
  141 #include "opt_ipsec.h"
  142 #include "opt_tcp_debug.h"
  143 
  144 #include <sys/param.h>
  145 #include <sys/systm.h>
  146 #include <sys/malloc.h>
  147 #include <sys/mbuf.h>
  148 #include <sys/protosw.h>
  149 #include <sys/socket.h>
  150 #include <sys/socketvar.h>
  151 #include <sys/errno.h>
  152 #include <sys/domain.h>
  153 #include <sys/kernel.h>
  154 #ifdef TCP_SIGNATURE
  155 #include <sys/md5.h>
  156 #endif
  157 
  158 #include <net/if.h>
  159 #include <net/route.h>
  160 
  161 #include <netinet/in.h>
  162 #include <netinet/in_systm.h>
  163 #include <netinet/ip.h>
  164 #include <netinet/in_pcb.h>
  165 #include <netinet/ip_var.h>
  166 
  167 #ifdef INET6
  168 #ifndef INET
  169 #include <netinet/in.h>
  170 #endif
  171 #include <netinet/ip6.h>
  172 #include <netinet6/in6_var.h>
  173 #include <netinet6/ip6_var.h>
  174 #include <netinet6/in6_pcb.h>
  175 #include <netinet6/nd6.h>
  176 #endif
  177 
  178 #ifdef FAST_IPSEC
  179 #include <netipsec/ipsec.h>
  180 #include <netipsec/key.h>
  181 #ifdef INET6
  182 #include <netipsec/ipsec6.h>
  183 #endif
  184 #endif  /* FAST_IPSEC*/
  185 #ifdef IPSEC
  186 #include <netinet6/ipsec.h>
  187 #endif
  188 
  189 #include <netinet/tcp.h>
  190 #define TCPOUTFLAGS
  191 #include <netinet/tcp_fsm.h>
  192 #include <netinet/tcp_seq.h>
  193 #include <netinet/tcp_timer.h>
  194 #include <netinet/tcp_var.h>
  195 #include <netinet/tcp_private.h>
  196 #include <netinet/tcp_congctl.h>
  197 #include <netinet/tcpip.h>
  198 #include <netinet/tcp_debug.h>
  199 #include <netinet/in_offload.h>
  200 #include <netinet6/in6_offload.h>
  201 
  202 #ifdef IPSEC
  203 #include <netkey/key.h>
  204 #endif
  205 
  206 #ifdef notyet
  207 extern struct mbuf *m_copypack();
  208 #endif
  209 
  210 /*
  211  * Knob to enable Congestion Window Monitoring, and control
  212  * the burst size it allows.  Default burst is 4 packets, per
  213  * the Internet draft.
  214  */
  215 int     tcp_cwm = 0;
  216 int     tcp_cwm_burstsize = 4;
  217 
  218 int     tcp_do_autosndbuf = 0;
  219 int     tcp_autosndbuf_inc = 8 * 1024;
  220 int     tcp_autosndbuf_max = 256 * 1024;
  221 
  222 #ifdef TCP_OUTPUT_COUNTERS
  223 #include <sys/device.h>
  224 
  225 extern struct evcnt tcp_output_bigheader;
  226 extern struct evcnt tcp_output_predict_hit;
  227 extern struct evcnt tcp_output_predict_miss;
  228 extern struct evcnt tcp_output_copysmall;
  229 extern struct evcnt tcp_output_copybig;
  230 extern struct evcnt tcp_output_refbig;
  231 
  232 #define TCP_OUTPUT_COUNTER_INCR(ev)     (ev)->ev_count++
  233 #else
  234 
  235 #define TCP_OUTPUT_COUNTER_INCR(ev)     /* nothing */
  236 
  237 #endif /* TCP_OUTPUT_COUNTERS */
  238 
  239 static
  240 #ifndef GPROF
  241 inline
  242 #endif
  243 int
  244 tcp_segsize(struct tcpcb *tp, int *txsegsizep, int *rxsegsizep,
  245     bool *alwaysfragp)
  246 {
  247 #ifdef INET
  248         struct inpcb *inp = tp->t_inpcb;
  249 #endif
  250 #ifdef INET6
  251         struct in6pcb *in6p = tp->t_in6pcb;
  252 #endif
  253         struct socket *so = NULL;
  254         struct rtentry *rt;
  255         struct ifnet *ifp;
  256         int size;
  257         int hdrlen;
  258         int optlen;
  259 
  260         *alwaysfragp = false;
  261 
  262 #ifdef DIAGNOSTIC
  263         if (tp->t_inpcb && tp->t_in6pcb)
  264                 panic("tcp_segsize: both t_inpcb and t_in6pcb are set");
  265 #endif
  266         switch (tp->t_family) {
  267 #ifdef INET
  268         case AF_INET:
  269                 hdrlen = sizeof(struct ip) + sizeof(struct tcphdr);
  270                 break;
  271 #endif
  272 #ifdef INET6
  273         case AF_INET6:
  274                 hdrlen = sizeof(struct ip6_hdr) + sizeof(struct tcphdr);
  275                 break;
  276 #endif
  277         default:
  278                 size = tcp_mssdflt;
  279                 goto out;
  280         }
  281 
  282         rt = NULL;
  283 #ifdef INET
  284         if (inp) {
  285                 rt = in_pcbrtentry(inp);
  286                 so = inp->inp_socket;
  287         }
  288 #endif
  289 #ifdef INET6
  290         if (in6p) {
  291                 rt = in6_pcbrtentry(in6p);
  292                 so = in6p->in6p_socket;
  293         }
  294 #endif
  295         if (rt == NULL) {
  296                 size = tcp_mssdflt;
  297                 goto out;
  298         }
  299 
  300         ifp = rt->rt_ifp;
  301 
  302         size = tcp_mssdflt;
  303         if (tp->t_mtudisc && rt->rt_rmx.rmx_mtu != 0) {
  304 #ifdef INET6
  305                 if (in6p && rt->rt_rmx.rmx_mtu < IPV6_MMTU) {
  306                         /*
  307                          * RFC2460 section 5, last paragraph: if path MTU is
  308                          * smaller than 1280, use 1280 as packet size and
  309                          * attach fragment header.
  310                          */
  311                         size = IPV6_MMTU - hdrlen - sizeof(struct ip6_frag);
  312                         *alwaysfragp = true;
  313                 } else
  314                         size = rt->rt_rmx.rmx_mtu - hdrlen;
  315 #else
  316                 size = rt->rt_rmx.rmx_mtu - hdrlen;
  317 #endif
  318         } else if (ifp->if_flags & IFF_LOOPBACK)
  319                 size = ifp->if_mtu - hdrlen;
  320 #ifdef INET
  321         else if (inp && tp->t_mtudisc)
  322                 size = ifp->if_mtu - hdrlen;
  323         else if (inp && in_localaddr(inp->inp_faddr))
  324                 size = ifp->if_mtu - hdrlen;
  325 #endif
  326 #ifdef INET6
  327         else if (in6p) {
  328 #ifdef INET
  329                 if (IN6_IS_ADDR_V4MAPPED(&in6p->in6p_faddr)) {
  330                         /* mapped addr case */
  331                         struct in_addr d;
  332                         bcopy(&in6p->in6p_faddr.s6_addr32[3], &d, sizeof(d));
  333                         if (tp->t_mtudisc || in_localaddr(d))
  334                                 size = ifp->if_mtu - hdrlen;
  335                 } else
  336 #endif
  337                 {
  338                         /*
  339                          * for IPv6, path MTU discovery is always turned on,
  340                          * or the node must use packet size <= 1280.
  341                          */
  342                         size = tp->t_mtudisc ? IN6_LINKMTU(ifp) : IPV6_MMTU;
  343                         size -= hdrlen;
  344                 }
  345         }
  346 #endif
  347  out:
  348         /*
  349          * Now we must make room for whatever extra TCP/IP options are in
  350          * the packet.
  351          */
  352         optlen = tcp_optlen(tp);
  353 
  354         /*
  355          * XXX tp->t_ourmss should have the right size, but without this code
  356          * fragmentation will occur... need more investigation
  357          */
  358 #ifdef INET
  359         if (inp) {
  360 #if defined(IPSEC) || defined(FAST_IPSEC)
  361                 if (! IPSEC_PCB_SKIP_IPSEC(inp->inp_sp, IPSEC_DIR_OUTBOUND))
  362                         optlen += ipsec4_hdrsiz_tcp(tp);
  363 #endif
  364                 optlen += ip_optlen(inp);
  365         }
  366 #endif
  367 #ifdef INET6
  368 #ifdef INET
  369         if (in6p && tp->t_family == AF_INET) {
  370 #if defined(IPSEC) || defined(FAST_IPSEC)
  371                 if (! IPSEC_PCB_SKIP_IPSEC(in6p->in6p_sp, IPSEC_DIR_OUTBOUND))
  372                         optlen += ipsec4_hdrsiz_tcp(tp);
  373 #endif
  374                 /* XXX size -= ip_optlen(in6p); */
  375         } else
  376 #endif
  377         if (in6p && tp->t_family == AF_INET6) {
  378 #if defined(IPSEC) || defined(FAST_IPSEC)
  379                 if (! IPSEC_PCB_SKIP_IPSEC(in6p->in6p_sp, IPSEC_DIR_OUTBOUND))
  380                         optlen += ipsec6_hdrsiz_tcp(tp);
  381 #endif
  382                 optlen += ip6_optlen(in6p);
  383         }
  384 #endif
  385         size -= optlen;
  386 
  387         /* there may not be any room for data if mtu is too small */
  388         if (size < 0)
  389                 return (EMSGSIZE);
  390 
  391         /*
  392          * *rxsegsizep holds *estimated* inbound segment size (estimation
  393          * assumes that path MTU is the same for both ways).  this is only
  394          * for silly window avoidance, do not use the value for other purposes.
  395          *
  396          * ipseclen is subtracted from both sides, this may not be right.
  397          * I'm not quite sure about this (could someone comment).
  398          */
  399         *txsegsizep = min(tp->t_peermss - optlen, size);
  400         /*
  401          * Never send more than half a buffer full.  This insures that we can
  402          * always keep 2 packets on the wire, no matter what SO_SNDBUF is, and
  403          * therefore acks will never be delayed unless we run out of data to
  404          * transmit.
  405          */
  406         if (so)
  407                 *txsegsizep = min(so->so_snd.sb_hiwat >> 1, *txsegsizep);
  408         *rxsegsizep = min(tp->t_ourmss - optlen, size);
  409 
  410         if (*txsegsizep != tp->t_segsz) {
  411                 /*
  412                  * If the new segment size is larger, we don't want to
  413                  * mess up the congestion window, but if it is smaller
  414                  * we'll have to reduce the congestion window to ensure
  415                  * that we don't get into trouble with initial windows
  416                  * and the rest.  In any case, if the segment size
  417                  * has changed, chances are the path has, too, and
  418                  * our congestion window will be different.
  419                  */
  420                 if (*txsegsizep < tp->t_segsz) {
  421                         tp->snd_cwnd = max((tp->snd_cwnd / tp->t_segsz)
  422                                            * *txsegsizep, *txsegsizep);
  423                         tp->snd_ssthresh = max((tp->snd_ssthresh / tp->t_segsz)
  424                                                 * *txsegsizep, *txsegsizep);
  425                 }
  426                 tp->t_segsz = *txsegsizep;
  427         }
  428 
  429         return (0);
  430 }
  431 
  432 static
  433 #ifndef GPROF
  434 inline
  435 #endif
  436 int
  437 tcp_build_datapkt(struct tcpcb *tp, struct socket *so, int off,
  438     long len, int hdrlen, struct mbuf **mp)
  439 {
  440         struct mbuf *m, *m0;
  441         uint64_t *tcps;
  442 
  443         tcps = TCP_STAT_GETREF();
  444         if (tp->t_force && len == 1)
  445                 tcps[TCP_STAT_SNDPROBE]++;
  446         else if (SEQ_LT(tp->snd_nxt, tp->snd_max)) {
  447                 tcps[TCP_STAT_SNDREXMITPACK]++;
  448                 tcps[TCP_STAT_SNDREXMITBYTE] += len;
  449         } else {
  450                 tcps[TCP_STAT_SNDPACK]++;
  451                 tcps[TCP_STAT_SNDBYTE] += len;
  452         }
  453         TCP_STAT_PUTREF();
  454 #ifdef notyet
  455         if ((m = m_copypack(so->so_snd.sb_mb, off,
  456             (int)len, max_linkhdr + hdrlen)) == 0)
  457                 return (ENOBUFS);
  458         /*
  459          * m_copypack left space for our hdr; use it.
  460          */
  461         m->m_len += hdrlen;
  462         m->m_data -= hdrlen;
  463 #else
  464         MGETHDR(m, M_DONTWAIT, MT_HEADER);
  465         if (__predict_false(m == NULL))
  466                 return (ENOBUFS);
  467         MCLAIM(m, &tcp_tx_mowner);
  468 
  469         /*
  470          * XXX Because other code assumes headers will fit in
  471          * XXX one header mbuf.
  472          *
  473          * (This code should almost *never* be run.)
  474          */
  475         if (__predict_false((max_linkhdr + hdrlen) > MHLEN)) {
  476                 TCP_OUTPUT_COUNTER_INCR(&tcp_output_bigheader);
  477                 MCLGET(m, M_DONTWAIT);
  478                 if ((m->m_flags & M_EXT) == 0) {
  479                         m_freem(m);
  480                         return (ENOBUFS);
  481                 }
  482         }
  483 
  484         m->m_data += max_linkhdr;
  485         m->m_len = hdrlen;
  486 
  487         /*
  488          * To avoid traversing the whole sb_mb chain for correct
  489          * data to send, remember last sent mbuf, its offset and
  490          * the sent size.  When called the next time, see if the
  491          * data to send is directly following the previous transfer.
  492          * This is important for large TCP windows.
  493          */
  494         if (off == 0 || tp->t_lastm == NULL ||
  495             (tp->t_lastoff + tp->t_lastlen) != off) {
  496                 TCP_OUTPUT_COUNTER_INCR(&tcp_output_predict_miss);
  497                 /*
  498                  * Either a new packet or a retransmit.
  499                  * Start from the beginning.
  500                  */
  501                 tp->t_lastm = so->so_snd.sb_mb;
  502                 tp->t_inoff = off;
  503         } else {
  504                 TCP_OUTPUT_COUNTER_INCR(&tcp_output_predict_hit);
  505                 tp->t_inoff += tp->t_lastlen;
  506         }
  507 
  508         /* Traverse forward to next packet */
  509         while (tp->t_inoff > 0) {
  510                 if (tp->t_lastm == NULL)
  511                         panic("tp->t_lastm == NULL");
  512                 if (tp->t_inoff < tp->t_lastm->m_len)
  513                         break;
  514                 tp->t_inoff -= tp->t_lastm->m_len;
  515                 tp->t_lastm = tp->t_lastm->m_next;
  516         }
  517 
  518         tp->t_lastoff = off;
  519         tp->t_lastlen = len;
  520         m0 = tp->t_lastm;
  521         off = tp->t_inoff;
  522 
  523         if (len <= M_TRAILINGSPACE(m)) {
  524                 m_copydata(m0, off, (int) len, mtod(m, char *) + hdrlen);
  525                 m->m_len += len;
  526                 TCP_OUTPUT_COUNTER_INCR(&tcp_output_copysmall);
  527         } else {
  528                 m->m_next = m_copym(m0, off, (int) len, M_DONTWAIT);
  529                 if (m->m_next == NULL) {
  530                         m_freem(m);
  531                         return (ENOBUFS);
  532                 }
  533 #ifdef TCP_OUTPUT_COUNTERS
  534                 if (m->m_next->m_flags & M_EXT)
  535                         TCP_OUTPUT_COUNTER_INCR(&tcp_output_refbig);
  536                 else
  537                         TCP_OUTPUT_COUNTER_INCR(&tcp_output_copybig);
  538 #endif /* TCP_OUTPUT_COUNTERS */
  539         }
  540 #endif
  541 
  542         *mp = m;
  543         return (0);
  544 }
  545 
  546 /*
  547  * Tcp output routine: figure out what should be sent and send it.
  548  */
  549 int
  550 tcp_output(struct tcpcb *tp)
  551 {
  552         struct rtentry *rt;
  553         struct socket *so;
  554         struct route *ro;
  555         long len, win;
  556         int off, flags, error;
  557         struct mbuf *m;
  558         struct ip *ip;
  559 #ifdef INET6
  560         struct ip6_hdr *ip6;
  561 #endif
  562         struct tcphdr *th;
  563         u_char opt[MAX_TCPOPTLEN];
  564         unsigned optlen, hdrlen, packetlen;
  565         unsigned int sack_numblks;
  566         int idle, sendalot, txsegsize, rxsegsize;
  567         int txsegsize_nosack;
  568         int maxburst = TCP_MAXBURST;
  569         int af;         /* address family on the wire */
  570         int iphdrlen;
  571         int has_tso4, has_tso6;
  572         int has_tso, use_tso;
  573         bool alwaysfrag;
  574         int sack_rxmit;
  575         int sack_bytes_rxmt;
  576         int ecn_tos;
  577         struct sackhole *p;
  578 #ifdef TCP_SIGNATURE
  579         int sigoff = 0;
  580 #endif
  581         uint64_t *tcps;
  582 
  583 #ifdef DIAGNOSTIC
  584         if (tp->t_inpcb && tp->t_in6pcb)
  585                 panic("tcp_output: both t_inpcb and t_in6pcb are set");
  586 #endif
  587         so = NULL;
  588         ro = NULL;
  589         if (tp->t_inpcb) {
  590                 so = tp->t_inpcb->inp_socket;
  591                 ro = &tp->t_inpcb->inp_route;
  592         }
  593 #ifdef INET6
  594         else if (tp->t_in6pcb) {
  595                 so = tp->t_in6pcb->in6p_socket;
  596                 ro = &tp->t_in6pcb->in6p_route;
  597         }
  598 #endif
  599 
  600         switch (af = tp->t_family) {
  601 #ifdef INET
  602         case AF_INET:
  603                 if (tp->t_inpcb)
  604                         break;
  605 #ifdef INET6
  606                 /* mapped addr case */
  607                 if (tp->t_in6pcb)
  608                         break;
  609 #endif
  610                 return (EINVAL);
  611 #endif
  612 #ifdef INET6
  613         case AF_INET6:
  614                 if (tp->t_in6pcb)
  615                         break;
  616                 return (EINVAL);
  617 #endif
  618         default:
  619                 return (EAFNOSUPPORT);
  620         }
  621 
  622         if (tcp_segsize(tp, &txsegsize, &rxsegsize, &alwaysfrag))
  623                 return (EMSGSIZE);
  624 
  625         idle = (tp->snd_max == tp->snd_una);
  626 
  627         /*
  628          * Determine if we can use TCP segmentation offload:
  629          * - If we're using IPv4
  630          * - If there is not an IPsec policy that prevents it
  631          * - If the interface can do it
  632          */
  633         has_tso4 = has_tso6 = false;
  634 #if defined(INET)
  635         has_tso4 = tp->t_inpcb != NULL &&
  636 #if defined(IPSEC) || defined(FAST_IPSEC)
  637                   IPSEC_PCB_SKIP_IPSEC(tp->t_inpcb->inp_sp,
  638                                        IPSEC_DIR_OUTBOUND) &&
  639 #endif
  640                   (rt = rtcache_validate(&tp->t_inpcb->inp_route)) != NULL &&
  641                   (rt->rt_ifp->if_capenable & IFCAP_TSOv4) != 0;
  642 #endif /* defined(INET) */
  643 #if defined(INET6)
  644         has_tso6 = tp->t_in6pcb != NULL &&
  645 #if defined(IPSEC) || defined(FAST_IPSEC)
  646                   IPSEC_PCB_SKIP_IPSEC(tp->t_in6pcb->in6p_sp,
  647                                        IPSEC_DIR_OUTBOUND) &&
  648 #endif
  649                   (rt = rtcache_validate(&tp->t_in6pcb->in6p_route)) != NULL &&
  650                   (rt->rt_ifp->if_capenable & IFCAP_TSOv6) != 0;
  651 #endif /* defined(INET6) */
  652         has_tso = (has_tso4 || has_tso6) && !alwaysfrag;
  653 
  654         /*
  655          * Restart Window computation.  From draft-floyd-incr-init-win-03:
  656          *
  657          *      Optionally, a TCP MAY set the restart window to the
  658          *      minimum of the value used for the initial window and
  659          *      the current value of cwnd (in other words, using a
  660          *      larger value for the restart window should never increase
  661          *      the size of cwnd).
  662          */
  663         if (tcp_cwm) {
  664                 /*
  665                  * Hughes/Touch/Heidemann Congestion Window Monitoring.
  666                  * Count the number of packets currently pending
  667                  * acknowledgement, and limit our congestion window
  668                  * to a pre-determined allowed burst size plus that count.
  669                  * This prevents bursting once all pending packets have
  670                  * been acknowledged (i.e. transmission is idle).
  671                  *
  672                  * XXX Link this to Initial Window?
  673                  */
  674                 tp->snd_cwnd = min(tp->snd_cwnd,
  675                     (tcp_cwm_burstsize * txsegsize) +
  676                     (tp->snd_nxt - tp->snd_una));
  677         } else {
  678                 if (idle && (tcp_now - tp->t_rcvtime) >= tp->t_rxtcur) {
  679                         /*
  680                          * We have been idle for "a while" and no acks are
  681                          * expected to clock out any data we send --
  682                          * slow start to get ack "clock" running again.
  683                          */
  684                         int ss = tcp_init_win;
  685 #ifdef INET
  686                         if (tp->t_inpcb &&
  687                             in_localaddr(tp->t_inpcb->inp_faddr))
  688                                 ss = tcp_init_win_local;
  689 #endif
  690 #ifdef INET6
  691                         if (tp->t_in6pcb &&
  692                             in6_localaddr(&tp->t_in6pcb->in6p_faddr))
  693                                 ss = tcp_init_win_local;
  694 #endif
  695                         tp->snd_cwnd = min(tp->snd_cwnd,
  696                             TCP_INITIAL_WINDOW(ss, txsegsize));
  697                 }
  698         }
  699 
  700         txsegsize_nosack = txsegsize;
  701 again:
  702         ecn_tos = 0;
  703         use_tso = has_tso;
  704         if ((tp->t_flags & (TF_ECN_SND_CWR|TF_ECN_SND_ECE)) != 0) {
  705                 /* don't duplicate CWR/ECE. */
  706                 use_tso = 0;
  707         }
  708         TCP_REASS_LOCK(tp);
  709         sack_numblks = tcp_sack_numblks(tp);
  710         if (sack_numblks) {
  711                 int sackoptlen;
  712 
  713                 sackoptlen = TCP_SACK_OPTLEN(sack_numblks);
  714                 if (sackoptlen > txsegsize_nosack) {
  715                         sack_numblks = 0; /* give up SACK */
  716                         txsegsize = txsegsize_nosack;
  717                 } else {
  718                         if ((tp->rcv_sack_flags & TCPSACK_HAVED) != 0) {
  719                                 /* don't duplicate D-SACK. */
  720                                 use_tso = 0;
  721                         }
  722                         txsegsize = txsegsize_nosack - sackoptlen;
  723                 }
  724         } else {
  725                 txsegsize = txsegsize_nosack;
  726         }
  727 
  728         /*
  729          * Determine length of data that should be transmitted, and
  730          * flags that should be used.  If there is some data or critical
  731          * controls (SYN, RST) to send, then transmit; otherwise,
  732          * investigate further.
  733          *
  734          * Readjust SACK information to avoid resending duplicate data.
  735          */
  736         if (TCP_SACK_ENABLED(tp) && SEQ_LT(tp->snd_nxt, tp->snd_max))
  737                 tcp_sack_adjust(tp);
  738         sendalot = 0;
  739         off = tp->snd_nxt - tp->snd_una;
  740         win = min(tp->snd_wnd, tp->snd_cwnd);
  741 
  742         flags = tcp_outflags[tp->t_state];
  743 
  744         /*
  745          * Send any SACK-generated retransmissions.  If we're explicitly trying
  746          * to send out new data (when sendalot is 1), bypass this function.
  747          * If we retransmit in fast recovery mode, decrement snd_cwnd, since
  748          * we're replacing a (future) new transmission with a retransmission
  749          * now, and we previously incremented snd_cwnd in tcp_input().
  750          */
  751         /*
  752          * Still in sack recovery , reset rxmit flag to zero.
  753          */
  754         sack_rxmit = 0;
  755         sack_bytes_rxmt = 0;
  756         len = 0;
  757         p = NULL;
  758         do {
  759                 long cwin;
  760                 if (!TCP_SACK_ENABLED(tp))
  761                         break;
  762                 if (tp->t_partialacks < 0) 
  763                         break;
  764                 p = tcp_sack_output(tp, &sack_bytes_rxmt);
  765                 if (p == NULL)
  766                         break;
  767                 
  768                 cwin = min(tp->snd_wnd, tp->snd_cwnd) - sack_bytes_rxmt;
  769                 if (cwin < 0)
  770                         cwin = 0;
  771                 /* Do not retransmit SACK segments beyond snd_recover */
  772                 if (SEQ_GT(p->end, tp->snd_recover)) {
  773                         /*
  774                          * (At least) part of sack hole extends beyond
  775                          * snd_recover. Check to see if we can rexmit data
  776                          * for this hole.
  777                          */
  778                         if (SEQ_GEQ(p->rxmit, tp->snd_recover)) {
  779                                 /*
  780                                  * Can't rexmit any more data for this hole.
  781                                  * That data will be rexmitted in the next
  782                                  * sack recovery episode, when snd_recover
  783                                  * moves past p->rxmit.
  784                                  */
  785                                 p = NULL;
  786                                 break;
  787                         }
  788                         /* Can rexmit part of the current hole */
  789                         len = ((long)ulmin(cwin, tp->snd_recover - p->rxmit));
  790                 } else
  791                         len = ((long)ulmin(cwin, p->end - p->rxmit));
  792                 off = p->rxmit - tp->snd_una;
  793                 if (off + len > so->so_snd.sb_cc) {
  794                         /* 1 for TH_FIN */
  795                         KASSERT(off + len == so->so_snd.sb_cc + 1);
  796                         KASSERT(p->rxmit + len == tp->snd_max);
  797                         len = so->so_snd.sb_cc - off;
  798                 }
  799                 if (len > 0) {
  800                         sack_rxmit = 1;
  801                         sendalot = 1;
  802                 }
  803         } while (/*CONSTCOND*/0);
  804 
  805         /*
  806          * If in persist timeout with window of 0, send 1 byte.
  807          * Otherwise, if window is small but nonzero
  808          * and timer expired, we will send what we can
  809          * and go to transmit state.
  810          */
  811         if (tp->t_force) {
  812                 if (win == 0) {
  813                         /*
  814                          * If we still have some data to send, then
  815                          * clear the FIN bit.  Usually this would
  816                          * happen below when it realizes that we
  817                          * aren't sending all the data.  However,
  818                          * if we have exactly 1 byte of unset data,
  819                          * then it won't clear the FIN bit below,
  820                          * and if we are in persist state, we wind
  821                          * up sending the packet without recording
  822                          * that we sent the FIN bit.
  823                          *
  824                          * We can't just blindly clear the FIN bit,
  825                          * because if we don't have any more data
  826                          * to send then the probe will be the FIN
  827                          * itself.
  828                          */
  829                         if (off < so->so_snd.sb_cc)
  830                                 flags &= ~TH_FIN;
  831                         win = 1;
  832                 } else {
  833                         TCP_TIMER_DISARM(tp, TCPT_PERSIST);
  834                         tp->t_rxtshift = 0;
  835                 }
  836         }
  837 
  838         if (sack_rxmit == 0) {
  839                 if (TCP_SACK_ENABLED(tp) && tp->t_partialacks >= 0) {
  840                         long cwin;
  841 
  842                         /*
  843                          * We are inside of a SACK recovery episode and are
  844                          * sending new data, having retransmitted all the
  845                          * data possible in the scoreboard.
  846                          */
  847                         if (tp->snd_wnd < so->so_snd.sb_cc) {
  848                                 len = tp->snd_wnd - off;
  849                                 flags &= ~TH_FIN;
  850                         } else {
  851                                 len = so->so_snd.sb_cc - off;
  852                         }
  853 
  854                         /*
  855                          * From FreeBSD:
  856                          *  Don't remove this (len > 0) check !
  857                          *  We explicitly check for len > 0 here (although it 
  858                          *  isn't really necessary), to work around a gcc 
  859                          *  optimization issue - to force gcc to compute
  860                          *  len above. Without this check, the computation
  861                          *  of len is bungled by the optimizer.
  862                          */
  863                         if (len > 0) {
  864                                 cwin = tp->snd_cwnd - 
  865                                     (tp->snd_nxt - tp->sack_newdata) -
  866                                     sack_bytes_rxmt;
  867                                 if (cwin < 0)
  868                                         cwin = 0;
  869                                 if (cwin < len) {
  870                                         len = cwin;
  871                                         flags &= ~TH_FIN;
  872                                 }
  873                         }
  874                 } else if (win < so->so_snd.sb_cc) {
  875                         len = win - off;
  876                         flags &= ~TH_FIN;
  877                 } else {
  878                         len = so->so_snd.sb_cc - off;
  879                 }
  880         }
  881 
  882         if (len < 0) {
  883                 /*
  884                  * If FIN has been sent but not acked,
  885                  * but we haven't been called to retransmit,
  886                  * len will be -1.  Otherwise, window shrank
  887                  * after we sent into it.  If window shrank to 0,
  888                  * cancel pending retransmit, pull snd_nxt back
  889                  * to (closed) window, and set the persist timer
  890                  * if it isn't already going.  If the window didn't
  891                  * close completely, just wait for an ACK.
  892                  *
  893                  * If we have a pending FIN, either it has already been
  894                  * transmitted or it is outside the window, so drop it.
  895                  * If the FIN has been transmitted, but this is not a
  896                  * retransmission, then len must be -1.  Therefore we also
  897                  * prevent here the sending of `gratuitous FINs'.  This
  898                  * eliminates the need to check for that case below (e.g.
  899                  * to back up snd_nxt before the FIN so that the sequence
  900                  * number is correct).
  901                  */
  902                 len = 0;
  903                 flags &= ~TH_FIN;
  904                 if (win == 0) {
  905                         TCP_TIMER_DISARM(tp, TCPT_REXMT);
  906                         tp->t_rxtshift = 0;
  907                         tp->snd_nxt = tp->snd_una;
  908                         if (TCP_TIMER_ISARMED(tp, TCPT_PERSIST) == 0)
  909                                 tcp_setpersist(tp);
  910                 }
  911         }
  912 
  913         /*
  914          * Automatic sizing enables the performance of large buffers
  915          * and most of the efficiency of small ones by only allocating
  916          * space when it is needed.
  917          *
  918          * The criteria to step up the send buffer one notch are:
  919          *  1. receive window of remote host is larger than send buffer
  920          *     (with a fudge factor of 5/4th);
  921          *  2. send buffer is filled to 7/8th with data (so we actually
  922          *     have data to make use of it);
  923          *  3. send buffer fill has not hit maximal automatic size;
  924          *  4. our send window (slow start and cogestion controlled) is
  925          *     larger than sent but unacknowledged data in send buffer.
  926          *
  927          * The remote host receive window scaling factor may limit the
  928          * growing of the send buffer before it reaches its allowed
  929          * maximum.
  930          *
  931          * It scales directly with slow start or congestion window
  932          * and does at most one step per received ACK.  This fast
  933          * scaling has the drawback of growing the send buffer beyond
  934          * what is strictly necessary to make full use of a given
  935          * delay*bandwith product.  However testing has shown this not
  936          * to be much of an problem.  At worst we are trading wasting
  937          * of available bandwith (the non-use of it) for wasting some
  938          * socket buffer memory.
  939          *
  940          * TODO: Shrink send buffer during idle periods together
  941          * with congestion window.  Requires another timer.
  942          */
  943         if (tcp_do_autosndbuf && so->so_snd.sb_flags & SB_AUTOSIZE) {
  944                 if ((tp->snd_wnd / 4 * 5) >= so->so_snd.sb_hiwat &&
  945                     so->so_snd.sb_cc >= (so->so_snd.sb_hiwat / 8 * 7) &&
  946                     so->so_snd.sb_cc < tcp_autosndbuf_max &&
  947                     win >= (so->so_snd.sb_cc - (tp->snd_nxt - tp->snd_una))) {
  948                         if (!sbreserve(&so->so_snd,
  949                             min(so->so_snd.sb_hiwat + tcp_autosndbuf_inc,
  950                              tcp_autosndbuf_max), so))
  951                                 so->so_snd.sb_flags &= ~SB_AUTOSIZE;
  952                 }
  953         }
  954 
  955         if (len > txsegsize) {
  956                 if (use_tso) {
  957                         /*
  958                          * Truncate TSO transfers to IP_MAXPACKET, and make
  959                          * sure that we send equal size transfers down the
  960                          * stack (rather than big-small-big-small-...).
  961                          */
  962 #ifdef INET6
  963 #if IPV6_MAXPACKET != IP_MAXPACKET
  964 #error IPV6_MAXPACKET != IP_MAXPACKET
  965 #endif
  966 #endif
  967                         len = (min(len, IP_MAXPACKET) / txsegsize) * txsegsize;
  968                         if (len <= txsegsize) {
  969                                 use_tso = 0;
  970                         }
  971                 } else
  972                         len = txsegsize;
  973                 flags &= ~TH_FIN;
  974                 sendalot = 1;
  975         } else
  976                 use_tso = 0;
  977         if (sack_rxmit) {
  978                 if (SEQ_LT(p->rxmit + len, tp->snd_una + so->so_snd.sb_cc))
  979                         flags &= ~TH_FIN;
  980         }
  981 
  982         win = sbspace(&so->so_rcv);
  983 
  984         /*
  985          * Sender silly window avoidance.  If connection is idle
  986          * and can send all data, a maximum segment,
  987          * at least a maximum default-size segment do it,
  988          * or are forced, do it; otherwise don't bother.
  989          * If peer's buffer is tiny, then send
  990          * when window is at least half open.
  991          * If retransmitting (possibly after persist timer forced us
  992          * to send into a small window), then must resend.
  993          */
  994         if (len) {
  995                 if (len >= txsegsize)
  996                         goto send;
  997                 if ((so->so_state & SS_MORETOCOME) == 0 &&
  998                     ((idle || tp->t_flags & TF_NODELAY) &&
  999                      len + off >= so->so_snd.sb_cc))
 1000                         goto send;
 1001                 if (tp->t_force)
 1002                         goto send;
 1003                 if (len >= tp->max_sndwnd / 2)
 1004                         goto send;
 1005                 if (SEQ_LT(tp->snd_nxt, tp->snd_max))
 1006                         goto send;
 1007                 if (sack_rxmit)
 1008                         goto send;
 1009         }
 1010 
 1011         /*
 1012          * Compare available window to amount of window known to peer
 1013          * (as advertised window less next expected input).  If the
 1014          * difference is at least twice the size of the largest segment
 1015          * we expect to receive (i.e. two segments) or at least 50% of
 1016          * the maximum possible window, then want to send a window update
 1017          * to peer.
 1018          */
 1019         if (win > 0) {
 1020                 /*
 1021                  * "adv" is the amount we can increase the window,
 1022                  * taking into account that we are limited by
 1023                  * TCP_MAXWIN << tp->rcv_scale.
 1024                  */
 1025                 long adv = min(win, (long)TCP_MAXWIN << tp->rcv_scale) -
 1026                         (tp->rcv_adv - tp->rcv_nxt);
 1027 
 1028                 if (adv >= (long) (2 * rxsegsize))
 1029                         goto send;
 1030                 if (2 * adv >= (long) so->so_rcv.sb_hiwat)
 1031                         goto send;
 1032         }
 1033 
 1034         /*
 1035          * Send if we owe peer an ACK.
 1036          */
 1037         if (tp->t_flags & TF_ACKNOW)
 1038                 goto send;
 1039         if (flags & (TH_SYN|TH_FIN|TH_RST))
 1040                 goto send;
 1041         if (SEQ_GT(tp->snd_up, tp->snd_una))
 1042                 goto send;
 1043         /*
 1044          * In SACK, it is possible for tcp_output to fail to send a segment
 1045          * after the retransmission timer has been turned off.  Make sure
 1046          * that the retransmission timer is set.
 1047          */
 1048         if (TCP_SACK_ENABLED(tp) && SEQ_GT(tp->snd_max, tp->snd_una) &&
 1049             !TCP_TIMER_ISARMED(tp, TCPT_REXMT) &&
 1050             !TCP_TIMER_ISARMED(tp, TCPT_PERSIST)) {
 1051                 TCP_TIMER_ARM(tp, TCPT_REXMT, tp->t_rxtcur);
 1052                 goto just_return;
 1053         }
 1054 
 1055         /*
 1056          * TCP window updates are not reliable, rather a polling protocol
 1057          * using ``persist'' packets is used to insure receipt of window
 1058          * updates.  The three ``states'' for the output side are:
 1059          *      idle                    not doing retransmits or persists
 1060          *      persisting              to move a small or zero window
 1061          *      (re)transmitting        and thereby not persisting
 1062          *
 1063          * tp->t_timer[TCPT_PERSIST]
 1064          *      is set when we are in persist state.
 1065          * tp->t_force
 1066          *      is set when we are called to send a persist packet.
 1067          * tp->t_timer[TCPT_REXMT]
 1068          *      is set when we are retransmitting
 1069          * The output side is idle when both timers are zero.
 1070          *
 1071          * If send window is too small, there is data to transmit, and no
 1072          * retransmit or persist is pending, then go to persist state.
 1073          * If nothing happens soon, send when timer expires:
 1074          * if window is nonzero, transmit what we can,
 1075          * otherwise force out a byte.
 1076          */
 1077         if (so->so_snd.sb_cc && TCP_TIMER_ISARMED(tp, TCPT_REXMT) == 0 &&
 1078             TCP_TIMER_ISARMED(tp, TCPT_PERSIST) == 0) {
 1079                 tp->t_rxtshift = 0;
 1080                 tcp_setpersist(tp);
 1081         }
 1082 
 1083         /*
 1084          * No reason to send a segment, just return.
 1085          */
 1086 just_return:
 1087         TCP_REASS_UNLOCK(tp);
 1088         return (0);
 1089 
 1090 send:
 1091         /*
 1092          * Before ESTABLISHED, force sending of initial options
 1093          * unless TCP set not to do any options.
 1094          * NOTE: we assume that the IP/TCP header plus TCP options
 1095          * always fit in a single mbuf, leaving room for a maximum
 1096          * link header, i.e.
 1097          *      max_linkhdr + sizeof (struct tcpiphdr) + optlen <= MCLBYTES
 1098          */
 1099         optlen = 0;
 1100         switch (af) {
 1101 #ifdef INET
 1102         case AF_INET:
 1103                 iphdrlen = sizeof(struct ip) + sizeof(struct tcphdr);
 1104                 break;
 1105 #endif
 1106 #ifdef INET6
 1107         case AF_INET6:
 1108                 iphdrlen = sizeof(struct ip6_hdr) + sizeof(struct tcphdr);
 1109                 break;
 1110 #endif
 1111         default:        /*pacify gcc*/
 1112                 iphdrlen = 0;
 1113                 break;
 1114         }
 1115         hdrlen = iphdrlen;
 1116         if (flags & TH_SYN) {
 1117                 struct rtentry *synrt;
 1118 
 1119                 synrt = NULL;
 1120 #ifdef INET
 1121                 if (tp->t_inpcb)
 1122                         synrt = in_pcbrtentry(tp->t_inpcb);
 1123 #endif
 1124 #ifdef INET6
 1125                 if (tp->t_in6pcb)
 1126                         synrt = in6_pcbrtentry(tp->t_in6pcb);
 1127 #endif
 1128 
 1129                 tp->snd_nxt = tp->iss;
 1130                 tp->t_ourmss = tcp_mss_to_advertise(synrt != NULL ?
 1131                                                     synrt->rt_ifp : NULL, af);
 1132                 if ((tp->t_flags & TF_NOOPT) == 0) {
 1133                         opt[0] = TCPOPT_MAXSEG;
 1134                         opt[1] = 4;
 1135                         opt[2] = (tp->t_ourmss >> 8) & 0xff;
 1136                         opt[3] = tp->t_ourmss & 0xff;
 1137                         optlen = 4;
 1138 
 1139                         if ((tp->t_flags & TF_REQ_SCALE) &&
 1140                             ((flags & TH_ACK) == 0 ||
 1141                             (tp->t_flags & TF_RCVD_SCALE))) {
 1142                                 *((u_int32_t *) (opt + optlen)) = htonl(
 1143                                         TCPOPT_NOP << 24 |
 1144                                         TCPOPT_WINDOW << 16 |
 1145                                         TCPOLEN_WINDOW << 8 |
 1146                                         tp->request_r_scale);
 1147                                 optlen += 4;
 1148                         }
 1149                         if (tcp_do_sack) {
 1150                                 u_int8_t *cp = (u_int8_t *)(opt + optlen);
 1151 
 1152                                 cp[0] = TCPOPT_SACK_PERMITTED;
 1153                                 cp[1] = 2;
 1154                                 cp[2] = TCPOPT_NOP;
 1155                                 cp[3] = TCPOPT_NOP;
 1156                                 optlen += 4;
 1157                         }
 1158                 }
 1159         }
 1160 
 1161         /*
 1162          * Send a timestamp and echo-reply if this is a SYN and our side
 1163          * wants to use timestamps (TF_REQ_TSTMP is set) or both our side
 1164          * and our peer have sent timestamps in our SYN's.
 1165          */
 1166         if ((tp->t_flags & (TF_REQ_TSTMP|TF_NOOPT)) == TF_REQ_TSTMP &&
 1167              (flags & TH_RST) == 0 &&
 1168             ((flags & (TH_SYN|TH_ACK)) == TH_SYN ||
 1169              (tp->t_flags & TF_RCVD_TSTMP))) {
 1170                 u_int32_t *lp = (u_int32_t *)(opt + optlen);
 1171 
 1172                 /* Form timestamp option as shown in appendix A of RFC 1323. */
 1173                 *lp++ = htonl(TCPOPT_TSTAMP_HDR);
 1174                 *lp++ = htonl(TCP_TIMESTAMP(tp));
 1175                 *lp   = htonl(tp->ts_recent);
 1176                 optlen += TCPOLEN_TSTAMP_APPA;
 1177 
 1178                 /* Set receive buffer autosizing timestamp. */
 1179                 if (tp->rfbuf_ts == 0 && (so->so_rcv.sb_flags & SB_AUTOSIZE))
 1180                         tp->rfbuf_ts = TCP_TIMESTAMP(tp);
 1181         }
 1182 
 1183         /*
 1184          * Tack on the SACK block if it is necessary.
 1185          */
 1186         if (sack_numblks) {
 1187                 int sack_len;
 1188                 u_char *bp = (u_char *)(opt + optlen);
 1189                 u_int32_t *lp = (u_int32_t *)(bp + 4);
 1190                 struct ipqent *tiqe;
 1191 
 1192                 sack_len = sack_numblks * 8 + 2;
 1193                 bp[0] = TCPOPT_NOP;
 1194                 bp[1] = TCPOPT_NOP;
 1195                 bp[2] = TCPOPT_SACK;
 1196                 bp[3] = sack_len;
 1197                 if ((tp->rcv_sack_flags & TCPSACK_HAVED) != 0) {
 1198                         sack_numblks--;
 1199                         *lp++ = htonl(tp->rcv_dsack_block.left);
 1200                         *lp++ = htonl(tp->rcv_dsack_block.right);
 1201                         tp->rcv_sack_flags &= ~TCPSACK_HAVED;
 1202                 }
 1203                 for (tiqe = TAILQ_FIRST(&tp->timeq);
 1204                     sack_numblks > 0; tiqe = TAILQ_NEXT(tiqe, ipqe_timeq)) {
 1205                         KASSERT(tiqe != NULL);
 1206                         sack_numblks--;
 1207                         *lp++ = htonl(tiqe->ipqe_seq);
 1208                         *lp++ = htonl(tiqe->ipqe_seq + tiqe->ipqe_len +
 1209                             ((tiqe->ipqe_flags & TH_FIN) != 0 ? 1 : 0));
 1210                 }
 1211                 optlen += sack_len + 2;
 1212         }
 1213         TCP_REASS_UNLOCK(tp);
 1214 
 1215 #ifdef TCP_SIGNATURE
 1216         if (tp->t_flags & TF_SIGNATURE) {
 1217                 u_char *bp;
 1218                 /*
 1219                  * Initialize TCP-MD5 option (RFC2385)
 1220                  */
 1221                 bp = (u_char *)opt + optlen;
 1222                 *bp++ = TCPOPT_SIGNATURE;
 1223                 *bp++ = TCPOLEN_SIGNATURE;
 1224                 sigoff = optlen + 2;
 1225                 bzero(bp, TCP_SIGLEN);
 1226                 bp += TCP_SIGLEN;
 1227                 optlen += TCPOLEN_SIGNATURE;
 1228                 /*
 1229                  * Terminate options list and maintain 32-bit alignment.
 1230                  */
 1231                 *bp++ = TCPOPT_NOP;
 1232                 *bp++ = TCPOPT_EOL;
 1233                 optlen += 2;
 1234         }
 1235 #endif /* TCP_SIGNATURE */
 1236 
 1237         hdrlen += optlen;
 1238 
 1239 #ifdef DIAGNOSTIC
 1240         if (!use_tso && len > txsegsize)
 1241                 panic("tcp data to be sent is larger than segment");
 1242         else if (use_tso && len > IP_MAXPACKET)
 1243                 panic("tcp data to be sent is larger than max TSO size");
 1244         if (max_linkhdr + hdrlen > MCLBYTES)
 1245                 panic("tcphdr too big");
 1246 #endif
 1247 
 1248         /*
 1249          * Grab a header mbuf, attaching a copy of data to
 1250          * be transmitted, and initialize the header from
 1251          * the template for sends on this connection.
 1252          */
 1253         if (len) {
 1254                 error = tcp_build_datapkt(tp, so, off, len, hdrlen, &m);
 1255                 if (error)
 1256                         goto out;
 1257                 /*
 1258                  * If we're sending everything we've got, set PUSH.
 1259                  * (This will keep happy those implementations which only
 1260                  * give data to the user when a buffer fills or
 1261                  * a PUSH comes in.)
 1262                  */
 1263                 if (off + len == so->so_snd.sb_cc)
 1264                         flags |= TH_PUSH;
 1265         } else {
 1266                 tcps = TCP_STAT_GETREF();
 1267                 if (tp->t_flags & TF_ACKNOW)
 1268                         tcps[TCP_STAT_SNDACKS]++;
 1269                 else if (flags & (TH_SYN|TH_FIN|TH_RST))
 1270                         tcps[TCP_STAT_SNDCTRL]++;
 1271                 else if (SEQ_GT(tp->snd_up, tp->snd_una))
 1272                         tcps[TCP_STAT_SNDURG]++;
 1273                 else
 1274                         tcps[TCP_STAT_SNDWINUP]++;
 1275                 TCP_STAT_PUTREF();
 1276 
 1277                 MGETHDR(m, M_DONTWAIT, MT_HEADER);
 1278                 if (m != NULL && max_linkhdr + hdrlen > MHLEN) {
 1279                         MCLGET(m, M_DONTWAIT);
 1280                         if ((m->m_flags & M_EXT) == 0) {
 1281                                 m_freem(m);
 1282                                 m = NULL;
 1283                         }
 1284                 }
 1285                 if (m == NULL) {
 1286                         error = ENOBUFS;
 1287                         goto out;
 1288                 }
 1289                 MCLAIM(m, &tcp_tx_mowner);
 1290                 m->m_data += max_linkhdr;
 1291                 m->m_len = hdrlen;
 1292         }
 1293         m->m_pkthdr.rcvif = (struct ifnet *)0;
 1294         switch (af) {
 1295 #ifdef INET
 1296         case AF_INET:
 1297                 ip = mtod(m, struct ip *);
 1298 #ifdef INET6
 1299                 ip6 = NULL;
 1300 #endif
 1301                 th = (struct tcphdr *)(ip + 1);
 1302                 break;
 1303 #endif
 1304 #ifdef INET6
 1305         case AF_INET6:
 1306                 ip = NULL;
 1307                 ip6 = mtod(m, struct ip6_hdr *);
 1308                 th = (struct tcphdr *)(ip6 + 1);
 1309                 break;
 1310 #endif
 1311         default:        /*pacify gcc*/
 1312                 ip = NULL;
 1313 #ifdef INET6
 1314                 ip6 = NULL;
 1315 #endif
 1316                 th = NULL;
 1317                 break;
 1318         }
 1319         if (tp->t_template == 0)
 1320                 panic("tcp_output");
 1321         if (tp->t_template->m_len < iphdrlen)
 1322                 panic("tcp_output");
 1323         bcopy(mtod(tp->t_template, void *), mtod(m, void *), iphdrlen);
 1324 
 1325         /*
 1326          * If we are starting a connection, send ECN setup
 1327          * SYN packet. If we are on a retransmit, we may
 1328          * resend those bits a number of times as per
 1329          * RFC 3168.
 1330          */
 1331         if (tp->t_state == TCPS_SYN_SENT && tcp_do_ecn) {
 1332                 if (tp->t_flags & TF_SYN_REXMT) {
 1333                         if (tp->t_ecn_retries--)
 1334                                 flags |= TH_ECE|TH_CWR;
 1335                 } else {
 1336                         flags |= TH_ECE|TH_CWR;
 1337                         tp->t_ecn_retries = tcp_ecn_maxretries;
 1338                 }
 1339         }
 1340 
 1341         if (TCP_ECN_ALLOWED(tp)) {
 1342                 /*
 1343                  * If the peer has ECN, mark data packets
 1344                  * ECN capable. Ignore pure ack packets, retransmissions
 1345                  * and window probes.
 1346                  */
 1347                 if (len > 0 && SEQ_GEQ(tp->snd_nxt, tp->snd_max) &&
 1348                     !(tp->t_force && len == 1)) {
 1349                         ecn_tos = IPTOS_ECN_ECT0;
 1350                         TCP_STATINC(TCP_STAT_ECN_ECT);
 1351                 }
 1352 
 1353                 /*
 1354                  * Reply with proper ECN notifications.
 1355                  */
 1356                 if (tp->t_flags & TF_ECN_SND_CWR) {
 1357                         flags |= TH_CWR;
 1358                         tp->t_flags &= ~TF_ECN_SND_CWR;
 1359                 } 
 1360                 if (tp->t_flags & TF_ECN_SND_ECE) {
 1361                         flags |= TH_ECE;
 1362                 }
 1363         }
 1364 
 1365 
 1366         /*
 1367          * If we are doing retransmissions, then snd_nxt will
 1368          * not reflect the first unsent octet.  For ACK only
 1369          * packets, we do not want the sequence number of the
 1370          * retransmitted packet, we want the sequence number
 1371          * of the next unsent octet.  So, if there is no data
 1372          * (and no SYN or FIN), use snd_max instead of snd_nxt
 1373          * when filling in ti_seq.  But if we are in persist
 1374          * state, snd_max might reflect one byte beyond the
 1375          * right edge of the window, so use snd_nxt in that
 1376          * case, since we know we aren't doing a retransmission.
 1377          * (retransmit and persist are mutually exclusive...)
 1378          */
 1379         if (TCP_SACK_ENABLED(tp) && sack_rxmit) {
 1380                 th->th_seq = htonl(p->rxmit);
 1381                 p->rxmit += len;
 1382         } else {
 1383                 if (len || (flags & (TH_SYN|TH_FIN)) ||
 1384                     TCP_TIMER_ISARMED(tp, TCPT_PERSIST))
 1385                         th->th_seq = htonl(tp->snd_nxt);
 1386                 else
 1387                         th->th_seq = htonl(tp->snd_max);
 1388         }
 1389         th->th_ack = htonl(tp->rcv_nxt);
 1390         if (optlen) {
 1391                 bcopy((void *)opt, (void *)(th + 1), optlen);
 1392                 th->th_off = (sizeof (struct tcphdr) + optlen) >> 2;
 1393         }
 1394         th->th_flags = flags;
 1395         /*
 1396          * Calculate receive window.  Don't shrink window,
 1397          * but avoid silly window syndrome.
 1398          */
 1399         if (win < (long)(so->so_rcv.sb_hiwat / 4) && win < (long)rxsegsize)
 1400                 win = 0;
 1401         if (win > (long)TCP_MAXWIN << tp->rcv_scale)
 1402                 win = (long)TCP_MAXWIN << tp->rcv_scale;
 1403         if (win < (long)(int32_t)(tp->rcv_adv - tp->rcv_nxt))
 1404                 win = (long)(int32_t)(tp->rcv_adv - tp->rcv_nxt);
 1405         th->th_win = htons((u_int16_t) (win>>tp->rcv_scale));
 1406         if (SEQ_GT(tp->snd_up, tp->snd_nxt)) {
 1407                 u_int32_t urp = tp->snd_up - tp->snd_nxt;
 1408                 if (urp > IP_MAXPACKET)
 1409                         urp = IP_MAXPACKET;
 1410                 th->th_urp = htons((u_int16_t)urp);
 1411                 th->th_flags |= TH_URG;
 1412         } else
 1413                 /*
 1414                  * If no urgent pointer to send, then we pull
 1415                  * the urgent pointer to the left edge of the send window
 1416                  * so that it doesn't drift into the send window on sequence
 1417                  * number wraparound.
 1418                  */
 1419                 tp->snd_up = tp->snd_una;               /* drag it along */
 1420 
 1421 #ifdef TCP_SIGNATURE
 1422         if (sigoff && (tp->t_flags & TF_SIGNATURE)) {
 1423                 struct secasvar *sav;
 1424                 u_int8_t *sigp;
 1425 
 1426                 sav = tcp_signature_getsav(m, th);
 1427 
 1428                 if (sav == NULL) {
 1429                         if (m)
 1430                                 m_freem(m);
 1431                         return (EPERM);
 1432                 }
 1433 
 1434                 m->m_pkthdr.len = hdrlen + len;
 1435                 sigp = (char *)th + sizeof(*th) + sigoff;
 1436                 tcp_signature(m, th, (char *)th - mtod(m, char *), sav, sigp);
 1437 
 1438                 key_sa_recordxfer(sav, m);
 1439 #ifdef FAST_IPSEC
 1440                 KEY_FREESAV(&sav);
 1441 #else
 1442                 key_freesav(sav);
 1443 #endif
 1444         }
 1445 #endif
 1446 
 1447         /*
 1448          * Set ourselves up to be checksummed just before the packet
 1449          * hits the wire.
 1450          */
 1451         switch (af) {
 1452 #ifdef INET
 1453         case AF_INET:
 1454                 m->m_pkthdr.csum_data = offsetof(struct tcphdr, th_sum);
 1455                 if (use_tso) {
 1456                         m->m_pkthdr.segsz = txsegsize;
 1457                         m->m_pkthdr.csum_flags = M_CSUM_TSOv4;
 1458                 } else {
 1459                         m->m_pkthdr.csum_flags = M_CSUM_TCPv4;
 1460                         if (len + optlen) {
 1461                                 /* Fixup the pseudo-header checksum. */
 1462                                 /* XXXJRT Not IP Jumbogram safe. */
 1463                                 th->th_sum = in_cksum_addword(th->th_sum,
 1464                                     htons((u_int16_t) (len + optlen)));
 1465                         }
 1466                 }
 1467                 break;
 1468 #endif
 1469 #ifdef INET6
 1470         case AF_INET6:
 1471                 m->m_pkthdr.csum_data = offsetof(struct tcphdr, th_sum);
 1472                 if (use_tso) {
 1473                         m->m_pkthdr.segsz = txsegsize;
 1474                         m->m_pkthdr.csum_flags = M_CSUM_TSOv6;
 1475                 } else {
 1476                         m->m_pkthdr.csum_flags = M_CSUM_TCPv6;
 1477                         if (len + optlen) {
 1478                                 /* Fixup the pseudo-header checksum. */
 1479                                 /* XXXJRT: Not IPv6 Jumbogram safe. */
 1480                                 th->th_sum = in_cksum_addword(th->th_sum,
 1481                                     htons((u_int16_t) (len + optlen)));
 1482                         }
 1483                 }
 1484                 break;
 1485 #endif
 1486         }
 1487 
 1488         /*
 1489          * In transmit state, time the transmission and arrange for
 1490          * the retransmit.  In persist state, just set snd_max.
 1491          */
 1492         if (tp->t_force == 0 || TCP_TIMER_ISARMED(tp, TCPT_PERSIST) == 0) {
 1493                 tcp_seq startseq = tp->snd_nxt;
 1494 
 1495                 /*
 1496                  * Advance snd_nxt over sequence space of this segment.
 1497                  * There are no states in which we send both a SYN and a FIN,
 1498                  * so we collapse the tests for these flags.
 1499                  */
 1500                 if (flags & (TH_SYN|TH_FIN))
 1501                         tp->snd_nxt++;
 1502                 if (sack_rxmit)
 1503                         goto timer;
 1504                 tp->snd_nxt += len;
 1505                 if (SEQ_GT(tp->snd_nxt, tp->snd_max)) {
 1506                         tp->snd_max = tp->snd_nxt;
 1507                         /*
 1508                          * Time this transmission if not a retransmission and
 1509                          * not currently timing anything.
 1510                          */
 1511                         if (tp->t_rtttime == 0) {
 1512                                 tp->t_rtttime = tcp_now;
 1513                                 tp->t_rtseq = startseq;
 1514                                 TCP_STATINC(TCP_STAT_SEGSTIMED);
 1515                         }
 1516                 }
 1517 
 1518                 /*
 1519                  * Set retransmit timer if not currently set,
 1520                  * and not doing an ack or a keep-alive probe.
 1521                  * Initial value for retransmit timer is smoothed
 1522                  * round-trip time + 2 * round-trip time variance.
 1523                  * Initialize shift counter which is used for backoff
 1524                  * of retransmit time.
 1525                  */
 1526 timer:
 1527                 if (TCP_TIMER_ISARMED(tp, TCPT_REXMT) == 0 &&
 1528                         ((sack_rxmit && tp->snd_nxt != tp->snd_max) ||
 1529                     tp->snd_nxt != tp->snd_una)) {
 1530                         if (TCP_TIMER_ISARMED(tp, TCPT_PERSIST)) {
 1531                                 TCP_TIMER_DISARM(tp, TCPT_PERSIST);
 1532                                 tp->t_rxtshift = 0;
 1533                         }
 1534                         TCP_TIMER_ARM(tp, TCPT_REXMT, tp->t_rxtcur);
 1535                 }
 1536         } else
 1537                 if (SEQ_GT(tp->snd_nxt + len, tp->snd_max))
 1538                         tp->snd_max = tp->snd_nxt + len;
 1539 
 1540 #ifdef TCP_DEBUG
 1541         /*
 1542          * Trace.
 1543          */
 1544         if (so->so_options & SO_DEBUG)
 1545                 tcp_trace(TA_OUTPUT, tp->t_state, tp, m, 0);
 1546 #endif
 1547 
 1548         /*
 1549          * Fill in IP length and desired time to live and
 1550          * send to IP level.  There should be a better way
 1551          * to handle ttl and tos; we could keep them in
 1552          * the template, but need a way to checksum without them.
 1553          */
 1554         m->m_pkthdr.len = hdrlen + len;
 1555 
 1556         switch (af) {
 1557 #ifdef INET
 1558         case AF_INET:
 1559                 ip->ip_len = htons(m->m_pkthdr.len);
 1560                 packetlen = m->m_pkthdr.len;
 1561                 if (tp->t_inpcb) {
 1562                         ip->ip_ttl = tp->t_inpcb->inp_ip.ip_ttl;
 1563                         ip->ip_tos = tp->t_inpcb->inp_ip.ip_tos | ecn_tos;
 1564                 }
 1565 #ifdef INET6
 1566                 else if (tp->t_in6pcb) {
 1567                         ip->ip_ttl = in6_selecthlim(tp->t_in6pcb, NULL); /*XXX*/
 1568                         ip->ip_tos = ecn_tos;   /*XXX*/
 1569                 }
 1570 #endif
 1571                 break;
 1572 #endif
 1573 #ifdef INET6
 1574         case AF_INET6:
 1575                 packetlen = m->m_pkthdr.len;
 1576                 ip6->ip6_nxt = IPPROTO_TCP;
 1577                 if (tp->t_in6pcb) {
 1578                         /*
 1579                          * we separately set hoplimit for every segment, since
 1580                          * the user might want to change the value via
 1581                          * setsockopt. Also, desired default hop limit might
 1582                          * be changed via Neighbor Discovery.
 1583                          */
 1584                         ip6->ip6_hlim = in6_selecthlim(tp->t_in6pcb,
 1585                                 (rt = rtcache_validate(ro)) != NULL ? rt->rt_ifp
 1586                                                                     : NULL);
 1587                 }
 1588                 ip6->ip6_flow |= htonl(ecn_tos << 20);
 1589                 /* ip6->ip6_flow = ??? (from template) */
 1590                 /* ip6_plen will be filled in ip6_output(). */
 1591                 break;
 1592 #endif
 1593         default:        /*pacify gcc*/
 1594                 packetlen = 0;
 1595                 break;
 1596         }
 1597 
 1598         switch (af) {
 1599 #ifdef INET
 1600         case AF_INET:
 1601             {
 1602                 struct mbuf *opts;
 1603 
 1604                 if (tp->t_inpcb)
 1605                         opts = tp->t_inpcb->inp_options;
 1606                 else
 1607                         opts = NULL;
 1608                 error = ip_output(m, opts, ro,
 1609                         (tp->t_mtudisc ? IP_MTUDISC : 0) |
 1610                         (so->so_options & SO_DONTROUTE),
 1611                         (struct ip_moptions *)0, so);
 1612                 break;
 1613             }
 1614 #endif
 1615 #ifdef INET6
 1616         case AF_INET6:
 1617             {
 1618                 struct ip6_pktopts *opts;
 1619 
 1620                 if (tp->t_in6pcb)
 1621                         opts = tp->t_in6pcb->in6p_outputopts;
 1622                 else
 1623                         opts = NULL;
 1624                 error = ip6_output(m, opts, ro, so->so_options & SO_DONTROUTE,
 1625                         NULL, so, NULL);
 1626                 break;
 1627             }
 1628 #endif
 1629         default:
 1630                 error = EAFNOSUPPORT;
 1631                 break;
 1632         }
 1633         if (error) {
 1634 out:
 1635                 if (error == ENOBUFS) {
 1636                         TCP_STATINC(TCP_STAT_SELFQUENCH);
 1637 #ifdef INET
 1638                         if (tp->t_inpcb)
 1639                                 tcp_quench(tp->t_inpcb, 0);
 1640 #endif
 1641 #ifdef INET6
 1642                         if (tp->t_in6pcb)
 1643                                 tcp6_quench(tp->t_in6pcb, 0);
 1644 #endif
 1645                         error = 0;
 1646                 } else if ((error == EHOSTUNREACH || error == ENETDOWN) &&
 1647                     TCPS_HAVERCVDSYN(tp->t_state)) {
 1648                         tp->t_softerror = error;
 1649                         error = 0;
 1650                 }
 1651 
 1652                 /* Back out the seqence number advance. */
 1653                 if (sack_rxmit)
 1654                         p->rxmit -= len;
 1655 
 1656                 /* Restart the delayed ACK timer, if necessary. */
 1657                 if (tp->t_flags & TF_DELACK)
 1658                         TCP_RESTART_DELACK(tp);
 1659 
 1660                 return (error);
 1661         }
 1662 
 1663         if (packetlen > tp->t_pmtud_mtu_sent)
 1664                 tp->t_pmtud_mtu_sent = packetlen;
 1665         
 1666         tcps = TCP_STAT_GETREF();
 1667         tcps[TCP_STAT_SNDTOTAL]++;
 1668         if (tp->t_flags & TF_DELACK)
 1669                 tcps[TCP_STAT_DELACK]++;
 1670         TCP_STAT_PUTREF();
 1671 
 1672         /*
 1673          * Data sent (as far as we can tell).
 1674          * If this advertises a larger window than any other segment,
 1675          * then remember the size of the advertised window.
 1676          * Any pending ACK has now been sent.
 1677          */
 1678         if (win > 0 && SEQ_GT(tp->rcv_nxt+win, tp->rcv_adv))
 1679                 tp->rcv_adv = tp->rcv_nxt + win;
 1680         tp->last_ack_sent = tp->rcv_nxt;
 1681         tp->t_flags &= ~TF_ACKNOW;
 1682         TCP_CLEAR_DELACK(tp);
 1683 #ifdef DIAGNOSTIC
 1684         if (maxburst < 0)
 1685                 printf("tcp_output: maxburst exceeded by %d\n", -maxburst);
 1686 #endif
 1687         if (sendalot && (tp->t_congctl == &tcp_reno_ctl || --maxburst))
 1688                 goto again;
 1689         return (0);
 1690 }
 1691 
 1692 void
 1693 tcp_setpersist(struct tcpcb *tp)
 1694 {
 1695         int t = ((tp->t_srtt >> 2) + tp->t_rttvar) >> (1 + 2);
 1696         int nticks;
 1697 
 1698         if (TCP_TIMER_ISARMED(tp, TCPT_REXMT))
 1699                 panic("tcp_output REXMT");
 1700         /*
 1701          * Start/restart persistance timer.
 1702          */
 1703         if (t < tp->t_rttmin)
 1704                 t = tp->t_rttmin;
 1705         TCPT_RANGESET(nticks, t * tcp_backoff[tp->t_rxtshift],
 1706             TCPTV_PERSMIN, TCPTV_PERSMAX);
 1707         TCP_TIMER_ARM(tp, TCPT_PERSIST, nticks);
 1708         if (tp->t_rxtshift < TCP_MAXRXTSHIFT)
 1709                 tp->t_rxtshift++;
 1710 }

Cache object: b47619426b281d6846035ccd71019968


[ source navigation ] [ diff markup ] [ identifier search ] [ freetext search ] [ file search ] [ list types ] [ track identifier ]


This page is part of the FreeBSD/Linux Linux Kernel Cross-Reference, and was automatically generated using a modified version of the LXR engine.