The Design and Implementation of the FreeBSD Operating System, Second Edition
Now available: The Design and Implementation of the FreeBSD Operating System (Second Edition)


[ source navigation ] [ diff markup ] [ identifier search ] [ freetext search ] [ file search ] [ list types ] [ track identifier ]

FreeBSD/Linux Kernel Cross Reference
sys/kern/uipc_socket.c

Version: -  FREEBSD  -  FREEBSD-13-STABLE  -  FREEBSD-13-0  -  FREEBSD-12-STABLE  -  FREEBSD-12-0  -  FREEBSD-11-STABLE  -  FREEBSD-11-0  -  FREEBSD-10-STABLE  -  FREEBSD-10-0  -  FREEBSD-9-STABLE  -  FREEBSD-9-0  -  FREEBSD-8-STABLE  -  FREEBSD-8-0  -  FREEBSD-7-STABLE  -  FREEBSD-7-0  -  FREEBSD-6-STABLE  -  FREEBSD-6-0  -  FREEBSD-5-STABLE  -  FREEBSD-5-0  -  FREEBSD-4-STABLE  -  FREEBSD-3-STABLE  -  FREEBSD22  -  l41  -  OPENBSD  -  linux-2.6  -  MK84  -  PLAN9  -  xnu-8792 
SearchContext: -  none  -  3  -  10 

    1 /*-
    2  * Copyright (c) 1982, 1986, 1988, 1990, 1993
    3  *      The Regents of the University of California.
    4  * Copyright (c) 2004 The FreeBSD Foundation
    5  * Copyright (c) 2004-2008 Robert N. M. Watson
    6  * All rights reserved.
    7  *
    8  * Redistribution and use in source and binary forms, with or without
    9  * modification, are permitted provided that the following conditions
   10  * are met:
   11  * 1. Redistributions of source code must retain the above copyright
   12  *    notice, this list of conditions and the following disclaimer.
   13  * 2. Redistributions in binary form must reproduce the above copyright
   14  *    notice, this list of conditions and the following disclaimer in the
   15  *    documentation and/or other materials provided with the distribution.
   16  * 4. Neither the name of the University nor the names of its contributors
   17  *    may be used to endorse or promote products derived from this software
   18  *    without specific prior written permission.
   19  *
   20  * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
   21  * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
   22  * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
   23  * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
   24  * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
   25  * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
   26  * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
   27  * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
   28  * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
   29  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
   30  * SUCH DAMAGE.
   31  *
   32  *      @(#)uipc_socket.c       8.3 (Berkeley) 4/15/94
   33  */
   34 
   35 /*
   36  * Comments on the socket life cycle:
   37  *
   38  * soalloc() sets of socket layer state for a socket, called only by
   39  * socreate() and sonewconn().  Socket layer private.
   40  *
   41  * sodealloc() tears down socket layer state for a socket, called only by
   42  * sofree() and sonewconn().  Socket layer private.
   43  *
   44  * pru_attach() associates protocol layer state with an allocated socket;
   45  * called only once, may fail, aborting socket allocation.  This is called
   46  * from socreate() and sonewconn().  Socket layer private.
   47  *
   48  * pru_detach() disassociates protocol layer state from an attached socket,
   49  * and will be called exactly once for sockets in which pru_attach() has
   50  * been successfully called.  If pru_attach() returned an error,
   51  * pru_detach() will not be called.  Socket layer private.
   52  *
   53  * pru_abort() and pru_close() notify the protocol layer that the last
   54  * consumer of a socket is starting to tear down the socket, and that the
   55  * protocol should terminate the connection.  Historically, pru_abort() also
   56  * detached protocol state from the socket state, but this is no longer the
   57  * case.
   58  *
   59  * socreate() creates a socket and attaches protocol state.  This is a public
   60  * interface that may be used by socket layer consumers to create new
   61  * sockets.
   62  *
   63  * sonewconn() creates a socket and attaches protocol state.  This is a
   64  * public interface  that may be used by protocols to create new sockets when
   65  * a new connection is received and will be available for accept() on a
   66  * listen socket.
   67  *
   68  * soclose() destroys a socket after possibly waiting for it to disconnect.
   69  * This is a public interface that socket consumers should use to close and
   70  * release a socket when done with it.
   71  *
   72  * soabort() destroys a socket without waiting for it to disconnect (used
   73  * only for incoming connections that are already partially or fully
   74  * connected).  This is used internally by the socket layer when clearing
   75  * listen socket queues (due to overflow or close on the listen socket), but
   76  * is also a public interface protocols may use to abort connections in
   77  * their incomplete listen queues should they no longer be required.  Sockets
   78  * placed in completed connection listen queues should not be aborted for
   79  * reasons described in the comment above the soclose() implementation.  This
   80  * is not a general purpose close routine, and except in the specific
   81  * circumstances described here, should not be used.
   82  *
   83  * sofree() will free a socket and its protocol state if all references on
   84  * the socket have been released, and is the public interface to attempt to
   85  * free a socket when a reference is removed.  This is a socket layer private
   86  * interface.
   87  *
   88  * NOTE: In addition to socreate() and soclose(), which provide a single
   89  * socket reference to the consumer to be managed as required, there are two
   90  * calls to explicitly manage socket references, soref(), and sorele().
   91  * Currently, these are generally required only when transitioning a socket
   92  * from a listen queue to a file descriptor, in order to prevent garbage
   93  * collection of the socket at an untimely moment.  For a number of reasons,
   94  * these interfaces are not preferred, and should be avoided.
   95  *
   96  * NOTE: With regard to VNETs the general rule is that callers do not set
   97  * curvnet. Exceptions to this rule include soabort(), sodisconnect(),
   98  * sofree() (and with that sorele(), sotryfree()), as well as sonewconn()
   99  * and sorflush(), which are usually called from a pre-set VNET context.
  100  * sopoll() currently does not need a VNET context to be set.
  101  */
  102 
  103 #include <sys/cdefs.h>
  104 __FBSDID("$FreeBSD: releng/10.2/sys/kern/uipc_socket.c 279516 2015-03-02 07:51:14Z ae $");
  105 
  106 #include "opt_inet.h"
  107 #include "opt_inet6.h"
  108 #include "opt_compat.h"
  109 
  110 #include <sys/param.h>
  111 #include <sys/systm.h>
  112 #include <sys/fcntl.h>
  113 #include <sys/limits.h>
  114 #include <sys/lock.h>
  115 #include <sys/mac.h>
  116 #include <sys/malloc.h>
  117 #include <sys/mbuf.h>
  118 #include <sys/mutex.h>
  119 #include <sys/domain.h>
  120 #include <sys/file.h>                   /* for struct knote */
  121 #include <sys/kernel.h>
  122 #include <sys/event.h>
  123 #include <sys/eventhandler.h>
  124 #include <sys/poll.h>
  125 #include <sys/proc.h>
  126 #include <sys/protosw.h>
  127 #include <sys/socket.h>
  128 #include <sys/socketvar.h>
  129 #include <sys/resourcevar.h>
  130 #include <net/route.h>
  131 #include <sys/signalvar.h>
  132 #include <sys/stat.h>
  133 #include <sys/sx.h>
  134 #include <sys/sysctl.h>
  135 #include <sys/uio.h>
  136 #include <sys/jail.h>
  137 #include <sys/syslog.h>
  138 #include <netinet/in.h>
  139 
  140 #include <net/vnet.h>
  141 
  142 #include <security/mac/mac_framework.h>
  143 
  144 #include <vm/uma.h>
  145 
  146 #ifdef COMPAT_FREEBSD32
  147 #include <sys/mount.h>
  148 #include <sys/sysent.h>
  149 #include <compat/freebsd32/freebsd32.h>
  150 #endif
  151 
  152 static int      soreceive_rcvoob(struct socket *so, struct uio *uio,
  153                     int flags);
  154 
  155 static void     filt_sordetach(struct knote *kn);
  156 static int      filt_soread(struct knote *kn, long hint);
  157 static void     filt_sowdetach(struct knote *kn);
  158 static int      filt_sowrite(struct knote *kn, long hint);
  159 static int      filt_solisten(struct knote *kn, long hint);
  160 
  161 static struct filterops solisten_filtops = {
  162         .f_isfd = 1,
  163         .f_detach = filt_sordetach,
  164         .f_event = filt_solisten,
  165 };
  166 static struct filterops soread_filtops = {
  167         .f_isfd = 1,
  168         .f_detach = filt_sordetach,
  169         .f_event = filt_soread,
  170 };
  171 static struct filterops sowrite_filtops = {
  172         .f_isfd = 1,
  173         .f_detach = filt_sowdetach,
  174         .f_event = filt_sowrite,
  175 };
  176 
  177 so_gen_t        so_gencnt;      /* generation count for sockets */
  178 
  179 MALLOC_DEFINE(M_SONAME, "soname", "socket name");
  180 MALLOC_DEFINE(M_PCB, "pcb", "protocol control block");
  181 
  182 #define VNET_SO_ASSERT(so)                                              \
  183         VNET_ASSERT(curvnet != NULL,                                    \
  184             ("%s:%d curvnet is NULL, so=%p", __func__, __LINE__, (so)));
  185 
  186 /*
  187  * Limit on the number of connections in the listen queue waiting
  188  * for accept(2).
  189  * NB: The orginal sysctl somaxconn is still available but hidden
  190  * to prevent confusion about the actual purpose of this number.
  191  */
  192 static int somaxconn = SOMAXCONN;
  193 
  194 static int
  195 sysctl_somaxconn(SYSCTL_HANDLER_ARGS)
  196 {
  197         int error;
  198         int val;
  199 
  200         val = somaxconn;
  201         error = sysctl_handle_int(oidp, &val, 0, req);
  202         if (error || !req->newptr )
  203                 return (error);
  204 
  205         if (val < 1 || val > USHRT_MAX)
  206                 return (EINVAL);
  207 
  208         somaxconn = val;
  209         return (0);
  210 }
  211 SYSCTL_PROC(_kern_ipc, OID_AUTO, soacceptqueue, CTLTYPE_UINT | CTLFLAG_RW,
  212     0, sizeof(int), sysctl_somaxconn, "I",
  213     "Maximum listen socket pending connection accept queue size");
  214 SYSCTL_PROC(_kern_ipc, KIPC_SOMAXCONN, somaxconn,
  215     CTLTYPE_UINT | CTLFLAG_RW | CTLFLAG_SKIP,
  216     0, sizeof(int), sysctl_somaxconn, "I",
  217     "Maximum listen socket pending connection accept queue size (compat)");
  218 
  219 static int numopensockets;
  220 SYSCTL_INT(_kern_ipc, OID_AUTO, numopensockets, CTLFLAG_RD,
  221     &numopensockets, 0, "Number of open sockets");
  222 
  223 /*
  224  * accept_mtx locks down per-socket fields relating to accept queues.  See
  225  * socketvar.h for an annotation of the protected fields of struct socket.
  226  */
  227 struct mtx accept_mtx;
  228 MTX_SYSINIT(accept_mtx, &accept_mtx, "accept", MTX_DEF);
  229 
  230 /*
  231  * so_global_mtx protects so_gencnt, numopensockets, and the per-socket
  232  * so_gencnt field.
  233  */
  234 static struct mtx so_global_mtx;
  235 MTX_SYSINIT(so_global_mtx, &so_global_mtx, "so_glabel", MTX_DEF);
  236 
  237 /*
  238  * General IPC sysctl name space, used by sockets and a variety of other IPC
  239  * types.
  240  */
  241 SYSCTL_NODE(_kern, KERN_IPC, ipc, CTLFLAG_RW, 0, "IPC");
  242 
  243 /*
  244  * Initialize the socket subsystem and set up the socket
  245  * memory allocator.
  246  */
  247 static uma_zone_t socket_zone;
  248 int     maxsockets;
  249 
  250 static void
  251 socket_zone_change(void *tag)
  252 {
  253 
  254         maxsockets = uma_zone_set_max(socket_zone, maxsockets);
  255 }
  256 
  257 static void
  258 socket_init(void *tag)
  259 {
  260 
  261         socket_zone = uma_zcreate("socket", sizeof(struct socket), NULL, NULL,
  262             NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_NOFREE);
  263         maxsockets = uma_zone_set_max(socket_zone, maxsockets);
  264         uma_zone_set_warning(socket_zone, "kern.ipc.maxsockets limit reached");
  265         EVENTHANDLER_REGISTER(maxsockets_change, socket_zone_change, NULL,
  266             EVENTHANDLER_PRI_FIRST);
  267 }
  268 SYSINIT(socket, SI_SUB_PROTO_DOMAININIT, SI_ORDER_ANY, socket_init, NULL);
  269 
  270 /*
  271  * Initialise maxsockets.  This SYSINIT must be run after
  272  * tunable_mbinit().
  273  */
  274 static void
  275 init_maxsockets(void *ignored)
  276 {
  277 
  278         TUNABLE_INT_FETCH("kern.ipc.maxsockets", &maxsockets);
  279         maxsockets = imax(maxsockets, maxfiles);
  280 }
  281 SYSINIT(param, SI_SUB_TUNABLES, SI_ORDER_ANY, init_maxsockets, NULL);
  282 
  283 /*
  284  * Sysctl to get and set the maximum global sockets limit.  Notify protocols
  285  * of the change so that they can update their dependent limits as required.
  286  */
  287 static int
  288 sysctl_maxsockets(SYSCTL_HANDLER_ARGS)
  289 {
  290         int error, newmaxsockets;
  291 
  292         newmaxsockets = maxsockets;
  293         error = sysctl_handle_int(oidp, &newmaxsockets, 0, req);
  294         if (error == 0 && req->newptr) {
  295                 if (newmaxsockets > maxsockets &&
  296                     newmaxsockets <= maxfiles) {
  297                         maxsockets = newmaxsockets;
  298                         EVENTHANDLER_INVOKE(maxsockets_change);
  299                 } else
  300                         error = EINVAL;
  301         }
  302         return (error);
  303 }
  304 SYSCTL_PROC(_kern_ipc, OID_AUTO, maxsockets, CTLTYPE_INT|CTLFLAG_RW,
  305     &maxsockets, 0, sysctl_maxsockets, "IU",
  306     "Maximum number of sockets avaliable");
  307 
  308 /*
  309  * Socket operation routines.  These routines are called by the routines in
  310  * sys_socket.c or from a system process, and implement the semantics of
  311  * socket operations by switching out to the protocol specific routines.
  312  */
  313 
  314 /*
  315  * Get a socket structure from our zone, and initialize it.  Note that it
  316  * would probably be better to allocate socket and PCB at the same time, but
  317  * I'm not convinced that all the protocols can be easily modified to do
  318  * this.
  319  *
  320  * soalloc() returns a socket with a ref count of 0.
  321  */
  322 static struct socket *
  323 soalloc(struct vnet *vnet)
  324 {
  325         struct socket *so;
  326 
  327         so = uma_zalloc(socket_zone, M_NOWAIT | M_ZERO);
  328         if (so == NULL)
  329                 return (NULL);
  330 #ifdef MAC
  331         if (mac_socket_init(so, M_NOWAIT) != 0) {
  332                 uma_zfree(socket_zone, so);
  333                 return (NULL);
  334         }
  335 #endif
  336         SOCKBUF_LOCK_INIT(&so->so_snd, "so_snd");
  337         SOCKBUF_LOCK_INIT(&so->so_rcv, "so_rcv");
  338         sx_init(&so->so_snd.sb_sx, "so_snd_sx");
  339         sx_init(&so->so_rcv.sb_sx, "so_rcv_sx");
  340         TAILQ_INIT(&so->so_aiojobq);
  341         mtx_lock(&so_global_mtx);
  342         so->so_gencnt = ++so_gencnt;
  343         ++numopensockets;
  344 #ifdef VIMAGE
  345         VNET_ASSERT(vnet != NULL, ("%s:%d vnet is NULL, so=%p",
  346             __func__, __LINE__, so));
  347         vnet->vnet_sockcnt++;
  348         so->so_vnet = vnet;
  349 #endif
  350         mtx_unlock(&so_global_mtx);
  351         return (so);
  352 }
  353 
  354 /*
  355  * Free the storage associated with a socket at the socket layer, tear down
  356  * locks, labels, etc.  All protocol state is assumed already to have been
  357  * torn down (and possibly never set up) by the caller.
  358  */
  359 static void
  360 sodealloc(struct socket *so)
  361 {
  362 
  363         KASSERT(so->so_count == 0, ("sodealloc(): so_count %d", so->so_count));
  364         KASSERT(so->so_pcb == NULL, ("sodealloc(): so_pcb != NULL"));
  365 
  366         mtx_lock(&so_global_mtx);
  367         so->so_gencnt = ++so_gencnt;
  368         --numopensockets;       /* Could be below, but faster here. */
  369 #ifdef VIMAGE
  370         VNET_ASSERT(so->so_vnet != NULL, ("%s:%d so_vnet is NULL, so=%p",
  371             __func__, __LINE__, so));
  372         so->so_vnet->vnet_sockcnt--;
  373 #endif
  374         mtx_unlock(&so_global_mtx);
  375         if (so->so_rcv.sb_hiwat)
  376                 (void)chgsbsize(so->so_cred->cr_uidinfo,
  377                     &so->so_rcv.sb_hiwat, 0, RLIM_INFINITY);
  378         if (so->so_snd.sb_hiwat)
  379                 (void)chgsbsize(so->so_cred->cr_uidinfo,
  380                     &so->so_snd.sb_hiwat, 0, RLIM_INFINITY);
  381 #ifdef INET
  382         /* remove acccept filter if one is present. */
  383         if (so->so_accf != NULL)
  384                 do_setopt_accept_filter(so, NULL);
  385 #endif
  386 #ifdef MAC
  387         mac_socket_destroy(so);
  388 #endif
  389         crfree(so->so_cred);
  390         sx_destroy(&so->so_snd.sb_sx);
  391         sx_destroy(&so->so_rcv.sb_sx);
  392         SOCKBUF_LOCK_DESTROY(&so->so_snd);
  393         SOCKBUF_LOCK_DESTROY(&so->so_rcv);
  394         uma_zfree(socket_zone, so);
  395 }
  396 
  397 /*
  398  * socreate returns a socket with a ref count of 1.  The socket should be
  399  * closed with soclose().
  400  */
  401 int
  402 socreate(int dom, struct socket **aso, int type, int proto,
  403     struct ucred *cred, struct thread *td)
  404 {
  405         struct protosw *prp;
  406         struct socket *so;
  407         int error;
  408 
  409         if (proto)
  410                 prp = pffindproto(dom, proto, type);
  411         else
  412                 prp = pffindtype(dom, type);
  413 
  414         if (prp == NULL) {
  415                 /* No support for domain. */
  416                 if (pffinddomain(dom) == NULL)
  417                         return (EAFNOSUPPORT);
  418                 /* No support for socket type. */
  419                 if (proto == 0 && type != 0)
  420                         return (EPROTOTYPE);
  421                 return (EPROTONOSUPPORT);
  422         }
  423         if (prp->pr_usrreqs->pru_attach == NULL ||
  424             prp->pr_usrreqs->pru_attach == pru_attach_notsupp)
  425                 return (EPROTONOSUPPORT);
  426 
  427         if (prison_check_af(cred, prp->pr_domain->dom_family) != 0)
  428                 return (EPROTONOSUPPORT);
  429 
  430         if (prp->pr_type != type)
  431                 return (EPROTOTYPE);
  432         so = soalloc(CRED_TO_VNET(cred));
  433         if (so == NULL)
  434                 return (ENOBUFS);
  435 
  436         TAILQ_INIT(&so->so_incomp);
  437         TAILQ_INIT(&so->so_comp);
  438         so->so_type = type;
  439         so->so_cred = crhold(cred);
  440         if ((prp->pr_domain->dom_family == PF_INET) ||
  441             (prp->pr_domain->dom_family == PF_INET6) ||
  442             (prp->pr_domain->dom_family == PF_ROUTE))
  443                 so->so_fibnum = td->td_proc->p_fibnum;
  444         else
  445                 so->so_fibnum = 0;
  446         so->so_proto = prp;
  447 #ifdef MAC
  448         mac_socket_create(cred, so);
  449 #endif
  450         knlist_init_mtx(&so->so_rcv.sb_sel.si_note, SOCKBUF_MTX(&so->so_rcv));
  451         knlist_init_mtx(&so->so_snd.sb_sel.si_note, SOCKBUF_MTX(&so->so_snd));
  452         so->so_count = 1;
  453         /*
  454          * Auto-sizing of socket buffers is managed by the protocols and
  455          * the appropriate flags must be set in the pru_attach function.
  456          */
  457         CURVNET_SET(so->so_vnet);
  458         error = (*prp->pr_usrreqs->pru_attach)(so, proto, td);
  459         CURVNET_RESTORE();
  460         if (error) {
  461                 KASSERT(so->so_count == 1, ("socreate: so_count %d",
  462                     so->so_count));
  463                 so->so_count = 0;
  464                 sodealloc(so);
  465                 return (error);
  466         }
  467         *aso = so;
  468         return (0);
  469 }
  470 
  471 #ifdef REGRESSION
  472 static int regression_sonewconn_earlytest = 1;
  473 SYSCTL_INT(_regression, OID_AUTO, sonewconn_earlytest, CTLFLAG_RW,
  474     &regression_sonewconn_earlytest, 0, "Perform early sonewconn limit test");
  475 #endif
  476 
  477 /*
  478  * When an attempt at a new connection is noted on a socket which accepts
  479  * connections, sonewconn is called.  If the connection is possible (subject
  480  * to space constraints, etc.) then we allocate a new structure, propoerly
  481  * linked into the data structure of the original socket, and return this.
  482  * Connstatus may be 0, or SO_ISCONFIRMING, or SO_ISCONNECTED.
  483  *
  484  * Note: the ref count on the socket is 0 on return.
  485  */
  486 struct socket *
  487 sonewconn(struct socket *head, int connstatus)
  488 {
  489         static struct timeval lastover;
  490         static struct timeval overinterval = { 60, 0 };
  491         static int overcount;
  492 
  493         struct socket *so;
  494         int over;
  495 
  496         ACCEPT_LOCK();
  497         over = (head->so_qlen > 3 * head->so_qlimit / 2);
  498         ACCEPT_UNLOCK();
  499 #ifdef REGRESSION
  500         if (regression_sonewconn_earlytest && over) {
  501 #else
  502         if (over) {
  503 #endif
  504                 overcount++;
  505 
  506                 if (ratecheck(&lastover, &overinterval)) {
  507                         log(LOG_DEBUG, "%s: pcb %p: Listen queue overflow: "
  508                             "%i already in queue awaiting acceptance "
  509                             "(%d occurrences)\n",
  510                             __func__, head->so_pcb, head->so_qlen, overcount);
  511 
  512                         overcount = 0;
  513                 }
  514 
  515                 return (NULL);
  516         }
  517         VNET_ASSERT(head->so_vnet != NULL, ("%s:%d so_vnet is NULL, head=%p",
  518             __func__, __LINE__, head));
  519         so = soalloc(head->so_vnet);
  520         if (so == NULL) {
  521                 log(LOG_DEBUG, "%s: pcb %p: New socket allocation failure: "
  522                     "limit reached or out of memory\n",
  523                     __func__, head->so_pcb);
  524                 return (NULL);
  525         }
  526         if ((head->so_options & SO_ACCEPTFILTER) != 0)
  527                 connstatus = 0;
  528         so->so_head = head;
  529         so->so_type = head->so_type;
  530         so->so_options = head->so_options &~ SO_ACCEPTCONN;
  531         so->so_linger = head->so_linger;
  532         so->so_state = head->so_state | SS_NOFDREF;
  533         so->so_fibnum = head->so_fibnum;
  534         so->so_proto = head->so_proto;
  535         so->so_cred = crhold(head->so_cred);
  536 #ifdef MAC
  537         mac_socket_newconn(head, so);
  538 #endif
  539         knlist_init_mtx(&so->so_rcv.sb_sel.si_note, SOCKBUF_MTX(&so->so_rcv));
  540         knlist_init_mtx(&so->so_snd.sb_sel.si_note, SOCKBUF_MTX(&so->so_snd));
  541         VNET_SO_ASSERT(head);
  542         if (soreserve(so, head->so_snd.sb_hiwat, head->so_rcv.sb_hiwat)) {
  543                 sodealloc(so);
  544                 log(LOG_DEBUG, "%s: pcb %p: soreserve() failed\n",
  545                     __func__, head->so_pcb);
  546                 return (NULL);
  547         }
  548         if ((*so->so_proto->pr_usrreqs->pru_attach)(so, 0, NULL)) {
  549                 sodealloc(so);
  550                 log(LOG_DEBUG, "%s: pcb %p: pru_attach() failed\n",
  551                     __func__, head->so_pcb);
  552                 return (NULL);
  553         }
  554         so->so_rcv.sb_lowat = head->so_rcv.sb_lowat;
  555         so->so_snd.sb_lowat = head->so_snd.sb_lowat;
  556         so->so_rcv.sb_timeo = head->so_rcv.sb_timeo;
  557         so->so_snd.sb_timeo = head->so_snd.sb_timeo;
  558         so->so_rcv.sb_flags |= head->so_rcv.sb_flags & SB_AUTOSIZE;
  559         so->so_snd.sb_flags |= head->so_snd.sb_flags & SB_AUTOSIZE;
  560         so->so_state |= connstatus;
  561         ACCEPT_LOCK();
  562         /*
  563          * The accept socket may be tearing down but we just
  564          * won a race on the ACCEPT_LOCK.
  565          * However, if sctp_peeloff() is called on a 1-to-many
  566          * style socket, the SO_ACCEPTCONN doesn't need to be set.
  567          */
  568         if (!(head->so_options & SO_ACCEPTCONN) &&
  569             ((head->so_proto->pr_protocol != IPPROTO_SCTP) ||
  570              (head->so_type != SOCK_SEQPACKET))) {
  571                 SOCK_LOCK(so);
  572                 so->so_head = NULL;
  573                 sofree(so);             /* NB: returns ACCEPT_UNLOCK'ed. */
  574                 return (NULL);
  575         }
  576         if (connstatus) {
  577                 TAILQ_INSERT_TAIL(&head->so_comp, so, so_list);
  578                 so->so_qstate |= SQ_COMP;
  579                 head->so_qlen++;
  580         } else {
  581                 /*
  582                  * Keep removing sockets from the head until there's room for
  583                  * us to insert on the tail.  In pre-locking revisions, this
  584                  * was a simple if(), but as we could be racing with other
  585                  * threads and soabort() requires dropping locks, we must
  586                  * loop waiting for the condition to be true.
  587                  */
  588                 while (head->so_incqlen > head->so_qlimit) {
  589                         struct socket *sp;
  590                         sp = TAILQ_FIRST(&head->so_incomp);
  591                         TAILQ_REMOVE(&head->so_incomp, sp, so_list);
  592                         head->so_incqlen--;
  593                         sp->so_qstate &= ~SQ_INCOMP;
  594                         sp->so_head = NULL;
  595                         ACCEPT_UNLOCK();
  596                         soabort(sp);
  597                         ACCEPT_LOCK();
  598                 }
  599                 TAILQ_INSERT_TAIL(&head->so_incomp, so, so_list);
  600                 so->so_qstate |= SQ_INCOMP;
  601                 head->so_incqlen++;
  602         }
  603         ACCEPT_UNLOCK();
  604         if (connstatus) {
  605                 sorwakeup(head);
  606                 wakeup_one(&head->so_timeo);
  607         }
  608         return (so);
  609 }
  610 
  611 int
  612 sobind(struct socket *so, struct sockaddr *nam, struct thread *td)
  613 {
  614         int error;
  615 
  616         CURVNET_SET(so->so_vnet);
  617         error = (*so->so_proto->pr_usrreqs->pru_bind)(so, nam, td);
  618         CURVNET_RESTORE();
  619         return (error);
  620 }
  621 
  622 int
  623 sobindat(int fd, struct socket *so, struct sockaddr *nam, struct thread *td)
  624 {
  625         int error;
  626 
  627         CURVNET_SET(so->so_vnet);
  628         error = (*so->so_proto->pr_usrreqs->pru_bindat)(fd, so, nam, td);
  629         CURVNET_RESTORE();
  630         return (error);
  631 }
  632 
  633 /*
  634  * solisten() transitions a socket from a non-listening state to a listening
  635  * state, but can also be used to update the listen queue depth on an
  636  * existing listen socket.  The protocol will call back into the sockets
  637  * layer using solisten_proto_check() and solisten_proto() to check and set
  638  * socket-layer listen state.  Call backs are used so that the protocol can
  639  * acquire both protocol and socket layer locks in whatever order is required
  640  * by the protocol.
  641  *
  642  * Protocol implementors are advised to hold the socket lock across the
  643  * socket-layer test and set to avoid races at the socket layer.
  644  */
  645 int
  646 solisten(struct socket *so, int backlog, struct thread *td)
  647 {
  648         int error;
  649 
  650         CURVNET_SET(so->so_vnet);
  651         error = (*so->so_proto->pr_usrreqs->pru_listen)(so, backlog, td);
  652         CURVNET_RESTORE();
  653         return (error);
  654 }
  655 
  656 int
  657 solisten_proto_check(struct socket *so)
  658 {
  659 
  660         SOCK_LOCK_ASSERT(so);
  661 
  662         if (so->so_state & (SS_ISCONNECTED | SS_ISCONNECTING |
  663             SS_ISDISCONNECTING))
  664                 return (EINVAL);
  665         return (0);
  666 }
  667 
  668 void
  669 solisten_proto(struct socket *so, int backlog)
  670 {
  671 
  672         SOCK_LOCK_ASSERT(so);
  673 
  674         if (backlog < 0 || backlog > somaxconn)
  675                 backlog = somaxconn;
  676         so->so_qlimit = backlog;
  677         so->so_options |= SO_ACCEPTCONN;
  678 }
  679 
  680 /*
  681  * Evaluate the reference count and named references on a socket; if no
  682  * references remain, free it.  This should be called whenever a reference is
  683  * released, such as in sorele(), but also when named reference flags are
  684  * cleared in socket or protocol code.
  685  *
  686  * sofree() will free the socket if:
  687  *
  688  * - There are no outstanding file descriptor references or related consumers
  689  *   (so_count == 0).
  690  *
  691  * - The socket has been closed by user space, if ever open (SS_NOFDREF).
  692  *
  693  * - The protocol does not have an outstanding strong reference on the socket
  694  *   (SS_PROTOREF).
  695  *
  696  * - The socket is not in a completed connection queue, so a process has been
  697  *   notified that it is present.  If it is removed, the user process may
  698  *   block in accept() despite select() saying the socket was ready.
  699  */
  700 void
  701 sofree(struct socket *so)
  702 {
  703         struct protosw *pr = so->so_proto;
  704         struct socket *head;
  705 
  706         ACCEPT_LOCK_ASSERT();
  707         SOCK_LOCK_ASSERT(so);
  708 
  709         if ((so->so_state & SS_NOFDREF) == 0 || so->so_count != 0 ||
  710             (so->so_state & SS_PROTOREF) || (so->so_qstate & SQ_COMP)) {
  711                 SOCK_UNLOCK(so);
  712                 ACCEPT_UNLOCK();
  713                 return;
  714         }
  715 
  716         head = so->so_head;
  717         if (head != NULL) {
  718                 KASSERT((so->so_qstate & SQ_COMP) != 0 ||
  719                     (so->so_qstate & SQ_INCOMP) != 0,
  720                     ("sofree: so_head != NULL, but neither SQ_COMP nor "
  721                     "SQ_INCOMP"));
  722                 KASSERT((so->so_qstate & SQ_COMP) == 0 ||
  723                     (so->so_qstate & SQ_INCOMP) == 0,
  724                     ("sofree: so->so_qstate is SQ_COMP and also SQ_INCOMP"));
  725                 TAILQ_REMOVE(&head->so_incomp, so, so_list);
  726                 head->so_incqlen--;
  727                 so->so_qstate &= ~SQ_INCOMP;
  728                 so->so_head = NULL;
  729         }
  730         KASSERT((so->so_qstate & SQ_COMP) == 0 &&
  731             (so->so_qstate & SQ_INCOMP) == 0,
  732             ("sofree: so_head == NULL, but still SQ_COMP(%d) or SQ_INCOMP(%d)",
  733             so->so_qstate & SQ_COMP, so->so_qstate & SQ_INCOMP));
  734         if (so->so_options & SO_ACCEPTCONN) {
  735                 KASSERT((TAILQ_EMPTY(&so->so_comp)),
  736                     ("sofree: so_comp populated"));
  737                 KASSERT((TAILQ_EMPTY(&so->so_incomp)),
  738                     ("sofree: so_incomp populated"));
  739         }
  740         SOCK_UNLOCK(so);
  741         ACCEPT_UNLOCK();
  742 
  743         VNET_SO_ASSERT(so);
  744         if (pr->pr_flags & PR_RIGHTS && pr->pr_domain->dom_dispose != NULL)
  745                 (*pr->pr_domain->dom_dispose)(so->so_rcv.sb_mb);
  746         if (pr->pr_usrreqs->pru_detach != NULL)
  747                 (*pr->pr_usrreqs->pru_detach)(so);
  748 
  749         /*
  750          * From this point on, we assume that no other references to this
  751          * socket exist anywhere else in the stack.  Therefore, no locks need
  752          * to be acquired or held.
  753          *
  754          * We used to do a lot of socket buffer and socket locking here, as
  755          * well as invoke sorflush() and perform wakeups.  The direct call to
  756          * dom_dispose() and sbrelease_internal() are an inlining of what was
  757          * necessary from sorflush().
  758          *
  759          * Notice that the socket buffer and kqueue state are torn down
  760          * before calling pru_detach.  This means that protocols shold not
  761          * assume they can perform socket wakeups, etc, in their detach code.
  762          */
  763         sbdestroy(&so->so_snd, so);
  764         sbdestroy(&so->so_rcv, so);
  765         seldrain(&so->so_snd.sb_sel);
  766         seldrain(&so->so_rcv.sb_sel);
  767         knlist_destroy(&so->so_rcv.sb_sel.si_note);
  768         knlist_destroy(&so->so_snd.sb_sel.si_note);
  769         sodealloc(so);
  770 }
  771 
  772 /*
  773  * Close a socket on last file table reference removal.  Initiate disconnect
  774  * if connected.  Free socket when disconnect complete.
  775  *
  776  * This function will sorele() the socket.  Note that soclose() may be called
  777  * prior to the ref count reaching zero.  The actual socket structure will
  778  * not be freed until the ref count reaches zero.
  779  */
  780 int
  781 soclose(struct socket *so)
  782 {
  783         int error = 0;
  784 
  785         KASSERT(!(so->so_state & SS_NOFDREF), ("soclose: SS_NOFDREF on enter"));
  786 
  787         CURVNET_SET(so->so_vnet);
  788         funsetown(&so->so_sigio);
  789         if (so->so_state & SS_ISCONNECTED) {
  790                 if ((so->so_state & SS_ISDISCONNECTING) == 0) {
  791                         error = sodisconnect(so);
  792                         if (error) {
  793                                 if (error == ENOTCONN)
  794                                         error = 0;
  795                                 goto drop;
  796                         }
  797                 }
  798                 if (so->so_options & SO_LINGER) {
  799                         if ((so->so_state & SS_ISDISCONNECTING) &&
  800                             (so->so_state & SS_NBIO))
  801                                 goto drop;
  802                         while (so->so_state & SS_ISCONNECTED) {
  803                                 error = tsleep(&so->so_timeo,
  804                                     PSOCK | PCATCH, "soclos",
  805                                     so->so_linger * hz);
  806                                 if (error)
  807                                         break;
  808                         }
  809                 }
  810         }
  811 
  812 drop:
  813         if (so->so_proto->pr_usrreqs->pru_close != NULL)
  814                 (*so->so_proto->pr_usrreqs->pru_close)(so);
  815         ACCEPT_LOCK();
  816         if (so->so_options & SO_ACCEPTCONN) {
  817                 struct socket *sp;
  818                 /*
  819                  * Prevent new additions to the accept queues due
  820                  * to ACCEPT_LOCK races while we are draining them.
  821                  */
  822                 so->so_options &= ~SO_ACCEPTCONN;
  823                 while ((sp = TAILQ_FIRST(&so->so_incomp)) != NULL) {
  824                         TAILQ_REMOVE(&so->so_incomp, sp, so_list);
  825                         so->so_incqlen--;
  826                         sp->so_qstate &= ~SQ_INCOMP;
  827                         sp->so_head = NULL;
  828                         ACCEPT_UNLOCK();
  829                         soabort(sp);
  830                         ACCEPT_LOCK();
  831                 }
  832                 while ((sp = TAILQ_FIRST(&so->so_comp)) != NULL) {
  833                         TAILQ_REMOVE(&so->so_comp, sp, so_list);
  834                         so->so_qlen--;
  835                         sp->so_qstate &= ~SQ_COMP;
  836                         sp->so_head = NULL;
  837                         ACCEPT_UNLOCK();
  838                         soabort(sp);
  839                         ACCEPT_LOCK();
  840                 }
  841                 KASSERT((TAILQ_EMPTY(&so->so_comp)),
  842                     ("%s: so_comp populated", __func__));
  843                 KASSERT((TAILQ_EMPTY(&so->so_incomp)),
  844                     ("%s: so_incomp populated", __func__));
  845         }
  846         SOCK_LOCK(so);
  847         KASSERT((so->so_state & SS_NOFDREF) == 0, ("soclose: NOFDREF"));
  848         so->so_state |= SS_NOFDREF;
  849         sorele(so);                     /* NB: Returns with ACCEPT_UNLOCK(). */
  850         CURVNET_RESTORE();
  851         return (error);
  852 }
  853 
  854 /*
  855  * soabort() is used to abruptly tear down a connection, such as when a
  856  * resource limit is reached (listen queue depth exceeded), or if a listen
  857  * socket is closed while there are sockets waiting to be accepted.
  858  *
  859  * This interface is tricky, because it is called on an unreferenced socket,
  860  * and must be called only by a thread that has actually removed the socket
  861  * from the listen queue it was on, or races with other threads are risked.
  862  *
  863  * This interface will call into the protocol code, so must not be called
  864  * with any socket locks held.  Protocols do call it while holding their own
  865  * recursible protocol mutexes, but this is something that should be subject
  866  * to review in the future.
  867  */
  868 void
  869 soabort(struct socket *so)
  870 {
  871 
  872         /*
  873          * In as much as is possible, assert that no references to this
  874          * socket are held.  This is not quite the same as asserting that the
  875          * current thread is responsible for arranging for no references, but
  876          * is as close as we can get for now.
  877          */
  878         KASSERT(so->so_count == 0, ("soabort: so_count"));
  879         KASSERT((so->so_state & SS_PROTOREF) == 0, ("soabort: SS_PROTOREF"));
  880         KASSERT(so->so_state & SS_NOFDREF, ("soabort: !SS_NOFDREF"));
  881         KASSERT((so->so_state & SQ_COMP) == 0, ("soabort: SQ_COMP"));
  882         KASSERT((so->so_state & SQ_INCOMP) == 0, ("soabort: SQ_INCOMP"));
  883         VNET_SO_ASSERT(so);
  884 
  885         if (so->so_proto->pr_usrreqs->pru_abort != NULL)
  886                 (*so->so_proto->pr_usrreqs->pru_abort)(so);
  887         ACCEPT_LOCK();
  888         SOCK_LOCK(so);
  889         sofree(so);
  890 }
  891 
  892 int
  893 soaccept(struct socket *so, struct sockaddr **nam)
  894 {
  895         int error;
  896 
  897         SOCK_LOCK(so);
  898         KASSERT((so->so_state & SS_NOFDREF) != 0, ("soaccept: !NOFDREF"));
  899         so->so_state &= ~SS_NOFDREF;
  900         SOCK_UNLOCK(so);
  901 
  902         CURVNET_SET(so->so_vnet);
  903         error = (*so->so_proto->pr_usrreqs->pru_accept)(so, nam);
  904         CURVNET_RESTORE();
  905         return (error);
  906 }
  907 
  908 int
  909 soconnect(struct socket *so, struct sockaddr *nam, struct thread *td)
  910 {
  911 
  912         return (soconnectat(AT_FDCWD, so, nam, td));
  913 }
  914 
  915 int
  916 soconnectat(int fd, struct socket *so, struct sockaddr *nam, struct thread *td)
  917 {
  918         int error;
  919 
  920         if (so->so_options & SO_ACCEPTCONN)
  921                 return (EOPNOTSUPP);
  922 
  923         CURVNET_SET(so->so_vnet);
  924         /*
  925          * If protocol is connection-based, can only connect once.
  926          * Otherwise, if connected, try to disconnect first.  This allows
  927          * user to disconnect by connecting to, e.g., a null address.
  928          */
  929         if (so->so_state & (SS_ISCONNECTED|SS_ISCONNECTING) &&
  930             ((so->so_proto->pr_flags & PR_CONNREQUIRED) ||
  931             (error = sodisconnect(so)))) {
  932                 error = EISCONN;
  933         } else {
  934                 /*
  935                  * Prevent accumulated error from previous connection from
  936                  * biting us.
  937                  */
  938                 so->so_error = 0;
  939                 if (fd == AT_FDCWD) {
  940                         error = (*so->so_proto->pr_usrreqs->pru_connect)(so,
  941                             nam, td);
  942                 } else {
  943                         error = (*so->so_proto->pr_usrreqs->pru_connectat)(fd,
  944                             so, nam, td);
  945                 }
  946         }
  947         CURVNET_RESTORE();
  948 
  949         return (error);
  950 }
  951 
  952 int
  953 soconnect2(struct socket *so1, struct socket *so2)
  954 {
  955         int error;
  956 
  957         CURVNET_SET(so1->so_vnet);
  958         error = (*so1->so_proto->pr_usrreqs->pru_connect2)(so1, so2);
  959         CURVNET_RESTORE();
  960         return (error);
  961 }
  962 
  963 int
  964 sodisconnect(struct socket *so)
  965 {
  966         int error;
  967 
  968         if ((so->so_state & SS_ISCONNECTED) == 0)
  969                 return (ENOTCONN);
  970         if (so->so_state & SS_ISDISCONNECTING)
  971                 return (EALREADY);
  972         VNET_SO_ASSERT(so);
  973         error = (*so->so_proto->pr_usrreqs->pru_disconnect)(so);
  974         return (error);
  975 }
  976 
  977 #define SBLOCKWAIT(f)   (((f) & MSG_DONTWAIT) ? 0 : SBL_WAIT)
  978 
  979 int
  980 sosend_dgram(struct socket *so, struct sockaddr *addr, struct uio *uio,
  981     struct mbuf *top, struct mbuf *control, int flags, struct thread *td)
  982 {
  983         long space;
  984         ssize_t resid;
  985         int clen = 0, error, dontroute;
  986 
  987         KASSERT(so->so_type == SOCK_DGRAM, ("sosend_dgram: !SOCK_DGRAM"));
  988         KASSERT(so->so_proto->pr_flags & PR_ATOMIC,
  989             ("sosend_dgram: !PR_ATOMIC"));
  990 
  991         if (uio != NULL)
  992                 resid = uio->uio_resid;
  993         else
  994                 resid = top->m_pkthdr.len;
  995         /*
  996          * In theory resid should be unsigned.  However, space must be
  997          * signed, as it might be less than 0 if we over-committed, and we
  998          * must use a signed comparison of space and resid.  On the other
  999          * hand, a negative resid causes us to loop sending 0-length
 1000          * segments to the protocol.
 1001          */
 1002         if (resid < 0) {
 1003                 error = EINVAL;
 1004                 goto out;
 1005         }
 1006 
 1007         dontroute =
 1008             (flags & MSG_DONTROUTE) && (so->so_options & SO_DONTROUTE) == 0;
 1009         if (td != NULL)
 1010                 td->td_ru.ru_msgsnd++;
 1011         if (control != NULL)
 1012                 clen = control->m_len;
 1013 
 1014         SOCKBUF_LOCK(&so->so_snd);
 1015         if (so->so_snd.sb_state & SBS_CANTSENDMORE) {
 1016                 SOCKBUF_UNLOCK(&so->so_snd);
 1017                 error = EPIPE;
 1018                 goto out;
 1019         }
 1020         if (so->so_error) {
 1021                 error = so->so_error;
 1022                 so->so_error = 0;
 1023                 SOCKBUF_UNLOCK(&so->so_snd);
 1024                 goto out;
 1025         }
 1026         if ((so->so_state & SS_ISCONNECTED) == 0) {
 1027                 /*
 1028                  * `sendto' and `sendmsg' is allowed on a connection-based
 1029                  * socket if it supports implied connect.  Return ENOTCONN if
 1030                  * not connected and no address is supplied.
 1031                  */
 1032                 if ((so->so_proto->pr_flags & PR_CONNREQUIRED) &&
 1033                     (so->so_proto->pr_flags & PR_IMPLOPCL) == 0) {
 1034                         if ((so->so_state & SS_ISCONFIRMING) == 0 &&
 1035                             !(resid == 0 && clen != 0)) {
 1036                                 SOCKBUF_UNLOCK(&so->so_snd);
 1037                                 error = ENOTCONN;
 1038                                 goto out;
 1039                         }
 1040                 } else if (addr == NULL) {
 1041                         if (so->so_proto->pr_flags & PR_CONNREQUIRED)
 1042                                 error = ENOTCONN;
 1043                         else
 1044                                 error = EDESTADDRREQ;
 1045                         SOCKBUF_UNLOCK(&so->so_snd);
 1046                         goto out;
 1047                 }
 1048         }
 1049 
 1050         /*
 1051          * Do we need MSG_OOB support in SOCK_DGRAM?  Signs here may be a
 1052          * problem and need fixing.
 1053          */
 1054         space = sbspace(&so->so_snd);
 1055         if (flags & MSG_OOB)
 1056                 space += 1024;
 1057         space -= clen;
 1058         SOCKBUF_UNLOCK(&so->so_snd);
 1059         if (resid > space) {
 1060                 error = EMSGSIZE;
 1061                 goto out;
 1062         }
 1063         if (uio == NULL) {
 1064                 resid = 0;
 1065                 if (flags & MSG_EOR)
 1066                         top->m_flags |= M_EOR;
 1067         } else {
 1068                 /*
 1069                  * Copy the data from userland into a mbuf chain.
 1070                  * If no data is to be copied in, a single empty mbuf
 1071                  * is returned.
 1072                  */
 1073                 top = m_uiotombuf(uio, M_WAITOK, space, max_hdr,
 1074                     (M_PKTHDR | ((flags & MSG_EOR) ? M_EOR : 0)));
 1075                 if (top == NULL) {
 1076                         error = EFAULT; /* only possible error */
 1077                         goto out;
 1078                 }
 1079                 space -= resid - uio->uio_resid;
 1080                 resid = uio->uio_resid;
 1081         }
 1082         KASSERT(resid == 0, ("sosend_dgram: resid != 0"));
 1083         /*
 1084          * XXXRW: Frobbing SO_DONTROUTE here is even worse without sblock
 1085          * than with.
 1086          */
 1087         if (dontroute) {
 1088                 SOCK_LOCK(so);
 1089                 so->so_options |= SO_DONTROUTE;
 1090                 SOCK_UNLOCK(so);
 1091         }
 1092         /*
 1093          * XXX all the SBS_CANTSENDMORE checks previously done could be out
 1094          * of date.  We could have recieved a reset packet in an interrupt or
 1095          * maybe we slept while doing page faults in uiomove() etc.  We could
 1096          * probably recheck again inside the locking protection here, but
 1097          * there are probably other places that this also happens.  We must
 1098          * rethink this.
 1099          */
 1100         VNET_SO_ASSERT(so);
 1101         error = (*so->so_proto->pr_usrreqs->pru_send)(so,
 1102             (flags & MSG_OOB) ? PRUS_OOB :
 1103         /*
 1104          * If the user set MSG_EOF, the protocol understands this flag and
 1105          * nothing left to send then use PRU_SEND_EOF instead of PRU_SEND.
 1106          */
 1107             ((flags & MSG_EOF) &&
 1108              (so->so_proto->pr_flags & PR_IMPLOPCL) &&
 1109              (resid <= 0)) ?
 1110                 PRUS_EOF :
 1111                 /* If there is more to send set PRUS_MORETOCOME */
 1112                 (resid > 0 && space > 0) ? PRUS_MORETOCOME : 0,
 1113                 top, addr, control, td);
 1114         if (dontroute) {
 1115                 SOCK_LOCK(so);
 1116                 so->so_options &= ~SO_DONTROUTE;
 1117                 SOCK_UNLOCK(so);
 1118         }
 1119         clen = 0;
 1120         control = NULL;
 1121         top = NULL;
 1122 out:
 1123         if (top != NULL)
 1124                 m_freem(top);
 1125         if (control != NULL)
 1126                 m_freem(control);
 1127         return (error);
 1128 }
 1129 
 1130 /*
 1131  * Send on a socket.  If send must go all at once and message is larger than
 1132  * send buffering, then hard error.  Lock against other senders.  If must go
 1133  * all at once and not enough room now, then inform user that this would
 1134  * block and do nothing.  Otherwise, if nonblocking, send as much as
 1135  * possible.  The data to be sent is described by "uio" if nonzero, otherwise
 1136  * by the mbuf chain "top" (which must be null if uio is not).  Data provided
 1137  * in mbuf chain must be small enough to send all at once.
 1138  *
 1139  * Returns nonzero on error, timeout or signal; callers must check for short
 1140  * counts if EINTR/ERESTART are returned.  Data and control buffers are freed
 1141  * on return.
 1142  */
 1143 int
 1144 sosend_generic(struct socket *so, struct sockaddr *addr, struct uio *uio,
 1145     struct mbuf *top, struct mbuf *control, int flags, struct thread *td)
 1146 {
 1147         long space;
 1148         ssize_t resid;
 1149         int clen = 0, error, dontroute;
 1150         int atomic = sosendallatonce(so) || top;
 1151 
 1152         if (uio != NULL)
 1153                 resid = uio->uio_resid;
 1154         else
 1155                 resid = top->m_pkthdr.len;
 1156         /*
 1157          * In theory resid should be unsigned.  However, space must be
 1158          * signed, as it might be less than 0 if we over-committed, and we
 1159          * must use a signed comparison of space and resid.  On the other
 1160          * hand, a negative resid causes us to loop sending 0-length
 1161          * segments to the protocol.
 1162          *
 1163          * Also check to make sure that MSG_EOR isn't used on SOCK_STREAM
 1164          * type sockets since that's an error.
 1165          */
 1166         if (resid < 0 || (so->so_type == SOCK_STREAM && (flags & MSG_EOR))) {
 1167                 error = EINVAL;
 1168                 goto out;
 1169         }
 1170 
 1171         dontroute =
 1172             (flags & MSG_DONTROUTE) && (so->so_options & SO_DONTROUTE) == 0 &&
 1173             (so->so_proto->pr_flags & PR_ATOMIC);
 1174         if (td != NULL)
 1175                 td->td_ru.ru_msgsnd++;
 1176         if (control != NULL)
 1177                 clen = control->m_len;
 1178 
 1179         error = sblock(&so->so_snd, SBLOCKWAIT(flags));
 1180         if (error)
 1181                 goto out;
 1182 
 1183 restart:
 1184         do {
 1185                 SOCKBUF_LOCK(&so->so_snd);
 1186                 if (so->so_snd.sb_state & SBS_CANTSENDMORE) {
 1187                         SOCKBUF_UNLOCK(&so->so_snd);
 1188                         error = EPIPE;
 1189                         goto release;
 1190                 }
 1191                 if (so->so_error) {
 1192                         error = so->so_error;
 1193                         so->so_error = 0;
 1194                         SOCKBUF_UNLOCK(&so->so_snd);
 1195                         goto release;
 1196                 }
 1197                 if ((so->so_state & SS_ISCONNECTED) == 0) {
 1198                         /*
 1199                          * `sendto' and `sendmsg' is allowed on a connection-
 1200                          * based socket if it supports implied connect.
 1201                          * Return ENOTCONN if not connected and no address is
 1202                          * supplied.
 1203                          */
 1204                         if ((so->so_proto->pr_flags & PR_CONNREQUIRED) &&
 1205                             (so->so_proto->pr_flags & PR_IMPLOPCL) == 0) {
 1206                                 if ((so->so_state & SS_ISCONFIRMING) == 0 &&
 1207                                     !(resid == 0 && clen != 0)) {
 1208                                         SOCKBUF_UNLOCK(&so->so_snd);
 1209                                         error = ENOTCONN;
 1210                                         goto release;
 1211                                 }
 1212                         } else if (addr == NULL) {
 1213                                 SOCKBUF_UNLOCK(&so->so_snd);
 1214                                 if (so->so_proto->pr_flags & PR_CONNREQUIRED)
 1215                                         error = ENOTCONN;
 1216                                 else
 1217                                         error = EDESTADDRREQ;
 1218                                 goto release;
 1219                         }
 1220                 }
 1221                 space = sbspace(&so->so_snd);
 1222                 if (flags & MSG_OOB)
 1223                         space += 1024;
 1224                 if ((atomic && resid > so->so_snd.sb_hiwat) ||
 1225                     clen > so->so_snd.sb_hiwat) {
 1226                         SOCKBUF_UNLOCK(&so->so_snd);
 1227                         error = EMSGSIZE;
 1228                         goto release;
 1229                 }
 1230                 if (space < resid + clen &&
 1231                     (atomic || space < so->so_snd.sb_lowat || space < clen)) {
 1232                         if ((so->so_state & SS_NBIO) || (flags & MSG_NBIO)) {
 1233                                 SOCKBUF_UNLOCK(&so->so_snd);
 1234                                 error = EWOULDBLOCK;
 1235                                 goto release;
 1236                         }
 1237                         error = sbwait(&so->so_snd);
 1238                         SOCKBUF_UNLOCK(&so->so_snd);
 1239                         if (error)
 1240                                 goto release;
 1241                         goto restart;
 1242                 }
 1243                 SOCKBUF_UNLOCK(&so->so_snd);
 1244                 space -= clen;
 1245                 do {
 1246                         if (uio == NULL) {
 1247                                 resid = 0;
 1248                                 if (flags & MSG_EOR)
 1249                                         top->m_flags |= M_EOR;
 1250                         } else {
 1251                                 /*
 1252                                  * Copy the data from userland into a mbuf
 1253                                  * chain.  If no data is to be copied in,
 1254                                  * a single empty mbuf is returned.
 1255                                  */
 1256                                 top = m_uiotombuf(uio, M_WAITOK, space,
 1257                                     (atomic ? max_hdr : 0),
 1258                                     (atomic ? M_PKTHDR : 0) |
 1259                                     ((flags & MSG_EOR) ? M_EOR : 0));
 1260                                 if (top == NULL) {
 1261                                         error = EFAULT; /* only possible error */
 1262                                         goto release;
 1263                                 }
 1264                                 space -= resid - uio->uio_resid;
 1265                                 resid = uio->uio_resid;
 1266                         }
 1267                         if (dontroute) {
 1268                                 SOCK_LOCK(so);
 1269                                 so->so_options |= SO_DONTROUTE;
 1270                                 SOCK_UNLOCK(so);
 1271                         }
 1272                         /*
 1273                          * XXX all the SBS_CANTSENDMORE checks previously
 1274                          * done could be out of date.  We could have recieved
 1275                          * a reset packet in an interrupt or maybe we slept
 1276                          * while doing page faults in uiomove() etc.  We
 1277                          * could probably recheck again inside the locking
 1278                          * protection here, but there are probably other
 1279                          * places that this also happens.  We must rethink
 1280                          * this.
 1281                          */
 1282                         VNET_SO_ASSERT(so);
 1283                         error = (*so->so_proto->pr_usrreqs->pru_send)(so,
 1284                             (flags & MSG_OOB) ? PRUS_OOB :
 1285                         /*
 1286                          * If the user set MSG_EOF, the protocol understands
 1287                          * this flag and nothing left to send then use
 1288                          * PRU_SEND_EOF instead of PRU_SEND.
 1289                          */
 1290                             ((flags & MSG_EOF) &&
 1291                              (so->so_proto->pr_flags & PR_IMPLOPCL) &&
 1292                              (resid <= 0)) ?
 1293                                 PRUS_EOF :
 1294                         /* If there is more to send set PRUS_MORETOCOME. */
 1295                             (resid > 0 && space > 0) ? PRUS_MORETOCOME : 0,
 1296                             top, addr, control, td);
 1297                         if (dontroute) {
 1298                                 SOCK_LOCK(so);
 1299                                 so->so_options &= ~SO_DONTROUTE;
 1300                                 SOCK_UNLOCK(so);
 1301                         }
 1302                         clen = 0;
 1303                         control = NULL;
 1304                         top = NULL;
 1305                         if (error)
 1306                                 goto release;
 1307                 } while (resid && space > 0);
 1308         } while (resid);
 1309 
 1310 release:
 1311         sbunlock(&so->so_snd);
 1312 out:
 1313         if (top != NULL)
 1314                 m_freem(top);
 1315         if (control != NULL)
 1316                 m_freem(control);
 1317         return (error);
 1318 }
 1319 
 1320 int
 1321 sosend(struct socket *so, struct sockaddr *addr, struct uio *uio,
 1322     struct mbuf *top, struct mbuf *control, int flags, struct thread *td)
 1323 {
 1324         int error;
 1325 
 1326         CURVNET_SET(so->so_vnet);
 1327         error = so->so_proto->pr_usrreqs->pru_sosend(so, addr, uio, top,
 1328             control, flags, td);
 1329         CURVNET_RESTORE();
 1330         return (error);
 1331 }
 1332 
 1333 /*
 1334  * The part of soreceive() that implements reading non-inline out-of-band
 1335  * data from a socket.  For more complete comments, see soreceive(), from
 1336  * which this code originated.
 1337  *
 1338  * Note that soreceive_rcvoob(), unlike the remainder of soreceive(), is
 1339  * unable to return an mbuf chain to the caller.
 1340  */
 1341 static int
 1342 soreceive_rcvoob(struct socket *so, struct uio *uio, int flags)
 1343 {
 1344         struct protosw *pr = so->so_proto;
 1345         struct mbuf *m;
 1346         int error;
 1347 
 1348         KASSERT(flags & MSG_OOB, ("soreceive_rcvoob: (flags & MSG_OOB) == 0"));
 1349         VNET_SO_ASSERT(so);
 1350 
 1351         m = m_get(M_WAITOK, MT_DATA);
 1352         error = (*pr->pr_usrreqs->pru_rcvoob)(so, m, flags & MSG_PEEK);
 1353         if (error)
 1354                 goto bad;
 1355         do {
 1356                 error = uiomove(mtod(m, void *),
 1357                     (int) min(uio->uio_resid, m->m_len), uio);
 1358                 m = m_free(m);
 1359         } while (uio->uio_resid && error == 0 && m);
 1360 bad:
 1361         if (m != NULL)
 1362                 m_freem(m);
 1363         return (error);
 1364 }
 1365 
 1366 /*
 1367  * Following replacement or removal of the first mbuf on the first mbuf chain
 1368  * of a socket buffer, push necessary state changes back into the socket
 1369  * buffer so that other consumers see the values consistently.  'nextrecord'
 1370  * is the callers locally stored value of the original value of
 1371  * sb->sb_mb->m_nextpkt which must be restored when the lead mbuf changes.
 1372  * NOTE: 'nextrecord' may be NULL.
 1373  */
 1374 static __inline void
 1375 sockbuf_pushsync(struct sockbuf *sb, struct mbuf *nextrecord)
 1376 {
 1377 
 1378         SOCKBUF_LOCK_ASSERT(sb);
 1379         /*
 1380          * First, update for the new value of nextrecord.  If necessary, make
 1381          * it the first record.
 1382          */
 1383         if (sb->sb_mb != NULL)
 1384                 sb->sb_mb->m_nextpkt = nextrecord;
 1385         else
 1386                 sb->sb_mb = nextrecord;
 1387 
 1388         /*
 1389          * Now update any dependent socket buffer fields to reflect the new
 1390          * state.  This is an expanded inline of SB_EMPTY_FIXUP(), with the
 1391          * addition of a second clause that takes care of the case where
 1392          * sb_mb has been updated, but remains the last record.
 1393          */
 1394         if (sb->sb_mb == NULL) {
 1395                 sb->sb_mbtail = NULL;
 1396                 sb->sb_lastrecord = NULL;
 1397         } else if (sb->sb_mb->m_nextpkt == NULL)
 1398                 sb->sb_lastrecord = sb->sb_mb;
 1399 }
 1400 
 1401 /*
 1402  * Implement receive operations on a socket.  We depend on the way that
 1403  * records are added to the sockbuf by sbappend.  In particular, each record
 1404  * (mbufs linked through m_next) must begin with an address if the protocol
 1405  * so specifies, followed by an optional mbuf or mbufs containing ancillary
 1406  * data, and then zero or more mbufs of data.  In order to allow parallelism
 1407  * between network receive and copying to user space, as well as avoid
 1408  * sleeping with a mutex held, we release the socket buffer mutex during the
 1409  * user space copy.  Although the sockbuf is locked, new data may still be
 1410  * appended, and thus we must maintain consistency of the sockbuf during that
 1411  * time.
 1412  *
 1413  * The caller may receive the data as a single mbuf chain by supplying an
 1414  * mbuf **mp0 for use in returning the chain.  The uio is then used only for
 1415  * the count in uio_resid.
 1416  */
 1417 int
 1418 soreceive_generic(struct socket *so, struct sockaddr **psa, struct uio *uio,
 1419     struct mbuf **mp0, struct mbuf **controlp, int *flagsp)
 1420 {
 1421         struct mbuf *m, **mp;
 1422         int flags, error, offset;
 1423         ssize_t len;
 1424         struct protosw *pr = so->so_proto;
 1425         struct mbuf *nextrecord;
 1426         int moff, type = 0;
 1427         ssize_t orig_resid = uio->uio_resid;
 1428 
 1429         mp = mp0;
 1430         if (psa != NULL)
 1431                 *psa = NULL;
 1432         if (controlp != NULL)
 1433                 *controlp = NULL;
 1434         if (flagsp != NULL)
 1435                 flags = *flagsp &~ MSG_EOR;
 1436         else
 1437                 flags = 0;
 1438         if (flags & MSG_OOB)
 1439                 return (soreceive_rcvoob(so, uio, flags));
 1440         if (mp != NULL)
 1441                 *mp = NULL;
 1442         if ((pr->pr_flags & PR_WANTRCVD) && (so->so_state & SS_ISCONFIRMING)
 1443             && uio->uio_resid) {
 1444                 VNET_SO_ASSERT(so);
 1445                 (*pr->pr_usrreqs->pru_rcvd)(so, 0);
 1446         }
 1447 
 1448         error = sblock(&so->so_rcv, SBLOCKWAIT(flags));
 1449         if (error)
 1450                 return (error);
 1451 
 1452 restart:
 1453         SOCKBUF_LOCK(&so->so_rcv);
 1454         m = so->so_rcv.sb_mb;
 1455         /*
 1456          * If we have less data than requested, block awaiting more (subject
 1457          * to any timeout) if:
 1458          *   1. the current count is less than the low water mark, or
 1459          *   2. MSG_DONTWAIT is not set
 1460          */
 1461         if (m == NULL || (((flags & MSG_DONTWAIT) == 0 &&
 1462             so->so_rcv.sb_cc < uio->uio_resid) &&
 1463             so->so_rcv.sb_cc < so->so_rcv.sb_lowat &&
 1464             m->m_nextpkt == NULL && (pr->pr_flags & PR_ATOMIC) == 0)) {
 1465                 KASSERT(m != NULL || !so->so_rcv.sb_cc,
 1466                     ("receive: m == %p so->so_rcv.sb_cc == %u",
 1467                     m, so->so_rcv.sb_cc));
 1468                 if (so->so_error) {
 1469                         if (m != NULL)
 1470                                 goto dontblock;
 1471                         error = so->so_error;
 1472                         if ((flags & MSG_PEEK) == 0)
 1473                                 so->so_error = 0;
 1474                         SOCKBUF_UNLOCK(&so->so_rcv);
 1475                         goto release;
 1476                 }
 1477                 SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 1478                 if (so->so_rcv.sb_state & SBS_CANTRCVMORE) {
 1479                         if (m == NULL) {
 1480                                 SOCKBUF_UNLOCK(&so->so_rcv);
 1481                                 goto release;
 1482                         } else
 1483                                 goto dontblock;
 1484                 }
 1485                 for (; m != NULL; m = m->m_next)
 1486                         if (m->m_type == MT_OOBDATA  || (m->m_flags & M_EOR)) {
 1487                                 m = so->so_rcv.sb_mb;
 1488                                 goto dontblock;
 1489                         }
 1490                 if ((so->so_state & (SS_ISCONNECTED|SS_ISCONNECTING)) == 0 &&
 1491                     (so->so_proto->pr_flags & PR_CONNREQUIRED)) {
 1492                         SOCKBUF_UNLOCK(&so->so_rcv);
 1493                         error = ENOTCONN;
 1494                         goto release;
 1495                 }
 1496                 if (uio->uio_resid == 0) {
 1497                         SOCKBUF_UNLOCK(&so->so_rcv);
 1498                         goto release;
 1499                 }
 1500                 if ((so->so_state & SS_NBIO) ||
 1501                     (flags & (MSG_DONTWAIT|MSG_NBIO))) {
 1502                         SOCKBUF_UNLOCK(&so->so_rcv);
 1503                         error = EWOULDBLOCK;
 1504                         goto release;
 1505                 }
 1506                 SBLASTRECORDCHK(&so->so_rcv);
 1507                 SBLASTMBUFCHK(&so->so_rcv);
 1508                 error = sbwait(&so->so_rcv);
 1509                 SOCKBUF_UNLOCK(&so->so_rcv);
 1510                 if (error)
 1511                         goto release;
 1512                 goto restart;
 1513         }
 1514 dontblock:
 1515         /*
 1516          * From this point onward, we maintain 'nextrecord' as a cache of the
 1517          * pointer to the next record in the socket buffer.  We must keep the
 1518          * various socket buffer pointers and local stack versions of the
 1519          * pointers in sync, pushing out modifications before dropping the
 1520          * socket buffer mutex, and re-reading them when picking it up.
 1521          *
 1522          * Otherwise, we will race with the network stack appending new data
 1523          * or records onto the socket buffer by using inconsistent/stale
 1524          * versions of the field, possibly resulting in socket buffer
 1525          * corruption.
 1526          *
 1527          * By holding the high-level sblock(), we prevent simultaneous
 1528          * readers from pulling off the front of the socket buffer.
 1529          */
 1530         SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 1531         if (uio->uio_td)
 1532                 uio->uio_td->td_ru.ru_msgrcv++;
 1533         KASSERT(m == so->so_rcv.sb_mb, ("soreceive: m != so->so_rcv.sb_mb"));
 1534         SBLASTRECORDCHK(&so->so_rcv);
 1535         SBLASTMBUFCHK(&so->so_rcv);
 1536         nextrecord = m->m_nextpkt;
 1537         if (pr->pr_flags & PR_ADDR) {
 1538                 KASSERT(m->m_type == MT_SONAME,
 1539                     ("m->m_type == %d", m->m_type));
 1540                 orig_resid = 0;
 1541                 if (psa != NULL)
 1542                         *psa = sodupsockaddr(mtod(m, struct sockaddr *),
 1543                             M_NOWAIT);
 1544                 if (flags & MSG_PEEK) {
 1545                         m = m->m_next;
 1546                 } else {
 1547                         sbfree(&so->so_rcv, m);
 1548                         so->so_rcv.sb_mb = m_free(m);
 1549                         m = so->so_rcv.sb_mb;
 1550                         sockbuf_pushsync(&so->so_rcv, nextrecord);
 1551                 }
 1552         }
 1553 
 1554         /*
 1555          * Process one or more MT_CONTROL mbufs present before any data mbufs
 1556          * in the first mbuf chain on the socket buffer.  If MSG_PEEK, we
 1557          * just copy the data; if !MSG_PEEK, we call into the protocol to
 1558          * perform externalization (or freeing if controlp == NULL).
 1559          */
 1560         if (m != NULL && m->m_type == MT_CONTROL) {
 1561                 struct mbuf *cm = NULL, *cmn;
 1562                 struct mbuf **cme = &cm;
 1563 
 1564                 do {
 1565                         if (flags & MSG_PEEK) {
 1566                                 if (controlp != NULL) {
 1567                                         *controlp = m_copy(m, 0, m->m_len);
 1568                                         controlp = &(*controlp)->m_next;
 1569                                 }
 1570                                 m = m->m_next;
 1571                         } else {
 1572                                 sbfree(&so->so_rcv, m);
 1573                                 so->so_rcv.sb_mb = m->m_next;
 1574                                 m->m_next = NULL;
 1575                                 *cme = m;
 1576                                 cme = &(*cme)->m_next;
 1577                                 m = so->so_rcv.sb_mb;
 1578                         }
 1579                 } while (m != NULL && m->m_type == MT_CONTROL);
 1580                 if ((flags & MSG_PEEK) == 0)
 1581                         sockbuf_pushsync(&so->so_rcv, nextrecord);
 1582                 while (cm != NULL) {
 1583                         cmn = cm->m_next;
 1584                         cm->m_next = NULL;
 1585                         if (pr->pr_domain->dom_externalize != NULL) {
 1586                                 SOCKBUF_UNLOCK(&so->so_rcv);
 1587                                 VNET_SO_ASSERT(so);
 1588                                 error = (*pr->pr_domain->dom_externalize)
 1589                                     (cm, controlp, flags);
 1590                                 SOCKBUF_LOCK(&so->so_rcv);
 1591                         } else if (controlp != NULL)
 1592                                 *controlp = cm;
 1593                         else
 1594                                 m_freem(cm);
 1595                         if (controlp != NULL) {
 1596                                 orig_resid = 0;
 1597                                 while (*controlp != NULL)
 1598                                         controlp = &(*controlp)->m_next;
 1599                         }
 1600                         cm = cmn;
 1601                 }
 1602                 if (m != NULL)
 1603                         nextrecord = so->so_rcv.sb_mb->m_nextpkt;
 1604                 else
 1605                         nextrecord = so->so_rcv.sb_mb;
 1606                 orig_resid = 0;
 1607         }
 1608         if (m != NULL) {
 1609                 if ((flags & MSG_PEEK) == 0) {
 1610                         KASSERT(m->m_nextpkt == nextrecord,
 1611                             ("soreceive: post-control, nextrecord !sync"));
 1612                         if (nextrecord == NULL) {
 1613                                 KASSERT(so->so_rcv.sb_mb == m,
 1614                                     ("soreceive: post-control, sb_mb!=m"));
 1615                                 KASSERT(so->so_rcv.sb_lastrecord == m,
 1616                                     ("soreceive: post-control, lastrecord!=m"));
 1617                         }
 1618                 }
 1619                 type = m->m_type;
 1620                 if (type == MT_OOBDATA)
 1621                         flags |= MSG_OOB;
 1622         } else {
 1623                 if ((flags & MSG_PEEK) == 0) {
 1624                         KASSERT(so->so_rcv.sb_mb == nextrecord,
 1625                             ("soreceive: sb_mb != nextrecord"));
 1626                         if (so->so_rcv.sb_mb == NULL) {
 1627                                 KASSERT(so->so_rcv.sb_lastrecord == NULL,
 1628                                     ("soreceive: sb_lastercord != NULL"));
 1629                         }
 1630                 }
 1631         }
 1632         SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 1633         SBLASTRECORDCHK(&so->so_rcv);
 1634         SBLASTMBUFCHK(&so->so_rcv);
 1635 
 1636         /*
 1637          * Now continue to read any data mbufs off of the head of the socket
 1638          * buffer until the read request is satisfied.  Note that 'type' is
 1639          * used to store the type of any mbuf reads that have happened so far
 1640          * such that soreceive() can stop reading if the type changes, which
 1641          * causes soreceive() to return only one of regular data and inline
 1642          * out-of-band data in a single socket receive operation.
 1643          */
 1644         moff = 0;
 1645         offset = 0;
 1646         while (m != NULL && uio->uio_resid > 0 && error == 0) {
 1647                 /*
 1648                  * If the type of mbuf has changed since the last mbuf
 1649                  * examined ('type'), end the receive operation.
 1650                  */
 1651                 SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 1652                 if (m->m_type == MT_OOBDATA || m->m_type == MT_CONTROL) {
 1653                         if (type != m->m_type)
 1654                                 break;
 1655                 } else if (type == MT_OOBDATA)
 1656                         break;
 1657                 else
 1658                     KASSERT(m->m_type == MT_DATA,
 1659                         ("m->m_type == %d", m->m_type));
 1660                 so->so_rcv.sb_state &= ~SBS_RCVATMARK;
 1661                 len = uio->uio_resid;
 1662                 if (so->so_oobmark && len > so->so_oobmark - offset)
 1663                         len = so->so_oobmark - offset;
 1664                 if (len > m->m_len - moff)
 1665                         len = m->m_len - moff;
 1666                 /*
 1667                  * If mp is set, just pass back the mbufs.  Otherwise copy
 1668                  * them out via the uio, then free.  Sockbuf must be
 1669                  * consistent here (points to current mbuf, it points to next
 1670                  * record) when we drop priority; we must note any additions
 1671                  * to the sockbuf when we block interrupts again.
 1672                  */
 1673                 if (mp == NULL) {
 1674                         SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 1675                         SBLASTRECORDCHK(&so->so_rcv);
 1676                         SBLASTMBUFCHK(&so->so_rcv);
 1677                         SOCKBUF_UNLOCK(&so->so_rcv);
 1678                         error = uiomove(mtod(m, char *) + moff, (int)len, uio);
 1679                         SOCKBUF_LOCK(&so->so_rcv);
 1680                         if (error) {
 1681                                 /*
 1682                                  * The MT_SONAME mbuf has already been removed
 1683                                  * from the record, so it is necessary to
 1684                                  * remove the data mbufs, if any, to preserve
 1685                                  * the invariant in the case of PR_ADDR that
 1686                                  * requires MT_SONAME mbufs at the head of
 1687                                  * each record.
 1688                                  */
 1689                                 if (m && pr->pr_flags & PR_ATOMIC &&
 1690                                     ((flags & MSG_PEEK) == 0))
 1691                                         (void)sbdroprecord_locked(&so->so_rcv);
 1692                                 SOCKBUF_UNLOCK(&so->so_rcv);
 1693                                 goto release;
 1694                         }
 1695                 } else
 1696                         uio->uio_resid -= len;
 1697                 SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 1698                 if (len == m->m_len - moff) {
 1699                         if (m->m_flags & M_EOR)
 1700                                 flags |= MSG_EOR;
 1701                         if (flags & MSG_PEEK) {
 1702                                 m = m->m_next;
 1703                                 moff = 0;
 1704                         } else {
 1705                                 nextrecord = m->m_nextpkt;
 1706                                 sbfree(&so->so_rcv, m);
 1707                                 if (mp != NULL) {
 1708                                         m->m_nextpkt = NULL;
 1709                                         *mp = m;
 1710                                         mp = &m->m_next;
 1711                                         so->so_rcv.sb_mb = m = m->m_next;
 1712                                         *mp = NULL;
 1713                                 } else {
 1714                                         so->so_rcv.sb_mb = m_free(m);
 1715                                         m = so->so_rcv.sb_mb;
 1716                                 }
 1717                                 sockbuf_pushsync(&so->so_rcv, nextrecord);
 1718                                 SBLASTRECORDCHK(&so->so_rcv);
 1719                                 SBLASTMBUFCHK(&so->so_rcv);
 1720                         }
 1721                 } else {
 1722                         if (flags & MSG_PEEK)
 1723                                 moff += len;
 1724                         else {
 1725                                 if (mp != NULL) {
 1726                                         int copy_flag;
 1727 
 1728                                         if (flags & MSG_DONTWAIT)
 1729                                                 copy_flag = M_NOWAIT;
 1730                                         else
 1731                                                 copy_flag = M_WAIT;
 1732                                         if (copy_flag == M_WAITOK)
 1733                                                 SOCKBUF_UNLOCK(&so->so_rcv);
 1734                                         *mp = m_copym(m, 0, len, copy_flag);
 1735                                         if (copy_flag == M_WAITOK)
 1736                                                 SOCKBUF_LOCK(&so->so_rcv);
 1737                                         if (*mp == NULL) {
 1738                                                 /*
 1739                                                  * m_copym() couldn't
 1740                                                  * allocate an mbuf.  Adjust
 1741                                                  * uio_resid back (it was
 1742                                                  * adjusted down by len
 1743                                                  * bytes, which we didn't end
 1744                                                  * up "copying" over).
 1745                                                  */
 1746                                                 uio->uio_resid += len;
 1747                                                 break;
 1748                                         }
 1749                                 }
 1750                                 m->m_data += len;
 1751                                 m->m_len -= len;
 1752                                 so->so_rcv.sb_cc -= len;
 1753                         }
 1754                 }
 1755                 SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 1756                 if (so->so_oobmark) {
 1757                         if ((flags & MSG_PEEK) == 0) {
 1758                                 so->so_oobmark -= len;
 1759                                 if (so->so_oobmark == 0) {
 1760                                         so->so_rcv.sb_state |= SBS_RCVATMARK;
 1761                                         break;
 1762                                 }
 1763                         } else {
 1764                                 offset += len;
 1765                                 if (offset == so->so_oobmark)
 1766                                         break;
 1767                         }
 1768                 }
 1769                 if (flags & MSG_EOR)
 1770                         break;
 1771                 /*
 1772                  * If the MSG_WAITALL flag is set (for non-atomic socket), we
 1773                  * must not quit until "uio->uio_resid == 0" or an error
 1774                  * termination.  If a signal/timeout occurs, return with a
 1775                  * short count but without error.  Keep sockbuf locked
 1776                  * against other readers.
 1777                  */
 1778                 while (flags & MSG_WAITALL && m == NULL && uio->uio_resid > 0 &&
 1779                     !sosendallatonce(so) && nextrecord == NULL) {
 1780                         SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 1781                         if (so->so_error ||
 1782                             so->so_rcv.sb_state & SBS_CANTRCVMORE)
 1783                                 break;
 1784                         /*
 1785                          * Notify the protocol that some data has been
 1786                          * drained before blocking.
 1787                          */
 1788                         if (pr->pr_flags & PR_WANTRCVD) {
 1789                                 SOCKBUF_UNLOCK(&so->so_rcv);
 1790                                 VNET_SO_ASSERT(so);
 1791                                 (*pr->pr_usrreqs->pru_rcvd)(so, flags);
 1792                                 SOCKBUF_LOCK(&so->so_rcv);
 1793                         }
 1794                         SBLASTRECORDCHK(&so->so_rcv);
 1795                         SBLASTMBUFCHK(&so->so_rcv);
 1796                         /*
 1797                          * We could receive some data while was notifying
 1798                          * the protocol. Skip blocking in this case.
 1799                          */
 1800                         if (so->so_rcv.sb_mb == NULL) {
 1801                                 error = sbwait(&so->so_rcv);
 1802                                 if (error) {
 1803                                         SOCKBUF_UNLOCK(&so->so_rcv);
 1804                                         goto release;
 1805                                 }
 1806                         }
 1807                         m = so->so_rcv.sb_mb;
 1808                         if (m != NULL)
 1809                                 nextrecord = m->m_nextpkt;
 1810                 }
 1811         }
 1812 
 1813         SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 1814         if (m != NULL && pr->pr_flags & PR_ATOMIC) {
 1815                 flags |= MSG_TRUNC;
 1816                 if ((flags & MSG_PEEK) == 0)
 1817                         (void) sbdroprecord_locked(&so->so_rcv);
 1818         }
 1819         if ((flags & MSG_PEEK) == 0) {
 1820                 if (m == NULL) {
 1821                         /*
 1822                          * First part is an inline SB_EMPTY_FIXUP().  Second
 1823                          * part makes sure sb_lastrecord is up-to-date if
 1824                          * there is still data in the socket buffer.
 1825                          */
 1826                         so->so_rcv.sb_mb = nextrecord;
 1827                         if (so->so_rcv.sb_mb == NULL) {
 1828                                 so->so_rcv.sb_mbtail = NULL;
 1829                                 so->so_rcv.sb_lastrecord = NULL;
 1830                         } else if (nextrecord->m_nextpkt == NULL)
 1831                                 so->so_rcv.sb_lastrecord = nextrecord;
 1832                 }
 1833                 SBLASTRECORDCHK(&so->so_rcv);
 1834                 SBLASTMBUFCHK(&so->so_rcv);
 1835                 /*
 1836                  * If soreceive() is being done from the socket callback,
 1837                  * then don't need to generate ACK to peer to update window,
 1838                  * since ACK will be generated on return to TCP.
 1839                  */
 1840                 if (!(flags & MSG_SOCALLBCK) &&
 1841                     (pr->pr_flags & PR_WANTRCVD)) {
 1842                         SOCKBUF_UNLOCK(&so->so_rcv);
 1843                         VNET_SO_ASSERT(so);
 1844                         (*pr->pr_usrreqs->pru_rcvd)(so, flags);
 1845                         SOCKBUF_LOCK(&so->so_rcv);
 1846                 }
 1847         }
 1848         SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 1849         if (orig_resid == uio->uio_resid && orig_resid &&
 1850             (flags & MSG_EOR) == 0 && (so->so_rcv.sb_state & SBS_CANTRCVMORE) == 0) {
 1851                 SOCKBUF_UNLOCK(&so->so_rcv);
 1852                 goto restart;
 1853         }
 1854         SOCKBUF_UNLOCK(&so->so_rcv);
 1855 
 1856         if (flagsp != NULL)
 1857                 *flagsp |= flags;
 1858 release:
 1859         sbunlock(&so->so_rcv);
 1860         return (error);
 1861 }
 1862 
 1863 /*
 1864  * Optimized version of soreceive() for stream (TCP) sockets.
 1865  * XXXAO: (MSG_WAITALL | MSG_PEEK) isn't properly handled.
 1866  */
 1867 int
 1868 soreceive_stream(struct socket *so, struct sockaddr **psa, struct uio *uio,
 1869     struct mbuf **mp0, struct mbuf **controlp, int *flagsp)
 1870 {
 1871         int len = 0, error = 0, flags, oresid;
 1872         struct sockbuf *sb;
 1873         struct mbuf *m, *n = NULL;
 1874 
 1875         /* We only do stream sockets. */
 1876         if (so->so_type != SOCK_STREAM)
 1877                 return (EINVAL);
 1878         if (psa != NULL)
 1879                 *psa = NULL;
 1880         if (controlp != NULL)
 1881                 return (EINVAL);
 1882         if (flagsp != NULL)
 1883                 flags = *flagsp &~ MSG_EOR;
 1884         else
 1885                 flags = 0;
 1886         if (flags & MSG_OOB)
 1887                 return (soreceive_rcvoob(so, uio, flags));
 1888         if (mp0 != NULL)
 1889                 *mp0 = NULL;
 1890 
 1891         sb = &so->so_rcv;
 1892 
 1893         /* Prevent other readers from entering the socket. */
 1894         error = sblock(sb, SBLOCKWAIT(flags));
 1895         if (error)
 1896                 goto out;
 1897         SOCKBUF_LOCK(sb);
 1898 
 1899         /* Easy one, no space to copyout anything. */
 1900         if (uio->uio_resid == 0) {
 1901                 error = EINVAL;
 1902                 goto out;
 1903         }
 1904         oresid = uio->uio_resid;
 1905 
 1906         /* We will never ever get anything unless we are or were connected. */
 1907         if (!(so->so_state & (SS_ISCONNECTED|SS_ISDISCONNECTED))) {
 1908                 error = ENOTCONN;
 1909                 goto out;
 1910         }
 1911 
 1912 restart:
 1913         SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 1914 
 1915         /* Abort if socket has reported problems. */
 1916         if (so->so_error) {
 1917                 if (sb->sb_cc > 0)
 1918                         goto deliver;
 1919                 if (oresid > uio->uio_resid)
 1920                         goto out;
 1921                 error = so->so_error;
 1922                 if (!(flags & MSG_PEEK))
 1923                         so->so_error = 0;
 1924                 goto out;
 1925         }
 1926 
 1927         /* Door is closed.  Deliver what is left, if any. */
 1928         if (sb->sb_state & SBS_CANTRCVMORE) {
 1929                 if (sb->sb_cc > 0)
 1930                         goto deliver;
 1931                 else
 1932                         goto out;
 1933         }
 1934 
 1935         /* Socket buffer is empty and we shall not block. */
 1936         if (sb->sb_cc == 0 &&
 1937             ((so->so_state & SS_NBIO) || (flags & (MSG_DONTWAIT|MSG_NBIO)))) {
 1938                 error = EAGAIN;
 1939                 goto out;
 1940         }
 1941 
 1942         /* Socket buffer got some data that we shall deliver now. */
 1943         if (sb->sb_cc > 0 && !(flags & MSG_WAITALL) &&
 1944             ((so->so_state & SS_NBIO) ||
 1945              (flags & (MSG_DONTWAIT|MSG_NBIO)) ||
 1946              sb->sb_cc >= sb->sb_lowat ||
 1947              sb->sb_cc >= uio->uio_resid ||
 1948              sb->sb_cc >= sb->sb_hiwat) ) {
 1949                 goto deliver;
 1950         }
 1951 
 1952         /* On MSG_WAITALL we must wait until all data or error arrives. */
 1953         if ((flags & MSG_WAITALL) &&
 1954             (sb->sb_cc >= uio->uio_resid || sb->sb_cc >= sb->sb_hiwat))
 1955                 goto deliver;
 1956 
 1957         /*
 1958          * Wait and block until (more) data comes in.
 1959          * NB: Drops the sockbuf lock during wait.
 1960          */
 1961         error = sbwait(sb);
 1962         if (error)
 1963                 goto out;
 1964         goto restart;
 1965 
 1966 deliver:
 1967         SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 1968         KASSERT(sb->sb_cc > 0, ("%s: sockbuf empty", __func__));
 1969         KASSERT(sb->sb_mb != NULL, ("%s: sb_mb == NULL", __func__));
 1970 
 1971         /* Statistics. */
 1972         if (uio->uio_td)
 1973                 uio->uio_td->td_ru.ru_msgrcv++;
 1974 
 1975         /* Fill uio until full or current end of socket buffer is reached. */
 1976         len = min(uio->uio_resid, sb->sb_cc);
 1977         if (mp0 != NULL) {
 1978                 /* Dequeue as many mbufs as possible. */
 1979                 if (!(flags & MSG_PEEK) && len >= sb->sb_mb->m_len) {
 1980                         if (*mp0 == NULL)
 1981                                 *mp0 = sb->sb_mb;
 1982                         else
 1983                                 m_cat(*mp0, sb->sb_mb);
 1984                         for (m = sb->sb_mb;
 1985                              m != NULL && m->m_len <= len;
 1986                              m = m->m_next) {
 1987                                 len -= m->m_len;
 1988                                 uio->uio_resid -= m->m_len;
 1989                                 sbfree(sb, m);
 1990                                 n = m;
 1991                         }
 1992                         n->m_next = NULL;
 1993                         sb->sb_mb = m;
 1994                         sb->sb_lastrecord = sb->sb_mb;
 1995                         if (sb->sb_mb == NULL)
 1996                                 SB_EMPTY_FIXUP(sb);
 1997                 }
 1998                 /* Copy the remainder. */
 1999                 if (len > 0) {
 2000                         KASSERT(sb->sb_mb != NULL,
 2001                             ("%s: len > 0 && sb->sb_mb empty", __func__));
 2002 
 2003                         m = m_copym(sb->sb_mb, 0, len, M_NOWAIT);
 2004                         if (m == NULL)
 2005                                 len = 0;        /* Don't flush data from sockbuf. */
 2006                         else
 2007                                 uio->uio_resid -= len;
 2008                         if (*mp0 != NULL)
 2009                                 m_cat(*mp0, m);
 2010                         else
 2011                                 *mp0 = m;
 2012                         if (*mp0 == NULL) {
 2013                                 error = ENOBUFS;
 2014                                 goto out;
 2015                         }
 2016                 }
 2017         } else {
 2018                 /* NB: Must unlock socket buffer as uiomove may sleep. */
 2019                 SOCKBUF_UNLOCK(sb);
 2020                 error = m_mbuftouio(uio, sb->sb_mb, len);
 2021                 SOCKBUF_LOCK(sb);
 2022                 if (error)
 2023                         goto out;
 2024         }
 2025         SBLASTRECORDCHK(sb);
 2026         SBLASTMBUFCHK(sb);
 2027 
 2028         /*
 2029          * Remove the delivered data from the socket buffer unless we
 2030          * were only peeking.
 2031          */
 2032         if (!(flags & MSG_PEEK)) {
 2033                 if (len > 0)
 2034                         sbdrop_locked(sb, len);
 2035 
 2036                 /* Notify protocol that we drained some data. */
 2037                 if ((so->so_proto->pr_flags & PR_WANTRCVD) &&
 2038                     (((flags & MSG_WAITALL) && uio->uio_resid > 0) ||
 2039                      !(flags & MSG_SOCALLBCK))) {
 2040                         SOCKBUF_UNLOCK(sb);
 2041                         VNET_SO_ASSERT(so);
 2042                         (*so->so_proto->pr_usrreqs->pru_rcvd)(so, flags);
 2043                         SOCKBUF_LOCK(sb);
 2044                 }
 2045         }
 2046 
 2047         /*
 2048          * For MSG_WAITALL we may have to loop again and wait for
 2049          * more data to come in.
 2050          */
 2051         if ((flags & MSG_WAITALL) && uio->uio_resid > 0)
 2052                 goto restart;
 2053 out:
 2054         SOCKBUF_LOCK_ASSERT(sb);
 2055         SBLASTRECORDCHK(sb);
 2056         SBLASTMBUFCHK(sb);
 2057         SOCKBUF_UNLOCK(sb);
 2058         sbunlock(sb);
 2059         return (error);
 2060 }
 2061 
 2062 /*
 2063  * Optimized version of soreceive() for simple datagram cases from userspace.
 2064  * Unlike in the stream case, we're able to drop a datagram if copyout()
 2065  * fails, and because we handle datagrams atomically, we don't need to use a
 2066  * sleep lock to prevent I/O interlacing.
 2067  */
 2068 int
 2069 soreceive_dgram(struct socket *so, struct sockaddr **psa, struct uio *uio,
 2070     struct mbuf **mp0, struct mbuf **controlp, int *flagsp)
 2071 {
 2072         struct mbuf *m, *m2;
 2073         int flags, error;
 2074         ssize_t len;
 2075         struct protosw *pr = so->so_proto;
 2076         struct mbuf *nextrecord;
 2077 
 2078         if (psa != NULL)
 2079                 *psa = NULL;
 2080         if (controlp != NULL)
 2081                 *controlp = NULL;
 2082         if (flagsp != NULL)
 2083                 flags = *flagsp &~ MSG_EOR;
 2084         else
 2085                 flags = 0;
 2086 
 2087         /*
 2088          * For any complicated cases, fall back to the full
 2089          * soreceive_generic().
 2090          */
 2091         if (mp0 != NULL || (flags & MSG_PEEK) || (flags & MSG_OOB))
 2092                 return (soreceive_generic(so, psa, uio, mp0, controlp,
 2093                     flagsp));
 2094 
 2095         /*
 2096          * Enforce restrictions on use.
 2097          */
 2098         KASSERT((pr->pr_flags & PR_WANTRCVD) == 0,
 2099             ("soreceive_dgram: wantrcvd"));
 2100         KASSERT(pr->pr_flags & PR_ATOMIC, ("soreceive_dgram: !atomic"));
 2101         KASSERT((so->so_rcv.sb_state & SBS_RCVATMARK) == 0,
 2102             ("soreceive_dgram: SBS_RCVATMARK"));
 2103         KASSERT((so->so_proto->pr_flags & PR_CONNREQUIRED) == 0,
 2104             ("soreceive_dgram: P_CONNREQUIRED"));
 2105 
 2106         /*
 2107          * Loop blocking while waiting for a datagram.
 2108          */
 2109         SOCKBUF_LOCK(&so->so_rcv);
 2110         while ((m = so->so_rcv.sb_mb) == NULL) {
 2111                 KASSERT(so->so_rcv.sb_cc == 0,
 2112                     ("soreceive_dgram: sb_mb NULL but sb_cc %u",
 2113                     so->so_rcv.sb_cc));
 2114                 if (so->so_error) {
 2115                         error = so->so_error;
 2116                         so->so_error = 0;
 2117                         SOCKBUF_UNLOCK(&so->so_rcv);
 2118                         return (error);
 2119                 }
 2120                 if (so->so_rcv.sb_state & SBS_CANTRCVMORE ||
 2121                     uio->uio_resid == 0) {
 2122                         SOCKBUF_UNLOCK(&so->so_rcv);
 2123                         return (0);
 2124                 }
 2125                 if ((so->so_state & SS_NBIO) ||
 2126                     (flags & (MSG_DONTWAIT|MSG_NBIO))) {
 2127                         SOCKBUF_UNLOCK(&so->so_rcv);
 2128                         return (EWOULDBLOCK);
 2129                 }
 2130                 SBLASTRECORDCHK(&so->so_rcv);
 2131                 SBLASTMBUFCHK(&so->so_rcv);
 2132                 error = sbwait(&so->so_rcv);
 2133                 if (error) {
 2134                         SOCKBUF_UNLOCK(&so->so_rcv);
 2135                         return (error);
 2136                 }
 2137         }
 2138         SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 2139 
 2140         if (uio->uio_td)
 2141                 uio->uio_td->td_ru.ru_msgrcv++;
 2142         SBLASTRECORDCHK(&so->so_rcv);
 2143         SBLASTMBUFCHK(&so->so_rcv);
 2144         nextrecord = m->m_nextpkt;
 2145         if (nextrecord == NULL) {
 2146                 KASSERT(so->so_rcv.sb_lastrecord == m,
 2147                     ("soreceive_dgram: lastrecord != m"));
 2148         }
 2149 
 2150         KASSERT(so->so_rcv.sb_mb->m_nextpkt == nextrecord,
 2151             ("soreceive_dgram: m_nextpkt != nextrecord"));
 2152 
 2153         /*
 2154          * Pull 'm' and its chain off the front of the packet queue.
 2155          */
 2156         so->so_rcv.sb_mb = NULL;
 2157         sockbuf_pushsync(&so->so_rcv, nextrecord);
 2158 
 2159         /*
 2160          * Walk 'm's chain and free that many bytes from the socket buffer.
 2161          */
 2162         for (m2 = m; m2 != NULL; m2 = m2->m_next)
 2163                 sbfree(&so->so_rcv, m2);
 2164 
 2165         /*
 2166          * Do a few last checks before we let go of the lock.
 2167          */
 2168         SBLASTRECORDCHK(&so->so_rcv);
 2169         SBLASTMBUFCHK(&so->so_rcv);
 2170         SOCKBUF_UNLOCK(&so->so_rcv);
 2171 
 2172         if (pr->pr_flags & PR_ADDR) {
 2173                 KASSERT(m->m_type == MT_SONAME,
 2174                     ("m->m_type == %d", m->m_type));
 2175                 if (psa != NULL)
 2176                         *psa = sodupsockaddr(mtod(m, struct sockaddr *),
 2177                             M_NOWAIT);
 2178                 m = m_free(m);
 2179         }
 2180         if (m == NULL) {
 2181                 /* XXXRW: Can this happen? */
 2182                 return (0);
 2183         }
 2184 
 2185         /*
 2186          * Packet to copyout() is now in 'm' and it is disconnected from the
 2187          * queue.
 2188          *
 2189          * Process one or more MT_CONTROL mbufs present before any data mbufs
 2190          * in the first mbuf chain on the socket buffer.  We call into the
 2191          * protocol to perform externalization (or freeing if controlp ==
 2192          * NULL). In some cases there can be only MT_CONTROL mbufs without
 2193          * MT_DATA mbufs.
 2194          */
 2195         if (m->m_type == MT_CONTROL) {
 2196                 struct mbuf *cm = NULL, *cmn;
 2197                 struct mbuf **cme = &cm;
 2198 
 2199                 do {
 2200                         m2 = m->m_next;
 2201                         m->m_next = NULL;
 2202                         *cme = m;
 2203                         cme = &(*cme)->m_next;
 2204                         m = m2;
 2205                 } while (m != NULL && m->m_type == MT_CONTROL);
 2206                 while (cm != NULL) {
 2207                         cmn = cm->m_next;
 2208                         cm->m_next = NULL;
 2209                         if (pr->pr_domain->dom_externalize != NULL) {
 2210                                 error = (*pr->pr_domain->dom_externalize)
 2211                                     (cm, controlp, flags);
 2212                         } else if (controlp != NULL)
 2213                                 *controlp = cm;
 2214                         else
 2215                                 m_freem(cm);
 2216                         if (controlp != NULL) {
 2217                                 while (*controlp != NULL)
 2218                                         controlp = &(*controlp)->m_next;
 2219                         }
 2220                         cm = cmn;
 2221                 }
 2222         }
 2223         KASSERT(m == NULL || m->m_type == MT_DATA,
 2224             ("soreceive_dgram: !data"));
 2225         while (m != NULL && uio->uio_resid > 0) {
 2226                 len = uio->uio_resid;
 2227                 if (len > m->m_len)
 2228                         len = m->m_len;
 2229                 error = uiomove(mtod(m, char *), (int)len, uio);
 2230                 if (error) {
 2231                         m_freem(m);
 2232                         return (error);
 2233                 }
 2234                 if (len == m->m_len)
 2235                         m = m_free(m);
 2236                 else {
 2237                         m->m_data += len;
 2238                         m->m_len -= len;
 2239                 }
 2240         }
 2241         if (m != NULL) {
 2242                 flags |= MSG_TRUNC;
 2243                 m_freem(m);
 2244         }
 2245         if (flagsp != NULL)
 2246                 *flagsp |= flags;
 2247         return (0);
 2248 }
 2249 
 2250 int
 2251 soreceive(struct socket *so, struct sockaddr **psa, struct uio *uio,
 2252     struct mbuf **mp0, struct mbuf **controlp, int *flagsp)
 2253 {
 2254         int error;
 2255 
 2256         CURVNET_SET(so->so_vnet);
 2257         error = (so->so_proto->pr_usrreqs->pru_soreceive(so, psa, uio, mp0,
 2258             controlp, flagsp));
 2259         CURVNET_RESTORE();
 2260         return (error);
 2261 }
 2262 
 2263 int
 2264 soshutdown(struct socket *so, int how)
 2265 {
 2266         struct protosw *pr = so->so_proto;
 2267         int error;
 2268 
 2269         if (!(how == SHUT_RD || how == SHUT_WR || how == SHUT_RDWR))
 2270                 return (EINVAL);
 2271 
 2272         CURVNET_SET(so->so_vnet);
 2273         if (pr->pr_usrreqs->pru_flush != NULL)
 2274                 (*pr->pr_usrreqs->pru_flush)(so, how);
 2275         if (how != SHUT_WR)
 2276                 sorflush(so);
 2277         if (how != SHUT_RD) {
 2278                 error = (*pr->pr_usrreqs->pru_shutdown)(so);
 2279                 wakeup(&so->so_timeo);
 2280                 CURVNET_RESTORE();
 2281                 return (error);
 2282         }
 2283         wakeup(&so->so_timeo);
 2284         CURVNET_RESTORE();
 2285         return (0);
 2286 }
 2287 
 2288 void
 2289 sorflush(struct socket *so)
 2290 {
 2291         struct sockbuf *sb = &so->so_rcv;
 2292         struct protosw *pr = so->so_proto;
 2293         struct sockbuf asb;
 2294 
 2295         VNET_SO_ASSERT(so);
 2296 
 2297         /*
 2298          * In order to avoid calling dom_dispose with the socket buffer mutex
 2299          * held, and in order to generally avoid holding the lock for a long
 2300          * time, we make a copy of the socket buffer and clear the original
 2301          * (except locks, state).  The new socket buffer copy won't have
 2302          * initialized locks so we can only call routines that won't use or
 2303          * assert those locks.
 2304          *
 2305          * Dislodge threads currently blocked in receive and wait to acquire
 2306          * a lock against other simultaneous readers before clearing the
 2307          * socket buffer.  Don't let our acquire be interrupted by a signal
 2308          * despite any existing socket disposition on interruptable waiting.
 2309          */
 2310         socantrcvmore(so);
 2311         (void) sblock(sb, SBL_WAIT | SBL_NOINTR);
 2312 
 2313         /*
 2314          * Invalidate/clear most of the sockbuf structure, but leave selinfo
 2315          * and mutex data unchanged.
 2316          */
 2317         SOCKBUF_LOCK(sb);
 2318         bzero(&asb, offsetof(struct sockbuf, sb_startzero));
 2319         bcopy(&sb->sb_startzero, &asb.sb_startzero,
 2320             sizeof(*sb) - offsetof(struct sockbuf, sb_startzero));
 2321         bzero(&sb->sb_startzero,
 2322             sizeof(*sb) - offsetof(struct sockbuf, sb_startzero));
 2323         SOCKBUF_UNLOCK(sb);
 2324         sbunlock(sb);
 2325 
 2326         /*
 2327          * Dispose of special rights and flush the socket buffer.  Don't call
 2328          * any unsafe routines (that rely on locks being initialized) on asb.
 2329          */
 2330         if (pr->pr_flags & PR_RIGHTS && pr->pr_domain->dom_dispose != NULL)
 2331                 (*pr->pr_domain->dom_dispose)(asb.sb_mb);
 2332         sbrelease_internal(&asb, so);
 2333 }
 2334 
 2335 /*
 2336  * Perhaps this routine, and sooptcopyout(), below, ought to come in an
 2337  * additional variant to handle the case where the option value needs to be
 2338  * some kind of integer, but not a specific size.  In addition to their use
 2339  * here, these functions are also called by the protocol-level pr_ctloutput()
 2340  * routines.
 2341  */
 2342 int
 2343 sooptcopyin(struct sockopt *sopt, void *buf, size_t len, size_t minlen)
 2344 {
 2345         size_t  valsize;
 2346 
 2347         /*
 2348          * If the user gives us more than we wanted, we ignore it, but if we
 2349          * don't get the minimum length the caller wants, we return EINVAL.
 2350          * On success, sopt->sopt_valsize is set to however much we actually
 2351          * retrieved.
 2352          */
 2353         if ((valsize = sopt->sopt_valsize) < minlen)
 2354                 return EINVAL;
 2355         if (valsize > len)
 2356                 sopt->sopt_valsize = valsize = len;
 2357 
 2358         if (sopt->sopt_td != NULL)
 2359                 return (copyin(sopt->sopt_val, buf, valsize));
 2360 
 2361         bcopy(sopt->sopt_val, buf, valsize);
 2362         return (0);
 2363 }
 2364 
 2365 /*
 2366  * Kernel version of setsockopt(2).
 2367  *
 2368  * XXX: optlen is size_t, not socklen_t
 2369  */
 2370 int
 2371 so_setsockopt(struct socket *so, int level, int optname, void *optval,
 2372     size_t optlen)
 2373 {
 2374         struct sockopt sopt;
 2375 
 2376         sopt.sopt_level = level;
 2377         sopt.sopt_name = optname;
 2378         sopt.sopt_dir = SOPT_SET;
 2379         sopt.sopt_val = optval;
 2380         sopt.sopt_valsize = optlen;
 2381         sopt.sopt_td = NULL;
 2382         return (sosetopt(so, &sopt));
 2383 }
 2384 
 2385 int
 2386 sosetopt(struct socket *so, struct sockopt *sopt)
 2387 {
 2388         int     error, optval;
 2389         struct  linger l;
 2390         struct  timeval tv;
 2391         sbintime_t val;
 2392         uint32_t val32;
 2393 #ifdef MAC
 2394         struct mac extmac;
 2395 #endif
 2396 
 2397         CURVNET_SET(so->so_vnet);
 2398         error = 0;
 2399         if (sopt->sopt_level != SOL_SOCKET) {
 2400                 if (so->so_proto->pr_ctloutput != NULL) {
 2401                         error = (*so->so_proto->pr_ctloutput)(so, sopt);
 2402                         CURVNET_RESTORE();
 2403                         return (error);
 2404                 }
 2405                 error = ENOPROTOOPT;
 2406         } else {
 2407                 switch (sopt->sopt_name) {
 2408 #ifdef INET
 2409                 case SO_ACCEPTFILTER:
 2410                         error = do_setopt_accept_filter(so, sopt);
 2411                         if (error)
 2412                                 goto bad;
 2413                         break;
 2414 #endif
 2415                 case SO_LINGER:
 2416                         error = sooptcopyin(sopt, &l, sizeof l, sizeof l);
 2417                         if (error)
 2418                                 goto bad;
 2419 
 2420                         SOCK_LOCK(so);
 2421                         so->so_linger = l.l_linger;
 2422                         if (l.l_onoff)
 2423                                 so->so_options |= SO_LINGER;
 2424                         else
 2425                                 so->so_options &= ~SO_LINGER;
 2426                         SOCK_UNLOCK(so);
 2427                         break;
 2428 
 2429                 case SO_DEBUG:
 2430                 case SO_KEEPALIVE:
 2431                 case SO_DONTROUTE:
 2432                 case SO_USELOOPBACK:
 2433                 case SO_BROADCAST:
 2434                 case SO_REUSEADDR:
 2435                 case SO_REUSEPORT:
 2436                 case SO_OOBINLINE:
 2437                 case SO_TIMESTAMP:
 2438                 case SO_BINTIME:
 2439                 case SO_NOSIGPIPE:
 2440                 case SO_NO_DDP:
 2441                 case SO_NO_OFFLOAD:
 2442                         error = sooptcopyin(sopt, &optval, sizeof optval,
 2443                             sizeof optval);
 2444                         if (error)
 2445                                 goto bad;
 2446                         SOCK_LOCK(so);
 2447                         if (optval)
 2448                                 so->so_options |= sopt->sopt_name;
 2449                         else
 2450                                 so->so_options &= ~sopt->sopt_name;
 2451                         SOCK_UNLOCK(so);
 2452                         break;
 2453 
 2454                 case SO_SETFIB:
 2455                         error = sooptcopyin(sopt, &optval, sizeof optval,
 2456                             sizeof optval);
 2457                         if (error)
 2458                                 goto bad;
 2459 
 2460                         if (optval < 0 || optval >= rt_numfibs) {
 2461                                 error = EINVAL;
 2462                                 goto bad;
 2463                         }
 2464                         if (((so->so_proto->pr_domain->dom_family == PF_INET) ||
 2465                            (so->so_proto->pr_domain->dom_family == PF_INET6) ||
 2466                            (so->so_proto->pr_domain->dom_family == PF_ROUTE)))
 2467                                 so->so_fibnum = optval;
 2468                         else
 2469                                 so->so_fibnum = 0;
 2470                         break;
 2471 
 2472                 case SO_USER_COOKIE:
 2473                         error = sooptcopyin(sopt, &val32, sizeof val32,
 2474                             sizeof val32);
 2475                         if (error)
 2476                                 goto bad;
 2477                         so->so_user_cookie = val32;
 2478                         break;
 2479 
 2480                 case SO_SNDBUF:
 2481                 case SO_RCVBUF:
 2482                 case SO_SNDLOWAT:
 2483                 case SO_RCVLOWAT:
 2484                         error = sooptcopyin(sopt, &optval, sizeof optval,
 2485                             sizeof optval);
 2486                         if (error)
 2487                                 goto bad;
 2488 
 2489                         /*
 2490                          * Values < 1 make no sense for any of these options,
 2491                          * so disallow them.
 2492                          */
 2493                         if (optval < 1) {
 2494                                 error = EINVAL;
 2495                                 goto bad;
 2496                         }
 2497 
 2498                         switch (sopt->sopt_name) {
 2499                         case SO_SNDBUF:
 2500                         case SO_RCVBUF:
 2501                                 if (sbreserve(sopt->sopt_name == SO_SNDBUF ?
 2502                                     &so->so_snd : &so->so_rcv, (u_long)optval,
 2503                                     so, curthread) == 0) {
 2504                                         error = ENOBUFS;
 2505                                         goto bad;
 2506                                 }
 2507                                 (sopt->sopt_name == SO_SNDBUF ? &so->so_snd :
 2508                                     &so->so_rcv)->sb_flags &= ~SB_AUTOSIZE;
 2509                                 break;
 2510 
 2511                         /*
 2512                          * Make sure the low-water is never greater than the
 2513                          * high-water.
 2514                          */
 2515                         case SO_SNDLOWAT:
 2516                                 SOCKBUF_LOCK(&so->so_snd);
 2517                                 so->so_snd.sb_lowat =
 2518                                     (optval > so->so_snd.sb_hiwat) ?
 2519                                     so->so_snd.sb_hiwat : optval;
 2520                                 SOCKBUF_UNLOCK(&so->so_snd);
 2521                                 break;
 2522                         case SO_RCVLOWAT:
 2523                                 SOCKBUF_LOCK(&so->so_rcv);
 2524                                 so->so_rcv.sb_lowat =
 2525                                     (optval > so->so_rcv.sb_hiwat) ?
 2526                                     so->so_rcv.sb_hiwat : optval;
 2527                                 SOCKBUF_UNLOCK(&so->so_rcv);
 2528                                 break;
 2529                         }
 2530                         break;
 2531 
 2532                 case SO_SNDTIMEO:
 2533                 case SO_RCVTIMEO:
 2534 #ifdef COMPAT_FREEBSD32
 2535                         if (SV_CURPROC_FLAG(SV_ILP32)) {
 2536                                 struct timeval32 tv32;
 2537 
 2538                                 error = sooptcopyin(sopt, &tv32, sizeof tv32,
 2539                                     sizeof tv32);
 2540                                 CP(tv32, tv, tv_sec);
 2541                                 CP(tv32, tv, tv_usec);
 2542                         } else
 2543 #endif
 2544                                 error = sooptcopyin(sopt, &tv, sizeof tv,
 2545                                     sizeof tv);
 2546                         if (error)
 2547                                 goto bad;
 2548                         if (tv.tv_sec < 0 || tv.tv_usec < 0 ||
 2549                             tv.tv_usec >= 1000000) {
 2550                                 error = EDOM;
 2551                                 goto bad;
 2552                         }
 2553                         if (tv.tv_sec > INT32_MAX)
 2554                                 val = SBT_MAX;
 2555                         else
 2556                                 val = tvtosbt(tv);
 2557                         switch (sopt->sopt_name) {
 2558                         case SO_SNDTIMEO:
 2559                                 so->so_snd.sb_timeo = val;
 2560                                 break;
 2561                         case SO_RCVTIMEO:
 2562                                 so->so_rcv.sb_timeo = val;
 2563                                 break;
 2564                         }
 2565                         break;
 2566 
 2567                 case SO_LABEL:
 2568 #ifdef MAC
 2569                         error = sooptcopyin(sopt, &extmac, sizeof extmac,
 2570                             sizeof extmac);
 2571                         if (error)
 2572                                 goto bad;
 2573                         error = mac_setsockopt_label(sopt->sopt_td->td_ucred,
 2574                             so, &extmac);
 2575 #else
 2576                         error = EOPNOTSUPP;
 2577 #endif
 2578                         break;
 2579 
 2580                 default:
 2581                         error = ENOPROTOOPT;
 2582                         break;
 2583                 }
 2584                 if (error == 0 && so->so_proto->pr_ctloutput != NULL)
 2585                         (void)(*so->so_proto->pr_ctloutput)(so, sopt);
 2586         }
 2587 bad:
 2588         CURVNET_RESTORE();
 2589         return (error);
 2590 }
 2591 
 2592 /*
 2593  * Helper routine for getsockopt.
 2594  */
 2595 int
 2596 sooptcopyout(struct sockopt *sopt, const void *buf, size_t len)
 2597 {
 2598         int     error;
 2599         size_t  valsize;
 2600 
 2601         error = 0;
 2602 
 2603         /*
 2604          * Documented get behavior is that we always return a value, possibly
 2605          * truncated to fit in the user's buffer.  Traditional behavior is
 2606          * that we always tell the user precisely how much we copied, rather
 2607          * than something useful like the total amount we had available for
 2608          * her.  Note that this interface is not idempotent; the entire
 2609          * answer must generated ahead of time.
 2610          */
 2611         valsize = min(len, sopt->sopt_valsize);
 2612         sopt->sopt_valsize = valsize;
 2613         if (sopt->sopt_val != NULL) {
 2614                 if (sopt->sopt_td != NULL)
 2615                         error = copyout(buf, sopt->sopt_val, valsize);
 2616                 else
 2617                         bcopy(buf, sopt->sopt_val, valsize);
 2618         }
 2619         return (error);
 2620 }
 2621 
 2622 int
 2623 sogetopt(struct socket *so, struct sockopt *sopt)
 2624 {
 2625         int     error, optval;
 2626         struct  linger l;
 2627         struct  timeval tv;
 2628 #ifdef MAC
 2629         struct mac extmac;
 2630 #endif
 2631 
 2632         CURVNET_SET(so->so_vnet);
 2633         error = 0;
 2634         if (sopt->sopt_level != SOL_SOCKET) {
 2635                 if (so->so_proto->pr_ctloutput != NULL)
 2636                         error = (*so->so_proto->pr_ctloutput)(so, sopt);
 2637                 else
 2638                         error = ENOPROTOOPT;
 2639                 CURVNET_RESTORE();
 2640                 return (error);
 2641         } else {
 2642                 switch (sopt->sopt_name) {
 2643 #ifdef INET
 2644                 case SO_ACCEPTFILTER:
 2645                         error = do_getopt_accept_filter(so, sopt);
 2646                         break;
 2647 #endif
 2648                 case SO_LINGER:
 2649                         SOCK_LOCK(so);
 2650                         l.l_onoff = so->so_options & SO_LINGER;
 2651                         l.l_linger = so->so_linger;
 2652                         SOCK_UNLOCK(so);
 2653                         error = sooptcopyout(sopt, &l, sizeof l);
 2654                         break;
 2655 
 2656                 case SO_USELOOPBACK:
 2657                 case SO_DONTROUTE:
 2658                 case SO_DEBUG:
 2659                 case SO_KEEPALIVE:
 2660                 case SO_REUSEADDR:
 2661                 case SO_REUSEPORT:
 2662                 case SO_BROADCAST:
 2663                 case SO_OOBINLINE:
 2664                 case SO_ACCEPTCONN:
 2665                 case SO_TIMESTAMP:
 2666                 case SO_BINTIME:
 2667                 case SO_NOSIGPIPE:
 2668                         optval = so->so_options & sopt->sopt_name;
 2669 integer:
 2670                         error = sooptcopyout(sopt, &optval, sizeof optval);
 2671                         break;
 2672 
 2673                 case SO_TYPE:
 2674                         optval = so->so_type;
 2675                         goto integer;
 2676 
 2677                 case SO_PROTOCOL:
 2678                         optval = so->so_proto->pr_protocol;
 2679                         goto integer;
 2680 
 2681                 case SO_ERROR:
 2682                         SOCK_LOCK(so);
 2683                         optval = so->so_error;
 2684                         so->so_error = 0;
 2685                         SOCK_UNLOCK(so);
 2686                         goto integer;
 2687 
 2688                 case SO_SNDBUF:
 2689                         optval = so->so_snd.sb_hiwat;
 2690                         goto integer;
 2691 
 2692                 case SO_RCVBUF:
 2693                         optval = so->so_rcv.sb_hiwat;
 2694                         goto integer;
 2695 
 2696                 case SO_SNDLOWAT:
 2697                         optval = so->so_snd.sb_lowat;
 2698                         goto integer;
 2699 
 2700                 case SO_RCVLOWAT:
 2701                         optval = so->so_rcv.sb_lowat;
 2702                         goto integer;
 2703 
 2704                 case SO_SNDTIMEO:
 2705                 case SO_RCVTIMEO:
 2706                         tv = sbttotv(sopt->sopt_name == SO_SNDTIMEO ?
 2707                             so->so_snd.sb_timeo : so->so_rcv.sb_timeo);
 2708 #ifdef COMPAT_FREEBSD32
 2709                         if (SV_CURPROC_FLAG(SV_ILP32)) {
 2710                                 struct timeval32 tv32;
 2711 
 2712                                 CP(tv, tv32, tv_sec);
 2713                                 CP(tv, tv32, tv_usec);
 2714                                 error = sooptcopyout(sopt, &tv32, sizeof tv32);
 2715                         } else
 2716 #endif
 2717                                 error = sooptcopyout(sopt, &tv, sizeof tv);
 2718                         break;
 2719 
 2720                 case SO_LABEL:
 2721 #ifdef MAC
 2722                         error = sooptcopyin(sopt, &extmac, sizeof(extmac),
 2723                             sizeof(extmac));
 2724                         if (error)
 2725                                 goto bad;
 2726                         error = mac_getsockopt_label(sopt->sopt_td->td_ucred,
 2727                             so, &extmac);
 2728                         if (error)
 2729                                 goto bad;
 2730                         error = sooptcopyout(sopt, &extmac, sizeof extmac);
 2731 #else
 2732                         error = EOPNOTSUPP;
 2733 #endif
 2734                         break;
 2735 
 2736                 case SO_PEERLABEL:
 2737 #ifdef MAC
 2738                         error = sooptcopyin(sopt, &extmac, sizeof(extmac),
 2739                             sizeof(extmac));
 2740                         if (error)
 2741                                 goto bad;
 2742                         error = mac_getsockopt_peerlabel(
 2743                             sopt->sopt_td->td_ucred, so, &extmac);
 2744                         if (error)
 2745                                 goto bad;
 2746                         error = sooptcopyout(sopt, &extmac, sizeof extmac);
 2747 #else
 2748                         error = EOPNOTSUPP;
 2749 #endif
 2750                         break;
 2751 
 2752                 case SO_LISTENQLIMIT:
 2753                         optval = so->so_qlimit;
 2754                         goto integer;
 2755 
 2756                 case SO_LISTENQLEN:
 2757                         optval = so->so_qlen;
 2758                         goto integer;
 2759 
 2760                 case SO_LISTENINCQLEN:
 2761                         optval = so->so_incqlen;
 2762                         goto integer;
 2763 
 2764                 default:
 2765                         error = ENOPROTOOPT;
 2766                         break;
 2767                 }
 2768         }
 2769 #ifdef MAC
 2770 bad:
 2771 #endif
 2772         CURVNET_RESTORE();
 2773         return (error);
 2774 }
 2775 
 2776 int
 2777 soopt_getm(struct sockopt *sopt, struct mbuf **mp)
 2778 {
 2779         struct mbuf *m, *m_prev;
 2780         int sopt_size = sopt->sopt_valsize;
 2781 
 2782         MGET(m, sopt->sopt_td ? M_WAITOK : M_NOWAIT, MT_DATA);
 2783         if (m == NULL)
 2784                 return ENOBUFS;
 2785         if (sopt_size > MLEN) {
 2786                 MCLGET(m, sopt->sopt_td ? M_WAITOK : M_NOWAIT);
 2787                 if ((m->m_flags & M_EXT) == 0) {
 2788                         m_free(m);
 2789                         return ENOBUFS;
 2790                 }
 2791                 m->m_len = min(MCLBYTES, sopt_size);
 2792         } else {
 2793                 m->m_len = min(MLEN, sopt_size);
 2794         }
 2795         sopt_size -= m->m_len;
 2796         *mp = m;
 2797         m_prev = m;
 2798 
 2799         while (sopt_size) {
 2800                 MGET(m, sopt->sopt_td ? M_WAITOK : M_NOWAIT, MT_DATA);
 2801                 if (m == NULL) {
 2802                         m_freem(*mp);
 2803                         return ENOBUFS;
 2804                 }
 2805                 if (sopt_size > MLEN) {
 2806                         MCLGET(m, sopt->sopt_td != NULL ? M_WAITOK :
 2807                             M_NOWAIT);
 2808                         if ((m->m_flags & M_EXT) == 0) {
 2809                                 m_freem(m);
 2810                                 m_freem(*mp);
 2811                                 return ENOBUFS;
 2812                         }
 2813                         m->m_len = min(MCLBYTES, sopt_size);
 2814                 } else {
 2815                         m->m_len = min(MLEN, sopt_size);
 2816                 }
 2817                 sopt_size -= m->m_len;
 2818                 m_prev->m_next = m;
 2819                 m_prev = m;
 2820         }
 2821         return (0);
 2822 }
 2823 
 2824 int
 2825 soopt_mcopyin(struct sockopt *sopt, struct mbuf *m)
 2826 {
 2827         struct mbuf *m0 = m;
 2828 
 2829         if (sopt->sopt_val == NULL)
 2830                 return (0);
 2831         while (m != NULL && sopt->sopt_valsize >= m->m_len) {
 2832                 if (sopt->sopt_td != NULL) {
 2833                         int error;
 2834 
 2835                         error = copyin(sopt->sopt_val, mtod(m, char *),
 2836                             m->m_len);
 2837                         if (error != 0) {
 2838                                 m_freem(m0);
 2839                                 return(error);
 2840                         }
 2841                 } else
 2842                         bcopy(sopt->sopt_val, mtod(m, char *), m->m_len);
 2843                 sopt->sopt_valsize -= m->m_len;
 2844                 sopt->sopt_val = (char *)sopt->sopt_val + m->m_len;
 2845                 m = m->m_next;
 2846         }
 2847         if (m != NULL) /* should be allocated enoughly at ip6_sooptmcopyin() */
 2848                 panic("ip6_sooptmcopyin");
 2849         return (0);
 2850 }
 2851 
 2852 int
 2853 soopt_mcopyout(struct sockopt *sopt, struct mbuf *m)
 2854 {
 2855         struct mbuf *m0 = m;
 2856         size_t valsize = 0;
 2857 
 2858         if (sopt->sopt_val == NULL)
 2859                 return (0);
 2860         while (m != NULL && sopt->sopt_valsize >= m->m_len) {
 2861                 if (sopt->sopt_td != NULL) {
 2862                         int error;
 2863 
 2864                         error = copyout(mtod(m, char *), sopt->sopt_val,
 2865                             m->m_len);
 2866                         if (error != 0) {
 2867                                 m_freem(m0);
 2868                                 return(error);
 2869                         }
 2870                 } else
 2871                         bcopy(mtod(m, char *), sopt->sopt_val, m->m_len);
 2872                 sopt->sopt_valsize -= m->m_len;
 2873                 sopt->sopt_val = (char *)sopt->sopt_val + m->m_len;
 2874                 valsize += m->m_len;
 2875                 m = m->m_next;
 2876         }
 2877         if (m != NULL) {
 2878                 /* enough soopt buffer should be given from user-land */
 2879                 m_freem(m0);
 2880                 return(EINVAL);
 2881         }
 2882         sopt->sopt_valsize = valsize;
 2883         return (0);
 2884 }
 2885 
 2886 /*
 2887  * sohasoutofband(): protocol notifies socket layer of the arrival of new
 2888  * out-of-band data, which will then notify socket consumers.
 2889  */
 2890 void
 2891 sohasoutofband(struct socket *so)
 2892 {
 2893 
 2894         if (so->so_sigio != NULL)
 2895                 pgsigio(&so->so_sigio, SIGURG, 0);
 2896         selwakeuppri(&so->so_rcv.sb_sel, PSOCK);
 2897 }
 2898 
 2899 int
 2900 sopoll(struct socket *so, int events, struct ucred *active_cred,
 2901     struct thread *td)
 2902 {
 2903 
 2904         /*
 2905          * We do not need to set or assert curvnet as long as everyone uses
 2906          * sopoll_generic().
 2907          */
 2908         return (so->so_proto->pr_usrreqs->pru_sopoll(so, events, active_cred,
 2909             td));
 2910 }
 2911 
 2912 int
 2913 sopoll_generic(struct socket *so, int events, struct ucred *active_cred,
 2914     struct thread *td)
 2915 {
 2916         int revents = 0;
 2917 
 2918         SOCKBUF_LOCK(&so->so_snd);
 2919         SOCKBUF_LOCK(&so->so_rcv);
 2920         if (events & (POLLIN | POLLRDNORM))
 2921                 if (soreadabledata(so))
 2922                         revents |= events & (POLLIN | POLLRDNORM);
 2923 
 2924         if (events & (POLLOUT | POLLWRNORM))
 2925                 if (sowriteable(so))
 2926                         revents |= events & (POLLOUT | POLLWRNORM);
 2927 
 2928         if (events & (POLLPRI | POLLRDBAND))
 2929                 if (so->so_oobmark || (so->so_rcv.sb_state & SBS_RCVATMARK))
 2930                         revents |= events & (POLLPRI | POLLRDBAND);
 2931 
 2932         if ((events & POLLINIGNEOF) == 0) {
 2933                 if (so->so_rcv.sb_state & SBS_CANTRCVMORE) {
 2934                         revents |= events & (POLLIN | POLLRDNORM);
 2935                         if (so->so_snd.sb_state & SBS_CANTSENDMORE)
 2936                                 revents |= POLLHUP;
 2937                 }
 2938         }
 2939 
 2940         if (revents == 0) {
 2941                 if (events & (POLLIN | POLLPRI | POLLRDNORM | POLLRDBAND)) {
 2942                         selrecord(td, &so->so_rcv.sb_sel);
 2943                         so->so_rcv.sb_flags |= SB_SEL;
 2944                 }
 2945 
 2946                 if (events & (POLLOUT | POLLWRNORM)) {
 2947                         selrecord(td, &so->so_snd.sb_sel);
 2948                         so->so_snd.sb_flags |= SB_SEL;
 2949                 }
 2950         }
 2951 
 2952         SOCKBUF_UNLOCK(&so->so_rcv);
 2953         SOCKBUF_UNLOCK(&so->so_snd);
 2954         return (revents);
 2955 }
 2956 
 2957 int
 2958 soo_kqfilter(struct file *fp, struct knote *kn)
 2959 {
 2960         struct socket *so = kn->kn_fp->f_data;
 2961         struct sockbuf *sb;
 2962 
 2963         switch (kn->kn_filter) {
 2964         case EVFILT_READ:
 2965                 if (so->so_options & SO_ACCEPTCONN)
 2966                         kn->kn_fop = &solisten_filtops;
 2967                 else
 2968                         kn->kn_fop = &soread_filtops;
 2969                 sb = &so->so_rcv;
 2970                 break;
 2971         case EVFILT_WRITE:
 2972                 kn->kn_fop = &sowrite_filtops;
 2973                 sb = &so->so_snd;
 2974                 break;
 2975         default:
 2976                 return (EINVAL);
 2977         }
 2978 
 2979         SOCKBUF_LOCK(sb);
 2980         knlist_add(&sb->sb_sel.si_note, kn, 1);
 2981         sb->sb_flags |= SB_KNOTE;
 2982         SOCKBUF_UNLOCK(sb);
 2983         return (0);
 2984 }
 2985 
 2986 /*
 2987  * Some routines that return EOPNOTSUPP for entry points that are not
 2988  * supported by a protocol.  Fill in as needed.
 2989  */
 2990 int
 2991 pru_accept_notsupp(struct socket *so, struct sockaddr **nam)
 2992 {
 2993 
 2994         return EOPNOTSUPP;
 2995 }
 2996 
 2997 int
 2998 pru_attach_notsupp(struct socket *so, int proto, struct thread *td)
 2999 {
 3000 
 3001         return EOPNOTSUPP;
 3002 }
 3003 
 3004 int
 3005 pru_bind_notsupp(struct socket *so, struct sockaddr *nam, struct thread *td)
 3006 {
 3007 
 3008         return EOPNOTSUPP;
 3009 }
 3010 
 3011 int
 3012 pru_bindat_notsupp(int fd, struct socket *so, struct sockaddr *nam,
 3013     struct thread *td)
 3014 {
 3015 
 3016         return EOPNOTSUPP;
 3017 }
 3018 
 3019 int
 3020 pru_connect_notsupp(struct socket *so, struct sockaddr *nam, struct thread *td)
 3021 {
 3022 
 3023         return EOPNOTSUPP;
 3024 }
 3025 
 3026 int
 3027 pru_connectat_notsupp(int fd, struct socket *so, struct sockaddr *nam,
 3028     struct thread *td)
 3029 {
 3030 
 3031         return EOPNOTSUPP;
 3032 }
 3033 
 3034 int
 3035 pru_connect2_notsupp(struct socket *so1, struct socket *so2)
 3036 {
 3037 
 3038         return EOPNOTSUPP;
 3039 }
 3040 
 3041 int
 3042 pru_control_notsupp(struct socket *so, u_long cmd, caddr_t data,
 3043     struct ifnet *ifp, struct thread *td)
 3044 {
 3045 
 3046         return EOPNOTSUPP;
 3047 }
 3048 
 3049 int
 3050 pru_disconnect_notsupp(struct socket *so)
 3051 {
 3052 
 3053         return EOPNOTSUPP;
 3054 }
 3055 
 3056 int
 3057 pru_listen_notsupp(struct socket *so, int backlog, struct thread *td)
 3058 {
 3059 
 3060         return EOPNOTSUPP;
 3061 }
 3062 
 3063 int
 3064 pru_peeraddr_notsupp(struct socket *so, struct sockaddr **nam)
 3065 {
 3066 
 3067         return EOPNOTSUPP;
 3068 }
 3069 
 3070 int
 3071 pru_rcvd_notsupp(struct socket *so, int flags)
 3072 {
 3073 
 3074         return EOPNOTSUPP;
 3075 }
 3076 
 3077 int
 3078 pru_rcvoob_notsupp(struct socket *so, struct mbuf *m, int flags)
 3079 {
 3080 
 3081         return EOPNOTSUPP;
 3082 }
 3083 
 3084 int
 3085 pru_send_notsupp(struct socket *so, int flags, struct mbuf *m,
 3086     struct sockaddr *addr, struct mbuf *control, struct thread *td)
 3087 {
 3088 
 3089         return EOPNOTSUPP;
 3090 }
 3091 
 3092 /*
 3093  * This isn't really a ``null'' operation, but it's the default one and
 3094  * doesn't do anything destructive.
 3095  */
 3096 int
 3097 pru_sense_null(struct socket *so, struct stat *sb)
 3098 {
 3099 
 3100         sb->st_blksize = so->so_snd.sb_hiwat;
 3101         return 0;
 3102 }
 3103 
 3104 int
 3105 pru_shutdown_notsupp(struct socket *so)
 3106 {
 3107 
 3108         return EOPNOTSUPP;
 3109 }
 3110 
 3111 int
 3112 pru_sockaddr_notsupp(struct socket *so, struct sockaddr **nam)
 3113 {
 3114 
 3115         return EOPNOTSUPP;
 3116 }
 3117 
 3118 int
 3119 pru_sosend_notsupp(struct socket *so, struct sockaddr *addr, struct uio *uio,
 3120     struct mbuf *top, struct mbuf *control, int flags, struct thread *td)
 3121 {
 3122 
 3123         return EOPNOTSUPP;
 3124 }
 3125 
 3126 int
 3127 pru_soreceive_notsupp(struct socket *so, struct sockaddr **paddr,
 3128     struct uio *uio, struct mbuf **mp0, struct mbuf **controlp, int *flagsp)
 3129 {
 3130 
 3131         return EOPNOTSUPP;
 3132 }
 3133 
 3134 int
 3135 pru_sopoll_notsupp(struct socket *so, int events, struct ucred *cred,
 3136     struct thread *td)
 3137 {
 3138 
 3139         return EOPNOTSUPP;
 3140 }
 3141 
 3142 static void
 3143 filt_sordetach(struct knote *kn)
 3144 {
 3145         struct socket *so = kn->kn_fp->f_data;
 3146 
 3147         SOCKBUF_LOCK(&so->so_rcv);
 3148         knlist_remove(&so->so_rcv.sb_sel.si_note, kn, 1);
 3149         if (knlist_empty(&so->so_rcv.sb_sel.si_note))
 3150                 so->so_rcv.sb_flags &= ~SB_KNOTE;
 3151         SOCKBUF_UNLOCK(&so->so_rcv);
 3152 }
 3153 
 3154 /*ARGSUSED*/
 3155 static int
 3156 filt_soread(struct knote *kn, long hint)
 3157 {
 3158         struct socket *so;
 3159 
 3160         so = kn->kn_fp->f_data;
 3161         SOCKBUF_LOCK_ASSERT(&so->so_rcv);
 3162 
 3163         kn->kn_data = so->so_rcv.sb_cc - so->so_rcv.sb_ctl;
 3164         if (so->so_rcv.sb_state & SBS_CANTRCVMORE) {
 3165                 kn->kn_flags |= EV_EOF;
 3166                 kn->kn_fflags = so->so_error;
 3167                 return (1);
 3168         } else if (so->so_error)        /* temporary udp error */
 3169                 return (1);
 3170         else if (kn->kn_sfflags & NOTE_LOWAT)
 3171                 return (kn->kn_data >= kn->kn_sdata);
 3172         else
 3173                 return (so->so_rcv.sb_cc >= so->so_rcv.sb_lowat);
 3174 }
 3175 
 3176 static void
 3177 filt_sowdetach(struct knote *kn)
 3178 {
 3179         struct socket *so = kn->kn_fp->f_data;
 3180 
 3181         SOCKBUF_LOCK(&so->so_snd);
 3182         knlist_remove(&so->so_snd.sb_sel.si_note, kn, 1);
 3183         if (knlist_empty(&so->so_snd.sb_sel.si_note))
 3184                 so->so_snd.sb_flags &= ~SB_KNOTE;
 3185         SOCKBUF_UNLOCK(&so->so_snd);
 3186 }
 3187 
 3188 /*ARGSUSED*/
 3189 static int
 3190 filt_sowrite(struct knote *kn, long hint)
 3191 {
 3192         struct socket *so;
 3193 
 3194         so = kn->kn_fp->f_data;
 3195         SOCKBUF_LOCK_ASSERT(&so->so_snd);
 3196         kn->kn_data = sbspace(&so->so_snd);
 3197         if (so->so_snd.sb_state & SBS_CANTSENDMORE) {
 3198                 kn->kn_flags |= EV_EOF;
 3199                 kn->kn_fflags = so->so_error;
 3200                 return (1);
 3201         } else if (so->so_error)        /* temporary udp error */
 3202                 return (1);
 3203         else if (((so->so_state & SS_ISCONNECTED) == 0) &&
 3204             (so->so_proto->pr_flags & PR_CONNREQUIRED))
 3205                 return (0);
 3206         else if (kn->kn_sfflags & NOTE_LOWAT)
 3207                 return (kn->kn_data >= kn->kn_sdata);
 3208         else
 3209                 return (kn->kn_data >= so->so_snd.sb_lowat);
 3210 }
 3211 
 3212 /*ARGSUSED*/
 3213 static int
 3214 filt_solisten(struct knote *kn, long hint)
 3215 {
 3216         struct socket *so = kn->kn_fp->f_data;
 3217 
 3218         kn->kn_data = so->so_qlen;
 3219         return (!TAILQ_EMPTY(&so->so_comp));
 3220 }
 3221 
 3222 int
 3223 socheckuid(struct socket *so, uid_t uid)
 3224 {
 3225 
 3226         if (so == NULL)
 3227                 return (EPERM);
 3228         if (so->so_cred->cr_uid != uid)
 3229                 return (EPERM);
 3230         return (0);
 3231 }
 3232 
 3233 /*
 3234  * These functions are used by protocols to notify the socket layer (and its
 3235  * consumers) of state changes in the sockets driven by protocol-side events.
 3236  */
 3237 
 3238 /*
 3239  * Procedures to manipulate state flags of socket and do appropriate wakeups.
 3240  *
 3241  * Normal sequence from the active (originating) side is that
 3242  * soisconnecting() is called during processing of connect() call, resulting
 3243  * in an eventual call to soisconnected() if/when the connection is
 3244  * established.  When the connection is torn down soisdisconnecting() is
 3245  * called during processing of disconnect() call, and soisdisconnected() is
 3246  * called when the connection to the peer is totally severed.  The semantics
 3247  * of these routines are such that connectionless protocols can call
 3248  * soisconnected() and soisdisconnected() only, bypassing the in-progress
 3249  * calls when setting up a ``connection'' takes no time.
 3250  *
 3251  * From the passive side, a socket is created with two queues of sockets:
 3252  * so_incomp for connections in progress and so_comp for connections already
 3253  * made and awaiting user acceptance.  As a protocol is preparing incoming
 3254  * connections, it creates a socket structure queued on so_incomp by calling
 3255  * sonewconn().  When the connection is established, soisconnected() is
 3256  * called, and transfers the socket structure to so_comp, making it available
 3257  * to accept().
 3258  *
 3259  * If a socket is closed with sockets on either so_incomp or so_comp, these
 3260  * sockets are dropped.
 3261  *
 3262  * If higher-level protocols are implemented in the kernel, the wakeups done
 3263  * here will sometimes cause software-interrupt process scheduling.
 3264  */
 3265 void
 3266 soisconnecting(struct socket *so)
 3267 {
 3268 
 3269         SOCK_LOCK(so);
 3270         so->so_state &= ~(SS_ISCONNECTED|SS_ISDISCONNECTING);
 3271         so->so_state |= SS_ISCONNECTING;
 3272         SOCK_UNLOCK(so);
 3273 }
 3274 
 3275 void
 3276 soisconnected(struct socket *so)
 3277 {
 3278         struct socket *head;
 3279         int ret;
 3280 
 3281 restart:
 3282         ACCEPT_LOCK();
 3283         SOCK_LOCK(so);
 3284         so->so_state &= ~(SS_ISCONNECTING|SS_ISDISCONNECTING|SS_ISCONFIRMING);
 3285         so->so_state |= SS_ISCONNECTED;
 3286         head = so->so_head;
 3287         if (head != NULL && (so->so_qstate & SQ_INCOMP)) {
 3288                 if ((so->so_options & SO_ACCEPTFILTER) == 0) {
 3289                         SOCK_UNLOCK(so);
 3290                         TAILQ_REMOVE(&head->so_incomp, so, so_list);
 3291                         head->so_incqlen--;
 3292                         so->so_qstate &= ~SQ_INCOMP;
 3293                         TAILQ_INSERT_TAIL(&head->so_comp, so, so_list);
 3294                         head->so_qlen++;
 3295                         so->so_qstate |= SQ_COMP;
 3296                         ACCEPT_UNLOCK();
 3297                         sorwakeup(head);
 3298                         wakeup_one(&head->so_timeo);
 3299                 } else {
 3300                         ACCEPT_UNLOCK();
 3301                         soupcall_set(so, SO_RCV,
 3302                             head->so_accf->so_accept_filter->accf_callback,
 3303                             head->so_accf->so_accept_filter_arg);
 3304                         so->so_options &= ~SO_ACCEPTFILTER;
 3305                         ret = head->so_accf->so_accept_filter->accf_callback(so,
 3306                             head->so_accf->so_accept_filter_arg, M_NOWAIT);
 3307                         if (ret == SU_ISCONNECTED)
 3308                                 soupcall_clear(so, SO_RCV);
 3309                         SOCK_UNLOCK(so);
 3310                         if (ret == SU_ISCONNECTED)
 3311                                 goto restart;
 3312                 }
 3313                 return;
 3314         }
 3315         SOCK_UNLOCK(so);
 3316         ACCEPT_UNLOCK();
 3317         wakeup(&so->so_timeo);
 3318         sorwakeup(so);
 3319         sowwakeup(so);
 3320 }
 3321 
 3322 void
 3323 soisdisconnecting(struct socket *so)
 3324 {
 3325 
 3326         /*
 3327          * Note: This code assumes that SOCK_LOCK(so) and
 3328          * SOCKBUF_LOCK(&so->so_rcv) are the same.
 3329          */
 3330         SOCKBUF_LOCK(&so->so_rcv);
 3331         so->so_state &= ~SS_ISCONNECTING;
 3332         so->so_state |= SS_ISDISCONNECTING;
 3333         so->so_rcv.sb_state |= SBS_CANTRCVMORE;
 3334         sorwakeup_locked(so);
 3335         SOCKBUF_LOCK(&so->so_snd);
 3336         so->so_snd.sb_state |= SBS_CANTSENDMORE;
 3337         sowwakeup_locked(so);
 3338         wakeup(&so->so_timeo);
 3339 }
 3340 
 3341 void
 3342 soisdisconnected(struct socket *so)
 3343 {
 3344 
 3345         /*
 3346          * Note: This code assumes that SOCK_LOCK(so) and
 3347          * SOCKBUF_LOCK(&so->so_rcv) are the same.
 3348          */
 3349         SOCKBUF_LOCK(&so->so_rcv);
 3350         so->so_state &= ~(SS_ISCONNECTING|SS_ISCONNECTED|SS_ISDISCONNECTING);
 3351         so->so_state |= SS_ISDISCONNECTED;
 3352         so->so_rcv.sb_state |= SBS_CANTRCVMORE;
 3353         sorwakeup_locked(so);
 3354         SOCKBUF_LOCK(&so->so_snd);
 3355         so->so_snd.sb_state |= SBS_CANTSENDMORE;
 3356         sbdrop_locked(&so->so_snd, so->so_snd.sb_cc);
 3357         sowwakeup_locked(so);
 3358         wakeup(&so->so_timeo);
 3359 }
 3360 
 3361 /*
 3362  * Make a copy of a sockaddr in a malloced buffer of type M_SONAME.
 3363  */
 3364 struct sockaddr *
 3365 sodupsockaddr(const struct sockaddr *sa, int mflags)
 3366 {
 3367         struct sockaddr *sa2;
 3368 
 3369         sa2 = malloc(sa->sa_len, M_SONAME, mflags);
 3370         if (sa2)
 3371                 bcopy(sa, sa2, sa->sa_len);
 3372         return sa2;
 3373 }
 3374 
 3375 /*
 3376  * Register per-socket buffer upcalls.
 3377  */
 3378 void
 3379 soupcall_set(struct socket *so, int which,
 3380     int (*func)(struct socket *, void *, int), void *arg)
 3381 {
 3382         struct sockbuf *sb;
 3383 
 3384         switch (which) {
 3385         case SO_RCV:
 3386                 sb = &so->so_rcv;
 3387                 break;
 3388         case SO_SND:
 3389                 sb = &so->so_snd;
 3390                 break;
 3391         default:
 3392                 panic("soupcall_set: bad which");
 3393         }
 3394         SOCKBUF_LOCK_ASSERT(sb);
 3395 #if 0
 3396         /* XXX: accf_http actually wants to do this on purpose. */
 3397         KASSERT(sb->sb_upcall == NULL, ("soupcall_set: overwriting upcall"));
 3398 #endif
 3399         sb->sb_upcall = func;
 3400         sb->sb_upcallarg = arg;
 3401         sb->sb_flags |= SB_UPCALL;
 3402 }
 3403 
 3404 void
 3405 soupcall_clear(struct socket *so, int which)
 3406 {
 3407         struct sockbuf *sb;
 3408 
 3409         switch (which) {
 3410         case SO_RCV:
 3411                 sb = &so->so_rcv;
 3412                 break;
 3413         case SO_SND:
 3414                 sb = &so->so_snd;
 3415                 break;
 3416         default:
 3417                 panic("soupcall_clear: bad which");
 3418         }
 3419         SOCKBUF_LOCK_ASSERT(sb);
 3420         KASSERT(sb->sb_upcall != NULL, ("soupcall_clear: no upcall to clear"));
 3421         sb->sb_upcall = NULL;
 3422         sb->sb_upcallarg = NULL;
 3423         sb->sb_flags &= ~SB_UPCALL;
 3424 }
 3425 
 3426 /*
 3427  * Create an external-format (``xsocket'') structure using the information in
 3428  * the kernel-format socket structure pointed to by so.  This is done to
 3429  * reduce the spew of irrelevant information over this interface, to isolate
 3430  * user code from changes in the kernel structure, and potentially to provide
 3431  * information-hiding if we decide that some of this information should be
 3432  * hidden from users.
 3433  */
 3434 void
 3435 sotoxsocket(struct socket *so, struct xsocket *xso)
 3436 {
 3437 
 3438         xso->xso_len = sizeof *xso;
 3439         xso->xso_so = so;
 3440         xso->so_type = so->so_type;
 3441         xso->so_options = so->so_options;
 3442         xso->so_linger = so->so_linger;
 3443         xso->so_state = so->so_state;
 3444         xso->so_pcb = so->so_pcb;
 3445         xso->xso_protocol = so->so_proto->pr_protocol;
 3446         xso->xso_family = so->so_proto->pr_domain->dom_family;
 3447         xso->so_qlen = so->so_qlen;
 3448         xso->so_incqlen = so->so_incqlen;
 3449         xso->so_qlimit = so->so_qlimit;
 3450         xso->so_timeo = so->so_timeo;
 3451         xso->so_error = so->so_error;
 3452         xso->so_pgid = so->so_sigio ? so->so_sigio->sio_pgid : 0;
 3453         xso->so_oobmark = so->so_oobmark;
 3454         sbtoxsockbuf(&so->so_snd, &xso->so_snd);
 3455         sbtoxsockbuf(&so->so_rcv, &xso->so_rcv);
 3456         xso->so_uid = so->so_cred->cr_uid;
 3457 }
 3458 
 3459 
 3460 /*
 3461  * Socket accessor functions to provide external consumers with
 3462  * a safe interface to socket state
 3463  *
 3464  */
 3465 
 3466 void
 3467 so_listeners_apply_all(struct socket *so, void (*func)(struct socket *, void *),
 3468     void *arg)
 3469 {
 3470 
 3471         TAILQ_FOREACH(so, &so->so_comp, so_list)
 3472                 func(so, arg);
 3473 }
 3474 
 3475 struct sockbuf *
 3476 so_sockbuf_rcv(struct socket *so)
 3477 {
 3478 
 3479         return (&so->so_rcv);
 3480 }
 3481 
 3482 struct sockbuf *
 3483 so_sockbuf_snd(struct socket *so)
 3484 {
 3485 
 3486         return (&so->so_snd);
 3487 }
 3488 
 3489 int
 3490 so_state_get(const struct socket *so)
 3491 {
 3492 
 3493         return (so->so_state);
 3494 }
 3495 
 3496 void
 3497 so_state_set(struct socket *so, int val)
 3498 {
 3499 
 3500         so->so_state = val;
 3501 }
 3502 
 3503 int
 3504 so_options_get(const struct socket *so)
 3505 {
 3506 
 3507         return (so->so_options);
 3508 }
 3509 
 3510 void
 3511 so_options_set(struct socket *so, int val)
 3512 {
 3513 
 3514         so->so_options = val;
 3515 }
 3516 
 3517 int
 3518 so_error_get(const struct socket *so)
 3519 {
 3520 
 3521         return (so->so_error);
 3522 }
 3523 
 3524 void
 3525 so_error_set(struct socket *so, int val)
 3526 {
 3527 
 3528         so->so_error = val;
 3529 }
 3530 
 3531 int
 3532 so_linger_get(const struct socket *so)
 3533 {
 3534 
 3535         return (so->so_linger);
 3536 }
 3537 
 3538 void
 3539 so_linger_set(struct socket *so, int val)
 3540 {
 3541 
 3542         so->so_linger = val;
 3543 }
 3544 
 3545 struct protosw *
 3546 so_protosw_get(const struct socket *so)
 3547 {
 3548 
 3549         return (so->so_proto);
 3550 }
 3551 
 3552 void
 3553 so_protosw_set(struct socket *so, struct protosw *val)
 3554 {
 3555 
 3556         so->so_proto = val;
 3557 }
 3558 
 3559 void
 3560 so_sorwakeup(struct socket *so)
 3561 {
 3562 
 3563         sorwakeup(so);
 3564 }
 3565 
 3566 void
 3567 so_sowwakeup(struct socket *so)
 3568 {
 3569 
 3570         sowwakeup(so);
 3571 }
 3572 
 3573 void
 3574 so_sorwakeup_locked(struct socket *so)
 3575 {
 3576 
 3577         sorwakeup_locked(so);
 3578 }
 3579 
 3580 void
 3581 so_sowwakeup_locked(struct socket *so)
 3582 {
 3583 
 3584         sowwakeup_locked(so);
 3585 }
 3586 
 3587 void
 3588 so_lock(struct socket *so)
 3589 {
 3590 
 3591         SOCK_LOCK(so);
 3592 }
 3593 
 3594 void
 3595 so_unlock(struct socket *so)
 3596 {
 3597 
 3598         SOCK_UNLOCK(so);
 3599 }

Cache object: 42d87af0fb169cb273d79d9a7555917d


[ source navigation ] [ diff markup ] [ identifier search ] [ freetext search ] [ file search ] [ list types ] [ track identifier ]


This page is part of the FreeBSD/Linux Linux Kernel Cross-Reference, and was automatically generated using a modified version of the LXR engine.