net: sock_def_readable() and friends RCU conversion
sk_callback_lock rwlock actually protects sk->sk_sleep pointer, so we need two atomic operations (and associated dirtying) per incoming packet. RCU conversion is pretty much needed : 1) Add a new structure, called "struct socket_wq" to hold all fields that will need rcu_read_lock() protection (currently: a wait_queue_head_t and a struct fasync_struct pointer). [Future patch will add a list anchor for wakeup coalescing] 2) Attach one of such structure to each "struct socket" created in sock_alloc_inode(). 3) Respect RCU grace period when freeing a "struct socket_wq" 4) Change sk_sleep pointer in "struct sock" by sk_wq, pointer to "struct socket_wq" 5) Change sk_sleep() function to use new sk->sk_wq instead of sk->sk_sleep 6) Change sk_has_sleeper() to wq_has_sleeper() that must be used inside a rcu_read_lock() section. 7) Change all sk_has_sleeper() callers to : - Use rcu_read_lock() instead of read_lock(&sk->sk_callback_lock) - Use wq_has_sleeper() to eventually wakeup tasks. - Use rcu_read_unlock() instead of read_unlock(&sk->sk_callback_lock) 8) sock_wake_async() is modified to use rcu protection as well. 9) Exceptions : macvtap, drivers/net/tun.c, af_unix use integrated "struct socket_wq" instead of dynamically allocated ones. They dont need rcu freeing. Some cleanups or followups are probably needed, (possible sk_callback_lock conversion to a spinlock for example...). Signed-off-by:Eric Dumazet <eric.dumazet@gmail.com> Signed-off-by:
David S. Miller <davem@davemloft.net>
Showing
- drivers/net/macvtap.c 9 additions, 4 deletionsdrivers/net/macvtap.c
- drivers/net/tun.c 12 additions, 9 deletionsdrivers/net/tun.c
- include/linux/net.h 9 additions, 5 deletionsinclude/linux/net.h
- include/net/af_unix.h 11 additions, 9 deletionsinclude/net/af_unix.h
- include/net/sock.h 19 additions, 19 deletionsinclude/net/sock.h
- net/atm/common.c 14 additions, 8 deletionsnet/atm/common.c
- net/core/sock.c 31 additions, 19 deletionsnet/core/sock.c
- net/core/stream.c 7 additions, 3 deletionsnet/core/stream.c
- net/dccp/output.c 6 additions, 4 deletionsnet/dccp/output.c
- net/iucv/af_iucv.c 7 additions, 4 deletionsnet/iucv/af_iucv.c
- net/phonet/pep.c 4 additions, 4 deletionsnet/phonet/pep.c
- net/phonet/socket.c 1 addition, 1 deletionnet/phonet/socket.c
- net/rxrpc/af_rxrpc.c 6 additions, 4 deletionsnet/rxrpc/af_rxrpc.c
- net/sctp/socket.c 1 addition, 1 deletionnet/sctp/socket.c
- net/socket.c 36 additions, 11 deletionsnet/socket.c
- net/unix/af_unix.c 8 additions, 9 deletionsnet/unix/af_unix.c
Loading
Please register or sign in to comment