q3k
/
linux
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
linux/lib
History
Oleg Nesterov a1fd3e24d8 percpu_rw_semaphore: reimplement to not block the readers unnecessarily 
Currently the writer does msleep() plus synchronize_sched() 3 times to
acquire/release the semaphore, and during this time the readers are
blocked completely.  Even if the "write" section was not actually started
or if it was already finished.

With this patch down_write/up_write does synchronize_sched() twice and
down_read/up_read are still possible during this time, just they use the
slow path.

percpu_down_write() first forces the readers to use rw_semaphore and
increment the "slow" counter to take the lock for reading, then it
takes that rw_semaphore for writing and blocks the readers.

Also.  With this patch the code relies on the documented behaviour of
synchronize_sched(), it doesn't try to pair synchronize_sched() with
barrier.

Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mikulas Patocka <mpatocka@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Cc: Anton Arapov <anton@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
 		2012-12-17 17:15:18 -08:00
..
lzo
…
mpi MPI: Fix compilation on MIPS with GCC 4.4 and newer 2012-11-23 18:57:17 +01:00
raid6 lib/raid6: fix sparse warnings in recovery functions 2012-05-28 14:10:22 +10:00
reed_solomon
…
xz XZ: Fix incorrect XZ_BUF_ERROR 2011-09-21 13:39:59 -07:00
zlib_deflate zlib: slim down zlib_deflate() workspace when possible 2011-03-22 17:44:17 -07:00
zlib_inflate
…
.gitignore X.509: Implement simple static OID registry 2012-10-08 13:50:18 +10:30
Kconfig X.509: Implement simple static OID registry 2012-10-08 13:50:18 +10:30
Kconfig.debug Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2012-12-13 12:00:02 -08:00
Kconfig.kgdb
…
Kconfig.kmemcheck
…
Makefile percpu_rw_semaphore: reimplement to not block the readers unnecessarily 2012-12-17 17:15:18 -08:00
argv_split.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
asn1_decoder.c ASN.1: Fix an indefinite length skip error 2012-12-05 11:27:39 +10:30
atomic64.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
atomic64_test.c atomic64_test: simplify the #ifdef for atomic64_dec_if_positive() test 2012-07-30 17:25:16 -07:00
audit.c audit: support the "standard" <asm-generic/unistd.h> 2011-05-04 14:41:28 -04:00
average.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
bcd.c usb/core: use bin2bcd() for bcdDevice in RH 2012-09-10 11:13:16 -07:00
bch.c lib: add shared BCH ECC library 2011-03-11 14:25:50 +00:00
bitmap.c propagate name change to comments in kernel source 2012-12-06 10:39:54 +01:00
bitrev.c
…
bsearch.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
btree.c btree: catch NULL value before it does harm 2012-06-07 14:43:55 -07:00
bug.c bugs, x86: Fix printk levels for panic, softlockups and stack dumps 2012-01-26 21:28:45 +01:00
build_OID_registry X.509: Implement simple static OID registry 2012-10-08 13:50:18 +10:30
bust_spinlocks.c
…
check_signature.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
checksum.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
clz_tab.c lib: Fix multiple definitions of clz_tab 2012-02-02 10:34:23 +11:00
cmdline.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
cordic.c Docs: wording: functions -> algorithm 2011-10-29 21:20:22 +02:00
cpu-notifier-error-inject.c cpu: rewrite cpu-notifier-error-inject module 2012-07-30 17:25:22 -07:00
cpu_rmap.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
cpumask.c bootmem: fix wrong call parameter for free_bootmem() 2012-12-11 17:22:28 -08:00
crc-ccitt.c
…
crc-itu-t.c
…
crc-t10dif.c
…
crc7.c
…
crc8.c lib: crc8: add new library module providing crc8 algorithm 2011-06-03 15:01:06 -04:00
crc16.c
…
crc32.c sections: fix const sections for crc32 table 2012-10-06 03:04:46 +09:00
crc32defs.h crc32: select an algorithm via Kconfig 2012-03-23 16:58:38 -07:00
ctype.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
debug_locks.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
debugobjects.c debugobjects: Fill_pool() returns void now 2012-04-18 13:38:48 +02:00
dec_and_lock.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
decompress.c lib/decompress.c add __init to decompress_method and data 2012-10-06 03:05:32 +09:00
decompress_bunzip2.c decompress_bunzip2: remove invalid vi modeline 2011-12-06 10:00:05 +01:00
decompress_inflate.c decompressors: check input size in decompress_inflate.c 2011-01-13 08:03:25 -08:00
decompress_unlzma.c treewide: Fix comment and string typo 'bufer' 2011-12-06 09:53:40 +01:00
decompress_unlzo.c unlzo: fix input buffer free 2012-01-12 20:13:13 -08:00
decompress_unxz.c Fix common misspellings 2011-03-31 11:26:23 -03:00
devres.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
digsig.c digsig: add hash size comparision on signature verification 2012-09-13 09:13:02 +08:00
div64.c lib: correct link to the original source for div64_u64 2012-06-28 11:51:39 +02:00
dma-debug.c lib/dma-debug.c: fix __hash_bucket_find() 2012-10-19 14:07:48 -07:00
dump_stack.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
dynamic_debug.c lib: dynamic_debug: use kbasename() 2012-12-17 17:15:17 -08:00
dynamic_queue_limits.c bql: Avoid possible inconsistent calculation. 2012-05-31 18:18:17 -04:00
earlycpio.c lib: Add early cpio decoder 2012-09-30 18:02:20 -07:00
extable.c
…
fault-inject.c fault-inject: avoid call to random32() if fault injection is disabled 2012-06-20 14:39:36 -07:00
fdt.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_ro.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_rw.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_strerror.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_sw.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_wip.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
find_last_bit.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
find_next_bit.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
flex_array.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
flex_proportions.c lib/flex_proportions.c: fix corruption of denominator in flexible proportions 2012-09-25 08:59:21 -07:00
gcd.c lib/gcd.c: prevent possible div by 0 2012-10-06 03:04:57 +09:00
gen_crc32table.c sections: fix const sections for crc32 table 2012-10-06 03:04:46 +09:00
genalloc.c genalloc: stop crashing the system when destroying a pool 2012-10-25 14:37:52 -07:00
halfmd4.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
hexdump.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
hweight.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
idr.c idr: rename MAX_LEVEL to MAX_IDR_LEVEL 2012-10-06 03:04:56 +09:00
inflate.c
…
int_sqrt.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
interval_tree.c mm: interval tree updates 2012-10-09 16:22:40 +09:00
interval_tree_test_main.c rbtree: add prio tree and interval tree tests 2012-10-09 16:22:39 +09:00
iomap.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
iomap_copy.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
iommu-helper.c The following text was taken from the original review request: 2012-03-24 10:24:31 -07:00
ioremap.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
irq_regs.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
is_single_threaded.c
…
jedec_ddr_data.c ddr: add LPDDR2 data from JESD209-2 2012-05-02 00:04:06 -07:00
kasprintf.c lib/kasprintf.c: use kmalloc_track_caller() to get accurate traces for kvasprintf 2012-10-11 08:50:15 +09:00
klist.c Revert "driver core: check start node in klist_iter_init_node" 2012-04-19 19:17:30 -07:00
kobject.c kobject: fix the uncorrect comment 2012-05-07 16:51:19 -07:00
kobject_uevent.c netlink: hide struct module parameter in netlink_kernel_create 2012-09-08 18:46:30 -04:00
kstrtox.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
kstrtox.h lib/kstrtox: common code between kstrto*() and simple_strto*() functions 2011-10-31 17:30:56 -07:00
lcm.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
libcrc32c.c
…
list_debug.c rcu: Fix broken strings in RCU's source code. 2012-07-06 06:01:49 -07:00
list_sort.c lib/list_sort: test: check element addresses 2010-10-26 16:52:19 -07:00
llist.c Disintegrate and delete asm/system.h 2012-03-28 15:58:21 -07:00
locking-selftest-hardirq.h
…
locking-selftest-mutex.h
…
locking-selftest-rlock-hardirq.h
…
locking-selftest-rlock-softirq.h
…
locking-selftest-rlock.h
…
locking-selftest-rsem.h
…
locking-selftest-softirq.h
…
locking-selftest-spin-hardirq.h
…
locking-selftest-spin-softirq.h
…
locking-selftest-spin.h
…
locking-selftest-wlock-hardirq.h
…
locking-selftest-wlock-softirq.h
…
locking-selftest-wlock.h
…
locking-selftest-wsem.h
…
locking-selftest.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
lru_cache.c lru_cache: use correct type in sizeof for allocation 2011-05-25 08:39:52 -07:00
md5.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
memory-notifier-error-inject.c memory: memory notifier error injection module 2012-07-30 17:25:22 -07:00
memweight.c string: introduce memweight() 2012-07-30 17:25:16 -07:00
nlattr.c netlink: add minlen validation for the new signed types 2012-08-30 13:11:46 -04:00
notifier-error-inject.c fault-injection: notifier error injection 2012-07-30 17:25:22 -07:00
notifier-error-inject.h fault-injection: notifier error injection 2012-07-30 17:25:22 -07:00
oid_registry.c X.509: Add utility functions to render OIDs as strings 2012-10-08 13:50:18 +10:30
pSeries-reconfig-notifier-error-inject.c powerpc: pSeries reconfig notifier error injection module 2012-07-30 17:25:22 -07:00
parser.c lib/parser.c: avoid overflow in match_number() 2012-10-06 03:04:56 +09:00
pci_iomap.c lib: add NO_GENERIC_PCI_IOPORT_MAP 2012-01-31 23:19:47 +02:00
percpu-rwsem.c percpu_rw_semaphore: reimplement to not block the readers unnecessarily 2012-12-17 17:15:18 -08:00
percpu_counter.c switch the protection of percpu_counter list to spinlock 2012-07-31 09:28:31 +04:00
plist.c lib/plist.c: make plist test announcements KERN_DEBUG 2012-10-06 03:04:58 +09:00
pm-notifier-error-inject.c PM: PM notifier error injection module 2012-07-30 17:25:22 -07:00
prio_heap.c
…
proportions.c locking, lib/proportions: Annotate prop_local_percpu::lock as raw 2011-09-13 11:11:50 +02:00
radix-tree.c radix-tree: fix contiguous iterator 2012-06-05 10:46:40 -07:00
random32.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
ratelimit.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
rational.c lib: Change mail address of Oskar Schirmer 2012-05-17 15:18:37 +02:00
rbtree.c rbtree: move augmented rbtree functionality to rbtree_augmented.h 2012-10-09 16:22:40 +09:00
rbtree_test.c rbtree: move augmented rbtree functionality to rbtree_augmented.h 2012-10-09 16:22:40 +09:00
reciprocal_div.c sch_red: Adaptative RED AQM 2011-12-08 19:52:43 -05:00
rwsem-spinlock.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
rwsem.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
scatterlist.c Merge branch 'for-3.7/core' of git://git.kernel.dk/linux-block 2012-10-11 09:04:23 +09:00
sha1.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
show_mem.c arch, mm: filter disallowed nodes from arch specific show_mem functions 2011-05-25 08:39:03 -07:00
smp_processor_id.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
sort.c
…
spinlock_debug.c lib/spinlock_debug: avoid livelock in do_raw_spin_lock() 2012-10-06 03:04:57 +09:00
stmp_device.c lib: add support for stmp-style devices 2012-04-20 23:27:08 +02:00
string.c The following text was taken from the original review request: 2012-03-24 10:24:31 -07:00
string_helpers.c lib/string_helpers.c: make arrays static 2012-05-29 16:22:32 -07:00
strncpy_from_user.c word-at-a-time: make the interfaces truly generic 2012-05-26 11:33:40 -07:00
strnlen_user.c lib: Fix generic strnlen_user for 32-bit big-endian machines 2012-05-27 20:59:46 -07:00
swiotlb.c swiotlb: Do not export swiotlb_bounce since there are no external consumers 2012-10-30 09:32:07 -04:00
syscall.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
test-kstrtox.c lib/test-kstrtox.c: mark const init data with __initconst instead of __initdata 2012-05-29 16:22:32 -07:00
textsearch.c textsearch: doc - fix spelling in lib/textsearch.c. 2011-01-24 23:33:30 -08:00
timerqueue.c The following text was taken from the original review request: 2012-03-24 10:24:31 -07:00
ts_bm.c
…
ts_fsm.c
…
ts_kmp.c
…
uuid.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
vsprintf.c sscanf: don't ignore field widths for numeric conversions 2012-12-17 17:15:18 -08:00