q3k
/
linux
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
linux/lib
History
Oleg Nesterov 9390ef0c85 percpu_rw_semaphore: kill ->writer_mutex, add ->write_ctr 
percpu_rw_semaphore->writer_mutex was only added to simplify the initial
rewrite, the only thing it protects is clear_fast_ctr() which otherwise
could be called by multiple writers.  ->rw_sem is enough to serialize the
writers.

Kill this mutex and add "atomic_t write_ctr" instead.  The writers
increment/decrement this counter, the readers check it is zero instead of
mutex_is_locked().

Move atomic_add(clear_fast_ctr(), slow_read_ctr) under down_write() to
avoid the race with other writers.  This is a bit sub-optimal, only the
first writer needs this and we do not need to exclude the readers at this
stage.  But this is simple, we do not want another internal lock until we
add more features.

And this speeds up the write-contended case.  Before this patch the racing
writers sleep in synchronize_sched_expedited() sequentially, with this
patch multiple synchronize_sched_expedited's can "overlap" with each
other.  Note: we can do more optimizations, this is only the first step.

Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Cc: Anton Arapov <anton@redhat.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Michal Marek <mmarek@suse.cz>
Cc: Mikulas Patocka <mpatocka@redhat.com>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
 		2012-12-17 17:15:18 -08:00
..
lzo
…
mpi MPI: Fix compilation on MIPS with GCC 4.4 and newer 2012-11-23 18:57:17 +01:00
raid6 lib/raid6: fix sparse warnings in recovery functions 2012-05-28 14:10:22 +10:00
reed_solomon
…
xz XZ: Fix incorrect XZ_BUF_ERROR 2011-09-21 13:39:59 -07:00
zlib_deflate zlib: slim down zlib_deflate() workspace when possible 2011-03-22 17:44:17 -07:00
zlib_inflate
…
.gitignore X.509: Implement simple static OID registry 2012-10-08 13:50:18 +10:30
Kconfig X.509: Implement simple static OID registry 2012-10-08 13:50:18 +10:30
Kconfig.debug Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2012-12-13 12:00:02 -08:00
Kconfig.kgdb
…
Kconfig.kmemcheck
…
Makefile percpu_rw_semaphore: reimplement to not block the readers unnecessarily 2012-12-17 17:15:18 -08:00
argv_split.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
asn1_decoder.c ASN.1: Fix an indefinite length skip error 2012-12-05 11:27:39 +10:30
atomic64.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
atomic64_test.c atomic64_test: simplify the #ifdef for atomic64_dec_if_positive() test 2012-07-30 17:25:16 -07:00
audit.c audit: support the "standard" <asm-generic/unistd.h> 2011-05-04 14:41:28 -04:00
average.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
bcd.c usb/core: use bin2bcd() for bcdDevice in RH 2012-09-10 11:13:16 -07:00
bch.c lib: add shared BCH ECC library 2011-03-11 14:25:50 +00:00
bitmap.c propagate name change to comments in kernel source 2012-12-06 10:39:54 +01:00
bitrev.c
…
bsearch.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
btree.c btree: catch NULL value before it does harm 2012-06-07 14:43:55 -07:00
bug.c bugs, x86: Fix printk levels for panic, softlockups and stack dumps 2012-01-26 21:28:45 +01:00
build_OID_registry X.509: Implement simple static OID registry 2012-10-08 13:50:18 +10:30
bust_spinlocks.c
…
check_signature.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
checksum.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
clz_tab.c lib: Fix multiple definitions of clz_tab 2012-02-02 10:34:23 +11:00
cmdline.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
cordic.c Docs: wording: functions -> algorithm 2011-10-29 21:20:22 +02:00
cpu-notifier-error-inject.c cpu: rewrite cpu-notifier-error-inject module 2012-07-30 17:25:22 -07:00
cpu_rmap.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
cpumask.c bootmem: fix wrong call parameter for free_bootmem() 2012-12-11 17:22:28 -08:00
crc-ccitt.c
…
crc-itu-t.c
…
crc-t10dif.c
…
crc7.c
…
crc8.c lib: crc8: add new library module providing crc8 algorithm 2011-06-03 15:01:06 -04:00
crc16.c
…
crc32.c sections: fix const sections for crc32 table 2012-10-06 03:04:46 +09:00
crc32defs.h crc32: select an algorithm via Kconfig 2012-03-23 16:58:38 -07:00
ctype.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
debug_locks.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
debugobjects.c debugobjects: Fill_pool() returns void now 2012-04-18 13:38:48 +02:00
dec_and_lock.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
decompress.c lib/decompress.c add __init to decompress_method and data 2012-10-06 03:05:32 +09:00
decompress_bunzip2.c decompress_bunzip2: remove invalid vi modeline 2011-12-06 10:00:05 +01:00
decompress_inflate.c decompressors: check input size in decompress_inflate.c 2011-01-13 08:03:25 -08:00
decompress_unlzma.c treewide: Fix comment and string typo 'bufer' 2011-12-06 09:53:40 +01:00
decompress_unlzo.c unlzo: fix input buffer free 2012-01-12 20:13:13 -08:00
decompress_unxz.c Fix common misspellings 2011-03-31 11:26:23 -03:00
devres.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
digsig.c digsig: add hash size comparision on signature verification 2012-09-13 09:13:02 +08:00
div64.c lib: correct link to the original source for div64_u64 2012-06-28 11:51:39 +02:00
dma-debug.c lib/dma-debug.c: fix __hash_bucket_find() 2012-10-19 14:07:48 -07:00
dump_stack.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
dynamic_debug.c lib: dynamic_debug: use kbasename() 2012-12-17 17:15:17 -08:00
dynamic_queue_limits.c bql: Avoid possible inconsistent calculation. 2012-05-31 18:18:17 -04:00
earlycpio.c lib: Add early cpio decoder 2012-09-30 18:02:20 -07:00
extable.c
…
fault-inject.c fault-inject: avoid call to random32() if fault injection is disabled 2012-06-20 14:39:36 -07:00
fdt.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_ro.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_rw.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_strerror.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_sw.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_wip.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
find_last_bit.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
find_next_bit.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
flex_array.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
flex_proportions.c lib/flex_proportions.c: fix corruption of denominator in flexible proportions 2012-09-25 08:59:21 -07:00
gcd.c lib/gcd.c: prevent possible div by 0 2012-10-06 03:04:57 +09:00
gen_crc32table.c sections: fix const sections for crc32 table 2012-10-06 03:04:46 +09:00
genalloc.c genalloc: stop crashing the system when destroying a pool 2012-10-25 14:37:52 -07:00
halfmd4.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
hexdump.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
hweight.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
idr.c idr: rename MAX_LEVEL to MAX_IDR_LEVEL 2012-10-06 03:04:56 +09:00
inflate.c
…
int_sqrt.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
interval_tree.c mm: interval tree updates 2012-10-09 16:22:40 +09:00
interval_tree_test_main.c rbtree: add prio tree and interval tree tests 2012-10-09 16:22:39 +09:00
iomap.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
iomap_copy.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
iommu-helper.c The following text was taken from the original review request: 2012-03-24 10:24:31 -07:00
ioremap.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
irq_regs.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
is_single_threaded.c
…
jedec_ddr_data.c ddr: add LPDDR2 data from JESD209-2 2012-05-02 00:04:06 -07:00
kasprintf.c lib/kasprintf.c: use kmalloc_track_caller() to get accurate traces for kvasprintf 2012-10-11 08:50:15 +09:00
klist.c Revert "driver core: check start node in klist_iter_init_node" 2012-04-19 19:17:30 -07:00
kobject.c kobject: fix the uncorrect comment 2012-05-07 16:51:19 -07:00
kobject_uevent.c netlink: hide struct module parameter in netlink_kernel_create 2012-09-08 18:46:30 -04:00
kstrtox.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
kstrtox.h lib/kstrtox: common code between kstrto*() and simple_strto*() functions 2011-10-31 17:30:56 -07:00
lcm.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
libcrc32c.c
…
list_debug.c rcu: Fix broken strings in RCU's source code. 2012-07-06 06:01:49 -07:00
list_sort.c lib/list_sort: test: check element addresses 2010-10-26 16:52:19 -07:00
llist.c Disintegrate and delete asm/system.h 2012-03-28 15:58:21 -07:00
locking-selftest-hardirq.h
…
locking-selftest-mutex.h
…
locking-selftest-rlock-hardirq.h
…
locking-selftest-rlock-softirq.h
…
locking-selftest-rlock.h
…
locking-selftest-rsem.h
…
locking-selftest-softirq.h
…
locking-selftest-spin-hardirq.h
…
locking-selftest-spin-softirq.h
…
locking-selftest-spin.h
…
locking-selftest-wlock-hardirq.h
…
locking-selftest-wlock-softirq.h
…
locking-selftest-wlock.h
…
locking-selftest-wsem.h
…
locking-selftest.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
lru_cache.c lru_cache: use correct type in sizeof for allocation 2011-05-25 08:39:52 -07:00
md5.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
memory-notifier-error-inject.c memory: memory notifier error injection module 2012-07-30 17:25:22 -07:00
memweight.c string: introduce memweight() 2012-07-30 17:25:16 -07:00
nlattr.c netlink: add minlen validation for the new signed types 2012-08-30 13:11:46 -04:00
notifier-error-inject.c fault-injection: notifier error injection 2012-07-30 17:25:22 -07:00
notifier-error-inject.h fault-injection: notifier error injection 2012-07-30 17:25:22 -07:00
oid_registry.c X.509: Add utility functions to render OIDs as strings 2012-10-08 13:50:18 +10:30
pSeries-reconfig-notifier-error-inject.c powerpc: pSeries reconfig notifier error injection module 2012-07-30 17:25:22 -07:00
parser.c lib/parser.c: avoid overflow in match_number() 2012-10-06 03:04:56 +09:00
pci_iomap.c lib: add NO_GENERIC_PCI_IOPORT_MAP 2012-01-31 23:19:47 +02:00
percpu-rwsem.c percpu_rw_semaphore: kill ->writer_mutex, add ->write_ctr 2012-12-17 17:15:18 -08:00
percpu_counter.c switch the protection of percpu_counter list to spinlock 2012-07-31 09:28:31 +04:00
plist.c lib/plist.c: make plist test announcements KERN_DEBUG 2012-10-06 03:04:58 +09:00
pm-notifier-error-inject.c PM: PM notifier error injection module 2012-07-30 17:25:22 -07:00
prio_heap.c
…
proportions.c locking, lib/proportions: Annotate prop_local_percpu::lock as raw 2011-09-13 11:11:50 +02:00
radix-tree.c radix-tree: fix contiguous iterator 2012-06-05 10:46:40 -07:00
random32.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
ratelimit.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
rational.c lib: Change mail address of Oskar Schirmer 2012-05-17 15:18:37 +02:00
rbtree.c rbtree: move augmented rbtree functionality to rbtree_augmented.h 2012-10-09 16:22:40 +09:00
rbtree_test.c rbtree: move augmented rbtree functionality to rbtree_augmented.h 2012-10-09 16:22:40 +09:00
reciprocal_div.c sch_red: Adaptative RED AQM 2011-12-08 19:52:43 -05:00
rwsem-spinlock.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
rwsem.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
scatterlist.c Merge branch 'for-3.7/core' of git://git.kernel.dk/linux-block 2012-10-11 09:04:23 +09:00
sha1.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
show_mem.c arch, mm: filter disallowed nodes from arch specific show_mem functions 2011-05-25 08:39:03 -07:00
smp_processor_id.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
sort.c
…
spinlock_debug.c lib/spinlock_debug: avoid livelock in do_raw_spin_lock() 2012-10-06 03:04:57 +09:00
stmp_device.c lib: add support for stmp-style devices 2012-04-20 23:27:08 +02:00
string.c The following text was taken from the original review request: 2012-03-24 10:24:31 -07:00
string_helpers.c lib/string_helpers.c: make arrays static 2012-05-29 16:22:32 -07:00
strncpy_from_user.c word-at-a-time: make the interfaces truly generic 2012-05-26 11:33:40 -07:00
strnlen_user.c lib: Fix generic strnlen_user for 32-bit big-endian machines 2012-05-27 20:59:46 -07:00
swiotlb.c swiotlb: Do not export swiotlb_bounce since there are no external consumers 2012-10-30 09:32:07 -04:00
syscall.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
test-kstrtox.c lib/test-kstrtox.c: mark const init data with __initconst instead of __initdata 2012-05-29 16:22:32 -07:00
textsearch.c textsearch: doc - fix spelling in lib/textsearch.c. 2011-01-24 23:33:30 -08:00
timerqueue.c The following text was taken from the original review request: 2012-03-24 10:24:31 -07:00
ts_bm.c
…
ts_fsm.c
…
ts_kmp.c
…
uuid.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
vsprintf.c sscanf: don't ignore field widths for numeric conversions 2012-12-17 17:15:18 -08:00