We currently get fairly poor behaviour with files which get many short
writes, such as system logs. This is because we end up with many tiny
data nodes, and the rbtree gets massive. None of these nodes are
actually obsolete, so they are counted as 'clean' space. Eraseblocks can
be entirely full of these nodes (which are REF_NORMAL instead of
REF_PRISTINE), and still they count entirely towards 'used_size' and the
eraseblocks can sit on the clean_list for a long time without being
picked for GC.
One way to alleviate this in the long term is to account REF_NORMAL
space separately from REF_PRISTINE space, rather than counting them both
towards used_size. Then these eraseblocks can be picked for GC and the
offending nodes will be garbage collected.
The short-term fix, though -- which probably makes sense even if we do
eventually implement the above -- is to merge these nodes as they're
written. When we write the last byte in a page, write the _whole_ page.
This obsoletes the earlier nodes in the page _immediately_ and we don't
even need to wait for the garbage collection to do it.
Original implementation from Ferenc Havasi <havasi@inf.u-szeged.hu>
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
When jffs2_sum_process_sum_data() found a JFFS2_NODETYPE_XATTR
which has duplicate xid and older version, an error was returned
without appropriate process.
In the result, mounting filesystem is failed.
This patch fix this problem. If jffs2_setup_xattr_datum() returned
-EEXIST, the caller marks this node as DIRTY_SPACE().
[1/2] jffs2-xattr-v5.2-01-fix-duplicate-xdatum.patch
Signed-off-by: KaiGai Kohei <kaigai@ak.jp.nec.com>
'#include <linux/list.h>' was added into xattr.h.
because 'struct list_head' is used in this header file.
[6/10] jffs2-xattr-v5.1-06-add_list.h.patch
Signed-off-by: KaiGai Kohei <kaigai@ak.jp.nec.com>
Remove jffs2_garbage_collect_xattr(c, ic).
jffs2_garbage_collect_xattr_datum/ref() are called from gc.c directly.
In original implementation, jffs2_garbage_collect_xattr(c, ic) returns
with holding a spinlock if 'ic' is inode_cache. But it returns after
releasing a spinlock if 'ic' is xattr_datum/ref.
It looks so confusable behavior. Thus, this patch makes caller manage
locking/unlocking.
[5/10] jffs2-xattr-v5.1-05-update_xattr_gc.patch
Signed-off-by: KaiGai Kohei <kaigai@ak.jp.nec.com>
This patch can reduce 4-byte of memory usage per inode_cache.
[4/10] jffs2-xattr-v5.1-04-remove_ilist_from_ic.patch
Signed-off-by: KaiGai Kohei <kaigai@ak.jp.nec.com>
Add a description about the c->xattr_sem read/write semaphore
into README.Locking.
[3/10] jffs2-xattr-v5.1-03-append_README.Locking.patch
Signed-off-by: KaiGai Kohei <kaigai@ak.jp.nec.com>
jffs2_acl_header, jffs2_acl_entry and jffs2_acl_entry_short were redefined
with using 'struct' instead of 'typedef' in kernel implementation.
[1/10] jffs2-xattr-v5.1-01-remove_typedef_kernel.patch
Signed-off-by: KaiGai Kohei <kaigai@ak.jp.nec.com>
This attached patches provide xattr support including POSIX-ACL and
SELinux support on JFFS2 (version.5).
There are some significant differences from previous version posted
at last December.
The biggest change is addition of EBS(Erase Block Summary) support.
Currently, both kernel and usermode utility (sumtool) can recognize
xattr nodes which have JFFS2_NODETYPE_XATTR/_XREF nodetype.
In addition, some bugs are fixed.
- A potential race condition was fixed.
- Unexpected fail when updating a xattr by same name/value pair was fixed.
- A bug when removing xattr name/value pair was fixed.
The fundamental structures (such as using two new nodetypes and exclusion
mechanism by rwsem) are unchanged. But most of implementation were reviewed
and updated if necessary.
Espacially, we had to change several internal implementations related to
load_xattr_datum() to avoid a potential race condition.
[1/2] xattr_on_jffs2.kernel.version-5.patch
[2/2] xattr_on_jffs2.utils.version-5.patch
Signed-off-by: KaiGai Kohei <kaigai@ak.jp.nec.com>
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Reduce the nr. of pointer dereferences in fs/jffs2/summary.c
Benefits:
- micro speed optimization due to fewer pointer derefs
- generated code is slightly smaller
- better readability
(The first two sound like a compiler problem but I'll go with the third. dwmw2).
Signed-off-by: Jesper Juhl <jesper.juhl@gmail.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
This file hasn't actually been used since the very early days of JFFS2
when Arjan was playing with compression methods. It can go now.
Signed-off-by: Domen Puncer <domen@coderock.org>
Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
Acked-by: Arjan van de Ven <arjan@infradead.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
It seems like there is a potential race in the function jffs2_do_setattr()
in the case when attributes of a symlink are updated. The symlink metadata
is read without having f->sem locked.
The following patch should fix the race.
Signed-off-by: Dmitry Bazhenov <atrey@emcraft.com>
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
For a while now, we've postponed CRC-checking of data nodes to be done
by the GC thread, instead of being done while the user is waiting for
mount to finish. The GC thread would iterate through all the inodes on
the system and check each of their data nodes. It would skip over inodes
which had already been used or were already being read in by
read_inode(), because their data nodes would have been examined anyway.
However, we could sometimes reach the end of the for-each-inode loop and
still have some unchecked space left, if an inode we'd skipped was
_still_ in the process of being read. This fixes that race by actually
waiting for read_inode() to finish rather than just moving on.
Thanks to Ladislav Michl for coming up with a reproducible test case and
helping to track it down.
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
this changes if() BUG(); constructs to BUG_ON() which is
cleaner, contains unlikely() and can better optimized away.
Signed-off-by: Eric Sesterhenn <snakebyte@gmx.de>
Signed-off-by: Adrian Bunk <bunk@stusta.de>
This is a conversion to make the various file_operations structs in fs/
const. Basically a regexp job, with a few manual fixups
The goal is both to increase correctness (harder to accidentally write to
shared datastructures) and reducing the false sharing of cachelines with
things that get dirty in .data (while .rodata is nicely read only and thus
cache clean)
Signed-off-by: Arjan van de Ven <arjan@infradead.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Semaphore to mutex conversion.
The conversion was generated via scripts, and the result was validated
automatically via a script as well.
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Cc: Eric Van Hensbergen <ericvh@ericvh.myip.org>
Cc: Robert Love <rml@tech9.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Neil Brown <neilb@cse.unsw.edu.au>
Cc: Trond Myklebust <trond.myklebust@fys.uio.no>
Cc: Dave Kleikamp <shaggy@austin.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Rewrap the overly long source code lines resulting from the previous
patch's addition of the slab cache flag SLAB_MEM_SPREAD. This patch
contains only formatting changes, and no function change.
Signed-off-by: Paul Jackson <pj@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Mark file system inode and similar slab caches subject to SLAB_MEM_SPREAD
memory spreading.
If a slab cache is marked SLAB_MEM_SPREAD, then anytime that a task that's
in a cpuset with the 'memory_spread_slab' option enabled goes to allocate
from such a slab cache, the allocations are spread evenly over all the
memory nodes (task->mems_allowed) allowed to that task, instead of favoring
allocation on the node local to the current cpu.
The following inode and similar caches are marked SLAB_MEM_SPREAD:
file cache
==== =====
fs/adfs/super.c adfs_inode_cache
fs/affs/super.c affs_inode_cache
fs/befs/linuxvfs.c befs_inode_cache
fs/bfs/inode.c bfs_inode_cache
fs/block_dev.c bdev_cache
fs/cifs/cifsfs.c cifs_inode_cache
fs/coda/inode.c coda_inode_cache
fs/dquot.c dquot
fs/efs/super.c efs_inode_cache
fs/ext2/super.c ext2_inode_cache
fs/ext2/xattr.c (fs/mbcache.c) ext2_xattr
fs/ext3/super.c ext3_inode_cache
fs/ext3/xattr.c (fs/mbcache.c) ext3_xattr
fs/fat/cache.c fat_cache
fs/fat/inode.c fat_inode_cache
fs/freevxfs/vxfs_super.c vxfs_inode
fs/hpfs/super.c hpfs_inode_cache
fs/isofs/inode.c isofs_inode_cache
fs/jffs/inode-v23.c jffs_fm
fs/jffs2/super.c jffs2_i
fs/jfs/super.c jfs_ip
fs/minix/inode.c minix_inode_cache
fs/ncpfs/inode.c ncp_inode_cache
fs/nfs/direct.c nfs_direct_cache
fs/nfs/inode.c nfs_inode_cache
fs/ntfs/super.c ntfs_big_inode_cache_name
fs/ntfs/super.c ntfs_inode_cache
fs/ocfs2/dlm/dlmfs.c dlmfs_inode_cache
fs/ocfs2/super.c ocfs2_inode_cache
fs/proc/inode.c proc_inode_cache
fs/qnx4/inode.c qnx4_inode_cache
fs/reiserfs/super.c reiser_inode_cache
fs/romfs/inode.c romfs_inode_cache
fs/smbfs/inode.c smb_inode_cache
fs/sysv/inode.c sysv_inode_cache
fs/udf/super.c udf_inode_cache
fs/ufs/super.c ufs_inode_cache
net/socket.c sock_inode_cache
net/sunrpc/rpc_pipe.c rpc_inode_cache
The choice of which slab caches to so mark was quite simple. I marked
those already marked SLAB_RECLAIM_ACCOUNT, except for fs/xfs, dentry_cache,
inode_cache, and buffer_head, which were marked in a previous patch. Even
though SLAB_RECLAIM_ACCOUNT is for a different purpose, it marks the same
potentially large file system i/o related slab caches as we need for memory
spreading.
Given that the rule now becomes "wherever you would have used a
SLAB_RECLAIM_ACCOUNT slab cache flag before (usually the inode cache), use
the SLAB_MEM_SPREAD flag too", this should be easy enough to maintain.
Future file system writers will just copy one of the existing file system
slab cache setups and tend to get it right without thinking.
Signed-off-by: Paul Jackson <pj@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The meaning of MS_VERBOSE is backwards; if the bit is set, it really means,
"don't be verbose". This is confusing and counter-intuitive.
In addition, there is also no way to set the MS_VERBOSE flag in the
mount(8) program in util-linux, but interesting, it does define options
which would do the right thing if MS_SILENT were defined, which
unfortunately we do not:
#ifdef MS_SILENT
{ "quiet", 0, 0, MS_SILENT }, /* be quiet */
{ "loud", 0, 1, MS_SILENT }, /* print out messages. */
#endif
So the obvious fix is to deprecate the use of MS_VERBOSE and replace it
with MS_SILENT.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Fix some bugs in mtd/jffs2 on 64bit platform.
The MEMGETBADBLOCK/MEMSETBADBLOCK ioctl are not listed in compat_ioctl.h.
And some variables in jffs2 are declared as uint32_t but used to hold
size_t values.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Cc: Thomas Gleixner <tglx@linutronix.de>
Acked-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Remove the "inline" keyword from a bunch of big functions in the kernel with
the goal of shrinking it by 30kb to 40kb
Signed-off-by: Arjan van de Ven <arjan@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Acked-by: Jeff Garzik <jgarzik@pobox.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
fs: Use <linux/capability.h> where capable() is used.
Signed-off-by: Randy Dunlap <rdunlap@xenotime.net>
Acked-by: Tim Schmielau <tim@physik3.uni-rostock.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
JFFS2 initialize f->sem mutex as "locked" in the slab constructor which is a
bug. Objects are freed with unlocked f->sem mutex. So, when they allocated
again, f->sem is unlocked because the slab cache constructor is not called for
them. The constructor is called only once when memory pages are allocated for
objects (namely, when the slab layer allocates new slabs). So, sometimes
'struct jffs2_inode_info' are allocated with unlocked f->sem, sometimes with
locked. This is a bug. Instead, initialize f->sem as unlocked in the
constructor. I.e., in the "constructed" state f->sem must be unlocked.
From: Keijiro Yano <keijiro_yano@yahoo.co.jp>
Acked-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Work around gcc-2.95.x macro expansion bug.
Cc: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This is the fs/ part of the big kfree cleanup patch.
Remove pointless checks for NULL prior to calling kfree() in fs/.
Signed-off-by: Jesper Juhl <jesper.juhl@gmail.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Some callers to block-layer commit_write function treat non-zero return as
error, notably the loopback mount driver sometimes used in conjunction with
JFFS2 on NAND flash for bad block avoidance, etc. Return zero for success
as do various other commit_write functions.
Signed-off-by: Todd Poynor <tpoynor@mvista.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
- assume wbuf may be of size which is not power of 2
- don't make strange assumption about not padding wbuf for DataFlash
- use wbuf = DataFlash page and eraseblock >= 8 Dataflash pages
From: Peter Menzebach <pm-mtd@mw-itcon.de>
Acked-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Simplify the debugging code further.
Update the TODO list
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Always keep valid data in reserved_size.
It did not cause problems, but the reservation code was unoptimal
when centralized summary was active or the size of the erase block
was very small.
Signed-off-by: Ferenc Havasi <havasi@inf.u-szeged.hu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Do the summary collection in the right place. If the device
was not writebuffered but had c->mtd->writev function
(e.g. blkmtd) the summary collector function was not called.
Signed-off-by: Ferenc Havasi <havasi@inf.u-szeged.hu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
The goal of summary is to speed up the mount time. Erase block summary (EBS)
stores summary information at the end of every (closed) erase block. It is
no longer necessary to scan all nodes separetly (and read all pages of them)
just read this "small" summary, where every information is stored which is
needed at mount time.
This summary information is stored in a JFFS2_FEATURE_RWCOMPAT_DELETE. During
the mount process if there is no summary info the orignal scan process will
be executed. EBS works with NAND and NOR flashes, too.
There is a user space tool called sumtool to generate this summary
information for a JFFS2 image.
Signed-off-by: Ferenc Havasi <havasi@inf.u-szeged.hu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Remove support for virtual blocks, which are build by
concatenation of multiple physical erase blocks.
For more information please read the MTD mailing list thread
"[PATCH] remove support for virtual blocks"
Signed-off-by: Ferenc Havasi <havasi@inf.u-szeged.hu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
When data starts from the beginning of NAND page, 'len' must be zero, not
c->wbuf_page.
Thanks to Zoltan Sogor for reporting this problem.
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
From: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Instead of building fragtree starting from node with the smallest version
number, start from the highest. This helps to avoid reading and checking
obsolete nodes.
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Replace the D1(printk()) style debugging with the new debug macros
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Move functions to read inodes into readinode.c
Move functions to handle fragtree and dentry lists into nodelist.[ch]
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Small comment cleanups. Remove a unused macro
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Rename functions to a name matching the functionality.
Remove stall debug code
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Various simplifiactions. printk format corrections.
Convert more code to use the new debug functions.
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
When JFFS22 is unable to read the root inode, the bad root inode object is not
freed and remains sticked in the jffs2_i slab cache. When we further try to
free the slab cache (e.g., on rmmod jffs2), slab allocator subsystem panics.
Fix this bug.
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
If debugging is disabled, define debugging functions as empty macros, instead
of using Dx() explicitly.
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
JFFS2 uses f->dents to store the pointer to the symlink target string (in case
the inode is symlink). This is somewhat ugly to use the same field for
different reasons. Introduce distinct field f->target for this purpose.
Note, f->fragtree, f->dents, f->target may probably be put in a union.
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Move debug functions into a seperate source file
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
I recently picked up my older work to remove unnecessary #includes of
sched.h, starting from a patch by Dave Jones to not include sched.h
from module.h. This reduces the number of indirect includes of sched.h
by ~300. Another ~400 pointless direct includes can be removed after
this disentangling (patch to follow later).
However, quite a few indirect includes need to be fixed up for this.
In order to feed the patches through -mm with as little disturbance as
possible, I've split out the fixes I accumulated up to now (complete for
i386 and x86_64, more archs to follow later) and post them before the real
patch. This way this large part of the patch is kept simple with only
adding #includes, and all hunks are independent of each other. So if any
hunk rejects or gets in the way of other patches, just drop it. My scripts
will pick it up again in the next round.
Signed-off-by: Tim Schmielau <tim@physik3.uni-rostock.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This patch removes prototypes for the generic_file_open and
generic_file_llseek functions.
Besides being superfluous because they are already present in fs.h, they
were also wrong because the actual functions aren't weak functions.
Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The current calling conventions for ->follow_link() are already fairly
complex.
What we have is
1) you can return -error; then you must release nameidata yourself
and ->put_link() will _not_ be called.
2) you can do nd_set_link(nd, ERR_PTR(-error)) and return 0
3) you can do nd_set_link(nd, path) and return 0
4) you can return 0 (after having moved nameidata yourself)
jffs2 follow_link() is broken - it has an exit where it returns
-EIO and leaks nameidata.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
In the rare case of failing to write the cleanmarker
the allocated node was not freed.
Pointed out by Forrest Zhao
Initial cleanup by Joern Engel
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
In case of a mount error locks might be uninitialized but
accessed by the resulting call to jffs2_kill_sb().
Signed-off-by: Artem B. Bityuckiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
We recently changed the method of collecting and sorting of
tmp_dnode objects to use a temporary RB-tree instead of a
temporary list. Rename function and update comments.
Signed-off-by: Artem B. Bityuckiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
It isn't _normal_ that we allow key collision in rbtrees,
but it does not matter as long as the two nodes with the same
version are together.
Signed-off-by: Artem B. Bityuckiy <dedekind@infradead.org>
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Use an rbtree instead of a simple linked list. We were wasting
an amazing amount of time in jffs2_add_tn_to_list().
Thanks to Artem Bityuckiy and Jarkko Jlavinen for noticing.
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Missed conversion in the swsusp cleanup.
Cc: Russell King <rmk@arm.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Do not call wait_for_completion, when the gc task is not there.
Prevent an oops when the gc thread was not started.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
First kill GC thread, then start clearing the internal structures
Signed-off-by: Artem B. Bityuckiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Fix the bug that caouses problems when compiling for NOR.
We read a newly erased block so we don't need to check ECC.
Define jffs2_is_writebuffered as zero if there is no wbuf.
Signed-off-by: Artem B. Bityuckiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
