Commit graph

26 commits

Author SHA1 Message Date
Artem Bityutskiy
91f2d53cd7 UBI: add layout volume information
Add more information about layout volume to make userspace tools
use the macros instead of constants. Also rename UBI_LAYOUT_VOL_ID
to make it consistent with other macros.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2008-01-25 16:41:25 +02:00
Artem Bityutskiy
4ccf8cffa9 UBI: add auto-resize feature
The problem: NAND flashes have different amount of initial bad physical
eraseblocks (marked as bad by the manufacturer). For example, for 256MiB
Samsung OneNAND flash there might be from 0 to 40 bad initial eraseblocks,
which is about 2%. When UBI is used as the base system, one needs to know
the exact amount of good physical eraseblocks, because this number is
needed to create the UBI image which is put to the devices during
production. But this number is not know, which forces us to use the
minimum number of good physical eraseblocks. And UBI additionally
reserves some percentage of physical eraseblocks for bad block handling
(default is 1%), so we have 1-3% of PEBs reserved at the end, depending
on the amount of initial bad PEBs. But it is desired to always have
1% (or more, depending on the configuration).

Solution: this patch adds an "auto-resize" flag to the volume table.
The volume which has the "auto-resize" flag will automatically be re-sized
(enlarged) on the first UBI initialization. UBI clears the flag when
the volume is re-sized. Only one volume may have the "auto-resize" flag.

So, the production UBI image may have one volume with "auto-resize"
flag set, and its size is automatically adjusted on the first boot
of the device.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2008-01-25 16:41:24 +02:00
Artem Bityutskiy
b9a06623d9 UBI: get rid of ubi_ltree_slab
This slab cache is not really needed since the number of objects
is low and the constructor does not make much sense because we
allocate oblects when doint I/O, which is way slower then allocation.

Suggested-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2008-01-25 16:41:24 +02:00
Artem Bityutskiy
e73f4459d9 UBI: add UBI devices reference counting
This is one more step on the way to "removable" UBI devices. It
adds reference counting for UBI devices. Every time a volume on
this device is opened - the device's refcount is increased. It
is also increased if someone is reading any sysfs file of this
UBI device or of one of its volumes.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-12-26 19:15:17 +02:00
Artem Bityutskiy
43f9b25a9c UBI: bugfix: protect from volume removal
When the WL worker is moving an LEB, the volume might go away
occasionally. UBI does not handle these situations correctly.

This patch introduces a new mutex which serializes wear-levelling
worker and the the 'ubi_wl_put_peb()' function. Now, if one puts
an LEB, and its PEB is being moved, it will wait on the mutex.
And because we unmap all LEBs when removing volumes, this will make
the volume remove function to wait while the LEB movement
finishes.

Below is an example of an oops which should be fixed by this patch:

Pid: 9167, comm: io_paral Not tainted (2.6.24-rc5-ubi-2.6.git #2)
EIP: 0060:[<f884a379>] EFLAGS: 00010246 CPU: 0
EIP is at prot_tree_del+0x2a/0x63 [ubi]
EAX: f39a90e0 EBX: 00000000 ECX: 00000000 EDX: 00000134
ESI: f39a90e0 EDI: f39a90e0 EBP: f2d55ddc ESP: f2d55dd4
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process io_paral (pid: 9167, ti=f2d54000 task=f72a8030 task.ti=f2d54000)
Stack: f39a95f8 ef6aae50 f2d55e08 f884a511 f88538e1 f884ecea 00000134 00000000
       f39a9604 f39a95f0 efea8280 00000000 f39a90e0 f2d55e40 f8847261 f8850c3c
       f884eaad 00000001 000000b9 00000134 00000172 000000b9 00000134 00000001
Call Trace:
 [<c0105227>] show_trace_log_lvl+0x1a/0x30
 [<c01052e2>] show_stack_log_lvl+0xa5/0xca
 [<c01053d6>] show_registers+0xcf/0x21b
 [<c0105648>] die+0x126/0x224
 [<c0119a62>] do_page_fault+0x27f/0x60d
 [<c037dd62>] error_code+0x72/0x78
 [<f884a511>] ubi_wl_put_peb+0xf0/0x191 [ubi]
 [<f8847261>] ubi_eba_unmap_leb+0xaf/0xcc [ubi]
 [<f8843c21>] ubi_remove_volume+0x102/0x1e8 [ubi]
 [<f8846077>] ubi_cdev_ioctl+0x22a/0x383 [ubi]
 [<c017d768>] do_ioctl+0x68/0x71
 [<c017d7c6>] vfs_ioctl+0x55/0x271
 [<c017da15>] sys_ioctl+0x33/0x52
 [<c0104152>] sysenter_past_esp+0x5f/0xa5
 =======================

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-12-26 19:15:16 +02:00
Artem Bityutskiy
d05c77a816 UBI: introduce volume refcounting
Add ref_count field to UBI volumes and remove weired "vol->removed"
field. This way things are better understandable and we do not have
to do whold show_attr operation under spinlock.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-12-26 19:15:16 +02:00
Artem Bityutskiy
c63a491d37 UBI: add some more comments
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-12-26 19:15:15 +02:00
Artem Bityutskiy
89b96b6929 UBI: improve internal interfaces
Pass volume description object to the EBA function which makes
more sense, and EBA function do not have to find the volume
description object by volume ID.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-12-26 19:15:15 +02:00
Artem Bityutskiy
3a8d464286 UBI: create ltree_entry slab on initialization
Since the ltree_entry slab cache is a global entity, which is
used by all UBI devices, it is more logical to create it on
module initialization time and destro on module exit time.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-12-26 19:15:14 +02:00
Artem Bityutskiy
393852ecfe UBI: add ubi_leb_map interface
The idea of this interface belongs to Adrian Hunter. The
interface is extremely useful when one has to have a guarantee
that an LEB will contain all 0xFFs even in case of an unclean
reboot. UBI does have an 'ubi_leb_erase()' call which may do
this, but it is stupid and ineffecient, because it flushes whole
queue. I should be re-worked to just be a pair of unmap,
map calls.

The user of the interfaci is UBIFS at the moment.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-12-26 19:15:14 +02:00
Artem Bityutskiy
94780d4de2 UBI: bugfix: allocate mandatory EBs first
First allocate the necessary eraseblocks, then the optional ones.
Otherwise it allocates all PEBs for bad EB handling, and fails
on then following EBA LEB allocation.

Reported-by: Adrian Hunter <ext-adrian.hunter@nokia.com>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-12-26 19:15:14 +02:00
Christoph Lameter
4ba9b9d0ba Slab API: remove useless ctor parameter and reorder parameters
Slab constructors currently have a flags parameter that is never used.  And
the order of the arguments is opposite to other slab functions.  The object
pointer is placed before the kmem_cache pointer.

Convert

        ctor(void *object, struct kmem_cache *s, unsigned long flags)

to

        ctor(struct kmem_cache *s, void *object)

throughout the kernel

[akpm@linux-foundation.org: coupla fixes]
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-17 08:42:45 -07:00
Artem Bityutskiy
cc5f4f2875 UBI: return correct error code
Fix the following warning:

drivers/mtd/ubi/eba.c: In function 'ubi_eba_init_scan':
drivers/mtd/ubi/eba.c:1116: warning: 'err' may be used uninitialized in this function

Pointed-to-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-10-14 13:10:21 +03:00
Artem Bityutskiy
e8823bd63d UBI: fix atomic LEB change problems
When the UBI device is nearly full, i.e. all LEBs are mapped, we have
only one spare LEB left - the one we reserved for WL purposes. Well,
I do not count the LEBs which were reserved for bad PEB handling -
suppose NOR flash for simplicity. If an "atomic LEB change operation"
is run, and the WL unit is moving a LEB, we have no spare LEBs to
finish the operation and fail, which is not good. Moreover, if there
are 2 or more simultanious "atomic LEB change" requests, only one of
them has chances to succeed, the other will fail with -ENOSPC. Not
good either.

This patch does 2 things:
1. Reserves one PEB for the "atomic LEB change" operation.
2. Serealize the operations so that only on of them may run
   at a time (by means of a mutex).

Pointed-to-by: Brijesh Singh <brijesh.s.singh@gmail.com>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-10-14 13:10:21 +03:00
Artem Bityutskiy
e88d6e10e5 UBI: do not use vmalloc on I/O path
Similar reason as in case of the previous patch: it causes
deadlocks if a filesystem with writeback support works on top
of UBI. So pre-allocate needed buffers when attaching MTD device.
We also need mutexes to protect the buffers, but they do not
cause much contantion because they are used in recovery, torture,
and WL copy routines, which are called seldom.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-10-14 13:10:21 +03:00
Artem Bityutskiy
33818bbb84 UBI: allocate memory with GFP_NOFS
Use GFP_NOFS flag when allocating memory on I/O path, because otherwise
we may deadlock the filesystem which works on top of us. We observed
the deadlocks with UBIFS. Example:

VFS->FS lock a lock->UBI->kmalloc()->VFS writeback->FS locks the same
lock again.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-10-14 13:10:20 +03:00
Paul Mundt
20c2df83d2 mm: Remove slab destructors from kmem_cache_create().
Slab destructors were no longer supported after Christoph's
c59def9f22 change. They've been
BUGs for both slab and slub, and slob never supported them
either.

This rips out support for the dtor pointer from kmem_cache_create()
completely and fixes up every single callsite in the kernel (there were
about 224, not including the slab allocator definitions themselves,
or the documentation references).

Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2007-07-20 10:11:58 +09:00
Artem Bityutskiy
84a9258062 UBI: fix bug in atomic_leb_change()
atomic_leb_change() is only allowed for dynamic volumes, so set
the volume type correctly.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-07-18 16:56:24 +03:00
Artem Bityutskiy
a443db48e0 UBI: bugfix in ubi_leb_change()
Do not call 'ubi_wl_put_peb()' if the LEB was unmapped.

Reported-by: Gabor Loki <loki@inf.u-szeged.hu>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-07-18 16:53:57 +03:00
Christoph Hellwig
3261ebd7d4 UBI: kill homegrown endian macros
Kill UBI's homegrown endianess handling and replace it with
the standard kernel endianess handling.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-07-18 16:53:49 +03:00
Artem Bityutskiy
92ad8f3750 UBI: use vmalloc for large buffers
UBI allocates temporary buffers of PEB size, which may be 256KiB.
Use vmalloc instead of kmalloc for such big temporary buffers.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2007-07-18 16:53:08 +03:00
Jeff Garzik
a6343afb6e drivers/*: mark variables with uninitialized_var()
Mark variables in drivers/* with uninitialized_var() if such a warning
appears, and analysis proves that the var is initialized properly on all
paths it is used.

Signed-off-by: Jeff Garzik <jeff@garzik.org>
2007-07-17 16:23:19 -04:00
Jeff Garzik
2ab934b8af drivers/mtd/ubi/eba: minor cleanup: tighten scope of a local var
Signed-off-by: Jeff Garzik <jeff@garzik.org>
2007-07-17 16:18:00 -04:00
Christoph Lameter
a35afb830f Remove SLAB_CTOR_CONSTRUCTOR
SLAB_CTOR_CONSTRUCTOR is always specified. No point in checking it.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Jens Axboe <jens.axboe@oracle.com>
Cc: Steven French <sfrench@us.ibm.com>
Cc: Michael Halcrow <mhalcrow@us.ibm.com>
Cc: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Cc: Miklos Szeredi <miklos@szeredi.hu>
Cc: Steven Whitehouse <swhiteho@redhat.com>
Cc: Roman Zippel <zippel@linux-m68k.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Dave Kleikamp <shaggy@austin.ibm.com>
Cc: Trond Myklebust <trond.myklebust@fys.uio.no>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Cc: Anton Altaparmakov <aia21@cantab.net>
Cc: Mark Fasheh <mark.fasheh@oracle.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Jan Kara <jack@ucw.cz>
Cc: David Chinner <dgc@sgi.com>
Cc: "David S. Miller" <davem@davemloft.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-05-17 05:23:04 -07:00
Christoph Lameter
50953fe9e0 slab allocators: Remove SLAB_DEBUG_INITIAL flag
I have never seen a use of SLAB_DEBUG_INITIAL.  It is only supported by
SLAB.

I think its purpose was to have a callback after an object has been freed
to verify that the state is the constructor state again?  The callback is
performed before each freeing of an object.

I would think that it is much easier to check the object state manually
before the free.  That also places the check near the code object
manipulation of the object.

Also the SLAB_DEBUG_INITIAL callback is only performed if the kernel was
compiled with SLAB debugging on.  If there would be code in a constructor
handling SLAB_DEBUG_INITIAL then it would have to be conditional on
SLAB_DEBUG otherwise it would just be dead code.  But there is no such code
in the kernel.  I think SLUB_DEBUG_INITIAL is too problematic to make real
use of, difficult to understand and there are easier ways to accomplish the
same effect (i.e.  add debug code before kfree).

There is a related flag SLAB_CTOR_VERIFY that is frequently checked to be
clear in fs inode caches.  Remove the pointless checks (they would even be
pointless without removeal of SLAB_DEBUG_INITIAL) from the fs constructors.

This is the last slab flag that SLUB did not support.  Remove the check for
unimplemented flags from SLUB.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-05-07 12:12:57 -07:00
Artem B. Bityutskiy
801c135ce7 UBI: Unsorted Block Images
UBI (Latin: "where?") manages multiple logical volumes on a single
flash device, specifically supporting NAND flash devices. UBI provides
a flexible partitioning concept which still allows for wear-levelling
across the whole flash device.

In a sense, UBI may be compared to the Logical Volume Manager
(LVM). Whereas LVM maps logical sector numbers to physical HDD sector
numbers, UBI maps logical eraseblocks to physical eraseblocks.

More information may be found at
http://www.linux-mtd.infradead.org/doc/ubi.html

Partitioning/Re-partitioning

  An UBI volume occupies a certain number of erase blocks. This is
  limited by a configured maximum volume size, which could also be
  viewed as the partition size. Each individual UBI volume's size can
  be changed independently of the other UBI volumes, provided that the
  sum of all volume sizes doesn't exceed a certain limit.

  UBI supports dynamic volumes and static volumes. Static volumes are
  read-only and their contents are protected by CRC check sums.

Bad eraseblocks handling

  UBI transparently handles bad eraseblocks. When a physical
  eraseblock becomes bad, it is substituted by a good physical
  eraseblock, and the user does not even notice this.

Scrubbing

  On a NAND flash bit flips can occur on any write operation,
  sometimes also on read. If bit flips persist on the device, at first
  they can still be corrected by ECC, but once they accumulate,
  correction will become impossible. Thus it is best to actively scrub
  the affected eraseblock, by first copying it to a free eraseblock
  and then erasing the original. The UBI layer performs this type of
  scrubbing under the covers, transparently to the UBI volume users.

Erase Counts

  UBI maintains an erase count header per eraseblock. This frees
  higher-level layers (like file systems) from doing this and allows
  for centralized erase count management instead. The erase counts are
  used by the wear-levelling algorithm in the UBI layer. The algorithm
  itself is exchangeable.

Booting from NAND

  For booting directly from NAND flash the hardware must at least be
  capable of fetching and executing a small portion of the NAND
  flash. Some NAND flash controllers have this kind of support. They
  usually limit the window to a few kilobytes in erase block 0. This
  "initial program loader" (IPL) must then contain sufficient logic to
  load and execute the next boot phase.

  Due to bad eraseblocks, which may be randomly scattered over the
  flash device, it is problematic to store the "secondary program
  loader" (SPL) statically. Also, due to bit-flips it may become
  corrupted over time. UBI allows to solve this problem gracefully by
  storing the SPL in a small static UBI volume.

UBI volumes vs. static partitions

  UBI volumes are still very similar to static MTD partitions:

    * both consist of eraseblocks (logical eraseblocks in case of UBI
      volumes, and physical eraseblocks in case of static partitions;
    * both support three basic operations - read, write, erase.

  But UBI volumes have the following advantages over traditional
  static MTD partitions:

    * there are no eraseblock wear-leveling constraints in case of UBI
      volumes, so the user should not care about this;
    * there are no bit-flips and bad eraseblocks in case of UBI volumes.

  So, UBI volumes may be considered as flash devices with relaxed
  restrictions.

Where can it be found?

  Documentation, kernel code and applications can be found in the MTD
  gits.

What are the applications for?

  The applications help to create binary flash images for two purposes: pfi
  files (partial flash images) for in-system update of UBI volumes, and plain
  binary images, with or without OOB data in case of NAND, for a manufacturing
  step. Furthermore some tools are/and will be created that allow flash content
  analysis after a system has crashed..

Who did UBI?

  The original ideas, where UBI is based on, were developed by Andreas
  Arnez, Frank Haverkamp and Thomas Gleixner. Josh W. Boyer and some others
  were involved too. The implementation of the kernel layer was done by Artem
  B. Bityutskiy. The user-space applications and tools were written by Oliver
  Lohmann with contributions from Frank Haverkamp, Andreas Arnez, and Artem.
  Joern Engel contributed a patch which modifies JFFS2 so that it can be run on
  a UBI volume. Thomas Gleixner did modifications to the NAND layer. Alexander
  Schmidt made some testing work as well as core functionality improvements.

Signed-off-by: Artem B. Bityutskiy <dedekind@linutronix.de>
Signed-off-by: Frank Haverkamp <haver@vnet.ibm.com>
2007-04-27 14:23:33 +03:00