q3k
/
linux
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
linux/fs
History
David Chinner 2a82b8be8a [XFS] Concurrent Multi-File Data Streams 
In media spaces, video is often stored in a frame-per-file format. When
dealing with uncompressed realtime HD video streams in this format, it is
crucial that files do not get fragmented and that multiple files a placed
contiguously on disk.

When multiple streams are being ingested and played out at the same time,
it is critical that the filesystem does not cross the streams and
interleave them together as this creates seek and readahead cache miss
latency and prevents both ingest and playout from meeting frame rate
targets.

This patch set creates a "stream of files" concept into the allocator to
place all the data from a single stream contiguously on disk so that RAID
array readahead can be used effectively. Each additional stream gets
placed in different allocation groups within the filesystem, thereby
ensuring that we don't cross any streams. When an AG fills up, we select a
new AG for the stream that is not in use.

The core of the functionality is the stream tracking - each inode that we
create in a directory needs to be associated with the directories' stream.
Hence every time we create a file, we look up the directories' stream
object and associate the new file with that object.

Once we have a stream object for a file, we use the AG that the stream
object point to for allocations. If we can't allocate in that AG (e.g. it
is full) we move the entire stream to another AG. Other inodes in the same
stream are moved to the new AG on their next allocation (i.e. lazy
update).

Stream objects are kept in a cache and hold a reference on the inode.
Hence the inode cannot be reclaimed while there is an outstanding stream
reference. This means that on unlink we need to remove the stream
association and we also need to flush all the associations on certain
events that want to reclaim all unreferenced inodes (e.g. filesystem
freeze).

SGI-PV: 964469
SGI-Modid: xfs-linux-melb:xfs-kern:29096a

Signed-off-by: David Chinner <dgc@sgi.com>
Signed-off-by: Barry Naujok <bnaujok@sgi.com>
Signed-off-by: Donald Douwsma <donaldd@sgi.com>
Signed-off-by: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Tim Shimmin <tes@sgi.com>
Signed-off-by: Vlad Apostolov <vapo@sgi.com>
 		2007-07-14 15:40:53 +10:00
..
9p Detach sched.h from mm.h 2007-05-21 09:18:19 -07:00
adfs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
affs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
afs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
autofs Replace pid_t in autofs with struct pid reference 2007-05-11 08:29:36 -07:00
autofs4 Fix some coding-style errors in autofs 2007-05-11 08:29:36 -07:00
befs Remove SLAB_CTOR_CONSTRUCTOR 2007-05-17 05:23:04 -07:00
bfs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
cifs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
coda sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
configfs Detach sched.h from mm.h 2007-05-21 09:18:19 -07:00
cramfs mm: make read_cache_page synchronous 2007-05-07 12:12:51 -07:00
debugfs remove "struct subsystem" as it is no longer needed 2007-05-02 18:57:59 -07:00
devpts devpts: add fsnotify create event 2007-05-08 11:14:59 -07:00
dlm [DLM] dump more lock values 2007-07-09 08:24:13 +01:00
ecryptfs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
efs Remove SLAB_CTOR_CONSTRUCTOR 2007-05-17 05:23:04 -07:00
exportfs header cleaning: don't include smp_lock.h when not used 2007-05-08 11:15:07 -07:00
ext2 xip sendfile removal 2007-07-10 08:04:15 +02:00
ext3 sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
ext4 sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
fat sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
freevxfs freevxfs: possible null pointer dereference fix 2007-05-08 11:14:59 -07:00
fuse sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
gfs2 Merge git://git.kernel.org/pub/scm/linux/kernel/git/steve/gfs2-2.6-nmw 2007-07-10 13:56:13 -07:00
hfs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
hfsplus sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
hostfs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
hpfs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
hppfs [PATCH] Mark struct super_operations const 2007-02-12 09:48:47 -08:00
hugetlbfs shm: fix the filename of hugetlb sysv shared memory 2007-06-16 13:16:16 -07:00
isofs Remove SLAB_CTOR_CONSTRUCTOR 2007-05-17 05:23:04 -07:00
jbd fix file specification in comments 2007-05-09 08:58:16 +02:00
jbd2 fix file specification in comments 2007-05-09 08:58:16 +02:00
jffs2 sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
jfs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
lockd NLM: Fix sparse warnings 2007-05-14 19:33:46 -04:00
minix sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
msdos [PATCH] mark struct inode_operations const 2 2007-02-12 09:48:46 -08:00
ncpfs Detach sched.h from mm.h 2007-05-21 09:18:19 -07:00
nfs sendfile: convert nfs to using splice_read() 2007-07-10 08:04:14 +02:00
nfs_common [PATCH] nfs_common endianness annotations 2006-10-20 10:26:41 -07:00
nfsd pipe: change the ->pin() operation to ->confirm() 2007-07-10 08:04:15 +02:00
nls [PATCH] fs: make nls_cp936.c handle some U00XY characters and U20AC correctly 2006-12-07 08:39:46 -08:00
ntfs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
ocfs2 pipe: change the ->pin() operation to ->confirm() 2007-07-10 08:04:15 +02:00
openpromfs Remove SLAB_CTOR_CONSTRUCTOR 2007-05-17 05:23:04 -07:00
partitions [S390] fixed cdl-format detection. 2007-07-10 11:24:44 +02:00
proc sched: scheduler debugging, core 2007-07-09 18:52:00 +02:00
qnx4 sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
ramfs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
reiserfs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
romfs Remove SLAB_CTOR_CONSTRUCTOR 2007-05-17 05:23:04 -07:00
smbfs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
sysfs sysfs: fix race condition around sd->s_dentry, take#2 2007-06-12 16:08:47 -07:00
sysv sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
udf sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
ufs sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
vfat [PATCH] mark struct inode_operations const 3 2007-02-12 09:48:46 -08:00
xfs [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
Kconfig Remove obsolete fat_cvf help text 2007-05-09 08:58:15 +02:00
Kconfig.binfmt fs: Kill sh dependency for binfmt_flat. 2007-05-21 14:34:00 +09:00
Makefile signal/timer/event: eventfd core 2007-05-11 08:29:36 -07:00
aio.c signal/timer/event: KAIO eventfd support example 2007-05-11 08:29:37 -07:00
anon_inodes.c signal/timer/event fds: anonymous inode source 2007-05-11 08:29:36 -07:00
attr.c header cleaning: don't include smp_lock.h when not used 2007-05-08 11:15:07 -07:00
bad_inode.c sendfile: remove bad_sendfile() from bad_file_ops 2007-07-10 08:04:15 +02:00
binfmt_aout.c [PATCH] VFS: change struct file to use struct path 2006-12-08 08:28:41 -08:00
binfmt_elf.c Fix elf_core_dump() when writing arch specific notes (spu coredumps) 2007-07-06 10:23:43 -07:00
binfmt_elf_fdpic.c header cleaning: don't include smp_lock.h when not used 2007-05-08 11:15:07 -07:00
binfmt_em86.c header cleaning: don't include smp_lock.h when not used 2007-05-08 11:15:07 -07:00
binfmt_flat.c nommu: report correct errno in message 2007-06-08 17:23:32 -07:00
binfmt_misc.c Detach sched.h from mm.h 2007-05-21 09:18:19 -07:00
binfmt_script.c header cleaning: don't include smp_lock.h when not used 2007-05-08 11:15:07 -07:00
binfmt_som.c [PARISC] Fix fs/binfmt_som.c 2006-10-04 06:51:26 -06:00
bio.c unexport bio_{,un}map_user 2007-07-10 08:03:34 +02:00
block_dev.c sendfile: remove .sendfile from filesystems that use generic_file_sendfile() 2007-07-10 08:04:13 +02:00
buffer.c Fix "fs: convert core functions to zero_user_page" 2007-05-21 09:15:32 -07:00
char_dev.c [PATCH] remove protection of LANANA-reserved majors 2007-04-04 21:12:47 -07:00
compat.c optimize compat_core_sys_select() by a using stack space for small fd sets 2007-05-23 20:14:12 -07:00
compat_ioctl.c fix compat console unimap regression 2007-05-25 17:37:46 -07:00
dcache.c header cleaning: don't include smp_lock.h when not used 2007-05-08 11:15:07 -07:00
dcookies.c [PATCH] slab: remove kmem_cache_t 2006-12-07 08:39:25 -08:00
direct-io.c dio: remove bogus refcounting BUG_ON 2007-07-03 18:23:23 -07:00
dnotify.c [PATCH] VFS: change struct file to use struct path 2006-12-08 08:28:41 -08:00
dquot.c circular locking dependency found in QUOTA OFF 2007-05-17 05:23:05 -07:00
drop_caches.c [PATCH] remove invalidate_inode_pages() 2007-02-11 10:51:31 -08:00
eventfd.c eventfd use waitqueue lock ... 2007-05-18 13:09:34 -07:00
eventpoll.c epoll: move kfree inside ep_free 2007-05-15 08:54:00 -07:00
exec.c uselib: add missing MNT_NOEXEC check 2007-05-23 20:14:13 -07:00
fcntl.c [PATCH] fdtable: Make fdarray and fdsets equal in size 2006-12-10 09:57:22 -08:00
fifo.c Detach sched.h from mm.h 2007-05-21 09:18:19 -07:00
file.c [PATCH] fdtable: Provide free_fdtable() wrapper 2006-12-22 08:55:50 -08:00
file_table.c header cleaning: don't include smp_lock.h when not used 2007-05-08 11:15:07 -07:00
filesystems.c add filesystem subtype support 2007-05-08 11:15:01 -07:00
fs-writeback.c Write back inode data pages even when the inode itself is locked 2007-01-26 12:53:20 -08:00
generic_acl.c [PATCH] Generic infrastructure for acls 2006-09-29 09:18:24 -07:00
inode.c Remove SLAB_CTOR_CONSTRUCTOR 2007-05-17 05:23:04 -07:00
inotify.c Introduce a handy list_first_entry macro 2007-05-08 11:15:11 -07:00
inotify_user.c [PATCH] inotify: read return val fix 2007-02-12 09:48:28 -08:00
internal.h cleanup compat ioctl handling 2007-05-08 11:15:09 -07:00
ioctl.c vanishing ioctl handler debugging 2007-06-04 13:25:10 -07:00
ioprio.c [PATCH] pid: replace do/while_each_task_pid with do/while_each_pid_task 2007-02-12 09:48:32 -08:00
libfs.c fs/libfs.c: >80 columns line break fix 2007-05-09 06:44:57 +02:00
locks.c Remove SLAB_CTOR_CONSTRUCTOR 2007-05-17 05:23:04 -07:00
mbcache.c [PATCH] slab: remove kmem_cache_t 2006-12-07 08:39:25 -08:00
mpage.c consolidate generic_writepages and mpage_writepages 2007-05-11 08:29:35 -07:00
namei.c [PATCH] complete message queue auditing 2007-05-11 05:38:26 -04:00
namespace.c check privileges before setting mount propagation 2007-05-08 11:15:12 -07:00
nfsctl.c Remove obsolete #include <linux/config.h> 2006-06-30 19:25:36 +02:00
no-block.c [PATCH] BLOCK: Make it possible to disable the block layer [try #6] 2006-09-30 20:52:31 +02:00
open.c Remove suid/sgid bits on [f]truncate() 2007-05-08 20:10:00 -07:00
pipe.c pipe: add documentation and comments 2007-07-10 08:04:16 +02:00
pnode.c Introduce a handy list_first_entry macro 2007-05-08 11:15:11 -07:00
pnode.h [PATCH] rename struct namespace to struct mnt_namespace 2006-12-08 08:28:51 -08:00
posix_acl.c [PATCH] kmemdup: some users 2006-10-01 00:39:19 -07:00
quota.c circular locking dependency found in QUOTA OFF 2007-05-17 05:23:05 -07:00
quota_v1.c
…
quota_v2.c
…
read_write.c Remove remnants of sendfile() 2007-07-10 08:04:15 +02:00
read_write.h [PATCH] Remove readv/writev methods and use aio_read/aio_write instead 2006-10-01 00:39:28 -07:00
readdir.c ROUND_UP macro cleanup in fs/(select|compat|readdir).c 2007-05-08 11:15:09 -07:00
select.c Style fix in fs/select.c 2007-05-09 07:10:02 +02:00
seq_file.c Make common helpers for seq_files that work with list_heads 2007-07-10 17:51:13 -07:00
signalfd.c avoid spurious POLLIN returns in signalfd 2007-06-28 11:34:54 -07:00
splice.c pipe: add documentation and comments 2007-07-10 08:04:16 +02:00
stack.c [PATCH] fs/stack.c: Copy i_nlink after all other attributes are copied 2007-02-19 14:21:50 -08:00
stat.c header cleaning: don't include smp_lock.h when not used 2007-05-08 11:15:07 -07:00
super.c add filesystem subtype support 2007-05-08 11:15:01 -07:00
sync.c Introduce fixed sys_sync_file_range2() syscall, implement on PowerPC and ARM 2007-06-28 11:38:30 -07:00
timerfd.c timerfd use waitqueue lock ... 2007-05-18 13:09:34 -07:00
utimes.c Fix permission checking for the new utimensat() system call 2007-07-08 12:02:55 -07:00
xattr.c [PATCH] audit inode for all xattr syscalls 2007-05-11 05:38:26 -04:00
xattr_acl.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00