Project

General

Profile

Bug #2803

HAMMER: Warning: UNDO area too small!

Added by ftigeot over 2 years ago. Updated over 2 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
VFS subsystem
Target version:
-
Start date:
03/11/2015
Due date:
% Done:

0%


Description

The kernel prints this message when mounting a ~20 TB HAMMER filesystem.

No special undo size was used with newfs_hammer and it didn't complain when it created the filesystem.

The system is running DragonFly 4.1 (current master).

History

#1 Updated by tkusumi over 2 years ago

  • Category set to VFS subsystem

I don't know details of the situation but maybe either should work ?
(A) is the straight forward way but (B) could be effective in certain cases.

(A) Specify larger ondisk undo fifo size by -u on newfs
-u undosize
Specify the size of the fixed UNDO/REDO FIFO. The undosize is
specified in bytes. By default 0.1% of the root volume's size is
used, with a reasonable minimum and a reasonable cap. The
UNDO/REDO FIFO is used to sequence meta-data out to the media for
instant crash recovery.

(B) Change size of on-memory lru list which prevents fs from consuming on-disk undo fifo if the same (offset, size) pair has recently been added to this lru. The size of lru is currently 1024 (not tunable) and this number seems to be some sort of heuristic.

I don't know if (B) is really that effective, but when the fs somehow happens to generate undos for the same (offset, size), having too small lru ends up generating duplicated on-disk undos which I think is not necessary and should be avoided. (by the way it actually uses rbtree to lookup for the cache instead of seeking the lru list, so having larger lru size isn't a big problem in terms of performance).

I actually made a patch for (B) that makes lru size tunable using sysctl about a month ago but never really tested.

---
commit 50e302b13040146bc9756ce2277c5e68c399d452
Author: Tomohiro Kusumi <xxx>
Date: Mon Feb 16 00:14:24 2015 +0900

.

diff --git a/sys/vfs/hammer/hammer.h b/sys/vfs/hammer/hammer.h
index 0921035..be118ce 100644
--- a/sys/vfs/hammer/hammer.h
+++ b/sys/vfs/hammer/hammer.h
@@ -826,10 +826,7 @@ typedef struct hammer_reserve *hammer_reserve_t;
/*
* The undo structure tracks recent undos to avoid laying down duplicate
* undos within a flush group, saving us a significant amount of overhead.
- *
- * This is strictly a heuristic.
*/
-#define HAMMER_MAX_UNDOS 1024
#define HAMMER_MAX_FLUSHERS 4

struct hammer_undo {
@@ -942,7 +939,7 @@ struct hammer_mount {
struct hammer_lock snapshot_lock;
struct hammer_lock volume_lock;
struct hammer_blockmap blockmap[HAMMER_MAX_ZONES];
- struct hammer_undo undos[HAMMER_MAX_UNDOS];
+ struct hammer_undo *undos;
int undo_alloc;
TAILQ_HEAD(, hammer_undo) undo_lru_list;
TAILQ_HEAD(, hammer_reserve) delay_list;
@@ -1065,6 +1062,7 @@ extern int hammer_double_buffer;
extern int hammer_btree_full_undo;
extern int hammer_yield_check;
extern int hammer_fsync_mode;
+extern int hammer_max_undos;
extern int hammer_autoflush;
extern int64_t hammer_contention_count;

diff --git a/sys/vfs/hammer/hammer_undo.c b/sys/vfs/hammer/hammer_undo.c
index 9e644a2..c48ab3f 100644
--- a/sys/vfs/hammer/hammer_undo.c
+++ b/sys/vfs/hammer/hammer_undo.c
@@ -443,7 +443,7 @@ hammer_enter_undo_history(hammer_mount_t hmp, hammer_off_t offset, int bytes)
node->bytes = bytes;
return(0);
}
- if (hmp->undo_alloc != HAMMER_MAX_UNDOS) {
+ if (hmp->undo_alloc < hammer_max_undos) {
node = &hmp->undos[hmp->undo_alloc++];
} else {
node = TAILQ_FIRST(&hmp->undo_lru_list);
diff --git a/sys/vfs/hammer/hammer_vfsops.c b/sys/vfs/hammer/hammer_vfsops.c
index 4e23479..d503f6c 100644
--- a/sys/vfs/hammer/hammer_vfsops.c
+++ b/sys/vfs/hammer/hammer_vfsops.c
@@ -115,6 +115,7 @@ int hammer_double_buffer;
int hammer_btree_full_undo = 1;
int hammer_yield_check = 16;
int hammer_fsync_mode = 3;
+int hammer_max_undos = 1024;
int64_t hammer_contention_count;
int64_t hammer_zone_limit;

@@ -287,6 +288,8 @@ SYSCTL_INT(_vfs_hammer, OID_AUTO, yield_check, CTLFLAG_RW,
&hammer_yield_check, 0, "");
SYSCTL_INT(_vfs_hammer, OID_AUTO, fsync_mode, CTLFLAG_RW,
&hammer_fsync_mode, 0, "");
+SYSCTL_INT(_vfs_hammer, OID_AUTO, max_undos, CTLFLAG_RW,
+ &hammer_max_undos, 0, "");

/* KTR_INFO_MASTER(hammer); */

@@ -496,6 +499,11 @@ hammer_vfs_mount(struct mount *mp, char *mntpt, caddr_t data,
hmp->snapshot_lock.refs = 1;
hmp->volume_lock.refs = 1;

+ hmp->undos = kmalloc(sizeof(*hmp->undos) * hammer_max_undos,
+ M_HAMMER, M_WAITOK | M_ZERO);
+
TAILQ_INIT(&hmp->delay_list);
TAILQ_INIT(&hmp->flush_group_list);
TAILQ_INIT(&hmp->objid_cache_list);
@@ -955,6 +963,7 @@ hammer_free_hmp(struct mount *mp)
kmalloc_destroy(&hmp->m_misc);
kmalloc_destroy(&hmp->m_inodes);
lwkt_reltoken(&hmp->fs_token);
+ kfree(hmp->undos, M_HAMMER);
kfree(hmp, M_HAMMER);
}

Also available in: Atom PDF