Bug #844

fsx on NFS fails

Added by thomas.nikolajsen about 7 years ago. Updated about 5 years ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:-
Target version:-

Description

Running fsx (usr/src/test/stress/fsx) on NFS mount fails after ~15 seconds.

This using RELEASE or DEVELOPMENT for server and client.
Both UDP and TCP mounts fail.
Using dfly client on fbsd7 server gives same error.
Using fbsd7 client in dfly server works.

No problem experienced on other NFS use, though.

-thomas

History

#1 Updated by thomas.nikolajsen about 7 years ago

More details:
fsx params. tried: fsx/dotest and default.

No problems reported by fsx when testing UFS or MFS.

-thomas
-
fsx/dotest output for NFS-mount dir is:
truncating to largest ever: 0x2abbd1
truncating to largest ever: 0x3abe49
truncating to largest ever: 0x3b1956
truncating to largest ever: 0x3e2816
truncating to largest ever: 0x3e8ae7
READ BAD DATA: offset = 0x124078, size = 0x345ac
OFFSET GOOD BAD RANGE
0x15306e 0x4946 0x4845 0x 55b6
operation# (mod 256) for the bad data may be 72
LOG DUMP (867 total operations):
1(1 mod 256): MAPWRITE 0x2ba358 thru 0x2cc3fb (0x120a4 bytes)
..
867(99 mod 256): READ 0x124078 thru 0x158623 (0x345ac bytes) ***RRRR***
Correct content saved for comparison
(maybe hexdump "test.fsx.dat" vs "test.fsx.dat.fsxgood")

-
$ ls -l test.fsx.dat /tmp/test.fsx.dat.fsxgood
-rw-r--r-- 1 thomas wheel 3963049 Nov 22 07:43 /tmp/test.fsx.dat.fsxgood
-rw-r--r-- 1 thomas thomas 3963049 Nov 22 07:43 test.fsx.dat
$ cmp -l test.fsx.dat /tmp/test.fsx.dat.fsxgood | wc -l
36754
$ cmp -l test.fsx.dat /tmp/test.fsx.dat.fsxgood | head
1388655 110 111
1388656 105 106
1388657 110 111
1388658 1 2
1388659 110 111
1388660 232 233
1388661 110 111
1388662 273 274
1388663 110 111
1388664 161 162

#2 Updated by dillon about 7 years ago

:Thomas Nikolajsen <> added the comment:
:
:More details:
:fsx params. tried: fsx/dotest and default.
:
:No problems reported by fsx when testing UFS or MFS.
:
: -thomas
:-
:fsx/dotest output for NFS-mount dir is:
:truncating to largest ever: 0x2abbd1
:truncating to largest ever: 0x3abe49
:truncating to largest ever: 0x3b1956
:truncating to largest ever: 0x3e2816
:truncating to largest ever: 0x3e8ae7
:READ BAD DATA: offset =3D 0x124078, size =3D 0x345ac
:OFFSET GOOD BAD RANGE

I've got this on my plate but may not be able to look into it for
a while, so if someone else wants to track down what happened please
do!

-Matt

#3 Updated by tuxillo over 5 years ago

Hi,

I've run test/stress/fsx/dotest for more than an hour in HAMMER without a
problem. (It's a slow VM)

> sudo /tmp/fsx -c 100000 -l 4194304 -o 262144 -P /tmp/ test.fsx.dat
Chance of close/open is 1 in 100000
truncating to largest ever: 0x2abbd1
truncating to largest ever: 0x3abe49
truncating to largest ever: 0x3b1956
truncating to largest ever: 0x3e2816
truncating to largest ever: 0x3e8ae7
truncating to largest ever: 0x3f959f
.....
testcalls = 6981963
>

But running on NFS produces almost instant failure still:

> sudo /tmp/fsx -c 100000 -l 4194304 -o 262144 -P /tmp/ test.fsx.dat
Chance of close/open is 1 in 100000
truncating to largest ever: 0x2abbd1
truncating to largest ever: 0x3abe49
READ BAD DATA: offset = 0x30eea8, size = 0x13534
OFFSET GOOD BAD RANGE
0x313000 0x0000 0x6a10 0x e38b
operation# (mod 256) for the bad data may be 106
LOG DUMP (16 total operations):
1(1 mod 256): MAPWRITE 0x2ba358 thru 0x2cc3fb (0x120a4 bytes)
2(2 mod 256): MAPWRITE 0x2ce205 thru 0x2f4dff (0x26bfb bytes)
3(3 mod 256): READ 0x1c0d46 thru 0x1f8c40 (0x37efb bytes)
4(4 mod 256): MAPREAD 0x28f55 thru 0x3c96a (0x13a16 bytes)
5(5 mod 256): TRUNCATE DOWN from 0x2f4e00 to 0x2abbd1
6(6 mod 256): TRUNCATE DOWN from 0x2abbd1 to 0x13ccb2
7(7 mod 256): MAPREAD 0x61fd thru 0xf62b (0x942f bytes)
8(8 mod 256): MAPWRITE 0x1cf8ff thru 0x1ddc87 (0xe389 bytes)
9(9 mod 256): MAPWRITE 0x312c01 thru 0x3314c9 (0x1e8c9 bytes) ******WWWW
10(10 mod 256): TRUNCATE UP from 0x3314ca to 0x3abe49
11(11 mod 256): MAPREAD 0x28e90 thru 0x41e0d (0x18f7e bytes)
12(12 mod 256): MAPWRITE 0xac797 thru 0xbf2cb (0x12b35 bytes)
13(13 mod 256): TRUNCATE DOWN from 0x3abe49 to 0x4e872 ******WWWW
14(14 mod 256): MAPWRITE 0x2dedbf thru 0x311249 (0x3248b bytes)
15(15 mod 256): WRITE 0x34faf8 thru 0x3870be (0x375c7 bytes) HOLE ***WWWW
16(16 mod 256): READ 0x30eea8 thru 0x3223db (0x13534 bytes) ***RRRR***
Correct content saved for comparison
(maybe hexdump "test.fsx.dat" vs "test.fsx.dat.fsxgood")
> cmp -l test.fsx.dat /tmp/test.fsx.dat.fsxgood | head
3440641 11 0
3440642 336 0
3440643 11 0
3440644 340 0
3440645 11 0
3440646 123 0
3440647 11 0
3440648 12 0
3440649 11 0
3440650 373 0

#4 Updated by tuxillo over 5 years ago

Hi,

There's a commit for this on master, 28953d39b23eec3fbd96a46ca2c4946bfe8d1a61

#5 Updated by tuxillo over 5 years ago

As Dillon says, fsx stills fails on NFS mounts, just that it takes longer to get
there.

#6 Updated by tuxillo about 5 years ago

Dillon agreed this is solved.

Also available in: Atom PDF