DragonFlyBSD

:Happened when I started 8 mirror-stream at the same time.
:
:cheers
: simon

Yours are easier to track down :-)  Try the patch below.

Tomokazu's are more difficult.

-Matt
                    Matthew Dillon 
                    <dillon@backplane.com>

diff --git a/sys/vfs/hammer/hammer_mirror.c b/sys/vfs/hammer/hammer_mirror.c
index 574b74d..05e12a0 100644
--- a/sys/vfs/hammer/hammer_mirror.c
++ b/sys/vfs/hammer/hammer_mirror.c
@ -462,8 +462,10 @ hammer_ioc_mirror_write(hammer_transaction_t trans, hammer_inode_t ip,
*/
if (error EDEADLK) {
while (error EDEADLK) {
hammer_sync_lock_sh(trans);
hammer_recover_cursor(&cursor);
error = hammer_cursor_upgrade(&cursor);
+ hammer_sync_unlock(trans);
}
} else {
if (error == EALREADY)

Contents of patch applied to 2.4, no ill effects noted. With the PFSes
mostly-synchronized I didn't have enough data left to sync to test for the same
failure with multiple `mirror-copy`s running, but I'll be setting up parallel
mirror-streams shortly as required for my setup. (Is there a convenient way to
destroy a PFS and make sure it gets recreated with the same PFS number?)

I briefly thought the low I/O figures on da1 (mirroring from da0 to da1) might
have been a new quirk:

1. hammer mirror-copy /DATA/ /Mirror/pfs/DATA
   histogram range 0000000107e975cf - 000000010d6514f0
   Mirror-read: Mirror from 0000000000000002 to 000000010d6514f0
   Mirror-read /DATA/ succeeded

iostat -w1

tty ad0 da0 da1 cpu
tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id
2 465 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 4 0 10 1 85
0 80 0.00 0 0.00 64.00 643 40.21 64.00 4 0.25 21 0 42 2 35
0 80 0.00 0 0.00 63.82 683 42.59 60.00 4 0.23 28 0 37 3 32
0 80 0.00 0 0.00 63.79 614 38.26 64.00 6 0.37 20 0 38 1 41

...but it completed in appropriate time and all the data appears to be there, so
I'm not sure if HAMMER was being smart enough to skip over what existed from a
previous interrupted copy or what. I don't really remember whether the
pre-patch stats actually looked any different for the destination disk.

[SATA disks on a mpt0, write caches enabled.]

:I briefly thought the low I/O figures on da1 (mirroring from da0 to da1) might
:have been a new quirk:
:
:> iostat -w1
: tty ad0 da0 da1 cpu
: tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id
: 2 465 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 4 0 10 1 85
: 0 80 0.00 0 0.00 64.00 643 40.21 64.00 4 0.25 21 0 42 2 35
: 0 80 0.00 0 0.00 63.82 683 42.59 60.00 4 0.23 28 0 37 3 32
: 0 80 0.00 0 0.00 63.79 614 38.26 64.00 6 0.37 20 0 38 1 41
:
:...but it completed in appropriate time and all the data appears to be there, so
:I'm not sure if HAMMER was being smart enough to skip over what existed from a
:previous interrupted copy or what. I don't really remember whether the
:pre-patch stats actually looked any different for the destination disk.
:
:[SATA disks on a mpt0, write caches enabled.]

The more likely case is that the mirror stream repeated the data but
    since the target already had it there was no need to re-write it on
    the target.  In that case the only disk activity on the target will
    be accessing and comparing the meta-data.

It tries to stage the mirroring stream so it can pick up where it
    left off but it's a bit of a hack.  If it managed to complete a
    batch then it certainly will pick up after the completion and not
    resend the whole batch.  I need to redo the code to actually
    calculate the batch size and split things up based on that instead
    of splitting things up based on B-Tree key ranges.

This applies to initial mirroring only, of course.  Once the initial
    copy is finished all later mirroring runs will be incremental.

-Matt
                    Matthew Dillon 
                    <dillon@backplane.com>