DragonFlyBSD

:Hi,
:
:I'm doing a "cpdup /usr/src /hammer/src" and a "hammer snapshot
:/hammer/snap-%Y" at the same time. "hammer snapshot" takes as long as
:"cpdup". Indeed, it does not return until the "cpdup" is completed.
:
:I think it's a problem in "hammer sync". But taking a snapshot is
:useless if one has to wait until no more file activity happens (which is
:rarely the case on a server).
:
:Regards,
:
: Michael

It isn't waiting for the activity to stop, it is trying to sync out
    the caches.  It should be able to do that in parallel with running
    activity but /usr/src is only ~450MB or so and it is likely that
    a large chunk of the target copy will have been cached, depending
    on how much memory you have.

It is possible to query the last synchronized transaction id
    and generate a softlink based on that, without doing a new sync.
    This would give you a snapshot as-of 0-60 seconds ago verses 'now'.
    That is not usually what the user desires, though.

You'd need to copy more data, on the order of a few gigs, to measure
    how long it takes snapshot to stage out the caches.  There could be
    a bug there but I'm pretty sure it is coded properly.

Remember that hard drives cannot do 70MBytes/sec worth of random I/O,
    they can only do that rate when reading or writing large linear swaths.
    When you get into more random I/O the data rate will drop to
    5-15 MBytes/sec.

-Matt
                    Matthew Dillon 
                    <dillon@backplane.com>

Okay, I'm trying now with cpdup /usr /hammer/usr2 and a "cp 10GB-file
/hammer" at the same time. But I'm having trouble to delete the
/hammer/usr2 directory structure afterwards:

1. rm -rf /hammer/usr2
   /hammer/usr2/src/usr.bin/paste/paste.1 rename-after-copy failed: No
   such file or directory
   /hammer/usr2/src/usr.bin/paste/paste.c create (uid 0, euid 0) failed:
   No such file or directory
   rm: usr2/src/usr.bin: Directory not empty
   rm: usr2/src: Directory not empty
   rm: usr2/: Directory not empty

When I did the "cp 10GB-file" paired with the cpdup, "hammer snapshot"
completed after 7 seconds. But if I don't do a "cp", it takes much
longer (I've aborted it).

I've seen 180 MB/sec read rate of a 1 GB large file, when read twice
(the first read "only" reaches 100 MB/sec). Does that mean a large part
of the file is cached by hammer (I've only 1 GB main memory)? If that is
the case, reading a big file would destroy the cache-set, which might
include a lot of small and frequently used files.

Good to know! I run "bonnie++", but with a wrong setting so it generated
1 million files in one directory, which slowed down the whole benchmark
a bit ;-). I don't want to try out the same for UFS (I did a sort of
this a few month ago and it showed that UFS can't create so many files
in one directory). Is there a usable limit of files per directory in
HAMMER?

Regards,

Michael

:Okay, I'm trying now with cpdup /usr /hammer/usr2 and a "cp 10GB-file
:/hammer" at the same time. But I'm having trouble to delete the
:/hammer/usr2 directory structure afterwards:
:
: # rm -rf /hammer/usr2
: /hammer/usr2/src/usr.bin/paste/paste.1 rename-after-copy failed: No
:such file or directory
: /hammer/usr2/src/usr.bin/paste/paste.c create (uid 0, euid 0) failed:
:No such file or directory
: rm: usr2/src/usr.bin: Directory not empty
: rm: usr2/src: Directory not empty
: rm: usr2/: Directory not empty

There was some instability a few days ago, which was fixed in
    commit 60H on the evening of July 7th.  If your kernel is older then
    that you may have to update and newfs_hammer, then re-test.

Hammer is still a bit of a moving target but I'm almost done with
    the mirroring code.

-Matt

Ah great. Right now, I've /home/dcvs and /usr/src on hammer, so it's
unwise to newfs_hammer :). But I will try later.

Thanks a lot!

Regards,

Michael

:> There was some instability a few days ago, which was fixed in
:> commit 60H on the evening of July 7th. If your kernel is older then
:> that you may have to update and newfs_hammer, then re-test.
:
:Ah great. Right now, I've /home/dcvs and /usr/src on hammer, so it's
:unwise to newfs_hammer :). But I will try later.
:
:Thanks a lot!
:
:Regards,
:
: Michael

Heh.  It isn't released yet!  But I appreciate the testing.

The bug that was fixed was that parallel delete_range() iterations
    could get off in the deadlock recovery code and delete records
    beyond the scope of the requested deletion.  Parallel iterations
    happen all the time in the flusher, even if the test program is
    just doing things one at a time.

In anycase, if you are going to run with it the damage should stay
    contained in that area.  Just rename the directory to something else
    and ignore it for now :-)  If you could re-test w/ the latest code
    please do, and tell me if any new problems occur.

-Matt
                    Matthew Dillon 
                    <dillon@backplane.com>