DragonFlyBSD

:dear dragonflyers!
:
:first of all: congratulations for such a great release!
:
:I tried hammer and it worked without a glitch,
:and I tried USB plugging deplugging, replugging
:and was impressed that the shaky USB implementation
:inherited from FreeBSD has finally been stabilized!
:
:So this is a mixture of a bug report, a success report and
:some ideas how to improve the user experience :)
:
:I tried the following:
:- plug in a USB stick
:- mount t msdos /dev/da0s1 /mnt/usbstick
: cd /mnt/usbstick
:- ls
:- unplug the stick
:... lot of warnings, but no panic
:- ls
:=> shows the old content, even if the stick isn't there any longer
:- dh
:=> shows /mnt/usbstick still mounted
:- plug in USB stick again
:=> surprise! now is da1 ! (same slot though)

Yah, because there are still references on da0.  Even though
    the filesystem has failed you still have to unmount it.
    (umount -f ought to work, theoretically).

:here I was able to unmount the "ghost" mount on /mnt/usbstick
:and to remount the "new" da1 under /mnt/usbstick again.
:
:however, I then tried to unplug the device while copying a larger
:file to it.
:no panic, but when I did
: ls /mnt/usbstick
:the system froze.
:
:I think it could be related with the fact that ls showed the old
:files on the previous test (some caching issue?).

Possibly.  More likely its because the buffer cache filled up
    with dirty buffers which couldn't be flushed because the
    device was pulled.

There still needs to be some work done in the that regard,
    like to try a few times and then discard the buffer.  There's
    a problem though in that the buffers represent modified meta-data
    that is synchronized with the mount.  If the kernel throws the
    buffers away the filesystem mount could become even more confused
    then just getting I/O errors.

:- would be silently* umounted when the device is plugged off,
: so ls should show nothing in the path

I think an auto-unmount of some sort would be a good idea.
    It would probably solve the remaining panics/lockups.

:- would put the processes trying to access the device into a
: waiting loop
:
:- would be silently* remounted if the same device** is plugged in
: again in the same slot within a certain timeframe (the 10 seconds)

Not possible.  Once the filesystem gets that many fatal errors
    due to the device being unplugged it is pretty much a lost cause.

If the filesystem were idle when the device got unplugged we might
    be able to swing it.  That's a big if.

:- would return errors to the waiting processes after the timeout
:
:This would be a cool way to "recover" from accidently interrupted
:connections or unreliable devices.

Yah, but I don't think it is in the cards.  There is no way
    for the kernel to know what buffers actually made it to the
    device or not.

:Just a thought. My experiences with C are very limited, so I do not
:qualify to implement such a beast, nor do I know how difficult it is
:or if it might break Posix.
:
:Whow, that one got longer than expected.
:
:Anyway: keep up the great work!
:
:Regards,
:Benny

They are all good ideas :-)  Just not so easy to implement.

-Matt

Matthew Dillon wrote:

This sounds to me like the umount command is waiting for the buffer to
flush while the buffer is not given up if the mount is in no proper
state. Is this what you are saying?

My naive approach is to let a forced umount blindly clear any related
buffers.

Well, I think it doesn't have to - in conjunction with the rsync based
changes to cp and mv (BTW I am only taking about storage devices, hard
disks, usb-sticks and the like in my posting, should have made this clear).

My naive idea is to let cp/mv simply resume by starting a new transfer
with only the differences, something like this

> cp starts
> device unplugged
> triggers umount -f, buffers cleared, sigwait to cp, locks mount point
> wait some secs
> device plugged
> triggers remount under same mount point
(if not possible: error, give up, release mount point)
> sighup (or some other signal) to cp
> cp resumes. i.e. starts copying the same things again, this time
behaving like rsync (I guess rsync solves incomplete transfers by
some kind of checksum / hashing)

Yeah, that was my fear :-). But since you are digging very deep anyway, it
could make a nice (sub-)project. Isn't the dragonfly project about proving
great ideas by cool implementations? Just to stimulate you ;-)

I had the impression that the reason why USB still isn't getting better in
FreeBSD is that nobody didn't really understood the the deeper internals
and since you just had fixed a lot of issues I thought you are in the topic
anyway and there could be a chance to fix the last remaining issues.

But if somebody who just wrote a filesystem tells me that this is not easy
to implement, this sounds like there is no chance to get somebody on such
work anytime soon...

Cheers,
Benny

If the disk layer ack'ed the block, it should be on stable storage,
right? Otherwise it isn't. To me that would mean that we just have to
write out all blocks in flight (those with errors) and the ones we
didn't even write yet. Of course that's only true for non-caching
media, but what can you do.

cheers
simon

benny wrote:

perhaps this one is not so far away :)
lets say the fs on both devices is hammer, then a cp could be thought of as
a simple mirroring (syncing) (and a mv as a cp with deletion on the
source). the only thing needed would be to be able to "mirror" a single
file and to "detach" the mirrored file / directory somehow from the master.
I don't know if it is possible to extend hammer that way though.

For other filesystems an rsync something would still be needed, but this
could be a start.

Regards,
Benny

benny wrote:

maybe a normal cross-device hammer-to-hammer cp could then be done like
this:
- create a pseudo fs on the target (marked with a special delete-me flag)
- sync the pseudo fs with the source
- transform it somehow into a regular entry (no pseudofs any longer)
- remove the special delete-me flag

if any interruption occurs, we got a pseudo fs entry with a special
delete-me flag that doesn't show up in normal operations and could be
removed by normal pruning

if the copying can be resumed, it would continue the mirroring until its
done

maybe single files could be handled on the target the same way
(pseudofs-root would be the filename and have some special flag an the file
in it)

just some food for thinking...I am sure you would come up with better
solutions..

benny

why not just use rsync?

cheers
simon

that depends on "who" should use rsync?

if you mean that the user could use rsync instead of cp
in the first place - then you are right

but that was the whole point of my suggestion - to give a better user
experience:
IMHO the user should not have to know and install a tool like rsync
(nor to know mirror-copy) only to be able to transfer data reliably and
resumeable from one device to another... I mean, it's such a basic thing
that one tool (cp) should do it.

if you mean that "cp" should use rsync as proposed - then there may be two
caveats against having it as part of the base:
- the license is GPL
- rsync is not maintained by you

so one might suggest a (partial) rewrite under BSD license. that could take
some time - the suggestion for "hammered" devices could serve as a starting
point.

cheers,
Benny

Under DF you can use cpdup.

I don't agree. Each tool does its job. cp does copy. rsync syncs
files. Anyways, I wonder how do you wind up with a partial copy? Do
you ^C it? I can't imagine any other situation. I believe that less
than 0.0001% uses of cp are trying to resume a copy.

cheers
simon

Simon 'corecode' Schubert wrote:

Well, if you are not in a server environment with perfect hardware and
restricted access, there are many situations where such things happens:

- defective (controllers | harddisks | cables | contacts)
- shaky slots
- fluctuation of current
- accidental removals of external drives and sticks by kids or while
travelling with your notebook

the real world is full of interrupted connections...

in this case I plugged it of to test the behaviour, but I had plenty of
situations where such a feature would have helped (mostly due to defective
harddisks).

cheers,
benny

Gergo Szakal wrote:

Thanks, I didn't know it. From a first view it looks like the only
missing piece would then be to make cpdup resumeable on a signal delivered
when the device is accessable again (remounted).

then it should be possible to gain the desired effect by
cpdup -o -s0 src dest

this could by aliased in shell for user comfort.

benny

:If the disk layer ack'ed the block, it should be on stable storage,
:right? Otherwise it isn't. To me that would mean that we just have to
:write out all blocks in flight (those with errors) and the ones we
:didn't even write yet. Of course that's only true for non-caching
:media, but what can you do.
:
:cheers
: simon

Nope.  The device is virtually guaranteed to ack the block before
    it reaches stable storage, particularly if it is a flash device.

-Matt
                    Matthew Dillon 
                    <dillon@backplane.com>