Hammer mirror copy causes extreme slowdown on new ssh connections
I ssh into the machine to do a mirror copy or mirror stream from a master PFS on a SATA drive to a newly created slave PFS on a different SATA drive (both drives are encrypted). During the mirror copy everything seems fine and the SSH session I'm in seems responsive enough. CPU usage as reported by top doesn't exceed 40%.
If I try to make a new SSH session right after the mirror copy starts it works normally. However, a few minutes after the mirror copy starts, any attempt to SSH takes a vey long time to connect. It appears that other types of network connections, like SMB are similarly affected. There is no activity on the box other than the mirror copy or stream. If I kill that, then things return to normal.
#2 Updated by t_dfbsd over 2 years ago
I should have said that the mirroring is occurring between two drives on the same machine, so it doesn't sound like the -b option would apply. Does the setting of splitsize only affect the initialization phase of the mirroring? The problem I'm seeing continues to occur well after the mirror has started.
#3 Updated by justin over 2 years ago
Hmm. I'm doing the exact same thing here on 3.4 (master -> slave, both drives in same machine) and not seeing the problem. I haven't modified anything from 'normal'. I don't have as many disks as you, though.
If you have the space for it - can you copy a large file, or large set of files, from one disk to the other? The problem happens a while after the hammer copy starts, so it's probably fine while the system figures out what to move in what parts, and then starts to slow down when the actual data transfer happens. If it's the data transit itself that causes the problem, a normal bulk copy from disk to disk should show the same symptoms.
Your SATA drives in your dmesg are coming up as daX devices - something that I thought would only happen with SCSI. Maybe I don't know what I'm talking about, though.
#6 Updated by t_dfbsd over 2 years ago
Just an update that this is still an issue on v188.8.131.520.g060fb-DEVELOPMENT. I was mirror copying a 4TB PFS from one SATA drive to another in the same machine. After running approximately 12 hours, I ran the command "hammer config" on a different volume and the SSH session I was in hung for over a minute. It eventually returned. While it was hung, I tried to open a new SSH session to the machine and that took over a minute to connect.