Project

General

Profile

Actions
Copy link

Bug #3106

closed

go from pkg install hanging when running tests

Added by driusan about 7 years ago. Updated about 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Vendor software
Target version:
6.2
Start date:
11/29/2017
Due date:
% Done:

0%

Estimated time:

Description

I reinstalled Dragonfly 5.0.1 and after re-installing Go with "pkg install go" (which installed go version 1.9) and running "go test" now seems to randomly freeze blocking for IO. It worked reliably on this hardware previously to this reinstall (it was running HAMMER2 when it was working, now it's running HAMMER. I don't think that's relevant because as far as I know Go uses /tmp which is tmpfs for its tests anyways. Unfortunately, I can't recall what version of Go was on it..)

I tried compiling 1.9.2 from source to see if it's something that's been fixed but just hasn't made it to dports yet, but there's enough tests in the go compiler that whenever I try at least one of them freezes and after a few minutes fails dumping a stack trace like the following:

goroutine 53 [IO wait, 2 minutes]:
internal/poll.runtime_pollWait(0x800788f20, 0x72, 0x1)
/home/driusan/go1.9.2/src/runtime/netpoll.go:173 +0x57
internal/poll.(*pollDesc).wait(0xc42040bb98, 0x72, 0xffffffffffffff01, 0x725b40, 0x7241e0)
/home/driusan/go1.9.2/src/internal/poll/fd_poll_runtime.go:85 +0xae
internal/poll.(*pollDesc).waitRead(0xc42040bb98, 0xc4204b4201, 0x200, 0x200)
/home/driusan/go1.9.2/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc42040bb80, 0xc4204b4200, 0x200, 0x200, 0x0, 0x0, 0x0)
/home/driusan/go1.9.2/src/internal/poll/fd_unix.go:126 +0x18a
os.(*File).read(0xc4203c4df0, 0xc4204b4200, 0x200, 0x200, 0x0, 0xc42047cde8, 0x4ce4cb)
/home/driusan/go1.9.2/src/os/file_unix.go:216 +0x4e
os.(*File).Read(0xc4203c4df0, 0xc4204b4200, 0x200, 0x200, 0x0, 0x0, 0x0)
/home/driusan/go1.9.2/src/os/file.go:103 +0x6d
bytes.(*Buffer).ReadFrom(0xc4200c4380, 0x725640, 0xc4203c4df0, 0x8007890c8, 0xc4200c4380, 0xc42047cf01)
/home/driusan/go1.9.2/src/bytes/buffer.go:209 +0x177
io.copyBuffer(0x725400, 0xc4200c4380, 0x725640, 0xc4203c4df0, 0x0, 0x0, 0x0, 0xc4200d00f0, 0x0, 0x0)
/home/driusan/go1.9.2/src/io/io.go:386 +0x2bb
io.Copy(0x725400, 0xc4200c4380, 0x725640, 0xc4203c4df0, 0x403663, 0xc4200aa000, 0xc42047cfb0)
/home/driusan/go1.9.2/src/io/io.go:362 +0x68
os/exec.(*Cmd).writerDescriptor.func1(0xc4200aa000, 0xc42047cfb0)
/home/driusan/go1.9.2/src/os/exec/exec.go:264 +0x4d
os/exec.(*Cmd).Start.func1(0xc4204f0580, 0xc42019d820)
/home/driusan/go1.9.2/src/os/exec/exec.go:380 +0x27
created by os/exec.(*Cmd).Start
/home/driusan/go1.9.2/src/os/exec/exec.go:379 +0x646
FAIL go/internal/gcimporter 180.018s

Which test freezes and eventually dies isn't deterministic as far as I can tell, but a stack trace like that with an IO timing out is pretty consistent. (Other threads also panic with timeouts but I've only included one here..)

I'm not sure if this bug report belongs with DragonFly or Go (or what details would be useful to help reproduce it), but since it's a (relatively) clean install of DragonFly with a go version installed from pkg, I thought I'd start here..

Files

Download all files
stderr (7.43 KB) stderr more stack traces of threads that seem to be deadlocked or somehow blocked driusan, 11/29/2017 07:38 PM
ktrace.out.xz (619 KB) ktrace.out.xz ktrace of a failing "go test" run. driusan, 11/30/2017 04:59 AM
Actions
Copy link
 #1

Updated by sepherosa about 7 years ago

Possible to get a ktrace?

On Thu, Nov 30, 2017 at 10:16 AM,
<> wrote:

Issue #3106 has been reported by driusan.

----------------------------------------
Bug #3106: go from pkg install hanging when running tests
http://bugs.dragonflybsd.org/issues/3106

  • Author: driusan
  • Status: New
  • Priority: Normal
  • Assignee:
  • Category: Vendor software
  • Target version: Latest stable
    ----------------------------------------
    I reinstalled Dragonfly 5.0.1 and after re-installing Go with "pkg install go" (which installed go version 1.9) and running "go test" now seems to randomly freeze blocking for IO. It worked reliably on this hardware previously to this reinstall (it was running HAMMER2 when it was working, now it's running HAMMER. I don't think that's relevant because as far as I know Go uses /tmp which is tmpfs for its tests anyways. Unfortunately, I can't recall what version of Go was on it..)

I tried compiling 1.9.2 from source to see if it's something that's been fixed but just hasn't made it to dports yet, but there's enough tests in the go compiler that whenever I try at least one of them freezes and after a few minutes fails dumping a stack trace like the following:

goroutine 53 [IO wait, 2 minutes]:
internal/poll.runtime_pollWait(0x800788f20, 0x72, 0x1)
/home/driusan/go1.9.2/src/runtime/netpoll.go:173 +0x57
internal/poll.(*pollDesc).wait(0xc42040bb98, 0x72, 0xffffffffffffff01, 0x725b40, 0x7241e0)
/home/driusan/go1.9.2/src/internal/poll/fd_poll_runtime.go:85 +0xae
internal/poll.(*pollDesc).waitRead(0xc42040bb98, 0xc4204b4201, 0x200, 0x200)
/home/driusan/go1.9.2/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc42040bb80, 0xc4204b4200, 0x200, 0x200, 0x0, 0x0, 0x0)
/home/driusan/go1.9.2/src/internal/poll/fd_unix.go:126 +0x18a
os.(*File).read(0xc4203c4df0, 0xc4204b4200, 0x200, 0x200, 0x0, 0xc42047cde8, 0x4ce4cb)
/home/driusan/go1.9.2/src/os/file_unix.go:216 +0x4e
os.(*File).Read(0xc4203c4df0, 0xc4204b4200, 0x200, 0x200, 0x0, 0x0, 0x0)
/home/driusan/go1.9.2/src/os/file.go:103 +0x6d
bytes.(*Buffer).ReadFrom(0xc4200c4380, 0x725640, 0xc4203c4df0, 0x8007890c8, 0xc4200c4380, 0xc42047cf01)
/home/driusan/go1.9.2/src/bytes/buffer.go:209 +0x177
io.copyBuffer(0x725400, 0xc4200c4380, 0x725640, 0xc4203c4df0, 0x0, 0x0, 0x0, 0xc4200d00f0, 0x0, 0x0)
/home/driusan/go1.9.2/src/io/io.go:386 +0x2bb
io.Copy(0x725400, 0xc4200c4380, 0x725640, 0xc4203c4df0, 0x403663, 0xc4200aa000, 0xc42047cfb0)
/home/driusan/go1.9.2/src/io/io.go:362 +0x68
os/exec.(*Cmd).writerDescriptor.func1(0xc4200aa000, 0xc42047cfb0)
/home/driusan/go1.9.2/src/os/exec/exec.go:264 +0x4d
os/exec.(*Cmd).Start.func1(0xc4204f0580, 0xc42019d820)
/home/driusan/go1.9.2/src/os/exec/exec.go:380 +0x27
created by os/exec.(*Cmd).Start
/home/driusan/go1.9.2/src/os/exec/exec.go:379 +0x646
FAIL go/internal/gcimporter 180.018s

Which test freezes and eventually dies isn't deterministic as far as I can tell, but a stack trace like that with an IO timing out is pretty consistent. (Other threads also panic with timeouts but I've only included one here..)

I'm not sure if this bug report belongs with DragonFly or Go (or what details would be useful to help reproduce it), but since it's a (relatively) clean install of DragonFly with a go version installed from pkg, I thought I'd start here..

--
You have received this notification because you have either subscribed to it, or are involved in it.
To change your notification preferences, please click here: http://bugs.dragonflybsd.org/my/account

--
Tomorrow Will Never Die

Actions
Copy link
 #2

Updated by driusan about 7 years ago

I can't attach a ktrace of the entire go toolchain build/test. I tried taking one of "ktrace go test -v" for my personal project written in Go where I initially encountered the problem since it's smaller but still fails with the same time out/stack trace as above in the panic, but it weighs in at 5.9Mb and there seems to be a 5Mb attachment limit.. I can attach it if there's some way to increase the limit by 1Mb, for now I've just attached stderr for the run I have a ktrace.out for as another slightly more detailed data point.

(This test suite took ~20 seconds on my previous install of DragonFly, now it's timing out after 10 minutes..)

Actions
Copy link
 #3

Updated by sepherosa about 7 years ago

Can you try attaching xz'ed ktrace output?

On Thu, Nov 30, 2017 at 11:48 AM,
<> wrote:

Issue #3106 has been updated by driusan.

File stderr added

I can't attach a ktrace of the entire go toolchain build/test. I tried taking one of "ktrace go test -v" for my personal project written in Go where I initially encountered the problem since it's smaller but still fails with the same time out/stack trace as above in the panic, but it weighs in at 5.9Mb and there seems to be a 5Mb attachment limit.. I can attach it if there's some way to increase the limit by 1Mb, for now I've just attached stderr for the run I have a ktrace.out for as another slightly more detailed data point.

(This test suite took ~20 seconds on my previous install of DragonFly, now it's timing out after 10 minutes..)

----------------------------------------
Bug #3106: go from pkg install hanging when running tests
http://bugs.dragonflybsd.org/issues/3106#change-13332

  • Author: driusan
  • Status: New
  • Priority: Normal
  • Assignee:
  • Category: Vendor software
  • Target version: Latest stable
    ----------------------------------------
    I reinstalled Dragonfly 5.0.1 and after re-installing Go with "pkg install go" (which installed go version 1.9) and running "go test" now seems to randomly freeze blocking for IO. It worked reliably on this hardware previously to this reinstall (it was running HAMMER2 when it was working, now it's running HAMMER. I don't think that's relevant because as far as I know Go uses /tmp which is tmpfs for its tests anyways. Unfortunately, I can't recall what version of Go was on it..)

I tried compiling 1.9.2 from source to see if it's something that's been fixed but just hasn't made it to dports yet, but there's enough tests in the go compiler that whenever I try at least one of them freezes and after a few minutes fails dumping a stack trace like the following:

goroutine 53 [IO wait, 2 minutes]:
internal/poll.runtime_pollWait(0x800788f20, 0x72, 0x1)
/home/driusan/go1.9.2/src/runtime/netpoll.go:173 +0x57
internal/poll.(*pollDesc).wait(0xc42040bb98, 0x72, 0xffffffffffffff01, 0x725b40, 0x7241e0)
/home/driusan/go1.9.2/src/internal/poll/fd_poll_runtime.go:85 +0xae
internal/poll.(*pollDesc).waitRead(0xc42040bb98, 0xc4204b4201, 0x200, 0x200)
/home/driusan/go1.9.2/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc42040bb80, 0xc4204b4200, 0x200, 0x200, 0x0, 0x0, 0x0)
/home/driusan/go1.9.2/src/internal/poll/fd_unix.go:126 +0x18a
os.(*File).read(0xc4203c4df0, 0xc4204b4200, 0x200, 0x200, 0x0, 0xc42047cde8, 0x4ce4cb)
/home/driusan/go1.9.2/src/os/file_unix.go:216 +0x4e
os.(*File).Read(0xc4203c4df0, 0xc4204b4200, 0x200, 0x200, 0x0, 0x0, 0x0)
/home/driusan/go1.9.2/src/os/file.go:103 +0x6d
bytes.(*Buffer).ReadFrom(0xc4200c4380, 0x725640, 0xc4203c4df0, 0x8007890c8, 0xc4200c4380, 0xc42047cf01)
/home/driusan/go1.9.2/src/bytes/buffer.go:209 +0x177
io.copyBuffer(0x725400, 0xc4200c4380, 0x725640, 0xc4203c4df0, 0x0, 0x0, 0x0, 0xc4200d00f0, 0x0, 0x0)
/home/driusan/go1.9.2/src/io/io.go:386 +0x2bb
io.Copy(0x725400, 0xc4200c4380, 0x725640, 0xc4203c4df0, 0x403663, 0xc4200aa000, 0xc42047cfb0)
/home/driusan/go1.9.2/src/io/io.go:362 +0x68
os/exec.(*Cmd).writerDescriptor.func1(0xc4200aa000, 0xc42047cfb0)
/home/driusan/go1.9.2/src/os/exec/exec.go:264 +0x4d
os/exec.(*Cmd).Start.func1(0xc4204f0580, 0xc42019d820)
/home/driusan/go1.9.2/src/os/exec/exec.go:380 +0x27
created by os/exec.(*Cmd).Start
/home/driusan/go1.9.2/src/os/exec/exec.go:379 +0x646
FAIL go/internal/gcimporter 180.018s

Which test freezes and eventually dies isn't deterministic as far as I can tell, but a stack trace like that with an IO timing out is pretty consistent. (Other threads also panic with timeouts but I've only included one here..)

I'm not sure if this bug report belongs with DragonFly or Go (or what details would be useful to help reproduce it), but since it's a (relatively) clean install of DragonFly with a go version installed from pkg, I thought I'd start here..

---Files--------------------------------
stderr (7.43 KB)

--
You have received this notification because you have either subscribed to it, or are involved in it.
To change your notification preferences, please click here: http://bugs.dragonflybsd.org/my/account

--
Tomorrow Will Never Die

Actions
Copy link
 #4

Updated by Anonymous about 7 years ago

When you reinstalled 5.0.1, did you make any changes to the install aside from switching from HAMMER2 to HAMMER?

Actions
Copy link
 #5

Updated by mneumann about 7 years ago

Have you tried master? Dillon fixed a pthread/libc bug, which lead to hangs in cargo (rust package manager):

https://gitweb.dragonflybsd.org/dragonfly.git/commit/e2caf0e77d5adc5ddec07633ec1c9ceaae464acf

Actions
Copy link
 #6

Updated by driusan about 7 years ago

It went from being a system that was upgraded from 4.5 to a fresh install of 5.0.1, so any changes that would have accumulated over the time it was installed, but nothing's configured differently other than going back to hammer1.

I've attached the ktrace. I don't know why I didn't think of compressing it before..

I'll try master tonight, "to avoid corruption when heavily threaded programs call fork()." sounds promising..

Actions
Copy link
 #7

Updated by driusan about 7 years ago

  • Status changed from New to Closed

Works after updating to master. Thanks!

Actions
Copy link

Also available in: Atom PDF