Project

General

Profile

Actions

Bug #2263

closed

nginx loses/garbles data if buffering to disk

Added by matthiasr almost 13 years ago. Updated about 10 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
12/17/2011
Due date:
% Done:

0%

Estimated time:

Description

I recently moved my web server to DragonFly and noticed that for very large responses from a web app accessed through nginx via fastcgi_pass or proxy_pass parts of the response are lost. This happens in the middle of the response, at more or less random locations (more towards the end). It used to work fine on Debian Linux.

I suspect that this is related to nginx buffering large responses to disk. I have created a testcase (test file, although anything large enough would work; nginx config snippet, repeated test download script, see attachments). With proxy_buffers left at the default, the downloaded file2.txt differs every few attempts. With the buffer size cranked up it works reliably. Both variants work reliably on OS X.

Note that this is also almost a Heisenbug – it takes a lot more tries until something goes wrong while ktrace'ing. ktrace available at http://2pktfkt.de/dfly/nginx_ktrace.out (~300MB – also http://2pktfkt.de/dfly/nginx_ktrace.out.xz soon)


Files

file1.txt (586 KB) file1.txt test file matthiasr, 12/17/2011 10:52 AM
localhost.conf (293 Bytes) localhost.conf nginx snipped matthiasr, 12/17/2011 10:52 AM
test.sh (188 Bytes) test.sh repeated test download script matthiasr, 12/17/2011 10:52 AM
ktrace.out.xz (640 KB) ktrace.out.xz sudo ktrace -idp <nginx master process>; ./test.sh; sudo ktrace -c <nginx master process> matthiasr, 12/17/2011 10:55 AM
Actions #1

Updated by matthiasr almost 13 years ago

compressed ktrace attached.

Actions #2

Updated by matthiasr almost 13 years ago

It very much seems sendfile is the culprit – after setting 'sendfile off;' in the nginx config I could let the test run for >1h straight without problems.

Actions #3

Updated by jorisgio about 10 years ago

  • Description updated (diff)
  • Status changed from New to Resolved

Fixed in 799312a2b6e22a583d98633ea6b62146afa8eb45

Actions

Also available in: Atom PDF