Bug #1998

kqueue issues with nginx

Added by sjg over 3 years ago. Updated about 3 years ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:sjg% Done:

0%

Category:-
Target version:-

Description

When nginx is used to reverse proxy requests for php pages to lighttpd, some
requests stall having only served 3316 bytes.

test-kqueue.c Magnifier (1.02 KB) ano, 09/07/2011 07:18 AM

History

#1 Updated by pavalos over 3 years ago

Try a simple config like:
server {
listen 8001;
location / {
proxy_set_header Host www.airwarriors.com;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://www.airwarriors.com:81;
proxy_redirect http://www.airwarriors.com:81/ http://www.airwarriors.com/;
}
}

Then try to grab http://localhost:8001/forum/AWVB3admincp/index.php

Only get 3316 bytes (should be 7674).

When I switch nginx to use select instead of kqueue, it works fine.

#2 Updated by ano about 3 years ago

I have some problems between nginx and backends too.
Nginx can not see when the backend closed connection.
The nginx developers said there is a kqueue problem: kqueue does not set EV_EOF
flag. They has made test program (attached), which works well under the FreeBSD
and NetBSD, but do not work properly under DragonFly.

Using "select" or "poll" methods in the nginx solves the problem, but kqueue is
preferred.

$ uname -iprs
DragonFly 2.10-RELEASE x86_64 X86_64_GENERIC_SMP

#3 Updated by sepherosa about 3 years ago

Well, I think we could handle the problem in kernel in the following way:
- Add EV_NODATA flag
- In the places that sets EV_NODATA, we also set the EV_EOF using the
logic before 8c4ed4265ed709e0ff2e164b292fd6b3febef5d3
- In the places that test EV_EOF, we change it to test EV_NODATA
- In the places that clears EV_EOF, we also clears EV_NODATA

On Wed, Sep 7, 2011 at 3:18 PM, Andrey N. Oktyabrski (via DragonFly
issue tracker) <> wrote:
>
> Andrey N. Oktyabrski <> added the comment:
>
> I have some problems between nginx and backends too.
> Nginx can not see when the backend closed connection.
> The nginx developers said there is a kqueue problem: kqueue does not set EV_EOF
> flag. They has made test program (attached), which works well under the FreeBSD
> and NetBSD, but do not work properly under DragonFly.
>
> Using "select" or "poll" methods in the nginx solves the problem, but kqueue is
> preferred.
>
> $ uname -iprs
> DragonFly 2.10-RELEASE x86_64 X86_64_GENERIC_SMP
>
> _____________________________________________________
> DragonFly issue tracker <>
> <http://bugs.dragonflybsd.org/issue1998>
> _____________________________________________________

#4 Updated by sepherosa about 3 years ago

On Wed, Sep 7, 2011 at 3:18 PM, Andrey N. Oktyabrski (via DragonFly
issue tracker) <> wrote:
>
> Andrey N. Oktyabrski <> added the comment:
>
> I have some problems between nginx and backends too.
> Nginx can not see when the backend closed connection.
> The nginx developers said there is a kqueue problem: kqueue does not set EV_EOF
> flag. They has made test program (attached), which works well under the FreeBSD
> and NetBSD, but do not work properly under DragonFly.

Please test the latest master:
3bcb6e5e80cb05cecaddd866d4b15b428334b9aa

should have been fixed, at least the test-kqueue.c runs properly.

Best Regards,
sephe

#5 Updated by ano about 3 years ago

I have not an "master" installed anywhere. I will set it up in the VirtualBox
this evening, then can test.

Is it possible to made the patch for 2.10? Or git' patch may be used for the
2.10?
(http://gitweb.dragonflybsd.org/dragonfly.git/patch/3bcb6e5e80cb05cecaddd866d4b15b428334b9aa)

#6 Updated by sepherosa about 3 years ago

On Fri, Sep 23, 2011 at 3:45 PM, Andrey N. Oktyabrski (via DragonFly
issue tracker) <> wrote:
>
> Andrey N. Oktyabrski <> added the comment:
>
> I have not an "master" installed anywhere. I will set it up in the VirtualBox
> this evening, then can test.
>
> Is it possible to made the patch for 2.10? Or git' patch may be used for the
> 2.10?
> (http://gitweb.dragonflybsd.org/dragonfly.git/patch/3bcb6e5e80cb05cecaddd866d4b15b428334b9aa)

You probably could cherry-pick in the following order:
ef7915fc4bf7204876b4aca6698fd84668676a41
23a031e34d5e83e747ad3907f5e72c1175f7ad94
3bcb6e5e80cb05cecaddd866d4b15b428334b9aa

Best Regards,
sephe

#7 Updated by ano about 3 years ago

Yes, it works for me on 2.10, thank you very much.

$ uname -iprs
DragonFly 2.10-RELEASE x86_64 X86_64_GENERIC_SMP

$ ./test-kqueue
kevent1 done, flags: 8021
kevent1 reported EOF, exiting

$ head /usr/pkg/etc/nginx/nginx.conf | tail -6 | grep -v '^ *#'
events {
worker_connections 10240;
use kqueue;
}

Also available in: Atom PDF