我有一个Ubuntu 10.10服务器,有足够的RAM,带宽和cpu.在从Apache和Nginx提供静态文件时,我在延迟分布中看到了一种奇怪的,可重复的模式.因为这两个http服务器都存在这个问题,所以我想知道我是否配置错误或调整了Ubuntu的网络或缓存参数.
ab -n 1000 -c 4 http：//apache-host/static-file.jpg：
Percentage of the requests served within a certain time (ms) 50% 5 66% 3007 75% 3009 80% 3011 90% 9021 95% 9032 98% 21068 99% 45105 100% 45105 (longest request)
ab -n 1000 -c 4 http：//Nginx-host/static-file.jpg：
Percentage of the requests served within a certain time (ms) 50% 19 66% 19 75% 3011 80% 3017 90% 9021 95% 12026 98% 12028 99% 18063 100% 18063 (longest request)
结果始终遵循这种模式 – 50％或更多的请求按预期服务,然后其余部分落入离散频段,最慢的几个数量级更慢.
Apache是2.x并且安装了mod_PHP. Nginx是1.0.x并且安装了Passenger(但是app服务器都不应该位于静态文件的关键路径中).每次测试运行时,平均负载大约为1(服务器有12个物理内核). 5GB免费RAM,7GB缓存交换.测试从localhost运行.
以下是我从ubuntu服务器10.10默认设置所做的配置更改：
/etc/sysctl.conf:
net.core.rmem_default = 65536
net.core.wmem_default = 65536
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_mem = 16777216 16777216 16777216
net.ipv4.tcp_window_scaling = 1
net.ipv4.route.flush = 1
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_moderate_rcvbuf = 1
net.core.somaxconn = 8192
/etc/security/limits.conf:
* hard nofile 65535
* soft nofile 65535
root hard nofile 65535
root soft nofile 65535
other config:
ifconfig eth0 txqueuelen 1000
如果这种问题响起,或者如果有关配置的更多信息会有所帮助,请告诉我.谢谢你的时间.
更新：这是我在增加net.netfilter.nf_conntrack_max之后看到的内容,如下所示：
Percentage of the requests served within a certain time (ms) 50% 2 66% 2 75% 2 80% 2 90% 3 95% 3 98% 3 99% 3 100% 5 (longest request)
解决方法
关闭你的评论,这是nf_conntrack完全问题,你可以增加conntrak表：
sysctl -w net.netfilter.nf_conntrack_max = 131072
或者,如果您已经位于防火墙后面,则可以从连接跟踪中免除HTTP流量：
# iptables -L -t raw Chain PREROUTING (policy ACCEPT) target prot opt source destination NOTRACK tcp -- anywhere anywhere tcp dpt:www NOTRACK tcp -- anywhere anywhere tcp spt:www Chain OUTPUT (policy ACCEPT) target prot opt source destination NOTRACK tcp -- anywhere anywhere tcp spt:www NOTRACK tcp -- anywhere anywhere tcp dpt:www
