生产环境,请求方调用我方地址,发生异常NoHttpResponseException,错误详情:
org.apache.http.NoHttpResponseException: 127.0.0.1:9000 failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at com.http.CommonHttpClient.sendPost(CommonHttpClient.java:96)
at com.http.CommonHttpClient.lambda$main$0(CommonHttpClient.java:121)
at java.util.stream.ForEachOps$ForEachOp$OfInt.accept(ForEachOps.java:205)
at java.util.Spliterators$IntArraySpliterator.forEachRemaining(Spliterators.java:1032)
at java.util.Spliterator$OfInt.forEachRemaining(Spliterator.java:693)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.execLocalTasks(ForkJoinPool.java:1040)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1058)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
看一下报错的源代码,相关注释提示服务端关闭了连接导致的:
想要解决问题,首先需要复现一下,于是本地启动服务,设置tomcat保活时间1s,
server:
tomcat:
keep-alive-timeout: 1000
使用common-httpclient4.5.10连接池,模拟一下请求,这里设置:
connManager.setMaxTotal(5);
connManager.setDefaultMaxPerRoute(5);
因为连接就是一组 ip:port 抽象,所以在模拟代码里获取一下本地的连接端口,在main测试中模拟20个请求,发送请求后休眠一会,推进连接过期,服务端关闭连接:
BasicHttpContext httpContext = new BasicHttpContext();
final long l = System.currentTimeMillis();
try {
response = httpClient.execute(httpPost, httpContext);
local = JSONObject.parseObject(JSON.toJSONString(httpContext.getAttribute("http.connection"))).getString("localPort");
HttpEntity httpEntity = response.getEntity();
result = EntityUtils.toString(httpEntity, "utf-8");
System.out.println("normal" + " : " + l + " : " + result + " : " + local);
} catch (Exception e) {
System.out.println("error " + " : " + l + " : " + local);
e.printStackTrace();
}
-----------------------------------------------------------------------------------
public static void main(String[] args) {
try {
IntStream.range(1, 20).parallel().sorted().forEach(e -> {
try {
sendPost("http://127.0.0.1:9000/receive", "{}");
TimeUnit.MILLISECONDS.sleep(995L);
} catch (Exception exception) {
exception.printStackTrace();
}
});
} catch (Exception e) {
e.printStackTrace();
}
}
可以看到本地和服务器共有5个连接,51573,51574,51575,51576,51577,其中有两笔异常数据报错:NoHttpResponseException,其中75,76两个端口连接后续没有被继续使用,怀疑已经被关闭了看一下抓包数据:
这两个端口处确实已经四次挥手关闭连接了,看样子应该是连接已不可用,发起请求时返回的错误。如果连接不可用,连接池会清除掉不可用连接,重新创建新的连接。再看下httpclient默认的保活策略:
问题也复现了,相关原因也基本确定了,解决办法就是保证conn的可用性:
1.关闭keep-alive,设置header Connection close
2.客户端实现一下保活策略,时间小于服务端的keep-alive-timeout,连接提前过期释放;当然应该配置合理的连接池大小,不必过大
3.客户端发生此类错误时,应该启用重试机制