同样的错误消息又回来了!
2017-09-29 18:50:22.497 ERROR 11099 --- [8080-Acceptor-0] org.apache.tomcat.util.net.NioEndpoint : Socket accept failed java.io.IOException: Too many open files at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) ~[na:1.8.0_141] at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) ~[na:1.8.0_141] at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) ~[na:1.8.0_141] at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:453) ~[tomcat-embed-core-8.5.15.jar!/:8.5.15] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_141] 2017-09-29 18:50:23.885 INFO 11099 --- [Thread-3] ationConfigEmbeddedWebApplicationContext : Closing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@5387f9e0: startup date [Wed Sep 27 03:14:35 UTC 2017]; root of context hierarchy 2017-09-29 18:50:23.890 INFO 11099 --- [Thread-3] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase 2147483647 2017-09-29 18:50:23.891 WARN 11099 --- [Thread-3] o.s.c.support.DefaultLifecycleProcessor : Failed to stop bean 'documentationPluginsBootstrapper' ... 7 common frames omitted 2017-09-29 18:50:53.891 WARN 11099 --- [Thread-3] o.s.c.support.DefaultLifecycleProcessor : Failed to shut down 1 bean with phase value 2147483647 within timeout of 30000: [documentationPluginsBootstrapper] 2017-09-29 18:50:53.891 INFO 11099 --- [Thread-3] o.s.j.e.a.AnnotationMBeanExporter : Unregistering JMX-exposed beans on shutdown 2017-09-29 18:50:53.894 INFO 11099 --- [Thread-3] com.app.controller.SearchController : Closing the ES REST client
我尝试使用上一篇文章中的解决方案。
ElasticsearchConfig: @Configuration public class ElasticsearchConfig { @Value("${elasticsearch.host}") private String host; @Value("${elasticsearch.port}") private int port; @Bean public RestClient restClient() { return RestClient.builder(new HttpHost(host, port)) .setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() { @Override public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) { return requestConfigBuilder.setConnectTimeout(5000).setSocketTimeout(60000); } }).setMaxRetryTimeoutMillis(60000).build(); }
SearchController:
@RestController @RequestMapping("/api/v1") public class SearchController { @Autowired private RestClient restClient; @RequestMapping(value = "/search", method = RequestMethod.GET, produces="application/json" ) public ResponseEntity<Object> getSearchQueryResults(@RequestParam(value = "criteria") String criteria) throws IOException { // Setup HTTP Headers HttpHeaders headers = new HttpHeaders(); headers.add("Content-Type", "application/json"); // Setup query and send and return ResponseEntity... Response response = this.restClient.performRequest(...); } @PreDestroy public void cleanup() { try { logger.info("Closing the ES REST client"); this.restClient.close(); } catch (IOException ioe) { logger.error("Problem occurred when closing the ES REST client", ioe); } } }
pom.xml:
<parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>1.5.4.RELEASE</version> </parent> <dependencies> <!-- Spring --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <!-- Elasticsearch --> <dependency> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> <version>5.5.0</version> </dependency> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>transport</artifactId> <version>5.5.0</version> </dependency> <!-- Apache Commons --> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-lang3</artifactId> <version>3.6</version> </dependency> <!-- Log4j --> <dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId> <version>1.2.17</version> </dependency> </dependencies>
这让我认为RestClient首先从来没有明确关闭连接…
这令人惊讶,因为我的基于Elasticsearch Spring Boot的微服务在两个不同的AWS EC-2服务器上实现了负载平衡。
日志文件报告该异常发生了2000次,直到最后prePrestroy()关闭了客户端。请参阅在堆栈跟踪末尾记录的@PreDestroy()清除方法中的INFO。
我是否需要在SearchController中显式放置一个finally子句并显式关闭RestClient连接?
至关重要的是,此IOException不再发生,因为此搜索微服务依赖于许多不同的移动客户端(iOS和Android)。
需要它具有容错能力和可伸缩性……或者至少不中断。
唯一的原因是在日志文件的底部:
2017-09-29 18:50:53.894 INFO 11099 --- [Thread-3] com.app.controller.SearchController : Closing the ES REST client
是因为我这样做了:
kill -3 jvm_pid
我应该保留@PreDestory cleanup()方法,但更改SearchController.getSearchResults()方法的内容以反映如下内容:
@RequestMapping(value = "/search", method = RequestMethod.GET, produces="application/json" ) public ResponseEntity<Object> getSearchQueryResults(@RequestParam(value = "criteria") String criteria) throws IOException { // Setup HTTP Headers HttpHeaders headers = new HttpHeaders(); headers.add("Content-Type", "application/json"); // Setup query and send and return ResponseEntity... Response response = null; try { // Submit Query and Obtain Response response = this.restClient.performRequest("POST", endPoint, Collections.singletonMap("pretty", "true"), entity); } catch(IOException ioe) { logger.error("Exception when performing POST request " + ioe); } finally { this.restClient.close(); } // return response as EsResponse(); }
这样,RestClient连接始终关闭…
如果有人可以帮助我,将不胜感激。
从我的角度来看,您做错的事情很少,但是我将直接解决。
我不会编写完整的解决方案(实际上,我没有执行或测试任何东西),但是重要的是要理解它。另外,最好将所有与数据访问相关的内容移到另一层。 无论如何,这只是一个例子,因此设计并不完美 。
步骤1:导入正确的库。
几乎与您的示例相同。我更新了示例以使用版本5.6.2中推荐的最后一个客户端库
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.acervera</groupId> <artifactId>elastic-example</artifactId> <version>1.0-SNAPSHOT</version> <packaging>jar</packaging> <name>elastic-example</name> <url>http://maven.apache.org</url> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <es.version>5.6.2</es.version> </properties> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>1.5.4.RELEASE</version> </parent> <dependencies> <!-- Spring --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <!-- Elasticsearch --> <dependency> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> <version>${es.version}</version> </dependency> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> <version>${es.version}</version> </dependency> <!-- Log4j --> <dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId> <version>1.2.17</version> </dependency> </dependencies> </project>
步骤2:在bean工厂中创建并关闭客户端。
在bean工厂中,创建并销毁它。您可以重用相同的客户端。
import org.apache.http.HttpHost; import org.elasticsearch.client.RestClient; import org.elasticsearch.client.RestHighLevelClient; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import javax.annotation.PostConstruct; import javax.annotation.PreDestroy; import java.io.IOException; @Configuration public class ElasticsearchConfig { // Here all init stuff with @Value(....) RestClient lowLevelRestClient; RestHighLevelClient client; @PostConstruct public void init() { lowLevelRestClient = RestClient.builder(new HttpHost("host", 9200, "http")).build(); client = new RestHighLevelClient(lowLevelRestClient); } @PreDestroy public void destroy() throws IOException { lowLevelRestClient.close(); } @Bean public RestHighLevelClient getClient() { return client; } }
步骤3:使用Java Transport Client执行查询。
使用Java Transport Client执行查询。
import org.elasticsearch.action.search.SearchRequest; import org.elasticsearch.action.search.SearchResponse; import org.elasticsearch.client.RestHighLevelClient; import org.elasticsearch.index.query.QueryBuilders; import org.elasticsearch.search.SearchHit; import org.elasticsearch.search.builder.SearchSourceBuilder; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.http.ResponseEntity; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMethod; import org.springframework.web.bind.annotation.RequestParam; import org.springframework.web.bind.annotation.RestController; import java.io.IOException; @RestController @RequestMapping("/api/v1") public class SearchController { @Autowired private RestHighLevelClient client; @RequestMapping(value = "/search", method = RequestMethod.GET, produces="application/json" ) public ResponseEntity<Tweet> getSearchQueryResults(@RequestParam(value = "criteria") String criteria) throws IOException { // This is only one example. Of course, this logic make non sense and you are going to put it in a DAO // layer with more logical stuff SearchRequest searchRequest = new SearchRequest(); SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.query(QueryBuilders.matchAllQuery()); SearchResponse searchResponse = client.search(searchRequest); if(searchResponse.getHits().totalHits > 0) { SearchHit searchHit = searchResponse.getHits().iterator().next(); // Deserialize to Java. The best option is to use response.getSource() and Jackson // This is other option. Tweet tweet = new Tweet(); tweet.setId(searchHit.getField("id").getValue().toString()); tweet.setTittle(searchHit.getField("tittle").getValue().toString()); return ResponseEntity.ok(tweet); } else { return ResponseEntity.notFound().build(); } } }
另外,使用Bean构建响应。
public class Tweet { private String id; private String tittle; public String getTittle() { return tittle; } public void setTittle(String tittle) { this.tittle = tittle; } public String getId() { return id; } public void setId(String id) { this.id = id; } // Here rest of bean stuff (equal, hash, etc) or Lombok }
第4步
享受Elasticsearch!
注意:Java REST客户端[5.6]»Java高级REST客户端
PS。有必要重构该示例。只是了解方式。