摘要:
我们在Spring-Boot 2.0.4和存在一个主要的性能问题Couchbase server 5.5.1
Spring-Boot 2.0.4
Couchbase server 5.5.1
随着线程数量的增加,数据库响应时间性能迅速下降。这是有关该问题的另一份报告。
详细信息:
Spring Boot正在运行500个线程:
server: tomcat: max-threads: 500 max-connections: 500
我们使用以下依赖项:
<dependency> <groupId>org.springframework.data</groupId> <artifactId>spring-data-couchbase</artifactId> <version>3.0.9.RELEASE</version> </dependency>
我们从DB的“选择”是通过Spring-Data存储库执行的:
Cat findFirstByOwnerIdAndNameAndColor(String ownerId, String name, String color);
我们有一个特别适合此查询的索引:
CREATE INDEX `cat_by_ownerId_name_and_color_idx` ON `pets`(`ownerId`,`name`,`color`) WHERE (`_class` = "com.example.Cat")
随着请求数量的增加,我们可以看到数据库回答查询的时间迅速减少。
例如,当每秒运行300个请求时,响应时间的99%约为 10秒! 而50的百分位数约为5秒。
返回的文档的平均大小约为 300字节 。这意味着我们正在尝试每秒提取大约90千字节。相对较低的数量。
编辑:
我在此处添加在Couchbase的UI中运行相同查询的结果:(在UI中,查询需要1.75毫秒才能完成)。
{ "plan": { "#operator": "Sequence", "~children": [ { "#operator": "IndexScan3", "index": "cats_by_ownerId_name_and_color_idx", "index_id": "c061141c2d373067", "index_projection": { "primary_key": true }, "keyspace": "pets", "namespace": "default", "spans": [ { "exact": true, "range": [ { "high": "\"bf23fa4c-22c3-42ac-b141-39cdc76bb2x5\"", "inclusion": 3, "low": "\"bf23fa4c-22c3-42ac-b141-39cdc76bb2x5\"" }, { "high": "\"Oscar\"", "inclusion": 3, "low": "\"Oscar\"" }, { "high": "\"red\"", "inclusion": 3, "low": "\"red\"" } ] } ], "using": "gsi" }, { "#operator": "Fetch", "keyspace": "pets", "namespace": "default" }, { "#operator": "Parallel", "~child": { "#operator": "Sequence", "~children": [ { "#operator": "Filter", "condition": "(((((`pets`.`_class`) = \"com.example.Cat\") and ((`pets`.`ownerId`) = \"bf23fa4c-22c3-42ac-b141-39cdc76bb2x5\")) and ((`pets`.`name`) = \"Oscar\")) and ((`pets`.`color`) = \"red\"))" }, { "#operator": "InitialProject", "result_terms": [ { "expr": "self", "star": true } ] }, { "#operator": "FinalProject" } ] } } ] }, "text": "select * from pets where _class=\"com.example.Cat\" and projectId=\"bf23fa4c-22c3-42ac-b141-39cdc76bb2x5\" and name=\"Oscar\" and color=\"red\""
}
编辑2
我们还尝试隐式编写N1ql查询,但结果是相同的。和以前一样,我们获得了许多TimeOutExceptions:
Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.dao.QueryTimeoutException: java.util.concurrent.TimeoutException: {"b":"pets","s":"n1ql","t":7500000,"i":"f8cdf670-d32a-4d74-858c-f9dd9789d264"}; nested exception is java.lang.RuntimeException: java.util.concurrent.TimeoutException: {"b":"pets","s":"n1ql","t":7500000,"i":"f8cdf670-d32a-4d74-858c-f9dd9789d264"}] with root cause java.util.concurrent.TimeoutException: {"b":"pets","s":"n1ql","t":7500000,"i":"f8cdf670-d32a-4d74-858c-f9dd9789d264"} at com.couchbase.client.java.bucket.api.Utils$1.call(Utils.java:131) ~[java-client-2.7.0.jar:na] at com.couchbase.client.java.bucket.api.Utils$1.call(Utils.java:127) ~[java-client-2.7.0.jar:na] at rx.internal.operators.OperatorOnErrorResumeNextViaFunction$4.onError(OperatorOnErrorResumeNextViaFunction.java:140) ~[rxjava-1.3.8.jar:1.3.8] at rx.internal.operators.OnSubscribeTimeoutTimedWithFallback$TimeoutMainSubscriber.onTimeout(OnSubscribeTimeoutTimedWithFallback.java:166) ~[rxjava-1.3.8.jar:1.3.8] at rx.internal.operators.OnSubscribeTimeoutTimedWithFallback$TimeoutMainSubscriber$TimeoutTask.call(OnSubscribeTimeoutTimedWithFallback.java:191) ~[rxjava-1.3.8.jar:1.3.8] at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55) ~[rxjava-1.3.8.jar:1.3.8] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_161] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_161] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_161] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_161] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_161] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_161] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
有没有办法解决这个问题,或者我们需要一个不同的数据库?
因此,在进一步调查后,在Spring-Data组件中发现了问题。
Spring-Data
为了克服这个问题,我们必须转向 非阻塞机制 。
我们做了两件事:
CompleteableFuture<Cat>
为了绕过Spring-Data与Couchbase的连接,我们使用实现代码创建了自己的存储库类,该实现代码如下所示:
Statement statement = select("*") .from(i(bucket.name())) .where(x("name").eq(s(name)) .and(x("ownerId").eq(s(ownerId))) .and(x("color").eq(s(color))) .and(x("_class").eq(s("com.example.Cat"))));
CompletableFuture completableFuture = new CompletableFuture(); bucket.async().query(statement) …
完成此操作后,即使在大约数百个并发请求期间,延迟问题也消失了,查询性能约为 2毫秒 。