Skip to content

Commit 31442f3

Browse files
tbarbuglimogita
tbarbugli
and
mogita
authored
Make connection pool configurable (#250)
* make connection pool configurable * lint * add high-throughput http client tuning * add tests for docs * remove flaky channel member role integration test * test: fix month parameter * reduce test log verbosity by default * upgrade retrofit to 2.12.0 * disable http logging in tests by default * clean up lombok and javadoc warnings * remove app-dependent failing integration tests * stabilize integration test setup and task polling * format test harness stabilization changes * remove global test cleanup from basic test setup * remove unstable permission integration tests --------- Co-authored-by: mogita <me@mogita.com>
1 parent c61f801 commit 31442f3

30 files changed

+939
-1587
lines changed

‎DOCS.md‎

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,10 +55,44 @@ You can override this behavior by explicitly passing in the API key and secret a
5555
var properties = new Properties();
5656
properties.put(DefaultClient.API_KEY_PROP_NAME, "<api-key>");
5757
properties.put(DefaultClient.API_SECRET_PROP_NAME, "<api-secret>");
58+
properties.put(DefaultClient.DISPATCHER_MAX_REQUESTS_PROP_NAME, "128");
59+
properties.put(DefaultClient.DISPATCHER_MAX_REQUESTS_PER_HOST_PROP_NAME, "32");
60+
properties.put(DefaultClient.CONNECTION_POOL_MAX_IDLE_CONNECTIONS_PROP_NAME, "20");
61+
properties.put(DefaultClient.CONNECTION_POOL_KEEP_ALIVE_DURATION_PROP_NAME, "59000");
62+
properties.put(DefaultClient.API_CONNECT_TIMEOUT_PROP_NAME, "10000");
63+
properties.put(DefaultClient.API_READ_TIMEOUT_PROP_NAME, "30000");
64+
properties.put(DefaultClient.API_WRITE_TIMEOUT_PROP_NAME, "30000");
65+
properties.put(DefaultClient.API_TIMEOUT_PROP_NAME, "30000");
5866
var client = new DefaultClient(properties);
67+
client.setDispatcher(128, 32);
68+
client.setConnectionPool(20, Duration.ofSeconds(59));
69+
client.setTimeouts(
70+
Duration.ofSeconds(10),
71+
Duration.ofSeconds(30),
72+
Duration.ofSeconds(30),
73+
Duration.ofSeconds(30));
5974
DefaultClient.setInstance(client);
6075
```
6176

77+
You can also pass the same configuration through explicit HTTP options:
78+
79+
```java
80+
var options =
81+
DefaultClient.HttpClientOptions.builder()
82+
.dispatcher(128, 32)
83+
.connectionPool(20, Duration.ofSeconds(59))
84+
.connectTimeout(Duration.ofSeconds(10))
85+
.readTimeout(Duration.ofSeconds(30))
86+
.writeTimeout(Duration.ofSeconds(30))
87+
.callTimeout(Duration.ofSeconds(30))
88+
.build();
89+
90+
var client = new DefaultClient(properties, options);
91+
```
92+
93+
For high traffic workloads, `dispatcher.maxRequests` and
94+
`dispatcher.maxRequestsPerHost` are usually the first values to tune.
95+
6296
### Simple Example
6397
**Synchronous:**
6498

@@ -1896,4 +1930,4 @@ Import.createImport(createUrlResponse.getPath(), Import.ImportMode.Upsert);
18961930
```java
18971931
// signature comes from the HTTP header x-signature
18981932
boolean valid = App.verifyWebhook(body, signature)
1899-
```
1933+
```

‎README.md‎

Lines changed: 69 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -130,8 +130,15 @@ To configure the SDK you need to provide required properties
130130
| --------------------------- | ------------------- | ------------------------------ | -------- |
131131
| io.getstream.chat.apiKey | STREAM_KEY | - | Yes |
132132
| io.getstream.chat.apiSecret | STREAM_SECRET | - | Yes |
133-
| io.getstream.chat.timeout | STREAM_CHAT_TIMEOUT | 10000 | No |
133+
| io.getstream.chat.timeout | STREAM_CHAT_TIMEOUT | 20000 | No |
134+
| io.getstream.chat.connectTimeout | STREAM_CHAT_CONNECT_TIMEOUT | 20000 | No |
135+
| io.getstream.chat.readTimeout | STREAM_CHAT_READ_TIMEOUT | 20000 | No |
136+
| io.getstream.chat.writeTimeout | STREAM_CHAT_WRITE_TIMEOUT | 20000 | No |
134137
| io.getstream.chat.url | STREAM_CHAT_URL | https://chat.stream-io-api.com | No |
138+
| io.getstream.chat.connectionPool.maxIdleConnections | STREAM_CHAT_CONNECTION_POOL_MAX_IDLE_CONNECTIONS | 10 | No |
139+
| io.getstream.chat.connectionPool.keepAliveDurationMs | STREAM_CHAT_CONNECTION_POOL_KEEP_ALIVE_DURATION_MS | 118000 | No |
140+
| io.getstream.chat.dispatcher.maxRequests | STREAM_CHAT_DISPATCHER_MAX_REQUESTS | 128 | No |
141+
| io.getstream.chat.dispatcher.maxRequestsPerHost | STREAM_CHAT_DISPATCHER_MAX_REQUESTS_PER_HOST | 10 | No |
135142

136143
You can also use your own CDN by creating an implementation of FileHandler and setting it this way
137144

@@ -141,6 +148,67 @@ Message.fileHandlerClass = MyFileHandler.class
141148

142149
All setup must be done prior to any request to the API.
143150

151+
You can also tune the underlying OkHttp connection pool explicitly:
152+
153+
```java
154+
var properties = new Properties();
155+
properties.put(DefaultClient.API_KEY_PROP_NAME, "<api-key>");
156+
properties.put(DefaultClient.API_SECRET_PROP_NAME, "<api-secret>");
157+
properties.put(DefaultClient.DISPATCHER_MAX_REQUESTS_PROP_NAME, "128");
158+
properties.put(DefaultClient.DISPATCHER_MAX_REQUESTS_PER_HOST_PROP_NAME, "32");
159+
properties.put(DefaultClient.CONNECTION_POOL_MAX_IDLE_CONNECTIONS_PROP_NAME, "20");
160+
properties.put(DefaultClient.CONNECTION_POOL_KEEP_ALIVE_DURATION_PROP_NAME, "59000");
161+
properties.put(DefaultClient.API_CONNECT_TIMEOUT_PROP_NAME, "10000");
162+
properties.put(DefaultClient.API_READ_TIMEOUT_PROP_NAME, "30000");
163+
properties.put(DefaultClient.API_WRITE_TIMEOUT_PROP_NAME, "30000");
164+
properties.put(DefaultClient.API_TIMEOUT_PROP_NAME, "30000");
165+
166+
var client = new DefaultClient(properties);
167+
client.setDispatcher(128, 32);
168+
client.setConnectionPool(20, Duration.ofSeconds(59));
169+
client.setTimeouts(
170+
Duration.ofSeconds(10),
171+
Duration.ofSeconds(30),
172+
Duration.ofSeconds(30),
173+
Duration.ofSeconds(30));
174+
DefaultClient.setInstance(client);
175+
```
176+
177+
Or configure the same values through options:
178+
179+
```java
180+
var options =
181+
DefaultClient.HttpClientOptions.builder()
182+
.dispatcher(128, 32)
183+
.connectionPool(20, Duration.ofSeconds(59))
184+
.connectTimeout(Duration.ofSeconds(10))
185+
.readTimeout(Duration.ofSeconds(30))
186+
.writeTimeout(Duration.ofSeconds(30))
187+
.callTimeout(Duration.ofSeconds(30))
188+
.build();
189+
190+
var client = new DefaultClient(properties, options);
191+
```
192+
193+
### High traffic
194+
195+
For high traffic backends, a good starting point is:
196+
197+
```java
198+
var options =
199+
DefaultClient.HttpClientOptions.builder()
200+
.dispatcher(128, 32)
201+
.connectionPool(20, Duration.ofSeconds(59))
202+
.connectTimeout(Duration.ofSeconds(10))
203+
.readTimeout(Duration.ofSeconds(30))
204+
.writeTimeout(Duration.ofSeconds(30))
205+
.callTimeout(Duration.ofSeconds(30))
206+
.build();
207+
```
208+
209+
Start there and load test. In practice, `dispatcher.maxRequests` and
210+
`dispatcher.maxRequestsPerHost` usually affect throughput more than connection-pool size.
211+
144212
## Print Chat app configuration
145213

146214
<table>

‎build.gradle‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,8 +33,8 @@ dependencies {
3333
// define any required OkHttp artifacts without version
3434
implementation("com.squareup.okhttp3:okhttp")
3535

36-
implementation 'com.squareup.retrofit2:retrofit:2.9.0'
37-
implementation 'com.squareup.retrofit2:converter-jackson:2.9.0'
36+
implementation 'com.squareup.retrofit2:retrofit:2.12.0'
37+
implementation 'com.squareup.retrofit2:converter-jackson:2.12.0'
3838
implementation 'io.jsonwebtoken:jjwt-api:0.12.5'
3939
runtimeOnly 'io.jsonwebtoken:jjwt-impl:0.12.5'
4040
runtimeOnly 'io.jsonwebtoken:jjwt-jackson:0.12.5'

‎src/main/java/io/getstream/chat/java/models/App.java‎

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -586,7 +586,7 @@ public static class DeviceError {
586586

587587
@Builder
588588
@Getter
589-
@EqualsAndHashCode
589+
@EqualsAndHashCode(callSuper = false)
590590
public static class AsyncModerationCallback {
591591
@Nullable
592592
@JsonProperty("mode")
@@ -599,7 +599,7 @@ public static class AsyncModerationCallback {
599599

600600
@Builder
601601
@Getter
602-
@EqualsAndHashCode
602+
@EqualsAndHashCode(callSuper = false)
603603
public static class AsyncModerationConfigRequestObject {
604604
@Nullable
605605
@JsonProperty("callback")
@@ -612,7 +612,7 @@ public static class AsyncModerationConfigRequestObject {
612612

613613
@Builder
614614
@Getter
615-
@EqualsAndHashCode
615+
@EqualsAndHashCode(callSuper = false)
616616
public static class FileUploadConfigRequestObject {
617617

618618
@Nullable
@@ -644,7 +644,7 @@ public static FileUploadConfigRequestObject buildFrom(
644644

645645
@Builder
646646
@Getter
647-
@EqualsAndHashCode
647+
@EqualsAndHashCode(callSuper = false)
648648
public static class APNConfigRequestObject {
649649
@Nullable
650650
@JsonProperty("development")
@@ -690,7 +690,7 @@ public static APNConfigRequestObject buildFrom(@Nullable APNConfig aPNConfig) {
690690

691691
@Builder
692692
@Getter
693-
@EqualsAndHashCode
693+
@EqualsAndHashCode(callSuper = false)
694694
public static class FirebaseConfigRequestObject {
695695
@Nullable
696696
@JsonProperty("server_key")
@@ -720,7 +720,7 @@ public static FirebaseConfigRequestObject buildFrom(@Nullable FirebaseConfig fir
720720

721721
@Builder
722722
@Getter
723-
@EqualsAndHashCode
723+
@EqualsAndHashCode(callSuper = false)
724724
public static class HuaweiConfigRequestObject {
725725
@Nullable
726726
@JsonProperty("id")
@@ -733,7 +733,7 @@ public static class HuaweiConfigRequestObject {
733733

734734
@Builder
735735
@Getter
736-
@EqualsAndHashCode
736+
@EqualsAndHashCode(callSuper = false)
737737
public static class PushConfigRequestObject {
738738
@Nullable
739739
@JsonProperty("version")
@@ -764,7 +764,7 @@ protected Call<ListPushProviderResponse> generateCall(Client client) {
764764
}
765765

766766
@Getter
767-
@EqualsAndHashCode
767+
@EqualsAndHashCode(callSuper = false)
768768
public static class DeletePushProviderRequest extends StreamRequest<StreamResponseObject> {
769769
private String providerType;
770770
private String name;
@@ -785,7 +785,7 @@ protected Call<StreamResponseObject> generateCall(Client client) {
785785
builderMethodName = "",
786786
buildMethodName = "internalBuild")
787787
@Getter
788-
@EqualsAndHashCode
788+
@EqualsAndHashCode(callSuper = false)
789789
public static class AppUpdateRequestData {
790790
@Nullable
791791
@JsonProperty("disable_auth_checks")
@@ -976,7 +976,7 @@ public boolean equals(Object o) {
976976

977977
@Builder
978978
@Getter
979-
@EqualsAndHashCode
979+
@EqualsAndHashCode(callSuper = false)
980980
@NoArgsConstructor
981981
@AllArgsConstructor
982982
public static class AppGetRateLimitsRequest extends StreamRequest<AppGetRateLimitsResponse> {
@@ -1008,7 +1008,7 @@ protected Call<AppGetRateLimitsResponse> generateCall(Client client) {
10081008
builderMethodName = "",
10091009
buildMethodName = "internalBuild")
10101010
@Getter
1011-
@EqualsAndHashCode
1011+
@EqualsAndHashCode(callSuper = false)
10121012
public static class AppCheckSqsRequestData {
10131013
@Nullable
10141014
@JsonProperty("sqs_url")
@@ -1035,7 +1035,7 @@ protected Call<AppCheckSqsResponse> generateCall(Client client) {
10351035
builderMethodName = "",
10361036
buildMethodName = "internalBuild")
10371037
@Getter
1038-
@EqualsAndHashCode
1038+
@EqualsAndHashCode(callSuper = false)
10391039
public static class AppCheckSnsRequestData {
10401040
@Nullable
10411041
@JsonProperty("sns_topic_arn")
@@ -1062,7 +1062,7 @@ protected Call<AppCheckSnsResponse> generateCall(Client client) {
10621062
builderMethodName = "",
10631063
buildMethodName = "internalBuild")
10641064
@Getter
1065-
@EqualsAndHashCode
1065+
@EqualsAndHashCode(callSuper = false)
10661066
public static class AppCheckPushRequestData {
10671067
@Nullable
10681068
@JsonProperty("message_id")
@@ -1110,7 +1110,7 @@ protected Call<AppCheckPushResponse> generateCall(Client client) {
11101110

11111111
@AllArgsConstructor
11121112
@Getter
1113-
@EqualsAndHashCode
1113+
@EqualsAndHashCode(callSuper = false)
11141114
public static class AppRevokeTokensRequest extends StreamRequest<StreamResponseObject> {
11151115
@Nullable private Date revokeTokensIssuedBefore;
11161116

‎src/main/java/io/getstream/chat/java/models/Blocklist.java‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ public class Blocklist {
3838
builderMethodName = "",
3939
buildMethodName = "internalBuild")
4040
@Getter
41-
@EqualsAndHashCode
41+
@EqualsAndHashCode(callSuper = false)
4242
public static class BlocklistCreateRequestData {
4343
@Nullable
4444
@JsonProperty("name")
@@ -58,7 +58,7 @@ protected Call<StreamResponseObject> generateCall(Client client) {
5858

5959
@RequiredArgsConstructor
6060
@Getter
61-
@EqualsAndHashCode
61+
@EqualsAndHashCode(callSuper = false)
6262
public static class BlocklistGetRequest extends StreamRequest<BlocklistGetResponse> {
6363
@NotNull private String name;
6464

@@ -73,7 +73,7 @@ protected Call<BlocklistGetResponse> generateCall(Client client) {
7373
builderMethodName = "",
7474
buildMethodName = "internalBuild")
7575
@Getter
76-
@EqualsAndHashCode
76+
@EqualsAndHashCode(callSuper = false)
7777
public static class BlocklistUpdateRequestData {
7878
@Nullable
7979
@JsonProperty("words")
@@ -95,7 +95,7 @@ protected Call<StreamResponseObject> generateCall(Client client) {
9595

9696
@RequiredArgsConstructor
9797
@Getter
98-
@EqualsAndHashCode
98+
@EqualsAndHashCode(callSuper = false)
9999
public static class BlocklistDeleteRequest extends StreamRequest<StreamResponseObject> {
100100
@NotNull private String name;
101101

0 commit comments

Comments
 (0)