Problem
Currently, the bee node connects to blockchain RPC endpoints using rpc.DialContext with Go's default HTTP client settings (pkg/node/chain.go). There is no way for node operators to configure the underlying HTTP transport (timeouts, keepalive, idle connections, etc.).
When communicating with RPC services behind HAProxy or similar load balancers/proxies, this causes intermittent EOF errors:
"could not get block number" "error"="Post \"http://rpc-sepolia-haproxy...\": EOF"
The root cause is a mismatch between Go's default HTTP client timeouts and the proxy/load balancer timeout configuration (e.g., HAProxy's timeout connect, timeout client, timeout server), leading to premature connection closures during RPC communication.
Current State
The RPC client is initialized with no HTTP client customization:
rpcClient, err := rpc.DialContext(ctx, endpoint)
The only existing configuration option is --blockchain-rpc-endpoint, which sets the URL. There are no options to tune the HTTP transport behavior.
Proposed Solution
Add new bee configuration flags to allow tuning the HTTP client used for blockchain RPC communication. Instead of relying on Go's http.DefaultTransport, construct a custom http.Client with operator-configurable timeouts and pass it via rpc.DialOptions with rpc.WithHTTPClient.
Suggested configuration flags (with sensible defaults):
| Flag |
Description |
Default |
--blockchain-rpc-dial-timeout |
TCP connection dial timeout |
30s |
--blockchain-rpc-tls-timeout |
TLS handshake timeout |
10s |
--blockchain-rpc-idle-timeout |
Idle connection timeout |
90s |
--blockchain-rpc-keepalive |
TCP keepalive interval |
30s |
This would allow node operators running behind proxies with aggressive timeout settings to tune these values to match their infrastructure, avoiding EOF and connection reset errors.
The same approach could also be considered for the ENS resolver client (pkg/resolver/client/ens/ens.go), which currently uses ethclient.Dial with defaults.
Background
This was originally explored in #5341 as a custom retry round-tripper. Review feedback concluded that properly configuring the HTTP client timeouts is a cleaner solution than adding transport-level retry logic, since:
- Retrying non-idempotent RPC requests (e.g.,
eth_sendRawTransaction) risks duplicate submissions
- The root cause is a timeout mismatch, not transient network failures
- Retry logic adds unnecessary complexity (body rewinding, backoff, context cancellation handling)
Problem
Currently, the bee node connects to blockchain RPC endpoints using
rpc.DialContextwith Go's default HTTP client settings (pkg/node/chain.go). There is no way for node operators to configure the underlying HTTP transport (timeouts, keepalive, idle connections, etc.).When communicating with RPC services behind HAProxy or similar load balancers/proxies, this causes intermittent EOF errors:
The root cause is a mismatch between Go's default HTTP client timeouts and the proxy/load balancer timeout configuration (e.g., HAProxy's
timeout connect,timeout client,timeout server), leading to premature connection closures during RPC communication.Current State
The RPC client is initialized with no HTTP client customization:
The only existing configuration option is
--blockchain-rpc-endpoint, which sets the URL. There are no options to tune the HTTP transport behavior.Proposed Solution
Add new bee configuration flags to allow tuning the HTTP client used for blockchain RPC communication. Instead of relying on Go's
http.DefaultTransport, construct a customhttp.Clientwith operator-configurable timeouts and pass it viarpc.DialOptionswithrpc.WithHTTPClient.Suggested configuration flags (with sensible defaults):
--blockchain-rpc-dial-timeout30s--blockchain-rpc-tls-timeout10s--blockchain-rpc-idle-timeout90s--blockchain-rpc-keepalive30sThis would allow node operators running behind proxies with aggressive timeout settings to tune these values to match their infrastructure, avoiding EOF and connection reset errors.
The same approach could also be considered for the ENS resolver client (
pkg/resolver/client/ens/ens.go), which currently usesethclient.Dialwith defaults.Background
This was originally explored in #5341 as a custom retry round-tripper. Review feedback concluded that properly configuring the HTTP client timeouts is a cleaner solution than adding transport-level retry logic, since:
eth_sendRawTransaction) risks duplicate submissions