Skip to content

Add a default timeout to all requests, make it configurable#226

Merged
amercader merged 2 commits intomasterfrom
timeouts
Mar 13, 2026
Merged

Add a default timeout to all requests, make it configurable#226
amercader merged 2 commits intomasterfrom
timeouts

Conversation

@amercader
Copy link
Copy Markdown
Member

Make sure the the timeout parameter is passed to requests before making an HTTP request. By default we use (5,5) which means 5 seconds for connect and read timeouts.

I added a separate option for the read timeout as it is specially important in the streaming requests we use to load/dump datasets.

@wardi @EricSoroos what do you think of the default value? currently 5 seconds for both

Make sure the the `timeout` parameter is passed to requests before
making an HTTP request. By default we use (5,5) which means 5 seconds
for connect and read timeouts.
@EricSoroos
Copy link
Copy Markdown

LGTM, 5 seconds is a decent default, IIRC the read timeout is not for completion, just for byte gaps. I think it's going to be rarely set, but may be useful in some specific cases.

To avoid this warning:

```
  ckanapi/ckanapi/cli/load.py:307: SyntaxWarning: "\." is an invalid
  escape sequence. Such sequences will not work in the
    future. Did you mean "\\."? A raw string is also an option.
        new_name = re.sub('[0-9\.-]','',name)
```
Comment thread ckanapi/cli/load.py
@wardi
Copy link
Copy Markdown
Contributor

wardi commented Feb 20, 2026

How does this interact with someone setting the timeout via requests_kwargs?

files=None, requests_kwargs=None):

@wardi
Copy link
Copy Markdown
Contributor

wardi commented Feb 20, 2026

Setting a new default timeout of 5 seconds would be a breaking change. Calling datastore_search on a 1M row table will take longer than that just because of computing the total row count.

The default should either be really long or unset to minimize breaking existing users' code.

@amercader
Copy link
Copy Markdown
Member Author

How does this interact with someone setting the timeout via requests_kwargs?

As we set the timeout in request_kwargs using setdefault(), if someone has explicitly set a timeout via requests_kwargs that will take precedence

@amercader
Copy link
Copy Markdown
Member Author

amercader commented Mar 10, 2026

We decided to keep the timeout unset by default to not break existing scripts (as it does requests by default). Users relying on a timely response can configure the timeouts explicitly in the API call (using requests_kwargs["timeout"]) or via config

@amercader amercader merged commit d6c97e5 into master Mar 13, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants