handle rate limiting

[djay]

Too many people logging in at once or polling at once, from a single IP (such as at an event) appears like abuse to github.

Need to work out if it was the polling or the

Options to solve

• Turn off polling by default
• Slow polling down
• use condition not-modified requests for polling requests
• require everyone login to see. logins have higher rate limits
• put up apporiate message if we are rate limited. headers can tell us
• somehow reduce initial requests to be less on login.
• use firebase or other service to replicate the data and push it out
• don't poll when tab is hidden (page visibility api)

[jean]

Host the hackmap via netlify.com, so that read requests hit their CDN.
Not sure how doable this is ..

[waf]

PR #28 implements conditional (if-not-modified) polling.

For the initial load, it's only 2 API requests. One to get the issue list, and one to get the comments on the BKK Main Thread issue. I don't think that's causing the issue, but it's possible. We could investigate GitHub's graphql API.

[waf]

#40 fixes put up appropriate message if we are rate limited. headers can tell us check list item.

[djay]

@waf was there any analysis on why we got rate limited last event?

[waf]

@djay sorry, just saw your comment (got lost in the noise).

• If you log in with your github account, you'll get a higher rate limit
• I've added a feature to only poll when the browser tab is visible, which should help when people open the hackmap and then switch tabs and let it sit there in the background.
• There's one more fix I haven't got around to yet (done! see When the user is logged in, apply their authentication to all API calls #44), which is the that the very first AJAX request is unauthenticated. So that first ajax request is subject to rate limiting and can be rejected. It's easy enough to make it authenticated.

[djay]

@waf sounds great. One more feature we could potentially add later is to adjust the polling rate based on the number of users visible. Also, is it the case that polling isn't the problem if there are no updates since it returns 403 not modified? So potentially we could slow it down just when there are lots of updates?
Another idea is perhaps change the data model so we can do a single query to get the data. like if x,y are in the title rather than the body of the comment?

[waf]

@djay sounds good. Yes, if the api returns 304 Not Modified, the polling shouldn't be an issue. Now that we have both features in (use auth for all calls, and poll only when the tab is visible), let's wait for the next hacknight and see how these features work. We can experiment with smarter polling rate adjustments and alternate data models if it's still a problem.