API Data Access Changes

[CodexVeritas]

Description:
Following the public announcement of API data access changes, the items below need to be implemented and rolled out by end of week. Italicized items from the post represent net-new changes that are not yet live.

Changes to the API gateway:

• all accounts have question text and resolution value access to any question it's ever predicted.
• all accounts have access to staff comments.
• all accounts have current CP access on ~50 questions in a "bot testing" project (id = 32977).
• there is a new access tier that gives access to a "bot benchmarking" project which will have ~250 open questions and ~250 resolved questions. Accounts with this access tier have access to the current CP of the open questions and the resolution value of the resolved questions. (id = 32979)

Other changes:

• MiniBench CP history – For MiniBench questions that ask bots to predict the Metaculus CP, inject additional historical CP aggregation values directly into the question text.
• document all this in the API docs.

Context: link to public post | link to Slack thread

[leonardthethird]

@elisescu how easy/hard is it to create functionality to give researcher-type people API access to only certain projects?

[lsabor]

I'm the one primarily implementing this change.
I think I've figured out a way to allow this with minimal effort. Could you spec out the exact behavior you're looking for? I can see if I can either wrap it into what I'm doing now or set it up so we can support that easily in the future.

[lsabor]

access to staff comments

The niave way of implementing this will just return all staff comments, including non-root comments.
I suggest that we only return root-level staff comments. (I have this implemented, but can pivot easily)

I also think we could consider including all comments that have a staff comment as a root - this way api users could see the full discussion in case there are nuances within the thread that require some back and forth.

[lsabor]

I think it's likely that the intention is that for the 50 questions where CP is available universally and 500 bot benchmarking questions, we want to also give access to the text fields on the question "description", "fine_print", and "resolution_criteria". I'm implementing that, but please correct me if that seems wrong.

[lsabor]

Previously, the api gave question text for open questions independent of predicting and project status. I assume we don't want to change that, yes?