Updated/new HTTP API Migration

[guseggert]

Context

Per the Delegated Routing HTTP API IPIP-337, we're expecting changes to ipfs/go-delegated-routing, either as a new HTTP Transport of Reframe or a "new" protocol. The specific path to be chosen is based on what's decided there. This tracking issue is for the resulting work and migrations for rolling it out in production.

Done Criteria

1. Parties interested in the Delegated Routing HTTP API have a clean/sane client and server implementation.
2. This has rolled out to Kubo, Hydras, and cid.contact
3. There is followup action outlining the communication steps we need to like updating https://blog.ipfs.tech/2022-09-02-introducing-reframe/

Why Important

This will enable more/easier adoption of delegated routing across IPFS implementations. See the IPIP for more information.

User/Customer

Delegated routing servers and clients.

Notes

Known systems requiring migration:

• Hydras
  • Tracking issue: Update Hydras to new HTTP Delegated Routing libp2p/hydra-booster#180
  • Dependent on cid.contact being available via the HTTP API
  • @guseggert will handle this
  • Effort: 2 days
• Bedrock (cid.contact)
  • Tracking issue: Update STI to new HTTP Delegated Routing ipni/storetheindex#1038
  • storetheindex
  • indexstar
  • @willscott has already committed to driving this
  • Effort: ?
  • Sequence of steps
    • Add a new endpoint, leaving the existing one alone
    • Hydras flip over to the new endpoint
    • Remove the old endpoint
• Kubo
  • Tracking issue: Update Kubo to new HTTP Delegated Routing kubo#9422
  • Add a new router type to the config, something like http or http-content-routing
  • Remove reframe router type until its issues are resolved
  • Best person for this is probably @ajnavarro, but @guseggert can also pick up if necessary
  • Effort: 2 days

Timeline

• go-delegated-routing changes merged and released ECD: 2022-11-22 2022-11-29
• Hydra codes changes merged using updated go-delegated-routing ECD: 2022-11-23 2022-12-08
• cid.contact code changes merged using updated go-delegated routing ECD: 2022-11-30 support http delegated content routing protocol ipni/indexstar#45
• cid.contact deployment ECD: 2022-12-01
• Hydra deployment ECD: 2022-11-28 2022-12-08
• Kubo code changes merge ECD: 2022-11-28 2022-12-05 2022-12-07 feature: delegated-routing: Add HTTP delegated routing. kubo#9453
• Kubo 0.18 release (RC 2022-12-08, final 2022-12-15): Release 0.18 kubo#9417

Cleanup is being tracked in ipfs/kubo#9479. We can close this once the items above are completed.

[BigLep]

@guseggert : I provided some more info on what I think the done criteria is. Feel free to adjust.

Also, some other things I think we need notes on:

1. Are we doing a new go major version?
2. What happens to the existing "reframe" implementation?
3. What is the migration sequence of events? For example, is cid.contact updating and going to break the hydras until Hydra updates?

[willscott]

3. What is the migration sequence of events? For example, is cid.contact updating and going to break the hydras until Hydra updates?

i imagine cid.contact can implement the new endpoint in existing to the current reframe one. Once that's in place, then hydras / kubo can migrate at their leisure.

from looking at the proposed spec, initially cid.contact will only implement the /v1/providers/{cid} endpoint, since double hashing does not yet exist, and the indexer does not accept provide requests.

[guseggert]

What is the migration sequence of events? For example, is cid.contact updating and going to break the hydras until Hydra updates?

We can do it gracefully, I added details to the issue.

[guseggert]

Are we doing a new go major version?

I don't see a practical reason to do this, if anyone has one then we can consider it but mv bumps are painful and IMO should be avoided when possible. If this library had a huge amount of consumers then sure, but the number of consumers is tiny.

[BigLep]

Also, for anyone watching this issue, I added a callout up at the top that it's still an open question on how we do an HTTP delegated routing API. There good/important open questions in ipfs/specs#337 and those should get resolved first. This issue is for accounting for making and rolling out whatever changes are decided on in IPIP-337.

[guseggert]

What happens to the existing "reframe" implementation?

It would exist in the Git history. If someone needs it they are free to fork it from the commit here and keep on using it, but I don't think it makes sense to keep it around when it is broken and we aren't using it. (By "broken" I'm referring to both implementation bugs such as this and the spec issues mentioned here.)

[BigLep]

@guseggert : thanks for the estimates on the work. Given this is actively being pursued and is important for any work involving potential adjustments to the hydras (https://pl-strflt.notion.site/2022Q4-Hydra-Dial-Down-c39f63d473e64f6fabcbfc38fb11d670 ), can you give please give an estimated timeline? It looks like we have these steps to track:

• go-delegated-routing changes merged and released
• Hydra codes changes merged using updated go-delegated-routing
• cid.contact code changes merged using updated go-delegated routing
• cid.contact deployment
• Hydra deployment
• Kubo code changes merge
• Kubo 0.18 release (RC 2022-12-08, final 2022-12-15): Release 0.18 kubo#9417

Those can be added to the description.

Thanks!