[WIP][POC] Update logic for constructing protected resource metadata URL to include the resource's path component, for compatibility with Oauth RFCs

[smurching]

Motivation and Context

Update logic for constructing the protected resource metadata URL for an MCP server to include the path component of the resource being accessed, as per section 3.1 of Oauth RFC 9728:

3.1. Protected Resource Metadata Request

A protected resource metadata document MUST be queried using an HTTP
GET request at the previously specified URL.

The consumer of the metadata would make the following request when
the resource identifier is https://resource.example.com and the well-
known URI path suffix is oauth-protected-resource to obtain the
metadata, since the resource identifier contains no path component:

 GET /.well-known/oauth-protected-resource HTTP/1.1
 Host: resource.example.com

If the resource identifier value contains a path or query component,
any terminating slash (/) following the host component MUST be
removed before inserting /.well-known/ and the well-known URI path
suffix between the host component and the path and/or query
components. The consumer of the metadata would make the following
request when the resource identifier is https://resource.example.com/
resource1 and the well-known URI path suffix is oauth-protected-
resource to obtain the metadata, since the resource identifier
contains a path component:

 GET /.well-known/oauth-protected-resource/resource1 HTTP/1.1
 Host: resource.example.com

Using path components enables supporting multiple resources per host.
This is required in some multi-tenant hosting configurations. This
use of .well-known is for supporting multiple resources per host;
unlike its use in [RFC8615], it does not provide general information
about the host.

To avoid breaking changes, we first attempt RFC-compliant behavior, e.g. for https://api.example.com/mcp, first try to query a PRM endpoint at https://api.example.com/.well-known/oauth-protected-resource/mcp. If that fails, we then fall back to querying https://api.example.com/.well-known/oauth-protected-resource (existing behavior, excluding the /mcp path component)

How Has This Been Tested?

Updated unit tests - TODO test this against some real example servers

Breaking Changes

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update

Checklist

• I have read the MCP Documentation
• My code follows the repository's style guidelines
• New and existing tests pass locally
• I have added appropriate error handling
• I have added or updated documentation as needed

Additional context

[smurching]

In practice we may want to keep retrying on other types of errors too, just to be safe

[felixweinberger]

Hi @smurching thank you for this contribution - discussed internally with @pcarleton as we have another PR #1334 open that tackles the same issue.

We think #1334 is the right approach here, so closing this one for now - please feel free to chime in on that PR if you have thoughts on the approach!