Add CI Job indexing pages after build to an AI Search instance

[ttmx]

Summary

Adds a CI Job that sends the changed contents of pages to a worker which indexes them to an AI Search instance.

The job also stores a state cache in CI, in order to only send diffs of what has changed between runs.

The indexer script looks at the generated html pages from the build step and correlates them to the original markdown, indexing the markdown pages split by the headers.
In the case of generated pages without a markdown source, we add content transformers, which generate the content to be indexed from the same config files.
In the case there is no matching transformer or markdown document available, we index a transformation of the html (mostly stripping what isn't content and converting things to plaintext)

IMPORTANT

Requires new secrets in the repository: (To be discussed internally)

• CF_ACCESS_CLIENT_ID: Access service-token client ID.

• CF_ACCESS_CLIENT_SECRET: Access service-token secret.

• AI_SEARCH_REINDEX_URL: Reindexing endpoint.

Documentation checklist

• Is there a changelog entry (guidelines)? If you don't add one for something awesome and new (however small) — how will our customers find out? Changelogs are automatically posted to RSS feeds, the Discord, and X.
• The change adheres to the documentation style guide.
• If a larger change - such as adding a new page- an issue has been opened in relation to any incorrect or out of date information that this PR fixes. -> Talked about internally, no issue
• Files which have changed name or location have been allocated redirects. -> No file changed.