feat(extension): expand t.co links when importing tweets#1352
Open
abhay-codes07 wants to merge 1 commit into
Open
feat(extension): expand t.co links when importing tweets#1352abhay-codes07 wants to merge 1 commit into
abhay-codes07 wants to merge 1 commit into
Conversation
A tweet's full_text embeds every link as an opaque `https://t.co/xxxx` shortlink, so imported tweets stored those instead of the real destinations, which is bad for reading and for search/recall over the memory. The real URL is already available on entities.urls (expanded_url + display_url), it just was not being used. Add expandTweetText, which rewrites each t.co shortlink in the text to a markdown link pointing at the expanded URL and labelled with the human-readable display_url, then run the tweet body through it in tweetToMarkdown. Entries missing a url or expanded_url are left untouched. Adds unit tests for single, repeated, and multiple links, the display_url fallback, and the no-op cases.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
A tweet's
full_textembeds every link as an opaquehttps://t.co/xxxxshortlink.tweetToMarkdownwrote that text verbatim, so imported tweets storedhttps://t.co/abc123instead of the real destination. That is hard to read, and worse for search/recall over the memory since the meaningful URL is gone. The real URL is already present onentities.urls(expanded_url+display_url); it just was not being used.Change
Add a pure
expandTweetText(text, urls)and run the tweet body through it intweetToMarkdown:It replaces each
t.coshortlink in the text with a markdown link to the expanded URL, labelled with the human-readabledisplay_url:check this https://t.co/abc123 out→
check this [example.com/article](https://example.com/article) outEntries missing a
urlorexpanded_urlare left untouched, and repeated shortlinks are all expanded. Rendering as a markdown link stays consistent with the rest of the markdown this function already emits (images, bold, etc.).Tests
twitter-expand-text.test.ts(bun:test): single link, multiple links with a repeat,display_url-empty fallback to the expanded URL, entries missing fields left untouched, and the no-op cases (no/empty url entities). 5 tests pass; extensiontscis clean for these files.