Reviewing Tests

[dale-wahl]

Instagram

[✔️] instagram.com: Tag overview — https://www.instagram.com/explore/tags/blessed/ — 0 / 0 (expected more after scroll)

• fixed here 71526f3

Tiktok

May be issue with wait; worked manual tests
[✔️] tiktok.com: Audio overview — https://www.tiktok.com/music/Oh-No-6586947002464996102 — 0 / 0 (expected more after scroll)

• media URL dead; replaced w/ new and all good

[✔️] tiktok.com: User page — https://www.tiktok.com/@eliminatehq — 0 / 0 (expected more after scroll)
[✔️] tiktok.com: Tag overview — https://www.tiktok.com/tag/blessed — 0 / 0 (expected more after scroll)
tiktok.com: Search results (video tab) — https://www.tiktok.com/search/video?q=ok boomer — 0 / 0 (expected more after scroll)

Tiktok comments

[✔️] tiktok-comments: Comments — https://www.tiktok.com/@eliminatehq/video/7340364794802752811 — 0 / 0 (expected more after scroll)

Imgur

[❌] imgur.com: Search results — https://imgur.com/search?q=yakuza — 0 / 0 (expected more after scroll)

LinkedIn

[❌] linkedin.com: Home feed — https://www.linkedin.com/feed/ — 0 / 0 (expected more after scroll)

X

[✔️] twitter.com: User page (with replies) — https://x.com/HIDEO_KOJIMA_EN/with_replies — 0 / 0 (expected more after scroll)
[❓] twitter.com: User page (likes) — https://x.com/HIDEO_KOJIMA_EN/likes — 0 / 0 (expected more after scroll)
Possibly does not exist; redirects to https://x.com/HIDEO_KOJIMA_EN
[✔️] twitter.com: Search results (hashtag) — https://x.com/search?q=%23blessed — 0 / 0 (expected more after scroll)
[✔️] twitter.com: List — https://x.com/i/lists/216472995 — 0 / 0 (expected more after scroll)
[❌] twitter.com: Single tweet with replies — https://x.com/dril/status/247222360309121024 — 0 / 0 (expected more after scroll)
Works after scroll, but first posts fail

Douyin

[❓] douyin.com: Search — https://www.douyin.com/search/%E6%90%9E%E7%AC%91 — 0 / 0 (expected more after scroll)
This requires a login to test which I do not have.

Xiaohongshu

[✔️] xiaohongshu.com: Explore (main, with scrolling) — https://www.xiaohongshu.com/ — 0 / 0 (expected more after scroll)
[✔️] xiaohongshu.com: Explore (category) — https://www.xiaohongshu.com/explore?channel_id=homefeed.fashion_v3 — 0 / 0 (expected more after scroll)
[✔️] xiaohongshu.com: Search results — https://www.xiaohongshu.com/search_result?keyword=yakuza&source=web_user_page — 0 / 0 (expected more after scroll)

[dale-wahl]

Douyin looks down now.

[dale-wahl]

Twitter also has issue with query URLs. Captures correctly, but missing author info (author id does exist).

{
    "source_urls_unique": [
      "https://x.com/search?q=clavicular%20until%3A2026-04-20%20since%3A2026-03-15&src=typed_query&f=live"
    ],
    "per_signature": [
      {
        "label": "KeyError 'result' at search_twitter.py:117",
        "count": 8,
        "source_urls": [
          "https://x.com/search?q=clavicular%20until%3A2026-04-20%20since%3A2026-03-15&src=typed_query&f=live"
        ],
        "item_ids": [
          "2045681907342487644",
          "2045682334700093686",
          "2045682422977556608",
          "2045683472233009531",
          "2045683740756832430",
          "2045683805999034617",
          "2045684308430459128",
          "2045684578736918705"
        ]
      },
      {
        "label": "KeyError 'core' at search_twitter.py:137",
        "count": 2,
        "source_urls": [
          "https://x.com/search?q=clavicular%20until%3A2026-04-20%20since%3A2026-03-15&src=typed_query&f=live"
        ],
        "item_ids": [
          "2045569838257442971",
          "2045569897258787102"
        ]
      }
    ]
  }

We should review and see if we are missing data from somewhere. Could be previous item has full data and other items reference it.

[dale-wahl]

rednote seems to be collecting some junk items:

{
          "index": 47,
          "item": {
            "cover": {
              "url_default": "http://sns-webpic-qc.xhscdn.com/202604300005/27e075e4c015190794e976c8d40ec1bb/1040g2sg31vgkl0fn3m705q69fuu8dv00nr7fopo!nc_n_webp_mw_1"
            },
            "xsec_token": "ABnvsD9EJsppGV4peDWxkSR_Eu9h9xpSfmMjwXkfbOHp0=",
            "note_id": "69f096e00000000035022946",
            "type": "video",
            "id": "69f096e00000000035022946",
            "__import_meta": {
              "nav_index": "19:20:NaN",
              "item_id": "69f096e00000000035022946",
              "timestamp_collected": 1777478744934,
              "source_platform": "xiaohongshu.com",
              "source_platform_url": "https://www.xiaohongshu.com/explore/69d7bc13000000001f001f2d?xsec_token=ABp4VcdiEF0T6qxwrGeMeif3AWKh_yDYBXnOsEaafAsSQ=&xsec_source=pc_search&source=web_search_result_notes",
              "source_url": "https://edith.xiaohongshu.com/api/sns/web/v1/user/hover_card?target_user_id=68c97fbc000000002102fc00&image_formats=jpg,webp,avif&xsec_source=pc_comment&xsec_token=ABjrsE-rAcEEnKglcnFcwnKwn1uPLSUNt-50wy1vTAybM%3D",
              "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:149.0) Gecko/20100101 Firefox/149.0",
              "id": 29054
            }
          }
        }

The source_url implies it is some pop up. Maybe hovering over something.