Skip to content

Latest commit

 

History

History
1086 lines (907 loc) · 48.1 KB

File metadata and controls

1086 lines (907 loc) · 48.1 KB

Cloudstream 3 Plugin Development Handbook

Parse the site, see how the video links are loaded or find as fast as possible with the shortest time with the mentioned techniques and uses of the Cloudstream 3 API and app capabilities.


Table of Contents

  1. Phase 1: Rapid Reconnaissance (Python & Terminal Prototyping)
  2. Phase 2: Hydration Framework Parsing (WordPress, SvelteKit, Next.js)
  3. Phase 3: Bypassing Barriers (CF WebView Fallback, Immediate Token Form Submission, AES Decryption)
  4. Phase 4: Advanced Redirection Gateways & WebSocket Heartbeat Bypasses
  5. Phase 5: Core MainAPI Implementation & Content Classification
  6. Phase 6: App Ecosystem Mappings (Movie/TV/Anime Metadata Tracker Integration)
  7. Phase 7: Custom Extractor Registry (GDFlix, HubCloud, BuzzServer)
  8. Phase 8: local JVM JUnit Testing Loop
  9. Phase 9: High-Efficiency Debugging & ADB Logging Workflow
  10. Phase 10: Dependency Compilation & JAR Packaging
  11. Phase 11: CI/CD Automation (GitHub Actions & Auto-Publishing)

[CRITICAL FOR AI AGENTS] Repository Isolation & Example Template Bootstrap

To prevent future AI agent runs from corrupting the main repository branch or polluting compiled binaries, AI agents must strictly adhere to the following guardrails:

1. Bootstrapping with ExampleProvider

  • The ExampleProvider module serves as the reference template. When creating a new provider, copy its skeleton structure (src/main/ folder and build.gradle.kts). Do not write project code inside the ExampleProvider module itself.

2. Isolated Gradle Compilation via settings.gradle.kts

  • Avoid executing global repository tasks like ./gradlew build or ./gradlew make. If any other plugin in the repository has a compile error, your compilation pipeline will break.
  • Isolate Your Build: Open settings.gradle.kts and add all other active provider modules to the disabled list, or temporarily restrict the build to your active module:
    // To compile and test ONLY your active target plugin:
    include("MyTargetPlugin")
  • Always invoke Gradle tasks scoped explicitly to your module:
    .\gradlew :<ModuleName>:make
    .\gradlew :<ModuleName>:test

3. Best Practice: Local Build Isolation vs Dynamic CI/CD Compilation

To speed up local development and prevent compilation failures from other unfinished or broken plugins, use local build isolation in combination with dynamic CI/CD compilation.

Local Development

Add any other plugins you are not currently working on to the disabled list inside settings.gradle.kts:

val disabled = listOf<String>("AnimeVerse", "PikaHD")

This ensures Gradle only loads and compiles your target module, avoiding global classpath validation overhead or compile errors in unrelated directories.

Dynamic CI/CD Compilation (GitHub Actions)

Since the production repository needs to compile and publish all plugins on push, the GitHub Actions runner must dynamically enable all plugins before invoking the compilation task. To do this, use an inline Python replacement inside the workflow steps (e.g., in .github/workflows/build.yml) to clear the disabled list on the fly before running Gradle:

      - name: Build Plugins
        run: |
          cd $GITHUB_WORKSPACE/src
          # Dynamically enable all plugins by clearing disabled list in settings.gradle.kts
          python3 -c "content = open('settings.gradle.kts').read(); open('settings.gradle.kts', 'w').write(content.replace('val disabled = listOf<String>(\"AnimeVerse\", \"PikaHD\")', 'val disabled = listOf<String>()'))"
          chmod +x gradlew
          ./gradlew make makePluginsJson

This provides the best of both worlds: robust local isolation for fast debugging, and automated full-repo distribution on git push.

4. Local Scratchpad Isolation

  • Write all temporary Python scraping tests, raw HTML dumps, and JSON logs under the conversation's persistent scratch directory: <appDataDir>\brain\<conversation-id>/scratch/. Never add scratch files to the plugin source paths.

5. Git Push Policy

  • Do not perform any git commits, branch creations, or pushes unless the user explicitly requests them. Perform all validations using local JUnit tests or via local ADB deployment to the test device.

Phase 1: Rapid Reconnaissance (Python & Terminal Prototyping)

Before writing target plugin code in Kotlin, developers should spin up lightweight Python scratch artifacts and terminal commands to test HTML/JS rendering, check cookie scopes, and validate API headers.

1. Terminal cURL Check

To inspect HTTP headers, redirection chains (Location header), and Set-Cookie keys:

curl -i -L -X GET "https://new.pikahd.co/oshi-no-ko-season-1-hindi" \
  -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36..." \
  -H "Referer: https://new.pikahd.co/"
  • Use -i to view HTTP response headers.
  • Omit -L to isolate challenge states.

2. Standalone Python Test Blueprint

For verifying CSS selectors, iframe extractions, and API endpoints:

# scratch/test_parser.py
import requests
import json
import re
from bs4 import BeautifulSoup

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
}

def analyze_target(url):
    session = requests.Session()
    session.headers.update(headers)
    
    r = session.get(url, timeout=10)
    if "Just a moment" in r.text:
        print("[!] Cloudflare challenge detected.")
        return
        
    soup = BeautifulSoup(r.text, 'html.parser')
    
    # Check for external catalogue links (e.g. IMDb)
    imdb_a = soup.find('a', href=re.compile(r'imdb\.com/title/'))
    if imdb_a:
        imdb_id = re.search(r'/title/(tt\d+)', imdb_a.get('href')).group(1)
        print(f"Parsed Catalogue ID: {imdb_id}")
        
    # Find direct player iframe sources
    iframes = soup.find_all('iframe')
    for iframe in iframes:
        src = iframe.get('src', '')
        print(f"Found Player Frame: {src}")

if __name__ == "__main__":
    analyze_target("https://new.pikahd.co/oshi-no-ko-season-1-hindi")

Phase 2: Hydration Framework Parsing

Modern sites do not expose raw links inside standard HTML elements; they serialize database objects inside client-side JS scripts.

1. WordPress (Server-Rendered HTML)

Standard selector querying:

val document = response.document
val items = document.select("article.gridlove-post").map { el ->
    val title = el.select("h1").text()
    val href = el.select("a").attr("href")
    val poster = el.select("img").attr("src")
    newMovieSearchResponse(title, href, TvType.Movie) { this.posterUrl = poster }
}

2. SvelteKit Hydration Parsing (__data.json)

SvelteKit pages load initial database states inside custom serialization lines. The API page structure maps data via the __data.json URL suffix.

The SvelteKit Devalue JSON Decoder

SvelteKit uses a serialization format where list values index back to a central registry. We can parse and resolve this structure programmatically using a devalue decoder:

fun resolveDevalue(dataList: List<Any?>): Any? {
    if (dataList.isEmpty()) return null
    val resolved = mutableMapOf<Int, Any?>()
    val visited = mutableSetOf<Int>()

    fun resolveVal(valObj: Any?): Any? {
        if (valObj is Number) {
            val idx = valObj.toInt()
            if (idx >= 0 && idx < dataList.size) {
                if (resolved.containsKey(idx)) return resolved[idx]
                if (visited.contains(idx)) return null
                visited.add(idx)

                val rawVal = dataList[idx]
                val res = when (rawVal) {
                    is Map<*, *> -> {
                        val resMap = mutableMapOf<String, Any?>()
                        resolved[idx] = resMap
                        for ((k, v) in rawVal) {
                            resMap[k.toString()] = resolveVal(v)
                        }
                        resMap
                    }
                    is List<*> -> {
                        val resList = mutableListOf<Any?>()
                        resolved[idx] = resList
                        for (item in rawVal) {
                            resList.add(resolveVal(item))
                        }
                        resList
                    }
                    else -> rawVal
                }
                visited.remove(idx)
                resolved[idx] = res
                return res
            }
        } else if (valObj is Map<*, *>) {
            return valObj.mapKeys { it.key.toString() }.mapValues { resolveVal(it.value) }
        } else if (valObj is List<*>) {
            return valObj.map { resolveVal(it) }
        }
        return valObj
    }
    return resolveVal(0)
}

Query the hydrated data by appending /__data.json to the target page URL:

suspend fun fetchSvelteData(pageUrl: String): List<Any?> {
    val jsonLines = app.get("$pageUrl/__data.json").text
    val decodedList = mutableListOf<Any?>()
    for (line in jsonLines.split("\n")) {
        val trimmed = line.trim()
        if (trimmed.isEmpty()) continue
        try {
            val lineData = parseJson<Map<String, Any?>>(trimmed)
            if (lineData["type"] == "chunk") {
                val raw = lineData["data"] as? List<Any?>
                if (raw != null) decodedList.add(resolveDevalue(raw))
            }
        } catch (_: Exception) {}
    }
    return decodedList
}

Phase 3: Bypassing Barriers

1. Cloudflare WebView Fallback

Leverage the built-in CloudflareKiller interceptor when a challenge page is detected:

import com.lagradost.cloudstream3.network.CloudflareKiller
import com.lagradost.nicehttp.NiceResponse

private suspend fun cfGet(url: String, referer: String? = null): NiceResponse {
    val headers = mutableMapOf<String, String>()
    if (referer != null) headers["Referer"] = referer

    var response = app.get(url, headers = headers)
    if (response.document.select("title").text() == "Just a moment") {
        // Run WebView challenge solver, store clearance cookies, and retry request
        response = app.get(url, headers = headers, interceptor = CloudflareKiller())
    }
    return response
}

2. Bypassing Countdown Timers

Countdown timers are almost always cosmetic. Find the target redirection endpoint and form tokens, then submit the unlock request immediately.

suspend fun bypassUnlockPage(pageUrl: String): String {
    // 1. Fetch page and harvest hidden inputs
    val doc = app.get(pageUrl).document
    val token = doc.selectFirst("input[name=token]")?.attr("value") ?: ""
    val id = doc.selectFirst("input[name=id]")?.attr("value") ?: ""
    
    // 2. Perform form submission immediately without delays
    val response = app.post(
        "https://links.kmhd.eu/locked?/unlock",
        data = mapOf("token" to token, "id" to id),
        headers = mapOf("Referer" to pageUrl),
        allowRedirects = false
    )
    return response.headers["location"] ?: ""
}

3. AES Decryption

For extracting sources from encrypted parameters:

import javax.crypto.Cipher
import javax.crypto.spec.IvParameterSpec
import javax.crypto.spec.SecretKeySpec
import android.util.Base64

fun decryptAES(encryptedBase64: String, key: String, iv: String): String {
    val cipher = Cipher.getInstance("AES/CBC/PKCS5Padding")
    val keySpec = SecretKeySpec(key.toByteArray(Charsets.UTF_8), "AES")
    val ivSpec = IvParameterSpec(iv.toByteArray(Charsets.UTF_8))
    cipher.init(Cipher.DECRYPT_MODE, keySpec, ivSpec)
    val decodedBytes = Base64.decode(encryptedBase64, Base64.DEFAULT)
    return String(cipher.doFinal(decodedBytes), Charsets.UTF_8)
}

Phase 4: Advanced Redirection Gateways & WebSocket Heartbeat Bypasses

Some premium domains protect link generators via server-side delays validated over WebSockets. A plain thread delay will fail; you must emulate client events.

1. The Hrefli Bypass (bypassHrefli)

Bypasses intermediate landing links that execute multi-stage redirects:

suspend fun bypassHrefli(url: String): String? {
    fun Document.getFormUrl(): String {
        return this.select("form#landing").attr("action")
    }

    fun Document.getFormData(): Map<String, String> {
        return this.select("form#landing input").associate { it.attr("name") to it.attr("value") }
    }

    val host = getBaseUrl(url)
    var res = app.get(url).document
    var formUrl = res.getFormUrl()
    var formData = res.getFormData()

    // Step 1: Submit Form 1
    res = app.post(formUrl, data = formData).document
    formUrl = res.getFormUrl()
    formData = res.getFormData()

    // Step 2: Submit Form 2
    res = app.post(formUrl, data = formData).document
    val skToken = res.selectFirst("script:containsData(?go=)")?.data()?.substringAfter("?go=")
        ?.substringBefore("\"") ?: return null
        
    // Step 3: Fetch Direct Refresh URL
    val driveUrl = app.get(
        "$host?go=$skToken", 
        cookies = mapOf(skToken to "${formData["_wp_http2"]}")
    ).document.selectFirst("meta[http-equiv=refresh]")?.attr("content")?.substringAfter("url=") ?: return null
    
    val path = app.get(driveUrl).text.substringAfter("replace(\"").substringBefore("\")")
    if (path == "/404") return null
    return fixUrl(path, getBaseUrl(driveUrl))
}

2. The Socket.IO WebSocket Bypass (bypassXD)

This emulates mouse clicks, binds credentials, and sends WebSocket heartbeats (Socket.IO) to trick the backend into generating a media download link:

import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.WebSocket
import okhttp3.WebSocketListener
import okhttp3.Response
import kotlinx.coroutines.CompletableDeferred
import kotlinx.coroutines.withTimeout

suspend fun bypassXD(url: String): String? {
    // 1. Get redirect target
    val redirect = app.get(url, allowRedirects = false).headers["location"] ?: return null
    val baseUrl = getBaseUrl(redirect)
    val code = redirect.substringAfterLast("/").takeIf { it.isNotEmpty() } ?: return null
    
    val mouseData = mapOf(
        "eventCount" to 220, "moveCount" to 185, "clickCount" to 3,
        "totalDistance" to 3800, "hasMovement" to true, "duration" to 27000
    )
    val baseHeaders = mapOf(
        "User-Agent" to "Mozilla/5.0...", "Origin" to baseUrl, "Referer" to "$baseUrl/r/$code"
    )

    // ── STEP 1: Create session ────────────────────────────────────────────────
    val sessionJson = JSONObject(
        app.post(
            "$baseUrl/api/session",
            json = mapOf("code" to code, "fingerprint" to "fp_hash", "mouseData" to mouseData),
            headers = baseHeaders
        ).text
    )
    val sessionId = sessionJson.optString("sessionId").takeIf { it.isNotEmpty() } ?: return null
    val cookieHeaders = baseHeaders + mapOf("Cookie" to "sid=$sessionId")

    // ── STEP 2: Rebind Session ────────────────────────────────────────────────
    val rebindJson = JSONObject(
        app.post(
            "$baseUrl/api/session/rebind",
            json = mapOf("fingerprint" to "fp_hash"),
            headers = cookieHeaders
        ).text
    )
    val rebindToken = rebindJson.optString("token").takeIf { it.isNotEmpty() } ?: return null

    // ── STEP 3: WebSocket Connection & Heartbeat ──────────────────────────────
    val wsBaseUrl = baseUrl.replace("https://", "wss://").replace("http://", "ws://")
    val visibleTimeDone = CompletableDeferred<Unit>()
    val okHttpClient = OkHttpClient()
    val wsRequest = Request.Builder()
        .url("$wsBaseUrl/socket.io/?EIO=4&transport=websocket")
        .addHeader("Origin", baseUrl).addHeader("Cookie", "sid=$sessionId").build()

    var heartbeatJob: kotlinx.coroutines.Job? = null
    val webSocket = okHttpClient.newWebSocket(wsRequest, object : WebSocketListener() {
        override fun onOpen(webSocket: WebSocket, response: Response) {
            webSocket.send("40") // Socket.IO connect
        }

        override fun onMessage(webSocket: WebSocket, text: String) {
            when {
                text == "2" -> webSocket.send("3") // Socket.IO ping response
                text.startsWith("40") -> {
                    webSocket.send("""42["bind","$rebindToken"]""")
                    webSocket.send("""42["visibility","visible"]""")

                    heartbeatJob = CoroutineScope(Dispatchers.IO).launch {
                        var elapsed = 0
                        while (elapsed < 28) {
                            delay(1000)
                            elapsed++
                            webSocket.send("""42["heartbeat"]""")
                            webSocket.send("""42["mouseActivity",{"duration":${elapsed * 1000}}]""")
                        }
                        visibleTimeDone.complete(Unit)
                    }
                }
            }
        }
    })

    try {
        withTimeout(40_000) { visibleTimeDone.await() }
    } catch (_: Exception) {
        return null
    } finally {
        heartbeatJob?.cancel()
        webSocket.close(1000, null)
    }

    // ── STEP 4: Complete Session ──────────────────────────────────────────────
    val completeJson = JSONObject(
        app.post(
            "$baseUrl/api/session/complete",
            json = mapOf("fingerprint" to "fp_hash", "mouseData" to mouseData, "honeypot" to ""),
            headers = cookieHeaders
        ).text
    )
    val finalToken = completeJson.optString("token")

    // ── STEP 5: Capture final location ────────────────────────────────────────
    return app.get("$baseUrl/go/$sessionId?t=$finalToken", allowRedirects = false, headers = cookieHeaders).headers["location"]
}

Phase 5: Core MainAPI Implementation & Content Classification

1. Classification Helper

private fun getTvType(title: String): TvType {
    val lower = title.lowercase()
    if (lower.contains("movie") || lower.contains("film")) {
        return if (lower.contains("anime")) TvType.AnimeMovie else TvType.Movie
    }
    val seriesKeywords = Regex("""\b(season|series|episode|episodes|ep|s\d+e\d+|s\d+)\b""")
    if (seriesKeywords.containsMatchIn(lower)) {
        return if (lower.contains("anime")) TvType.Anime else TvType.TvSeries
    }
    return TvType.Movie
}

2. MainAPI Implementation Scaffolding

class ExampleProvider : MainAPI() {
    override var mainUrl = "https://new.pikahd.co"
    override var name = "PikaHD"
    override val supportedTypes = setOf(TvType.Movie, TvType.TvSeries, TvType.Anime)
    override var lang = "en"
    override val hasMainPage = true

    override val mainPage = mainPageOf(
        Pair("trending", "Trending Content"),
        Pair("latest", "Latest Uploads")
    )

    override suspend fun getMainPage(page: Int, request: MainPageRequest): HomePageResponse {
        val list = mutableListOf<SearchResponse>()
        // Parse Svelte __data.json elements...
        return newHomePageResponse(request.name, list, hasNext = true)
    }

    override suspend fun search(query: String): List<SearchResponse> = emptyList()
    
    override suspend fun load(url: String): LoadResponse {
        // Parse detail pages...
        val episodes = mutableListOf<Episode>()
        return newTvSeriesLoadResponse(name, url, TvType.TvSeries, episodes) {
            this.posterUrl = "https://..."
            this.plot = "Synopsis..."
        }
    }
}

3. Core API Edge Case & Fail-Safe Checklist

  • Search / Listing Empty Responses: Always wrap selector queries in safe accessor scopes (selectFirst()?.text().orEmpty()). If a query returns no results, return emptyList() rather than throwing an exception.
  • Malformed or Encoded Titles: Raw HTML often contains entities like &amp;, &#039;, or &quot;. Use a decoder or sanitize strings before presenting them to the UI:
    fun cleanHTMLString(input: String): String {
        return input.replace("&amp;", "&")
                    .replace("&#039;", "'")
                    .replace("&quot;", "\"")
                    .trim()
    }
  • Poster URL Fallbacks: When a scraper fails to parse a poster image, never leave posterUrl empty or null if possible. Use a standard placeholder icon or skip the item safely:
    this.posterUrl = parsedPosterUrl.takeIf { it.startsWith("http") } ?: "https://raw.githubusercontent.com/username/repo/branch/placeholder.png"

Phase 6: App Ecosystem Mappings

Connecting streams to catalogue databases registers items into the tracking system (Simkl, Trakt, AniList, MAL).

                          [Extract Catalogue ID]
                                     │
                    ┌────────────────┴────────────────┐
                    ▼                                 ▼
             [If Movie / TV]                     [If Anime]
                    │                                 │
         Query Stremio Cinemeta               Query Ani.zip API
                    │                                 │
     Get TMDB, Plot, Backdrop, Cast       Get MAL, AniList, Kitsu, EPs
                    │                                 │
             addTMDbId(tmdbId)                  addAniListId(anilistId)
             addImdbId(imdbId)                  addMalId(malId)

1. Movie & TV Series Catalogue Link (via Cinemeta)

Query Stremio's Cinemeta API:

https://v3-cinemeta.strem.io/meta/{movie_or_series}/{imdb_id}.json
import com.lagradost.cloudstream3.LoadResponse.Companion.addImdbId
import com.lagradost.cloudstream3.LoadResponse.Companion.addTMDbId
import com.lagradost.cloudstream3.Actor
import com.lagradost.cloudstream3.ActorData
import com.lagradost.cloudstream3.Score

suspend fun loadMovieOrTv(url: String, imdbId: String, isSeries: Boolean): LoadResponse {
    val metaType = if (isSeries) "series" else "movie"
    val rawJson = app.get("https://v3-cinemeta.strem.io/meta/$metaType/$imdbId.json").text
    val meta = parseJson<MetaPayload>(rawJson).meta

    val actorsList = meta.cast.map { ActorData(Actor(it), role = null) }

    if (isSeries) {
        val episodes = mutableListOf<Episode>()
        // Map videos -> episodes matching season/episode indices
        return newTvSeriesLoadResponse(meta.name, url, TvType.TvSeries, episodes) {
            this.plot = meta.description
            this.posterUrl = meta.poster
            this.backgroundPosterUrl = meta.background
            this.actors = actorsList
            this.score = Score.from10(meta.imdbRating)
            addImdbId(imdbId)
            addTMDbId(meta.moviedb_id.toString())
        }
    } else {
        return newMovieLoadResponse(meta.name, url, TvType.Movie, "link-data") {
            this.plot = meta.description
            this.posterUrl = meta.poster
            this.backgroundPosterUrl = meta.background
            this.actors = actorsList
            this.score = Score.from10(meta.imdbRating)
            addImdbId(imdbId)
            addTMDbId(meta.moviedb_id.toString())
        }
    }
}

2. Anime Catalogue Link (via Ani.zip)

Query the Ani.zip API using a MAL ID:

https://api.ani.zip/mappings?mal_id={mal_id}
import com.lagradost.cloudstream3.LoadResponse.Companion.addMalId
import com.lagradost.cloudstream3.LoadResponse.Companion.addAniListId
import com.lagradost.cloudstream3.LoadResponse.Companion.addKitsuId

suspend fun loadAnime(url: String, malId: Int): LoadResponse {
    val syncText = app.get("https://api.ani.zip/mappings?mal_id=$malId").text
    val mappings = parseJson<AniZipResponse>(syncText)

    val episodes = mutableListOf<Episode>()
    // Build episodes using mapping data: mappings.episodes[epNumber.toString()]
    
    return newAnimeLoadResponse(mappings.titles["en"] ?: "Anime", url, TvType.Anime, episodes) {
        addMalId(malId)
        addAniListId(mappings.mappings.anilistId)
        addKitsuId(mappings.mappings.kitsuId)
    }
}

3. Rate-Limiting & API Failure Resiliency

External mapping services (like Cinemeta, Ani.zip, or TMDB) can be down, geo-blocked, or rate-limited (HTTP 429). The load function must never crash due to external mapping failures.

  • Resiliency Pattern: Wrap API calls in try-catch blocks. If the mapping fails, fall back to basic scraped site metadata (title, plot, poster) so the user can still access the episodes:
    var tmdbId: String? = null
    try {
        val response = app.get("https://api.ani.zip/mappings?imdb_id=$imdbId")
        if (response.code == 200) {
            val data = parseJson<AniZipResponse>(response.text)
            tmdbId = data.mappings?.themoviedbId?.toString()
        }
    } catch (e: Exception) {
        Log.e("PikaHD", "Ani.zip metadata mapping failed: ${e.message}")
        // Continue execution safely with local scraped fallback metadata
    }

Phase 7: Custom Extractor Registry

If the hosting servers are not resolved by the app core by default, register custom extractors.

1. GDFlix Extractor

open class GDFlix : ExtractorApi() {
    override val name = "GDFlix"
    override val mainUrl = "https://gdflix.cfd"
    override val requiresReferer = false

    override suspend fun getUrl(
        url: String,
        referer: String?,
        subtitleCallback: (SubtitleFile) -> Unit,
        callback: (ExtractorLink) -> Unit
    ) {
        val document = app.get(url).document
        val fileSize = document.select("ul > li:contains(Size)").text().substringAfter("Size : ")
        val quality = getQualityFromName(document.title())

        document.select("div.text-center a").forEach { anchor ->
            val text = anchor.text()
            val link = anchor.attr("href")

            when {
                text.contains("DIRECT DL", true) -> {
                    callback.invoke(
                        newExtractorLink("GDFlix [Direct]", "GDFlix [Direct] [$fileSize]", link) { this.quality = quality }
                    )
                }
                text.contains("PixeLServer", true) || text.contains("Pixeldrain", true) -> {
                    val finalURL = if (link.contains("download")) link else "${getBaseUrl(link)}/api/file/${link.substringAfterLast("/")}?download"
                    callback.invoke(
                        newExtractorLink("GDFlix [Pixeldrain]", "GDFlix [Pixeldrain] [$fileSize]", finalURL) { this.quality = quality }
                    )
                }
            }
        }
    }
}

2. HubCloud Extractor

open class HubCloud : ExtractorApi() {
    override val name = "Hub-Cloud"
    override val mainUrl = "https://hubcloud.club"
    override val requiresReferer = false

    override suspend fun getUrl(
        url: String,
        referer: String?,
        subtitleCallback: (SubtitleFile) -> Unit,
        callback: (ExtractorLink) -> Unit
    ) {
        val href = if ("hubcloud.php" in url) url else {
            app.get(url).document.selectFirst("#download")?.attr("href").orEmpty()
        }
        if (href.isBlank()) return

        val document = app.get(href).document
        val size = document.selectFirst("i#size")?.text().orEmpty()
        val quality = getQualityFromName(document.title())

        document.select("a.btn").forEach { element ->
            val link = element.attr("href")
            val label = element.ownText().lowercase()

            when {
                "fslv2" in label || "fsl" in label -> {
                    callback(newExtractorLink("HubCloud FSL", "HubCloud [FSL] [$size]", link) { this.quality = quality })
                }
                "buzzserver" in label -> {
                    val resp = app.get("$link/download", referer = link, allowRedirects = false)
                    val dlink = resp.headers["hx-redirect"] ?: resp.headers["HX-Redirect"].orEmpty()
                    if (dlink.isNotBlank()) {
                        callback(newExtractorLink("BuzzServer", "BuzzServer [$size]", dlink) { this.quality = quality })
                    }
                }
                "pixeldrain" in label || "pixel server" in label -> {
                    val finalUrl = if ("download" in link) link else "${getBaseUrl(link)}/api/file/${link.substringAfterLast("/")}?download"
                    callback(newExtractorLink("Pixeldrain", "Pixeldrain [$size]", finalUrl) { this.quality = quality })
                }
            }
        }
    }
}

3. UI Label Sanitization & Stream Integrity Checklist

  • Stripping Metadata Counters: Scoped file sizes or quality lists on index pages often contain view/download counters (e.g. 1.2 GB | Views : 2123). Pre-sanitize the size strings before injecting them into newExtractorLink:
    val cleanSize = rawSize.substringBefore("|").replace(Regex("(?i)views.*"), "").trim()
  • Referer Sanitization: Many CDNs proxying raw files return 403 Forbidden if a referrer is present. By default, do NOT assign this.referer inside the newExtractorLink lambda unless the server is verified to check the referrer.

Phase 8: local JVM JUnit Testing Loop

Run unit tests directly on the local JVM:

Create <ModuleName>/src/test/kotlin/com/example/ExampleTest.kt:

package com.example

import kotlinx.coroutines.runBlocking
import org.junit.jupiter.api.Test

class ExampleTest {
    private val provider = ExampleProvider()

    @Test
    fun testSearch() = runBlocking {
        val results = provider.search("Naruto")
        assert(results.isNotEmpty())
        println("Success. Found: ${results.size} matches.")
    }
}

Run test suite via Gradle:

.\gradlew :<ModuleName>:test --info

Output variables, stack traces, and parsing errors will log directly into your console.

2. Comprehensive Test Assertions Boilerplate

To catch edge cases in search results, season numbers, and episode descriptions:

package com.example

import kotlinx.coroutines.runBlocking
import org.junit.jupiter.api.Test
import com.lagradost.cloudstream3.TvType
import com.lagradost.cloudstream3.Episode

class ExampleTest {
    private val provider = ExampleProvider()
    private val testSeriesUrl = "https://new.pikahd.co/oshi-no-ko-season-1-hindi"

    @Test
    fun testProviderMetadata() = runBlocking {
        println("[*] Loading URL: $testSeriesUrl")
        val response = provider.load(testSeriesUrl)
        
        // 1. Verify Core Details
        assert(response.name.isNotBlank()) { "Error: Title card is blank!" }
        assert(response.plot?.isNotBlank() == true) { "Error: Plot synopsis is missing!" }
        assert(response.posterUrl?.startsWith("http") == true) { "Error: Poster URL is invalid: ${response.posterUrl}" }
        
        println("Title: ${response.name}")
        println("Poster: ${response.posterUrl}")
        println("Plot: ${response.plot}")

        // 2. Verify Episode Indexing
        val episodes = if (response is com.lagradost.cloudstream3.TvSeriesLoadResponse) {
            response.episodes
        } else emptyList()

        assert(episodes.isNotEmpty()) { "Error: Mapped episode list is empty!" }
        println("Total Mapped Episodes: ${episodes.size}")

        // 3. Verify Individual Episode Integrity
        episodes.forEachIndexed { index, ep ->
            assert(ep.episode > 0) { "Error: Episode number at index $index is invalid: ${ep.episode}" }
            assert(ep.name?.isNotBlank() == true) { "Error: Episode ${ep.episode} has a blank title!" }
            assert(ep.data.isNotBlank()) { "Error: Playback payload for Episode ${ep.episode} is empty!" }
            
            println("  - Ep ${ep.episode}: ${ep.name} | Payload: ${ep.data.take(100)}...")
        }
    }

    // 4. Verify Real-time Video Stream Playability (Recommended JVM Validation)
    @Test
    fun testVideoStreamIntegrity() = runBlocking {
        println("[*] Fetching and verifying playback stream links directly...")
        val response = provider.load(testSeriesUrl)
        val epData = if (response is com.lagradost.cloudstream3.TvSeriesLoadResponse) {
            response.episodes.firstOrNull()?.data
        } else if (response is com.lagradost.cloudstream3.MovieLoadResponse) {
            response.data
        } else null

        assert(epData != null) { "Error: No episode/movie data payload found!" }

        val resolvedLinks = mutableListOf<com.lagradost.cloudstream3.utils.ExtractorLink>()
        provider.loadLinks(epData!!, isCasting = false, subtitleCallback = {}, callback = { link ->
            resolvedLinks.add(link)
        })

        assert(resolvedLinks.isNotEmpty()) { "Error: Extractor failed to resolve any playable links!" }

        resolvedLinks.forEach { link ->
            println("[*] Validating resolved stream: ${link.name} -> ${link.url}")
            try {
                // Perform a quick GET request to verify HTTP status and avoid 403/404s
                val checkHeaders = mutableMapOf<String, String>()
                link.referer.takeIf { !it.isNullOrBlank() }?.let { checkHeaders["Referer"] = it }
                link.headers.forEach { (k, v) -> checkHeaders[k] = v }

                val streamCheck = app.get(link.url, headers = checkHeaders, timeout = 5000L)
                val status = streamCheck.code
                println("  -> Stream Status: HTTP $status (Content-Type: ${streamCheck.headers["Content-Type"]})")
                
                assert(status in 200..399) {
                    "Error: Resolved link returns invalid status HTTP $status! (Potential referer/token block)"
                }
            } catch (e: Exception) {
                println("  -> Connection Failed: ${e.message}")
                if ("403" in e.message.orEmpty() || "404" in e.message.orEmpty()) {
                    throw e
                }
            }
        }
    }
}

5. Advanced Mocking & Redirects Handling

  • Redirect Loop Prevention: Set allowRedirects = false in app.get() or app.post() when checking redirections manually to capture intermediate headers (like Location or hx-redirect) without hitting a redirect cycle or HTTP 307 loops.
  • OkHttp Cookie Handshake: If an extractor relies on session state, initialize the HTTP client cookies by running a dummy GET to the landing domain, extracting the returned cookies, and passing them to subsequent POST requests.

Phase 9: High-Efficiency Debugging & ADB Logging Workflow

Once the code passes local JVM unit tests, compile and push it to a connected Android test device. If the plugin fails to load or links do not resolve on-device, use Android's ADB logcat to capture runtime stack traces.

1. Compile & Deploy Script

Save this block as a terminal alias or run it as a unified command to build, push, and force-restart the Cloudstream application in under 5 seconds:

# Compile the plugin binary (.cs3 file)
.\gradlew :PikaHD:make

# Remove old binary instances from the application plugin directories
adb shell rm -f /sdcard/Android/data/com.lagradost.cloudstream3/files/plugins/PikaHD.cs3
adb shell rm -f /sdcard/Cloudstream3/plugins/PikaHD.cs3

# Push the newly compiled binary to the device
adb push "PikaHD\build\PikaHD.cs3" /sdcard/Android/data/com.lagradost.cloudstream3/files/plugins/PikaHD.cs3

# Force-stop and relaunch the application to reload the plugin classes
adb shell am force-stop com.lagradost.cloudstream3
adb shell monkey -p com.lagradost.cloudstream3 -c android.intent.category.LAUNCHER 1

2. Monitoring Device Logs (adb logcat)

If the application crashes, a stream doesn't play, or the plugin fails to register:

  1. Open a terminal and clear the log cache:
    adb logcat -c
  2. Spawn a live log listener, filtering specifically for the plugin class, the Cloudstream player, and standard JVM outputs:
    adb logcat -s Extractor:* Cloudstream:* System.out:I System.err:E *:S

3. Troubleshooting Common On-Device Failures

A. Class Loading / Linker Errors

  • Symptom: The plugin installs but is missing from the providers list, or the app logs java.lang.NoClassDefFoundError or java.lang.ClassNotFoundException.
  • Cause: You are using external library packages (e.g. cryptography or networking jars) that are not bundled inside the target .cs3 dex classpath.
  • Fix: Ensure helper utilities are implemented directly inside the plugin Kotlin files or compiled statically. Avoid importing external libraries in build.gradle.kts unless they are explicitly marked as compile-only dependencies provided by the Cloudstream core app wrapper.

B. SSL / Handshake Failures

  • Symptom: Networks call succeed locally on the JVM but throw javax.net.ssl.SSLHandshakeException: Trust anchor for certification path not found on-device.
  • Cause: Older Android devices lack updated Root Certificates to trust Let's Encrypt certificates served by the streaming providers.
  • Fix: Force the HTTP client to ignore certificate verification only for trust-broken debug links (or wrap the requests in custom user-agents to bypass regional ISP network blocks).

C. Cloudflare challenge Loop

  • Symptom: The WebView challenge triggers, solves, but the request still fails on-device.
  • Cause: The webview's User-Agent solved header does not match the User-Agent header passed in app.get().
  • Fix: Use the app's default User-Agent for all HTTP requests to ensure solved clearance cookies align with the client identity.

D. Playback Failures (Referer / Token Blocks)

  • Symptom: The list of links resolves successfully in the app, but clicking play throws a loading spinner forever or a playback failed error message.
  • Cause: Explicit referers or tokens are appended incorrectly, or a required referer is missing. Many file-hosting CDNs (like Amazon S3, Google Drive workers, FSL, Mega, and Pixeldrain) reject connections with an HTTP 403/404 if the player passes a webpage domain in the Referer header (to prevent bandwidth hotlinking).
  • Fix:
    • Rule of Thumb: Assume stream links do not want a referer unless verified. Avoid setting this.referer = newUrl or this.referer = href in the newExtractorLink block unless explicitly required by the server (e.g. VidStack / VidPlay).
    • If a stream throws a playback error, try setting this.referer = null or removing the referer field entirely to let the player request it referer-less.
    • Double check if the extractor URL has short-lived tokens that expired between extraction and playback.

4. Playback Diagnostics & ADB Monitoring Workflow

To verify if a video actually plays and debug player-level streaming blocks in real-time:

  1. Clear logcat:
    adb logcat -c
  2. Trigger playback on the device.
  3. Capture Player and Media Extractor logs:
    adb logcat | Select-String -Pattern "Stagefright", "ExoPlayer", "HttpDataSource", "setDataSource", "0x80000000", "403"
  4. Identify key Native Player Error Codes:
    • status = 0x80000000: The standard Stagefright/Android MediaPlayer error meaning the source failed to load (most commonly an HTTP 403 Forbidden or 404 Not Found returned by the proxy worker).
    • HttpDataSource$InvalidResponseCodeException: Response code: 403: ExoPlayer threw an error because the stream server returned a 403 (usually a referer/User-Agent mismatch or expired token).

5. ADB Cheat Sheet for Plugin Management

  • Clear app cache: adb shell pm clear com.lagradost.cloudstream3 (forces reload of local storage and configurations).
  • Check installed plugin directory permissions: adb shell ls -la /sdcard/Android/data/com.lagradost.cloudstream3/files/plugins/
  • Check connected devices: adb devices
  • Restart ADB server: adb kill-server && adb start-server (resolves connection drop issues).

Phase 10: Dependency Compilation & JAR Packaging

When building Cloudstream plugins, you do not distribute or install separate .jar files to the Android application. Instead, all external dependencies and local JAR libraries must be dexed and packaged directly into the unified classes.dex inside the .cs3 plugin file.

1. How the Build System Compiles JARs

During the ./gradlew make task:

  1. Gradle compiles the Kotlin/Java source files of the plugin.
  2. The compilation pipeline fetches all declared implementation configurations (e.g. Maven coordinates or local JARs).
  3. The Android D8 (or R8) dexer processes all compiled classes plus all dependency classes from the external JARs.
  4. It outputs a single, consolidated Dalvik Executable file: classes.dex.
  5. This DEX file is zipped into the final .cs3 package. When loaded, a DexClassLoader mounts the class paths in the Android app space.

2. Declaration Configurations in build.gradle.kts

A. Maven Dependencies

Add Maven dependencies under the dependencies block using implementation to package them inside the DEX:

dependencies {
    // Compiled and packaged directly inside classes.dex
    implementation("org.bouncycastle:bcpkix-jdk18on:1.84")
    implementation("org.mozilla:rhino:1.8.1")
}

B. Local .jar Libraries

If you have a custom or proprietary .jar file that is not hosted on Maven Central/Jitpack:

  1. Create a directory named libs inside the module directory (e.g., PikaHD/libs/).
  2. Put the my-library.jar file inside this libs/ folder.
  3. Reference it in the module's build.gradle.kts file:
    dependencies {
        // Package all .jar files in the libs/ folder into the DEX
        implementation(fileTree(mapOf("dir" to "libs", "include" to listOf("*.jar"))))
        
        // Or reference a single jar explicitly:
        // implementation(files("libs/my-library.jar"))
    }

3. Difference Between implementation and cloudstream Configurations

In your Gradle scripts, pay close attention to the dependency configurations:

  • implementation(...): Tells the compiler to compile and package the library classes directly inside the plugin's classes.dex. Use this for all helper libraries (e.g. rhino, custom decrypters, specific JSON engines) that are not provided by the app.
  • cloudstream(...) or compileOnly(...): Tells the compiler that the class libraries are already present in the core Cloudstream application runtime. D8 will not package them into your plugin. Using implementation on the Cloudstream core library will cause class-loading crashes or build size bloat on-device.

Phase 11: CI/CD Automation (GitHub Actions & Auto-Publishing)

To automate compiling and hosting your plugins, configure a GitHub Actions workflow. The runner compiles the modules, creates a repository manifest list (plugins.json), and automatically pushes the built .cs3 files to a dedicated builds branch.

1. The GitHub Actions Workflow Config

Save the following config as .github/workflows/build.yml in your repository:

name: Build and Host Plugins

concurrency:
  group: "build"
  cancel-in-progress: true

on:
  push:
    branches:
      - master
      - main
    paths-ignore:
      - '*.md'

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      # 1. Checkout the source code branch (master/main) into /src
      - name: Checkout Source
        uses: actions/checkout@v6
        with:
          path: "src"

      # 2. Checkout the builds hosting branch into /builds
      - name: Checkout Builds Branch
        uses: actions/checkout@v6
        with:
          ref: "builds"
          path: "builds"

      # 3. Clean old releases inside builds directory
      - name: Clean Old Releases
        run: |
          rm $GITHUB_WORKSPACE/builds/*.cs3 || true
          rm $GITHUB_WORKSPACE/builds/*.jar || true
          rm $GITHUB_WORKSPACE/builds/plugins.json || true

      # 4. Set up Java Environment
      - name: Set up JDK 17
        uses: actions/setup-java@v5
        with:
          distribution: "temurin"
          java-version: "17"

      # 5. Setup Gradle caching wrapper
      - name: Setup Gradle
        uses: gradle/actions/setup-gradle@v5
        with:
          cache-read-only: false

      # 6. Write API Key secrets to local.properties (if your providers use keys)
      - name: Populate Secrets
        env:
          TMDB_API_KEY: ${{ secrets.TMDB_API_KEY }}
        run: |
          cd $GITHUB_WORKSPACE/src
          echo TMDB_API_KEY=$TMDB_API_KEY >> local.properties

      # 7. Compile Plugins & Generate plugins.json Manifest
      - name: Build Plugins
        run: |
          cd $GITHUB_WORKSPACE/src
          chmod +x gradlew
          
          # makePluginsJson task compiles all modules, dexes them, and creates plugins.json
          ./gradlew makePluginsJson
          
          # Copy outputs (both dex .cs3 and class .jar files)
          cp **/build/*.cs3 $GITHUB_WORKSPACE/builds/
          cp **/build/*.jar $GITHUB_WORKSPACE/builds/ || true
          cp build/plugins.json $GITHUB_WORKSPACE/builds/

      # 8. Commit and Push outputs back to the builds branch
      - name: Push Builds
        run: |
          cd $GITHUB_WORKSPACE/builds
          git config --local user.email "actions@github.com"
          git config --local user.name "GitHub Actions"
          git add .
          # Commits outputs, keeping history clean by amending the previous build commit
          git commit --amend -m "Build $GITHUB_SHA" || exit 0
          git push --force

2. Required Repository Configuration

For this workflow to complete successfully:

  1. Create the builds Branch: Create an empty branch named builds in your repository.
  2. Enable Workflow Write Permissions: Go to Settings -> Actions -> General -> Workflow permissions and select Read and write permissions. This is mandatory to allow the GitHub Actions runner to push back compiled builds to your repository.
  3. Repository secrets: If you define API keys (e.g. TMDB_API_KEY) under local.properties locally, go to Settings -> Secrets and variables -> Actions and declare them so the runner compiles them into your dex files.

3. Plugin Versioning & Release Updates

To ensure that users who have already installed your plugin receive updates, you must manage versioning properly:

  1. Version Code Declaration: Inside each provider module's build.gradle.kts (e.g. AnimeVerse/build.gradle.kts), the plugin version is specified as an integer:

    version = 2

    Note: This must be a simple integer (1, 2, 3, etc.). Do not use decimals like 1.1 or 2.0.

  2. Triggering Client Updates:

    • The CI/CD workflow compiles the plugin and generates a plugins.json manifest representing the latest builds.
    • When a user launches the Cloudstream app, it periodically refetches plugins.json and compares the version code of the installed plugin with the one in the manifest.
    • If the remote version is higher than the installed version, the app displays a green "Update" button in Settings -> Plugins next to the provider.
  3. When to Increment:

    • Always increment the version code right before pushing code to GitHub for any release update (bug fixes, parser updates, new stream sources).
    • If you push changes without incrementing the version, users will not see the "Update" button, forcing them to manually uninstall and reinstall the plugin to get the fixes.