YouTube Subtitle Markdown

This project is a Node.js pipeline that reads a Markdown file containing a list of YouTube videos and generates a single Markdown file with full subtitle transcriptions, in the original language of each video, whenever available.

It is designed to be:

• resilient to failures
• restartable
• parallelized
• compatible with modern YouTube subtitle quirks

---

✨ Features

• Reads a .md file with YouTube links
• Downloads subtitles using yt-dlp
• Automatically selects the best available subtitle language
• Supports manual subtitles, auto-generated subtitles, and *-orig tracks
• Cleans VTT files (removes timestamps, tags, and formatting)
• Writes a clean, readable transcription per video
• Parallel processing (configurable)
• Retry mechanism with failure tracking
• Persistent progress (can resume after crashes or network loss)
• Detailed logging to file
• Modular architecture for easy maintenance and testing

---

📁 Project Structure

.
├── src/
│   ├── config/
│   │   └── constants.js           # All configuration constants
│   ├── utils/
│   │   ├── file.js                # File operations
│   │   ├── logger.js              # Logging system
│   │   └── vtt.js                 # VTT cleaning utilities
│   ├── services/
│   │   ├── markdown.js            # Input MD parsing
│   │   ├── ytdlp.js               # yt-dlp integration
│   │   └── progress.js            # Progress tracking
│   ├── workers/
│   │   └── videoProcessor.js      # Video processing logic
│   └── index.js                   # Main entry point
├── video-list.md                  # Input file (list of videos)
├── video-list-transcription.md    # Output file (generated)
├── progress.json                  # Progress tracking
├── tmp_subs/                      # Temporary subtitle downloads
├── logs/
│   └── app.log                    # Detailed execution logs
├── package.json
└── README.md

---

🏗️ Architecture

The project follows a modular architecture with clear separation of concerns:

config/

Centralized configuration and constants

• Paths, timeouts, parallel limits, log levels

utils/

Reusable utility functions

• logger.js: Multi-level logging system
• file.js: File and directory operations
• vtt.js: VTT subtitle cleaning

services/

Business logic and external integrations

• markdown.js: Parse input Markdown file
• ytdlp.js: All yt-dlp interactions (download, list subs, etc.)
• progress.js: Save/load progress tracking

workers/

Processing and orchestration

• videoProcessor.js: Individual video processing and parallel execution

index.js

Main entry point that orchestrates the entire pipeline

---

📝 Input Format

The input file must be a Markdown file with links in this format:

[Video Title](https://www.youtube.com/watch?v=VIDEO_ID)

Example:

[How to Build a CLI Tool](https://www.youtube.com/watch?v=abc123)
[Node.js Best Practices](https://www.youtube.com/watch?v=xyz789)

---

📤 Output Format

## Video Title
https://www.youtube.com/watch?v=VIDEO_ID

Full transcription text goes here...

## Another Video Title
https://www.youtube.com/watch?v=ANOTHER_ID

Another full transcription...

---

🧠 Subtitle Selection Logic

1. *-orig subtitles (original language track)
2. Single available manual subtitle
3. Auto-generated subtitle (fallback)
4. Fail only if no subtitles exist

This ensures you always get the highest quality subtitle available.

---

🚀 Usage

# Using npm scripts
npm start

# Or directly with Node.js
node src/index.js

# Development mode with auto-reload
npm run dev

You can stop and restart at any time. Progress is saved automatically in progress.json.

---

⚙️ Configuration

Edit src/config/constants.js to customize:

export const SETTINGS = {
  MAX_PARALLEL: 6,      // Number of concurrent downloads
  MAX_RETRIES: 3,       // Retry attempts per video
  TIMEOUT_MS: 30_000,   // Timeout for each operation
};

---

📦 Requirements

• Node.js 18+ (ES modules support)
• yt-dlp executable (must be in project root or PATH)
• Recommended: ffmpeg, Node.js runtime for yt-dlp

Installing yt-dlp

# Windows
# Download from https://github.com/yt-dlp/yt-dlp/releases

# macOS/Linux
pip install yt-dlp
# or
brew install yt-dlp

---

🔧 Development

Adding New Features

1. New utilities → src/utils/
2. New services → src/services/
3. New processing logic → src/workers/
4. Configuration changes → src/config/constants.js

Testing

The modular structure makes it easy to test individual components:

// Example: Testing VTT cleaning
import { cleanVtt } from './src/utils/vtt.js';

const dirty = 'WEBVTT\n\n00:00:01.000 --> 00:00:05.000\nHello world';
const clean = cleanVtt(dirty);
console.log(clean); // "Hello world"

---

📊 Logging

Logs are written to logs/app.log with timestamps and severity levels:

• DEBUG: Detailed execution info
• INFO: General progress updates
• WARN: Non-critical issues
• ERROR: Failures with retry info
• FATAL: Critical errors that stop execution

---

🔄 Progress Tracking

The progress.json file tracks:

• ✅ "done": Successfully processed
• ❌ "failed": Failed after all retries

Delete this file to reprocess all videos.

---

🐛 Troubleshooting

No subtitles found

• Check if the video has subtitles enabled
• Try running yt-dlp --list-subs VIDEO_URL manually

Timeout errors

• Increase TIMEOUT_MS in src/config/constants.js
• Check your internet connection

yt-dlp not found

• Ensure yt-dlp.exe is in the project root
• Or update PATHS.YT_DLP in src/config/constants.js

---

📜 License

This project is under MIT license. Check the file LICENSE for details.

---

Made with 💗 by Renan Oliveira