Skip to content

Filipponik/rustamanga-mangalib-parser

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

334 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸ“š Mangalib Parser

A fast and efficient Rust-based parser for Mangalib with multiple integration options

Rust Version License CI - Tests Last Commit


✨ Features

  • πŸš€ Fast & Efficient - Built with Rust for maximum performance
  • 🌐 Web Server Mode - Built-in HTTP server with REST API
  • 🐰 RabbitMQ Integration - Consume tasks from message queues
  • 🎯 Multiple Deployment Options - CLI, server, or consumer mode
  • πŸ”„ Concurrent Processing - Handle multiple manga scraping tasks simultaneously
  • 🎨 Modern Stack - Uses Axum + Tokio

πŸ“‹ Table of Contents


πŸ”§ Prerequisites

Before you begin, ensure you have the following installed:

  • Rust 1.70+ - For building from source (optional if using pre-built binaries)

πŸ“¦ Installation

Option 1: Build from Source

git clone git@github.com:Filipponik/rustamanga-mangalib-parser.git
cd rustamanga-mangalib-parser
cargo build --release
cd target/release

Option 2: Download Pre-built Binary

Download the latest release from the GitHub releases page.


πŸš€ Usage

RabbitMQ Consumer

Start a RabbitMQ consumer to process manga scraping tasks from a message queue.

Usage: ./rustamanga-mangalib-parser consume [OPTIONS]

Options:
      --url <URL>            AMQP URI
      --proxy <PROXY>        Proxy URI
  -h, --help                 Print help

Example:

./rustamanga-mangalib-parser consume --url=amqp://guest:guest@localhost:5672

Web Server Mode

Start an HTTP server that accepts POST requests to scrape manga data.

Usage: ./rustamanga-mangalib-parser serve [OPTIONS]

Options:
      --port <PORT>          Web server port
  -h, --help                 Print help

Example:

./rustamanga-mangalib-parser serve --port=12345

The server will be available at http://localhost:{PORT}

API Endpoints

  1. POST /async-command - execute command asynchronously
  2. POST /sync-command - Same, but action will be executed synchronously.

Request Body:

Command: full

{
  "command": "full",
  "params": {
    "slug": "manga-slug",
    "callback_url": "https://example.com",
    "after_chapter": "1",
    "after_volume": "1"
  }
}

Command: only_chapters

{
  "command": "only_chapters",
  "params": {
    "slug": "manga-slug",
    "callback_url": "https://example.com",
  }
}

Command: get_user_list

{
  "command": "get_user_list",
  "params": {
    "user_id": 1,
    "callback_url": "https://example.com",
  }
}

Send Resource

Send a manga resource to a specified callback URL.

Usage: ./rustamanga-mangalib-parser send-resource [OPTIONS]

Options:
      --url <URL>  URL where we should send this resource
  -h, --help       Print help

Example:

./rustamanga-mangalib-parser send-resource --url=https://example.com

πŸ› οΈ Development

Running Tests

cargo test

Project Structure

rustamanga-mangalib-parser/
β”œβ”€β”€ src/                    # Source code
β”œβ”€β”€ tests/                  # Test files
β”‚   └── fixtures/          # Test data
β”œβ”€β”€ resource/              # Helper resources
β”‚   └── json/             # JSON resources
β”œβ”€β”€ Cargo.toml            # Project dependencies
└── README.md             # This file

Configuration

  • Port Settings - Configure via --port flag in serve mode
  • AMQP URL - Set RabbitMQ connection string with --url in consume mode

🀝 Contributing

Contributions are welcome! Here's how you can help:

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add some amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

Guidelines

  • Write clear commit messages
  • Add tests for new features
  • Update documentation as needed
  • Follow Rust coding conventions
  • Ensure all tests pass before submitting

πŸ“ Additional Notes

  • βš™οΈ All configuration options are provided via CLI flags for flexibility
  • πŸ“Š Test fixtures are available in tests/fixtures/ for development
  • πŸ” Static resources are located in resource/json/

πŸ“„ License

This project is licensed under the MIT License - see the LICENSE file for details.


πŸ‘€ Author

Filipponik


Made with ❀️ and πŸ¦€ Rust

About

πŸ“š A fast and efficient Rust-based parser for Mangalib with multiple integration options

Resources

License

Stars

Watchers

Forks

Contributors