RClone-on-Cloud

Description

The RClone-on-Cloud project aims to provide a secure, scalable, and stateless web service for rclone. It stores the rclone configuration files in a secure, centralized MongoDB storage, encrypted with AES-256-GCM and authenticated with a Google OAuth2 + JWT gateway.

The RClone-on-Cloud project is a full stack web development project. It is comprised of several components: the Web API, Web UI, and the CLI Client.

Table of Contents

• Walkthrough
• Installation
• Usage
• Credits
• License

Walkthrough of this project

This project consists of several components, each responsible for performing a certain task to manage your cloud configurations securely. The diagram below illustrates the system architecture of the project.

graph LR
    subgraph "User's Terminal"
        CLI[CLI Client]
    end

    subgraph "Cloud / Storage"
        MongoDB[(RClone Encrypted Configs \nin MongoDB)]
    end

    subgraph "Cloud / Web"
        WebAPI[Web API]
    end

    subgraph "User's Browser"
        WebUI[Web UI]
    end

    WebUI -->|"Makes \nHTTP requests"| WebAPI

    WebAPI <-->|"Fetches, updates, and watches config changes from DB"| MongoDB

    CLI <-->|"Fetches, updates, and watches config changes from DB"| MongoDB

Loading

Users can use the front-end web application to browse their cloud files from their remotes directly from their browser.

The application includes a file viewers to easily preview your content stored on the cloud.

Users can also modify content in the cloud, such as uploading new content, renaming files / directories, moving files / directories, and deleting files / directories.

It also provides a mobile-responsive interface for managing your files on the go.

Installation

Required Programs and Tools:

• Go 1.25+
• Node.js & npm
• MongoDB 7.0+ (Local or Atlas)

Set up the database

• Install MongoDB on your machine or use a MongoDB Atlas instance and remember its connection string.

• Generate a 32-byte encryption key for the MongoDB configs by running:

  openssl rand -base64 32

  and remember it.

Set up the Web API:

1. Set up your Google Cloud OAuth2 credentials here.

2. Navigate to the apps/web-api directory.

3. Generate an RSA or Ed25519 key pair for JWT signing and add the PEM strings to your .env (refer to the Web API README for details).

4. Create a .env file to set the following minimum required environment variables:

   RCLONE_CONFIG_MONGO_KEY=<your-mongo-key>
   RCLONE_CONFIG_MONGO_URI=<your-mongo-uri>
   RCLONE_CONFIG_MONGO_DB=rclone
   RCLONE_CONFIG_MONGO_COL=configs
   
   AUTH_GOOGLE_CLIENT_ID=123
   AUTH_GOOGLE_CLIENT_SECRET=<your-google-oauth-client-secret>
   AUTH_ALLOWED_GOOGLE_IDS=<your-google-id>
   AUTH_GOOGLE_REDIRECT_URL=http://localhost:4200/auth/v1/google/callback
   
   AUTH_JWT_PRIVATE_KEY=<your-jwt-private-key>
   AUTH_JWT_PUBLIC_KEY=<your-jwt-public-key>
   
   LISTEN_ADDR=:3000
   CORS_ALLOWED_URLS=http://localhost:4200

5. Download dependencies and run the API:

   go mod download
   go run .

   or run via Docker:

   docker build -t rclone-cloud-web-api .
   docker run -d --name rclone-web-api --env-file .env rclone-cloud-web-api

Set up the Web UI:

1. Navigate to the apps/web-ui directory.

2. Install the project's dependencies:

   npm install

3. Create a .env file to store your API endpoints:

   NG_APP_LOGIN_URL=http://localhost:3000/auth/v1/google
   NG_APP_WEB_API_ENDPOINT=http://localhost:3000
   

4. Run the development server:

   npm run dev

5. Now, on your browser, go to http://localhost:4200. Log in, and you should get a 403. In the logs, there will be a message like Unauthorized user: XYZ. Copy-paste XYZ and set it to the AUTH_GOOGLE_CLIENT_ID environment variable in your web api's .env file like:

   ...
   AUTH_GOOGLE_CLIENT_ID=XYZ
   ...

6. Restart the server (if you launched your server with Docker, run docker stop rclone-web-api) and re-login to the web page. You should see a blank page in the Remotes page.

Set up the CLI:

1. Navigate to the apps/cli directory.

2. Build the binary locally:

   go build -o rclone-cloud .

3. Set your environment variables (MONGO_URL and MONGO_KEY) to match those used by the Web API:

   export MONGO_URL="<your-mongo-url>"
   export MONGO_KEY="<your-mongo-key>"

4. Migrate your existing rclone.conf to the MongoDB storage by running:

   ./rclone-cloud migrate --from-file <path/to/rclone.conf>

5. Now, you can run any RClone command such as:

   ./rclone-cloud listremotes
   ./rclone-cloud config
   ./rclone-cloud lsd <remote>:
   ./rclone-cloud ls <remote>:
   ./rclone-cloud sync . <remote>:Documents
   ./rclone-cloud copy file.txt <remote>:
   ...

   You can find more available commands at https://rclone.org/commands.

• Note: each time the configs change in MongoDB, the CLI and the Web API automatically picks up the new changes.

Usage

Please note that this project is used for educational purposes and is not intended to be used commercially. We are not liable for any damages/changes done by this project.

Credits

Emilio Kartono, who made the entire project.

License

This project is protected under the GNU licence. Please refer to the LICENSE for more information.