|
| 1 | +# scripted-renderer-dev |
| 2 | + |
| 3 | +This project is a **3D rendering tool** that takes any 2 Blender objects and renders them on a plane with customizable orientations, positions, distances, and camera angles. |
| 4 | + |
| 5 | +## Setup |
| 6 | + |
| 7 | +### Step 1: Clone the Repository |
| 8 | + |
| 9 | +Clone this repository to your local machine: |
| 10 | + |
| 11 | +```bash |
| 12 | +git clone https://github.com/compling-wat/scripted-renderer-dev.git |
| 13 | +cd scripted-renderer-dev |
| 14 | +``` |
| 15 | + |
| 16 | +### Step 2: Install Blender 4.3 |
| 17 | + |
| 18 | +Download and install **[Blender 4.3](https://www.blender.org/download/)**. |
| 19 | + |
| 20 | +### Step 3: Set Up the Environment |
| 21 | + |
| 22 | +1. Locate where Blender 4.3 is installed: |
| 23 | + ```bash |
| 24 | + which/where blender |
| 25 | + ``` |
| 26 | +2. Open a terminal and navigate into the Blender installation directory (path varies by operating system): |
| 27 | + ```bash |
| 28 | + cd /path/to/blender/site-packages/ |
| 29 | + ``` |
| 30 | +3. Create a `.pth` file named `scripted-renderer-dev.pth` and add the absolute path to the `src` directory of the cloned repository: |
| 31 | + ```bash |
| 32 | + echo "/path/to/scripted-renderer-dev/src" > /path/to/blender/site-packages/scripted-renderer-dev.pth |
| 33 | + ``` |
| 34 | + - _Note: if you encounter errors on Windows, ensure the file encoding is **UTF-8 with BOM**._ |
| 35 | + |
| 36 | +## Usage |
| 37 | + |
| 38 | +The repository contains a `scripts` folder with example shell scripts for using this tool. First **cd** into this directory. |
| 39 | + |
| 40 | +### **Camera Settings** |
| 41 | + |
| 42 | +To set the camera configurations for the rendered images, you can add the following parameters to your script: |
| 43 | + ```bash |
| 44 | + --camera-tilt X # Set camera tilt in degrees |
| 45 | + --camera-pan X # Set camera pan in degrees |
| 46 | + --camera-height X # Set camera height in meters |
| 47 | + --camera-focal-length X # Set focal length in millimeters |
| 48 | + ``` |
| 49 | + |
| 50 | +### **Rendering Multiple Images** |
| 51 | + |
| 52 | +To render all possible combinations of 2 objects under the `data` directory, use the `--render-random` parameter as shown in the `render_multiple.sh` script. |
| 53 | +- The generates images into a **main directory** for each combination of 2 objects within into the **specified image output folder**, along with **4 subdirectories** within the **main directory**: |
| 54 | + |
| 55 | + - `[object1]_[object2]_left` (object2 to the left of object1) |
| 56 | + - `[object1]_[object2]_right` (object2 to the right of object1) |
| 57 | + - `[object1]_[object2]_front` (object2 to the front of object1) |
| 58 | + - `[object1]_[object2]_behind` (object2 to the back of object1) |
| 59 | + |
| 60 | +- For each image, it also generates a corresponding `json` file with detailed information on the camera settings, and positions, rotations, orientations, and relative perspectives of the objects in the **specified scene output folder**. |
| 61 | + |
| 62 | +- You can specify the following constraints: |
| 63 | + ```bash |
| 64 | + --objects [list of objects] # Select specific objects to render |
| 65 | + --distance-between-objects X # Set object distance in meters (default: 3) |
| 66 | + --max-images N # Set max images (orientations combinations) per subdirectory (default: 1) |
| 67 | + --max-camera-configs N # Set max number of camera configurations per image (default: 1) |
| 68 | + ``` |
| 69 | + |
| 70 | +- If you do not specify `--objects`, the list of objects will be set to all objects in the `data` folder. |
| 71 | +- If you do not specify `--max-camera-configs`, you must fix a specific camera configuration for all images in the script. |
| 72 | + |
| 73 | +_This script can be used for generating an extensive **spatial reasoning dataset** for **vision-language models**._ |
| 74 | + |
| 75 | +### **Rendering a Single Image** |
| 76 | + |
| 77 | +For controlled rendering of 2 selected objects, navigate to the `render_single.sh` script for an example usage. |
| 78 | + |
| 79 | +- Unlike rendering multiple |
| 80 | + |
| 81 | +- Now, since you are only generating 1 image, you have the following additional customizations: |
| 82 | + ```bash |
| 83 | + --direction [left, right, behind, front] # Set object 2’s position relative to object1 |
| 84 | + --object1-rotation X # Set object 1's rotation to X degrees clockwise (default: 0) |
| 85 | + --object2-rotation X # Set object 2's rotation to X degrees clockwise (default: 0) |
| 86 | + ``` |
| 87 | + |
| 88 | +- In this case, you must specify a camera setting as well as the `--direction` parameter. |
| 89 | + |
| 90 | +_This script is useful for **testing and fine-tuning** object placements before batch rendering._ |
| 91 | + |
| 92 | +### **Adding AI-generated Backgrounds (EXTRA)** |
| 93 | + |
| 94 | +All the images rendered are just 2 objects on a white surface with a light grey "sky". If you would like to add a custom (more realistic) background to each image in your image output folder, you can navigate to the `generate_backgrounds.sh` script. Note: this feature is still under experimentation. |
| 95 | + |
| 96 | +- Here, you can customize the `PROMPT` argument to guide the background generation process. |
| 97 | + |
| 98 | +- The process utilizes the [**Stable Diffusion XL inpainting model**](https://huggingface.co/diffusers/stable-diffusion-xl-1.0-inpainting-0.1) to modify specific regions of images with custom backgrounds. |
| 99 | + |
| 100 | +_The goal is to enable comparisons between a fine-tuned spatial reasoning model’s performance on the original dataset versus a version with more realistic backgrounds._ |
| 101 | + |
| 102 | +## Configuration |
| 103 | + |
| 104 | +The tool's settings, such as **output directories, image resolution, and other scene configurations**, can be customized in `config.json` under the `src` directory. |
| 105 | + |
| 106 | +You can also add your own objects by placing `.blend` files inside the `data` folder and specifying their properties in `properties.json`. |
0 commit comments