diff --git a/docs/doc/assets/maixhub_home.png b/docs/doc/assets/maixhub_home.png new file mode 100644 index 00000000..7524a90f Binary files /dev/null and b/docs/doc/assets/maixhub_home.png differ diff --git a/docs/doc/assets/maixhub_train_annotate_page.png b/docs/doc/assets/maixhub_train_annotate_page.png new file mode 100644 index 00000000..ca917ab4 Binary files /dev/null and b/docs/doc/assets/maixhub_train_annotate_page.png differ diff --git a/docs/doc/assets/maixhub_train_device_qrcode.png b/docs/doc/assets/maixhub_train_device_qrcode.png new file mode 100644 index 00000000..a4bdb2ac Binary files /dev/null and b/docs/doc/assets/maixhub_train_device_qrcode.png differ diff --git a/docs/doc/assets/maixhub_train_manual_deploy_maixvision.png b/docs/doc/assets/maixhub_train_manual_deploy_maixvision.png new file mode 100644 index 00000000..1a716807 Binary files /dev/null and b/docs/doc/assets/maixhub_train_manual_deploy_maixvision.png differ diff --git a/docs/doc/assets/maixhub_train_manual_deploy_package.png b/docs/doc/assets/maixhub_train_manual_deploy_package.png new file mode 100644 index 00000000..83636739 Binary files /dev/null and b/docs/doc/assets/maixhub_train_manual_deploy_package.png differ diff --git a/docs/doc/assets/maixhub_train_manual_deploy_page.png b/docs/doc/assets/maixhub_train_manual_deploy_page.png new file mode 100644 index 00000000..5051dd1a Binary files /dev/null and b/docs/doc/assets/maixhub_train_manual_deploy_page.png differ diff --git a/docs/doc/assets/maixhub_train_manual_deploy_result.png b/docs/doc/assets/maixhub_train_manual_deploy_result.png new file mode 100644 index 00000000..3f045533 Binary files /dev/null and b/docs/doc/assets/maixhub_train_manual_deploy_result.png differ diff --git a/docs/doc/assets/maixhub_train_manual_deploy_upload.png b/docs/doc/assets/maixhub_train_manual_deploy_upload.png new file mode 100644 index 00000000..2552998e Binary files /dev/null and b/docs/doc/assets/maixhub_train_manual_deploy_upload.png differ diff --git a/docs/doc/assets/maixhub_train_model_type.png b/docs/doc/assets/maixhub_train_model_type.png new file mode 100644 index 00000000..5f284905 Binary files /dev/null and b/docs/doc/assets/maixhub_train_model_type.png differ diff --git a/docs/doc/assets/maixhub_train_parameters.png b/docs/doc/assets/maixhub_train_parameters.png new file mode 100644 index 00000000..52575709 Binary files /dev/null and b/docs/doc/assets/maixhub_train_parameters.png differ diff --git a/docs/doc/assets/maixhub_train_result.png b/docs/doc/assets/maixhub_train_result.png new file mode 100644 index 00000000..a0c14444 Binary files /dev/null and b/docs/doc/assets/maixhub_train_result.png differ diff --git a/docs/doc/assets/maixhub_train_split_validation.png b/docs/doc/assets/maixhub_train_split_validation.png new file mode 100644 index 00000000..e1e4b5a1 Binary files /dev/null and b/docs/doc/assets/maixhub_train_split_validation.png differ diff --git a/docs/doc/assets/maixhub_train_validation_failed.png b/docs/doc/assets/maixhub_train_validation_failed.png new file mode 100644 index 00000000..61784aa8 Binary files /dev/null and b/docs/doc/assets/maixhub_train_validation_failed.png differ diff --git a/docs/doc/assets/maixhub_train_video_entries.png b/docs/doc/assets/maixhub_train_video_entries.png new file mode 100644 index 00000000..85d03811 Binary files /dev/null and b/docs/doc/assets/maixhub_train_video_entries.png differ diff --git a/docs/doc/en/README.md b/docs/doc/en/README.md index c32864ed..7fc64d37 100644 --- a/docs/doc/en/README.md +++ b/docs/doc/en/README.md @@ -7,6 +7,41 @@ title: MaixCAM MaixPy Quick Start width: 100%; display: table; } + .device-list { + display: flex; + flex-direction: column; + gap: 0.8em; + margin: 1.2em 0; + } + .device-item { + display: grid; + grid-template-columns: 13em 1fr auto; + gap: 1em; + align-items: center; + border: 1px solid #e6e8ef; + border-radius: 8px; + padding: 0.8em; + background: #fff; + } + .device-item img { + width: 100%; + height: 6.8em; + object-fit: contain; + background: #f7f8fb; + border-radius: 6px; + } + .device-name { + display: block; + font-weight: 700; + font-size: 1.05em; + } + .device-desc { + margin: 0; + } + .device-link { + white-space: nowrap; + font-weight: 600; + } @media screen and (max-width: 900px){ #head_links th, #head_links td { @@ -14,6 +49,15 @@ title: MaixCAM MaixPy Quick Start font-size: 0.9em; padding: 0.1em 0.05em; } + .device-item { + grid-template-columns: 1fr; + } + .device-item img { + height: 8em; + } + .device-link { + white-space: normal; + } } @@ -42,15 +86,29 @@ title: MaixCAM MaixPy Quick Start > For an introduction to MaixPy, please see the [MaixPy official website homepage](../../README.md) > Please give the [MaixPy project](https://github.com/sipeed/MaixPy) a Star ⭐️ to encourage us to develop more features if you like MaixPy. - +## Choose Your Device First -## Quick Preview +There are several MaixCAM series products. If this is your first time using one, check the model name on the package, order page, or device enclosure first, then open the corresponding getting-started guide. Choosing the right guide first can save a lot of detours. -
MaixCAM2(The Upgraded Version of MaixCAM~)
- +
+
+ MaixCAM2 +

MaixCAM2

+ Quick Start MaixCAM2 +
+
+ MaixCAM and MaixCAM-Pro +

MaixCAM / MaixCAM-Pro

+ Quick Start MaixCAM +
+
+ MaixCAM Lite +

MaixCAM Lite / screenless version

+ Quick Start MaixCAM screenless version +
+
-
MaixCAM
- + ## Before Start @@ -82,17 +140,6 @@ Purchase Links:[Sipeed Taobao](https://item.taobao.com/item.htm?id=846226367137) >! Note that currently only the MaixCAM development board is supported. Other development boards with the same chip are not supported, including Sipeed's development boards with the same chip. Please be careful not to purchase the wrong board, which could result in unnecessary waste of time and money. - -## Getting Started - -Please select the documentation corresponding to your hardware platform to proceed: - -| Hardware Platform | Getting Started Guide | -|-|-| -|MaixCAM Lite|[Quick Start MaixCAM(Screenless Version)](./README_no_screen.md)| -|MaixCAM/MaixCAM Pro|[Quick Start MaixCAM](./README_MaixCAM.md)| -|MaixCAM2|[Quick Start MaixCAM2](./README_MaixCAM2.md)| - ## Next Steps If you like what you've seen so far, **please be sure to give the MaixPy open-source project a star on [GitHub](https://github.com/sipeed/MaixPy) (you need to log in to GitHub first). Your star and recognition is the motivation for us to continue maintaining and adding new features!** diff --git a/docs/doc/en/README_MaixCAM.md b/docs/doc/en/README_MaixCAM.md index d25aaa0c..f9e09867 100644 --- a/docs/doc/en/README_MaixCAM.md +++ b/docs/doc/en/README_MaixCAM.md @@ -7,6 +7,12 @@ title: MaixCAM MaixPy Quick Start width: 100%; display: table; } + .biliiframe { + width: 100%; + min-height: 30em; + border-radius: 0.5em; + border: 1em solid white; + } @media screen and (max-width: 900px){ #head_links th, #head_links td { @@ -19,6 +25,10 @@ title: MaixCAM MaixPy Quick Start ## Getting Started +### Getting Started Video + + + ### Prepare the TF Image Card and Insert it into the Device If the package you purchased includes a TF card, it already contains the factory image. If the TF card was not installed in the device at the factory, you will first need to carefully open the case (be careful not to tear the ribbon cables inside) and then insert the TF card. Additionally, since the firmware from the factory may be outdated, it is highly recommended to follow the instructions on [Upgrading and Flashing the System](./basic/os.md) to upgrade the system to the latest version. @@ -158,4 +168,4 @@ Here is the translation: If you want the program to start automatically on boot, you can set it in `Settings -> Boot Startup`. - More MaixVision usage refer to [MaixVision documentation](./basic/maixvision.md)。 \ No newline at end of file + More MaixVision usage refer to [MaixVision documentation](./basic/maixvision.md)。 diff --git a/docs/doc/en/README_MaixCAM2.md b/docs/doc/en/README_MaixCAM2.md index 0295eae2..ec0314fa 100644 --- a/docs/doc/en/README_MaixCAM2.md +++ b/docs/doc/en/README_MaixCAM2.md @@ -7,6 +7,12 @@ title: MaixCAM2 MaixPy Quick Start width: 100%; display: table; } + .biliiframe { + width: 100%; + min-height: 30em; + border-radius: 0.5em; + border: 1em solid white; + } @media screen and (max-width: 900px){ #head_links th, #head_links td { @@ -19,7 +25,11 @@ title: MaixCAM2 MaixPy Quick Start ## Getting Started ->! MaixCAM2 has built-in eMMC storage, so it can operate without requiring a TF card. If you need to upgrade or flash the system, please refer directly to the System Upgrade and Flashing guide. +>! MaixCAM2 is available in versions with and without eMMC storage. The 32GB eMMC version normally boots from eMMC and does not require a TF card for daily use; the version without eMMC must use a TF card with a flashed system to boot. A TF card can also be used to boot from TF card, or to flash or recover the system to eMMC. For system upgrade or flashing, see System Upgrade and Flashing. + +### Getting Started Video + + ### Power On @@ -154,4 +164,4 @@ Here is the translation: If you want the program to start automatically on boot, you can set it in `Settings -> Boot Startup`. - More MaixVision usage refer to [MaixVision documentation](./basic/maixvision.md)。 \ No newline at end of file + More MaixVision usage refer to [MaixVision documentation](./basic/maixvision.md)。 diff --git a/docs/doc/en/ai_model_converter/ai_model_deploy.md b/docs/doc/en/ai_model_converter/ai_model_deploy.md new file mode 100644 index 00000000..2479920e --- /dev/null +++ b/docs/doc/en/ai_model_converter/ai_model_deploy.md @@ -0,0 +1,15 @@ +--- +title: AI Model Download, Debugging, and Deployment Guide +--- + +## Choose a Model Deployment Workflow + +Before deploying a local model on MaixCAM2, first identify the model source and deployment path. Choose the workflow that matches your current resources instead of starting with ONNX conversion immediately. + +| Goal | Recommended workflow | Documentation | +| --- | --- | --- | +| Use built-in or ready-made models | Use built-in models first. For more resolutions or class sets, filter MaixCAM2 models in [MaixHub Model Zoo](https://maixhub.com/model/zoo?platform=maixcam2). Model packages usually include a `.mud` file and corresponding `.axmodel` files, which should be placed in the same directory on the device | [Model and dataset sources](../pro/datasets.md) | +| Train a custom recognition target | Use MaixHub online training to complete data collection, annotation, training, and deployment | [MaixHub online training](../vision/maixhub_train.md) | +| Deploy an ONNX model | Follow the MaixCAM2 conversion workflow to convert ONNX into `.mud` + `.axmodel` files before deployment | [MaixCAM2 model conversion](./maixcam2.md) | + +After choosing your workflow, continue with the corresponding document. diff --git a/docs/doc/en/ai_model_converter/maixcam2.md b/docs/doc/en/ai_model_converter/maixcam2.md index 81263de7..66cb7f6b 100644 --- a/docs/doc/en/ai_model_converter/maixcam2.md +++ b/docs/doc/en/ai_model_converter/maixcam2.md @@ -10,6 +10,8 @@ Models trained on a computer cannot be directly used on MaixCAM2 due to its limi This article describes how to convert an ONNX model into a model usable by MaixCAM2 (MUD model). +If you are not sure whether to download an existing model, train online, or convert ONNX yourself, read the [beginner model download, debugging, and deployment guide](./ai_model_deploy.md) first. + ## Supported Model File Format for MaixCAM2 MUD (Model Universal Description) is a model description file supported by MaixPy that unifies models across different platforms, making MaixPy code cross-platform. It is a text file in `ini` format and can be edited with a text editor. @@ -178,10 +180,50 @@ Edit as per your model. For YOLO11: The `[basic]` section is required. Once set, you can load and run the model using `maix.nn.NN` in `MaixPy` or `MaixCDK`. +## Deploy to the Device and Verify Quickly + +MaixPy usually loads the `.mud` file. The `.mud` file then points to the actual `.axmodel` files. The simplest approach is to put them in the same directory, for example: + +```text +/root/models/my_model.mud +/root/models/my_model_npu.axmodel +/root/models/my_model_vnpu.axmodel +``` + +Then first confirm that the model can be loaded: + +```python +from maix import nn + +model = nn.NN("/root/models/my_model.mud") +print(model) +``` + +If the model type is already supported by MaixPy, prefer the wrapped API. For example, for YOLO, see the [YOLO object detection documentation](../vision/yolov5.md): + +```python +from maix import nn + +detector = nn.YOLO11(model="/root/models/yolo11n.mud", dual_buff=True) +``` + +## Debug Checklist + +If the model cannot be loaded or the result is wrong, check these items first: + +1. Whether the `.mud` and `.axmodel` files are in the same directory, and whether `model_npu` and `model_vnpu` in the `.mud` file use the correct file names. +2. Whether the path used on the device really exists, such as `/root/models/xxx.mud`. +3. Whether `labels` exactly matches the class count and class order used during training. +4. Whether `model_type` is supported by MaixPy, such as `yolo11`, `yolov8`, or `classifier`. +5. Whether input resolution, `mean`, `scale`, and RGB/BGR order match training, export, and conversion settings. +6. Whether the ONNX output nodes exactly match `output_processors` in `config.json`. +7. If you are not sure whether the problem is the model or your code, test with a MaixHub model or a built-in model first. After an official model runs correctly, debug your own model. + +After these basic checks pass, continue with the specific model documentation or conversion workflow. + If MaixPy doesn't support your model, define your own `extra` fields and write decoding logic. You can either: * Use Python in `MaixPy` to load the model via `maix.nn.NN`, run `forward`/`forward_image`, and process the outputs in Python (easier but slower); * Or, for better performance and reusable integration, write C++ logic in `MaixCDK`, see [YOLOv5 example](https://github.com/sipeed/MaixCDK/blob/71d5b3980788e6b35514434bd84cd6eeee80d085/components/nn/include/maix_nn_yolov5.hpp). Once done, consider submitting a PR to `MaixPy` or share your model on [MaixHub](https://maixhub.com/share) to earn rewards ranging from ¥30 to ¥2000! - diff --git a/docs/doc/en/basic/upgrade.md b/docs/doc/en/basic/upgrade.md index 1714c26c..06444737 100644 --- a/docs/doc/en/basic/upgrade.md +++ b/docs/doc/en/basic/upgrade.md @@ -8,20 +8,24 @@ First, let's distinguish between **`System`** and **`MaixPy`**: * **System**: The foundation for running all software, including the operating system and drivers, serving as the cornerstone for software operation. * **MaixPy**: A software package that relies on system drivers to function. -## Getting the Latest System +## Get the Latest System and Flash It to Hardware -Find the latest system image files on the [MaixPy Releases page](https://github.com/sipeed/MaixPy/releases), for example: +Before flashing, confirm both the device model and where the system is stored. MaixCAM / MaixCAM-Pro run the system from a TF card, so a TF card is required. MaixCAM2 normally runs from onboard eMMC, so a TF card is usually not required for regular updates. For regular updates, use the USB flashing workflow in the corresponding flashing page first; when booting from a TF card, recovering the system, or replacing storage media, follow the method described on that page. -* `maixcam_os_20240401_maixpy_v4.1.0.xz`: MaixCAM system image including MaixPy v4.1.0. -* `maixcam-pro_os_20240401_maixpy_v4.1.0.xz`: MaixCAM Pro system image including MaixPy v4.1.0. -* `maixcam2_os_20250801_maixpy_v4.11.0.xz`: MaixCAM2 system image including MaixPy v4.11.0. Since the MaixCAM system image exceeds 2GB, it will only be available on Sourceforge. Users in China can download it via the shared files in the QQ group. +After confirmation, use the table below to choose the correct system image and flashing page. -**Make sure to download the system image that corresponds to your device model**. Downloading the wrong image may cause device damage. +| Device | Download entry | File to choose | Flashing page | +| --- | --- | --- | --- | +| MaixCAM | [MaixPy Releases page](https://github.com/sipeed/MaixPy/releases) | `maixcam-*.img.xz` | [MaixCAM System Flashing](https://wiki.sipeed.com/hardware/zh/maixcam/os.html) | +| MaixCAM-Pro | [MaixPy Releases page](https://github.com/sipeed/MaixPy/releases) | `maixcam-pro-*.img.xz` | [MaixCAM System Flashing](https://wiki.sipeed.com/hardware/zh/maixcam/os.html) | +| MaixCAM2 | [Baidu Netdisk (extraction code: vjex)](https://pan.baidu.com/s/1r4ECNlaTVxhWIafNBZOztg) | `maixcam2-*-maixpy-*_sd.img.7z.*` | [MaixCAM2 System Flashing](https://wiki.sipeed.com/hardware/zh/maixcam/maixcam2_os.html) | -> Users in China with slow download speeds can use tools like Xunlei for faster downloads. -> Alternatively, use proxy sites such as [github.abskoop.workers.dev](https://github.abskoop.workers.dev/) for downloads. +**Make sure to download the system image that corresponds to your device model**. Downloading the wrong image may cause abnormal behavior and may require recovery flashing. + +When downloading, choose the newest file that matches the "File to choose" column in the table. For MaixCAM and MaixCAM-Pro, select the corresponding image on the releases page. MaixCAM2 images are split 7z archives; download all `.7z.00x` parts of the same version from Baidu Netdisk before extracting and flashing. -Backup mirror: [Sourceforge](https://sourceforge.net/projects/maixpy/files/) (may not be up-to-date, so prefer the above official sources) + +> Users in China with slow download speeds can use tools like Xunlei for faster downloads. ## Backup Your Data @@ -37,18 +41,6 @@ Backup methods: * Remove the storage media and use a card reader to copy files directly. Note: the root filesystem is formatted as `ext4`, which Windows does not support by default (you can use third-party software like DiskGenius to read it). -## Flashing the System to Hardware - -| Item | MaixCAM / MaixCAM-Pro | MaixCAM2 | -| --------------- | ---------- | ------ | -| Flashing Docs | [MaixCAM System Flashing](https://wiki.sipeed.com/hardware/zh/maixcam/os.html) | [MaixCAM2 System Flashing](https://wiki.sipeed.com/hardware/zh/maixcam/maixcam2_os.html) | -| System Storage | TF Card | Built-in EMMC (/TF Card) | -| TF Card Required | Yes | No | -| Flashing Method | USB flashing or card reader flashing | USB flashing or card reader flashing | -| Recommended Method | USB flashing | USB flashing | -| Recovery Flashing | Card reader flashing | USB flashing / card reader flashing | - - ## When to Update the System vs. Updating MaixPy Only To simplify the process and avoid issues, it is **recommended to update the system whenever upgrading MaixPy**. diff --git a/docs/doc/en/pro/datasets.md b/docs/doc/en/pro/datasets.md index 8e58ed23..7388ea35 100644 --- a/docs/doc/en/pro/datasets.md +++ b/docs/doc/en/pro/datasets.md @@ -8,6 +8,14 @@ title: MaixCAM MaixP Where to Find Models and Datasets Visit the [MaixHub Model Library](https://maixhub.com/model/zoo) and filter by the corresponding hardware platform to find suitable models. +If you only want to verify that the workflow works, start with a built-in model. The system usually includes common models in `/root/models`, so you can reference the corresponding `.mud` file directly in your code without downloading anything. + +For more models, open [MaixHub Model Zoo](https://maixhub.com/model/zoo?platform=maixcam2), filter by the MaixCAM2 platform, and download a model package. After downloading, check whether it contains: + +- `.mud`: the model description file. MaixPy code usually loads this file. +- `.axmodel`: the actual model file that runs on MaixCAM2. +- Example code or instructions: if the model page provides an example, run that first. + ## What Are Datasets Used For? First, check the [MaixHub Model Library](https://maixhub.com/model/zoo) to see if there’s a model you need. If not, you can train your own model. Training a model requires a dataset, which provides the data needed for training. diff --git a/docs/doc/en/sidebar.yaml b/docs/doc/en/sidebar.yaml index 2d9d4439..e10cd422 100644 --- a/docs/doc/en/sidebar.yaml +++ b/docs/doc/en/sidebar.yaml @@ -137,24 +137,26 @@ items: label: Whisper Speech-Recognition Model - file: mllm/tts_melotts.md label: MeloTTS Speech-Synthesis Model -- label: Local NPU deployment of AI models +- label: Local deployment of AI models items: + - file: ai_model_converter/ai_model_deploy.md + label: AI model download and deployment guide + - file: pro/datasets.md + label: Where to find models & datasets + - file: vision/maixhub_train.md + label: MaixHub online AI training + - file: vision/customize_model_yolo.md + label: YOLO model offline training - file: ai_model_converter/maixcam2.md - label: ONNX model to MaixCAM2's + label: Convert ONNX model for MaixCAM2 - file: ai_model_converter/maixcam.md - label: ONNX model to MaixCAM's + label: Convert ONNX model for MaixCAM - file: ai_model_converter/onnx_export.md label: Trim ONNX model output nodes - file: ai_model_converter/web_converter.md label: Convert YOLO models with Web UI - - file: pro/datasets.md - label: Where to find models & datasets - file: pro/customize_model.md - label: Customize new AI model - - file: vision/maixhub_train.md - label: MaixHub online AI training - - file: vision/customize_model_yolo.md - label: YOLO model offline training + label: Port a new AI model diff --git a/docs/doc/en/vision/maixhub_train.md b/docs/doc/en/vision/maixhub_train.md index 741d8542..ddd8d068 100644 --- a/docs/doc/en/vision/maixhub_train.md +++ b/docs/doc/en/vision/maixhub_train.md @@ -1,52 +1,136 @@ --- -title: Using MaixHub to Train AI Models for MaixCAM MaixPy -update: - - date: 2024-04-03 - author: neucrack - version: 1.0.0 - content: Initial document +title: Train AI Models Online with MaixHub --- ## Introduction -MaixHub offers the functionality to train AI models online, directly within a browser. This eliminates the need for expensive hardware, complex development environments, or coding skills, making it highly suitable for beginners as well as experts who prefer not to delve into code. +[MaixHub](https://maixhub.com/) provides online AI model training. You can collect data, upload images, annotate images, train models, and deploy models in a browser without setting up a local training environment or configuring a GPU. -## Basic Steps to Train a Model Using MaixHub +This page follows the official example video workflow and uses an **image detection model** as the example. If you only want to try AI features quickly, first check the [MaixHub Model Zoo](https://maixhub.com/model/zoo) to see whether a ready-to-use model already exists. Use online training when you need to recognize custom targets. -### Identify the Data and Model Types +![](../../assets/maixhub_home.png) -To train an AI model, you first need to determine the type of data and model. As of April 2024, MaixHub provides models for image data including `Object Classification Models` and `Object Detection Models`. Object classification models are simpler than object detection models, as the latter require marking the position of objects within images, which can be more cumbersome. Object classification merely requires identifying what is in the image without needing coordinates, making it simpler and recommended for beginners. +> The screenshots below are for workflow guidance. Account, project, dataset, image file names, QR codes, and training task IDs have been blurred. -### Collect Data +## Official Video Demos -As discussed in AI basics, training a model requires a dataset for the AI to learn from. For image training, you need to create a dataset and upload images to it. +MaixHub provides two official videos on the home page. Watch **Quick Start** first to understand the overall online training workflow. When you need to follow the page step by step, watch **Tutorial** as a detailed walkthrough. After logging in to [MaixHub](https://maixhub.com/), click **Play Video** in the video area at the top of the home page to watch. -Ensure the device is connected to the internet (WiFi). -Open the MaixHub app on your device and choose to collect data to take photos and upload them directly to MaixHub. You need to create a dataset on MaixHub first, then click on device upload data, which will display a QR code. Scan this QR code with your device to connect to MaixHub. +![MaixHub official video entries](../../assets/maixhub_train_video_entries.png) -It's important to distinguish between training and validation datasets. To ensure the performance during actual operation matches the training results, the validation dataset must be of the same image quality as those taken during actual operation. It's also advisable to use images taken by the device for the training set. If using internet images, restrict them to the training set only, as the closer the dataset is to actual operational conditions, the better. +## MaixHub Model Training Workflow + +This section keeps the complete tutorial in one workflow category and follows the actual operation order: create a project, prepare data, annotate images, train the model, and deploy it. + +| Step | Operation | +| --- | --- | +| Create a project | Select the model type and target hardware platform | +| Prepare a dataset | Upload training and validation images from the device or local files | +| Annotate images | Detection models require boxes for each target | +| Create a training task | Select the model, augmentation, and training parameters | +| Check training results | Review curves and validation examples | +| Deploy to device | Download the model package and upload it to the device with MaixVision | + +### Create a Training Project + +From the MaixHub home page, enter **Model Training** and create a new training project. Select the model type and hardware platform during project creation: + +![MaixHub training project type selection](../../assets/maixhub_train_model_type.png) + +This guide uses an **image detection model** in the following steps. It is suitable when the model needs to locate target positions in an image. + +After the project is created, follow the left navigation to complete dataset preparation, annotation, training, and deployment. + +### Prepare the Dataset + +Create a dataset after entering the project. The data type and annotation type must match the project. For example, a detection project should use image data and detection annotation. + +It is recommended to collect images with the MaixHub app on the device. Device-captured images are closer to the real camera angle, resolution, and lighting used during deployment, so the trained model is more likely to run reliably on the device. + +On the web page, open **Collect Data**, choose whether the images should go to the training set or validation set, and generate a QR code: + +![MaixHub device collection QR code](../../assets/maixhub_train_device_qrcode.png) + +Basic workflow: + +1. Make sure the device is connected to WiFi. +2. Create and select a dataset on the web page. +3. Choose the training set or validation set, then generate the QR code. +4. Open the MaixHub app on the device, scan the QR code, and upload captured images. + +The training set is used for learning target features, while the validation set is used for evaluating training quality. For detection models, keep at least 5 validation images for each label, otherwise training may not start. The validation set should not duplicate the training set and should use real-scene images whenever possible. + +In the dataset page, select images in batches and move them to the training set or validation set as needed: + +![MaixHub organize training and validation sets](../../assets/maixhub_train_split_validation.png) ### Annotate Data -For classification models, images are annotated during upload by selecting the appropriate category for each image. +Image classification only requires selecting a category for each image. Image detection requires drawing boxes for targets and assigning labels to those boxes. + +Open **Annotate Data**, create annotations, draw target boxes, and save the result: + +![MaixHub annotation page](../../assets/maixhub_train_annotate_page.png) + +Annotation tips: + +* Keep each box close to the target object and avoid including too much background. +* Use consistent annotation rules for the same target class. +* Do not miss targets that should be detected. +* Leave blurry, heavily occluded, or uncertain images out of the training set first. + +For larger datasets, complete one full training and deployment cycle with a small dataset first, then gradually add more data to improve results. + +### Create a Training Task + +After checking the dataset and annotations, open **Create Task**. The page mainly contains image augmentation, model selection, and training parameter panels: + +![MaixHub training parameter settings](../../assets/maixhub_train_parameters.png) + +Suggestions for beginners: + +* Select the actual deployment device, such as MaixCAM or MaixCAM2. +* Use the default recommended model network first. +* Enable image augmentation as needed, but do not use it as a replacement for real-scene data. +* Enable data balancing when the number of images differs greatly between classes. +* Add negative samples when background false positives need to be reduced. + +After confirming the model information and parameters, click **Create Training Task**, enter a task name, and start training. + +### Check Training Results + +After training starts, open **Training Records** to view progress, logs, dataset statistics, and training parameters. When training is complete, the result page shows loss curves, accuracy curves, and validation examples: + +![MaixHub training result](../../assets/maixhub_train_result.png) + +Check the following first: + +* Whether the curves are stable and free of obvious anomalies. +* Whether the validation examples are recognized correctly. +* Whether wrong results are concentrated under certain angles, lighting conditions, or backgrounds. + +If training fails, check the log on the right first. In the example below, the failure is caused by fewer than 5 validation images for one label. Return to the dataset page, add enough validation images, and create a new training task. + +![MaixHub training failed because validation images are insufficient](../../assets/maixhub_train_validation_failed.png) + +### Deploy to MaixCAM / MaixCAM2 + +After training is complete and the validation result is acceptable, open the project deployment page and select the training record to deploy. Choose **Manual Deployment**, then click **Download Model** to download the model package. -For object detection models, after uploading, you need to manually annotate each image by marking the coordinates, size, and category of the objects to be recognized. -This annotation process can also be done offline on your own computer using software like labelimg, then imported into MaixHub using the dataset import feature. -Utilize shortcuts during annotation to speed up the process. MaixHub will also add more annotation aids and automatic annotation tools in the future (there is already an automatic annotation tool available for videos that you can try). +MaixHub manual deployment page -### Train the Model +Unzip the downloaded package first. For a MaixCAM2 image detection model, the package usually contains files such as `model_xxx.mud`, `model_xxx_npu.axmodel`, `model_xxx_vnpu.axmodel`, `main.py`, `app.yaml`, and `report.json`. The `.mud` and `.axmodel` files are the model files that need to be uploaded during manual deployment. `main.py` and `app.yaml` can be used as references for the example program or app configuration. -Select training parameters, choose the corresponding device platform, select maixcam, and wait in the training queue. You can monitor the training progress in real-time and wait for it to complete. +MaixHub model package contents -### Deploy the Model +Open MaixVision and connect to the device. Open **Device File Manager** on the left, then enter `/root/models` on the device. This directory is used to store model files. -Once training is complete, you can use the deploy function in the MaixHub app on your device to scan a code and deploy. -The device will automatically download and run the model, storing it locally for future use. +MaixVision device file manager -If you find the recognition results satisfactory, you can share the model to the model library with a single click for others to use. +Click **Upload File**, select `model_xxx.mud`, `model_xxx_npu.axmodel`, and `model_xxx_vnpu.axmodel` from the unzipped folder, and upload them to `/root/models` on the device. -## How to Use +Upload MaixHub model files -Please visit [MaixHub](https://maixhub.com) to register an account, then log in. There are video tutorials on the homepage for learning. +After uploading, run the example program on the device or load the corresponding `.mud` file from your project code. Test the model in the real scene and check whether the detection box position, confidence, and false detections meet expectations. -Note that if the tutorial uses the M2dock development board, the process is similar for MaixCAM, although the MaixHub application on the device might differ slightly. The overall process is the same, so please apply the knowledge flexibly. +MaixHub model running on device diff --git a/docs/doc/zh/README.md b/docs/doc/zh/README.md index 6f285e89..914ec993 100644 --- a/docs/doc/zh/README.md +++ b/docs/doc/zh/README.md @@ -14,6 +14,41 @@ title: MaixCAM MaixPy 快速开始 border-radius: 0.5em; border: 1em solid white; } + .device-list { + display: flex; + flex-direction: column; + gap: 0.8em; + margin: 1.2em 0; + } + .device-item { + display: grid; + grid-template-columns: 13em 1fr auto; + gap: 1em; + align-items: center; + border: 1px solid #e6e8ef; + border-radius: 8px; + padding: 0.8em; + background: #fff; + } + .device-item img { + width: 100%; + height: 6.8em; + object-fit: contain; + background: #f7f8fb; + border-radius: 6px; + } + .device-name { + display: block; + font-weight: 700; + font-size: 1.05em; + } + .device-desc { + margin: 0; + } + .device-link { + white-space: nowrap; + font-weight: 600; + } @media screen and (max-width: 900px){ #head_links th, #head_links td { @@ -21,6 +56,15 @@ title: MaixCAM MaixPy 快速开始 font-size: 0.9em; padding: 0.1em 0.05em; } + .device-item { + grid-template-columns: 1fr; + } + .device-item img { + height: 8em; + } + .device-link { + white-space: normal; + } } @@ -50,12 +94,27 @@ title: MaixCAM MaixPy 快速开始 > 关于 MaixPy 介绍请看 [MaixPy 官网首页](../../README.md) > 喜欢 MaixPy 请给 [ MaixPy 项目](https://github.com/sipeed/MaixPy) 点个 Star ⭐️ 以鼓励我们开发更多功能。 -## 快速预览 -
MaixCAM2(MaixCAM的升级版~)
- +## 先选择你的设备 -
MaixCAM
- +MaixCAM 系列型号较多,如果你是第一次使用,先看包装、订单名称或者设备外壳上的型号,再进入对应的上手文档。找对型号后再继续往下看视频和教程,可以少走很多弯路。 + +
+
+ MaixCAM2 +

MaixCAM2

+ 快速开始 MaixCAM2 +
+
+ MaixCAM and MaixCAM-Pro +

MaixCAM / MaixCAM-Pro

+ 快速开始 MaixCAM +
+
+ MaixCAM Lite +

MaixCAM Lite / 无屏幕版本

+ 快速开始 MaixCAM 无屏幕版本 +
+
## 写在前面 @@ -92,16 +151,6 @@ title: MaixCAM MaixPy 快速开始 >! 注意,目前只支持 MaixCAM 系列开发板,其它同型号芯片的开发板均不支持,包括 Sipeed 的同型号芯片开发板,请注意不要买错造成不必要的时间和金钱浪费。 -## 开始上手 - -请选择对应平台的文档进行操作: - -|硬件平台|上手文档| -|-|-| -|MaixCAM Lite|[快速开始MaixCAM(无屏幕版本)](./README_no_screen.md)| -|MaixCAM/MaixCAM Pro|[快速开始MaixCAM](./README_MaixCAM.md)| -|MaixCAM2|[快速开始MaixCAM2](./README_MaixCAM2.md)| - ## 下一步 看到这里,如果你觉得不错,**请务必来 [github](https://github.com/sipeed/MaixPy) 给 MaixPy 开源项目点一个 star(需要先登录 github), 你的 star 和认同是我们不断维护和添加新功能的动力!** diff --git a/docs/doc/zh/README_MaixCAM.md b/docs/doc/zh/README_MaixCAM.md index 712184f9..b2b64af7 100644 --- a/docs/doc/zh/README_MaixCAM.md +++ b/docs/doc/zh/README_MaixCAM.md @@ -26,6 +26,10 @@ title: MaixCAM MaixPy 快速开始 ## 上手配置 +### 视频演示 + + + ### 准备 TF 镜像卡和插入到设备 >! MaixCAM2内置emmc, 因此不需要插入TF卡. 如果需要升级和烧录系统, 请直接看升级和烧录系统 @@ -162,4 +166,4 @@ while not app.need_exit(): # 一直循环,直到程序退出(可以 如果想让程序开机自启动,可以在 `设置 -> 开机启动` 中设置。 -更多 MaixVision 使用请看 [MaixVision 文档](./basic/maixvision.md)。 \ No newline at end of file +更多 MaixVision 使用请看 [MaixVision 文档](./basic/maixvision.md)。 diff --git a/docs/doc/zh/README_MaixCAM2.md b/docs/doc/zh/README_MaixCAM2.md index 96c6b810..944bd319 100644 --- a/docs/doc/zh/README_MaixCAM2.md +++ b/docs/doc/zh/README_MaixCAM2.md @@ -26,7 +26,11 @@ title: MaixCAM2 MaixPy 快速开始 ## 上手配置 ->! MaixCAM2内置emmc, 因此不需要插入TF卡就能运行. 如果需要升级和烧录系统, 请直接看升级和烧录系统 +>! MaixCAM2 有带 eMMC 和不带 eMMC 的版本。带 32GB eMMC 的版本正常情况下从 eMMC 启动,日常运行不需要插入 TF 卡;不带 eMMC 的版本需要插入已烧录系统的 TF 卡才能启动。TF 卡也可用于从 TF 卡启动系统,或用于向 eMMC 烧录、恢复系统。如需升级或烧录系统,请查看升级和烧录系统。 + +### 视频演示 + + ### 上电开机 @@ -156,4 +160,4 @@ while not app.need_exit(): # 一直循环,直到程序退出(可以 如果想让程序开机自启动,可以在 `设置 -> 开机启动` 中设置。 -更多 MaixVision 使用请看 [MaixVision 文档](./basic/maixvision.md)。 \ No newline at end of file +更多 MaixVision 使用请看 [MaixVision 文档](./basic/maixvision.md)。 diff --git a/docs/doc/zh/ai_model_converter/ai_model_deploy.md b/docs/doc/zh/ai_model_converter/ai_model_deploy.md new file mode 100644 index 00000000..6fb5173d --- /dev/null +++ b/docs/doc/zh/ai_model_converter/ai_model_deploy.md @@ -0,0 +1,15 @@ +--- +title: AI 模型下载、调试和部署指南 +--- + +## 选择模型部署方式 + +首次进行 MaixCAM2 本地模型部署时,建议先明确模型来源与部署方式。请根据当前已有资源选择对应流程,避免一开始就直接进行 ONNX 转换。 + +| 使用目标 | 建议流程 | 查看文档 | +| --- | --- | --- | +| 使用预置或现成模型 | 优先使用系统预置模型;如需更多分辨率或类别,可在 [MaixHub 模型库](https://maixhub.com/model/zoo?platform=maixcam2) 筛选 MaixCAM2 平台模型。模型文件通常包含 `.mud` 与对应 `.axmodel`,部署时放入设备同一目录 | [模型与数据集来源](../pro/datasets.md) | +| 训练自定义识别目标 | 使用 MaixHub 在线训练完成数据采集、标注、训练与部署 | [MaixHub 在线训练](../vision/maixhub_train.md) | +| 部署 ONNX 模型 | 按 MaixCAM2 模型转换流程,将 ONNX 转换为 MaixCAM2 可用的 `.mud` + `.axmodel` 后再部署 | [MaixCAM2 模型转换](./maixcam2.md) | + +选定流程后,进入对应文档继续操作即可。 diff --git a/docs/doc/zh/ai_model_converter/maixcam2.md b/docs/doc/zh/ai_model_converter/maixcam2.md index a4670b95..948190cd 100644 --- a/docs/doc/zh/ai_model_converter/maixcam2.md +++ b/docs/doc/zh/ai_model_converter/maixcam2.md @@ -10,6 +10,8 @@ title: 将 ONNX 模型转换为 MaixCAM2 MaixPy 可以使用的模型(MUD) 本文介绍如何将 ONNX 模型转换为 MaixCAM2 能使用的模型(MUD模型)。 +如果你还不确定应该下载现成模型、在线训练,还是自己转换 ONNX,请先看[新手模型下载、调试和部署指南](./ai_model_deploy.md)。 + ## MaixCAM2 支持的模型文件格式 @@ -281,6 +283,47 @@ scale = 0.00392156862745098, 0.00392156862745098, 0.00392156862745098 实际用这个模型的时候将三个文件放在同一个目录下即可。 +## 部署到设备和快速验证 + +MaixPy 加载模型时通常只需要指定 `.mud` 文件路径,`.mud` 中会再指向实际的 `.axmodel` 文件。最省心的做法是把它们放在同一个目录,例如: + +```text +/root/models/my_model.mud +/root/models/my_model_npu.axmodel +/root/models/my_model_vnpu.axmodel +``` + +然后在代码中先确认模型能被加载: + +```python +from maix import nn + +model = nn.NN("/root/models/my_model.mud") +print(model) +``` + +如果是 MaixPy 已经支持的模型类型,优先使用对应封装好的 API,例如 YOLO 可以参考 [YOLO 目标检测文档](../vision/yolov5.md): + +```python +from maix import nn + +detector = nn.YOLO11(model="/root/models/yolo11n.mud", dual_buff=True) +``` + +## 调试时先检查这些问题 + +模型无法加载或运行结果不对时,先按这个顺序排查: + +1. `.mud` 和 `.axmodel` 是否在同一个目录,`.mud` 里面的 `model_npu`、`model_vnpu` 文件名是否写对。 +2. 设备上填写的路径是否真实存在,比如 `/root/models/xxx.mud`。 +3. `labels` 是否和训练时的类别数量、顺序完全一致。 +4. `model_type` 是否是 MaixPy 已经支持的类型,比如 `yolo11`、`yolov8`、`classifier` 等。 +5. 输入分辨率、`mean`、`scale`、RGB/BGR 顺序是否和训练、导出、转换时保持一致。 +6. ONNX 输出节点和 `config.json` 里的 `output_processors` 是否完全一致。 +7. 如果不确定是模型问题还是代码问题,先换 MaixHub 或系统内置模型测试;官方模型能跑通后,再调试自己的模型。 + +确认这些基础项没有问题后,再继续看具体模型文档或转换流程。 + 根据你的模型情况修改,比如对于 YOLO11,修改成你训练的`axmodel`名字和`labels`就好了。 这里`basic`部分指定了模型文件类别和模型文件路径,是必要的参数,有了这个参数就能用`MaixPy`或者`MaixCDK`中的`maix.nn.NN`类来加载并运行模型了。 @@ -299,4 +342,3 @@ scale = 0.00392156862745098, 0.00392156862745098, 0.00392156862745098 - diff --git a/docs/doc/zh/basic/upgrade.md b/docs/doc/zh/basic/upgrade.md index faff2ff9..21e88af8 100644 --- a/docs/doc/zh/basic/upgrade.md +++ b/docs/doc/zh/basic/upgrade.md @@ -8,18 +8,24 @@ title: MaixCAM MaixPy 升级和烧录系统 * **系统**: 运行所有软件的基础,包含了操作系统和驱动等,所有软件运行的基石。 * **MaixPy**: 软件包,依赖系统的驱动运行。 -## 获得最新系统 +## 获得最新系统并烧录到硬件 -在 [MaixPy 发布页面](https://github.com/sipeed/MaixPy/releases) 找到最新的系统镜像文件,比如: -* `maixcam_os_20240401_maixpy_v4.1.0.xz`: MaixCAM 系统镜像,包含了 MaixPy v4.1.0。 -* `maixcam-pro_os_20240401_maixpy_v4.1.0.xz`: MaixCAM Pro 系统镜像,包含了 MaixPy v4.1.0。 -* `maixcam2_os_20250801_maixpy_v4.11.0.xz`: MaixCAM2 系统镜像,包含了 MaixPy v4.11.0。由于MaixCAM2系统镜像超过2G, 因此只会放在Sourceforge上, 中国国内用户可以通过QQ群的群共享下载 -注意一定要下载对应型号的系统镜像,下载错误可能导致设备损坏。 +烧录前请先确认设备型号和系统存储位置。MaixCAM / MaixCAM-Pro 的系统运行在 TF 卡中,需要准备 TF 卡;MaixCAM2 默认运行在板载 eMMC 中,日常升级通常不需要 TF 卡。常规升级建议优先使用对应烧录页面中的 USB 烧录流程;需要从 TF 卡启动、恢复系统或更换存储介质时,再按页面说明选择对应方式。 -> 中国国内用户下载速度慢可以用迅雷下载,速度可能会快一些。 -> 或者使用例如 [github.abskoop.workers.dev](https://github.abskoop.workers.dev/) 这种代理网站下载。 +确认后,按下表选择对应的系统镜像和烧录页面。 + +| 设备型号 | 下载入口 | 选择文件 | 烧录页面 | +| --- | --- | --- | --- | +| MaixCAM | [MaixPy 发布页面](https://github.com/sipeed/MaixPy/releases) | `maixcam-*.img.xz` | [MaixCAM 系统烧录](https://wiki.sipeed.com/hardware/zh/maixcam/os.html) | +| MaixCAM-Pro | [MaixPy 发布页面](https://github.com/sipeed/MaixPy/releases) | `maixcam-pro-*.img.xz` | [MaixCAM 系统烧录](https://wiki.sipeed.com/hardware/zh/maixcam/os.html) | +| MaixCAM2 | [百度网盘(提取码:vjex)](https://pan.baidu.com/s/1r4ECNlaTVxhWIafNBZOztg) | `maixcam2-*-maixpy-*_sd.img.7z.*` | [MaixCAM2 系统烧录](https://wiki.sipeed.com/hardware/zh/maixcam/maixcam2_os.html) | + +注意一定要下载对应型号的系统镜像,下载错误可能导致设备异常,甚至需要重新救砖烧录。 -备用地址:[Sourceforge](https://sourceforge.net/projects/maixpy/files/) (同步可能不及时,建议优先上面的方式) +下载时请按表格里的“选择文件”列选择同型号的最新文件。MaixCAM 和 MaixCAM-Pro 请在发布页面中选择对应镜像;MaixCAM2 镜像为分卷压缩包,请在百度网盘中下载同一版本的全部 `.7z.00x` 文件后再解压烧录。 + + +> 中国国内用户下载速度慢可以用迅雷下载,速度可能会快一些。 ## 备份数据 @@ -33,18 +39,6 @@ title: MaixCAM MaixPy 升级和烧录系统 * 直接用读卡器插到电脑拷贝。注意根目录是`ext4`格式,`Windows`默认不支持(可以用三方软件比如diskgenius 读取)。 -## 烧录系统到硬件 - -| 项目 | MaixCAM / MaixCAM-Pro | MaixCAM2 | -| --- | --- | --- | -| 烧录文档 | [MaixCAM 系统烧录](https://wiki.sipeed.com/hardware/zh/maixcam/os.html) | [MaixCAM2 系统烧录](https://wiki.sipeed.com/hardware/zh/maixcam/maixcam2_os.html) | -| 系统存放位置 | TF 卡 | 内置EMMC(/TF卡) | -| 必须 TF 卡 | 是 | 否 | -| 烧录方式 | USB 烧录 或 读卡器烧录 | USB 烧录 或 读卡器烧录 | -| 推荐烧录方式 | USB 烧录 | USB 烧录 | -| 救砖烧录方式 | 读卡器烧录 | USB烧录/读卡器烧录 | - - ## 什么时候需要更新系统,什么时候可以只更新 MaixPy 为了简单并且不出问题,升级 `MaixPy` 一律**推荐直接更新系统**。 diff --git a/docs/doc/zh/pro/datasets.md b/docs/doc/zh/pro/datasets.md index 8450eb08..eb0fe47b 100644 --- a/docs/doc/zh/pro/datasets.md +++ b/docs/doc/zh/pro/datasets.md @@ -6,6 +6,14 @@ title: MaixCAM MaixPy 训练模型哪里能找到模型和数据集 到[MaixHub 模型库](https://maixhub.com/model/zoo) 筛选对应的硬件平台即可找到。 +如果只是想验证功能,先用系统内置模型最省时间。设备系统一般会在 `/root/models` 目录内置常用模型,不需要额外下载,直接在代码里引用对应的 `.mud` 文件即可。 + +如果需要更多模型,可以进入 [MaixHub 模型库](https://maixhub.com/model/zoo?platform=maixcam2),筛选 MaixCAM2 平台,下载模型包。下载后重点确认文件里是否包含: + +- `.mud`:模型描述文件,MaixPy 代码通常加载这个文件。 +- `.axmodel`:实际运行在 MaixCAM2 上的模型文件。 +- 示例代码或说明文件:如果模型库页面提供了示例,优先按示例运行。 + ## 数据集有什么用 可以先到[MaixHub 模型库](https://maixhub.com/model/zoo)看看有没有你需要的模型,如果没有,你可以自己训练模型,训练模型需要数据集,数据集就是用来训练模型的。 diff --git a/docs/doc/zh/sidebar.yaml b/docs/doc/zh/sidebar.yaml index 289c4f0a..c8ba3f3c 100644 --- a/docs/doc/zh/sidebar.yaml +++ b/docs/doc/zh/sidebar.yaml @@ -137,24 +137,26 @@ items: label: Whisper 语音识别模型 - file: mllm/tts_melotts.md label: MeloTTS 语音合成模型 -- label: AI 模型本地NPU部署 +- label: AI 模型本地部署 items: + - file: ai_model_converter/ai_model_deploy.md + label: AI 模型下载和部署指南 + - file: pro/datasets.md + label: 哪里找模型和数据集 + - file: vision/maixhub_train.md + label: MaixHub 在线训练 AI 模型 + - file: vision/customize_model_yolo.md + label: 离线训练 YOLO 模型 - file: ai_model_converter/maixcam2.md - label: ONNX 模型转给 MaixCAM2 用 + label: ONNX 模型转换为 MaixCAM2 模型 - file: ai_model_converter/maixcam.md - label: ONNX 模型转给 MaixCAM 用 + label: ONNX 模型转换为 MaixCAM 模型 - file: ai_model_converter/onnx_export.md - label: 裁剪 ONNX 模型节点教程 + label: 裁剪 ONNX 模型输出节点 - file: ai_model_converter/web_converter.md - label: 图形化模型转换平台 - - file: pro/datasets.md - label: 哪里找模型和数据集 + label: 使用网页工具转换 YOLO 模型 - file: pro/customize_model.md - label: 移植新模型 - - file: vision/customize_model_yolo.md - label: 离线训练 YOLO 模型 - - file: vision/maixhub_train.md - label: MaixHub 在线训练 AI 模型 + label: 移植新的 AI 模型 # - label: YOLO模型零算法基础入门使用 diff --git a/docs/doc/zh/vision/maixhub_train.md b/docs/doc/zh/vision/maixhub_train.md index ca630d33..65ed021e 100644 --- a/docs/doc/zh/vision/maixhub_train.md +++ b/docs/doc/zh/vision/maixhub_train.md @@ -1,61 +1,136 @@ --- -title: MaixCAM MaixPy 使用 MaixHub 在线训练 AI 模型给 MaixPy 使用 -update: - - date: 2024-04-03 - author: neucrack - version: 1.0.0 - content: 初版文档 +title: MaixHub 在线训练 AI 模型 --- ## 简介 -MaixHub 提供了在线训练 AI 模型的功能,可以直接在浏览器中训练模型,不需要购买昂贵的机器,不需要搭建复杂的开发环境,也不需要写代码,非常适合入门,也适合懒得翻代码的老手。 +[MaixHub](https://maixhub.com/) 提供在线训练 AI 模型的功能,可以在浏览器中完成数据采集、上传、标注、训练和部署,不需要本地安装训练环境,也不需要自行配置 GPU。 -## 使用 MaixHub 训练模型的基本步骤 +本文按官方示例视频的流程整理,以 **图像检测模型** 为例说明完整操作步骤。若只是想快速体验 AI 功能,可先到 [MaixHub 模型库](https://maixhub.com/model/zoo) 查找是否已有可直接使用的模型;若需要识别自定义目标,再使用在线训练功能。 +![](../../assets/maixhub_home.png) -### 确认要识别的数据类型和模型类型 +> 下方截图仅用于说明操作流程,账号、项目、数据集、图片文件名、二维码和训练任务编号等隐私信息已做模糊处理。 -要训练一个 AI 模型,需要先确定是什么数据和模型,目前 MaixHub(2024.4)提供了图像数据的`物体分类模型`和`物体检测模型`,都是图像识别模型, `物体分类模型` 比 `物体检测模型` 更简单,因为物体检测需要标注物体在图中的位置,会比较麻烦,物体分类则只需要给出图像中是什么,不需要坐标,所以更简单, 如果是初学者建议先从物体分类开始。 +## 官方视频演示 +MaixHub 首页提供两段官方视频。建议先观看“快速上手”,了解在线训练的整体流程;需要跟随页面逐步操作时,再观看“详细教程”。登录 [MaixHub](https://maixhub.com/) 后,在首页顶部的视频区域点击“立即观看”即可观看。 -### 采集数据 +![MaixHub 官方视频演示入口](../../assets/maixhub_train_video_entries.png) -如前面的 AI 基础所说,要训练模型,必须准备训练用的数据集让 AI 学习,对于图像训练,我们需要创建一个数据集,并且上传图片到数据集。 +## MaixHub 训练模型流程 -保证设备已经连接网络(WiFi)。 -打开设备上的 MaixHub 应用选择 采集数据 来拍照并一键上传到 MaixHub。需要先在 MaixHub 创建数据集,然后点击 设备 上传数据,会出现一个 二维码,设备扫描二维码来与MaixHub 建立连接。 +本节将完整流程放在同一个教程分类下,按实际操作顺序完成项目创建、数据准备、标注、训练和部署。 -注意要分清训练集和验证集的区别,要想实机运行的效果和训练效果相当,验证集的数据一定要和实机运行拍摄的图像质量一样,训练集也建议用设备拍摄的,如果要用网上的图片,一定只能用在训练集,不要用在验证集,因为数据量小,数据集与实机运行越接近越好。 +| 步骤 | 操作内容 | +| --- | --- | +| 创建项目 | 选择模型类型和目标硬件平台 | +| 准备数据集 | 使用设备或本地图片上传训练集、验证集 | +| 标注图片 | 检测模型需要逐张框选目标 | +| 创建训练任务 | 选择模型、图像增强和训练参数 | +| 查看训练结果 | 查看训练曲线和验证集抽样结果 | +| 部署到设备 | 下载模型包,并通过 MaixVision 手动上传到设备 | + +### 创建训练项目 + +进入 MaixHub 首页后,选择“模型训练”,创建新的训练项目。创建项目时需要选择模型类型和硬件平台: + +![MaixHub 选择训练项目类型](../../assets/maixhub_train_model_type.png) + +本文后续以 **图像检测模型** 为例进行说明,适用于需要在画面中定位目标位置的场景。 + +项目创建完成后,按页面左侧导航依次完成数据集、标注、训练和部署。 + +### 准备数据集 + +进入项目后先创建数据集。数据类型和标注类型需与项目一致,例如检测模型应选择图像数据和检测标注。 + +推荐使用设备端 MaixHub 应用采集图片。设备采集的数据更接近实际部署时的镜头、分辨率和光照条件,训练后的模型更容易在设备上稳定运行。 + +网页端进入“采集数据”页面,选择采集到训练集或验证集,然后生成二维码: + +![MaixHub 设备扫码采集](../../assets/maixhub_train_device_qrcode.png) + +基本流程: + +1. 确认设备已连接 WiFi。 +2. 在网页端创建并选择数据集。 +3. 选择采集到训练集或验证集,生成二维码。 +4. 在设备端 MaixHub 应用中扫码采集并上传图片。 + +训练集用于学习目标特征,验证集用于评估训练效果。检测模型建议每个标签在验证集中至少保留 5 张图片,否则可能无法开始训练。验证集不要与训练集重复,并尽量使用真实场景图片。 + +可在数据集页面批量选择图片,并移动到训练集或验证集: + +![MaixHub 整理训练集和验证集](../../assets/maixhub_train_split_validation.png) ### 标注数据 -对于分类模型,在上传的时候就顺便已经标注好了,即上传时选择好了图片属于那个分类。 +图像分类模型只需为图片选择类别。图像检测模型需要逐张框选目标,并为每个框选择标签。 + +进入“标注数据”页面后,创建标注、框选目标并保存: + +![MaixHub 标注数据页面](../../assets/maixhub_train_annotate_page.png) + +标注时注意: + +* 边框尽量贴合目标主体,不要包含过多背景。 +* 同一类目标使用一致的框选标准。 +* 图片中出现了需要识别的目标时,不要漏标。 +* 模糊、遮挡严重或无法判断类别的图片,可先不放入训练集。 + +数据较多时,建议先用少量图片跑通一次完整流程,再逐步增加数据优化效果。 + +### 创建训练任务 + +数据和标注检查完成后,进入“创建任务”页面。页面主要包含图像增强、选择模型和训练参数三部分: + +![MaixHub 配置训练参数](../../assets/maixhub_train_parameters.png) + +新手建议: + +* 部署平台选择实际设备,例如 MaixCAM 或 MaixCAM2。 +* 模型网络先使用页面默认推荐项。 +* 图像增强可适当开启,但不能替代真实场景数据。 +* 类别数量不均衡时,可开启数据均衡。 +* 需要降低背景误识别时,可加入不含目标的负样本。 + +确认模型信息和参数后,点击“创建训练任务”,输入任务名称并开始训练。 + +### 查看训练结果 + +训练开始后,可在“训练记录”中查看进度、日志、数据集统计和训练参数。训练完成后,结果页会显示损失曲线、准确率曲线和验证集抽样结果: -对于目标检测模型,上传完成后需要进行手动标注,即在每一张图中框出要被识别物体的坐标大小和分类。 -这个标注过程你也可以选择自己在自己的电脑中离线用比如 labelimg 这样的软件标注完毕后使用数据集中的导入功能导入到 MaixHub。 -标注时善用快捷键标注起来会更快,后面MaixHub 也会增加更多辅助标注和自动标注工具(目前在上传视频处有自动标注工具也可以尝试使用)。 +![MaixHub 训练完成结果](../../assets/maixhub_train_result.png) +优先检查: -### 训练模型 +* 曲线是否稳定,是否出现明显异常。 +* 验证集示例是否识别正确。 +* 错误结果是否集中出现在某些角度、光照或背景条件下。 -选择训练参数训练,选择对应的设备平台,选择 maixcam,等待排队训练,可以实时看到训练进度,等待完成即可。 +如果训练失败,先查看右侧训练日志。如下示例中,失败原因是验证集中某个标签的图片数量不足 5 张,需要回到数据集补充验证图片后重新训练。 -### 部署模型 +![MaixHub 验证集数量不足导致训练失败](../../assets/maixhub_train_validation_failed.png) -训练完成后,可以设备的 MaixHub 应用中选择 部署 功能,扫码进行部署。 -设备开会自动下载模型并且运行起来,模型会被存在本地,后面也能选择再次运行。 +### 部署到 MaixCAM / MaixCAM2 -如果你觉得识别效果很不错,可以一键分享到模型库让更多人使用。 +训练完成并确认验证效果后,进入项目的“部署模型”页面,选择需要部署的训练记录。部署方式选择“手动部署”,点击“下载模型”获取模型压缩包。 +MaixHub 手动部署页面 +模型包下载完成后先解压。以 MaixCAM2 图像检测模型为例,压缩包中通常包含 `model_xxx.mud`、`model_xxx_npu.axmodel`、`model_xxx_vnpu.axmodel`、`main.py`、`app.yaml` 和 `report.json` 等文件。其中 `.mud` 和 `.axmodel` 文件是手动部署时需要上传到设备的模型文件,`main.py` 和 `app.yaml` 可作为示例程序或应用配置参考。 -## 使用方法 +MaixHub 模型压缩包内容 -请到 [MaixHub](https://maixhub.com) 注册账号,然后登录,主页有视频教程,学习即可。 +打开 MaixVision 并连接设备,进入左侧“设备文件管理器”。在设备端进入 `/root/models` 目录,用于存放模型文件。 -注意教程如果是使用了 M2dock 这个开发板,和 MaixCAM也是类似的,只是设备(板子)上使用的 MaixHub 应用可能稍微有点区别,大体上是相同的,请注意举一反三。 +MaixVision 设备文件管理器 +点击“上传文件”,选择解压目录中的 `model_xxx.mud`、`model_xxx_npu.axmodel` 和 `model_xxx_vnpu.axmodel`,上传到设备的 `/root/models` 目录。 +上传 MaixHub 模型文件 +上传完成后,可在设备端运行示例程序,或在项目代码中加载对应的 `.mud` 文件。模型运行后,需要在真实场景中复测检测效果,确认目标框位置、置信度和误检情况是否符合预期。 +MaixHub 模型设备端运行效果