Data Tools

The Data Tools page provides useful tools for editing and managing datasets/models as follows:

Tool	Description
Hugging Face Upload & Download	Upload your datasets/models to Hugging Face or download existing ones
Merging Datasets	Combine multiple datasets into a single dataset
Removing Episodes	Remove unwanted episodes from your dataset

Hugging Face Upload & Download

You can upload collected datasets or trained models to Hugging Face. You can also download datasets or models from Hugging Face.

1. Upload

First, select the Hugging Face user id and data type for upload

INFO

If there is no registered User ID, click the Change button and enter your Hugging Face token to register a User ID.

Set the toggle button to upload.

Select the path of the dataset or model you want to upload in the right panel.

↓

Enter the repository name to be created on Hugging Face.

Click the Upload button. The upload progress will be displayed.

INFO

You can cancel the upload by clicking Cancel during the upload process. However, the repository will still be created on Hugging Face.

After canceling, clicking the Upload button again will resume the upload from where it left off.

2. Download

First, select the Hugging Face user id and data type for download

Set the toggle button to download.

Enter the repository to download.

Click the Download button.

• Dataset: Progress bar is displayed

• Model: Spinner is displayed

INFO

You can cancel the download by clicking Cancel during the download process. However, the folder created in the local path will remain.

INFO

The download paths are as follows (based on the Docker container’s file system):

• Dataset: /root/.cache/huggingface/lerobot/
• Model: /root/ros2_ws/src/physical_ai_tools/lerobot/outputs/train/

TIP

⚠️ Network recommendation

Policy models are typically large in size. When downloading a policy model, a wired Ethernet connection is highly recommended to ensure stable and fast download speeds.

Merging Datasets

You can merge multiple datasets into a single dataset.

1. Enter Datasets to Merge

In the Merge Datasets > Enter Datasets to Merge section:

Click the folder button to open the file explorer and select your desired dataset. (You can also type the path directly in the text input field)

↓

Click the + Add Dataset button to add a new dataset input field and select your dataset.

Repeat the above steps to add as many datasets as needed.

2. Enter Output Path

In the Enter Output Path section, select the path where the merged dataset result will be saved.

Click the folder button to open the file explorer and select your desired folder. (You can also type the path directly in the text input field)

↓

Enter the name for the merged result dataset folder.

Finally, click the Merge button to merge the datasets.

Removing Episodes

You can remove episodes that were incorrectly recorded during the data collection process.

1. Select Dataset to Edit

In the Delete Episodes section, click the folder button to open the file explorer and select your desired dataset. (You can also type the path directly in the text input field)

↓

2. Specify Episodes to Remove

Enter the episode numbers you want to remove in the input field. You can remove multiple episodes at once by separating the numbers with commas.

INFO

After deletion, the remaining episode numbers and metadata will be automatically reorganized and reindexed.

Finally, click the Delete button to remove the specified episodes from the dataset.