Create a FiftyOne dataset from a directory of media files (images, videos, point clouds), optionally import labels in common formats (COCO, YOLO, VOC), run model inference, and store predictions. Use when users want to load local files into FiftyOne, apply ML models for detection, classification, or segmentation, or build end-to-end inference pipelines.
This skill inherits all available tools. When active, it can use any tool Claude has access to.
Create FiftyOne datasets from local directories, import labels in standard formats, and run model inference to generate predictions.
Use this skill when:
@voxel51/io plugin for importing data@voxel51/zoo plugin for model inference@voxel51/utils plugin for dataset managementALWAYS follow these rules:
Scan the user's directory before importing to detect media types and label formats.
Present findings and get confirmation before creating datasets or running inference.
set_context(dataset_name="my-dataset")
launch_app(dataset_name="my-dataset")
Always ask the user for:
close_app()
Use Bash to scan the user's directory:
ls -la /path/to/directory
find /path/to/directory -type f | head -20
Identify media files and label files. See Supported Dataset Types section for format detection.
Before creating the dataset, confirm with the user:
I found the following in /path/to/directory:
- 150 image files (.jpg, .png)
- Labels: COCO format (annotations.json)
Proposed dataset name: "my-dataset"
Label field: "ground_truth"
Should I proceed with these settings?
execute_operator(
operator_uri="@voxel51/utils/create_dataset",
params={
"name": "my-dataset",
"persistent": true
}
)
Set context to the newly created dataset before importing:
set_context(dataset_name="my-dataset")
For media only (no labels):
execute_operator(
operator_uri="@voxel51/io/import_samples",
params={
"import_type": "MEDIA_ONLY",
"style": "DIRECTORY",
"directory": {"absolute_path": "/path/to/images"}
}
)
For media with labels:
execute_operator(
operator_uri="@voxel51/io/import_samples",
params={
"import_type": "MEDIA_AND_LABELS",
"dataset_type": "COCO",
"data_path": {"absolute_path": "/path/to/images"},
"labels_path": {"absolute_path": "/path/to/annotations.json"},
"label_field": "ground_truth"
}
)
Verify samples imported correctly by comparing with source:
load_dataset(name="my-dataset")
Compare num_samples with the file count from Step 1. Report any discrepancy to the user.
launch_app(dataset_name="my-dataset")
Ask user for model name and label field for predictions.
execute_operator(
operator_uri="@voxel51/zoo/apply_zoo_model",
params={
"tab": "BUILTIN",
"model": "yolov8n-coco-torch",
"label_field": "predictions"
}
)
set_view(exists=["predictions"])
close_app()
| Extensions | Media Type |
|---|---|
.jpg, .jpeg, .png, .gif, .bmp, .webp | image |
.mp4, .avi, .mov, .mkv, .webm | video |
.pcd | point-cloud |
.fo3d | 3d |
| Value | File Pattern | Label Types |
|---|---|---|
Image Classification Directory Tree | Folder per class | classification |
Video Classification Directory Tree | Folder per class | classification |
COCO | *.json | detections, segmentations, keypoints |
VOC | *.xml per image | detections |
KITTI | *.txt per image | detections |
YOLOv4 | *.txt + classes.txt | detections |
YOLOv5 | data.yaml + labels/*.txt | detections |
CVAT Image | Single *.xml file | classifications, detections, polylines, keypoints |
CVAT Video | XML directory | frame labels |
TF Image Classification | TFRecords | classification |
TF Object Detection | TFRecords | detections |
Popular models for apply_zoo_model. Some models require additional packages - if a model fails with a dependency error, the response includes the install_command. Offer to run it for the user.
Detection (PyTorch only):
faster-rcnn-resnet50-fpn-coco-torch - Faster R-CNN (no extra deps)retinanet-resnet50-fpn-coco-torch - RetinaNet (no extra deps)Detection (requires ultralytics):
yolov8n-coco-torch - YOLOv8 nano (fast)yolov8s-coco-torch - YOLOv8 smallyolov8m-coco-torch - YOLOv8 mediumClassification:
resnet50-imagenet-torch - ResNet-50mobilenet-v2-imagenet-torch - MobileNet v2Segmentation:
sam-vit-base-hq-torch - Segment Anythingdeeplabv3-resnet101-coco-torch - DeepLabV3Embeddings:
clip-vit-base32-torch - CLIP embeddingsdinov2-vits14-torch - DINOv2 embeddingsexecute_operator(
operator_uri="@voxel51/utils/create_dataset",
params={"name": "my-images", "persistent": true}
)
set_context(dataset_name="my-images")
execute_operator(
operator_uri="@voxel51/io/import_samples",
params={
"import_type": "MEDIA_ONLY",
"style": "DIRECTORY",
"directory": {"absolute_path": "/path/to/images"}
}
)
load_dataset(name="my-images") # Validate import
launch_app(dataset_name="my-images")
execute_operator(
operator_uri="@voxel51/zoo/apply_zoo_model",
params={
"tab": "BUILTIN",
"model": "faster-rcnn-resnet50-fpn-coco-torch",
"label_field": "predictions"
}
)
set_view(exists=["predictions"])
execute_operator(
operator_uri="@voxel51/utils/create_dataset",
params={"name": "coco-dataset", "persistent": true}
)
set_context(dataset_name="coco-dataset")
execute_operator(
operator_uri="@voxel51/io/import_samples",
params={
"import_type": "MEDIA_AND_LABELS",
"dataset_type": "COCO",
"data_path": {"absolute_path": "/path/to/images"},
"labels_path": {"absolute_path": "/path/to/annotations.json"},
"label_field": "ground_truth"
}
)
load_dataset(name="coco-dataset") # Validate import
launch_app(dataset_name="coco-dataset")
execute_operator(
operator_uri="@voxel51/zoo/apply_zoo_model",
params={
"tab": "BUILTIN",
"model": "faster-rcnn-resnet50-fpn-coco-torch",
"label_field": "predictions"
}
)
set_view(exists=["predictions"])
execute_operator(
operator_uri="@voxel51/utils/create_dataset",
params={"name": "yolo-dataset", "persistent": true}
)
set_context(dataset_name="yolo-dataset")
execute_operator(
operator_uri="@voxel51/io/import_samples",
params={
"import_type": "MEDIA_AND_LABELS",
"dataset_type": "YOLOv5",
"dataset_dir": {"absolute_path": "/path/to/yolo/dataset"},
"label_field": "ground_truth"
}
)
load_dataset(name="yolo-dataset")
launch_app(dataset_name="yolo-dataset")
For a folder structure like:
/dataset/
/cats/
cat1.jpg
cat2.jpg
/dogs/
dog1.jpg
dog2.jpg
execute_operator(
operator_uri="@voxel51/utils/create_dataset",
params={"name": "classification-dataset", "persistent": true}
)
set_context(dataset_name="classification-dataset")
execute_operator(
operator_uri="@voxel51/io/import_samples",
params={
"import_type": "MEDIA_AND_LABELS",
"dataset_type": "Image Classification Directory Tree",
"dataset_dir": {"absolute_path": "/path/to/dataset"},
"label_field": "ground_truth"
}
)
load_dataset(name="classification-dataset")
launch_app(dataset_name="classification-dataset")
Error: "Dataset already exists"
@voxel51/utils/delete_datasetError: "No samples found"
recursive=true if needed)Error: "Labels path not found"
Error: "Model not found"
list_operators() and get_operator_schema() to discover available modelsError: "Missing dependency" (e.g., torch, ultralytics)
missing_package and install_commandSlow inference
yolov8n instead of yolov8x)ground_truth vs predictions)get_operator_schema() to discover available parametersCopyright 2017-2025, Voxel51, Inc. Apache 2.0 License