Skip to content

Commit

Permalink
Merge pull request #124 from opendatalab/schema-dsdl
Browse files Browse the repository at this point in the history 
Schema dsdl
  • Loading branch information
@wangbinDL
wangbinDL committed May 10, 2023
2 parents 9f339cf + 97b6904 commit 89ab525
Show file tree
Hide file tree
Showing 156 changed files with 7,151 additions and 54,918 deletions.
25 changes: 13 additions & 12 deletions README.md
View file
Expand Up @@ -56,6 +56,7 @@ python setup.py install

#### Use dsdl parser to deserialize the Yaml file to Python code
```bash
cd <dsdl project directory>
dsdl parse --yaml demo/coco_demo.yaml
```

Expand All @@ -66,7 +67,7 @@ Create a configuration file `config.py` with the following contents（for now ds
```python
local = dict(
type="LocalFileReader",
working_dir="local path of your media",
working_dir="demo",
)

ali_oss = dict(
Expand All @@ -86,21 +87,21 @@ ali_oss = dict(
#### Visualize samples

```bash
dsdl view -y <yaml-name>.yaml -c <config.py> -l ali-oss -n 10 -r -v -f Label BBox Attributes
dsdl view -y <yaml-name>.yaml -c <config.py> -l ali-oss -n 5 -r -v -f Label BBox
```

The description of each argument is shown below:

| simplified argument | argument | description |
| -------------------- | ------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| -y | `--yaml` | The path of dsdl yaml file. |
| -c | `--config` | The path of location configuration file. |
| -l | `--location` | `local` or `ali-oss`，which means read media from local or aliyun oss. |
| -n | `--num` | The number of samples to be visualized. |
| -r | `--random` | Whether to load the samples in a random order. |
| -v | `--visualize` | Whether to visualize the samples or just print the information in console. |
| -f | `--field` | The field type to visualize, e.g. `-f BBox`means show the bounding box in samples, `-f Attributes`means show the attributes of a sample in the console . One can specify multiple field types simultaneously, such as `-f Label BBox Attributes`. |
| -t | `--task` | The task you are working on, for example, `-t detection` is equivalent to `-f Label BBox Polygon Attributes`. |
| simplified argument | argument | description |
| -------------------- | ------------- | :----------------------------------------------------------- |
| -y | `--yaml` | The path of dsdl yaml file. |
| -c | `--config` | The path of location configuration file. |
| -l | `--location` | `local` or `ali-oss`，which means read media from local or aliyun oss. |
| -n | `--num` | The number of samples to be visualized. |
| -r | `--random` | Whether to load the samples in a random order. |
| -v | `--visualize` | Whether to visualize the samples or just print the information in console. |
| -f | `--field` | The field type to visualize, e.g. `-f BBox`means show the bounding box in samples, `-f Label`means show the labels of a sample in the console . One can specify multiple field types simultaneously, such as `-f Label BBox`. |
| -t | `--task` | The task you are working on, for example, `-t detection` is equivalent to `-f Label BBox Polygon`. |

## Citation

Expand Down
24 changes: 12 additions & 12 deletions README_zh-CN.md
View file
Expand Up @@ -66,7 +66,7 @@ dsdl parse --yaml demo/coco_demo.yaml
```python
local = dict(
type="LocalFileReader",
working_dir="local path of your media",
working_dir="demo",
)

ali_oss = dict(
Expand All @@ -86,21 +86,21 @@ ali_oss = dict(
#### 可视化功能展示

```bash
dsdl view -y <yaml-name>.yaml -c <config.py> -l ali-oss -n 10 -r -v -f Label BBox Attributes
dsdl view -y <yaml-name>.yaml -c <config.py> -l ali-oss -n 10 -r -v -f Label BBox
```

每个参数的意义为：

| 参数简写 | 参数全写 | 参数解释 |
| -------- | ------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------- |
| -y | `--yaml` | 定义所有samples的yaml文件的路径 |
| -c | `--config` | 配置文件（`config.py`）的路径 |
| -l | `--location` | 只可以指定为`local`或是`ali-oss`，分别表示读取本地的数据与读取阿里云的数据 |
| -n | `--num` | 加载数据集的样本数量 |
| -r | `--random` | 在加载数据集中的样本时是否随机选取样本，如果不指定的话就按顺序从开始选取样本 |
| -v | `--visualize` | 是否将加载的数据进行可视化展示 |
| -f | `--field` | 选择需要进行可视化的字段，如`-f BBox`表示可视化bbox，`-f Attributes`表示对样本的attributes进行可视化等等，可以同时选择多个，如`-f Label BBox Attributes` |
| -t | `--task` | 可以选择当前需要可视化的任务类型，如果选择`-t detection`，则等价于`-f Label BBox Polygon Attributes` |
| 参数简写 | 参数全写 | 参数解释 |
| -------- | ------------- | :----------------------------------------------------------- |
| -y | `--yaml` | 定义所有samples的yaml文件的路径 |
| -c | `--config` | 配置文件（`config.py`）的路径 |
| -l | `--location` | 只可以指定为`local`或是`ali-oss`，分别表示读取本地的数据与读取阿里云的数据 |
| -n | `--num` | 加载数据集的样本数量 |
| -r | `--random` | 在加载数据集中的样本时是否随机选取样本，如果不指定的话就按顺序从开始选取样本 |
| -v | `--visualize` | 是否将加载的数据进行可视化展示 |
| -f | `--field` | 选择需要进行可视化的字段，如`-f BBox`表示可视化bbox，`-f Label`表示对样本的label进行可视化等等，可以同时选择多个，如`-f Label BBox` |
| -t | `--task` | 可以选择当前需要可视化的任务类型，如果选择`-t detection`，则等价于`-f Label BBox Polygon` |

## 引用

Expand Down
53 changes: 0 additions & 53 deletions demo/README.md
View file

This file was deleted.

86 changes: 1 addition & 85 deletions ...dsdl_library/class/coco-class-domain.yaml → demo/coco-class-domain.yaml
View file
@@ -1,89 +1,5 @@
$dsdl-version: "0.5.0"

COCOClassDom:
$def: class_domain
classes:
- person
- bicycle
- car
- motorcycle
- airplane
- bus
- train
- truck
- boat
- traffic light
- fire hydrant
- stop sign
- parking meter
- bench
- bird
- cat
- dog
- horse
- sheep
- cow
- elephant
- bear
- zebra
- giraffe
- backpack
- umbrella
- handbag
- tie
- suitcase
- frisbee
- skis
- snowboard
- sports ball
- kite
- baseball bat
- baseball glove
- skateboard
- surfboard
- tennis racket
- bottle
- wine glass
- cup
- fork
- knife
- spoon
- bowl
- banana
- apple
- sandwich
- orange
- broccoli
- carrot
- hot dog
- pizza
- donut
- cake
- chair
- couch
- potted plant
- bed
- dining table
- toilet
- tv
- laptop
- mouse
- remote
- keyboard
- cell phone
- microwave
- oven
- toaster
- sink
- refrigerator
- book
- clock
- vase
- scissors
- teddy bear
- hair drier
- toothbrush

COCOClassFullDom:
$def: class_domain
classes:
Expand Down Expand Up @@ -177,4 +93,4 @@ COCOClassFullDom:
- teddy bear
- hair drier
- toothbrush
- hair brush
- hair brush
28 changes: 28 additions & 0 deletions demo/coco_demo.py
View file
@@ -0,0 +1,28 @@
# Generated by the dsdl parser. DO NOT EDIT!
from dsdl.geometry import ClassDomain
from dsdl.fields import *


class LocalObjectEntry(Struct):
__params__ = ['cdom']
__fields__ = {
"bbox": BBox(),
"label": Label(dom="$cdom"),
"is_crowd": Bool(),
}
__optional__ = ['label']


ClassDomain(
name = "COCOClassDom",
classes = ["person","bicycle","car","motorcycle","airplane","bus","train","truck","boat","traffic light","fire hydrant","stop sign","parking meter","bench","bird","cat","dog","horse","sheep","cow","elephant","bear","zebra","giraffe","backpack","umbrella","handbag","tie","suitcase","frisbee","skis","snowboard","sports ball","kite","baseball bat","baseball glove","skateboard","surfboard","tennis racket","bottle","wine glass","cup","fork","knife","spoon","bowl","banana","apple","sandwich","orange","broccoli","carrot","hot dog","pizza","donut","cake","chair","couch","potted plant","bed","dining table","toilet","tv","laptop","mouse","remote","keyboard","cell phone","microwave","oven","toaster","sink","refrigerator","book","clock","vase","scissors","teddy bear","hair drier","toothbrush",],
)


class ObjectDetectionSample(Struct):
__params__ = ['cdom']
__fields__ = {
"image": Image(),
"objects": List(etype=LocalObjectEntry(cdom="$cdom")),
}

0 comments on commit 89ab525

Please sign in to comment.