mirror of
https://github.com/FunAudioLLM/SenseVoice.git
synced 2025-09-15 15:08:35 +08:00
update docs
This commit is contained in:
北念
parent
14cbaf90be
commit
6156332296
85
README.md
85
README.md
@ -247,6 +247,91 @@ Data examples
|
||||
|
||||
Full ref to `data/train_example.jsonl`
|
||||
|
||||
Description：
|
||||
- `key`: audio file unique ID
|
||||
- `source`：path to the audio file
|
||||
- `source_len`：number of fbank frames of the audio file
|
||||
- `target`：transcription
|
||||
- `target_len`：length of target
|
||||
- `text_language`：language id of the audio file
|
||||
- `emo_target`：emotion label of the audio file
|
||||
- `event_target`：event label of the audio file
|
||||
- `with_or_wo_itn`：whether includes punctuation and inverse text normalization
|
||||
|
||||
|
||||
`train_text.txt`
|
||||
|
||||
|
||||
```bash
|
||||
BAC009S0764W0121 甚至出现交易几乎停滞的情况
|
||||
BAC009S0916W0489 湖北一公司以员工名义贷款数十员工负债千万
|
||||
asr_example_cn_en 所有只要处理 data 不管你是做 machine learning 做 deep learning 做 data analytics 做 data science 也好 scientist 也好通通都要都做的基本功啊那 again 先先对有一些>也许对
|
||||
ID0012W0014 he tried to think how it could be
|
||||
```
|
||||
|
||||
`train_wav.scp`
|
||||
|
||||
|
||||
|
||||
```bash
|
||||
BAC009S0764W0121 https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav
|
||||
BAC009S0916W0489 https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0916W0489.wav
|
||||
asr_example_cn_en https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_cn_en.wav
|
||||
ID0012W0014 https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_en.wav
|
||||
```
|
||||
|
||||
`train_text_language.txt`
|
||||
|
||||
The language ids include `<|zh|>`、`<|en|>`、`<|yue|>`、`<|ja|>` and `<|ko|>`.
|
||||
|
||||
```bash
|
||||
BAC009S0764W0121 <|zh|>
|
||||
BAC009S0916W0489 <|zh|>
|
||||
asr_example_cn_en <|zh|>
|
||||
ID0012W0014 <|en|>
|
||||
```
|
||||
|
||||
`train_emo.txt`
|
||||
|
||||
The emotion labels include`<|HAPPY|>`、`<|SAD|>`、`<|ANGRY|>`、`<|NEUTRAL|>`、`<|FEARFUL|>`、`<|DISGUSTED|>` and `<|SURPRISED|>`.
|
||||
|
||||
```bash
|
||||
BAC009S0764W0121 <|NEUTRAL|>
|
||||
BAC009S0916W0489 <|NEUTRAL|>
|
||||
asr_example_cn_en <|NEUTRAL|>
|
||||
ID0012W0014 <|NEUTRAL|>
|
||||
```
|
||||
|
||||
`train_event.txt`
|
||||
|
||||
The event labels include`<|BGM|>`、`<|Speech|>`、`<|Applause|>`、`<|Laughter|>`、`<|Cry|>`、`<|Sneeze|>`、`<|Breath|>` and `<|Cough|>`.
|
||||
|
||||
```bash
|
||||
BAC009S0764W0121 <|Speech|>
|
||||
BAC009S0916W0489 <|Speech|>
|
||||
asr_example_cn_en <|Speech|>
|
||||
ID0012W0014 <|Speech|>
|
||||
```
|
||||
|
||||
`Command`
|
||||
```shell
|
||||
# generate train.jsonl and val.jsonl from wav.scp, text.txt, text_language.txt, emo_target.txt, event_target.txt
|
||||
sensevoice2jsonl \
|
||||
++scp_file_list='["../../../data/list/train_wav.scp", "../../../data/list/train_text.txt", "../../../data/list/train_text_language.txt", "../../../data/list/train_emo.txt", "../../../data/list/train_event.txt"]' \
|
||||
++data_type_list='["source", "target", "text_language", "emo_target", "event_target"]' \
|
||||
++jsonl_file_out="../../../data/list/train.jsonl"
|
||||
```
|
||||
|
||||
If there is no `train_text_language.txt`, `train_emo_target.txt` and `train_event_target.txt`, the language, emotion and event label will be predicted automatically by using the `SenseVoice` model.
|
||||
```shell
|
||||
# generate train.jsonl and val.jsonl from wav.scp and text.txt
|
||||
sensevoice2jsonl \
|
||||
++scp_file_list='["../../../data/list/train_wav.scp", "../../../data/list/train_text.txt"]' \
|
||||
++data_type_list='["source", "target"]' \
|
||||
++jsonl_file_out="../../../data/list/train.jsonl"
|
||||
```
|
||||
|
||||
|
||||
### Finetune
|
||||
|
||||
Ensure to modify the train_tool in finetune.sh to the absolute path of `funasr/bin/train_ds.py` from the FunASR installation directory you have set up earlier.
|
||||
|
||||
91
README_zh.md
91
README_zh.md
@ -254,6 +254,97 @@ pip3 install -e ./
|
||||
```
|
||||
详细可以参考：`data/train_example.jsonl`
|
||||
|
||||
字段说明：
|
||||
- `key`: 数据唯一ID
|
||||
- `source`：音频文件的路径
|
||||
- `source_len`：音频文件的fbank帧数
|
||||
- `target`：音频文件标注文本
|
||||
- `target_len`：音频文件标注文本长度
|
||||
- `text_language`：音频文件的语种标签
|
||||
- `emo_target`：音频文件的情感标签
|
||||
- `event_target`：音频文件的事件标签
|
||||
- `with_or_wo_itn`：标注文本中是否包含标点与逆文本正则化
|
||||
|
||||
可以用指令 `sensevoice2jsonl` 从train_wav.scp、train_text.txt、train_text_language.txt、train_emo_target.txt和train_event_target.txt生成，准备过程如下：
|
||||
|
||||
`train_text.txt`
|
||||
|
||||
左边为数据唯一ID，需与`train_wav.scp`中的`ID`一一对应
|
||||
右边为音频文件标注文本，格式如下：
|
||||
|
||||
```bash
|
||||
BAC009S0764W0121 甚至出现交易几乎停滞的情况
|
||||
BAC009S0916W0489 湖北一公司以员工名义贷款数十员工负债千万
|
||||
asr_example_cn_en 所有只要处理 data 不管你是做 machine learning 做 deep learning 做 data analytics 做 data science 也好 scientist 也好通通都要都做的基本功啊那 again 先先对有一些>也许对
|
||||
ID0012W0014 he tried to think how it could be
|
||||
```
|
||||
|
||||
`train_wav.scp`
|
||||
|
||||
左边为数据唯一ID，需与`train_text.txt`中的`ID`一一对应
|
||||
右边为音频文件的路径，格式如下
|
||||
|
||||
```bash
|
||||
BAC009S0764W0121 https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav
|
||||
BAC009S0916W0489 https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0916W0489.wav
|
||||
asr_example_cn_en https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_cn_en.wav
|
||||
ID0012W0014 https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_en.wav
|
||||
```
|
||||
|
||||
`train_text_language.txt`
|
||||
|
||||
左边为数据唯一ID，需与`train_text_language.txt`中的`ID`一一对应
|
||||
右边为音频文件的语种标签，支持`<|zh|>`、`<|en|>`、`<|yue|>`、`<|ja|>`和`<|ko|>`，格式如下
|
||||
|
||||
```bash
|
||||
BAC009S0764W0121 <|zh|>
|
||||
BAC009S0916W0489 <|zh|>
|
||||
asr_example_cn_en <|zh|>
|
||||
ID0012W0014 <|en|>
|
||||
```
|
||||
|
||||
`train_emo.txt`
|
||||
|
||||
左边为数据唯一ID，需与`train_emo.txt`中的`ID`一一对应
|
||||
右边为音频文件的情感标签，支持`<|HAPPY|>`、`<|SAD|>`、`<|ANGRY|>`、`<|NEUTRAL|>`、`<|FEARFUL|>`、`<|DISGUSTED|>`和`<|SURPRISED|>`，格式如下
|
||||
|
||||
```bash
|
||||
BAC009S0764W0121 <|NEUTRAL|>
|
||||
BAC009S0916W0489 <|NEUTRAL|>
|
||||
asr_example_cn_en <|NEUTRAL|>
|
||||
ID0012W0014 <|NEUTRAL|>
|
||||
```
|
||||
|
||||
`train_event.txt`
|
||||
|
||||
左边为数据唯一ID，需与`train_event.txt`中的`ID`一一对应
|
||||
右边为音频文件的事件标签，支持`<|BGM|>`、`<|Speech|>`、`<|Applause|>`、`<|Laughter|>`、`<|Cry|>`、`<|Sneeze|>`、`<|Breath|>`和`<|Cough|>`，格式如下
|
||||
|
||||
```bash
|
||||
BAC009S0764W0121 <|Speech|>
|
||||
BAC009S0916W0489 <|Speech|>
|
||||
asr_example_cn_en <|Speech|>
|
||||
ID0012W0014 <|Speech|>
|
||||
```
|
||||
|
||||
`生成指令`
|
||||
```shell
|
||||
# generate train.jsonl and val.jsonl from wav.scp, text.txt, text_language.txt, emo_target.txt, event_target.txt
|
||||
sensevoice2jsonl \
|
||||
++scp_file_list='["../../../data/list/train_wav.scp", "../../../data/list/train_text.txt", "../../../data/list/train_text_language.txt", "../../../data/list/train_emo.txt", "../../../data/list/train_event.txt"]' \
|
||||
++data_type_list='["source", "target", "text_language", "emo_target", "event_target"]' \
|
||||
++jsonl_file_out="../../../data/list/train.jsonl"
|
||||
```
|
||||
|
||||
若无train_text_language.txt、train_emo_target.txt和train_event_target.txt，则自动通过使用`SenseVoice`模型对语种、情感和事件打标。
|
||||
```shell
|
||||
# generate train.jsonl and val.jsonl from wav.scp and text.txt
|
||||
sensevoice2jsonl \
|
||||
++scp_file_list='["../../../data/list/train_wav.scp", "../../../data/list/train_text.txt"]' \
|
||||
++data_type_list='["source", "target"]' \
|
||||
++jsonl_file_out="../../../data/list/train.jsonl"
|
||||
```
|
||||
|
||||
### 启动训练
|
||||
|
||||
注意修改 `finetune.sh` 中 `train_tool` 为你前面安装FunASR路径中`funasr/bin/train_ds.py`绝对路径
|
||||
|
||||
Loading…
Reference in New Issue
Block a user