From 2a870daed228d11d185e876a44a236c8b19f40f2 Mon Sep 17 00:00:00 2001 From: Wang Wang Wang Date: Mon, 11 Nov 2019 17:26:11 +0800 Subject: [PATCH] doc/Update MultiNet doc --- README.md | 4 +--- README_cn.md | 4 +--- speech_command_recognition/README.md | 27 +++++++++++++++++++++---- speech_command_recognition/README_cn.md | 23 ++++++++++++++++++--- 4 files changed, 45 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index be75064..18bb392 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,4 @@ Currently, Espressif has not only provided an official wake word "Hi, Lexin" to Espressif's speech command recognition model [MultiNet](speech_command_recognition/README.md) is specially designed to provide a flexible off-line speech command recognition model. With this model, you can easily add your own speech commands, eliminating the need to train model again. -Currently, Espressif **MultiNet** supports up to 100 Chinese speech commands, such as “打开空调” (Turn on the air conditioner) and “打开卧室灯” (Turn on the bedroom light). - -We will add supports for English commands in the next release. +Currently, Espressif **MultiNet** supports up to 100 Chinese or English speech commands, such as “打开空调” (Turn on the air conditioner) and “打开卧室灯” (Turn on the bedroom light). diff --git a/README_cn.md b/README_cn.md index 62c2f05..6bf781e 100644 --- a/README_cn.md +++ b/README_cn.md @@ -18,7 +18,5 @@ esp_sr 提供语音识别相关方向算法模型，目前主要包括三个模 命令词识别模型 [MultiNet](speech_command_recognition/README_cn.md) ，致力于提供一个灵活的离线语音命词识别框架。用户可方便根据需求自定义语音命令，无需重新训练模型。 -目前模型支持类似“打开空调”，“打开卧室灯”等中文命令词识别，自定义语音命令词最大个数为 100。 - -英文命令词定义将在下一版提供支持。 +目前模型支持类似“打开空调”，“打开卧室灯”等中文命令词识别和"Turn on/off the light" 等英文命令词识别，自定义语音命令词最大个数为 100。 diff --git a/speech_command_recognition/README.md b/speech_command_recognition/README.md index 392187a..192745e 100644 --- a/speech_command_recognition/README.md +++ b/speech_command_recognition/README.md @@ -24,7 +24,9 @@ Please see the flow diagram below: ### User-defined Command -Currently, users can define their own speech commands by using the command `make menuconfig`. You can refer to the method of adding speech commands in `menuconfig->Component config > ESP Speech Recognition->Add speech commands`, there are already 20 commands pre-stored in sdkconfig. +Currently, users can define their own speech commands by using the command `make menuconfig`. You can refer to the method of adding speech commands in `menuconfig->Component config > ESP Speech Recognition->Add speech commands`, there are already 20 chinese commands and 7 english commands pre-stored in sdkconfig. + +**Chinese** |Command ID|Command|Command ID|Command|Command ID|Command|Command ID|Command| |:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:| @@ -33,14 +35,31 @@ Currently, users can define their own speech commands by using the command `make |2|增大风速 (Give me more wind)|7|制冷模式 (Cooling mode)|12| 关闭除湿模式 (Disable dehumidifying mode)|17| 定时一小时 (Set timer to 1 hour) |3|减少风速 (Give me less wind)|8|送风模式 (Ventilating mode)|13| 打开蓝牙 (Enable the Bluetooth)|18| 打开电灯 (Turn on the light) |4| 升高一度 (Increase by one degree)|9|节能模式 (Power-saving mode)|10| 关闭蓝牙 (Disable the Bluetooth)|19| 关闭电灯 (Turn off the light) - + +**English** + +|Command ID|Command|Command ID|Command| +|:---:|:---:|:---:|:---:| +|0|turn on the light|4|red mode| +|1|turn off the light|5|blue mode| +|2|lighting mode|6|yellow mode| +|3|reading mode| + MultiNet supports user-defined commands. You can add your own commands to MultiNet. Note that the newly added command should obtain its command ID before it can be recognized by MultiNet. ### Add Speech Command -Users can define their own speech commands in the `menuconfig` in Pinyin, for example: +Now, the MultiNet model predifine some speech commands. Users also can define their own speech commands and the number of speech commands ID in the `menuconfig -> Component config -> ESP Speech Recognition -> Add speech commands` and `The number of speech commands`. -the command of “打开空调”, which means turn on the air conditioner, should be provided to the blank as "da kai kong tiao". +##### Chinese Speech Command Recognition + +The speech commands should be provided in Pinyin with spaces in between. For example, the command of “打开空调”, which means to turn on the air conditioner, should be provided as "da kai kong tiao". + +##### English Speech Command Recognition + +The speech commands should be provided in specific phonetic symbol with spaces in between. Please use the `general_label_EN/general_label_en.py` script in the tools directory of the skainet root directory to generate the phonetic symbols corresponding to the command words. For details, please refer to [the phonetic symbol generation method](https://github.com/espressif/esp-skainet/tree/master/tools/general_label_EN/README.md). + +**Note:** - One speech commands ID can correspond to multiple speech command phrases; - Up to 100 speech commands ID or speech command phrases, including customized commands, are supported; diff --git a/speech_command_recognition/README_cn.md b/speech_command_recognition/README_cn.md index e4321db..d7d7c4b 100644 --- a/speech_command_recognition/README_cn.md +++ b/speech_command_recognition/README_cn.md @@ -26,7 +26,9 @@ MultiNet 输入为音频经过 **MFCC** 处理后的特征值，输出为汉语/ ### 命令词 -目前，用户可以使用 `make menuconfig` 命令来添加自定义命令词。可以通过 `menuconfig->Component config > ESP Speech Recognition->Add speech commands` 添加命令词，目前已经添加有 20 个命令词，如下表所示： +目前，用户可以使用 `make menuconfig` 命令来添加自定义命令词。可以通过 `menuconfig->Component config > ESP Speech Recognition->Add speech commands` 添加命令词，目前已经添加有 20 个中文命令词和 7 个英文命令词，分别如下表所示： + +**中文** |Command ID|命令词|Command ID|命令词|Command ID|命令词|Command ID|命令词| |:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:| @@ -36,14 +38,29 @@ MultiNet 输入为音频经过 **MFCC** 处理后的特征值，输出为汉语/ |3|减少风速|8|送风模式|13| 打开蓝牙|18| 打开电灯 |4| 升高一度|9|节能模式|10| 关闭蓝牙|19| 关闭电灯 +**英文** + +|Command ID|命令词|Command ID|命令词| +|:---:|:---:|:---:|:---:| +|0|turn on the light|4|red mode| +|1|turn off the light|5|blue mode| +|2|lighting mode|6|yellow mode| +|3|reading mode| + 网络支持自定义命令词，用户可以将自己想要的设置的命令词加入 MultiNet，注意新添加的命令词需要有其的对应 Command ID 已便于 MultiNet 时候后输出。 ### 添加自定义命令词 +目前，MultiNet 模型中已经预定义了一些命令词。用户可以通过 `menuconfig -> Component config -> ESP Speech Recognition -> Add speech commands` and `The number of speech commands`来定义自己的语音命令词和语音命令的数目。 -可以通过 `make menuconfig > Component config > ESP Speech Recognition->Add speech commands` 命令使用拼音添加对应的命令词，比如： +##### 中文命令词识别 -当添加“打开空调”命令时，应该输入 "da kai kong tiao"，请注意： +在填充命令词时应该使用拼音，并且每个字的拼音拼写间要间隔一个空格。比如“打开空调”，应该填入 "da kai kong tiao". +##### 英文命令词识别 + +在填充命令词时应该使用特定音标，请使用 skainet 根目录 `tools` 目录下的 `general_label_EN/general_label_en.py` 脚本生成命令词对应的音标，具体使用方法请参考 [音标生成方法](https://github.com/espressif/esp-skainet/tree/master/tools/general_label_EN/README.md) . + +**注意：** - 一个 Commnad ID 可以对应多个命令短语 - 最多支持 100 个 Command ID 或者 命令短语 - 同一个 Command ID 对应的几条命令短语之间应该由 "," 隔开