mirror of
https://github.com/espressif/esp-sr.git
synced 2025-09-15 15:28:44 +08:00
241 lines
10 KiB
ReStructuredText
241 lines
10 KiB
ReStructuredText
性能测试结果
|
||
==============
|
||
|
||
:link_to_translation:`en:[English]`
|
||
|
||
AFE
|
||
---
|
||
|
||
资源消耗
|
||
~~~~~~~~
|
||
|
||
.. only:: esp32
|
||
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
| Algorithm Type | RAM | Average cpu | Frame Length |
|
||
| | | loading(compute | |
|
||
| | | with 2 cores) | |
|
||
+=================+=================+=================+=================+
|
||
| AEC(HIGH_PERF) | 114 KB | 11% | 32 ms |
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
| NS | 27 KB | 5% | 10 ms |
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
| AFE Layer | 73 KB | | |
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
|
||
.. only:: esp32s3
|
||
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
| Algorithm Type | RAM | Average cpu | Frame Length |
|
||
| | | loading(compute | |
|
||
| | | with 2 cores) | |
|
||
+=================+=================+=================+=================+
|
||
| AEC(LOW_COST) | 152.3 KB | 8% | 32 ms |
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
| AEC(HIGH_PERF) | 166 KB | 11% | 32 ms |
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
| BSS(LOW_COST) | 198.7 KB | 6% | 64 ms |
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
| BSS(HIGH_PERF) | 215.5 KB | 7% | 64 ms |
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
| NS(NS_MODE_SSP) | 27 KB | 5% | 10 ms |
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
| NS(nsnet1) | 885 KB | 25% | 16 ms |
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
| NS(nsnet2) | 375 KB | 12% | 32 ms |
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
| MISO | 56 KB | 8% | 16 ms |
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
| AFE Layer | 227 KB | | |
|
||
+-----------------+-----------------+-----------------+-----------------+
|
||
|
||
|
||
WakeNet
|
||
-------
|
||
|
||
.. _resource-occupancyesp32-1:
|
||
|
||
资源消耗
|
||
~~~~~~~~
|
||
|
||
.. only:: esp32
|
||
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
| Model Type | Parameter | RAM | Average | Frame |
|
||
| | Num | | Running | Length |
|
||
| | | | Time per | |
|
||
| | | | Frame | |
|
||
+=============+=============+=============+=============+=============+
|
||
| Quantised | 41 K | 15 KB | 5.5 ms | 30 ms |
|
||
| WakeNet5 | | | | |
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
| Quantised | 165 K | 20 KB | 10.5 ms | 30 ms |
|
||
| WakeNet5X2 | | | | |
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
| Quantised | 371 K | 24 KB | 18 ms | 30 ms |
|
||
| WakeNet5X3 | | | | |
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
|
||
.. _resource-occupancyesp32s3-1:
|
||
|
||
.. only:: esp32s3
|
||
|
||
+----------------+-------+---------+----------------+--------------+
|
||
| Model Type | RAM | PSRAM | Average | Frame Length |
|
||
| | | | Running Time | |
|
||
| | | | per Frame | |
|
||
+================+=======+=========+================+==============+
|
||
| Quantised | 50 KB | 1640 KB | 10.0 ms | 32 ms |
|
||
| WakeNet8 @ 2 | | | | |
|
||
| channel | | | | |
|
||
+----------------+-------+---------+----------------+--------------+
|
||
| Quantised | 16 KB | 324 KB | 3.0 ms | 32 ms |
|
||
| WakeNet9 @ 2 | | | | |
|
||
| channel | | | | |
|
||
+----------------+-------+---------+----------------+--------------+
|
||
| Quantised | 20 KB | 347 KB | 4.3 ms | 32 ms |
|
||
| WakeNet9 @ 3 | | | | |
|
||
| channel | | | | |
|
||
+----------------+-------+---------+----------------+--------------+
|
||
|
||
性能测试
|
||
~~~~~~~~
|
||
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
| Distance | Quiet | Stationary | Speech | AEC |
|
||
| | | Noise (SNR | Noise (SNR | I |
|
||
| | | = 4 dB) | = 4 dB) | nterruption |
|
||
| | | | | (-10 dB) |
|
||
+=============+=============+=============+=============+=============+
|
||
| 1 m | 98% | 96% | 94% | 96% |
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
| 3 m | 98% | 96% | 94% | 94% |
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
|
||
误触发率：12 小时 1 次
|
||
|
||
.. note::
|
||
|
||
我们在测试中使用了 ESP32-S3-Korvo V4.0 开发板和 WakeNet9(Alexa) 模型。
|
||
|
||
MultiNet
|
||
--------
|
||
|
||
.. _resource-occupancyesp32-2:
|
||
|
||
资源消耗
|
||
~~~~~~~~
|
||
|
||
.. only:: esp32
|
||
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
| Model Type | Internal | PSRAM | Average | Frame |
|
||
| | RAM | | Running | Length |
|
||
| | | | Time per | |
|
||
| | | | Frame | |
|
||
+=============+=============+=============+=============+=============+
|
||
| MultiNet 2 | 13.3 KB | 9KB | 38 ms | 30 ms |
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
|
||
.. _resource-occupancyesp32s3-2:
|
||
|
||
.. only:: esp32s3
|
||
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
| Model Type | Internal | PSRAM | Average | Frame |
|
||
| | RAM | | Running | Length |
|
||
| | | | Time per | |
|
||
| | | | Frame | |
|
||
+=============+=============+=============+=============+=============+
|
||
| MultiNet 4 | 16.8KB | 1866 KB | 18 ms | 32 ms |
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
| MultiNet 4 | 10.5 KB | 1009 KB | 11 ms | 32 ms |
|
||
| Q8 | | | | |
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
| MultiNet 5 | 16 KB | 2310 KB | 12 ms | 32 ms |
|
||
| Q8 | | | | |
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
| MultiNet 6 | 32 KB | 4100 KB | 12 ms | 32 ms |
|
||
+-------------+-------------+-------------+-------------+-------------+
|
||
|
||
Word Error Rate 性能测试
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
+-----------+-----------+
|
||
| Model | aishell |
|
||
| Type | test |
|
||
+===========+===========+
|
||
| MultiNet | 9.5% |
|
||
| 5_cn | |
|
||
+-----------+-----------+
|
||
| MultiNet | 5.2% |
|
||
| 6_cn | |
|
||
+-----------+-----------+
|
||
|
||
.. note::
|
||
|
||
中文使用没有声调的拼音单元去计算WER。
|
||
|
||
Speech Commands 性能测试(空调控制场景)
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
+-----------+-----------+----------+------------+-------------+
|
||
| Model | Distance | Quiet | Stationary | Speech |
|
||
| Type | | | Noise | Noise |
|
||
| | | | (SNR=5~10dB| (SNR=5~10dB |
|
||
| | | | dB) | dB) |
|
||
+===========+===========+==========+============+=============+
|
||
| MultiNet | 3 m | 88.9% | 66.1% | 67.5% |
|
||
| 5_cn | | | | |
|
||
+-----------+-----------+----------+------------+-------------+
|
||
| MultiNet | 3 m | 98.8% | 88.3% | 88.0% |
|
||
| 6_cn | | | | |
|
||
+-----------+-----------+----------+------------+-------------+
|
||
| MultiNet | 3 m | 97.1% | 95.1% | 96.8% |
|
||
| 6_cn_ac | | | | |
|
||
+-----------+-----------+----------+------------+-------------+
|
||
|
||
.. note::
|
||
|
||
MultiNet6_cn_ac在空调场景数据集上进行了进一步的微调，所以在空调控制场景具有更好的性能。
|
||
|
||
|
||
TTS
|
||
---
|
||
|
||
资源消耗
|
||
~~~~~~~~
|
||
|
||
Flash image size: 2.2 MB
|
||
|
||
RAM runtime: 20 KB
|
||
|
||
|
||
性能测试
|
||
~~~~~~~~
|
||
|
||
CPU 负载测试（ESP32 @240 MHz）：
|
||
|
||
+------------------------------+------+------+------+------+------+------+
|
||
| Speech Rate | 0 | 1 | 2 | 3 | 4 | 5 |
|
||
+==============================+======+======+======+======+======+======+
|
||
| Times faster than real time | 4.5 | 3.2 | 2.9 | 2.5 | 2.2 | 1.8 |
|
||
+------------------------------+------+------+------+------+------+------+
|
||
|
||
|
||
NSNET
|
||
-----
|
||
|
||
性能测试
|
||
~~~~~~~~
|
||
数据集：array_onemic_nnoise_20230608(按照亚马逊声学认证标准录制测试集)
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
+------------------+--------+
|
||
| | dnsmos |
|
||
+==================+========+
|
||
| nsnet1 | 2.4 |
|
||
+------------------+--------+
|
||
| nsnet2 | 2.71 |
|
||
+------------------+--------+
|