性能测试结果
==============

:link_to_translation:`en:[English]`

AFE
---

资源消耗
~~~~~~~~

.. only:: esp32

    +-----------------+-----------------+-----------------+-----------------+
    | Algorithm Type  | RAM             | Average cpu     | Frame Length    |
    |                 |                 | loading(compute |                 |
    |                 |                 | with 2 cores)   |                 |
    +=================+=================+=================+=================+
    | AEC(HIGH_PERF)  | 114 KB          | 11%             | 32 ms           |
    +-----------------+-----------------+-----------------+-----------------+
    | NS              | 27 KB           | 5%              | 10 ms           |
    +-----------------+-----------------+-----------------+-----------------+
    | AFE Layer       | 73 KB           |                 |                 |
    +-----------------+-----------------+-----------------+-----------------+

.. only:: esp32s3

    +-----------------+-----------------+-----------------+-----------------+
    | Algorithm Type  | RAM             | Average cpu     | Frame Length    |
    |                 |                 | loading(compute |                 |
    |                 |                 | with 2 cores)   |                 |
    +=================+=================+=================+=================+
    | AEC(LOW_COST)   | 152.3 KB        | 8%              | 32 ms           |
    +-----------------+-----------------+-----------------+-----------------+
    | AEC(HIGH_PERF)  | 166 KB          | 11%             | 32 ms           |
    +-----------------+-----------------+-----------------+-----------------+
    | BSS(LOW_COST)   | 198.7 KB        | 6%              | 64 ms           |
    +-----------------+-----------------+-----------------+-----------------+
    | BSS(HIGH_PERF)  | 215.5 KB        | 7%              | 64 ms           |
    +-----------------+-----------------+-----------------+-----------------+
    | NS(NS_MODE_SSP) | 27 KB           | 5%              | 10 ms           |
    +-----------------+-----------------+-----------------+-----------------+
    | NS(nsnet1)      | 885 KB          | 25%             | 16 ms           |
    +-----------------+-----------------+-----------------+-----------------+
    | NS(nsnet2)      | 375 KB          | 12%             | 32 ms           |
    +-----------------+-----------------+-----------------+-----------------+
    | MISO            | 56 KB           | 8%              | 16 ms           |
    +-----------------+-----------------+-----------------+-----------------+
    | AFE Layer       | 227 KB          |                 |                 |
    +-----------------+-----------------+-----------------+-----------------+


WakeNet
-------

.. _resource-occupancyesp32-1:

资源消耗
~~~~~~~~

.. only:: esp32

    +-------------+-------------+-------------+-------------+-------------+
    | Model Type  | Parameter   | RAM         | Average     | Frame       |
    |             | Num         |             | Running     | Length      |
    |             |             |             | Time per    |             |
    |             |             |             | Frame       |             |
    +=============+=============+=============+=============+=============+
    | Quantised   | 41 K        | 15 KB       | 5.5 ms      | 30 ms       |
    | WakeNet5    |             |             |             |             |
    +-------------+-------------+-------------+-------------+-------------+
    | Quantised   | 165 K       | 20 KB       | 10.5 ms     | 30 ms       |
    | WakeNet5X2  |             |             |             |             |
    +-------------+-------------+-------------+-------------+-------------+
    | Quantised   | 371 K       | 24 KB       | 18 ms       | 30 ms       |
    | WakeNet5X3  |             |             |             |             |
    +-------------+-------------+-------------+-------------+-------------+

.. _resource-occupancyesp32s3-1:

.. only:: esp32s3

    +----------------+-------+---------+----------------+--------------+
    | Model Type     | RAM   | PSRAM   | Average        | Frame Length |
    |                |       |         | Running Time   |              |
    |                |       |         | per Frame      |              |
    +================+=======+=========+================+==============+
    | Quantised      | 50 KB | 1640 KB | 10.0 ms        | 32 ms        |
    | WakeNet8 @ 2   |       |         |                |              |
    | channel        |       |         |                |              |
    +----------------+-------+---------+----------------+--------------+
    | Quantised      | 16 KB | 324 KB  | 3.0 ms         | 32 ms        |
    | WakeNet9 @ 2   |       |         |                |              |
    | channel        |       |         |                |              |
    +----------------+-------+---------+----------------+--------------+
    | Quantised      | 20 KB | 347 KB  | 4.3 ms         | 32 ms        |
    | WakeNet9 @ 3   |       |         |                |              |
    | channel        |       |         |                |              |
    +----------------+-------+---------+----------------+--------------+

性能测试
~~~~~~~~

+-------------+-------------+-------------+-------------+-------------+
| Distance    | Quiet       | Stationary  | Speech      | AEC         |
|             |             | Noise (SNR  | Noise (SNR  | I           |
|             |             | = 4 dB)     | = 4 dB)     | nterruption |
|             |             |             |             | (-10 dB)    |
+=============+=============+=============+=============+=============+
| 1 m         | 98%         | 96%         | 94%         | 96%         |
+-------------+-------------+-------------+-------------+-------------+
| 3 m         | 98%         | 96%         | 94%         | 94%         |
+-------------+-------------+-------------+-------------+-------------+

误触发率：12 小时 1 次

.. note::

    我们在测试中使用了 ESP32-S3-Korvo V4.0 开发板和 WakeNet9(Alexa) 模型。

MultiNet
--------

.. _resource-occupancyesp32-2:

资源消耗
~~~~~~~~

.. only:: esp32

    +-------------+-------------+-------------+-------------+-------------+
    | Model Type  | Internal    | PSRAM       | Average     | Frame       |
    |             | RAM         |             | Running     | Length      |
    |             |             |             | Time per    |             |
    |             |             |             | Frame       |             |
    +=============+=============+=============+=============+=============+
    | MultiNet 2  | 13.3 KB     | 9KB         | 38 ms       | 30 ms       |
    +-------------+-------------+-------------+-------------+-------------+

.. _resource-occupancyesp32s3-2:

.. only:: esp32s3

    +-------------+-------------+-------------+-------------+-------------+
    | Model Type  | Internal    | PSRAM       | Average     | Frame       |
    |             | RAM         |             | Running     | Length      |
    |             |             |             | Time per    |             |
    |             |             |             | Frame       |             |
    +=============+=============+=============+=============+=============+
    | MultiNet 4  | 16.8KB      | 1866 KB     | 18 ms       | 32 ms       |
    +-------------+-------------+-------------+-------------+-------------+
    | MultiNet 4  | 10.5 KB     | 1009 KB     | 11 ms       | 32 ms       |
    | Q8          |             |             |             |             |
    +-------------+-------------+-------------+-------------+-------------+
    | MultiNet 5  | 16 KB       | 2310 KB     | 12 ms       | 32 ms       |
    | Q8          |             |             |             |             |
    +-------------+-------------+-------------+-------------+-------------+
    | MultiNet 6  | 32 KB       | 4100 KB     | 12 ms       | 32 ms       |
    +-------------+-------------+-------------+-------------+-------------+

Word Error Rate 性能测试
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

+-----------+-----------+
| Model     | aishell   |
| Type      | test      |
+===========+===========+
| MultiNet  |    9.5%   |
| 5_cn      |           |
+-----------+-----------+
| MultiNet  |    5.2%   |
| 6_cn      |           |
+-----------+-----------+

.. note::

    中文使用没有声调的拼音单元去计算WER。

Speech Commands 性能测试(空调控制场景)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

+-----------+-----------+----------+------------+-------------+
| Model     | Distance  | Quiet    | Stationary | Speech      |
| Type      |           |          | Noise      | Noise       |
|           |           |          | (SNR=5~10dB| (SNR=5~10dB |
|           |           |          | dB)        | dB)         |
+===========+===========+==========+============+=============+
| MultiNet  | 3 m       |  88.9%   |   66.1%    |   67.5%     |
| 5_cn      |           |          |            |             |
+-----------+-----------+----------+------------+-------------+
| MultiNet  | 3 m       |  98.8%   |   88.3%    |   88.0%     |
| 6_cn      |           |          |            |             |
+-----------+-----------+----------+------------+-------------+
| MultiNet  | 3 m       |  97.1%   |   95.1%    |   96.8%     |
| 6_cn_ac   |           |          |            |             |
+-----------+-----------+----------+------------+-------------+

.. note::

    MultiNet6_cn_ac在空调场景数据集上进行了进一步的微调，所以在空调控制场景具有更好的性能。


TTS
---

资源消耗
~~~~~~~~

Flash image size: 2.2 MB

RAM runtime: 20 KB


性能测试
~~~~~~~~

CPU 负载测试（ESP32 @240 MHz）：

+------------------------------+------+------+------+------+------+------+
| Speech Rate                  | 0    | 1    | 2    | 3    | 4    | 5    |
+==============================+======+======+======+======+======+======+
| Times faster than real time  | 4.5  | 3.2  | 2.9  | 2.5  | 2.2  | 1.8  |
+------------------------------+------+------+------+------+------+------+


NSNET
-----

性能测试
~~~~~~~~
数据集：array_onemic_nnoise_20230608(按照亚马逊声学认证标准录制测试集)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

+------------------+--------+
|                  | dnsmos |
+==================+========+
| nsnet1           | 2.4    |
+------------------+--------+ 
| nsnet2           | 2.71   |
+------------------+--------+