speech_recognition/FunASR
1
0
Fork 0
mirror of https://github.com/modelscope/FunASR synced 2025-09-15 14:48:36 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #519 from alibaba-damo-academy/main

Browse Source 
update with main
This commit is contained in:
zhifu gao 2023-05-17 00:05:31 +08:00 committed by GitHub
commit 7e63de52f0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
17 changed files with 2203 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
README.md
View File

@ -75,8 +75,8 @@ If you have any questions about FunASR, please contact us by
## Contributors
| <div align="left"><img src="docs/images/damo.png" width="180"/> | <div align="left"><img src="docs/images/nwpu.png" width="260"/> | <img src="docs/images/China_Telecom.png" width="200"/> </div> | <img src="docs/images/RapidAI.png" width="200"/> </div> | <img src="docs/images/DeepScience.png" width="200"/> </div> |
|:---------------------------------------------------------------:|:---------------------------------------------------------------:|:--------------------------------------------------------------:|:-------------------------------------------------------:|:-----------------------------------------------------------:|
| <div align="left"><img src="docs/images/damo.png" width="180"/> | <div align="left"><img src="docs/images/nwpu.png" width="260"/> | <img src="docs/images/China_Telecom.png" width="200"/> </div> | <img src="docs/images/RapidAI.png" width="200"/> </div> | <img src="docs/images/DeepScience.png" width="200"/> </div> | <img src="docs/images/aihealthx.png" width="200"/> </div> |
|:---------------------------------------------------------------:|:---------------------------------------------------------------:|:--------------------------------------------------------------:|:-------------------------------------------------------:|:-----------------------------------------------------------:|:-----------------------------------------------------------:|
## Acknowledge
@ -86,6 +86,7 @@ If you have any questions about FunASR, please contact us by
4. We acknowledge [ChinaTelecom](https://github.com/zhuzizyf/damo-fsmn-vad-infer-httpserver) for contributing the VAD runtime.
5. We acknowledge [RapidAI](https://github.com/RapidAI) for contributing the Paraformer and CT_Transformer-punc runtime.
6. We acknowledge [DeepScience](https://www.deepscience.cn) for contributing the grpc service.
6. We acknowledge [AiHealthx](http://www.aihealthx.com/) for contributing the websocket service and html5.
## License
This project is licensed under the [The MIT License](https://opensource.org/licenses/MIT). FunASR also contains various third-party components and some code modified from other repos under other open source licenses.

BIN
docs/images/aihealthx.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB

BIN
funasr/runtime/html5/demo.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 228 KiB

65
funasr/runtime/html5/h5Server.py Normal file
View File

@ -0,0 +1,65 @@
# -*- coding: utf-8 -*-
###
### Copyright FunASR (https://github.com/alibaba-damo-academy/FunASR). All Rights
### Reserved. MIT License (https://opensource.org/licenses/MIT)
###
### 2022-2023 by zhaoming,mali aihealthx.com
from flask import Flask,render_template,request,send_from_directory,jsonify
#from gevent.pywsgi import WSGIServer
import datetime
import random
import string
import time
import argparse
app = Flask(__name__,static_folder='static',static_url_path="/static")
@app.route('/')
def homePage():
return render_template('recorderapp_test.html')
parser = argparse.ArgumentParser()
parser.add_argument("--host",
type=str,
default="0.0.0.0",
required=False,
help="host ip, localhost, 0.0.0.0")
parser.add_argument("--port",
type=int,
default=1337,
required=False,
help="html5 server port")
parser.add_argument("--certfile",
type=str,
default="server.crt",
required=False,
help="certfile for ssl")
parser.add_argument("--keyfile",
type=str,
default="server.key",
required=False,
help="keyfile for ssl")
if __name__ == '__main__':
args = parser.parse_args()
port=args.port
#WSGIServer
#ssl = {
# 'certfile': 'server.crt',
# 'keyfile': 'server.key'
#}
#httpsServer = WSGIServer(("0.0.0.0",port), app, **ssl)
#httpsServer.serve_forever()
#flask
print("srv run on ",port)
app.run(debug=True,host=args.host,port=port, ssl_context=(args.certfile,args.keyfile))

109
funasr/runtime/html5/readme.md Normal file
View File

@ -0,0 +1,109 @@
# online asr demo for html5
## requirement
### python
```shell
flask
gevent
pyOpenSSL
```
### javascript
[html5录音](https://github.com/xiangyuecn/Recorder)
```shell
Recorder
```
### demo页面如下
![img](https://github.com/alibaba-damo-academy/FunASR/blob/for-html5-demo/funasr/runtime/html5/demo.gif)
## 两种ws_server_online连接模式
### 1)直接连接模式，浏览器https麦克风 --> html5 demo服务 --> js wss接口 --> wss asr online srv(证书生成请往后看)
### 2)nginx中转，浏览器https麦克风 --> html5 demo服务 --> js wss接口 --> nginx服务 --> ws asr online srv
## 1.html5 demo服务启动
### 启动html5服务，需要ssl证书(自己生成请往后看)
```shell
usage: h5Server.py [-h] [--host HOST] [--port PORT] [--certfile CERTFILE]
[--keyfile KEYFILE]
python h5Server.py --port 1337
```
## 2.启动ws or wss asr online srv
[具体请看online asr](https://github.com/alibaba-damo-academy/FunASR/tree/main/funasr/runtime/python/websocket)
online asr提供两种ws和wss模式，wss模式可以直接启动，无需nginx中转。否则需要通过nginx将wss转发到该online asr的ws端口上
### wss方式
```shell
python ws_server_online.py --certfile server.crt --keyfile server.key --port 5921
```
### ws方式
```shell
python ws_server_online.py --port 5921
```
## 3.修改wsconnecter.js里asr接口地址
wsconnecter.js里配置online asr服务地址路径，这里配置的是wss端口
var Uri = "wss://xxx:xxx/"
## 4.浏览器打开地址测试
https://127.0.0.1:1337/static/index.html
## 自行生成证书
生成证书(注意这种证书并不能被所有浏览器认可，部分手动授权可以访问,最好使用其他认证的官方ssl证书)
```shell
### 1)生成私钥，按照提示填写内容
openssl genrsa -des3 -out server.key 1024
### 2)生成csr文件 ，按照提示填写内容
openssl req -new -key server.key -out server.csr
### 去掉pass
cp server.key server.key.org
openssl rsa -in server.key.org -out server.key
### 生成crt文件，有效期1年（365天）
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
```
## nginx配置说明(了解的可以跳过)
h5打开麦克风需要https协议，同时后端的asr websocket也必须是wss协议，如果[online asr](https://github.com/alibaba-damo-academy/FunASR/tree/main/funasr/runtime/python/websocket)以ws方式运行，我们可以通过nginx配置实现wss协议到ws协议的转换。
### nginx转发配置示例
```shell
events { [0/1548]
worker_connections 1024;
accept_mutex on;
}
http {
error_log error.log;
access_log access.log;
server {
listen 5921 ssl http2; # nginx listen port for wss
server_name www.test.com;
ssl_certificate /funasr/server.crt;
ssl_certificate_key /funasr/server.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5;
location /wss/ {
proxy_pass http://127.0.0.1:1111/; # asr online model ws address and port
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 600s;
}
}
```
## Acknowledge
1. This project is maintained by [FunASR community](https://github.com/alibaba-damo-academy/FunASR).
2. We acknowledge [AiHealthx](http://www.aihealthx.com/) for contributing the html5 demo.

3
funasr/runtime/html5/requirement.txt Normal file
View File

@ -0,0 +1,3 @@
flask
gevent
pyOpenSSL

15
funasr/runtime/html5/server.crt Normal file
View File

@ -0,0 +1,15 @@
-----BEGIN CERTIFICATE-----
MIICSDCCAbECFCObiVAMkMlCGmMDGDFx5Nx3XYvOMA0GCSqGSIb3DQEBCwUAMGMx
CzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMRAwDgYDVQQHDAdCZWlqaW5n
MRAwDgYDVQQKDAdhbGliYWJhMQwwCgYDVQQLDANhc3IxEDAOBgNVBAMMB2FsaWJh
YmEwHhcNMjMwNTEyMTQzNjAxWhcNMjQwNTExMTQzNjAxWjBjMQswCQYDVQQGEwJD
TjEQMA4GA1UECAwHQmVpamluZzEQMA4GA1UEBwwHQmVpamluZzEQMA4GA1UECgwH
YWxpYmFiYTEMMAoGA1UECwwDYXNyMRAwDgYDVQQDDAdhbGliYWJhMIGfMA0GCSqG
SIb3DQEBAQUAA4GNADCBiQKBgQDEINLLMasJtJQPoesCfcwJsjiUkx3hLnoUyETS
NBrrRfjbBv6ucAgZIF+/V15IfJZR6u2ULpJN0wUg8xNQReu4kdpjSdNGuQ0aoWbc
38+VLo9UjjsoOeoeCro6b0u+GosPoEuI4t7Ky09zw+FBibD95daJ3GDY1DGCbDdL
mV/toQIDAQABMA0GCSqGSIb3DQEBCwUAA4GBAB5KNWF1XIIYD1geMsyT6/ZRnGNA
dmeUyMcwYvIlQG3boSipNk/JI4W5fFOg1O2sAqflYHmwZfmasAQsC2e5bSzHZ+PB
uMJhKYxfj81p175GumHTw5Lbp2CvFSLrnuVB0ThRdcCqEh1MDt0D3QBuBr/ZKgGS
hXtozVCgkSJzX6uD
-----END CERTIFICATE-----

15
funasr/runtime/html5/server.key Normal file
View File

@ -0,0 +1,15 @@
-----BEGIN RSA PRIVATE KEY-----
MIICXQIBAAKBgQDEINLLMasJtJQPoesCfcwJsjiUkx3hLnoUyETSNBrrRfjbBv6u
cAgZIF+/V15IfJZR6u2ULpJN0wUg8xNQReu4kdpjSdNGuQ0aoWbc38+VLo9Ujjso
OeoeCro6b0u+GosPoEuI4t7Ky09zw+FBibD95daJ3GDY1DGCbDdLmV/toQIDAQAB
AoGARpA0pwygp+ZDWvh7kDLoZRitCK+BkZHiNHX1ZNeAU+Oh7FOw79u43ilqqXHq
pxPEFYb7oVO8Kanhb4BlE32EmApBlvhd3SW07kn0dS7WVGsTvPFwKKpF88W8E+pc
2i8At5tr2O1DZhvqNdIN7r8FRrGQ/Hpm3ItypUdz2lZnMwECQQD3dILOMJ84O2JE
NxUwk8iOYefMJftQUO57Gm7XBVke/i3r9uajSqB2xmOvUaSyaHoJfx/mmfgfxYcD
M+Re6mERAkEAyuaV5+eD82eG2I8PgxJ2p5SOb1x5F5qpb4KuKAlfHEkdolttMwN3
7vl1ZWUZLVu2rHnUmvbYV2gkQO1os7/DkQJBAIDYfbN2xbC12vjB5ZqhmG/qspMt
w6mSOlqG7OewtTLaDncq2/RySxMNQaJr1GHA3KpNMwMTcIq6gw472tFBIMECQF0z
fjiASEROkcp4LI/ws0BXJPZSa+1DxgDK7mTFqUK88zfY91gvh6/mNt7UibQkJM0l
SVvFd6ru03hflXC77YECQQDDQrB9ApwVOMGQw+pwbxn9p8tPYVi3oBiUfYgd1RDO
uhcRgxv7gT4BSiyI4nFBMCYyI28azTLlUiJhMr9MNUpB
-----END RSA PRIVATE KEY-----

43
funasr/runtime/html5/static/index.html Normal file
View File

@ -0,0 +1,43 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width,initial-scale=1" />
<title>语音识别</title>
</head>
<body style="margin-left: 3%">
<script src="recorder-core.js" charset="UTF-8"></script>
<script src="wav.js" charset="UTF-8"></script>
<script src="pcm.js" charset="UTF-8"></script>
<div class="div_class_topArea">
<div class="div_class_recordControl">
语音识别结果显示：
<br>
<textarea rows="10" id="varArea" readonly="true" style=" width: 100%;height:100%" ></textarea>
<br>
<div id="info_div">请点击开始</div>
<div class="div_class_buttons">
<button id="btnStart">开始</button>
<button id="btnStop">停止</button>
</div>
<audio id="audio_record" type="audio/wav" controls style="margin-top: 12px; width: 100%;"></audio>
</div>
</div>
<script src="wsconnecter.js" charset="utf-8"></script>
<script src="main.js" charset="utf-8"></script>
</body>
</html>

167
funasr/runtime/html5/static/main.js Normal file
View File

@ -0,0 +1,167 @@
/**
* Copyright FunASR (https://github.com/alibaba-damo-academy/FunASR). All Rights
* Reserved. MIT License (https://opensource.org/licenses/MIT)
*/
/* 2022-2023 by zhaoming,mali aihealthx.com */
// 连接; 定义socket连接类对象与语音对象
var wsconnecter = new WebSocketConnectMethod({msgHandle:getJsonMessage,stateHandle:getConnState});
var audioBlob;
// 录音; 定义录音对象,wav格式
var rec = Recorder({
type:"pcm",
bitRate:16,
sampleRate:16000,
onProcess:recProcess
});
var sampleBuf=new Int16Array();
// 定义按钮响应事件
var btnStart = document.getElementById('btnStart');
btnStart.onclick = start;
var btnStop = document.getElementById('btnStop');
btnStop.onclick = stop;
btnStop.disabled = true;
var rec_text=""
var info_div = document.getElementById('info_div');
// 语音识别结果; 对jsonMsg数据解析,将识别结果附加到编辑框中
function getJsonMessage( jsonMsg ) {
console.log( "message: " + JSON.parse(jsonMsg.data)['text'] );
var rectxt=""+JSON.parse(jsonMsg.data)['text'];
var varArea=document.getElementById('varArea');
rec_text=rec_text+rectxt.replace(/ +/g,"");
varArea.value=rec_text;
}
// 连接状态响应
function getConnState( connState ) {
if ( connState === 0 ) {
rec.open( function(){
rec.start();
console.log("开始录音");
});
} else if ( connState === 1 ) {
//stop();
} else if ( connState === 2 ) {
stop();
console.log( 'connecttion error' );
setTimeout(function(){btnStart.disabled = true;info_div.innerHTML='connecttion error';}, 4000 );
}
}
// 识别启动、停止、清空操作
function start() {
info_div.innerHTML="正在连接asr服务器，请等待...";
// 清除显示
clear();
//控件状态更新
isRec = true;
btnStart.disabled = true;
btnStop.disabled = false;
//启动连接
wsconnecter.wsStart();
}
function stop() {
var chunk_size = new Array( 5, 10, 5 );
var request = {
"chunk_size": chunk_size,
"wav_name": "h5",
"is_speaking": false,
"chunk_interval":10,
};
if(sampleBuf.length>0){
wsconnecter.wsSend(sampleBuf,false);
console.log("sampleBuf.length"+sampleBuf.length);
sampleBuf=new Int16Array();
}
wsconnecter.wsSend( JSON.stringify(request) ,false);
// 控件状态更新
isRec = false;
info_div.innerHTML="请等候...";
btnStop.disabled = true;
setTimeout(function(){btnStart.disabled = false;info_div.innerHTML="请点击开始";}, 3000 );
rec.stop(function(blob,duration){
console.log(blob);
var audioBlob = Recorder.pcm2wav(data = {sampleRate:16000, bitRate:16, blob:blob},
function(theblob,duration){
console.log(theblob);
var audio_record = document.getElementById('audio_record');
audio_record.src = (window.URL||webkitURL).createObjectURL(theblob);
audio_record.controls=true;
audio_record.play();
} ,function(msg){
console.log(msg);
}
);
},function(errMsg){
console.log("errMsg: " + errMsg);
});
// 停止连接
}
function clear() {
var varArea=document.getElementById('varArea');
varArea.value="";
rec_text="";
}
function recProcess( buffer, powerLevel, bufferDuration, bufferSampleRate,newBufferIdx,asyncEnd ) {
if ( isRec === true ) {
var data_48k = buffer[buffer.length-1];
var array_48k = new Array(data_48k);
var data_16k=Recorder.SampleData(array_48k,bufferSampleRate,16000).data;
sampleBuf = Int16Array.from([...sampleBuf, ...data_16k]);
var chunk_size=960; // for asr chunk_size [5, 10, 5]
info_div.innerHTML=""+bufferDuration/1000+"s";
while(sampleBuf.length>=chunk_size){
sendBuf=sampleBuf.slice(0,chunk_size);
sampleBuf=sampleBuf.slice(chunk_size,sampleBuf.length);
wsconnecter.wsSend(sendBuf,false);
}
}
}

96
funasr/runtime/html5/static/pcm.js Normal file
View File

@ -0,0 +1,96 @@
/*
pcm编码器+编码引擎
https://github.com/xiangyuecn/Recorder
编码原理：本编码器输出的pcm格式数据其实就是Recorder中的buffers原始数据（经过了重新采样），16位时为LE小端模式（Little Endian），并未经过任何编码处理
编码的代码和wav.js区别不大，pcm加上一个44字节wav头即成wav文件；所以要播放pcm就很简单了，直接转成wav文件来播放，已提供转换函数 Recorder.pcm2wav
*/
(function(){
"use strict";
Recorder.prototype.enc_pcm={
stable:true
,testmsg:"pcm为未封装的原始音频数据，pcm数据文件无法直接播放；支持位数8位、16位（填在比特率里面），采样率取值无限制"
};
Recorder.prototype.pcm=function(res,True,False){
var This=this,set=This.set
,size=res.length
,bitRate=set.bitRate==8?8:16;
var buffer=new ArrayBuffer(size*(bitRate/8));
var data=new DataView(buffer);
var offset=0;
// 写入采样数据
if(bitRate==8) {
for(var i=0;i<size;i++,offset++) {
//16转8据说是雷霄骅的 https://blog.csdn.net/sevennight1989/article/details/85376149 细节比blqw的按比例的算法清晰点，虽然都有明显杂音
var val=(res[i]>>8)+128;
data.setInt8(offset,val,true);
};
}else{
for (var i=0;i<size;i++,offset+=2){
data.setInt16(offset,res[i],true);
};
};
True(new Blob([data.buffer],{type:"audio/pcm"}));
};
/**pcm直接转码成wav，可以直接用来播放；需同时引入wav.js
data: {
sampleRate:16000 pcm的采样率
bitRate:16 pcm的位数 取值：8 或 16
blob:blob对象
}
data如果直接提供的blob将默认使用16位16khz的配置，仅用于测试
True(wavBlob,duration)
False(msg)
**/
Recorder.pcm2wav=function(data,True,False){
if(data.slice && data.type!=null){//Blob 测试用
data={blob:data};
};
var sampleRate=data.sampleRate||16000,bitRate=data.bitRate||16;
if(!data.sampleRate || !data.bitRate){
console.warn("pcm2wav必须提供sampleRate和bitRate");
};
if(!Recorder.prototype.wav){
False("pcm2wav必须先加载wav编码器wav.js");
return;
};
var reader=new FileReader();
reader.onloadend=function(){
var pcm;
if(bitRate==8){
//8位转成16位
var u8arr=new Uint8Array(reader.result);
pcm=new Int16Array(u8arr.length);
for(var j=0;j<u8arr.length;j++){
pcm[j]=(u8arr[j]-128)<<8;
};
}else{
pcm=new Int16Array(reader.result);
};
Recorder({
type:"wav"
,sampleRate:sampleRate
,bitRate:bitRate
}).mock(pcm,sampleRate).stop(function(wavBlob,duration){
True(wavBlob,duration);
},False);
};
reader.readAsArrayBuffer(data.blob);
};
})();

1493
funasr/runtime/html5/static/recorder-core.js Normal file
View File

File diff suppressed because it is too large Load Diff

86
funasr/runtime/html5/static/wav.js Normal file
View File

@ -0,0 +1,86 @@
/*
wav编码器+编码引擎
https://github.com/xiangyuecn/Recorder
当然最佳推荐使用mp3、wav格式，代码也是优先照顾这两种格式
浏览器支持情况
https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats
编码原理：给pcm数据加上一个44直接的wav头即成wav文件；pcm数据就是Recorder中的buffers原始数据（重新采样），16位时为LE小端模式（Little Endian），实质上是未经过任何编码处理
*/
(function(){
"use strict";
Recorder.prototype.enc_wav={
stable:true
,testmsg:"支持位数8位、16位（填在比特率里面），采样率取值无限制"
};
Recorder.prototype.wav=function(res,True,False){
var This=this,set=This.set
,size=res.length
,sampleRate=set.sampleRate
,bitRate=set.bitRate==8?8:16;
//编码数据 https://github.com/mattdiamond/Recorderjs https://www.cnblogs.com/blqw/p/3782420.html https://www.cnblogs.com/xiaoqi/p/6993912.html
var dataLength=size*(bitRate/8);
var buffer=new ArrayBuffer(44+dataLength);
var data=new DataView(buffer);
var offset=0;
var writeString=function(str){
for (var i=0;i<str.length;i++,offset++) {
data.setUint8(offset,str.charCodeAt(i));
};
};
var write16=function(v){
data.setUint16(offset,v,true);
offset+=2;
};
var write32=function(v){
data.setUint32(offset,v,true);
offset+=4;
};
/* RIFF identifier */
writeString('RIFF');
/* RIFF chunk length */
write32(36+dataLength);
/* RIFF type */
writeString('WAVE');
/* format chunk identifier */
writeString('fmt ');
/* format chunk length */
write32(16);
/* sample format (raw) */
write16(1);
/* channel count */
write16(1);
/* sample rate */
write32(sampleRate);
/* byte rate (sample rate * block align) */
write32(sampleRate*(bitRate/8));// *1 声道
/* block align (channel count * bytes per sample) */
write16(bitRate/8);// *1 声道
/* bits per sample */
write16(bitRate);
/* data chunk identifier */
writeString('data');
/* data chunk length */
write32(dataLength);
// 写入采样数据
if(bitRate==8) {
for(var i=0;i<size;i++,offset++) {
//16转8据说是雷霄骅的 https://blog.csdn.net/sevennight1989/article/details/85376149 细节比blqw的按比例的算法清晰点，虽然都有明显杂音
var val=(res[i]>>8)+128;
data.setInt8(offset,val,true);
};
}else{
for (var i=0;i<size;i++,offset+=2){
data.setInt16(offset,res[i],true);
};
};
True(new Blob([data.buffer],{type:"audio/wav"}));
}
})();

81
funasr/runtime/html5/static/wsconnecter.js Normal file
View File

@ -0,0 +1,81 @@
/**
* Copyright FunASR (https://github.com/alibaba-damo-academy/FunASR). All Rights
* Reserved. MIT License (https://opensource.org/licenses/MIT)
*/
/* 2021-2023 by zhaoming,mali aihealthx.com */
function WebSocketConnectMethod( config ) { //定义socket连接方法类
var Uri = "wss://111.205.137.58:5821/wss/" //设置wss asr online接口地址 如 wss://X.X.X.X:port/wss/
var speechSokt;
var connKeeperID;
var msgHandle = config.msgHandle;
var stateHandle = config.stateHandle;
this.wsStart = function () {
if ( 'WebSocket' in window ) {
speechSokt = new WebSocket( Uri ); // 定义socket连接对象
speechSokt.onopen = function(e){onOpen(e);}; // 定义响应函数
speechSokt.onclose = function(e){onClose(e);};
speechSokt.onmessage = function(e){onMessage(e);};
speechSokt.onerror = function(e){onError(e);};
}
else {
alert('当前浏览器不支持 WebSocket');
}
};
// 定义停止与发送函数
this.wsStop = function () {
if(speechSokt != undefined) {
speechSokt.close();
}
};
this.wsSend = function ( oneData,stop ) {
if(speechSokt == undefined) return;
if ( speechSokt.readyState === 1 ) { // 0:CONNECTING, 1:OPEN, 2:CLOSING, 3:CLOSED
speechSokt.send( oneData );
if(stop){
setTimeout(speechSokt.close(), 3000 );
}
}
};
// SOCEKT连接中的消息与状态响应
function onOpen( e ) {
// 发送json
var chunk_size = new Array( 5, 10, 5 );
var request = {
"chunk_size": chunk_size,
"wav_name": "h5",
"is_speaking": true,
"chunk_interval":10,
};
speechSokt.send( JSON.stringify(request) );
console.log("连接成功");
stateHandle(0);
}
function onClose( e ) {
stateHandle(1);
}
function onMessage( e ) {
msgHandle( e );
}
function onError( e ) {
info_div.innerHTML="连接"+e;
console.log(e);
stateHandle(2);
}
}

2
funasr/runtime/onnxruntime/src/e2e-vad.h
View File

@ -306,7 +306,7 @@ public:
max_end_sil_frame_cnt_thresh = max_end_sil - vad_opts.speech_to_sil_time_thres;
this->waveform = waveform;
this->vad_opts.max_single_segment_time = max_single_segment_time;
this->vad_opts.speech_noise_thres = speech_noise_thres;
this->speech_noise_thres = speech_noise_thres;
this->vad_opts.sample_rate = sample_rate;
ComputeDecibel();

12
funasr/runtime/python/websocket/parse_args.py
View File

@ -35,6 +35,16 @@ parser.add_argument("--ncpu",
type=int,
default=1,
help="cpu cores")
parser.add_argument("--certfile",
type=str,
default="",
required=False,
help="certfile for ssl")
parser.add_argument("--keyfile",
type=str,
default="",
required=False,
help="keyfile for ssl")
args = parser.parse_args()
print(args)

18
funasr/runtime/python/websocket/ws_server_online.py
View File

@ -7,7 +7,7 @@ import threading
import logging
import tracemalloc
import numpy as np
import ssl
from parse_args import args
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
@ -53,6 +53,9 @@ async def ws_serve(websocket, path):
if "is_speaking" in messagejson:
websocket.is_speaking = messagejson["is_speaking"]
websocket.param_dict_asr_online["is_final"] = not websocket.is_speaking
# need to fire engine manually if no data received any more
if not websocket.is_speaking:
await async_asr_online(websocket,b"")
if "chunk_interval" in messagejson:
websocket.chunk_interval=messagejson["chunk_interval"]
if "wav_name" in messagejson:
@ -82,7 +85,7 @@ async def ws_serve(websocket, path):
async def async_asr_online(websocket,audio_in):
if len(audio_in) > 0:
if len(audio_in) >=0:
audio_in = load_bytes(audio_in)
rec_result = inference_pipeline_asr_online(audio_in=audio_in,
param_dict=websocket.param_dict_asr_online)
@ -94,7 +97,16 @@ async def async_asr_online(websocket,audio_in):
await websocket.send(message)
if len(args.certfile)>0:
ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
start_server = websockets.serve(ws_serve, args.host, args.port, subprotocols=["binary"], ping_interval=None)
# Generate with Lets Encrypt, copied to this location, chown to current user and 400 permissions
ssl_cert = args.certfile
ssl_key = args.keyfile
ssl_context.load_cert_chain(ssl_cert, keyfile=ssl_key)
start_server = websockets.serve(ws_serve, args.host, args.port, subprotocols=["binary"], ping_interval=None,ssl=ssl_context)
else:
start_server = websockets.serve(ws_serve, args.host, args.port, subprotocols=["binary"], ping_interval=None)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()