木守
bd4309ea3c
refactor: adjust parameters for websocket server
2024-09-09 20:31:08 +08:00
zhifu.gzf
0941f8edad
Merge branch dev_gzf_llm2 into dev_gzf_deepspeed
...
Title: wss
本次代码评审主要加入了时间戳打印，用于追踪WebSocket服务器中语音处理的关键步骤时间点，并新增了一个多轮对话处理的WebSocket服务器实现，包括语音合成和模型推理过程，提高了代码的可追溯性和诊断能力。
Link: https://code.alibaba-inc.com/zhifu.gzf/FunASR/codereview/18228062
2024-09-04 15:25:11 +08:00
游雁
b10a1ab523
wss
2024-09-04 12:53:40 +08:00
木守
245ba8fd45
Merge branch 'dev_gzf_deepspeed' of http://gitlab.alibaba-inc.com/zhifu.gzf/FunASR into dev_gzf_deepspeed
2024-09-03 19:45:35 +08:00
木守
281ebc3202
streaming
2024-09-03 19:45:29 +08:00
游雁
ef02bf322e
wss
2024-09-03 14:42:27 +08:00
木守
467056fc2f
streaming
2024-09-03 14:08:05 +08:00
zhifu.gzf
963472437c
feat: Resolve conflict, auto committed by CodeFlow
2024-09-03 13:54:42 +08:00
游雁
a9668ad075
lora
2024-09-03 12:51:16 +08:00
游雁
9f8107ff9e
lora
2024-09-03 11:48:49 +08:00
游雁
29717f4361
lora
2024-09-03 11:23:34 +08:00
游雁
8fb3ce8796
ws
2024-09-02 19:15:59 +08:00
木守
57729f40a6
streaming
2024-09-02 19:03:48 +08:00
木守
013f02e3a3
streaming
2024-09-02 15:49:30 +08:00
木守
561de8db92
streaming
2024-09-02 15:02:31 +08:00
木守
b60b9b6454
streaming
2024-09-02 14:27:09 +08:00
木守
b7f0e6894f
streaming
2024-09-02 14:23:45 +08:00
zhifu.gzf
623fd16f34
Merge branch dev_lr_deepspeed into dev_gzf_deepspeed
...
Title: Add llm tts to client process.
本次代码评审主要增强了 WebSocket 服务端和客户端的语音合成（TTS）功能，添加了语音数据发送、接收处理和计数逻辑，优化了模块结构，引入了`NlsTtsSynthesizer`类来管理语音合成流程，并调整了错误处理和连接管理，使得语音传输更稳定且可追踪。
Link: https://code.alibaba-inc.com/zhifu.gzf/FunASR/codereview/18178626
2024-09-02 10:22:55 +08:00
木守
32657bf651
streaming asr/s2tt
2024-08-29 18:53:24 +08:00
木守
47492abac5
streaming asr/s2tt
2024-08-29 16:05:08 +08:00
jichi.lr
e99201c9b0
Add llm tts to client process.
2024-08-29 11:42:20 +08:00
yangyexin.yyx
038f752e58
streaming
2024-08-28 14:29:04 +08:00
yangyexin.yyx
71b6ecbb39
streaming
2024-08-28 14:20:14 +08:00
yangyexin.yyx
366603d4ed
streaming asr
2024-08-28 09:48:06 +08:00
游雁
032d429a94
wss llm
2024-08-26 17:52:30 +08:00
游雁
260d037d55
wss llm
2024-08-26 15:59:35 +08:00
游雁
f2af56b678
wss llm
2024-08-26 15:00:41 +08:00
游雁
70bdbabcb2
docs
2024-08-22 11:32:22 +08:00
雾聪
e78d649ddb
update readme
2024-06-28 14:28:43 +08:00
雾聪
f170a8e07f
update readme
2024-06-28 11:12:07 +08:00
lingji-yidong
c880db5364
Fix: Return tuple ('', []) when char_list is empty to prevent ValueError ( #1857 )
...
This commit fixes an issue where an empty char_list causes a ValueError due to insufficient values to unpack. The function now returns a tuple ('', []) when char_list is empty.
2024-06-28 01:28:24 +08:00
雾聪
3c50a034b2
update sdk_roadmap.jpg
2024-06-27 17:49:33 +08:00
Yabin Li
d9529818f5
Add files via upload
2024-06-27 17:44:55 +08:00
Yabin Li
5853ebc98f
Merge Dev blade ( #1856 )
...
* update readme
* add benchmark_libtorch_cpp
* add benchmark_libtorch_cpp
* update readme
* update readme
* update readme
* update readme
2024-06-27 17:38:19 +08:00
雾聪
38c1f6393a
add warmup for paraformer-torch
2024-06-26 11:39:19 +08:00
Yabin Li
b7060884fa
Merge Dev tclas ( #1847 )
...
* support clas torchscripts
* fix CompileHotwordEmbedding
* add batch for tensor_hw_emb
* fix func of TimestampOnnx
* fix func of TimestampOnnx
* fix func of TimestampOnnx
* fix paraformer-torch fwd
* fix paraformer-torch fwd
* fix paraformer-torch fwd
* fix ~paraformer-torch
* update funasr-onnx-offline-rtf
* update funasr-onnx-offline-rtf
* update funasr-onnx-offline-rtf
* change tos model names
* fix results of ParaformerTorch::Forward
* fix results of ParaformerTorch::Forward
* add FusionStrategy for torch
* fix paraformer torch
* sync to main (#1826 )
* resume from step
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* train_loss_avg train_acc_avg
* train_loss_avg train_acc_avg
* train_loss_avg train_acc_avg
* log step
* wav is not exist
* wav is not exist
* decoding
* decoding
* decoding
* wechat
* decoding key
* decoding key
* decoding key
* decoding key
* decoding key
* decoding key
* dynamic batch
* start_data_split_i=0
* total_time/accum_grad
* total_time/accum_grad
* total_time/accum_grad
* update avg slice
* update avg slice
* sensevoice sanm
* sensevoice sanm
* add
* add
* add
* add
* deepspeed
* update with main (#1731 )
* c++ runtime adapt to 1.0 (#1724 )
* adapt vad runtime to 1.0
* add json
* change yml name
* add func LoadVocabFromJson
* add token file for InitAsr
* add token path for OfflineStream
* add funcOpenYaml
* add token file for InitPunc
* add token file for stream
* update punc-model
* update funasr-wss-server
* update runtime_sdk_download_tool.py
* update docker list
* Delete docs/images/wechat.png
* Add files via upload
* Emo2Vec限定选择的情感类别 (#1730 )
* 限定选择的情感类别
* 使用none来禁用情感标签输出
* 修改输出接口
* 使用unuse来禁用token
---------
Co-authored-by: 常材 <gaochangfeng.gcf@alibaba-inc.com>
* bugfix
* v1.0.27
* update docs
* hf hub
* Fix incorrect assignment of 'end' attribute to 'start' in sentences list comprehension (#1680 )
---------
Co-authored-by: Yabin Li <wucong.lyb@alibaba-inc.com>
Co-authored-by: gaochangfeng <54253717+gaochangfeng@users.noreply.github.com>
Co-authored-by: 常材 <gaochangfeng.gcf@alibaba-inc.com>
Co-authored-by: nsdou <168500039+nsdou@users.noreply.github.com>
* docs
* docs
* deepspeed
* deepspeed
* deepspeed
* deepspeed
* update
* ds
* ds
* ds
* ds
* ds
* ds
* ds
* add
* add
* bugfix
* add
* wenetspeech
* wenetspeech
* wenetspeech
* wenetspeech
* wenetspeech
* wenetspeech
* update export
* update export
* update export name
* update
* docs
* update wechat QRcode
* Add python funasr api support for websocket srv (#1777 )
* add python funasr_api supoort
* change little to README.md
* add core tools stream
* modified a little
* fix bug for timeout
* support for buffer decode
* add ffmpeg decode for buffer
* libtorch demo
* update libtorch infer
* update utils
* update demo
* update demo
* update libtorch inference
* update model class
* update seaco paraformer
* bug fix
* bug fix
* auto frontend
* auto frontend
* update with main (#1783 )
* add cmakelist
* add paraformer-torch
* add debug for funasr-onnx-offline
* fix redefinition of jieba StdExtension.hpp
* add loading torch models
* update funasr-onnx-offline
* add SwitchArg for wss-server
* add SwitchArg for funasr-onnx-offline
* update cmakelist
* update funasr-onnx-offline-rtf
* add define condition
* add gpu define for offlne-stream
* update com define
* update offline-stream
* update cmakelist
* update func CompileHotwordEmbedding
* add timestamp for paraformer-torch
* add C10_USE_GLOG for paraformer-torch
* update paraformer-torch
* fix func FunASRWfstDecoderInit
* update model.h
* fix func FunASRWfstDecoderInit
* fix tpass_stream
* update paraformer-torch
* add bladedisc for funasr-onnx-offline
* update comdefine
* update funasr-wss-server
* add log for torch
* fix GetValue BLADEDISC
* fix log
* update cmakelist
* update warmup to 10
* update funasrruntime
* add batch_size for wss-server
* add batch for bins
* add batch for offline-stream
* add batch for paraformer
* add batch for offline-stream
* fix func SetBatchSize
* add SetBatchSize for model
* add SetBatchSize for model
* fix func Forward
* fix padding
* update funasrruntime
* add dec reset for batch
* set batch default value
* add argv for CutSplit
* sort frame_queue
* sorted msgs
* fix FunOfflineInfer
* add dynamic batch for fetch
* fix FetchDynamic
* update run_server.sh
* update run_server.sh
* cpp http post server support (#1739 )
* add cpp http server
* add some comment
* remove some comments
* del debug infos
* restore run_server.sh
* adapt to new model struct
* 修复了onnxruntime在macos下编译失败的错误 (#1748 )
* Add files via upload
增加macos的编译支持
* Add files via upload
增加macos支持
* Add files via upload
target_link_directories(funasr PUBLIC ${ONNXRUNTIME_DIR}/lib)
target_link_directories(funasr PUBLIC ${FFMPEG_DIR}/lib)
添加 if(APPLE) 限制
---------
Co-authored-by: Yabin Li <wucong.lyb@alibaba-inc.com>
* Delete docs/images/wechat.png
* Add files via upload
* fixed the issues about seaco-onnx timestamp
* fix bug (#1764 )
当语音识别结果包含 `http` 时，标点符号预测会把它会被当成 url
* fix empty asr result (#1765 )
解码结果为空的语音片段，text 用空字符串
* docs
* docs
* docs
* docs
* docs
* keep empty speech result (#1772 )
* docs
* docs
* update wechat QRcode
* Add python funasr api support for websocket srv (#1777 )
* add python funasr_api supoort
* change little to README.md
* add core tools stream
* modified a little
* fix bug for timeout
* support for buffer decode
* add ffmpeg decode for buffer
* auto frontend
* auto frontend
---------
Co-authored-by: 雾聪 <wucong.lyb@alibaba-inc.com>
Co-authored-by: zhaomingwork <61895407+zhaomingwork@users.noreply.github.com>
Co-authored-by: szsteven008 <97944818+szsteven008@users.noreply.github.com>
Co-authored-by: Ephemeroptera <605686962@qq.com>
Co-authored-by: 彭震东 <zhendong.peng@qq.com>
Co-authored-by: Shi Xian <40013335+R1ckShi@users.noreply.github.com>
Co-authored-by: 维石 <shixian.shi@alibaba-inc.com>
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* Dev gzf exp (#1785 )
* resume from step
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* train_loss_avg train_acc_avg
* train_loss_avg train_acc_avg
* train_loss_avg train_acc_avg
* log step
* wav is not exist
* wav is not exist
* decoding
* decoding
* decoding
* wechat
* decoding key
* decoding key
* decoding key
* decoding key
* decoding key
* decoding key
* dynamic batch
* start_data_split_i=0
* total_time/accum_grad
* total_time/accum_grad
* total_time/accum_grad
* update avg slice
* update avg slice
* sensevoice sanm
* sensevoice sanm
* sensevoice sanm
---------
Co-authored-by: 北念 <lzr265946@alibaba-inc.com>
* auto frontend
* update with main (#1786 )
* add cmakelist
* add paraformer-torch
* add debug for funasr-onnx-offline
* fix redefinition of jieba StdExtension.hpp
* add loading torch models
* update funasr-onnx-offline
* add SwitchArg for wss-server
* add SwitchArg for funasr-onnx-offline
* update cmakelist
* update funasr-onnx-offline-rtf
* add define condition
* add gpu define for offlne-stream
* update com define
* update offline-stream
* update cmakelist
* update func CompileHotwordEmbedding
* add timestamp for paraformer-torch
* add C10_USE_GLOG for paraformer-torch
* update paraformer-torch
* fix func FunASRWfstDecoderInit
* update model.h
* fix func FunASRWfstDecoderInit
* fix tpass_stream
* update paraformer-torch
* add bladedisc for funasr-onnx-offline
* update comdefine
* update funasr-wss-server
* add log for torch
* fix GetValue BLADEDISC
* fix log
* update cmakelist
* update warmup to 10
* update funasrruntime
* add batch_size for wss-server
* add batch for bins
* add batch for offline-stream
* add batch for paraformer
* add batch for offline-stream
* fix func SetBatchSize
* add SetBatchSize for model
* add SetBatchSize for model
* fix func Forward
* fix padding
* update funasrruntime
* add dec reset for batch
* set batch default value
* add argv for CutSplit
* sort frame_queue
* sorted msgs
* fix FunOfflineInfer
* add dynamic batch for fetch
* fix FetchDynamic
* update run_server.sh
* update run_server.sh
* cpp http post server support (#1739 )
* add cpp http server
* add some comment
* remove some comments
* del debug infos
* restore run_server.sh
* adapt to new model struct
* 修复了onnxruntime在macos下编译失败的错误 (#1748 )
* Add files via upload
增加macos的编译支持
* Add files via upload
增加macos支持
* Add files via upload
target_link_directories(funasr PUBLIC ${ONNXRUNTIME_DIR}/lib)
target_link_directories(funasr PUBLIC ${FFMPEG_DIR}/lib)
添加 if(APPLE) 限制
---------
Co-authored-by: Yabin Li <wucong.lyb@alibaba-inc.com>
* Delete docs/images/wechat.png
* Add files via upload
* fixed the issues about seaco-onnx timestamp
* fix bug (#1764 )
当语音识别结果包含 `http` 时，标点符号预测会把它会被当成 url
* fix empty asr result (#1765 )
解码结果为空的语音片段，text 用空字符串
* docs
* docs
* docs
* docs
* docs
* keep empty speech result (#1772 )
* docs
* docs
* update wechat QRcode
* Add python funasr api support for websocket srv (#1777 )
* add python funasr_api supoort
* change little to README.md
* add core tools stream
* modified a little
* fix bug for timeout
* support for buffer decode
* add ffmpeg decode for buffer
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* Dev gzf exp (#1785 )
* resume from step
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* batch
* train_loss_avg train_acc_avg
* train_loss_avg train_acc_avg
* train_loss_avg train_acc_avg
* log step
* wav is not exist
* wav is not exist
* decoding
* decoding
* decoding
* wechat
* decoding key
* decoding key
* decoding key
* decoding key
* decoding key
* decoding key
* dynamic batch
* start_data_split_i=0
* total_time/accum_grad
* total_time/accum_grad
* total_time/accum_grad
* update avg slice
* update avg slice
* sensevoice sanm
* sensevoice sanm
* sensevoice sanm
---------
Co-authored-by: 北念 <lzr265946@alibaba-inc.com>
* auto frontend
---------
Co-authored-by: 雾聪 <wucong.lyb@alibaba-inc.com>
Co-authored-by: zhaomingwork <61895407+zhaomingwork@users.noreply.github.com>
Co-authored-by: szsteven008 <97944818+szsteven008@users.noreply.github.com>
Co-authored-by: Ephemeroptera <605686962@qq.com>
Co-authored-by: 彭震东 <zhendong.peng@qq.com>
Co-authored-by: Shi Xian <40013335+R1ckShi@users.noreply.github.com>
Co-authored-by: 维石 <shixian.shi@alibaba-inc.com>
Co-authored-by: 北念 <lzr265946@alibaba-inc.com>
* update paraformer timestamp
* auto frontend
* auto frontend
* [Optimization] support bladedisc fp16 optimization (#1790 )
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* auto frontend
* add cif_v1 and cif_export
* auto frontend
* Update SDK_advanced_guide_offline_zh.md
* add cif_wo_hidden_v1
* auto frontend
* auto frontend
* auto frontend
* fix bug
* [fix] fix empty asr result (#1794 )
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fix bug
* fp16
* english timestamp for valilla paraformer
* fp16
* wechat
* fixbug
* [fix] better solution for handling empty result (#1796 )
* update scripts
* modify the qformer adaptor (#1804 )
Co-authored-by: nichongjia-2007 <nichongjia@gmail.com>
* add ctc inference code (#1806 )
Co-authored-by: haoneng.lhn <haoneng.lhn@alibaba-inc.com>
* Update auto_model.py
修复空字串进入speaker model时报raw_text变量不存在的bug
* Update auto_model.py
修复识别出空串后spk_model内变量未定义问题
* update model name
* fix paramter 'quantize' unused issue (#1813 )
Co-authored-by: ZihanLiao <liaozihan1@xdf.cn>
* wechat
* Update cif_predictor.py (#1811 )
* Update cif_predictor.py
* modify cif_v1_export
under extreme cases, max_label_len calculated by batch_len misaligns with token_num
* Update cif_predictor.py
torch.cumsum precision degradation, using float64 instead
* update code
---------
Co-authored-by: 游雁 <zhifu.gzf@alibaba-inc.com>
Co-authored-by: gaochangfeng <54253717+gaochangfeng@users.noreply.github.com>
Co-authored-by: 常材 <gaochangfeng.gcf@alibaba-inc.com>
Co-authored-by: nsdou <168500039+nsdou@users.noreply.github.com>
Co-authored-by: 维石 <shixian.shi@alibaba-inc.com>
Co-authored-by: zhaomingwork <61895407+zhaomingwork@users.noreply.github.com>
Co-authored-by: szsteven008 <97944818+szsteven008@users.noreply.github.com>
Co-authored-by: Ephemeroptera <605686962@qq.com>
Co-authored-by: 彭震东 <zhendong.peng@qq.com>
Co-authored-by: Shi Xian <40013335+R1ckShi@users.noreply.github.com>
Co-authored-by: 北念 <lzr265946@alibaba-inc.com>
Co-authored-by: xiaowan0322 <wanchen.swc@alibaba-inc.com>
Co-authored-by: zhuangzhong <zhuangzhong@corp.netease.com>
Co-authored-by: Xingchen Song(宋星辰) <xingchensong1996@163.com>
Co-authored-by: nichongjia-2007 <nichongjia@gmail.com>
Co-authored-by: haoneng.lhn <haoneng.lhn@alibaba-inc.com>
Co-authored-by: liugz18 <57401541+liugz18@users.noreply.github.com>
Co-authored-by: Marlowe <54339989+ZihanLiao@users.noreply.github.com>
Co-authored-by: ZihanLiao <liaozihan1@xdf.cn>
Co-authored-by: zhong zhuang <zhuangz@lamda.nju.edu.cn>
* update runtime_sdk_download_tool
* update funasr-wss-server
* update vad_revision
* update funasr-wss-server
* update funasr-wss-server
* update punc quant
* rename torchscript
* Delete examples/industrial_data_pretraining/ctc/infer_from_local.py
* resolve conflicts
---------
Co-authored-by: 游雁 <zhifu.gzf@alibaba-inc.com>
Co-authored-by: gaochangfeng <54253717+gaochangfeng@users.noreply.github.com>
Co-authored-by: 常材 <gaochangfeng.gcf@alibaba-inc.com>
Co-authored-by: nsdou <168500039+nsdou@users.noreply.github.com>
Co-authored-by: 维石 <shixian.shi@alibaba-inc.com>
Co-authored-by: zhaomingwork <61895407+zhaomingwork@users.noreply.github.com>
Co-authored-by: szsteven008 <97944818+szsteven008@users.noreply.github.com>
Co-authored-by: Ephemeroptera <605686962@qq.com>
Co-authored-by: 彭震东 <zhendong.peng@qq.com>
Co-authored-by: Shi Xian <40013335+R1ckShi@users.noreply.github.com>
Co-authored-by: 北念 <lzr265946@alibaba-inc.com>
Co-authored-by: xiaowan0322 <wanchen.swc@alibaba-inc.com>
Co-authored-by: zhuangzhong <zhuangzhong@corp.netease.com>
Co-authored-by: Xingchen Song(宋星辰) <xingchensong1996@163.com>
Co-authored-by: nichongjia-2007 <nichongjia@gmail.com>
Co-authored-by: haoneng.lhn <haoneng.lhn@alibaba-inc.com>
Co-authored-by: liugz18 <57401541+liugz18@users.noreply.github.com>
Co-authored-by: Marlowe <54339989+ZihanLiao@users.noreply.github.com>
Co-authored-by: ZihanLiao <liaozihan1@xdf.cn>
Co-authored-by: zhong zhuang <zhuangz@lamda.nju.edu.cn>
2024-06-25 17:38:04 +08:00
游雁
1596f6f414
fixbug hotwords
2024-06-24 11:55:17 +08:00
Shi Xian
6c467e6f0a
Merge pull request #1825 from modelscope/dev_libt
...
Dev libt
2024-06-18 10:01:56 +08:00
维石
9377eed41e
update code
2024-06-17 20:20:00 +08:00
Marlowe
6fd83d23ee
fix paramter 'quantize' unused issue ( #1813 )
...
Co-authored-by: ZihanLiao <liaozihan1@xdf.cn>
2024-06-14 10:36:28 +08:00
R1ckShi
0f1247d7a8
update scripts
2024-06-11 14:41:21 +08:00
Yabin Li
84b6c48f55
Update SDK_advanced_guide_offline_zh.md
2024-06-07 14:28:57 +08:00
R1ckShi
22e51ec95f
bug fix
2024-06-03 15:56:39 +08:00
维石
487189b949
bug fix
2024-06-03 15:52:20 +08:00
维石
c0e5107b39
update seaco paraformer
2024-06-03 15:48:36 +08:00
维石
d2e9bf0142
update model class
2024-06-03 15:47:00 +08:00
维石
fd0992af3d
update libtorch inference
2024-06-03 15:32:34 +08:00
维石
c5339e8302
update demo
2024-06-03 15:27:16 +08:00
维石
1c96d9aec2
update demo
2024-06-03 15:20:37 +08:00
维石
02667efa75
update utils
2024-06-03 15:18:13 +08:00
