speech_recognition/FunASR
1
0
Fork 0
mirror of https://github.com/modelscope/FunASR synced 2025-09-15 14:48:36 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

FunTextProcessing

Browse Source
This commit is contained in:
onlybetheone 2022-12-22 17:06:50 +08:00
parent 96c56e556e
commit 70a23e14cd
1 changed files with 18 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
fun_text_processing/README.md
View File

@ -10,17 +10,32 @@ FunTextProcessing is a Python toolkit for fundamental text processing in ASR inc
- FunTextProcessing supports inverse text processing (ITN), text processing (TN).
- FunTextProcessing supports multilingual, 10+ languages for ITN, 5 languages for TN.
#### Installation
Fun Text Processing, specifically (Inverse) Text Normalization, requires Pynini to be installed.
```
bash fun_text_processing/install_pynini.sh
```
### Example
#### Inverse Text Processing (ITN)
Given text inputs, such as speech recognition results, use `fun_text_processing/inverse_text_normalization/inverse_normalize.py` to output ITN results. You may refer to the following example scripts.
```
test_file=fun_text_processing/inverse_text_normalization/id/id_itn_test_input.txt
python fun_text_processing/inverse_text_normalization/inverse_normalize.py --input_file $test_file --cache_dir ./itn_model/ --output_file output.txt --language=id
python fun_text_processing/inverse_text_normalization/inverse_normalize.py --text="ratus dua puluh tiga" --language=id
```
Arguments:
- text - Input text. Should not exceed 500 words.
- input_file - Input file with lines of input text. Only one of text or input_file is accepted.
- output_file - Output file to save normalizations. Needed if input_file is specified.
- language - language id.
- input_case - Only for text normalization. lower_cased or cased.
- verbose - Outputs intermediate information.
- cache_dir - Specifies a cache directory for compiled grammars. If grammars exist, this significantly improves speed.
- overwrite_cache - Updates grammars in cache.
- whitelist - TSV file with custom mappings of written text to spoken form.
### Acknowledge
1. We borrowed a lot of codes from [NeMo](https://github.com/NVIDIA/NeMo).