Async Japanese Tokenizer Native Plugin for React Native for iOS and Android
npm install react-native-japanese-tokenizerAsync Japanese tokenizer native plugin for React Native that works on iOS and Android.
It tokenizes Japanese text in background thread so it won't block UI. It comes in handy for such as indexing a lot of texts for full-text search.
It utilizes CFStringTokenizer to tokenize strings into words.
The implementation is based on TinySegmenter. It is lightweight word segmenter which works without dictionaries.
``shell`
$ npm install react-native-japanese-tokenizer --save
`shell`
$ react-native link react-native-japanese-tokenizer
#### iOS
1. In XCode, in the project navigator, right click Libraries ➜ Add Files to [your project's name]node_modules
2. Go to ➜ react-native-japanese-tokenizer and add RNJapaneseTokenizer.xcodeprojlibRNJapaneseTokenizer.a
3. In Xcode, in the project navigator, select your project. Add to your project's Build Phases ➜ Link Binary With LibrariesCmd+R
4. Run your project ()
#### Android
1. Open up android/app/src/main/java/[...]/MainActivity.javaimport dog.craftz.japanese_tokenizer.RNJapaneseTokenizerPackage;
- Add to the imports at the top of the filenew RNJapaneseTokenizerPackage()
- Add to the list returned by the getPackages() methodandroid/settings.gradle
2. Append the following lines to :`
`
include ':react-native-japanese-tokenizer'
project(':react-native-japanese-tokenizer').projectDir = new File(rootProject.projectDir, '../node_modules/react-native-japanese-tokenizer/android')
android/app/build.gradle
3. Insert the following lines inside the dependencies block in :`
`
compile project(':react-native-japanese-tokenizer')
Using Promise:
`javascript
import Tokenizer from 'react-native-japanese-tokenizer'
var text = "週休七日で働きたい"
Tokenizer.tokenize(text).then(tokens => {
...
})
`
async/await:
`javascript
import Tokenizer from 'react-native-japanese-tokenizer'
var text = "週休七日で働きたい"
const tokens = await Tokenizer.tokenize(text)
``
Also check out the test to learn more.
MIT License. By Takuya Matsuyama (@craftzdog).
Note that this module is based on following libraries with respective license. Thank you for the fantastic works!
* TinySegmenter: Javascriptだけで実装されたコンパクトな分かち書きソフトウェア
* BSD-3-Clause
* TinySegmenter for Java by takscape
* Public Domain