如何使用Lucene Analyzer对字符串进行标记?
有没有一种简单的方法可以使用Lucene的任何子类来解析/标记化一个?Analyzer
String
像这样:
String to_be_parsed = "car window seven";
Analyzer analyzer = new StandardAnalyzer(...);
List<String> tokenized_string = analyzer.analyze(to_be_parsed);
有没有一种简单的方法可以使用Lucene的任何子类来解析/标记化一个?Analyzer
String
像这样:
String to_be_parsed = "car window seven";
Analyzer analyzer = new StandardAnalyzer(...);
List<String> tokenized_string = analyzer.analyze(to_be_parsed);
根据上面的答案,这稍微修改了一下,可以与Lucene 4.0一起使用。
public final class LuceneUtil {
private LuceneUtil() {}
public static List<String> tokenizeString(Analyzer analyzer, String string) {
List<String> result = new ArrayList<String>();
try {
TokenStream stream = analyzer.tokenStream(null, new StringReader(string));
stream.reset();
while (stream.incrementToken()) {
result.add(stream.getAttribute(CharTermAttribute.class).toString());
}
} catch (IOException e) {
// not thrown b/c we're using a string reader...
throw new RuntimeException(e);
}
return result;
}
}
据我所知,你必须自己编写循环。像这样的东西(直接取自我的源树):
public final class LuceneUtils {
public static List<String> parseKeywords(Analyzer analyzer, String field, String keywords) {
List<String> result = new ArrayList<String>();
TokenStream stream = analyzer.tokenStream(field, new StringReader(keywords));
try {
while(stream.incrementToken()) {
result.add(stream.getAttribute(TermAttribute.class).term());
}
}
catch(IOException e) {
// not thrown b/c we're using a string reader...
}
return result;
}
}