Question

我有数千个同义词的清单。我还有数万个文件,我想搜索这些术语。使用python(或假编码)什么是有效的方法?

# this would work for single word synonyms, but there are multiple word synonyms too
synonymSet = set([...])
wordsInDocument = set([...])
synonymsInDocument = synonymSet.intersection(wordsInDocument)

# this would work, but sounds slow
matches = []
for document in documents:
    for synonym in synonymSet:
        if synonym in document:
            matches.append(synonym)

Is there a good solution to this problem, or will it just take a while? Thank you in advance

Answer 1

从您的同义词列表中建立正则表达式如何 :

import re

pattern = "|".join(synonymList)
regex = re.compile(pattern)

matches = regex.findall(document) # get a list of the matched synonyms
matchedSynonyms = set(matches)    # eliminate duplicates using a set

友情链接