使用Python从文本文件中删除停止词

print('====THIS IS HOW TO REMOVE STOP WORS====')

with open('one.txt','r')as myFile:

    str1=myFile.read()

    stop_words ="not", "is", "it", "By","between","This","By","A","when","And","up","Then","was","by","It","If","can","an","he","This","or","And","a","i","it","am","at","on","in","of","to","is","so","too","my","the","and","but","are","very","here","even","from","them","then","than","this","that","though","be","But","these"

    myList=[]

    myList.extend(str1.split(" "))

    for i in myList:

        if i not in stop_words:

            print ("____________")

            print(i,end='\n')

这里有一个没有正则表达式的解决方案。

import string

input_text = "!where??and!!or$$then:)"
punctuation_replacer = string.maketrans(string.punctuation, ' '*len(string.punctuation))    
print ' '.join(input_text.translate(punctuation_replacer).split()).strip()

Output>> where and or then

用空格替换标点用单个空格替换单词之间的多个空格删除尾随空格（如果有）条带（）

myString.translate(None, string.punctuation)

这可能不是最好的解决方案，但我就是这样做的。

import string
f = lambda x: ''.join([i for i in x if i not in string.punctuation])

这是我写的一个函数。它不是很有效，但很简单，您可以添加或删除任何您想要的标点符号：

def stripPunc(wordList):
    """Strips punctuation from list of words"""
    puncList = [".",";",":","!","?","/","\\",",","#","@","$","&",")","(","\""]
    for punc in puncList:
        for word in wordList:
            wordList=[word.replace(punc,'') for word in wordList]
    return wordList

字符串标点符号漏掉了现实世界中常用的大量标点符号。一个适用于非ASCII标点符号的解决方案怎么样？

import regex
s = u"string. With. Some・Really Weird、Non？ASCII。 「（Punctuation）」?"
remove = regex.compile(ur'[\p{C}|\p{M}|\p{P}|\p{S}|\p{Z}]+', regex.UNICODE)
remove.sub(u" ", s).strip()

我个人认为，这是在Python中删除字符串标点符号的最佳方法，因为：

它删除所有Unicode标点符号它很容易修改，例如，如果您想删除标点符号，可以删除\｛s｝，但保留像$这样的符号。您可以非常具体地了解要保留的内容和要删除的内容，例如，Pd只会删除破折号。此正则表达式还规范了空白。它将制表符、回车符和其他奇怪的字符映射到漂亮的单个空格。

这使用了Unicode字符财产，您可以在Wikipedia上阅读更多有关该属性的信息。