from mechanize import Browser
br = Browser()
br.open('http://somewebpage')
html = br.response().readlines()
for line in html:
print line
当在HTML文件中打印一行时，我试图找到一种方法，只显示每个HTML元素的内容，而不是格式本身。如果它发现'<a href="等等。例如">some text</a>'，它只会打印'some text'， '<b>hello</b>'打印'hello'，等等。该怎么做呢?
当前回答
我正在解析Github自述，我发现下面的工作真的很好:
import re
import lxml.html
def strip_markdown(x):
links_sub = re.sub(r'\[(.+)\]\([^\)]+\)', r'\1', x)
bold_sub = re.sub(r'\*\*([^*]+)\*\*', r'\1', links_sub)
emph_sub = re.sub(r'\*([^*]+)\*', r'\1', bold_sub)
return emph_sub
def strip_html(x):
return lxml.html.fromstring(x).text_content() if x else ''
然后
readme = """<img src="https://raw.githubusercontent.com/kootenpv/sky/master/resources/skylogo.png" />
sky is a web scraping framework, implemented with the latest python versions in mind (3.4+).
It uses the asynchronous `asyncio` framework, as well as many popular modules
and extensions.
Most importantly, it aims for **next generation** web crawling where machine intelligence
is used to speed up the development/maintainance/reliability of crawling.
It mainly does this by considering the user to be interested in content
from *domains*, not just a collection of *single pages*
([templating approach](#templating-approach))."""
strip_markdown(strip_html(readme))
正确移除所有markdown和html。
其他回答
你可以编写自己的函数:
def StripTags(text):
finished = 0
while not finished:
finished = 1
start = text.find("<")
if start >= 0:
stop = text[start:].find(">")
if stop >= 0:
text = text[:start] + text[start+stop+1:]
finished = 0
return text
我就是这么做的，但我不知道我在做什么。我通过剥离HTML标记从HTML表中获取数据。
它接受字符串“name”并返回不带HTML标记的字符串“name1”。
x = 0
anglebrackets = 0
name1 = ""
while x < len(name):
if name[x] == "<":
anglebrackets = anglebrackets + 1
if name[x] == ">":
anglebrackets = anglebrackets - 1
if anglebrackets == 0:
if name[x] != ">":
name1 = name1 + name[x]
x = x + 1
您可以使用不同的HTML解析器(如lxml或Beautiful Soup)——它提供只提取文本的函数。或者，您可以在行字符串上运行一个regex来删除标记。请参阅Python文档了解更多信息。
# This is a regex solution.
import re
def removeHtml(html):
if not html: return html
# Remove comments first
innerText = re.compile('<!--[\s\S]*?-->').sub('',html)
while innerText.find('>')>=0: # Loop through nested Tags
text = re.compile('<[^<>]+?>').sub('',innerText)
if text == innerText:
break
innerText = text
return innerText.strip()
如果你想去掉所有HTML标签，我发现最简单的方法是使用BeautifulSoup:
from bs4 import BeautifulSoup # Or from BeautifulSoup import BeautifulSoup
def stripHtmlTags(htmlTxt):
if htmlTxt is None:
return None
else:
return ''.join(BeautifulSoup(htmlTxt).findAll(text=True))
我尝试了接受的答案的代码，但我得到了“RuntimeError:最大递归深度超出”，这没有发生在上面的代码块。
