我发现最好的一段代码用于提取文本，而不需要javascript或不需要的东西：import urllibfrom bs4 import BeautifulSoupurl = "http://news.bbc.co.uk/2/hi/health/2284783.stm"html = urllib.urlopen(url).read()soup = BeautifulSoup(html)# kill all script and style elementsfor script in soup(["script", "style"]):
script.extract() # rip it out# get texttext = soup.get_text()# break into lines and remove leading and trailing space on eachlines = (line.strip() for line in text.splitlines())# break multi-headlines into a line eachchunks = (phrase.strip() for line in lines for phrase in line.split(" "))# drop blank linestext = '\n'.join(chunk for chunk in chunks if chunk)print(text)
你只需先安装BeautifulSoup：pip install beautifulsoup4
