Question

It s difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.

Closed 11 years ago.

Hi evry one i need to crun a court by using python (my python edition is 2.6.6) only using urls.

first i need to open this link(http://www.google.com/) and crawl,and i should collect all http links which are present in this page;
second, from the collected http links,i should go to each and every link, through this link i need to open and do the crawling process and should save the crawled links in a separate folder. iam just trying in this way,if any one is know the solution plz reply i need your help..

Answer 1

页: 1

步骤2:使用lxml或Beautiful Soup , 引自你的网页上的所有链接。

步骤3:现在再次通过这些链接与红利或红利,并节省你的网页。

也可提供 /Scrapy或Mechanize 。

这里是使用<条码>xpath表述和 lxml的一个例子。您可使用<条码>射线/代码>获取任何成分的xpath。

import lxml
from lxml import etree
import urllib2

response = urllib2.urlopen( your url )
html = response.read()
link = etree.HTML(html)
links = link.xpath( xpath expr )

友情链接