Question

我有兴趣学习Perl。我正在使用《学习手册》和泛泛网站,供参考。

我期待着使用Perl进行一些网络/文本的报废应用,以应用我所学到的一切。

请允许我提出一些很好的选择。

(这不是家庭工作。) 想在Perl做有助于我利用基本东西的东西。固定特征

Answer 1

If the web pages you want to scrape require JavaScript to function properly, you are going to need more than what WWW::Mechanize can provide you. You might even have to resort to controlling a specific browser via Perl (e.g. using Win32::IE::Mechanize or WWW::Mechanize::Firefox).

我先尝试过,但还有WW:Roder plugin。

Answer 2

正如其他人所说的那样,

http://search.cpan.org/dist/Scrappy”rel=“noreferer”> 同样值得看一看——它让你做许多手法很少——其文件就是一例:


    my $spidy = Scrappy->new;

    $spidy->crawl( http://search.cpan.org/recent , {
         #cpansearch li a  => sub {
            print shift->text, "
";
        }
    });

Scrappy使用在你可能希望把自己看作是另一个选择的情况下。

另外,如果你需要从超文本表格中提取数据,。传真:TableExtract 使这一死亡变得容易——你可以找到你重新感兴趣的桌子,点名标题,并非常容易地提取数据,例如:


    use HTML::TableExtract;
    $te = HTML::TableExtract->new( headers => [qw(Date Price Cost)] );
    $te->parse($html_string) or die "Didn t find table";
    foreach $row ($te->rows) {
        print join( , , @$row), "
";
    }

Answer 3

The most popular web scraping module for Perl is WWW::Mechanize, which is excellent if you can t just retrieve your destination page but need to navigate to it using links or forms, for instance, to log in. Have a look at its documentation for inspiration. If your needs are simple, you can extract the information you need from the HTML using regular expressions (but beware your sanity), otherwise it might be better to use a module such as HTML::TreeBuilder to do the job.

一种似乎有趣的单元,但我没有经过真正的尝试,是。它是WWWWWWWWW的一个子类:机械化,但支持Javascript和AJAX,并融入了

Answer 4

D. 访问网络 Perl模块。 http://teusje.wordpress.com/ 201005/02/web-scraping-with-perl/"rel=“nofollow”>beginners tutorial can be found here 。

It s safe, easy to use and fast.

Answer 5

你或许也想看一下我对贾瓦·赫特姆勒大学的新任的佩尔勒总结。非常容易使用,例如,在这里看看快速辅导:

By tomorrow I will publish some detailed installation instructions and a first release. Unlike Mechanize and alike you get some JavaScript support and it is way faster and less memory demanding than screen scraping.

友情链接