# NAME

Web::Query - Yet another scraping library like jQuery

# VERSION

version 0.33

# SYNOPSIS

    use Web::Query;

    wq('http://www.w3.org/TR/html401/')
        ->find('div.head dt')
        ->each(sub {
            my $i = shift;
            printf("%d %s\n", $i+1, $_->text);
        });

# DESCRIPTION

Web::Query is a yet another scraping framework, have a jQuery like interface.

Yes, I know Ingy's [pQuery](https://metacpan.org/pod/pQuery). But it's just a alpha quality. It doesn't works.
Web::Query built at top of the CPAN modules, [HTML::TreeBuilder::XPath](https://metacpan.org/pod/HTML::TreeBuilder::XPath), [LWP::UserAgent](https://metacpan.org/pod/LWP::UserAgent), and [HTML::Selector::XPath](https://metacpan.org/pod/HTML::Selector::XPath).

So, this module uses [HTML::Selector::XPath](https://metacpan.org/pod/HTML::Selector::XPath) and only supports the CSS 3
selector supported by that module.
Web::Query doesn't support jQuery's extended queries(yet?). If a selector is 
passed as a scalar ref, it'll be taken as a straight XPath expression.

    $wq( '<div><p>hello</p><p>there</p></div>' )->find( 'p' );       # css selector
    $wq( '<div><p>hello</p><p>there</p></div>' )->find( \'/div/p' ); # xpath selector

**THIS LIBRARY IS UNDER DEVELOPMENT. ANY API MAY CHANGE WITHOUT NOTICE**.

# FUNCTIONS

- `wq($stuff)`

    This is a shortcut for `Web::Query->new($stuff)`. This function is exported by default.

# METHODS

## CONSTRUCTORS

- my $q = Web::Query->new($stuff, \\%options )

    Create new instance of Web::Query. You can make the instance from URL(http, https, file scheme), HTML in string, URL in string, [URI](https://metacpan.org/pod/URI) object, and instance of [HTML::Element](https://metacpan.org/pod/HTML::Element).

    This method throw the exception on unknown $stuff.

    This method returns undefined value on non-successful response with URL.

    Currently, the only two valid options are _indent_, which will be used as
    the indentation string if the object is printed, and _no\_space\_compacting_, 
    which will prevent the compaction of whitespace characters in text blocks.

- my $q = Web::Query->new\_from\_element($element: HTML::Element)

    Create new instance of Web::Query from instance of [HTML::Element](https://metacpan.org/pod/HTML::Element).

- `my $q = Web::Query->new_from_html($html: Str)`

    Create new instance of Web::Query from HTML.

- my $q = Web::Query->new\_from\_url($url: Str)

    Create new instance of Web::Query from URL.

    If the response is not success(It means /^20\[0-9\]$/), this method returns undefined value.

    You can get a last result of response, use the `$Web::Query::RESPONSE`.

    Here is a best practical code:

        my $url = 'http://example.com/';
        my $q = Web::Query->new_from_url($url)
            or die "Cannot get a resource from $url: " . Web::Query->last_response()->status_line;

- my $q = Web::Query->new\_from\_file($file\_name: Str)

    Create new instance of Web::Query from file name.

## TRAVERSING

### add

Returns a new object augmented with the new element(s).

- add($html)

    An HTML fragment to add to the set of matched elements.

- add(@elements)

    One or more @elements to add to the set of matched elements. 

    @elements that already are part of the set are not added a second time.

        my $group = $wq->find('#foo');         # collection has 1 element
        $group = $group->add( '#bar', $wq );   # 2 elements
        $group->add( '#foo', $wq );            # still 2 elements

- add($wq)

    An existing Web::Query object to add to the set of matched elements.

- add($selector, $context)

    $selector is a string representing a selector expression to find additional elements to add to the set of matched elements.

    $context is the point in the document at which the selector should begin matching

### contents

Get the immediate children of each element in the set of matched elements, including text and comment nodes.

### each

Visit each nodes. `$i` is a counter value, 0 origin. `$elem` is iteration item.
`$_` is localized by `$elem`.

    $q->each(sub { my ($i, $elem) = @_; ... })

### end

Back to the before context like jQuery.

### filter

Reduce the elements to those that pass the function's test.

    $q->filter(sub { my ($i, $elem) = @_; ... })

### find

Get the descendants of each element in the current set of matched elements, filtered by a selector.

    my $q2 = $q->find($selector); # $selector is a CSS3 selector.

**NOTE** If you want to match the element itself, use ["filter"](#filter).

**INCOMPATIBLE CHANGE** 
From v0.14 to v0.19 (inclusive) find() also matched the element itself, which is not jQuery compatible.
You can achieve that result using `filter()`, `add()` and `find()`:

    my $wq = wq('<div class="foo"><p class="foo">bar</p></div>'); # needed because we don't have a global document like jQuery does
    print $wq->filter('.foo')->add($wq->find('.foo'))->as_html; # <div class="foo"><p class="foo">bar</p></div><p class="foo">bar</p>

### first

Return the first matching element.

This method constructs a new Web::Query object from the first matching element.

### last

Return the last matching element.

This method constructs a new Web::Query object from the last matching element.

### not($selector)

Return all the elements not matching the `$selector`.

    # $do_for_love will be every thing, except #that
    my $do_for_love = $wq->find('thing')->not('#that');

### and\_back

Add the previous set of elements to the current one.

    # get the h1 plus everything until the next h1
    $wq->find('h1')->next_until('h1')->and_back;

### map

Creates a new array with the results of calling a provided function on every element.

    $q->map(sub { my ($i, $elem) = @_; ... })

### parent

Get the parent of each element in the current set of matched elements.

### prev

Get the previous node of each element in the current set of matched elements.

    my $prev = $q->prev;

### next

Get the next node of each element in the current set of matched elements.

    my $next = $q->next;

### next\_until( $selector )

Get all subsequent siblings, up to (but not including) the next node matched `$selector`.

## MANIPULATION

### add\_class

Adds the specified class(es) to each of the set of matched elements.

    # add class 'foo' to <p> elements
    wq('<div><p>foo</p><p>bar</p></div>')->find('p')->add_class('foo'); 

### toggle\_class( @classes )

Toggles the given class or classes on each of the element. I.e., if the element had the class, it'll be removed,
and if it hadn't, it'll be added.

Classes are toggled once, no matter how many times they appear in the argument list.

    $q->toggle_class( 'foo', 'foo', 'bar' );

    # equivalent to
    
    $q->toggle_class('foo')->toggle_class('bar');

    # and not

    $q->toggle_class('foo')->toggle_class('foo')->toggle_class('bar');

### after

Insert content, specified by the parameter, after each element in the set of matched elements.

    wq('<div><p>foo</p></div>')->find('p')
                               ->after('<b>bar</b>')
                               ->end
                               ->as_html; # <div><p>foo</p><b>bar</b></div>

The content can be anything accepted by ["new"](#new).

### append

Insert content, specified by the parameter, to the end of each element in the set of matched elements.

    wq('<div></div>')->append('<p>foo</p>')->as_html; # <div><p>foo</p></div>

The content can be anything accepted by ["new"](#new).

### as\_html

Return the elements associated with the object as strings. 
If called in a scalar context, only return the string representation
of the first element.

### ` attr `

Get/set the attribute value in element.

In getter mode, it'll return either the values of the attribute
for all elements of the set, or only the first one depending of the calling context.

    my @values = $q->attr('style');      # style of all elements
    my $first_value = $q->attr('style'); # style of first element

In setter mode, it'll set the attribute value for all elements, and return back
the original object for easy chaining.

    $q->attr( 'alt' => 'a picture' )->find( ... );

### ` id `

Get/set the elements's id attribute.

In getter mode, it behaves just like `attr()`.

In setter mode, it behaves like `attr()`, but with the following exceptions.

If the attribute value is a scalar, it'll be only assigned to
the first element of the set (as ids are supposed to be unique), and the returned object will only contain
that first element.

    my $first_element = $q->id('the_one');

It's possible to set the ids of all the elements by passing a sub to `id()`. The sub is given the same arguments as for
`each()`, and its return value is taken to be the new id of the elements.

    $q->id( sub { my $i = shift;  'foo_' . $i } );

### ` name `

Get/set the elements's 'name' attribute.

    my $name = $q->name;  # equivalent to $q->attr( 'name' );

    $q->name( 'foo' );    # equivalent to $q->attr( name => 'foo' );

### ` data `

Get/set the elements's 'data-\*name\*' attributes.

    my $data = $q->data('foo');  # equivalent to $q->attr( 'data-foo' );

    $q->data( 'foo' => 'bar' );  # equivalent to $q->attr( 'data-foo' => 'bar' );

### tagname

Get/Set the tag name of elements.

    my $name = $q->tagname;

    $q->tagname($new_name);

### before

Insert content, specified by the parameter, before each element in the set of matched elements.

    wq('<div><p>foo</p></div>')->find('p')
                               ->before('<b>bar</b>')
                               ->end
                               ->as_html; # <div><b>bar</b><p>foo</p></div>

The content can be anything accepted by ["new"](#new).

### clone

Create a deep copy of the set of matched elements.

### detach

Remove the set of matched elements from the DOM.

### has\_class

Determine whether any of the matched elements are assigned the given class.

### ` html `

Get/Set the innerHTML.

    my @html = $q->html();

    my $html = $q->html(); # 1st matching element only

    $q->html('<p>foo</p>');

### insert\_before

Insert every element in the set of matched elements before the target.

### insert\_after

Insert every element in the set of matched elements after the target.

### ` prepend `

Insert content, specified by the parameter, to the beginning of each element in the set of matched elements. 

### remove

Delete the elements associated with the object from the DOM.

    # remove all <blink> tags from the document
    $q->find('blink')->remove;

### remove\_class

Remove a single class, multiple classes, or all classes from each element in the set of matched elements.

### replace\_with

Replace the elements of the object with the provided replacement. 
The replacement can be a string, a `Web::Query` object or an 
anonymous function. The anonymous function is passed the index of the current 
node and the node itself (with is also localized as `$_`).

    my $q = wq( '<p><b>Abra</b><i>cada</i><u>bra</u></p>' );

    $q->find('b')->replace_with('<a>Ocus</a>);
        # <p><a>Ocus</a><i>cada</i><u>bra</u></p>

    $q->find('u')->replace_with($q->find('b'));
        # <p><i>cada</i><b>Abra</b></p>

    $q->find('i')->replace_with(sub{ 
        my $name = $_->text;
        return "<$name></$name>";
    });
        # <p><b>Abra</b><cada></cada><u>bra</u></p>

### size

Return the number of elements in the Web::Query object.

    wq('<div><p>foo</p><p>bar</p></div>')->find('p')->size; # 2

### text

Get/Set the text.

    my @text = $q->text();

    my $text = $q->text(); # 1st matching element only

    $q->text('text');

If called in a scalar context, only return the string representation
of the first element

## OTHERS

- Web::Query->last\_response()

    Returns last HTTP response status that generated by `new_from_url()`.

# HOW DO I CUSTOMIZE USER AGENT?

You can specify your own instance of [LWP::UserAgent](https://metacpan.org/pod/LWP::UserAgent).

    $Web::Query::UserAgent = LWP::UserAgent->new( agent => 'Mozilla/5.0' );

# INCOMPATIBLE CHANGES

- 0.10

    new\_from\_url() is no longer throws exception on bad response from HTTP server.

# AUTHOR

Tokuhiro Matsuno <tokuhirom AAJKLFJEF@ GMAIL COM>

# SEE ALSO

[pQuery](https://metacpan.org/pod/pQuery)

# LICENSE

Copyright (C) Tokuhiro Matsuno

This library is free software; you can redistribute it and/or modify
it under the same terms as Perl itself.

# BUGS

Please report any bugs or feature requests on the bugtracker website
https://github.com/tokuhirom/Web-Query/issues

When submitting a bug or request, please include a test-file or a
patch to an existing test-file that illustrates the bug or desired
feature.
