File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -27,18 +27,26 @@ You can find out more about Composer here: https://getcomposer.org/
2727
2828
2929## Usage
30- Fetch headers automatically :
30+ Get rules for a specific UserAgent :
3131``` php
3232$parser = new \vipnytt\XRobotsTagParser('http://example.com/', 'myUserAgent');
33- $rules = $parser->getRules();
33+ $array = $parser->getRules();
3434```
3535
36- Export all rules :
36+ Use existing headers :
3737``` php
38- $parser = new \vipnytt\XRobotsTagParser('http://example.com/');
39- $rules = $parser->export ();
38+ $parser = new \vipnytt\XRobotsTagParser('http://example.com/', 'myUserAgent', ['headers' => $headers] );
39+ $array = $parser->getRules ();
4040```
4141
42+ Export rules for all UserAgents:
43+ ``` php
44+ $parser = new \vipnytt\XRobotsTagParser('http://example.com/', 'myUserAgent');
45+ $array = $parser->export();
46+ ```
47+
48+
49+
4250## Directives
4351- [x] ```` all ```` - There are no restrictions for indexing or serving.
4452- [x] ```` none ```` - Equivalent to ```` noindex ```` , ```` nofollow ````
You can’t perform that action at this time.
0 commit comments