Go to file
2021-01-03 17:39:04 +00:00
docker add docker support 2021-01-03 17:39:04 +00:00
example add example 2020-03-10 10:52:13 +00:00
src add setUserAgent support 2020-11-16 12:58:28 +00:00
tests revert fix redirectloop test 2020-11-15 18:34:28 +00:00
.drone.yml fix 2020-11-15 14:33:08 +00:00
.gitignore inital 2020-02-20 17:06:44 +00:00
composer.json downgrade 2020-11-15 18:33:05 +00:00
composer.lock downgrade 2020-11-15 18:33:05 +00:00
README.md fix composer url moved 2021-01-03 17:12:28 +00:00

Similar project to spatie/http-status-check but this will collect ALL the found on URL's (not just first occurrence) and return the results as an array for further processing in your own application (no console command).

Install

composer config repositories.jhodges composer https://composer.jhodges.co.uk/
composer require jhodges/sitemap

Usage

Note: start the test server as shown below if you want to use the localhost:8080 site as per usage example.

use \JHodges\Sitemap\Crawler;

$crawler=new Crawler();
$crawler->crawl('https://localhost:8080');
$sitemap=$crawler->getResults();
print_r($sitemap);

To crawl multiple areas of the same site that are not interlinked, you may:

use \JHodges\Sitemap\Crawler;

$crawler=new Crawler();
$crawler->crawl('https://localhost:8080/interlinked1');
$crawler->crawl('https://localhost:8080/deeplinked1');
$sitemap=$crawler->getResults();
print_r($sitemap);

Tests

Start the test server, will listen on localhost:8080

cd tests/server
npm install
node server.js

Run the tests:

vendor/bin/phpunit tests