Go to file
James 4e29ca154e
Some checks failed
continuous-integration/drone/push Build is failing
downgrade
2020-11-15 18:33:05 +00:00
example add example 2020-03-10 10:52:13 +00:00
src crawl externals too 2020-11-15 14:43:23 +00:00
tests fix redirect loop test 2020-11-15 14:54:32 +00:00
.drone.yml fix 2020-11-15 14:33:08 +00:00
.gitignore inital 2020-02-20 17:06:44 +00:00
composer.json downgrade 2020-11-15 18:33:05 +00:00
composer.lock downgrade 2020-11-15 18:33:05 +00:00
README.md Add drone.yml and fix tests to be compatable with docker pipeline (#1) 2020-09-30 15:13:56 +01:00

Similar project to spatie/http-status-check but this will collect ALL the found on URL's (not just first occurrence) and return the results as an array for further processing in your own application (no console command).

Install

composer config repositories.jhodges composer https://git.jhodges.co.uk/composer
composer require jhodges/sitemap

Usage

Note: start the test server as shown below if you want to use the localhost:8080 site as per usage example.

use \JHodges\Sitemap\Crawler;

$crawler=new Crawler();
$crawler->crawl('https://localhost:8080');
$sitemap=$crawler->getResults();
print_r($sitemap);

To crawl multiple areas of the same site that are not interlinked, you may:

use \JHodges\Sitemap\Crawler;

$crawler=new Crawler();
$crawler->crawl('https://localhost:8080/interlinked1');
$crawler->crawl('https://localhost:8080/deeplinked1');
$sitemap=$crawler->getResults();
print_r($sitemap);

Tests

Start the test server, will listen on localhost:8080

cd tests/server
npm install
node server.js

Run the tests:

vendor/bin/phpunit tests