James
|
7f2b819861
|
remove the console command
|
2020-03-06 13:34:37 +00:00 |
|
James
|
3ab51ccff9
|
better tests for interlinked and redirect tracking
|
2020-02-24 09:07:58 +00:00 |
|
James
|
340e78de13
|
crawler takes request options in constructor, update docs
|
2020-02-23 18:23:36 +00:00 |
|
James
|
adaaaef3c2
|
more comprehensive tests and tidy up
|
2020-02-23 01:19:31 +00:00 |
|
James
|
9c0961fd87
|
fix tests
|
2020-02-23 01:08:14 +00:00 |
|
James
|
fe45abaf49
|
make baseurl (internal links only) optional in crawler constructor
|
2020-02-22 13:57:43 +00:00 |
|
James
|
ef4ebf7cd0
|
add simple console command
|
2020-02-22 13:57:43 +00:00 |
|
James
|
9a84ec204d
|
test and fix redirect tracking
|
2020-02-22 10:06:18 +00:00 |
|
James
|
d81dc86381
|
bugfix inverted logic
|
2020-02-21 13:05:20 +00:00 |
|
James
|
c81551e2bf
|
add tests
|
2020-02-21 10:59:58 +00:00 |
|
James
|
ac7b849036
|
make observer collate all found on urls
|
2020-02-21 10:24:34 +00:00 |
|
James
|
6a986a62c0
|
inital
|
2020-02-20 17:06:44 +00:00 |
|