Project

Reverse Dependencies for spidr

The projects listed here declare spidr as a runtime or development dependency

0.03
Low commit activity in last 3 years
No release in over a year
validate-website is a web crawler for checking the markup validity with XML Schema / DTD and not found urls.
2019
2020
2021
2022
2023
2024
0.03
Low commit activity in last 3 years
No release in over a year
Post URLs to Wayback Machine (Internet Archive), using a crawler, from Sitemap(s) or a list of URLs.
2019
2020
2021
2022
2023
2024
0.01
No commit activity in last 3 years
No release in over 3 years
Hacker-gems installs most needed gems for hackers.
2019
2020
2021
2022
2023
2024
0.0
No commit activity in last 3 years
No release in over 3 years
A Gem to produce competitive intelligence data
2019
2020
2021
2022
2023
2024
0.0
No commit activity in last 3 years
No release in over 3 years
Crawler that auto generates SitePrism Page Objects
2019
2020
2021
2022
2023
2024
0.0
No release in over a year
ronin-web-spider is a collection of common web spidering routines using the spidr gem.
2019
2020
2021
2022
2023
2024
0.0
No release in over 3 years
Low commit activity in last 3 years
There's a lot of open issues
Crawl a site and check various health indicators, such as: HTTP 4XX, 5XX status, valid HTML/XML/JSON. Missing image alt attributes/missing HTML title/description.
2019
2020
2021
2022
2023
2024
0.0
No release in over 3 years
Low commit activity in last 3 years
Command Line Interface (CLI) for the spidr gem.
2019
2020
2021
2022
2023
2024
0.0
No release in over 3 years
Low commit activity in last 3 years
Stupid crawler that looks for URLs on a given site. Result is saved as two CSV files one with found URLs and another with failed URLs.
2019
2020
2021
2022
2023
2024