simplecrawler v1.1.3 Release Notes

Release Date: 2017-06-21 // almost 7 years ago
  • 🆕 New features

    • #376 - added Crawler#sortQueryParameters option. This option will sort the query parameters in a URL, making it simple to avoid fetching duplicate pages on sites that use a lot of query parameters. Thanks @HaroldPutman!