The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

502.