Skip to content

@arcalex Web Archiving at BA

Change the repository type filter

All

    Repositories list

    24 repositories

    • moodwarc

      Public
      Analysis of how content on the web affects mood over time
      GNU General Public License v3.0
      •0•0•0•0•Updated Jul 30, 2024Jul 30, 2024

    • link-indexer

      Public
      linked data collection tool for web archive graph visualization (LinkGate)
      Python
      •
      GNU General Public License v3.0
      •2•0•6•1•Updated Feb 22, 2024Feb 22, 2024

    • warclassifier

      Public
      Machine learning-based content classification for web archives
      GNU General Public License v3.0
      •1•0•0•0•Updated Oct 5, 2023Oct 5, 2023

    • warchtml

      Public
      Extract HTML data from WARC files
      GNU General Public License v3.0
      •0•0•0•0•Updated Aug 25, 2023Aug 25, 2023

    • meshwarc

      Public
      Wxploring the semantic-based web archive graph using MeshWARC
      GNU General Public License v3.0
      •0•0•0•0•Updated Aug 25, 2023Aug 25, 2023

    • txtcrawl-rs

      Public
      Crawl the web for text (Rust implementation)
      GNU General Public License v3.0
      •1•0•0•0•Updated Aug 25, 2023Aug 25, 2023

    • waget

      Public
      Incrementally fetch web archive data files and run actions
      Shell
      •
      GNU General Public License v3.0
      •1•2•0•0•Updated Apr 18, 2023Apr 18, 2023

    • link-serv

      Public
      versioned graph data service for web archive graph visualization (LinkGate)
      Java
      •
      GNU General Public License v3.0
      •2•0•13•8•Updated Dec 10, 2022Dec 10, 2022

    • iipc-collections

      Public
      Republishing IIPC collections through alternative interfaces for researcher access
      Shell
      •
      GNU General Public License v3.0
      •0•0•0•0•Updated Sep 8, 2022Sep 8, 2022

    • txtcrawl

      Public
      Crawl the web for text
      Python
      •
      GNU General Public License v3.0
      •1•0•0•0•Updated Aug 25, 2022Aug 25, 2022

    • warc-serv

      Public
      Serve records from WARC files just like you serve files from a web server's root directory
      Python
      •
      GNU General Public License v3.0
      •3•0•0•0•Updated Aug 25, 2022Aug 25, 2022

    • data-transfer-apis

      Public
      WASAPI data transfer APIs
      Python
      •6•0•0•0•Updated Apr 23, 2022Apr 23, 2022

    • linkgate

      Public
      common material for IIPC Project LinkGate, including research use cases for web archive graph visualization
      Shell
      •
      GNU General Public License v3.0
      •1•4•1•0•Updated Jan 13, 2022Jan 13, 2022

    • link-viz

      Public
      temporal graph rendering and exploration web frontend for web archive visualization (LinkGate)
      JavaScript
      •
      GNU General Public License v3.0
      •1•0•19•0•Updated Sep 29, 2021Sep 29, 2021

    • llx

      Public
      Parallel execution of processes based on a command template and input fields
      Perl
      •
      GNU General Public License v3.0
      •0•0•0•0•Updated Sep 23, 2021Sep 23, 2021

    • bibalex-crawl-config

      Public
      Recipe for crawling the web, by the Bibliotheca Alexandrina
      Shell
      •
      GNU General Public License v3.0
      •1•0•0•0•Updated Dec 2, 2020Dec 2, 2020

    • pywb

      Public
      Core Python Web Archiving Toolkit for replay and recording of web archives
      Python
      •
      GNU General Public License v3.0
      •217•0•0•0•Updated Oct 12, 2020Oct 12, 2020

    • content-scoper

      Public
      Example scripts for identifying content that falls within scope of a web crawl using machine learning
      Python
      •1•1•0•0•Updated May 14, 2019May 14, 2019

    • crawl-log-animator

      Public
      Crawl Log Animator
      0•0•0•0•Updated Apr 3, 2019Apr 3, 2019

    • warcrefs

      Public
      Web archive deduplication tools
      Java
      •1•6•11•0•Updated Oct 18, 2018Oct 18, 2018

    • racktk

      Public
      Command-line tools for computer clusters
      Perl
      •
      GNU General Public License v3.0
      •1•1•0•0•Updated Aug 17, 2017Aug 17, 2017

    • gzmulti

      Public
      Manipulate multi-member GZIP files
      C
      •
      GNU General Public License v3.0
      •0•0•0•0•Updated May 16, 2017May 16, 2017

    • purge-old-kernels

      Public
      Purge old kernel packages on a Debian system
      Shell
      •
      GNU General Public License v3.0
      •0•1•0•0•Updated Mar 12, 2017Mar 12, 2017

    • warcsum

      Public
      Web archive checksum
      C
      •
      GNU General Public License v3.0
      •0•4•0•0•Updated May 12, 2016May 12, 2016