Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v0.10.0 / Firefox 77.0.1 release #682

Merged
merged 16 commits into from
Jun 22, 2020
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 27 additions & 2 deletions CHANGELOG
Original file line number Diff line number Diff line change
@@ -1,8 +1,33 @@
v0.10.0 - under development
v0.10.0 - 2020-06-11
=======

This release is a long overdue release of OpenWPM, and contains too many
changes to list here. Instead, we highlight the major architectural since the
previous release. The instrumentation has been completely rewritten as part of
this release to the new WebExtensions framework. Older versions of OpenWPM
should not be used.

Changes:
* Migrate to WebExtensions from the addon-sdk.
* Migrate instrumentation from the addon-sdk framework to WebExtensions.
* Migrate to unbranded builds of Firefox, and off of the ESR channel to the
Release channel
* Add support for MacOS development
* Add an S3Aggregator that saves data in Parquet format on S3
* Use conda for dependency management
* Disable stateful crawling due to intermittent loss of profiles and
geckodriver incompatibilities
* Refactor extension instumentation to live in a separate module
* Re-write logger and add support for logging to sentry
* Add a crawler.py crawl script that can be used for cloud deployments like
the type documented in https://github.com/mozilla/openwpm-crawler
* Add support for Firefox's native headless mode alongside XVFB
* Add Dockerfile and automatically deploy builds to dockerhub
* Add a Navigation instrument that records navigation events
* Drop support for Python 2
* Remove support for Flash
* Numerous stability and data saving improvements (particularly for cloud
crawls / the S3Aggregator)
* Numerous bugfixes + improved testing

v0.9.0 - 2019-04-15
======
Expand Down
43 changes: 22 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Table of Contents <!-- omit in toc -->

* [Installation](#installation)
* [Quick Start](#quick-start)
* [Advice for Measurement Researchers](#advice-for-measurement-researchers)
* [Instrumentation and Data Access](#instrumentation-and-data-access)
* [Output Format](#output-format)
* [Local Databases](#local-databases)
Expand All @@ -39,7 +40,6 @@ Table of Contents <!-- omit in toc -->
* [Building the Docker Container](#building-the-docker-container)
* [Running Measurements from inside the Container](#running-measurements-from-inside-the-container)
* [MacOS GUI applications in Docker](#macos-gui-applications-in-docker)
* [Disclaimer](#disclaimer)
* [Citation](#citation)
* [License](#license)

Expand Down Expand Up @@ -113,11 +113,28 @@ tutorial, including a
[platform demo](https://github.com/citp/OpenWPM/wiki/Platform-Demo)
and a description of the
[additional commands](https://github.com/citp/OpenWPM/wiki/Available-Commands)
available. You can also take a look at two of our past studies, which use the
infrastructure:
available.

1. [The Web Never Forgets](https://github.com/citp/TheWebNeverForgets)
2. [Cookies that Give You Away](https://github.com/englehardt/cookies-that-give-you-away)

Advice for Measurement Researchers
----------------------------------

OpenWPM is [often used](https://webtap.princeton.edu/software/) for web
measurement research. We recommend the following for researchers using the tool:

**Use a versioned [release](https://github.com/mozilla/OpenWPM/releases).** We
aim to follow Firefox's release cadence, which is roughly once every four
weeks. If we happen to fall behind on checking in new releases, please file an
issue. Versions more than a few months out of date will use unsupported
versions of Firefox, which are likely to have known security
vulnerabilities. Versions less than v0.10.0 are from a previous architecture
and should not be used.

**Include the OpenWPM version number in your publication.** As of v0.10.0
OpenWPM pins all python, npm, and system dependencies. Including this
information alongside your work will allow other researchers to contextualize
the results, and can be helpful if future versions of OpenWPM have
instrumentation bugs that impact results.

Instrumentation and Data Access
-------------------------------
Expand Down Expand Up @@ -657,22 +674,6 @@ Or, run commands directly:
./run-on-osx-via-docker.sh python -m pytest
./run-on-osx-via-docker.sh python -m pytest -vv -s

Disclaimer
-----------

Note that OpenWPM is under active development, and should be considered
experimental software. The repository may contain experimental features that
aren't fully tested. We recommend using a [tagged
release](https://github.com/citp/OpenWPM/releases).

Although OpenWPM is actively used by our group for research studies and we
regularly use of the data collected, it is still possible there are unknown bugs
in the infrastructure. We are in the process of writing comprehensive tests to
verify the integrity of all included instrumentation. Prior to using OpenWPM
for your own research we encourage you to write tests (and submit pull
requests!) for any instrumentation that isn't currently included in our test
scripts.

Citation
--------

Expand Down
Loading