Merge pull request #243 from se-sic/dev

Version 4.3

Merged-by: Christian Hechtl <hechtl@cs.uni-saarland.de>

.github/workflows/pull_request.yml

@@ -0,0 +1,68 @@
+## This file is part of coronet, which is free software: you
+## can redistribute it and/or modify it under the terms of the GNU General
+## Public License as published by  the Free Software Foundation, version 2.
+##
+## This program is distributed in the hope that it will be useful,
+## but WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+## GNU General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License along
+## with this program; if not, write to the Free Software Foundation, Inc.,
+## 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+##
+## Copyright 2023 by Maximilian Löffler <s8maloef@cs.uni-saarland.de>
+## All Rights Reserved.
+
+name: Build Status
+
+on:
+  pull_request:
+    branches: [ master, dev ]
+    types: [ opened, reopened, synchronize ]
+  push:
+    branches: [ master, dev ]
+
+permissions:
+  contents: read
+
+jobs:
+  build:
+    name: Build
+
+    # change to 'runs-on: self-hosted' to run on self-hosted runners (https://docs.github.com/en/actions/using-jobs/choosing-the-runner-for-a-job)
+    runs-on: ubuntu-latest
+
+    strategy:
+      fail-fast: false
+      matrix:
+        r-version: ['3.6', '4.0', '4.1', '4.2', 'latest']
+
+    steps:
+      - name: Checkout Repo 
+        uses: actions/checkout@v3
+
+      - name: Update system
+        run: |
+          sudo apt-get update -y
+          sudo apt-get install --assume-yes libxml2
+          sudo apt-get install --assume-yes libxml2-dev
+          sudo apt-get install --assume-yes libglpk-dev
+          sudo apt-get install --assume-yes libfontconfig1-dev
+          sudo su -c "echo 'deb https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/' >> /etc/apt/sources.list"
+          wget -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc | sudo tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc
+
+      - name: Set up R ${{ matrix.r-version }}
+        uses: r-lib/actions/setup-r@v2
+        with:
+          r-version: ${{ matrix.r-version }}
+
+      - name: Install dependencies
+        run: Rscript install.R
+
+      - name: Run Tests
+        run: Rscript tests.R
+
+      - name: Run Showcase
+        run: Rscript showcase.R
+        if: always()

CONTRIBUTING.md

@@ -114,8 +114,8 @@ In our development process, we pursue the following idea:
 - The current development will be performed on the branch `dev`, i.e., all incoming pull requests are against this branch.
 
 The current build status is as follows:
-- `master`: [![Build Status](https://cloud.drone.io/api/badges/se-sic/coronet/status.svg)](https://cloud.drone.io/se-sic/coronet)
-- `dev`: [![Build Status](https://cloud.drone.io/api/badges/se-sic/coronet/status.svg?ref=refs/heads/dev)](https://cloud.drone.io/se-sic/coronet)
+- `master`: ![Build Status](https://github.com/se-sic/coronet/actions/workflows/pull_request.yml/badge.svg?branch=master)
+- `dev`: ![Build Status](https://github.com/se-sic/coronet/actions/workflows/pull_request.yml/badge.svg?branch=dev)
 
 ### Pull Requests
 

NEWS.md

@@ -2,6 +2,34 @@
 
 # coronet – Changelog
 
+## 4.3
+
+### Added
+
+- Add function `verify.data.frame.columns` to check that a dataframe includes all required columns, optionally with a specified datatype (PR #231, d1d9a039f50480ec5b442dc7e8b518648d1f9d9d)
+- Add helper function `is.single.na` to check whether an element is of length 1 and is `NA` (ddff2b8bbca6405f5c7c1cf4e7e97374fb1426ca, ccfc2d12a68dfa412f05159e8e3b03118694e748)
+- Add CI support for GitHub Actions (PR #234, fa1fc4af65751402ae6b23298dd4ed821930c6d2)
+
+### Changed/Improved
+
+- Include structural verification to almost all functions that read dataframes from files or set a dataframe (setter-functions) (PR #231, b7a95881da72ccaa548c6cd5d94bd558a25caa6f)
+- Include removal of empty and deleted users in the setters of mails, commits, issues, and authors. For commits, also the `committer.name` column is now checked for deleted or empty users (PR #235, 08fbd3e11e33d060f42cbc6f729eaf60b48a6de7)
+- Check for empty values (i.e., values of length < 1) when updating configuration attributes and throw an error if a value is empty (9f36c544637ab4f4173408152d223b9b5098ce5a)
+
+### Fixed
+
+- Fix check for empty input files in utility read functions. Compared to unpresent files, empty files do not throw an error when reading them, a check for `nrow(commit.data) < 1` is therefore required (PR #231, ecfa643cbc15975c3062af95c50ead02730b580f)
+- Fix various problems regarding the default classes of edge attributes and vertex attributes, and also make sure that the edge attributes for bipartite edges are chosen correctly (PR #240, 4275b93867c78d20d0bd116749c1e7603cd9d473, 98a6deb1b178a1fcf799c741906e99770c46a8d0, b8232c09b91df3412f703dd26c21c685bacd0321, a9535550d93207f466b315f33ea263a50e6c8924, 820a7631093d03ac5ccb7bf9923bd498f669120a)
+- Add argument to `construct.edge.list.from.key.value.list` function which differentiates if constructed edges are supposed to be artifact edges, in which case we check if the `artifact` attribute is present for edges and replace it by `author.name`. (PR #238, e2c9d6c39fb757c566ef4c4b18780cca247477cb, 7f42a91d4aa84e8c28c048925190637051e358a9)
+- Change edge construction algorithm for cochange-based artifact networks to respect the temporal order of data. This avoids duplicate edges. (PR #238, e2c9d6c39fb757c566ef4c4b18780cca247477cb)
+- Clarify that edges in issue-based artifact-networks are not available yet in `README.md`. (PR #238, 18a54f0241a28675dba4cdcbd433e88ec68d515a)
+- Fix bugs related to expanded adjacency matrices and update the initiation of sparse matrices to the most recent version of package Matrix, to replace deprecated and disfunct function calls. Due to this update, package versions prior to 1.3.0 of the Matrix package cannot be used any more. If the 'install.R' detects that a version prior to 1.3.0 is installed, it now automatically tries to re-install package Matrix once (PR #241, 573fab22a290e826e2bdd6e1f063cd2e87ed2167, 2f06252750354e4ed53b768bd212aacf1a350217)
+- Prevent R warnings `'length(x) = 2 > 1' in coercion to 'logical(1)'` in `if` conditions for updating configuration values, in update functions of additional data sources, and in `get.first.activity.data()` (PR #237, PR #241, ddff2b8bbca6405f5c7c1cf4e7e97374fb1426ca, e1579cab9bf8cdfee4105426c144350d092fffbd)
+- Prevent R warnings `In xtfrm.data.frame(x) : cannot xtfrm data frames` (PR #237, c24aee7d8f0b6ff4b641c8922e6ee1dce6f5999c)
+- Fix wrong bracket in pasted logging message (PR #241, 50c68cb60114b49c32dc5be15014745cb8d42ded)
+- Replace deprecated R function calls (PR #237, ed433821c04711a96501887b315d1b0ea8681f5a)
+
+
 ## 4.2
 
 ### Added

README.md

@@ -256,7 +256,7 @@ Relations determine which information is used to construct edges among the verti
 
 - `issue`
     * For author networks (configured via `author.relation` in the [`NetworkConf`](#networkconf)), authors who contribute to the same issue are connected with an edge.
-    * For artifact networks (configured via `artifact.relation` in the [`NetworkConf`](#networkconf)), issues are connected when they reference each other.
+    * For artifact networks (configured via `artifact.relation` in the [`NetworkConf`](#networkconf)), issues are connected when they reference each other. (**Note:** There are no edges available right now.)
     * For bipartite networks (configured via `artifact.relation` in the [`NetworkConf`](#networkconf)), authors get linked to all issues they have contributed to.
 
 - `callgraph`

install.R

@@ -16,8 +16,7 @@
 ## Copyright 2015 by Wolfgang Mauerer <wolfgang.mauerer@oth-regensburg.de>
 ## Copyright 2015-2017 by Claus Hunsen <hunsen@fim.uni-passau.de>
 ## Copyright 2017 by Thomas Bock <bockthom@fim.uni-passau.de>
-## Copyright 2022 by Thomas Bock <bockthom@cs.uni-saarland.de>
-## Copyright 2020-2021 by Thomas Bock <bockthom@cs.uni-saarland.de>
+## Copyright 2020-2023 by Thomas Bock <bockthom@cs.uni-saarland.de>
 ## Copyright 2019 by Anselm Fehnker <fehnker@fim.uni-passau.de>
 ## Copyright 2021 by Christian Hechtl <hechtl@cs.uni-saarland.de>
 ## All Rights Reserved.
@@ -69,4 +68,14 @@ if (length(p) > 0) {
     if (compareVersion(igraph.version, "1.3.0") == -1) {
         print("WARNING: igraph version 1.3.0 or higher is recommended for using coronet.")
     }
+
+    Matrix.version = installed.packages()[rownames(installed.packages()) == "Matrix", "Version"]
+    if (compareVersion(Matrix.version, "1.3.0") == -1) {
+        print("WARNING: Matrix version 1.3.0 or higher is necessary for using coronet. Re-install package Matrix...")
+        install.packages("Matrix", dependencies = NA, verbose = TRUE, quiet = TRUE)
+        Matrix.version = installed.packages()[rownames(installed.packages()) == "Matrix", "Version"]
+        if (compareVersion(Matrix.version, "1.3.0") == -1) {
+            print("WARNING: Re-installation of package Matrix did not end up in the necessary packge version.")
+        }
+    }
 }

showcase.R

@@ -402,7 +402,7 @@ p = p +
         plot.title = ggplot2::element_text(hjust = 0.5, size = 15),
         plot.margin = ggplot2::unit(c(0.5, 0.5, 0.5, 0.5), "cm")
     )
-ggplot2::ggsave("plot-multi.png", plot = p, width = 6.57, height = 4.114)
+ggplot2::ggsave("plot-multi.png", plot = p, bg = "white", width = 6.57, height = 4.114)
 
 
 ## / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / /

tests/codeface-data/results/testing/test_feature/feature/commits.list

@@ -9,3 +9,5 @@
 32716;"2016-07-12 16:06:30";"Thomas";"thomas@example.org";"2016-07-12 16:06:30";"Thomas";"thomas@example.org";"d01921773fae4bed8186b0aa411d6a2f7a6626e6";1;1;0;1;"";"";"";0
 32711;"2016-07-12 16:06:32";"Thomas";"thomas@example.org";"2016-07-12 16:06:32";"Thomas";"thomas@example.org";"0a1a5c523d835459c42f33e863623138555e2526";1;1;0;1;"test2.c";"Base_Feature";"Feature";1
 32711;"2016-07-12 16:06:32";"Thomas";"thomas@example.org";"2016-07-12 16:06:32";"Thomas";"thomas@example.org";"0a1a5c523d835459c42f33e863623138555e2526";1;1;0;1;"test2.c";"foo";"Feature";1
+31711;"2016-07-12 16:06:33";"Thomas";"thomas@example.org";"2016-07-12 16:06:33";"";"thomas@example.org";"2ef7bde608ce5404e97d5f042f95f89f1c232871";1;1;0;1;"test2.c";"foo";"Feature";1
+30711;"2016-07-12 16:06:34";"Thomas";"thomas@example.org";"2016-07-12 16:06:34";"deleted user";"thomas@example.org";"c6954cb75e3eeec5b827f64e97b6a4ba187c0d55";1;1;0;1;"test2.c";"foo";"Feature";1

tests/codeface-data/results/testing/test_proximity/proximity/commits.list

@@ -5,3 +5,5 @@
 32715;"2016-07-12 16:06:32";"Thomas";"thomas@example.org";"2016-07-12 16:06:32";"Thomas";"thomas@example.org";"0a1a5c523d835459c42f33e863623138555e2526";1;1;0;1;"test2.c";"File_Level";"Function";1
 32720;"2016-07-12 16:06:20";"Karl";"karl@example.org";"2016-07-12 16:06:20";"Karl";"karl@example.org";"418d1dc4929ad1df251d2aeb833dd45757b04a6f";1;1;0;1;"";"";"";0
 32721;"2016-07-12 16:06:30";"Thomas";"thomas@example.org";"2016-07-12 16:06:30";"Thomas";"thomas@example.org";"d01921773fae4bed8186b0aa411d6a2f7a6626e6";1;1;0;1;"";"";"";0
+32811;"2016-07-12 16:06:33";"Thomas";"thomas@example.org";"2016-07-12 16:06:33";"";"thomas@example.org";"2ef7bde608ce5404e97d5f042f95f89f1c232871";1;1;0;1;"test2.c";"foo";"Feature";1
+32911;"2016-07-12 16:06:34";"Thomas";"thomas@example.org";"2016-07-12 16:06:34";"deleted user";"thomas@example.org";"c6954cb75e3eeec5b827f64e97b6a4ba187c0d55";1;1;0;1;"test2.c";"foo";"Feature";1

tests/test-core-peripheral.R

@@ -79,7 +79,7 @@ test_that("Eigenvector classification", {
     row.names(result[["core"]]) = NULL
     row.names(result[["peripheral"]]) = NULL
 
-    expect_equal(expected, result, tolerance = 0.0001)
+    ## expect_equal(expected, result, tolerance = 0.0001)
     ## TODO: Find a way to directly test for equality without the need of taking care of different orders of author
     ##       names. For the moment, we take the following workaround:
 

tests/test-misc.R

@@ -14,6 +14,8 @@
 ## Copyright 2017 by Felix Prasse <prassefe@fim.uni-passau.de>
 ## Copyright 2017-2018 by Claus Hunsen <hunsen@fim.uni-passau.de>
 ## Copyright 2017-2018 by Thomas Bock <bockthom@fim.uni-passau.de>
+## Copyright 2023 by Thomas Bock <bockthom@cs.uni-saarland.de>
+## Copyright 2022-2023 by Maximilian Löffler <s8maloef@stud.uni-saarland.de>
 ## All Rights Reserved.
 
 
@@ -107,6 +109,109 @@ test_that("Match argument or take default.", {
     expect_equal(actual.result, expected.result, info = "Multiple choices with ignored default, two choices")
 })
 
+##
+## Check presence and datatype of data frame columns.
+##
+
+test_that("Check presence and datatype of data frame columns.", {
+
+    user.names = c("John", "Peter", "Maria", "Susanne")
+
+    ## contains NaN to verify functionality does not break
+    age = c(42, 50, NaN, 66)
+
+    ## contains NA to verify functionality does not break
+    is.male = c(TRUE, TRUE, FALSE, NA)
+
+    ## construct simple testing dataframe
+    data.frame = data.frame(user.names, age, is.male)
+
+    ## 1) Check base functionality (benign use-case)
+    expect_error(verify.data.frame.columns(
+        data.frame, c("user.names", "age", "is.male"), c("character", "numeric", "logical")),
+        NA,
+        message = "All columns present and well-typed.")
+    ## Expect no error
+
+    ## 2) Base test with reordered columns
+    expect_error(verify.data.frame.columns(
+        data.frame, c("is.male", "age", "user.names"), c("logical", "numeric", "character")),
+        NA,
+        message = "Order of columns does not matter.")
+    ## Expect no error
+
+    ## 3) Specify less columns than present (Allow optional columns)
+    expect_error(verify.data.frame.columns(
+        data.frame, c("user.names", "age"), c("character", "numeric")),
+        NA,
+        message = "Optional columns are allowed.")
+    ## Expect no error
+
+    ## 4) Unequal amount of column names and datatypes
+    expect_error(verify.data.frame.columns(
+        data.frame, c("user.names", "age", "is.male"), c("character", "numeric")),
+        message = "More column names specified than datatypes.")
+    expect_error(verify.data.frame.columns(
+        data.frame, c("user.names", "age"), c("character", "numeric", "logical")),
+        message = "More column names specified than datatypes.")
+
+    ## 5) Datatypes do not match column names
+    expect_error(verify.data.frame.columns(
+        data.frame, c("user.names", "age", "is.male"), c("logical", "character", "numeric")),
+        message = "Column names do not match datatypes.")
+
+    ## 6) Invalid column / Column not present in dataframe (Typo)
+    expect_error(verify.data.frame.columns(
+        data.frame, c("user.name"), c("character")),
+        message = "Column name 'user.name' should not be in dataframe.")
+
+    ## 7) No datatypes specified and column names are present
+    expect_error(verify.data.frame.columns(
+        data.frame, c("user.names", "age", "is.male")),
+        NA,
+        message = "Column names do not match datatypes.")
+    ## Expect no error
+
+    ## 8) No datatypes specified and column names are not specified correctly (Typo)
+    expect_error(verify.data.frame.columns(
+        data.frame, c("user.name")),
+        message = "Column name 'user.name' should not be in dataframe.")
+
+    ## 9) Too many column names and no datatypes specified
+    expect_error(verify.data.frame.columns(
+        data.frame, c("user.names", "age", "is.male", "job.orientation")),
+        message = "More column names specififed than present in the dataframe.")
+
+})
+
+
+## / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / /
+## Vector misc--------------------------------------------------------------
+
+##
+## Check if a value is a single NA value.
+##
+
+test_that("Check if a value is a single NA value", {
+
+    ## 1) Tests for single NA
+    expect_true(is.single.na(NA))
+    expect_true(is.single.na(list(NA)))
+    expect_true(is.single.na(data.frame(NA)))
+
+    ## 2) Tests for values other than a single NA
+    expect_false(is.single.na(0))
+    expect_false(is.single.na("na"))
+    expect_false(is.single.na(NULL))
+    expect_false(is.single.na(logical(0)))
+    expect_false(is.single.na(FALSE))
+    expect_false(is.single.na(c(NA, NA)))
+    expect_false(is.single.na(c(3, NA)))
+    expect_false(is.single.na(list(NA, NA)))
+    expect_false(is.single.na(data.frame(NA, NA)))
+})
+
+
 ## / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / /
 ## Date handling -----------------------------------------------------------
 

tests/test-networks-artifact.R

@@ -15,6 +15,7 @@
 ## Copyright 2017-2019 by Claus Hunsen <hunsen@fim.uni-passau.de>
 ## Copyright 2018 by Barbara Eckl <ecklbarb@fim.uni-passau.de>
 ## Copyright 2018 by Jakob Kronawitter <kronawij@fim.uni-passau.de>
+## Copyright 2023 by Maximilian Löffler <s8maloef@stud.uni-saarland.de>
 ## All Rights Reserved.
 
 
@@ -42,13 +43,13 @@ test_that("Network construction of the undirected artifact-cochange network", {
                           type = TYPE.ARTIFACT)
     ## 2) edges
     edges = data.frame(
-        from = c("Base_Feature", "Base_Feature"),
-        to = c("foo", "foo"),
-        date = get.date.from.string(c("2016-07-12 16:06:32", "2016-07-12 16:06:32")),
-        artifact.type = c("Feature", "Feature"),
-        hash = c("0a1a5c523d835459c42f33e863623138555e2526", "0a1a5c523d835459c42f33e863623138555e2526"),
-        file = c("test2.c", "test2.c"),
-        artifact = c("Base_Feature", "foo"),
+        from = "Base_Feature",
+        to = "foo",
+        date = get.date.from.string("2016-07-12 16:06:32"),
+        artifact.type = "Feature",
+        hash = "0a1a5c523d835459c42f33e863623138555e2526",
+        file = "test2.c",
+        author.name = "Thomas",
         weight = 1,
         type = TYPE.EDGES.INTRA,
         relation = "cochange"

tests/test-networks-author.R

@@ -13,9 +13,11 @@
 ##
 ## Copyright 2017, 2019 by Claus Hunsen <hunsen@fim.uni-passau.de>
 ## Copyright 2017 by Christian Hechtl <hechtl@fim.uni-passau.de>
+## Copyright 2023 by Christian Hechtl <hechtl@cs.uni-saarland.de>
 ## Copyright 2017 by Felix Prasse <prassefe@fim.uni-passau.de>
 ## Copyright 2018 by Barbara Eckl <ecklbarb@fim.uni-passau.de>
 ## Copyright 2018 by Thomas Bock <bockthom@fim.uni-passau.de>
+## Copyright 2023 by Thomas Bock <bockthom@cs.uni-saarland.de>
 ## Copyright 2018 by Jakob Kronawitter <kronawij@fim.uni-passau.de>
 ## Copyright 2018-2019 by Anselm Fehnker <fehnker@fim.uni-passau.de>
 ## Copyright 2021 by Johannes Hostert <s8johost@stud.uni-saarland.de>
@@ -421,12 +423,19 @@ test_that("Network construction of the undirected simplified author-cochange net
                          kind = TYPE.AUTHOR,
                          type = TYPE.AUTHOR)
 
+    ## make test independent of igraph version
+    date.attr = igraph::get.edge.attribute(network.built, "date")
+    date.conversion.function = ifelse(all(sapply(date.attr, lubridate::is.POSIXct)),
+                                      get.date.from.unix.timestamp, identity)
+
     ## edge attributes
     data = data.frame(
         from = c("Björn", "Olaf", "Olaf", "Karl"),
         to = c("Olaf", "Karl", "Thomas", "Thomas"),
-        date = I(list(c(1468339139, 1468339245), c(1468339541, 1468339570), c(1468339541, 1468339592),
-                      c(1468339570, 1468339592))),
+        date = I(list(date.conversion.function(c(1468339139, 1468339245)),
+                      date.conversion.function(c(1468339541, 1468339570)),
+                      date.conversion.function(c(1468339541, 1468339592)),
+                      date.conversion.function(c(1468339570, 1468339592)))),
         artifact.type = I(list(c("Feature", "Feature"), c("Feature", "Feature"), c("Feature", "Feature"),
                                c("Feature", "Feature"))),
         hash = I(list(
@@ -442,6 +451,13 @@ test_that("Network construction of the undirected simplified author-cochange net
         relation = "cochange"
     )
 
+    ## remove the 'AsIs' class from the edge attributes that have been inserted via `I(...)`
+    data[["date"]] = unclass(data[["date"]])
+    data[["artifact.type"]] = unclass(data[["artifact.type"]])
+    data[["hash"]] = unclass(data[["hash"]])
+    data[["file"]] = unclass(data[["file"]])
+    data[["artifact"]] = unclass(data[["artifact"]])
+
     ## build expected network
     network.expected = igraph::graph.data.frame(data, directed = FALSE, vertices = authors)