Skip to content

ORT Community Meeting

Jump to bottom
Sebastian Schuberth edited this page Nov 28, 2024 · 301 revisions

The ORT community hosts weekly meetings on Thursdays, 10.00 - 11.00 CET, that are open to any interested party (reach out to @mnonnenmacher if you are having problems with the above invitation link).

An important goal of these meetings is to share and align on the ideas for the future development of ORT, and to maintain a community roadmap. Also, e.g. open pull requests may be discussed where verbal communication might be more expedient than the usual textual conversation.

To shape these meetings efficient, the aim is to have an agenda before each meeting, and stick to the agenda topics before discussing anything else. In general, for support questions please prefer to reach out to the ORT community on Slack.

Future Topics

Meeting Minutes

2024-12-05 (planned)

2024-11-28

2024-11-21

  • Release Notes for version 41.0.0.

2024-11-14

  • Release Notes for version 40.0.0.
  • Update of WebApp report coming soon (@tsteenbe). Looking for testers.

2024-11-07

  • Release Notes for version 39.0.0.
    • Note that third-party plugins now need to be copied into the plugin directory; previously they were tolerated in the lib directory of an ORT installation as well.
  • Remove PR labels for breaking changes as we have them in release notes, and having both causes confusion.
  • Add a "Project" suffix to IDs of projects (like "MavenProject") to solve the specific problem of project vs package ID duplicates.
    • No additional concerns, proceed with the implementation.

2024-10-31

  • Release Notes for version 38.0.0.
    • Halloween edition 🎃
  • Discussions about how to read in invalid SPDX produced by Snyk (Copyright text is explicitly set to empty)

2024-10-24

2024-10-17

  • Release Notes for version 36.0.0.
  • Further discussion of how to handle Bazel monorepos where a singl emodule file refers to all projects in the monorepo, but only a subset of that should be analyzed.
    • One idea is a package-manager-specific option for Bazel to filter projects parsed from the module file before analyzing them.

2024-10-10

  • Release Notes for version 34.0.0.
  • Release Notes for version 35.0.0.
  • First Release Candidate for the ORT Server.
  • Discussion about different root causes for duplicate identifiers
    • The same "thing" could be seen as both a dependency and a project in a (mono-)repo that has project A with dependency package B, but the source code of B is hosted in the same repo.
    • E.g. NPM and Yarn projects could depend on the same (?) package, but it gets pulled from different (private) registries.
    • ORT's logic to come up with project names for projects without (enough) metadata (like Python's requirements.txt files) could create name / id clashes.
    • See https://github.com/oss-review-toolkit/ort/issues/9417.
  • Alliander has published their OSPO Code Scanner and ORT is part of it 😃

2024-10-03

  • No weekly call on October 3rd as it a national holiday in Germany so most people will have a day off. Next call will be October 10th. Althought there will not be a call the ORT project will have a release this week as usual.
  • Release Notes for version 34.0.0 (will be discussed on 2024-10-10).

2024-09-26

  • Release Notes for version 33.1.0.
  • Any objections to move to Java 21?
    • No objections let's be bold and remove it to see if something breaks.
    • But first add the javaVersion feature to "legacy" Gradle analyzer.
  • Any objections to remove the Nexus IQ advisor?
    • No objections, no one seems to be using it still.

2024-09-19

2024-09-12

  • Release Notes for version 32.0.0.

2024-09-05

  • Release Notes for version 31.0.0.
  • @mnonnenmacher to talk about the new plugin API.
  • Welcome @nnobelis as a new commiter to the ORT project!
  • Reto continues to work on and off on Conan 2 support
    • Can we limit ourselves to only support lockfiles?
  • Enrico introduces himself
  • We need to properoly maintain the (or a new version of) GitHub action.

2024-08-29

2024-08-22

  • Release Notes for version 29.0.0.
    • No more vulnerabilities, thanks to the removal of the GitHub defects advisor.
  • Release Notes for version 29.1.0.
  • Handing multiple SCANOSS line ranges for snippets.
  • Heads up: AOSD 2.0 will get an update to 2.1 soon.
  • Discussion about vulnerabilities reported for ORT Docker images; the majority probably comes from the base image.

2024-08-15 (first meeting after summer break)

2024-07-18 (last meeting before summer break)

  • No release due to Sonatype issues. Edit: The issue was fixed shortly after the meeting.
  • Release Notes for version 27.0.0.

2024-07-11

2024-07-04

2024-06-27

2024-06-20

  • Release Notes for version 24.0.0.
  • Sneak peek at the ORT Server frontend UI hosted at Double Open / Scaleway.

2024-06-13

2024-06-06

2024-05-30

  • Release Notes for version 22.6.0.
  • Ok to remove JitPack builds?
    • No objection, but probably ask again next week with more participants.

2024-05-23

  • Release Notes for version 22.4.0.
  • Release Notes for version 22.5.0.
  • Issue with scancode missing from the Docker image since 22.3.0 is gone with release 22.5.0.

2024-05-16

  • Release Notes for version 22.3.0.
  • CycloneDX 1.6 support delayed due to bugs in their core library.
    • Probably stick with writing out CycloneDX 1.5 by default until more tools support 1.6 features.

2024-05-09 (skipped)

Skipped due to a public holiday in Germany and elsewhere.

2024-05-02

  • Release Notes for version 22.2.0.
  • Backlog grooming of issues was made on 2024-04-29 by @sschuberth @tsteenbe @mnonnenmacher and @fviernau - closed tickets with sentence "Closed as part of backlog grooming. Feel free to comment if you would like to contribute to this."
  • Discussion about how to display effective licenses in the web-app report
    • Community favorite is to consolidate declared, detected and effective license information in a combined "Licensing" tab
    • Details about how to summarize licensing information (summary header vs. licensing tab) still need to be figured out
    • Generally tabs should not be reordered automatically or hidden if there is no content, but the first tab to act upon in a usual workflow should be made active by default

2024-04-25

  • Release Notes for version 22.1.0.

2024-04-18

2024-04-11

  • Release Notes for version 21.0.0.
  • Discussion on solution on improving copyright holders within ORT
    • Anton from VW showed table with examples, see slide 15 in https://github.com/oss-review-toolkit/ort/files/14692577/ORT-at-Volkswagen.pdf.
    • Is post-processing done by ScanCode an intentionally feature or unwanted side-effect
      • Philippe: Not unwanted side affect, no dependency on third party libs that introduces the issues - it's a design decision from ScanCode to harmonize
      • Martin: What about just getting the original copyrights directly from ScanCode? But that boils down - does someone have resources to work on this?
      • Philippe: Can't out of the top of my head say effort required. Could set up project on OpenCollective to pull funding.
      • Rishi: Think it worth to do some research and put a price / effort required to fix this so ORT users can bring this back to their organizations.
      • Thomas propose to set up a group of folks:
        • Users: clear description of all issue, provide test data and willingness support testing any solution or raise resulting document within their organizations
        • Scancode: Willingness to do initial research to determine parts of the code impacted, possible solutions and coach devs on how to fix issues
        • Both sides: Estimate effort required
        • Agreement to work on this
        • Anton: happy to provide test data and solution statement, create issue on ScanCode
        • Philippe: Happy to support
        • Rishi: To provide test data support, feedback on problem statement
        • Thomas: Will take coordination from ORT TSC
  • Possible solution:
    • ORT users such as VW provide raw copyrights issues or results from internal research to the ScanCode project so they become aware of issues. More issues/ edge cases reported will in better solution
    • ScanCode to add option to include unprocessed copyrights in the scan results
    • Implement copyright curations within ORT that enable curations based on processed and unprocessed copyrights
    • Fix things upstream in ScanCode, @pombredanne says it possible but lack resources to work on it - he open to contributions/funding
    • Include unprocessed copyrights from ScanCode into the ORT result optionally
      • @mnonnemacher: Would like to see the format coming from ScanCode before inclusion, let discuss this further
  • Philippe reports they are working on copyrights to make it faster so more input they receive it the better they can make it.

2024-04-04

2024-03-28

2024-03-21

2024-03-14

  • Release Notes for version 18.0.0.

2024-03-07 (ORT Community Days Edition)

  • Release Notes for version 17.1.0.

2024-02-29

  • Release Notes for version 17.0.0.

2024-02-22

2024-02-15

2024-02-08

  • Release Notes for version 15.2.0.
  • Discussion about current Docker image build issues (exceeding storage limits; probably arm64 related).

2024-02-01

2024-01-25

2024-01-18

2024-01-11

First weekly meeting in 2024, happy new year!

2023-12-21

2023-12-14

  • Release Notes for version 10.0.0.

2023-12-07

  • Release Notes for version 9.0.0.
  • Please migrate from --skip-excluded CLI options to -P ort.<tool>.skipExcluded=true CLI options for the respective <tool> or configuration options in config.yml.
  • There is a performance problem with the "DefaultDir" package curation provider with file systems mounted over NFS as it walks all files. Ideas are:
    • Create an option for the provider to assume an ort-config-like layout which allows for a direct lookup.
    • Create a new provider which does the same thing.
    • Create an option for the "OrtConfig" provider to read from a directory instead of Git repository.
  • Discussion about whether path excludes should ignore scan timeout errors only for packages or also projects.
    • Probably limit this feature to packages. For project still an issue resolution would be the way tro go.

2023-11-30

2023-11-23

  • Release Notes for version 7.1.0.
    • Hiccup in publishing artifacts to Maven Central, will republish later today.
  • Discuss reasons to use / keep the static HTML report.
    • Helio: Still in use, uses less memory than generating the web app report for large projects.
    • Frank: Static HTML is faster to generate, source code link feature still unique, searching through the whole content is easier as not things need to expand with the mouse.
    • Jens: Faster to get / search the necessary information.
  • No objection to make ReportTableModel non-public.
  • Does the Maven package manager support Eclipse p2 repositories based on Tycho? (@oheger-bosch)
  • Investigate Slack Huddles for future community meetings to give attendees access to text chat, and probably attract more people to join our Slack workspace in general.

2023-11-16

  • Release Notes for version 7.0.0.
  • Reminder: The latest tag for Docker images points to the image build from the most recent Git version tag, while the snapshot Docker tag points to the most recent image built from main.

2023-11-09

2023-11-02

  • Release Notes for versions 5.1.0 and 6.0.0.
  • Creation of new ORT subdomain for Slack http://slack.oss-review-toolkit.org. Issue is that invite link is expired although it shouldn't, instead of updating the invite link into multiple places proposal is to include subdomain everywhere so we only have to update the link in one place. Also subdomain is easier to communicate (@tsteenbe)
  • Epic created to improve ORT's docs https://github.com/oss-review-toolkit/ort/issues/7793. Looking for contributors. (@tsteenbe)

2023-10-26 (skipped)

  • General unavailability of participants (vacation, conferences).

2023-10-19

2023-10-12

  • Release Notes.
  • Discussion about recent major version bumps and how to avoid breaking changes
    • Split out code that is not core to ORT and version it independently (clients)
    • Make more code internal so that API changes are not public / breaking
    • Better document the area of breakage (API, configuration, etc.) similar to like we had for PRs
  • Continue with moving the docs and tutorial directories to the root?
    • @mnonnemacher and @tsteenbe will catch up with each other.
  • People who want to help with organizing the ORT Community Day(s) 2024, reach out to @tsteenbe.

2023-10-05

  • Release Notes.
  • Talk from @vw-anton about Volkswagen's current and future use of ORT.

2023-09-28

  • Release Notes.
  • ORT Docker images published. Align on next steps.
  • Update from @vw-anton about the ORT talk. -> Next week, 🎉!
  • ORT Community Day(s) planned for 6th & 7th of March 2024 at Bosch.IO Campus in Berlin-Tempelhof (FOSS Backstage is 4th & 5th of March 2024)

2023-09-21

2023-09-14

2023-09-07

2023-08-31

  • Release Notes.
  • Plans to soon-ish tag version 1.0.0 of ORT and have weekly releases to Maven Central from then on.
    • There's demand to keep the JitPack releases.
    • JARs go to Maven Central (no wrapper shell scripts there).
    • Distribution archive (incl. wrapper shell scripts) should get attached to GitHub releases.
    • Any Docker image would get published to the GitHub container registry.
  • The community should have a look at https://github.com/oss-review-toolkit/ort-config/issues/132 and chime in on upcoming rules.
  • Initial discussion about how to improve performance for applying license choices.

2023-08-24

2023-08-17 (first meeting after summer break)

2023-07-20 (last meeting before summer break)

  • Release Notes.
  • Discussion about versioning for releases
    • Stick with plain SemVer for simplicity (remembering the date as part of a version if you want to upgrade is hard)
  • Summer break after today's meeting
  • Note about DNF / licnese choise performance issues, see #6721 and #7316

2023-07-13

2023-07-06

  • Release Notes.
  • Are people still there next Thursday? -> @mmurto could give a presentation about dos(-ort).

2023-06-29

  • Release Notes.
  • Anyone having issues using yq on ORT result files?
    • No issues with parsing YAML; e.g. Porsche is using a custom reporter anyway to not rely on ORT's internal result file format 👍

2023-06-22

2023-06-15

2023-06-08

2023-06-01

2023-05-25

2023-05-18 (skipped)

Public holiday in Germany.

2023-05-11

2023-05-04

2023-04-27

2023-04-20

2023-04-13

2023-04-06

2023-03-30

2023-03-23

  • Weekly Release Notes.
  • Feedback on the ORT Community Day:
    • Have dedicated moderation to avoid people taking up too much speaking time / have a more structured discussion
    • Thursday discussion were hard to follow remotely (mixed Jitsi / Teams experience)
    • It was nice to meet people in person / good number of (international) attendees

2023-03-16 (skipped)

2023-03-09

2023-03-02

  • Weekly Release Notes.
  • Status update on the ORT Community Day. (@tsteenbe)
    • Venue will likely be Bosch IoT Campus, Ullsteinstraße 128, Tempelhof, 12109 Berlin, Germany.
    • Need to agree on the schedule, which type of talks or group discussion would people like to see?

2023-02-23

2023-02-16

2023-02-09

2023-02-02

  • Weekly Release Notes.
  • Breaking changes in #6391:
    • Possible solutions to deal with such breaking changes in the future
      • Separate PR label for format changing PRs.
      • Versioned ORT result.
      • ORT releases: #1901

2023-01-26

  • Weekly Release Notes.
  • Is #5629 already sufficiently covered by ProjectSourceRules now?
    • Yes. Anything as specific as checking for inclusice language should be impplemented as an evaluator rule.
  • Discuss giving advice about package health.
    • Upvote the issue to show interest, eventually add comments and remarks.

2023-01-19

2023-01-12

2023-01-05 (first meeting after holiday season)

2022-12-15 (last meeting before holiday season)

2022-12-08

  • Weekly Release Notes.
  • Status of Java 17 in ORT (@oheger-bosch)
    • Also the revert at https://github.com/oss-review-toolkit/ort/pull/4948
    • Ideas:
      • Use custom Docker images
      • Tell the Gradle tooling API to use a different JDK than the one running ORT
      • Just try analyzing the project with a newer version of Gradle than specified in the wrapper
      • Projects should be able to specify their required JDK via Gradle's toolchain mechanism
        • But that probably won't help as the JVM running the Gradle build would still be ORT's
  • Status for the GitHub Action:
    • @tsteenbe is working in Markdown stage summaries
    • Please test the action and give feedback

2022-12-01

  • Weekly Release Notes.
  • Show number of resolved vulnerabilities in web-app report (@mnonnenmacher).
    • Discussion about whether we should keep the feature to resolve vulnerabilities independently of rule violations at all.
      • Yes, because:
        • people might use ORT without the evaluator,
        • want to have a way to resolve unapplicable vulnerabilities even if they don't trigger rule violations.
  • Remove bootstrapping of scanners (see this for some context, @sschuberth).
    • Let's do it. For running funTests, scanners need to be either installed on the GitHub action runner, or in the Docker image and also scanner funTests need to use Batect to run against the Docker image.
  • Update on GitHub Action for ORT (@tsteenbe).

2022-11-24

  • Recently merged PRs.
  • Declared license mappings: Only take 100% indisputable ones?
    • https://github.com/oss-review-toolkit/ort/pull/3960
    • https://github.com/oss-review-toolkit/ort/pull/5127
    • Conclusions:
      • Aim to only have 100% unambiguous mappings
      • Make exceptions only for cases where we are very certain that a specific version of a license is meant (despite it being mentioned explicitly), e.g. because other versions of that license existed only for a very short period of time (e.g. Apache-1.0, LGPL-2.0-only)
        • All exceptions need to come with comments in the YAML file
        • If the version is missing in the declared license, usually map to -only, unless the license itself defines that omission of a version means "or later"
      • Package-specific mappings should be added as package curations with declared license mappings to the ort-config repository
      • Propopsal: Create an new initial PR that just separates existing entries into sections, one for 100% unambiguous mapping, and others (for both declared license and simple mappings)

2022-11-17

2022-11-10

2022-11-03

2022-10-27

2022-10-20

2022-10-13

  • Recently merged PRs.
  • Quick discussion about how to provide pip-related configuration (e.g. credentials for private package registries) with the new Dockerfile. -> Make sure to use a login shell.
  • Closing early due to no more open topics.

2022-10-06

  • Recently merged PRs.
  • Experimental scanner will be merged later today
  • Discussion about renaming the "ORT Developer Meeting" to something less developer-specific, as attendees are not exclusively developers
    • Currently exsiting meetings:
      • ORT Developer Meeting
      • TSC Meeting
      • Kotlin Core Developer Meeting
    • Proposed new meetings:
      • ORT Community Meeting (regular former "ORT Developer Meeting", deal with more specific topics towards the end)
        • Agreed on the renaming
      • ORT Drop-in Hours (irregular, for users from different time zones that want to join the community)
        • @tsteenbe will do shout-out on Social Media to figure out which hours we need
      • TSC Meeting (irregular, about every 1-2 month)
      • Kotlin Core Developer Meeting (regular, once a week)
    • @mnonnenmacher will create a new Teams invite to rename the ORT Developer Meeting to the ORT Community Meeting
    • Probably need to look at alternatives to Teams again, @tsteenbe already has a Jitsi channel

2022-09-29

2022-09-22

  • Recently merged PRs.
  • Multi-stage Docker status
    • @mnonnenmacher will take over the PR to enhance commit messages.
  • Discussion about splitting out examples to ort-config
    • When "disconnecting" examples from ORT core they might break due to ORT changes
    • Proposal: Keep simplifed examples in ORT core, and "real world" examples in ort-config
    • We don't want to have redundancy between examples from different locations
    • Proposal: Move current (simple) examples in ORT core to funTests / assets only, have "real world" examples in ort-config
    • Keep in mind that the evaluator-rules and notifications direct set up Gradle "wrapper projects" to get auto-comnpletion in ORT scripts

2022-09-15

2022-09-08

2022-09-01

  • Recently merged PRs.
  • Update on publishing Docker image
    • @nicorikken @heliocastro and @tsteenbe are working on creating a ORT "sources" image e.g. the complete corresponding source code to satisfy open source license obligations.
    • We are currently meeting on a weekly basis directly after ORT dev meeting
    • Our work is using https://github.com/oss-review-toolkit/ort/pull/4746 as a base
    • @marcelbochtler and @mnonnenmacher we hereby notifying Bosch of our intent to merge as agreed in earlier ORT dev meeting - please prepare Bosch infrastructure to use DockerFile from docker/legacy/Dockerfile if you wish to continue using the old image
    • @bennati PR to update ORT for GitLab to make Dockerfile location configurable https://github.com/oss-review-toolkit/ort-gitlab-ci/pull/23
  • Let's talk about Excel reporters
    • Do we still need the original ExcelReporter?
    • If only "some" Excel report is needed (the original Excel format was "made up" after all), can we replace it with the AOSD-compatible one from https://github.com/oss-review-toolkit/ort/pull/5518?
      • First investigate if there is a JSON -> Excel converter (maybe Pandas, Dasel) that could fulfill the same purpose, and convert an ORT analyer result in JSON format or the evaluated model format to the Excel format that AOSD expects.

2022-08-25 (first meeting after summer break)

2022-07-28 (last meeting before summer break)

  • Recently merged PRs.
  • OSB Alliance / Sovereign Cloud Stack funding for security tooling planned
    • @sschuberth will schedule a TSC meeting with @itrich if funding is indeed going to happen
  • Discuss a new reason to eventually switch the logging framework: Log4j2 does not work with Graal native-images.
    • SLF4J for the API should work.
    • Logback for the implementation should work.
    • No objections from the community to move forward with a migration.
  • Consider consolidating NOTICE_default and NOTICE_summary to something that makes @LeChasseur happy 😉
    • @sschuberth will come up with a proposal
  • Merging of ORT results aka "I want a single report for my microservices repos"
  • Extend the Evaluator to support "repolinter style" rules https://github.com/oss-review-toolkit/ort/issues/5621
    • Maybe record all scanned files (not only those with findings) in scan results to avoid the need to clone the source code again in the evaluator stage
      • This would also allow us to get rid of the pre-calculated SPDX package verification code if we also record the SHA1s of all scanned files
    • Consider to first finish @mnonnenmacher's rules DSL refactoring and generalize RuleViolation class to not be tied to licenses
  • The ORT developer community will take a summer break until 2022-08-25 😀

2022-07-21

  • Recently merged PRs.
  • @tsteenbe: @heliocastro I did not work on continueing multi-stage docker PR 4746 as I was out with food poisoning
  • Separate authors and copyright holders @tsteenbe: Set up a dedicated meeting for
    • Curating detected copyrights in file findings: add, remove, map to license
    • Mapping concluded copyrights to concluded licenses
    • Deriving declared copyright holders
  • Action item for @tsteenbe Write email on @sschuberth @porsche-rbieniek @fviernau @rishi @marcelbochtler to explain option to curate copyright holders plus doodle for a meeting. Doodle link https://doodle.com/meeting/participate/id/er03A92b
  • Question where scanning R project is supported @tsteenbe: Yes, it should work but it will be listed Unmanaged, if you want to add package metadata I recommend you add package.spdx.yml to the project.

2022-07-14

Agenda

Separate authors and copyright holders

@porsche-rbieniek: Thank you @fviernau for doing a review, I see the PR is approved now what does it take to get it merged @tsteenbe: I asked @fviernau not to merge it as I wanted to review the behavior/signature myself. @fviernau: I see two issues with this PR but non-blocking its merger, as I presume they will be fixed in the future:

  • Currently authors-based copyrights can only be completely overwritten using curations instead being able to add and remove copyrights entries
  • Existing bug scanner code assumes the addToCopyrights() innnns provided to evaluator and reporter @tsteenbe: What I believe the workflow this PR is enabling is "detected copyrights a,b,c -> manual analysis copyright d is applicable so via this PR one enable creation to say package curations with copyrights a,b,c,d". This is inconsistent with how ORT works, curations in curations.yml are fixing package metadata only and package configurations should be used for fixing source code finding. We know concluded license is in curations.yml but we had discussion amongst ORT TSC members that we should move concluded_license out of curations to make things more consistent and easy to understand. @porsche-rbieniek: Below an example of curation we have:
- id: "NPM:estraversa:4.3.0"
  curations:
    comment: "Add copyrights for subfile"
    copyright_holders:
       - Copyright (c) 2009-2019 by the contributors listed in CREDITS.TXT
       - Copyright (c) 2009-2014 by the contributors listed in CREDITS.TXT, see https://github.com/llvm/llvm-project/blob/main/...
  vcs:
    type: "git"
    url: "https://github.com/estools/estraverse.git"

@tsteenbe: This estraversa curation is incorrect, the package.json per its reference only support an author field (single person) and contributors (multiple people) field see https://docs.npmjs.com/cli/v8/configuring-npm/package-json#people-fields-author-contributors @tsteenbe: As you can see in https://registry.npmjs.com/estraverse for this package no copyrights have been declared. The "Copyright (c) 2009-2019 by the contributors listed in CREDITS.TXT" in this curation refers a finding in source code so this should be done so adding the copyrights should be done with package configurations, not a curation @tsteenbe: As copyrights are associated with licenses it makes sense implementation of adding, removing, mapping, overwritting declared copyrights matches declared licenses @mnonnenmacher: Agree, one first step could be to update PR 5504 to rename copyrightHolders to declaredCopyrightHolders so it aligns with declaredLicenses @porsche-rbieniek: Please add a comment to PR 5504 to inform @porsche-rbieniek

Discussion between @fviernau, @mnonnemacher, @tsteenbe and @porsche-rbieniek on various ideas how curating copyrights could look like that is consistent with how licenses are currently curated.

Three ways to curate copyrights:

A. Use curations to fix-up curate copyrights

The declared copyrights comes package metadata collected by ORT analyzer which can be fixed up using curations. Note that a lot of package managers do not have copyrights fields only author, contributors, developers which in some case are "mis-used" to convey copyright information. We could implement a parseCopyrights() to from parseAuthors() find copyright statements e.g. entries starting with "copyright" or "(c)"

Below several ideas for how curating declared licenses in curations.yml could look like.

Remove a declared copyright

declared_copyright_mapping:
   "MIT": ""

Add a declared copyright

declared_copyright_mapping:
   "": "Copyright (C) 2022 John Doe"

Overwrite all declared copyrights with a single one:

declared_copyright_mapping:
   "*": ""
   "": "Copyright (C) 2022 John Doe"

Overwrite all declared copyrights with a more than one:

declared_copyright_mapping:
   "*": ""
   "": "Copyright (C) 2022 John Doe"
   "": "Copyright (C) 2019 Jane Doe"
   "": "Copyright (C) 2012 Example, Inc"

Associate declared licenses with declared copyrights:

declared_license_copyrights_mapping:
   "MIT": "Copyright (C) 2022 John Doe"
   "MIT": "Copyright (C) 2019 Jane Doe"
   "Apache-2.0": "Copyright (C) 2012 Example, Inc"

B. Use package configurations to curate detected copyrights

Introduce a copyright_finding_curation in package configurations to curate detected copyrights:

id: "NPM::ansi-styles:4.2.1"
copyright_finding_curations:
  - path: "README.md"
    start_lines: "3"
    line_count: 11
    detected_copyright: ""
    reason: "INCORRECT"
    comment: "Copyright only written on project website."
    concluded_copyright: "Copyright (C) 2022 John Doe"

d. Associate detected licenses to detected copyrights in a package configuration

id: "NPM::ansi-styles:4.2.1"
license_copyright_finding_curation:
  - license: "MIT"
    copyrights:
       - "Copyright (C) 2022 John Doe"
       - "Copyright (C) 2019 Jane Doe"
       - "Copyright (C) 2012 Example, Inc."

C. Use concluded copyright to overwrite both declared and detected

Introduce the concept of a concluded copyright? Should we introduce this, believe concluded_license should be removed from curations as it applies to both declared and detected licenses

concluded_copyright: "Copyright (C) 2022 John Doe"

Conclusion of the discussion

Update the PR 5504 per below listed points, Porsche to accept signature likely will change in the future.

The following action points

  1. Rename copyrightHolders to declaredCopyrightHolders so it aligns with declaredLicenses (@tsteenbe)
  2. Fill an issue to address the existing bug where scanner code assumes the addToCopyrights is provided to evaluator and reporter. See also https://github.com/oss-review-toolkit/ort/pull/5504#discussion_r915825030 (@fviernau)
  3. Organize a workshop to discuss curating both declared and detected copyrights, fill tickets afterwards once we agree on solution. (@tsteenbe) Topics to be discussed:
    • Signature to use to add, remove, map and overwrite declared copyrights and licenses
    • Signature to use to add, remove, and overwrite detected copyrights
    • Association of licenses and copyrights
    • Add a license finding to a file using a package configuration
  4. Fill an issue to move concluded_license out of curations, possibly into it's own config file (@tsteenbe)
  5. Fill an issue to introduce concluded_copyrights to be aligned with concluded_license. (@tsteenbe)

2022-07-07

2022-06-30

2022-06-23

  • Recently merged PRs.
  • Ongoing work from @fviernau to improve GoMod support
    • Almost done, some VCS path issues left
    • Need to ensure that changes to expected results are actually correct
  • Discussion about https://github.com/oss-review-toolkit/ort/issues/5483
    • Seems like we have conflicting use-cases for SPDX document files to define project / packages
    • Need to rework the logic that distinguishes between SPDX document files that describe a project vs. a package
    • @tsteenbe will come up with a list of SPDX document scenarios that ORT needs to support
  • Discussion about an NPM / Yarn problem with its resolutions directive
    • Requires more debugging

2022-06-16

2022-06-09

2022-06-02

2022-05-26 (skipped)

  • Meeting was skipped due to a German holiday.

2022-05-19

  • Recently merged PRs.
  • Discussion about the PR to move away from Log4j (@porsche-rbieniek)
  • Idea to allow configurable Regex-based rewriting of URLs in the Downloader to cope with funky corporate proxy requirements, like prefixing URLs of changing protocols (@heliocastro)
  • @sschuberth will plan for a "contributor training" meeting

2022-05-12

  • Recently merged PRs.
  • Update CONTRIBUTING.md with more details about PR workflow; maybe also have a separate meeting for new / "young" contributors.
  • @pombredanne gives outlook for new ScanCode version (to be released in a month from now)
    • Ask for testing on Apple silicon (@MarcelBochtler will do)
  • @tsteenbe has another meeting with SVT tomorrow (they have an interesting workflow)

2022-05-05

  • Recently merged PRs.
  • BMW Car IT gives demo about their ORT process and technical implementation.

2022-04-28

2022-04-21

  • Recently merged PRs.
  • Updates from Porsche
    • Update on ongoing BlackDuck integration. (@porsche-rishisaxena)
    • Consider getting rid of Log4j because of its bad reputation by now, and technical issues with the BlackDuck integration. Proposal is to use Logback. (@porsche-rbieniek)
    • Clarified on misunderstandings about https://github.com/oss-review-toolkit/ort/issues/4519, there's still interest from Porsche to contribute this. (@porsche-rbieniek)
  • Taking scan-by-repo into use (@mnonnenmacher).
  • Make use of upvoting issues.
  • Use logos of ORT users into README.md / a dedicated landing page.

2022-04-14

  • Recently merged PRs.
  • Discussion about credential management in ORT
    • In the analyzer, we need to pass credentials to the respective package manager (e.g. to NPM via .npmrc)
    • In the downloader, we also need to have the same set of credentials, but for OkHttp
    • Where to store credentials in a central place, from where package manager specific / OkHttp authentication could maybe generated?

2022-04-07

2022-03-31

2022-03-24

  • Recently merged PRs.
  • Porsche gives demo about their ORT process and technical implementation.

2022-03-17

  • Recently merged PRs.
  • Discussion about the Python version problem
    • It's rather complex and manifold, thus no one is actively working on it, although several users are affacted
    • Work-around: Use a custom Docker image with the required Python version included
  • Discussion about ScanCode false-positives again
    • @porsche-rishisaxena will set up a dedicated meeting with @pombredanne to ask what more data he needs to be able to address the issue

2022-03-10

  • Recently merged PRs.
  • Need to revise the logic to generate unique SPDX ids in order to add transitive relationships.
  • Proposal to agree to a fixed directory structure for curation files (see OrtConfigPackageCurationProvider.kt).
  • @tsteenbe is leaving HERE Technologies by end of March (new employer to be disclosed 😉, but will still be working with ORT)

2022-03-03

  • Recently merged PRs.
  • Problem with automatic numbering of SPDX packages (see https://github.com/oss-review-toolkit/ort/pull/5126)
    • Skipped as no one from HERE attended.
  • Call for sharing of package configurations / license findings curations with ScanCode.
  • How does Porsche implement the upload of pre-created analyzer results from the project teams?
    • Porsche will give a demo on 2022-03-17.
  • ScanCode will support a (long) list of files to be included for a scan (instead of specifying files to exclude from a tree).
    • See:
    • Background: Incremental scans in ORT.
    • Idea: Use a daemon to speed up incremental runs of ScanCode, similar to Gradle.

2022-02-24

2022-02-17

  • Recently merged PRs.
  • Check on https://github.com/oss-review-toolkit/ort/issues/4196 again.
    • Idea: Feature toggle for both curation and configuration version ranges; enable both by default. Ping @fviernau again about this.
  • Wild-cards for package curation's namespaces / names to tackle NuGet::Microsoft.NETCore.Platforms:1.0.1 etc.
    • HERE uses (wildcard) issue resolutions to suppress downloader issues / policy violations. In notice generation custom text based on SDK usage is inserted.
    • Idea: Maybe allow wildcards (if enabled) only in global curations, but not in project-local ones?
    • Idea: Introduce "alias names" / placeholders to generally trust SDK vendors.
  • High number of LicenseRef-scancode-* false-positives in ScanCode 30.1.0.
    • Assemble a list of false-positive keys (and where they occurred) for @pombredanne (ideally also the matched rule).
    • False-positives likely to go down when taking the new main / primary license feature into use (probably not in next stable release yet).

2022-02-10

2022-02-03

  • Recently merged PRs.
  • HERE will be submit declared license mapping it found as after scanning all of its code repositories (inbound next week)
  • @porsche-rishisaxena: Missing API functionality in SW360 for the AttachmentController to upload attachments (no DELETE / POST implemented). Does someone else already implemented this functionality? If not Porsche will work on it. @heliocastro documentation is not up to date with the code. SW360 has developer call every Wednesday 10 AM via https://meet.jit.si/sw360-community and Slack channel
  • @porsche-rishisaxena having issue with large analyzer result file where it runs ORT runs out of memory. @fviernau do you pass -Xmx? We in HERE use JAVA_OPTS="-Xmx64G". @rbieniek What I would like see is rule of thumb if I have Analyzer results then it's recommend to have 4, 8 GB. @oheger-bosch we have pipelines with different t-shirt sizes s, l, xl with memory sizes machines doing scan. Action item for @rbieniek to create an issue describing the issue and if possible add numbers (nr of packages vs nr of package references), @heliocastro suggested to use https://github.com/bmwcarit/joynr/ as a test case. Action item for @fviernau for scan joynr to see if he can recreate the issue and add outcome to ticket created by @rbieniek.
  • SPDXDocument files does not support downloaded from PackageDownloadLocation, @tsteenbe and @heliocastro need this functionality. @oheger-bosch Sebastian is working on integration of SPDX with other package manager. Define a SPDX and reference other projects that use a package manager.
  • @heliocastro In the ORT conf file we have multiple time the same configurations e.g multiple same entries for scanner and archive configuration. See https://github.com/oss-review-toolkit/ort/blob/master/model/src/main/resources/reference.conf it has multiple sw360Configuration. @oheger-bosch we use secrets via environment variables, @rbieniek recommends against doing secrets via environment vars.

2022-01-27

2022-01-20

2022-01-13

  • Recently merged PRs.
  • Should we support scanning local directories that are not under version control? (E.g. by introducing a LocalFileProvenence.)
    • Sometimes source code from inaccessible repos needs to be analyzed / scanned, and then the source code is sent over in a ZIP file beforehand.
    • Idea: Let a LocalFileProvenence record absolute paths, requiring to run analyzer and scanner on the same machine.
      • @sschuberth will create issue for this feature.
  • Change to a generic Copyright header in ORT source code
    • Create issue for people to comment / signoff / consent to the generic Copyright
    • Maybe use a NOTICE file to globally collect Copyright entities

2022-01-06

2021-12-16

2021-12-09

2021-12-02

2021-11-25

2021-11-18

2021-11-11

  • Recently merged PRs.
  • Sharing of updates regarding making the ScanCode version configurable (#4523 will hopefully be merged today).
  • Discussion about organizing our work in the common roadmap (added an "Organization" column; Porsche plans for BlackDuck integration in Q1 2022).
  • Create a public high-level roadmap for contributing parties to align. (@sschuberth)

2021-11-04

  • Recently merged PRs.
  • Should severeIssueThreshold also apply to RuleViolations? (#4642, @sschuberth)
    • No, introduce severeRuleViolationThreshold.
  • How to map vulnerabilities to Evaluated Model stats? (https://github.com/oss-review-toolkit/ort/issues/4663, @tsteenbe)
    • No mapping of vulnerablites per se, list as is, and additionally create rule violations if you want vulnerabilities to be acted upon.
  • Should config file CLI options be moved to the root command to reduce repetition? (@mnonnenmacher)
    • Let's do it: Move common configuration override options to ort.conf, implicitly allowing to override via -P (which should be documented in OrtMain's help output).
  • Align on WoW for new features- propose we clarify new features and their context with issues prior to PRs being created (@tsteenbe)

2021-10-28

2021-10-21

  • AsciiDocTemplate Reporter
  • General way of working
    • Problems:
      • Breaking changes are merged, and not properly documented
      • Lack of transparency outside of Bosch and HERE.
    • Proposals:
      • Align on what is the expected behaviour of ORT.
      • Create principles document.
      • Document what we want to implement and what we don't want wo implement (and in include the why).
      • Draft a Roadmap in a dedicated roadmap meeting (@tsteenbe)
      • Create issues and discuss the proposal for all non-trivial PRs.
      • Improve documentation regarding the "unspoken functionality".
      • Create and maintain a changelog.
      • Commit messages do not document the "bigger picture".
    • Outcome:
      • Create working group (3-4 people / 1 per company) to draft a document for "ways of working on ORT".
      • Thomas, Helio, Sebastian, somebody from Porsche, Nico
      • Create an issue for this topic and set up meeting for all the nominated persons. (https://github.com/oss-review-toolkit/ort/issues/4629, @tsteenbe)
  • Epic for ScanCode upgrade needs to be created.
  • Blackduck integration
    • Issue has to be created first.

2021-10-14

2021-10-07

  • How to deal with C/C++ packages that are used with different compile flags depending on the project, and thus need different curations based on the project context? (@sschuberth)
    • One approach is to use separate ort.yml files for each variant of the project.
    • Use build tool like Conan to provide information about different product variants. (@heliocastro)
  • Demo of the Opossum integration (#4533). (@maxhbr)
    • Currently runs ScanCode in parallel to ORT to capture information from the ScanCode result that is not captured by ORT, for example the full list of files or the content of archives. There are already plans to add at least some of that information to ORT as well. @maxhbr to create an ORT issue for further discussion of the topic.
  • Allows curating the PURL, that would also be a useful feature for ORT (https://github.com/oss-review-toolkit/ort/issues/4565, @tsteenbe) -> Yes, makes sense.

2021-09-30

  • Porsche is testing the helper-cli tool to merge analyzer results (@porsche-rishisaxena)
  • Where to put additional copyright holders, PackageCurationData or PackageConfiguration? -> Preference for PackageCurationData to have it next to authors.
  • Clarify in the OpenChain community that OppossumUI is not an (ORT) server frontend, but "just" a tool to browse results.
    • What is @maxhbr referring to if he claims that ORT run by ScanCode does not capture the same information as running ScanCode directly?
    • October 13th, we do a deep dive on using ORT via the tool + deep dive into ORT internals engineering.
  • Sum up the issues we potentially see with a ScanCode upgrade to finally get it done. (@mnonnenmacher)
  • Should the developer meeting be held weekly? As we never manage to talk about all agenda items. (@MarcelBochtler)
    • Yes, let's do weekly meetings.
  • Rename master to main? (@sschuberth)
    • Yes, plan for doing it. -> Create issue and pre-announcement. (@sschuberth)
  • Use originator as the "author" in SpdxDocumentFiles? (#4058, @sschuberth)
    • Yes, @tsteenbe agrees to do that, for both person and organization (not supplier).

2021-09-16

  • Update on merging result files (#4364, @porsche-rishisaxena)
    • Porsche-internal testing is on-going, will file a PR soon
  • Discuss a "delta-analysis" feature (#4347, @porsche-rishisaxena)
    • On hold for in in favor of re-creating AOSD statements anew each time
  • RepositoryConfiguration naming (@sschuberth)

2021-09-02

2021-08-19

  • BlackDuck Hub integration (@sschuberth)
    • Porsche (and probably Cariad) would be interested, and could eventually do the contribution
  • SARIF file support (@heliocastro, @nicorikken)
    • Proposal is to make SARIF the native ORT format instead of its own YAML files -> needs research if technically possible (e.g. how to represent dependency graphs)
  • @tsteenbe announces coming new repo to showcase how to do full GitLab license management integration
    • @tsteenbe, please add links here to public GitLab issues to upvote
  • Number of Unique Packages for Violations in Web App Report, see https://github.com/oss-review-toolkit/ort/issues/4017 (@sschuberth)
    • Attracting front-end developers via OpenChain / ACT?
    • Use Kotlin to generate JavaScript / React code?
  • Re-introduce Reviewable as it can review commit message now? See https://github.com/Reviewable/Reviewable/issues/507. (@sschuberth)
    • Probably not, stick to current workflow for simplicity.
  • Possibility to waive license rule violations of 3rd party packages on project level (@MarcelBochtler)
    • Currently neither possible with resolutions nor with curations.
    • How could such a feature look like?
      • Project-local curations and package-configurations in .ort.yml.

2021-08-05

  • Breaking changes to FossID integration (@nnobelis) -> Ok to do breaking changes, no one except for Bosch is using FossID integration currently
  • Updates on various topics:
    • Scan-by-repo (store scan results by repo instead of by package, @mnonnenmacher)
      • No progress recently due to vacation time
      • Finished by end of August
    • ScanCode upgrade (to version 21.7.30, @fviernau)
      • Python version issue
      • Scan result compatibility issue
      • Finished by end of next week
  • Ideas
    • Extend the requirements command to not check the version (range), but really for "parsability" of the output generated by the tool
    • Rework bootstrapping: Allow to disable it completely, clearly separate accepted versions from the version to (eventually) bootstrap, align the pub / flutter bootstrapping.
    • Allow to only run findManagedFiles from CLI
  • Configure the mapping of authors (https://github.com/oss-review-toolkit/ort/issues/4314, @sschuberth)
    • Basic agreement on the feature, need to configure applicable licenses.

2021-07-22 (skipped)

  • Meeting was skipped due to vacation time.

2021-07-08

2021-06-24

2021-06-10

  • ORT curations

    • How can we "automatically" reuse curations with concluded licenses for (minor / patch level) version upgrades of the same package? -> HERE is not fine with extending package configuration to version ranges due to potential misuse / risks. -> The general topic of storing configuration data like curations and / or package configurations in a DB should be discussed in the context of a potential "ORT backend" where storing things in a DB becomes a topic naturally. -> Again this is a topic that could be solved by a UI... either a local UI or a web frontend UI -> @tsteenbe agreed to port the feature to link to the source code of detected license findings from the Static HTML reporter to the Web App reporter

  • Issues

2021-05-27

  • originalVcsInfo
  • resolvedRevision
    • Analyzer should be able to set the resolvedRevision, by reading the meta data from an existing lockfile.
    • Revision field might be symbolic but can also be fixed. resolvedRevision is guaranteed to be fixed.
    • Downloader needs to be changed to only download from the resolvedRevision (remove fallback).
    • Changes will be made in the "scan-by-repository" change.
    • Maybe rename resolvedRevision to make clear that it is the fixed revision.
  • Concluded license in nested SPDX projects

2021-05-20

  • PRs
    • Clean-up old PRs -> everybody please clean up
    • Also clean up remote branches -> everybody please clean up
    • Who looks at "external" PRs? -> assigned PRs
    • In the future, do a triage of external PRs as part of the developer meetings
  • Do not insist on data-class-deserialization vs iterator-based JSON / YAML (at the example of the Cocoapods PR)

2021-04-29

2021-04-15

  • Code Style
    • Should we start using trailing commas? (@mnonnenmacher)
    • Static private helper functions / properties: Put in companion object or top-level (before or after the class)? (@sschuberth)
      • Need to find out which syntax is better callable from Java. Do we need @JvmStatic?
      • Public top-level functions pollute global namespace.
      • Better discoverability / auto-completion for companion object functions.
      • For now, try: Static functions that are public go to companion object / top-level objects, private ones go to top-level.
  • PR discussion
    • Make analyzer results non-nullable in ORT result files? (@sschuberth)
      • @fviernau will also review. @sschuberth evtl. needs to reword commit message for clarity.
  • ORT curations
    • Move to Postgres? (@sschuberth)
      • HERE has no interest as their workflow is code-review based (Gerrit, GitLab)
      • Rather split into multiple files, like for PackageConfigurations
      • Bosch might add a Postgres provider for curations

2021-03-18

  • Talk about ways to merge / create unified reports for projects (in the project management sense, not in the ORT sense) that span multiple repositories. (@sschuberth)

    • HERE does not really have that problem currently.
    • Idea: Artificial "parent" Git repo + submodules, or git-repo manifest.
    • Idea: (Helper-)command to merge ORT (scan) results before passing to reporter. Unclear how to resolve potential conflicts, though.

  • Should we implement adding actual and expect files to functional test results in Azure? Azure cuts of big diff in its reports making it difficult for users without a full local setup to determine why a function test is failing. (@tsteenbe)

2021-03-04

  • Decide a format for user's license choice configuration. See: #3396 (@MarcelBochtler, @neubs-bsi)
    • Clarified that input for choices should be only one expression, not multiple.
    • Multiple choices should be incremental, i.e. output of first choice is input for second choice.
    • given field is optional, if not specified mean "complete expression".
  • ORT notifications: How to notify users? Idea: Create notification module and generalize evaluator to "post-processor" than can perform "any" action, e.g. also send mails. (@sschuberth)
    • Agreement that the idea is good and fits into ORT.

2021-02-18

  • Best practices for code review:

    • Author documents with 👍🏻 whether a review comment was addressed without any further discussion needed. For simple changes the author may as well resolve the conversion, but for more complex discussion the reviewer should resolve to confirm that the change was addressed as intended. If the author addresses the review comment differently, a comment should be added, and the reviewer must resolve the discussion.
    • Subsequent reviews need to be explicitly requested via the "re-request review" button. (As pushing of new commits does not necessarily mean the PR is ready for another review.)
    • Enable Slack notifications for lacking reviews (separate "notifications" channel).

  • Move the declared authors PR forward:

    • Rename declaredAuthors field to authors and put it between purl and declaredLicense.
    • @fviernau will do follow-up PR to remove declaredLicenses from PackageCurationData as declaredLicensesMapping is sufficient.

2021-02-04 (first meeting ever)

  • Bosch topics:
    • Agreed to address curating copyright holders for now via PackageCurationData, which means we need to start capturing author information from package managers.
  • HERE topics:
    • Plans to publish a PR to introduce "dynamic" How-to-Fix-hints within the next weeks.
    • Publish a new DSL for rules by end of February.
 ______________________________
/        \_______   \__    ___/ The OSS Review Toolkit, version 1.0.0.
|    |   | |       _/ |    |
|    |   | |    |   \ |    |    Running 'wiki' as 'ort' under Java on GitHub
\________/ |____|___/ |____|    with a lot of CPUs and a maximum amount of memory.

Community

Clone this wiki locally