-
Notifications
You must be signed in to change notification settings - Fork 219
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove Coveralls support / Coveralls setup broken #1463
Comments
I vote to remove Coveralls service and account because I see no value in it and there is no one who want to take care of it. Coverage as a measurement has its values but could be done locally on the machine of each developer. The maintenance cost is high. In regards to #1489 I investigated and modified the make system. I discovered that there is some exceptional code handling coveralls on travisci. Without coveralls the travisci exceptions could be removed and the code is easier to maintain. Maybe there is away to remove the travisci-exception code from configure-make but keeping coveralls. But for that we need someone who understand coveralls and travisci configuration. |
If it's unmaintained and causes complexity, then I agree with removing it. |
It seems that we have admin rights to the coveralls project. So we can delete it our self. But I'll contact Germar first via mail. |
Just change like this will fix the badge
Like I said in #1553 (comment)_ I used coveralls.io to monitor the impact of tests. For me coveralls.io always worked flawlessly in background. These three lines do the trick: Lines 50 to 52 in 6df8c90
|
I am still not convinced and vote to remove coveralls support. To "monitor the impact of tests" someone can clone the repo and run coveralls on their local machine. I see no value in doing this in public. In my understanding based on my experience and studying of that topic the coverage of productive code by testing code only counts for unit tests. Integration and system tests do not count because they do not test one isolated behavior (sometimes misinterpreted as a block of code). They always test multiple behaviors at once. Currently there is no pure unit test in BITs test suite. Because of my previous argument the coverage value is misleading. The code is nearly uncovered. One intention of unit tests is to make it secure to do modifications in the code. I need to be sure that the modification of one line won't affect other parts of the code and its behavior. In the current situation I can not be sure. |
I think
I also don't expect much maintenance overhead since So I would like to keep |
I am aware of the marketing aspect and also like it. But the cost is higher then the benefit.
In the current state it hurts because the number is incorrect even if we count our tests as real unit tests. I am not sure but it seems to me that the coverage calculation do take the unit test code and its coverage also into account. And we need someone to maintain it. Currently there is no one. The coveralls.io is stupid. Example: Top right "dev" branch is selected. But the list shows a pull request that belong to another branch. And "0 of 0 relevant lines covered". WTF!? I can not find a list of files and their values. It is unclear/nontransparent where the current "75%" comes from. I assume that this value comes from a very old build long ago in the past. The UI is not only bad it is broken. And I don't want to invest more time into it. EDIT: I triggered a manual build of the As you can see here. The test files are included in the calculation and the EDIT2: Even on my local machine it is not possible to generate an evident coverage report because of our project setup. See #1575 . Partly "code coverage" (not test coverage) just in
Be aware that some *.py files with productive code ( Interpretation of the result: The value "63%" is optimistic because some files with productive code are missing. So make it 55% I would say. But just for Using coveralls.io is IMHO not an indicator of quality in our case. It is the opposite. EDIT3: Just for your information here an Issue-based discussion about why EDIT4: As more I get into this topic (e.g. my tec demo) and the unexpected behavior of coverage.py I come to the conclusion that coverage values of most of the projects out there are just wrong because coverage.py is not correct configured and its default behavior is useless in context of code quality and test coverage. |
There have been no new arguments on this matter since it was last discussed here. Since this blocks the upcoming release, the active maintainer team has made a decision to remove coveralls from the current project. After migration to modern python packaging standards (#1575) |
I'm sorry I have to open this as an Issue because I don't know whats going on here. It seems that the setup on https://coveralls.io/github/bit-team/backintime is not correct. I do have admin rights (based on my bit-team membership). But I don't understand that UI. So I hope that Germar, Aryoda or someone else can help out here.
The badge on README.md do point to
master
which should bedev
of course. Easy to fix. But I'm not sure if or how coveralls can check for "dev" instead of "master".I'm not sure how coveralls get's its data but I know there are some coveralls-calls in our TravisCI build setup. Not sure if this is bound to a specific branch.
The landing page looks like this to me and confuses me a lot
In the top right you see "dev" with 50% and also "dev" as default branch. But in the bottom there is another branch.
The branch-dropdown-menu do not offer me "dev" even after clicking on "sync branches".
I checked the settings but before I break something I thought it is better to ask.
Related to #1282
The text was updated successfully, but these errors were encountered: