CI: Linting with azure instead of travis #22854

datapythonista · 2018-09-27T13:28:20Z

Moving all the linting, other code checks and doctests to azure.

TomAugspurger

I'm interested to see what the reporting looks like. Could you introduce a few changes that fail each one?

In particular, we want all of the steps to run, even if an early one fails.

For checks longer than a line, I think making a collections of scripts in ci/lint/ makes sense. Then in the azure config we call - script: ./ci/lint/sphinx_directives.sh.

TomAugspurger · 2018-09-27T13:32:26Z

azure-pipelines.yml

+  #C410   # Unnecessary (list/tuple) passed to list() - (remove the outer call to list()/rewrite as a list literal).
+
+  # pandas/_libs/src is C code, so no need to search there.
+  - script: flake8 pandas --filename=*.py --exclude pandas/_libs/src --ignore=C406,C408,C409,E402,E731,E741,W503


Is this --ignore necessary? I would think that this would be picked up from our setup.cfg. Better to delete it here so that we don't have to update two places.

That's a good point. That was in the original lint.sh, but will check if the setting in setup.cfg is used.

datapythonista · 2018-09-27T13:46:16Z

Looks like this requires a bit more research. I'm not activating the conda environment, so I was guessing whether this would fail or not. And it seems it does.

This answers your question. Defining them as steps, when one fails, the job is interrupted. An alternative would be to create a job for each, but I think that will create too many jobs, and it'll make it difficult to find the jobs of the tests. Will do some research and see if it's possible to continue the steps when one is failing.

I think yaml should support having scripts with multiple lines, I'll give that a try. I think that should be neater than having a script for each check with a loop.

TomAugspurger · 2018-09-27T13:51:08Z

https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts#job

Looks like there's a continueOnError field that determines this. That can be set at the job or script level. We'll want true.

…s that used to be loops). Installing missing packages, splitting linting and checks, and continuing on errors

codecov · 2018-09-27T14:47:47Z

Codecov Report

Merging #22854 into master will decrease coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #22854      +/-   ##
==========================================
- Coverage   42.45%   42.44%   -0.01%     
==========================================
  Files         161      161              
  Lines       51561    51559       -2     
==========================================
- Hits        21888    21886       -2     
  Misses      29673    29673

Flag	Coverage Δ
#single	`42.44% <100%> (-0.01%)`	⬇️

Impacted Files	Coverage Δ
pandas/core/panel.py	`42.02% <ø> (ø)`	⬆️
pandas/core/tools/timedeltas.py	`66.66% <ø> (ø)`	⬆️
pandas/core/frame.py	`38.7% <ø> (ø)`	⬆️
pandas/io/json/json.py	`16.66% <ø> (ø)`	⬆️
pandas/io/gbq.py	`25% <ø> (ø)`	⬆️
pandas/core/window.py	`28.93% <100%> (ø)`	⬆️
pandas/core/arrays/period.py	`36.97% <0%> (-0.15%)`	⬇️
pandas/core/arrays/datetimes.py	`63.41% <0%> (-0.08%)`	⬇️
pandas/core/strings.py	`33% <0%> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 45f880b...498cebb. Read the comment docs.

…when lines are found (exit 0)

… with or without quotes

… seems to fail because of it

…of script solves the exclamation mark problems

…pep8 issue in scripts/tests

pep8speaks · 2018-09-27T19:17:58Z

Hello @datapythonista! Thanks for updating the PR.

There are no PEP8 issues in the file scripts/generate_pip_deps_from_conda.py !

Comment last updated on November 12, 2018 at 16:58 Hours UTC

datapythonista · 2018-09-27T20:08:07Z

@TomAugspurger seems like the continueOnError is not working at the job level (it should based on the docs), it needs to be at step level. But more important, continuing means that the build will be successful when an step fails, so it's not an option.

I think the log is much clearer, I personally thing it'd be great if we could use azure for the linting. But I think the drawbacks are probably deal breakers, and I don't think they can be fixed:

We can't run all the linting/checks, seems like when one in each job fails it has to stop. And I don't think we want to have one job per test, that would make things much more complicated (both in the dashboard and in the code).
Having the checking in the yaml, we can't run ./ci/lint.sh locally. We could have a script that parses the yaml and executes all the checks, or we could build the yaml from a script. But I don't think any of them is actually worth.

Thoughts?

CC @chrisrpatterson

…tion, which runs the task, but doesn't change the outcome of the build

datapythonista · 2018-09-29T10:12:57Z

Looks like continueOnError is converting the error in a warning, but adding condition: true is what makes that a step is executed no matter the outcome of the previous.

This looks cool now:
https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=367&view=logs

And seems like we can have links to the source code line with the linting error, which may also be helpful in some cases.

I guess we still want to be able to run all the linting and checks locally. I think a script that parses the yaml and execute every command on it should be really easy, and would do the work for now.

Thoughts? @TomAugspurger @jreback

TomAugspurger · 2018-09-29T11:28:16Z

Yes, that looks quite nice. Thanks. As an alternative to parsing the YAML, could we have a directory of scripts like `ci/lint/lint-py.sh`, `ci/lint/lint-c.sh`. Each of those would be one of the `script` lines in your YAML. Then we could have a `ci/lint.sh` that loops over all the `ci/lint/*.sh` to run them all?

…

On Sat, Sep 29, 2018 at 5:13 AM Marc Garcia ***@***.***> wrote: Looks like continueOnError is converting the error in a warning, but adding condition: true is what makes that a step is executed no matter the outcome of the previous. This looks cool now: https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=367&view=logs And seems like we can have links to the source code line with the linting error, which may also be helpful in some cases. I guess we still want to be able to run all the linting and checks locally. I think a script that parses the yaml and execute every command on it should be really easy, and would do the work for now. Thoughts? @TomAugspurger <https://github.com/TomAugspurger> @jreback <https://github.com/jreback> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#22854 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIqDamrYYcQ5DYWaC0mzikWqSj0Ziks5uf0exgaJpZM4W8qbh> .

datapythonista · 2018-09-29T12:20:52Z

I unified all the linting of *.py files in #22863 (adcda47), which I think that simplifies things, and adds few value to have them separate, even in azure.

But even with that we'd have more than 10 scripts. And while that would surely guarantee the syncronization between azure and lint.sh in the exact commands, I think we can end up having differences on which scripts are called in each place. And, I think we'd loose the command being executed (e.g. we'd see in the logs ./ci/check_deprecated_directive.sh instead of grep -v ".. deprecated::, which I think makes things more difficult to debug).

So, I think I prefer another option (which look much simpler to me):

We could simply not have anything to call all checks (linting .py files is as simple as flake8 . with the current state of STYLE: Fixing and refactoring linting #22863, and the other checks we can probably wait for the CI)
Have the lint.sh that iterates over azure-pipelines.yml and run its commands

jbrockmendel · 2018-09-29T16:44:53Z

But even with that we'd have more than 10 scripts.

My only real thought here is that I prefer a centralized ci/lint.sh to less-centralized. I have a hard enough time keeping track of what is checked where as it is.

…nd it shows them without having to enter the log)

…ving to the command for now

…in a PR from the same pandas repo)

datapythonista · 2018-12-01T03:11:02Z

@vtbassmatt I'm testing the publishing of the docs from a branch in the pandas repo, and looks like that indeed fixes the secret variables problem. But I'm getting an error that the specified container doesn't exist. I created a container named web, but still the same error.

This is the container:

And you can see the error here: https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=4427

2018-12-01T03:01:23.6821015Z ##[section]Starting: Publishing docs (Azure storage)
2018-12-01T03:01:23.6824359Z ==============================================================================
2018-12-01T03:01:23.6824504Z Task         : Command Line
2018-12-01T03:01:23.6824616Z Description  : Run a command line script using cmd.exe on Windows and bash on macOS and Linux.
2018-12-01T03:01:23.6824745Z Version      : 2.142.2
2018-12-01T03:01:23.6824854Z Author       : Microsoft Corporation
2018-12-01T03:01:23.6824979Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkID=613735)
2018-12-01T03:01:23.6825218Z ==============================================================================
2018-12-01T03:01:23.8249356Z Generating script.
2018-12-01T03:01:23.8313839Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/8cf84557-3f35-4912-8280-707aacf05035.sh
2018-12-01T03:01:44.2572732Z WARNING: The installed extension 'storage-preview' is in preview.
2018-12-01T03:01:47.2522240Z WARNING: uploading /home/vsts/work/1/s/doc/build/html/indexing.html
2018-12-01T03:01:47.7692495Z ERROR: The specified container does not exist. ErrorCode: ContainerNotFound
2018-12-01T03:01:47.7694905Z <?xml version="1.0" encoding="utf-8"?><Error><Code>ContainerNotFound</Code><Message>The specified container does not exist.
2018-12-01T03:01:47.7695666Z RequestId:8b38210c-d01e-0077-7022-894d0e000000
2018-12-01T03:01:47.7696273Z Time:2018-12-01T03:01:47.6226510Z</Message></Error>
2018-12-01T03:01:47.7696599Z Traceback (most recent call last):
2018-12-01T03:01:47.7697153Z   File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 197, in invoke
2018-12-01T03:01:47.7697639Z     cmd_result = self.invocation.execute(args)
2018-12-01T03:01:47.7698160Z   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 366, in execute
2018-12-01T03:01:47.7698452Z     cmd.exception_handler(ex)
2018-12-01T03:01:47.7699073Z   File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/__init__.py", line 234, in new_handler
2018-12-01T03:01:47.7699376Z     handler(ex)
2018-12-01T03:01:47.7699845Z   File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/__init__.py", line 177, in handler
2018-12-01T03:01:47.7700128Z     raise ex
2018-12-01T03:01:47.7700564Z   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 343, in execute
2018-12-01T03:01:47.7700851Z     result = cmd(params)
2018-12-01T03:01:47.7701305Z   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 182, in __call__
2018-12-01T03:01:47.7701598Z     return self.handler(*args, **kwargs)
2018-12-01T03:01:47.7702308Z   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 436, in default_command_handler
2018-12-01T03:01:47.7702870Z     result = op(**command_args)
2018-12-01T03:01:47.7703609Z   File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 199, in storage_blob_upload_batch
2018-12-01T03:01:47.7704364Z     if_none_match=if_none_match, timeout=timeout)
2018-12-01T03:01:47.7705003Z   File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/util.py", line 218, in wrapper
2018-12-01T03:01:47.7705366Z     return True, func(*args, **kwargs)
2018-12-01T03:01:47.7705945Z   File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 185, in _upload_blob
2018-12-01T03:01:47.7706305Z     return upload_blob(*args, **kwargs)
2018-12-01T03:01:47.7706901Z   File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 286, in upload_blob
2018-12-01T03:01:47.7707444Z     return type_func[blob_type]()
2018-12-01T03:01:47.7707938Z   File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 279, in upload_block_blob
2018-12-01T03:01:47.7708231Z     return client.create_blob_from_path(**create_blob_args)
2018-12-01T03:01:47.7708733Z   File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/blob/blockblobservice.py", line 463, in create_blob_from_path
2018-12-01T03:01:47.7709230Z     timeout=timeout)
2018-12-01T03:01:47.7709760Z   File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/blob/blockblobservice.py", line 582, in create_blob_from_stream
2018-12-01T03:01:47.7710080Z     timeout=timeout)
2018-12-01T03:01:47.7710585Z   File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/blob/blockblobservice.py", line 971, in _put_blob
2018-12-01T03:01:47.7710905Z     return self._perform_request(request, _parse_base_properties)
2018-12-01T03:01:47.7711432Z   File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/common/storageclient.py", line 381, in _perform_request
2018-12-01T03:01:47.7711750Z     raise ex
2018-12-01T03:01:47.7712243Z   File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/common/storageclient.py", line 306, in _perform_request
2018-12-01T03:01:47.7712548Z     raise ex
2018-12-01T03:01:47.7713076Z   File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/common/storageclient.py", line 292, in _perform_request
2018-12-01T03:01:47.7713572Z     HTTPError(response.status, response.message, response.headers, response.body))
2018-12-01T03:01:47.7714756Z   File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/common/_error.py", line 115, in _http_error_handler
2018-12-01T03:01:47.7715115Z     raise ex
2018-12-01T03:01:47.7715419Z azure.common.AzureMissingResourceHttpError: The specified container does not exist. ErrorCode: ContainerNotFound
2018-12-01T03:01:47.7716037Z <?xml version="1.0" encoding="utf-8"?><Error><Code>ContainerNotFound</Code><Message>The specified container does not exist.
2018-12-01T03:01:47.7716650Z RequestId:8b38210c-d01e-0077-7022-894d0e000000
2018-12-01T03:01:47.7717804Z Time:2018-12-01T03:01:47.6226510Z</Message></Error>
2018-12-01T03:01:47.8524094Z Documentation uploaded to https://pandas.blob.core.windows.net
2018-12-01T03:01:47.8535123Z ##[section]Finishing: Publishing docs (Azure storage)

Is web the right name of the container? Do I need to give permissions or something?

Btw, is it normal that if a command in the step fails, it continues and shows the job as success? Should I do a set -e, or there is another way to make it fail?

vtbassmatt · 2018-12-01T10:46:05Z

Caveat, I'm new at this Azure publishing feature myself. But when I did it, Azure created the container for me when I enabled the "static sites" feature. Also, the container is literally called `$web` (leading dollar sign) so you'll have to put it in single quotes to avoid Bash trying to interpret it.

…

On Fri, Nov 30, 2018 at 10:11 PM Marc Garcia ***@***.***> wrote: @vtbassmatt <https://github.com/vtbassmatt> I'm testing the publishing of the docs from a branch in the pandas repo, and looks like that indeed fixes the secret variables problem. But I'm getting an error that the specified container doesn't exist. I created a container named web, but still the same error. This is the container: [image: azure_storage_container] <https://user-images.githubusercontent.com/10058240/49323617-0f534080-f516-11e8-9148-b9213fd6caaf.png> And you can see the error here: https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=4427 2018-12-01T03:01:23.6821015Z ##[section]Starting: Publishing docs (Azure storage) 2018-12-01T03:01:23.6824359Z ============================================================================== 2018-12-01T03:01:23.6824504Z Task : Command Line 2018-12-01T03:01:23.6824616Z Description : Run a command line script using cmd.exe on Windows and bash on macOS and Linux. 2018-12-01T03:01:23.6824745Z Version : 2.142.2 2018-12-01T03:01:23.6824854Z Author : Microsoft Corporation 2018-12-01T03:01:23.6824979Z Help : [More Information](https://go.microsoft.com/fwlink/?LinkID=613735) 2018-12-01T03:01:23.6825218Z ============================================================================== 2018-12-01T03:01:23.8249356Z Generating script. 2018-12-01T03:01:23.8313839Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/8cf84557-3f35-4912-8280-707aacf05035.sh 2018-12-01T03:01:44.2572732Z WARNING: The installed extension 'storage-preview' is in preview. 2018-12-01T03:01:47.2522240Z WARNING: uploading /home/vsts/work/1/s/doc/build/html/indexing.html 2018-12-01T03:01:47.7692495Z ERROR: The specified container does not exist. ErrorCode: ContainerNotFound 2018-12-01T03:01:47.7694905Z <?xml version="1.0" encoding="utf-8"?><Error><Code>ContainerNotFound</Code><Message>The specified container does not exist. 2018-12-01T03:01:47.7695666Z RequestId:8b38210c-d01e-0077-7022-894d0e000000 2018-12-01T03:01:47.7696273Z Time:2018-12-01T03:01:47.6226510Z</Message></Error> 2018-12-01T03:01:47.7696599Z Traceback (most recent call last): 2018-12-01T03:01:47.7697153Z File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 197, in invoke 2018-12-01T03:01:47.7697639Z cmd_result = self.invocation.execute(args) 2018-12-01T03:01:47.7698160Z File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 366, in execute 2018-12-01T03:01:47.7698452Z cmd.exception_handler(ex) 2018-12-01T03:01:47.7699073Z File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/__init__.py", line 234, in new_handler 2018-12-01T03:01:47.7699376Z handler(ex) 2018-12-01T03:01:47.7699845Z File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/__init__.py", line 177, in handler 2018-12-01T03:01:47.7700128Z raise ex 2018-12-01T03:01:47.7700564Z File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 343, in execute 2018-12-01T03:01:47.7700851Z result = cmd(params) 2018-12-01T03:01:47.7701305Z File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 182, in __call__ 2018-12-01T03:01:47.7701598Z return self.handler(*args, **kwargs) 2018-12-01T03:01:47.7702308Z File "/opt/az/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 436, in default_command_handler 2018-12-01T03:01:47.7702870Z result = op(**command_args) 2018-12-01T03:01:47.7703609Z File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 199, in storage_blob_upload_batch 2018-12-01T03:01:47.7704364Z if_none_match=if_none_match, timeout=timeout) 2018-12-01T03:01:47.7705003Z File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/util.py", line 218, in wrapper 2018-12-01T03:01:47.7705366Z return True, func(*args, **kwargs) 2018-12-01T03:01:47.7705945Z File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 185, in _upload_blob 2018-12-01T03:01:47.7706305Z return upload_blob(*args, **kwargs) 2018-12-01T03:01:47.7706901Z File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 286, in upload_blob 2018-12-01T03:01:47.7707444Z return type_func[blob_type]() 2018-12-01T03:01:47.7707938Z File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 279, in upload_block_blob 2018-12-01T03:01:47.7708231Z return client.create_blob_from_path(**create_blob_args) 2018-12-01T03:01:47.7708733Z File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/blob/blockblobservice.py", line 463, in create_blob_from_path 2018-12-01T03:01:47.7709230Z timeout=timeout) 2018-12-01T03:01:47.7709760Z File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/blob/blockblobservice.py", line 582, in create_blob_from_stream 2018-12-01T03:01:47.7710080Z timeout=timeout) 2018-12-01T03:01:47.7710585Z File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/blob/blockblobservice.py", line 971, in _put_blob 2018-12-01T03:01:47.7710905Z return self._perform_request(request, _parse_base_properties) 2018-12-01T03:01:47.7711432Z File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/common/storageclient.py", line 381, in _perform_request 2018-12-01T03:01:47.7711750Z raise ex 2018-12-01T03:01:47.7712243Z File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/common/storageclient.py", line 306, in _perform_request 2018-12-01T03:01:47.7712548Z raise ex 2018-12-01T03:01:47.7713076Z File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/common/storageclient.py", line 292, in _perform_request 2018-12-01T03:01:47.7713572Z HTTPError(response.status, response.message, response.headers, response.body)) 2018-12-01T03:01:47.7714756Z File "/opt/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_03_28/common/_error.py", line 115, in _http_error_handler 2018-12-01T03:01:47.7715115Z raise ex 2018-12-01T03:01:47.7715419Z azure.common.AzureMissingResourceHttpError: The specified container does not exist. ErrorCode: ContainerNotFound 2018-12-01T03:01:47.7716037Z <?xml version="1.0" encoding="utf-8"?><Error><Code>ContainerNotFound</Code><Message>The specified container does not exist. 2018-12-01T03:01:47.7716650Z RequestId:8b38210c-d01e-0077-7022-894d0e000000 2018-12-01T03:01:47.7717804Z Time:2018-12-01T03:01:47.6226510Z</Message></Error> 2018-12-01T03:01:47.8524094Z Documentation uploaded to https://pandas.blob.core.windows.net 2018-12-01T03:01:47.8535123Z ##[section]Finishing: Publishing docs (Azure storage) Is web the right name of the container? Do I need to give permissions or something? Btw, is it normal that if a command in the step fails, it continues and shows the job as success? Should I do a set -e, or there is another way to make it fail? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#22854 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AArmK9mV0MueblbX93svawmdPQwABRzgks5u0fNqgaJpZM4W8qbh> .

…one process in the build

h-vetinari · 2018-12-01T11:08:03Z

ci/code_checks.sh

    RET=$(($RET + $?)) ; echo $MSG "DONE"

    MSG='Check for incorrect sphinx directives' ; echo $MSG
-    ! grep -R --include="*.py" --include="*.pyx" --include="*.rst" -E "\.\. (autosummary|contents|currentmodule|deprecated|function|image|important|include|ipython|literalinclude|math|module|note|raw|seealso|toctree|versionadded|versionchanged|warning):[^:]" ./pandas ./doc/source
+    invgrep -R --include="*.py" --include="*.pyx" --include="*.rst" -E "\.\. (autosummary|contents|currentmodule|deprecated|function|image|important|include|ipython|literalinclude|math|module|note|raw|seealso|toctree|versionadded|versionchanged|warning):[^:]" ./pandas ./doc/source
    RET=$(($RET + $?)) ; echo $MSG "DONE"

    MSG='Check that the deprecated `assert_raises_regex` is not used (`pytest.raises(match=pattern)` should be used instead)' ; echo $MSG
    ! grep -R --exclude=*.pyc --exclude=testing.py --exclude=test_testing.py assert_raises_regex pandas


Missed an invgrep here.

Thanks @h-vetinari, that was added later, and I didn't see it after merging from master. Well spotted, I fixed it.

datapythonista · 2018-12-01T11:15:36Z

Thanks @vtbassmatt, I've got the '$web' as in your example, I think what's missing is enabling the static files.

I couldn't find where to activate it, but I saw the plan we've got is a free trial expiring in 3 weeks. As for now, we're just able to publish the dev documentation committed to master (and not the built documentation of every PR as initially planned), I think we can stay in GitHub pages.

So, I'm removing the part that builds and uploads the documentation in azure for now (it's still in travis). So, we should be able to merge this PR with all the linting, and I'll check in a new PR what can be done with the documentation.

datapythonista · 2018-12-01T14:51:18Z

@jreback, all green now, can you take another look please?

jreback · 2018-12-02T21:59:48Z

@datapythonista can you rebase.

also can you point to a failed build (where the fail is on the liniting). E.g. I want to see how easy it is for a user to navigate to this. (maybe just put a lint error in here for it to fail)

datapythonista · 2018-12-02T22:07:32Z

you can see how an error looks like here: #22854 (comment)

much easier to find the errors than in travis for sure :)

jreback · 2018-12-02T22:27:37Z

lgtm merge on green

datapythonista · 2018-12-03T00:06:27Z

thanks for the review @jreback, and thanks a lot for all the help with this @vtbassmatt

gfyoung · 2018-12-03T09:30:16Z

azure-pipelines.yml

+          echo "Benchmarks did not run, no changes detected"
+      fi
+    displayName: 'Running benchmarks'
+    condition: true


Why did you add running benchmarks when the original issue was just to migrate lint.sh to Azure? IMO, this should have been considered in a separate PR.

Also, as your changes did not actually modify any Python benchmark code, it was not possible to review how it would look in the Azure logs. This log here from another PR (#23752) is unfortunately impossible to read:

https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=4580

Also, it's somewhat debatable as to whether we even want to just run benchmarks when there's only a diff in asv_bench. We often ask people to run asv whenever there are changes to C and Cython code in general, just to make sure that no benchmarks were affected as a result of the changes.

gfyoung · 2018-12-03T09:30:46Z

azure-pipelines.yml

+      source activate pandas-dev
+      ci/code_checks.sh doctests
+    displayName: 'Running doctests'
+    condition: true


Hmm...IMO, I think running doctests should be a separate build entirely from linting. They are two semantically different things, and the separation would allow us to provide more semantically clear display names such as "Doctests" and "Linting" instead of the slightly vaguer mashup of "Checks_and_doc".

I see there was a concern about not having to use an extra build for this due to resource constraints (the conversation is a very long in this PR...)? However, I'm not sure I fully understand...

Also, based on the conversation, are the master docs no longer being published on https://pandas-docs.github.io/pandas-docs-travis? In which case, we need to update the GITHUB_ISSUE template that we have here.

datapythonista · 2018-12-03T10:23:42Z

@gfyoung I agree running benchmarks (and building the docs, which was implemented here), could be better addressed in separate PRs. Except that this PR took 2 months to be able to be ready, for different problems, and those things couldn't be implemented in parallel. And I didn't want to have them blocked for so long.

I don't understand your concerns.

The dev documentation is built in the same exact way as before, that wasn't touched.

I don't think anyone had a problem of running the linting in one of the tests builds. And now you don't want to have it with the doctests. We have around 12 of 14 builds that run the test suite with different configurations and for different platforms. I think having everything else in a separate build makes it very easy to know where to find things. This is not yet true for the docs, that we couldn't move them yet. And as the builds spend most of the time creating the environment and building pandas, I think it's much better to have a single build with everything that is not running the main tests, than having 10, because the linting is semantically different than the greps, than the docs, than the doctests, than the benchmarks, than the docstring formats...

Not sure which log is impossible to read. It takes me around 3 minutes to find the linting problems in travis, and it takes me around 10 seconds to find them in azure. I added screenshots and links on how things looked while they were broken in azure. And left them broken for more than a month.

It can make sense to run the benchmarks more often. But running them when the benchmarks itself, if you're happy to run them in the CI, is better than not running them, like until now.

I think this PR makes things much much better in the CI, and will save us lots of time. And I don't quite understand all the post-merge comments like if this was a mistake. Of course it'd be great for any of the things you propose, to open issues, so we can discuss there, and possibly implement then. And you can tag me, so I can share my experience.

datapythonista · 2018-12-03T10:57:56Z

@gfyoung I think I misunderstood you regarding the logs difficult to read. I think you meant the benchmarks only, right? I created #24061 as there is a bug in the code, and line breaks are not being displayed.

jreback · 2018-12-03T11:19:00Z

just to be clear

running code checks (including doc tests) and the doc build is fine; this would include
a code check that the benchmarks actually are semantically correct not actually running them - which is not done in in this repo at all

rather in :
https://travis-ci.org/pandas-dev/pandas-ci

though this often breaks because of the timeout

datapythonista · 2018-12-03T12:12:47Z

the implemented code check is a dry-run of the benchmark (and we are also linting the benchmarks code)

Moving one-line linting and checks from lint.sh to azure steps

46620f2

TomAugspurger reviewed Sep 27, 2018

View reviewed changes

Moving everything from lint.sh to azure (so far repeating the command…

dc8c528

…s that used to be loops). Installing missing packages, splitting linting and checks, and continuing on errors

datapythonista added 9 commits September 27, 2018 16:15

Fixing continueOnError settings

adeb0ca

Reverting the exit status of the grep commands, we want them to fail …

dcfb203

…when lines are found (exit 0)

Testing the continueOnError, doesn't seem to be working at job level,…

b765df8

… with or without quotes

Escaping exclamation marks to reverse the exist status of grep, azure…

3c77fc9

… seems to fail because of it

Adding the continueOnError to every step, and trying if bash instead …

3d8af7d

…of script solves the exclamation mark problems

Fixes to missing continueOnError, and to the grep command exit status

ed1064d

Testing couple of ways to invert the exit status of grep

8cb7b0f

Fixing azure config syntax error

dee5a08

Fixing the checks (they should be ok now), and adding an intentional …

54f9e98

…pep8 issue in scripts/tests

Removing intentional pep8 issue, and fixing last known problems

20cb360

datapythonista added 2 commits September 27, 2018 21:48

fixing style of azure settings

50d2757

Replacing continueOnError (that converts errors in warnings) to condi…

19a213c

…tion, which runs the task, but doesn't change the outcome of the build

datapythonista changed the title ~~WIP: Moving one-line linting and checks from lint.sh to azure steps~~ WIP: Linting with azure instead of travis (tests to see how it looks like) Sep 29, 2018

Fixing multiline in yaml, and removing job level continueOnError

0cf5da9

datapythonista mentioned this pull request Sep 30, 2018

STYLE: Fixing and refactoring linting #22863

Merged

datapythonista added 3 commits September 30, 2018 19:10

Changing the format of flake8 output to azure (so it creates links, a…

4d10702

…nd it shows them without having to enter the log)

Removing unnecessary quotes in flake8 format config

a356f03

flake8 format breaks pip (ConfigParser) when present in setup.cfg, mo…

50cb867

…ving to the command for now

Always uploading docs (removed if to test if uploading the docs work …

c88911a

…in a PR from the same pandas repo)

datapythonista mentioned this pull request Dec 1, 2018

Duplicate PR of #22854 (testing publishing docs from azure) #24030

Closed

Adding parallelization to build and docs

75b89eb

Removing documentation build in azure, and reverting using more than …

011950e

…one process in the build

h-vetinari reviewed Dec 1, 2018

View reviewed changes

datapythonista added 3 commits December 1, 2018 11:19

Made an invgrep a newly added pattern validation

01942b9

Merging from master

0a14165

Regenerating pip dependencies

705eb9d

Merging from master

498cebb

jreback added this to the 0.24.0 milestone Dec 2, 2018

jreback approved these changes Dec 2, 2018

View reviewed changes

datapythonista merged commit 022f458 into pandas-dev:master Dec 3, 2018

datapythonista mentioned this pull request Dec 3, 2018

Fix ASV imports #23085

Merged

gfyoung mentioned this pull request Dec 3, 2018

API: rename MultiIndex.labels to MultiIndex.codes #23752

Merged

4 tasks

gfyoung reviewed Dec 3, 2018

View reviewed changes

Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

CI: Linting with azure instead of travis (pandas-dev#22854)

e930029

Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

CI: Linting with azure instead of travis (pandas-dev#22854)

8c3bb7c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: Linting with azure instead of travis #22854

CI: Linting with azure instead of travis #22854

datapythonista commented Sep 27, 2018 •

edited

Loading

TomAugspurger left a comment

TomAugspurger Sep 27, 2018

datapythonista Sep 27, 2018

datapythonista commented Sep 27, 2018

TomAugspurger commented Sep 27, 2018

codecov bot commented Sep 27, 2018 •

edited

Loading

pep8speaks commented Sep 27, 2018 •

edited

Loading

datapythonista commented Sep 27, 2018

datapythonista commented Sep 29, 2018

TomAugspurger commented Sep 29, 2018 via email

datapythonista commented Sep 29, 2018

jbrockmendel commented Sep 29, 2018

datapythonista commented Dec 1, 2018

vtbassmatt commented Dec 1, 2018 via email

h-vetinari Dec 1, 2018

datapythonista Dec 1, 2018

datapythonista commented Dec 1, 2018

datapythonista commented Dec 1, 2018

jreback commented Dec 2, 2018

datapythonista commented Dec 2, 2018

jreback commented Dec 2, 2018

datapythonista commented Dec 3, 2018

gfyoung Dec 3, 2018 •

edited

Loading

gfyoung Dec 3, 2018 •

edited

Loading

gfyoung Dec 3, 2018 •

edited

Loading

gfyoung Dec 3, 2018 •

edited

Loading

datapythonista commented Dec 3, 2018

datapythonista commented Dec 3, 2018

jreback commented Dec 3, 2018 •

edited

Loading

datapythonista commented Dec 3, 2018

CI: Linting with azure instead of travis #22854

CI: Linting with azure instead of travis #22854

Conversation

datapythonista commented Sep 27, 2018 • edited Loading

TomAugspurger left a comment

Choose a reason for hiding this comment

TomAugspurger Sep 27, 2018

Choose a reason for hiding this comment

datapythonista Sep 27, 2018

Choose a reason for hiding this comment

datapythonista commented Sep 27, 2018

TomAugspurger commented Sep 27, 2018

codecov bot commented Sep 27, 2018 • edited Loading

Codecov Report

pep8speaks commented Sep 27, 2018 • edited Loading

Comment last updated on November 12, 2018 at 16:58 Hours UTC

datapythonista commented Sep 27, 2018

datapythonista commented Sep 29, 2018

TomAugspurger commented Sep 29, 2018 via email

datapythonista commented Sep 29, 2018

jbrockmendel commented Sep 29, 2018

datapythonista commented Dec 1, 2018

vtbassmatt commented Dec 1, 2018 via email

h-vetinari Dec 1, 2018

Choose a reason for hiding this comment

datapythonista Dec 1, 2018

Choose a reason for hiding this comment

datapythonista commented Dec 1, 2018

datapythonista commented Dec 1, 2018

jreback commented Dec 2, 2018

datapythonista commented Dec 2, 2018

jreback commented Dec 2, 2018

datapythonista commented Dec 3, 2018

gfyoung Dec 3, 2018 • edited Loading

Choose a reason for hiding this comment

gfyoung Dec 3, 2018 • edited Loading

Choose a reason for hiding this comment

gfyoung Dec 3, 2018 • edited Loading

Choose a reason for hiding this comment

gfyoung Dec 3, 2018 • edited Loading

Choose a reason for hiding this comment

datapythonista commented Dec 3, 2018

datapythonista commented Dec 3, 2018

jreback commented Dec 3, 2018 • edited Loading

datapythonista commented Dec 3, 2018

datapythonista commented Sep 27, 2018 •

edited

Loading

codecov bot commented Sep 27, 2018 •

edited

Loading

pep8speaks commented Sep 27, 2018 •

edited

Loading

gfyoung Dec 3, 2018 •

edited

Loading

gfyoung Dec 3, 2018 •

edited

Loading

gfyoung Dec 3, 2018 •

edited

Loading

gfyoung Dec 3, 2018 •

edited

Loading

jreback commented Dec 3, 2018 •

edited

Loading