Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tools: fix GYP ninja generator for Python 3 #29416

Closed
wants to merge 2 commits into from
Closed

tools: fix GYP ninja generator for Python 3 #29416

wants to merge 2 commits into from

Conversation

targos
Copy link
Member

@targos targos commented Sep 3, 2019

No description provided.

@targos targos requested a review from cclauss September 3, 2019 07:50
@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. tools Issues and PRs related to the tools directory. labels Sep 3, 2019
cclauss
cclauss requested changes Sep 3, 2019
View reviewed changes
Copy link
Contributor

@cclauss cclauss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please replace the filter() calls with list comprehensions as discussed at https://docs.python.org/3.0/library/functions.html#filter

@targos
Copy link
Member Author

targos commented Sep 3, 2019

@cclauss better?

@cclauss cclauss added the python PRs and issues that require attention from people who are familiar with Python. label Sep 3, 2019
@cclauss cclauss mentioned this pull request Sep 3, 2019
Closed
63 tasks
sam-github
sam-github reviewed Sep 3, 2019
View reviewed changes
Copy link
Contributor

@sam-github sam-github left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this do? How do I test it? AFAICT, python3 doesn't run on configure.py before or after this PR:

w/node (master $ u=) % python3 configure.py --ninja
Traceback (most recent call last):
  File "configure.py", line 28, in <module>
    from gyp.common import GetFlavor
  File "tools/gyp/pylib/gyp/__init__.py", line 37
    print '%s:%s:%d:%s %s' % (mode.upper(), os.path.basename(ctx[0]),
                         ^
SyntaxError: invalid syntax
w/node (master $ u=) % git pr 29416 upstream
remote: Enumerating objects: 3044, done.
remote: Counting objects: 100% (3044/3044), done.
remote: Total 4164 (delta 3044), reused 3044 (delta 3044), pack-reused 1120
Receiving objects: 100% (4164/4164), 4.95 MiB | 4.01 MiB/s, done.
Resolving deltas: 100% (3346/3346), completed with 2070 local objects.
From github.com:nodejs/node
 * [new ref]               refs/pull/29416/head -> upstream/pr/29416
w/node (master $ u=) % python3 configure.py --ninja
Traceback (most recent call last):
  File "configure.py", line 28, in <module>
    from gyp.common import GetFlavor
  File "tools/gyp/pylib/gyp/__init__.py", line 37
    print '%s:%s:%d:%s %s' % (mode.upper(), os.path.basename(ctx[0]),
                         ^
SyntaxError: invalid syntax

@Trott
Copy link
Member

Trott commented Sep 3, 2019

Lite CI: https://ci.nodejs.org/job/node-test-pull-request-lite-pipeline/3878/

@Trott
Copy link
Member

Trott commented Sep 3, 2019

What does this do?

I'm sure @targos and/or @cclauss will have better answers, but my approval is based on the fact that filter() returns different things in Python 2 vs. Python 3 (list vs. generator) so for compatibility purposes, best to refactor it out.

@targos
Copy link
Member Author

targos commented Sep 3, 2019

This fixes python3 configure.py --ninja on my end. I don't have the invalid syntax like you @sam-github.

@Trott
Copy link
Member

Trott commented Sep 3, 2019

This fixes python3 configure.py --ninja on my end. I don't have the invalid syntax like you @sam-github.

Python 3.6 vs Python 3.7 maybe?

@sam-github
Copy link
Contributor

I'm on python 3.7.4, which versions are you all using?

w/node (upstream/pr/29416 $) % git co upstream/pr/29416; git log --oneline | head -n 1; python3 --version; python3 configure.py --ninja
Already on 'upstream/pr/29416'
3b2ed104b7 fixup! tools: fix GYP ninja generator for Python 3
Python 3.7.4
INFO: Using floating patch "tools/icu/patches/64/source/common/putil.cpp" from "tools/icu"
INFO: Using floating patch "tools/icu/patches/64/source/i18n/dtptngen.cpp" from "tools/icu"
Traceback (most recent call last):
  File "configure.py", line 1716, in <module>
    run_gyp(gyp_args)
  File "tools/gyp_node.py", line 54, in run_gyp
    rc = gyp.main(args)
  File "tools/gyp/pylib/gyp/__init__.py", line 547, in main
    return gyp_main(args)
  File "tools/gyp/pylib/gyp/__init__.py", line 523, in gyp_main
    options.duplicate_basename_check)
  File "tools/gyp/pylib/gyp/__init__.py", line 139, in Load
    params['parallel'], params['root_targets'])
  File "tools/gyp/pylib/gyp/input.py", line 2779, in Load
    variables, includes, depth, check, True)
  File "tools/gyp/pylib/gyp/input.py", line 459, in LoadTargetBuildFile
    includes, depth, check, load_dependencies)
  File "tools/gyp/pylib/gyp/input.py", line 408, in LoadTargetBuildFile
    build_file_data, PHASE_EARLY, variables, build_file_path)
  File "tools/gyp/pylib/gyp/input.py", line 1290, in ProcessVariablesAndConditionsInDict
    build_file)
  File "tools/gyp/pylib/gyp/input.py", line 1305, in ProcessVariablesAndConditionsInList
    ProcessVariablesAndConditionsInDict(item, phase, variables, build_file)
  File "tools/gyp/pylib/gyp/input.py", line 1264, in ProcessVariablesAndConditionsInDict
    ProcessConditionsInDict(the_dict, phase, variables, build_file)
  File "tools/gyp/pylib/gyp/input.py", line 1137, in ProcessConditionsInDict
    build_file)
  File "tools/gyp/pylib/gyp/input.py", line 1061, in EvalCondition
    cond_expr, true_dict, false_dict, phase, variables, build_file)
  File "tools/gyp/pylib/gyp/input.py", line 1087, in EvalSingleCondition
    if eval(ast_code, {'__builtins__': None}, variables):
  File "<string>", line 1, in <module>
TypeError: '>=' not supported between instances of 'int' and 'str' while loading dependencies of /Users/samroberts/w/node/node.gyp while trying to load /Users/samroberts/w/node/node.gyp

@targos
Copy link
Member Author

targos commented Sep 3, 2019

TypeError: '>=' not supported between instances of 'int' and 'str' while loading dependencies of /Users/samroberts/w/node/node.gyp while trying to load /Users/samroberts/w/node/node.gyp

This is a macOS-only issue that @ryzokuken is working on.

ryzokuken
ryzokuken approved these changes Sep 4, 2019
View reviewed changes
Copy link
Contributor

@ryzokuken ryzokuken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much!

@cclauss
Copy link
Contributor

cclauss commented Sep 4, 2019 •
edited
Loading

TypeError: '>=' not supported between instances of 'int' and 'str'

This is usually a sign that the thing on the left is bytes while the thing on the right is str.

@ryzokuken
Copy link
Contributor

@cclauss

This is usually a sign that the thing on the left is bytes while the thing on the right is str.

Noted! I'll try to dig deeper.

@targos
Copy link
Member Author

targos commented Sep 5, 2019

Landed in af161f0

@targos targos closed this Sep 5, 2019
@targos targos deleted the gyp-ninja-py3 branch September 5, 2019 11:54
targos added a commit that referenced this pull request Sep 5, 2019
@targos
tools: fix GYP ninja generator for Python 3
af161f0 
PR-URL: #29416
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ujjwal Sharma <usharma1998@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
@sam-github
Copy link
Contributor

@nodejs/lts this needs to land on 12.x-staging, ./configure --ninja is broken on it ATM

@targos
Copy link
Member Author

targos commented Sep 13, 2019 •
edited
Loading

12.x isn't LTS yet. This commit will be in the next release

@cclauss
Copy link
Contributor

cclauss commented Sep 13, 2019

Related to #29415

@sam-github
Copy link
Contributor

Note that just cherry-picking af161f0 onto 12.x-staging doesn't fix ninja. I started to try to figure out why, but I've been called away to look at something else before making progress.

targos added a commit that referenced this pull request Sep 20, 2019
@targos
tools: fix GYP ninja generator for Python 3
2f81d59 
PR-URL: #29416
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ujjwal Sharma <usharma1998@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
@BridgeAR BridgeAR mentioned this pull request Sep 24, 2019
Merged
This was referenced Sep 25, 2019
Closed
Closed
Closed
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sam-github sam-github sam-github left review comments

@ryzokuken ryzokuken ryzokuken approved these changes

@cclauss cclauss cclauss approved these changes

@Trott Trott Trott approved these changes

@bnoordhuis bnoordhuis bnoordhuis approved these changes

@richardlau richardlau Awaiting requested review from richardlau

Assignees
No one assigned
Labels
build Issues and PRs related to build files or the CI. python PRs and issues that require attention from people who are familiar with Python. tools Issues and PRs related to the tools directory.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@targos @Trott @sam-github @cclauss @ryzokuken @bnoordhuis @nodejs-github-bot