feat: removed pkg_resources from all test files and moved importlib into pandas extra

[kiraksi]

In this PR

• removed all instances of pkg_resources in tests and introduced importlib instead
• similar to feat: introduce compatibility with native namespace packages python-bigquery-storage#706
• should resolve any further issues we had stemming from pkg_resources

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

• Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
• Ensure the tests and linter pass
• Code coverage does not decrease (if any source code was changed)
• Appropriate docs were updated (if necessary)

Fixes #1725 🦕
Fixes #313 🦕

[snippet-bot]

No region tags are edited in this PR.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

• Refresh this comment

[parthea]

LGTM. Added one observation for potentially unnecessary version checks in test files.

[chalmerlowe]

LGTM

[chalmerlowe]

This has several elements in the list so it is a bit easier to grok what is happening here.

[tswast]

This breaks BigQuery DataFrames e2e tests. Let's revert.

ImportError while importing test module '/tmpfs/src/github/python-bigquery-dataframes/tests/unit/ml/test_golden_sql.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/[usr/local/lib/python3.11/importlib/__init__.py:126](https://cs.corp.google.com/piper///depot/google3/usr/local/lib/python3.11/importlib/__init__.py?l=126): in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
[tests/unit/ml/test_golden_sql.py:17](https://cs.corp.google.com/piper///depot/google3/tests/unit/ml/test_golden_sql.py?l=17): in <module>
    from google.cloud import bigquery
E   ImportError: cannot import name 'bigquery' from 'google.cloud' (/tmpfs/src/github/python-bigquery-dataframes/.nox/unit_prerelease/src/google-cloud-bigquery-storage/google/cloud/__init__.py)

https://fusion2.corp.google.com/invocations/2be1e8d5-99f1-41aa-97e4-4cabb94bd441/targets/bigframes%2Fpresubmit%2Fe2e/log

[parthea]

Good catch @tswast ! I can produce the issue when using an editable install using egg as it appears in the build log

When I install using an editable install using egg, the import fails

pip install -e git+https://github.com/googleapis/python-bigquery.git@1f4ebb1eca4f9380a31172fc8cb2fae125f8c5a2#egg=google_cloud_bigquery --ignore-installed

When I install without an editable install, the import succeeds

pip install https://github.com/googleapis/python-bigquery/archive/main.zip --ignore-installed

I can produce the same issue with the google-auth package. The import of google.auth fails when I use

pip install -e git+https://github.com/googleapis/google-auth-library-python.git@7ab0fced0008ad9283c955b7dd7e2b7db55ae5e7#egg=google_auth --ignore-installed`

but not with

pip install https://github.com/googleapis/google-auth-library-python/archive/main.zip

The change to move to native namespace packages was released in July for google-auth.

We should investigate whether we want to block on this issue before reverting the change as we do want to support python 3.12 without using pkg_resources.

We could also consider the change to move to native namespace packages as a potential breaking change and bump the major version.

If this only affects editable installs, IMO we should consider this a minor version instead of a major version bump as there are limitations for editable installations:
https://setuptools.pypa.io/en/latest/userguide/development_mode.html#limitations

[tswast]

Thanks for the investigation! I think we can remove the "-e" in the BigFrames tests.