Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: modifying subprocess calls and removing try except continue statements #3474

Merged
merged 92 commits into from
Oct 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
40d04f6
fix: update ``.gitignore`` file
clatapie Jun 19, 2024
29ad4dc
fix: fixing minor vulnerabilities
clatapie Jun 19, 2024
7b1058c
Merge branch 'main' into maint/vulnerability_checks
clatapie Jun 24, 2024
9148131
fix: try/except/pass vulnerabilities
clatapie Jun 25, 2024
2bb7123
Merge branch `main`into `maint/vulnerability_checks`
clatapie Sep 23, 2024
a7bc106
Merge branch 'main' into maint/vulnerability_checks
clatapie Oct 7, 2024
d8c56b3
Merge branch 'main' into maint/vulnerability_checks
clatapie Oct 9, 2024
d2734dd
fix: ``subprocess`` calls
clatapie Oct 9, 2024
e920b10
chore: adding changelog file 3474.maintenance.md [dependabot-skip]
pyansys-ci-bot Oct 9, 2024
579d41f
ci: auto fixes from pre-commit.com hooks.
pre-commit-ci[bot] Oct 9, 2024
3deca3e
fix: test
clatapie Oct 9, 2024
410cd2f
Merge branch 'maint/pymapdl_maintenance' of https://github.com/ansys/…
clatapie Oct 9, 2024
e244650
ci: auto fixes from pre-commit.com hooks.
pre-commit-ci[bot] Oct 9, 2024
06af544
chore: adding changelog file 3474.added.md [dependabot-skip]
pyansys-ci-bot Oct 9, 2024
cdd479f
fix: removing pipe in shell command
clatapie Oct 9, 2024
60b560c
ci: auto fixes from pre-commit.com hooks.
pre-commit-ci[bot] Oct 9, 2024
d0373c8
fix: pre-commit
clatapie Oct 9, 2024
123d5fa
Merge branch 'maint/pymapdl_maintenance' of https://github.com/ansys/…
clatapie Oct 9, 2024
82e53b3
test: removing subprocess change
clatapie Oct 9, 2024
d79c6df
Merge branch 'main' into maint/pymapdl_maintenance
clatapie Oct 9, 2024
cae9462
revert: reverting ``subprocess`` changes
clatapie Oct 9, 2024
283407a
Merge branch 'maint/pymapdl_maintenance' of https://github.com/ansys/…
clatapie Oct 9, 2024
1f927bc
fix: import error
clatapie Oct 9, 2024
f519bfe
fix: import error with minimal requirements
clatapie Oct 10, 2024
8f30ec4
fix: warning issue
clatapie Oct 10, 2024
7fb9abf
fix: ``_retrieve_file``
clatapie Oct 10, 2024
c7270c2
fix: ``test_failed_download``
clatapie Oct 10, 2024
63172ee
ci: auto fixes from pre-commit.com hooks.
pre-commit-ci[bot] Oct 10, 2024
b07404e
fix: ``test_examples.py``
clatapie Oct 10, 2024
2a5d650
Merge branch 'maint/pymapdl_maintenance' of https://github.com/ansys/…
clatapie Oct 10, 2024
b31af98
Merge branch 'main' into maint/pymapdl_maintenance
clatapie Oct 10, 2024
425bd60
fix: ``download_examples``
clatapie Oct 10, 2024
9830881
fix: adding ``requests`` in ``minimum_requirements`` and reverting ch…
clatapie Oct 10, 2024
750003b
ci: auto fixes from pre-commit.com hooks.
pre-commit-ci[bot] Oct 10, 2024
5880358
fix: typo
clatapie Oct 10, 2024
5cf1885
fix: removing a warning
clatapie Oct 10, 2024
8afc259
feat: removing ``shell=True``
clatapie Oct 10, 2024
9e3c559
fix: attemp to fix the ``PermissionError``
clatapie Oct 10, 2024
7e2c39c
Merge branch 'main' into maint/pymapdl_maintenance
clatapie Oct 11, 2024
545ee64
test: providing higher permissions to run the exec file
clatapie Oct 11, 2024
8169058
Merge branch 'maint/pymapdl_maintenance' of https://github.com/ansys/…
clatapie Oct 11, 2024
e1f7b3a
Merge branch 'main' into maint/pymapdl_maintenance
clatapie Oct 11, 2024
172d890
test: using Python 3.12 in CICD for better understanding of the output
clatapie Oct 11, 2024
9812f68
fix: removing `"` character
clatapie Oct 11, 2024
ab6b7e3
fix: adding `'` character
clatapie Oct 11, 2024
23781d2
fix: reverting changes in subprocess
clatapie Oct 11, 2024
3f84961
fix: checking whether other changes are correct or not
clatapie Oct 11, 2024
86218d0
fix: `` test__is_ubuntu``
clatapie Oct 11, 2024
21fe1fb
fix: using ``executable`` in ``subprocess.Popen``
clatapie Oct 11, 2024
b920a65
fix: ``proc``
clatapie Oct 11, 2024
bf8cd48
fix: attempt to fix test errors
clatapie Oct 11, 2024
5f57e33
fix: fix typo
clatapie Oct 11, 2024
5932bfa
Merge branch 'main' into maint/pymapdl_maintenance
clatapie Oct 14, 2024
87722db
fix: ``_is_ubuntu``
clatapie Oct 14, 2024
df93a73
Merge branch 'maint/pymapdl_maintenance' of https://github.com/ansys/…
clatapie Oct 14, 2024
b6565df
fix: testing another approach for ``is_ubuntu`` method
clatapie Oct 14, 2024
3bc7f7c
fix: attempt to use ``shell=False`` in ``launch_grpc``
clatapie Oct 14, 2024
b440434
fix: removing empty args
clatapie Oct 14, 2024
f60012e
test: attempt to fix ``command_parm``
clatapie Oct 14, 2024
f490b90
test: attempt to fix ``command_parm`` - 2
clatapie Oct 14, 2024
c2e3781
ci: auto fixes from pre-commit.com hooks.
pre-commit-ci[bot] Oct 14, 2024
a94714b
Update src/ansys/mapdl/core/examples/downloads.py
clatapie Oct 14, 2024
effd48e
test: attempt to fix ``command_parm`` - 3
clatapie Oct 14, 2024
53a8dc6
fix: ``call`` in ``open_gui``
clatapie Oct 14, 2024
31e2e91
fix: removing unused variable ``e``
clatapie Oct 14, 2024
247752b
revert: reverting change for ``MAIN_PYTHON_VERSION``
clatapie Oct 14, 2024
3b9e847
chore: adding changelog file 3474.added.md [dependabot-skip]
pyansys-ci-bot Oct 14, 2024
646d6aa
Merge branch 'main' into maint/pymapdl_maintenance
clatapie Oct 15, 2024
2bc1915
Merge branch 'main' into maint/pymapdl_maintenance
clatapie Oct 16, 2024
365a74d
Apply suggestions from code review
clatapie Oct 16, 2024
61cd47d
Apply suggestions from code review
clatapie Oct 16, 2024
c644fb6
chore: merge remote-tracking branch 'origin/main' into maint/pymapdl_…
germa89 Oct 16, 2024
c751eae
test: fix tests
germa89 Oct 16, 2024
3acc746
test: reverting change in ``f'"{exec_file}"'``
clatapie Oct 16, 2024
4e059a6
fix: ``exec_file`` errors
clatapie Oct 17, 2024
027bfa5
Merge branch 'main' into maint/pymapdl_maintenance
clatapie Oct 18, 2024
41f716e
chore: merge remote-tracking branch 'origin/main' into maint/pymapdl_…
germa89 Oct 21, 2024
cbbae17
Merge branch 'main' into maint/pymapdl_maintenance
clatapie Oct 21, 2024
09a9a25
review: applying @germa89's suggestions
clatapie Oct 21, 2024
7b846be
Apply suggestions from code review
clatapie Oct 21, 2024
25a9db4
review: applying @germa89's suggestions
clatapie Oct 21, 2024
19cbea9
Update src/ansys/mapdl/core/mapdl_core.py
clatapie Oct 21, 2024
2235a19
review: applying @germa89's suggestions - 2
clatapie Oct 21, 2024
4e0688c
fux: using ``self._log.debug``
clatapie Oct 21, 2024
d845e24
fix: ``test_examples.py``
clatapie Oct 21, 2024
7b857d3
fix: ``generate_mapdl_launch_command``
clatapie Oct 21, 2024
8304fd4
fix: ``_checkout_license``
clatapie Oct 21, 2024
52c7b8c
fix: ``launch_grpc`` for Linux
clatapie Oct 21, 2024
752f735
Apply suggestions from code review
clatapie Oct 22, 2024
30aae0a
review: applying @germa89 and @RobPasMue's suggestions
clatapie Oct 22, 2024
0ddb90b
review: applying code reviewers suggestions
clatapie Oct 22, 2024
a1154bf
fix: using ``cwd`` arg in ``subprocess.call``
clatapie Oct 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/changelog.d/3474.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
refactor: modifying ``subprocess`` calls and removing ``try except continue`` statements
2 changes: 1 addition & 1 deletion minimum_requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ importlib-metadata==8.5.0
numpy==2.1.2
platformdirs==4.3.6
psutil==6.1.0
pyansys-tools-versioning==0.6.0
pyansys-tools-versioning==0.6.0
48 changes: 23 additions & 25 deletions src/ansys/mapdl/core/examples/downloads.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,13 @@
from functools import wraps
import os
import shutil
import urllib.request
import zipfile

try:
import requests

_HAS_REQUESTS = True

except ModuleNotFoundError:
_HAS_REQUESTS = False

Expand Down Expand Up @@ -83,14 +83,8 @@ def _get_file_url(filename, directory=None):


def _check_url_exist(url):
if not _HAS_REQUESTS:
raise ModuleNotFoundError("Examples module requires request module")

response = requests.get(url)
if response.status_code == 200:
return [True]
else:
return [False]
response = requests.get(url, timeout=10) # 10 seconds timeout
return response.status_code == 200


@check_directory_exist(pymapdl.EXAMPLES_PATH)
Expand All @@ -103,24 +97,28 @@ def _retrieve_file(url, filename, _test=False):
local_path = os.path.join(pymapdl.EXAMPLES_PATH, os.path.basename(filename))
local_path_no_zip = local_path.replace(".zip", "")
if os.path.isfile(local_path_no_zip) or os.path.isdir(local_path_no_zip):
return local_path_no_zip, None
return local_path_no_zip

# Perform download
saved_file, resp = urllib.request.urlretrieve(url)
shutil.move(saved_file, local_path)
requested_file = requests.get(url, timeout=10)
requested_file.raise_for_status()

with open(local_path, "wb") as f:
f.write(requested_file.content)

if get_ext(local_path) in [".zip"]:
_decompress(local_path)
local_path = local_path[:-4]
return local_path, resp
return local_path
clatapie marked this conversation as resolved.
Show resolved Hide resolved


def _download_file(filename, directory=None, _test=False):
url = _get_file_url(filename, directory)
try:
return _retrieve_file(url, filename, _test)
except Exception as e: # Genering exception
raise RuntimeError(
"For the reason mentioned below, retrieving the file from internet failed.\n"
except requests.exceptions.HTTPError as e:
raise requests.exceptions.HTTPError(
"Retrieving the file from internet failed.\n"
"You can download this file from:\n"
f"{url}\n"
"\n"
Expand All @@ -140,27 +138,27 @@ def download_bracket():
'/home/user/.local/share/ansys_mapdl_core/examples/bracket.iges'

"""
return _download_file("bracket.iges", "geometry")[0]
return _download_file("bracket.iges", "geometry")


def download_tech_demo_data(example, filename):
"""Download Tech Demos external data."""
example = "tech_demos/" + example
return _download_file(filename=filename, directory=example)[0]
return _download_file(filename=filename, directory=example)


def download_vtk_rotor():
"""Download rotor vtk file."""
return _download_file("rotor.vtk", "geometry")[0]
return _download_file("rotor.vtk", "geometry")


def _download_rotor_tech_demo_vtk():
"""Download the rotor surface VTK file."""
return _download_file("rotor2.vtk", "geometry")[0]
return _download_file("rotor2.vtk", "geometry")


def download_example_data(filename, directory=None):
return _download_file(filename, directory=directory)[0]
return _download_file(filename, directory=directory)


def download_manifold_example_data() -> dict:
Expand Down Expand Up @@ -188,10 +186,10 @@ def download_manifold_example_data() -> dict:
return {
"geometry": _download_file(
filename="manifold_geometry.anf", directory=files_dir
)[0],
),
"mapping_data": _download_file(
filename="manifold_cht-final_temp.csv", directory=files_dir
)[0],
),
}


Expand Down Expand Up @@ -219,6 +217,6 @@ def download_cfx_mapping_example_data() -> dict:
return {
"data": _download_file(
filename="11_blades_mode_1_ND_0.csv", directory=files_dir
)[0],
"model": _download_file(filename="ExampleMapping.db", directory=files_dir)[0],
),
"model": _download_file(filename="ExampleMapping.db", directory=files_dir),
clatapie marked this conversation as resolved.
Show resolved Hide resolved
}
26 changes: 13 additions & 13 deletions src/ansys/mapdl/core/launcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -166,8 +166,7 @@ def _is_ubuntu() -> bool:
return False

proc = subprocess.Popen(
"awk -F= '/^NAME/{print $2}' /etc/os-release",
shell=True,
["awk", "-F=", "/^NAME/{print $2}", "/etc/os-release"],
stdout=subprocess.PIPE,
)
if "ubuntu" in proc.stdout.read().decode().lower():
Expand Down Expand Up @@ -309,7 +308,7 @@ def generate_mapdl_launch_command(
ram: Optional[int] = None,
port: int = MAPDL_DEFAULT_PORT,
additional_switches: str = "",
) -> str:
) -> list[str]:
"""Generate the command line to start MAPDL in gRPC mode.

Parameters
Expand Down Expand Up @@ -346,7 +345,7 @@ def generate_mapdl_launch_command(

Returns
-------
str
list[str]
Command

"""
Expand All @@ -369,10 +368,10 @@ def generate_mapdl_launch_command(

# Windows will spawn a new window, special treatment
if os.name == "nt":
exec_file = f'"{exec_file}"'
# must start in batch mode on windows to hide APDL window
tmp_inp = ".__tmp__.inp"
command_parm = [
'"%s"' % exec_file,
job_sw,
cpu_sw,
ram_sw,
Expand All @@ -388,7 +387,6 @@ def generate_mapdl_launch_command(

else: # linux
command_parm = [
'"%s"' % exec_file,
job_sw,
cpu_sw,
ram_sw,
Expand All @@ -398,16 +396,19 @@ def generate_mapdl_launch_command(
]

command_parm = [
each for each in command_parm if command_parm
each for each in command_parm if each.strip()
] # cleaning empty args.
command = " ".join(command_parm)

LOG.debug(f"Generated command: {command}")
return command
# removing spaces in cells
command_parm = " ".join(command_parm).split(" ")
command_parm.insert(0, f"{exec_file}")
clatapie marked this conversation as resolved.
Show resolved Hide resolved

LOG.debug(f"Generated command: {' '.join(command_parm)}")
return command_parm


def launch_grpc(
cmd: str,
cmd: list[str],
run_location: str = None,
env_vars: Optional[Dict[str, str]] = None,
) -> subprocess.Popen:
Expand Down Expand Up @@ -442,15 +443,14 @@ def launch_grpc(

if os.name == "nt":
# getting tmp file name
tmp_inp = cmd.split()[cmd.split().index("-i") + 1]
tmp_inp = cmd[cmd.index("-i") + 1]
with open(os.path.join(run_location, tmp_inp), "w") as f:
f.write("FINISH\r\n")
LOG.debug(f"Writing temporary input file: {tmp_inp} with 'FINISH' command.")

LOG.debug("MAPDL starting in background.")
process = subprocess.Popen(
cmd,
shell=os.name != "nt",
cwd=run_location,
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
Expand Down
3 changes: 1 addition & 2 deletions src/ansys/mapdl/core/licensing.py
Original file line number Diff line number Diff line change
Expand Up @@ -329,11 +329,10 @@ def _checkout_license(self, lic, host=None, port=2325):

tstart = time.time()
process = subprocess.Popen(
f'"{ansysli_util_path}" -checkout {lic}',
[f'"{ansysli_util_path}"', "-checkout", f"{lic}"],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
env=env,
shell=True,
)
output = process.stdout.read().decode()

Expand Down
5 changes: 3 additions & 2 deletions src/ansys/mapdl/core/mapdl_console.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import re
import time

from ansys.mapdl.core import LOG
from ansys.mapdl.core.errors import MapdlExitedError, MapdlRuntimeError
from ansys.mapdl.core.mapdl import MapdlBase
from ansys.mapdl.core.misc import requires_package
Expand Down Expand Up @@ -284,8 +285,8 @@ def exit(self, close_log=True, timeout=3):
try:
self._process.sendline("FINISH")
self._process.sendline("EXIT")
except:
pass
except Exception as e:
LOG.warning(f"Unable to exit ANSYS MAPDL: {e}")

if close_log:
self._close_apdl_log()
Expand Down
26 changes: 19 additions & 7 deletions src/ansys/mapdl/core/mapdl_core.py
Original file line number Diff line number Diff line change
Expand Up @@ -500,8 +500,10 @@
while (not self._path and i > 5) or i == 0:
try:
self._path = self.inquire("", "DIRECTORY")
except Exception: # pragma: no cover
pass
except Exception as e: # pragma: no cover
logger.warning(
f"Failed to get the directory due to the following error: {e}"
)
clatapie marked this conversation as resolved.
Show resolved Hide resolved
i += 1
if not self._path: # pragma: no cover
time.sleep(0.1)
Expand Down Expand Up @@ -676,8 +678,8 @@
"""
try:
self._jobname = self.inquire("", "JOBNAME")
except Exception:
pass
except Exception as e:
logger.warning(f"Failed to get the jobname due to the following error: {e}")
return self._jobname

@jobname.setter
Expand Down Expand Up @@ -1693,11 +1695,21 @@
"MAPDL GUI has been opened using 'inplace' kwarg. "
f"The changes you make will overwrite the files in {run_dir}."
)
add_sw = add_sw.split()
exec_array = [
clatapie marked this conversation as resolved.
Show resolved Hide resolved
f"{exec_file}",
"-g",
"-j",
f"{name}",
"-np",
f"{nproc}",
*add_sw,
]

call(
f'cd "{run_dir}" && "{exec_file}" -g -j {name} -np {nproc} {add_sw}',
shell=True,
exec_array,
stdout=DEVNULL,
cwd=run_dir,
)

# Going back
Expand Down Expand Up @@ -2298,7 +2310,7 @@
try: # logger might be closed
if self._log is not None:
self._log.error("exit: %s", str(e))
except Exception:
except ValueError:

Check warning on line 2313 in src/ansys/mapdl/core/mapdl_core.py

View check run for this annotation

Codecov / codecov/patch

src/ansys/mapdl/core/mapdl_core.py#L2313

Added line #L2313 was not covered by tests
pass

def _cleanup_loggers(self):
Expand Down
8 changes: 5 additions & 3 deletions src/ansys/mapdl/core/mapdl_extended.py
Original file line number Diff line number Diff line change
Expand Up @@ -807,9 +807,11 @@

for surf in surfs:
anum = np.unique(surf["entity_num"])
assert (
len(anum) == 1
), f"The pv.Unstructured from the entity {anum[0]} contains entities from other entities {anum}" # Sanity check
if len(anum) != 1:
raise RuntimeError(

Check warning on line 811 in src/ansys/mapdl/core/mapdl_extended.py

View check run for this annotation

Codecov / codecov/patch

src/ansys/mapdl/core/mapdl_extended.py#L811

Added line #L811 was not covered by tests
f"The pv.Unstructured from the entity {anum[0]} contains entities"
f"from other entities {anum}" # Sanity check
)

area = surf.extract_cells(surf["entity_num"] == anum)
centers.append(area.center)
Expand Down
9 changes: 5 additions & 4 deletions src/ansys/mapdl/core/mapdl_grpc.py
Original file line number Diff line number Diff line change
Expand Up @@ -881,8 +881,9 @@
self.prep7()
success = True
break
except:
pass
except MapdlRuntimeError:
time.sleep(1)
warn("PyMAPDL is taking longer than expected to connect to the server.")

Check warning on line 886 in src/ansys/mapdl/core/mapdl_grpc.py

View check run for this annotation

Codecov / codecov/patch

src/ansys/mapdl/core/mapdl_grpc.py#L884-L886

Added lines #L884 - L886 were not covered by tests

if not success:
raise MapdlConnectionError("Unable to reconnect to MAPDL")
Expand Down Expand Up @@ -1025,7 +1026,7 @@
return "".join(response)

def _threaded_heartbeat(self):
"""To be called from a thread to verify mapdl instance is alive"""
"""To be called from a thread to verify MAPDL instance is alive"""
self._initialised.set()
while True:
if self._exited:
Expand All @@ -1038,7 +1039,7 @@
except ReferenceError:
break
except Exception:
continue
self._log.debug("Checking if MAPDL instance is still alive.")

Check warning on line 1042 in src/ansys/mapdl/core/mapdl_grpc.py

View check run for this annotation

Codecov / codecov/patch

src/ansys/mapdl/core/mapdl_grpc.py#L1042

Added line #L1042 was not covered by tests

@protect_from(ValueError, "I/O operation on closed file.")
def exit(self, save=False, force=False, **kwargs):
Expand Down
5 changes: 3 additions & 2 deletions src/ansys/mapdl/core/misc.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@
import os
from pathlib import Path
import platform
import random
import socket
import string
import tempfile
Expand Down Expand Up @@ -378,7 +377,9 @@ def is_float(input_string):

def random_string(stringLength=10, letters=string.ascii_lowercase):
"""Generate a random string of fixed length"""
return "".join(random.choice(letters) for i in range(stringLength))
import secrets

return "".join(secrets.choice(letters) for _ in range(stringLength))


def _check_has_ansys():
Expand Down
Loading
Loading