Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.0.22 #11

Merged
merged 17 commits into from
Nov 13, 2023
5 changes: 5 additions & 0 deletions .codecov.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
coverage:
status:
project:
default:
threshold: 80%
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ test.py
.coverage.*
tests/cli/data/chpok
tests/cli/data/pok
.idea
101 changes: 64 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,15 @@ Thanks to this package, it is very easy to manage the lifecycle of packages.
## Table of contents

- [**Quick start**](#quick-start)
- [**REPL mode**](#repl-mode)
- [**Script launch mode**](#script-launch-mode)
- [**Special comment language**](#special-comment-language)
- [**Using multiple environments**](#using-multiple-environments)
- [**Context manager mode**](#context-manager-mode)
- [**Installing multiple packages**](#installing-multiple-packages)
- [**Options**](#options)
- [**Using an existing virtual environment**](#using-an-existing-virtual-environment)
- [**Output and logging**](#output-and-logging)
- [**Special comment language**](#special-comment-language)
- [**Using multiple environments**](#using-multiple-environments)
- [**How does it work?**](#how-does-it-work)


Expand All @@ -39,14 +40,16 @@ Install [it](https://pypi.org/project/instld/):
pip install instld
```

And use the library in one of two ways: by running your script through it or by importing a context manager from there.
And use the library in one of three ways: by typing commands via REPL, by running your script through it or by importing a context manager from there.

If you run the script [like this](#script-launch-mode), all dependencies will be automatically installed when the application starts and deleted when it stops:

```bash
instld script.py
```

The [REPL mode](#repl-mode) works in a similar way, you just need to type `instld` in the console to enter it.

You can also call the [context manager](#context-manager-mode) from your code:

```python
Expand All @@ -59,57 +62,39 @@ with instld('some_package'):
Read more about each method, its capabilities and limitations below.


## Script launch mode
## REPL mode

You can use `instld` to run your script. To do this, you need to run a command like this in the console:
REPL mode is the fastest and easiest way to try out other people's libraries for your code. Just type this in your console:

```bash
instld script.py
instld
```

The contents of the script will be executed in the same way as if you were running it through the `python script.py` command. If necessary, you can pass additional arguments to the command line, as if you are running a regular Python script. However, if your program has imports of any packages other than the built-in ones, they will be installed automatically. Installed packages are automatically cleaned up when you exit the program, so they don't leave any garbage behind.


### Special comment language

When using script launch mode, you can specify additional parameters for each import inside your program. To do this, you need to write immediately after it (but always in the same line!) a comment that starts with "instld:", separating key and value pairs with commas.

As example, if the name of the imported module and the package name are different, this code imports the `f` function from the [`fazy`](https://github.com/pomponchik/fazy) library version `0.0.3`:

```python
import f # instld: version 0.0.3, package fazy
After that you will see a welcome message similar to this:

print(f('some string'))
```
⚡ INSTLD REPL based on
Python 3.11.6 (main, Oct 2 2023, 13:45:54) [Clang 15.0.0 (clang-1500.0.40.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

You can also specify only the version or only the package name in the comment, they do not have to be specified together.
>>>
```

Enjoy the regular Python [interactive console mode](https://docs.python.org/3/tutorial/interpreter.html#interactive-mode)! Any libraries that you ask for will be installed within the session, and after exiting it, they will be deleted without a trace. You don't need to "clean up" anything after exiting the console.

### Using multiple environments
In this mode, a [special comment language](#special-comment-language) is fully supported.

The instld script launch mode provides a unique opportunity to use multiple virtual environments at the same time.
## Script launch mode

Firstly, you can run scripts in the main virtual environment, and it will work exactly as you expect:
You can use `instld` to run your script from a file. To do this, you need to run a command like this in the console:

```bash
python3 -m venv venv
source venv/bin/activate
instld script.py
```

When the "import" command is executed in your script, the package will first be searched in the activated virtual environment, and only then downloaded if it is not found there. Note that by default, the activated virtual environment is read-only. That is, it is assumed that you will install all the necessary libraries there before running your script. If you want to install packages in runtime in a specific virtual environment - read about the second method further.

Secondly, you can specify the path to the virtual environment directly [in the comments](#special-comment-language) to a specific import using the `where` directive:

```python
import something # instld: where path/to/the/venv
```

If the path you specified does not exist when you first run the script, it will be automatically created. Libraries installed in this way are not deleted when the script is stopped, therefore, starting from the second launch, the download is no longer required.
The contents of the script will be executed in the same way as if you were running it through the `python script.py` command. If necessary, you can pass additional arguments to the command line, as if you are running a regular Python script. However, if your program has imports of any packages other than the built-in ones, they will be installed automatically. Installed packages are automatically cleaned up when you exit the program, so they don't leave any garbage behind.

Note that the path to the virtual environment in this case should not contain spaces. In addition, there is no multiplatform way to specify directory paths using a comment. Therefore, it is not recommended to use paths consisting of more than one part.

Since script launch mode uses a context manager to install packages "under the hood", you should also read about the features of installing packages in this way in the [corresponding section](#using-an-existing-virtual-environment).
In this mode, as in [REPL](#repl-mode), a [special comment language](#special-comment-language) is fully supported.


## Context manager mode
Expand Down Expand Up @@ -174,7 +159,7 @@ with instld('flask==2.0.2') as context_1:
> ⚠️ Keep in mind that although inter-thread isolation is used inside the library, working with contexts is not completely thread-safe. You can write code in such a way that two different contexts import different modules in separate threads at the same time. In this case, you may get paradoxical results. Therefore, it is recommended to additionally isolate with mutexes all cases where you import something from contexts in different threads.


### Options
## Options

You can use [any options](https://pip.pypa.io/en/stable/cli/pip_install/) available for `pip`. To do this, you need to slightly change the name of the option, replacing the hyphens with underscores, and pass it as an argument to `instld`. Here is an example of how using the `--index-url` option will look like:

Expand Down Expand Up @@ -284,6 +269,48 @@ with instld('flask', catch_output=True):
The `INFO` [level](https://docs.python.org/3/library/logging.html#logging-levels) is used by default. For errors - `ERROR`.


## Special comment language

When using script launch or REPL mode, you can specify additional parameters for each import inside your program. To do this, you need to write immediately after it (but always in the same line!) a comment that starts with "instld:", separating key and value pairs with commas.

As example, if the name of the imported module and the package name are different, this code imports the `f` function from the [`fazy`](https://github.com/pomponchik/fazy) library version `0.0.3`:

```python
import f # instld: version 0.0.3, package fazy

print(f('some string'))
```

You can also specify only the version or only the package name in the comment, they do not have to be specified together.


## Using multiple environments

The instld script launch mode and REPL mode provides a unique opportunity to use multiple virtual environments at the same time.

Firstly, you can run scripts in the main virtual environment, and it will work exactly as you expect:

```bash
python3 -m venv venv
source venv/bin/activate
instld script.py
```

When the "import" command is executed in your script, the package will first be searched in the activated virtual environment, and only then downloaded if it is not found there. Note that by default, the activated virtual environment is read-only. That is, it is assumed that you will install all the necessary libraries there before running your script. If you want to install packages in runtime in a specific virtual environment - read about the second method further.

Secondly, you can specify the path to the virtual environment directly [in the comments](#special-comment-language) to a specific import using the `where` directive:

```python
import something # instld: where path/to/the/venv
```

If the path you specified does not exist when you first run the script, it will be automatically created. Libraries installed in this way are not deleted when the script is stopped, therefore, starting from the second launch, the download is no longer required.

Note that the path to the virtual environment in this case should not contain spaces. In addition, there is no multiplatform way to specify directory paths using a comment. Therefore, it is not recommended to use paths consisting of more than one part.

Since script launch mode uses a context manager to install packages "under the hood", you should also read about the features of installing packages in this way in the [corresponding section](#using-an-existing-virtual-environment).


## How does it work?

This package is essentially a wrapper for `venv` and `pip`.
Expand Down
55 changes: 46 additions & 9 deletions instld/cli/main.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import os
import sys
import code
import builtins
import importlib
import inspect
Expand All @@ -8,13 +9,16 @@
from threading import RLock

import instld
from instld.cli.parsing_comments.get_options_from_comments import get_options_from_comments
from instld.cli.parsing_comments.get_options_from_comments import get_options_from_comments_by_frame
from instld.cli.parsing_arguments.get_python_file import get_python_file
from instld.cli.traceback_cutting.cutting import set_cutting_excepthook
from instld.state_management.storage import state_storage, RunType
from instld.errors import CommentFormatError


def main():
python_file = get_python_file()
state_storage.run_type = RunType.script

with instld() as context:
lock = RLock()
Expand Down Expand Up @@ -49,21 +53,29 @@
last_name = splitted_name[-1]

current_frame = inspect.currentframe()
options = get_options_from_comments(current_frame.f_back)
options = get_options_from_comments_by_frame(current_frame.f_back)

package_name = options.pop('package', base_name)

if 'version' in options:
package_name = f'{package_name}=={options.pop("version")}'

catch_output = options.pop('catch_output', 'no').lower()
if catch_output in ('yes', 'on', 'true'):
catch_output = True

Check warning on line 65 in instld/cli/main.py

View check run for this annotation

Codecov / codecov/patch

instld/cli/main.py#L65

Added line #L65 was not covered by tests
elif catch_output in ('no', 'off', 'false'):
catch_output = False
else:
raise CommentFormatError('For option "catch_output" you can use the following values: "yes", "on", "true", "no", "off", "false".')

Check warning on line 69 in instld/cli/main.py

View check run for this annotation

Codecov / codecov/patch

instld/cli/main.py#L69

Added line #L69 was not covered by tests

current_context = get_current_context(options.pop('where', None))

with lock:
with set_import():
try:
result = __import__(name, *args, **kwargs)
except (ModuleNotFoundError, ImportError) as e:
current_context.install(package_name)
current_context.install(package_name, catch_output=catch_output, **options)
result = current_context.import_here(base_name)
sys.modules[base_name] = result

Expand All @@ -78,13 +90,38 @@

return result

builtins.__import__ = import_wrapper
if python_file is None:
try:
import readline
except ImportError:
pass

Check warning on line 97 in instld/cli/main.py

View check run for this annotation

Codecov / codecov/patch

instld/cli/main.py#L94-L97

Added lines #L94 - L97 were not covered by tests

state_storage.run_type = RunType.REPL
builtins.__import__ = import_wrapper

Check warning on line 100 in instld/cli/main.py

View check run for this annotation

Codecov / codecov/patch

instld/cli/main.py#L99-L100

Added lines #L99 - L100 were not covered by tests

class REPL(code.InteractiveConsole):
def push(self, line):
state_storage.last_string = line
return super().push(line)

Check warning on line 105 in instld/cli/main.py

View check run for this annotation

Codecov / codecov/patch

instld/cli/main.py#L102-L105

Added lines #L102 - L105 were not covered by tests


banner_strings = [

Check warning on line 108 in instld/cli/main.py

View check run for this annotation

Codecov / codecov/patch

instld/cli/main.py#L108

Added line #L108 was not covered by tests
'⚡ INSTLD REPL based on\n'
'Python %s on %s\n' % (sys.version, sys.platform),
'Type "help", "copyright", "credits" or "license" for more information.\n',
]
banner = ''.join(banner_strings)

Check warning on line 113 in instld/cli/main.py

View check run for this annotation

Codecov / codecov/patch

instld/cli/main.py#L113

Added line #L113 was not covered by tests

REPL().interact(banner=banner)

Check warning on line 115 in instld/cli/main.py

View check run for this annotation

Codecov / codecov/patch

instld/cli/main.py#L115

Added line #L115 was not covered by tests


spec = importlib.util.spec_from_file_location('kek', os.path.abspath(python_file))
module = importlib.util.module_from_spec(spec)
sys.modules['__main__'] = module
set_cutting_excepthook(4)
spec.loader.exec_module(module)
else:
builtins.__import__ = import_wrapper
spec = importlib.util.spec_from_file_location('kek', os.path.abspath(python_file))
module = importlib.util.module_from_spec(spec)
sys.modules['__main__'] = module
set_cutting_excepthook(4)
spec.loader.exec_module(module)


if __name__ == "__main__":
Expand Down
7 changes: 2 additions & 5 deletions instld/cli/parsing_arguments/get_python_file.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,5 @@


def get_python_file():
if len(sys.argv) < 2:
print('usage: instld python_file.py [argv ...]', file=sys.stderr)
sys.exit(1)

return sys.argv[1]
if len(sys.argv) >= 2:
return sys.argv[1]
39 changes: 23 additions & 16 deletions instld/cli/parsing_comments/get_comment_string.py
Original file line number Diff line number Diff line change
@@ -1,32 +1,39 @@
from functools import lru_cache

from instld.errors import InstallingPackageError
from instld.state_management.storage import state_storage, RunType


def get_comment_substring_from_string(string):
splitted_line = string.split('#')
right_part = splitted_line[1:]
right_part = '#'.join(right_part)
right_part = right_part.strip()
if right_part.startswith('instld:'):
right_part = right_part[7:].strip()
if right_part:
return right_part
else:
raise InstallingPackageError('An empty list of options in the comment.')

@lru_cache()
def get_comment_string_from_file(line_number, file_name):
try:
with open(file_name, 'r') as file:
for index, line in enumerate(file):
if index + 1 == line_number:
splitted_line = line.split('#')
right_part = splitted_line[1:]
right_part = '#'.join(right_part)
right_part = right_part.strip()
if right_part.startswith('instld:'):
right_part = right_part[7:].strip()
if right_part:
return right_part
else:
raise InstallingPackageError('An empty list of options in the comment.')
break
return get_comment_substring_from_string(line)

except (FileNotFoundError, OSError):
return None

def get_comment_string(frame):
line_number = frame.f_lineno
code = frame.f_code
file_name = code.co_filename
def get_comment_string_by_frame(frame):
if state_storage.run_type == RunType.script:
line_number = frame.f_lineno
code = frame.f_code
file_name = code.co_filename

return get_comment_string_from_file(line_number, file_name)

return get_comment_string_from_file(line_number, file_name)
elif state_storage.run_type == RunType.REPL:
return get_comment_substring_from_string(state_storage.last_string)

Check warning on line 39 in instld/cli/parsing_comments/get_comment_string.py

View check run for this annotation

Codecov / codecov/patch

instld/cli/parsing_comments/get_comment_string.py#L38-L39

Added lines #L38 - L39 were not covered by tests
13 changes: 9 additions & 4 deletions instld/cli/parsing_comments/get_options_from_comments.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
from instld.errors import InstallingPackageError
from instld.cli.parsing_comments.get_comment_string import get_comment_string
from instld.cli.parsing_comments.get_comment_string import get_comment_string_by_frame


def get_options_from_comments(frame):
comment_string = get_comment_string(frame)

def get_options_from_comments(comment_string):
result = {}

if comment_string is not None:
Expand All @@ -21,4 +19,11 @@ def get_options_from_comments(frame):
option_value = splitted_option[1].strip().lower()
result[option_name] = option_value

result.pop('doc', None)
result.pop('comment', None)

return result

def get_options_from_comments_by_frame(frame):
comment_string = get_comment_string_by_frame(frame)
return get_options_from_comments(comment_string)
3 changes: 3 additions & 0 deletions instld/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,6 @@ class RestartingCommandError(Exception):

class RunningCommandError(Exception):
pass

class CommentFormatError(Exception):
pass
4 changes: 2 additions & 2 deletions instld/module/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,10 @@ def new_path(self, module_name):
yield
sys.path = old_path

def install(self, *package_names, **options):
def install(self, *package_names, catch_output=False, **options):
if not package_names:
raise ValueError('You need to pass at least one package name.')

options = convert_options(options)
with self.installer(package_names, options=options):
with self.installer(package_names, catch_output=catch_output, options=options):
pass
Loading
Loading