Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sm 989 create build pipeline for the python language wrapper #369

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
9d67fd0
correct case-sensitive package name
tangowithfoxtrot Nov 13, 2023
fae55d4
init python workflow
tangowithfoxtrot Nov 13, 2023
887477e
replace deprecated setup script with pyproject
tangowithfoxtrot Nov 14, 2023
fa70c28
add module init
tangowithfoxtrot Nov 14, 2023
02d1db0
fix package name
tangowithfoxtrot Nov 14, 2023
7d7d5db
use maturin for builds
tangowithfoxtrot Nov 14, 2023
12de7b6
add push trigger for easier testing
tangowithfoxtrot Nov 14, 2023
f448028
set working dir
tangowithfoxtrot Nov 14, 2023
f16ebe8
troubleshoot schemas
tangowithfoxtrot Nov 14, 2023
5e5d32f
fix schemas being a dir?
tangowithfoxtrot Nov 14, 2023
5ce0c56
rm schema troubleshooting
tangowithfoxtrot Nov 14, 2023
57fb86a
simpler init
tangowithfoxtrot Nov 14, 2023
0b657da
try handling openssl dep for linux
tangowithfoxtrot Nov 14, 2023
8cdf7bb
try building manylinux on host
tangowithfoxtrot Nov 14, 2023
ac05db6
revert building on Linux host; fails manylinux compat
tangowithfoxtrot Nov 14, 2023
57563f8
try setting openssl lib dir
tangowithfoxtrot Nov 20, 2023
224f259
remove sudo?
tangowithfoxtrot Nov 20, 2023
d43a061
apt instead of apt-get; missing on amd64 runner
tangowithfoxtrot Nov 20, 2023
8b108dd
configure libssl earlier
tangowithfoxtrot Nov 20, 2023
b6b17eb
still trying to configure openssl ¯\_(ツ)_/¯
tangowithfoxtrot Nov 20, 2023
5bc824f
add pkgconfig path
tangowithfoxtrot Nov 20, 2023
79b3818
append to existing pkgconfig path
tangowithfoxtrot Nov 20, 2023
a376198
try all the pkgconfig paths
tangowithfoxtrot Nov 20, 2023
c72e4ba
try vendored openssl in `bitwarden-py`
tangowithfoxtrot Nov 20, 2023
ebd103e
Add feature flag for RustTLS support
dani-garcia Nov 20, 2023
d8f0b84
Merge branch 'ps/rusttls' into SM-989-...
tangowithfoxtrot Nov 20, 2023
2207d35
specialize build prep for Windows arm64
tangowithfoxtrot Nov 20, 2023
0cb4f6b
set path in powershell
tangowithfoxtrot Nov 20, 2023
86c41d3
try setting PATH at the wheel build step
tangowithfoxtrot Nov 20, 2023
c58fa70
try setting shell to bash ¯\_(ツ)_/¯
tangowithfoxtrot Nov 20, 2023
3685933
delete file
tangowithfoxtrot Nov 21, 2023
6c9770d
try setting path var at system-level
tangowithfoxtrot Nov 21, 2023
4c5ea05
use workflow path
tangowithfoxtrot Nov 21, 2023
be0e5d8
.
tangowithfoxtrot Nov 21, 2023
46f9012
try setting aarch64 linux linker
tangowithfoxtrot Nov 21, 2023
4f4d251
configure more vars ¯\_(ツ)_/¯
tangowithfoxtrot Nov 21, 2023
94da33c
update
tangowithfoxtrot Nov 21, 2023
8ee5f2c
update
tangowithfoxtrot Nov 21, 2023
5756646
update
tangowithfoxtrot Nov 21, 2023
7ab8d1d
try setting arm var in container
tangowithfoxtrot Nov 21, 2023
04048d4
update
tangowithfoxtrot Nov 21, 2023
d192c9a
update
tangowithfoxtrot Nov 21, 2023
126a200
un-break amd64 linux
tangowithfoxtrot Nov 21, 2023
1af3b46
update
tangowithfoxtrot Nov 21, 2023
adaacdf
Update build-python-wheels.yml
tangowithfoxtrot Nov 21, 2023
e16511c
remove unused
tangowithfoxtrot Nov 21, 2023
4aa2515
add comment for arm linux
tangowithfoxtrot Nov 21, 2023
8b6f329
add ARM64 build tools and clang for win arm
tangowithfoxtrot Nov 21, 2023
3468b6c
fix windows arm target
tangowithfoxtrot Nov 21, 2023
09bb280
echo to `"$GITHUB_PATH"`
tangowithfoxtrot Nov 21, 2023
4cced58
troubleshoot win arm
tangowithfoxtrot Nov 21, 2023
487a08b
troubleshoot win arm
tangowithfoxtrot Nov 21, 2023
41b44dc
use wildcard in bash path
tangowithfoxtrot Nov 21, 2023
9831906
use pwsh to list build tool dir
tangowithfoxtrot Nov 21, 2023
02a9b25
try different build tools dir
tangowithfoxtrot Nov 21, 2023
f781bef
troubleshoot win arm
tangowithfoxtrot Nov 21, 2023
6167b1e
close missing double-quote
tangowithfoxtrot Nov 21, 2023
03dd20d
change arm win build target
tangowithfoxtrot Nov 21, 2023
268f55b
try multiple toolchains for win arm
tangowithfoxtrot Nov 22, 2023
ba584cd
try space-separated target string
tangowithfoxtrot Nov 22, 2023
4908d77
try YAML list
tangowithfoxtrot Nov 22, 2023
4c53545
try unquoted target
tangowithfoxtrot Nov 22, 2023
784d549
update
tangowithfoxtrot Nov 22, 2023
7505afc
try x86 path for build tools
tangowithfoxtrot Nov 22, 2023
c9559da
fix YAML indentation
tangowithfoxtrot Nov 22, 2023
e8eaf3a
plz work
tangowithfoxtrot Nov 22, 2023
a5b5cad
remove Win ARM builds; provide sdist
tangowithfoxtrot Nov 22, 2023
515ca7e
fix sdist artifact name; rm push trigger
tangowithfoxtrot Nov 22, 2023
bcdf143
Merge branch 'master' into SM-989...
tangowithfoxtrot Nov 27, 2023
676a009
Merge branch 'master' into SM-989-create-build-and-publish-pipelines-…
tangowithfoxtrot Nov 27, 2023
b79fc6f
prettier
tangowithfoxtrot Nov 27, 2023
27bfff8
fix version number
tangowithfoxtrot Nov 27, 2023
ffc8cf2
update example
tangowithfoxtrot Nov 27, 2023
2b5c6ce
update with new build instructions
tangowithfoxtrot Nov 27, 2023
bf3772d
simplify imports
tangowithfoxtrot Nov 27, 2023
34364a4
optimize imports; apply PEP formatting
tangowithfoxtrot Nov 27, 2023
2b3fb0b
re-add comment
tangowithfoxtrot Nov 27, 2023
2440843
update min python version
tangowithfoxtrot Nov 27, 2023
a0c7367
Update README.md
tangowithfoxtrot Nov 27, 2023
f8ef3a1
add classifiers
tangowithfoxtrot Nov 28, 2023
b7da700
add beta classifier
tangowithfoxtrot Nov 28, 2023
6274e8d
Merge branch 'master' into SM-989-create-build-and-publish-pipelines-…
tangowithfoxtrot Nov 29, 2023
0399eae
revert to openssl for ARM Linux; builds are failing
tangowithfoxtrot Nov 29, 2023
ac133f9
fix pip not found
tangowithfoxtrot Nov 29, 2023
6795ea8
add -y
tangowithfoxtrot Nov 29, 2023
b704cd0
try pinned container tag for amd linux
tangowithfoxtrot Nov 29, 2023
90a7eba
revert separate builds for amd64 linux
tangowithfoxtrot Nov 29, 2023
25c0506
install pip for LInux x86_64
tangowithfoxtrot Nov 29, 2023
e1d4424
use tagged container image
tangowithfoxtrot Nov 29, 2023
d415251
use openssl in cli
tangowithfoxtrot Nov 30, 2023
ee2ce66
rename to "bitwarden_sdk"
tangowithfoxtrot Nov 30, 2023
b238851
Merge branch 'master' into SM-989-create-build-and-publish-pipelines-…
tangowithfoxtrot Nov 30, 2023
1558df2
Merge branch 'master' into SM-989-create-build-and-publish-pipelines-…
tangowithfoxtrot Nov 30, 2023
0a259f3
Revert "Add feature flag for RustTLS support"
tangowithfoxtrot Dec 5, 2023
3011f4b
Revert "rename to "bitwarden_sdk""
tangowithfoxtrot Dec 5, 2023
56db9f1
Merge branch 'master' into SM-989-create-build-and-publish-pipelines-…
tangowithfoxtrot Dec 5, 2023
489df29
satisfy version req
tangowithfoxtrot Dec 5, 2023
060f1c9
add shebang
tangowithfoxtrot Dec 5, 2023
fe9e684
target 2_28 manylinux to match workflow
tangowithfoxtrot Dec 5, 2023
0b32f06
update tempfile
tangowithfoxtrot Dec 5, 2023
85490df
try workaround for python3
tangowithfoxtrot Dec 7, 2023
df68586
Merge branch 'master' into SM-989-create-build-and-publish-pipelines-…
tangowithfoxtrot Dec 7, 2023
00819d0
update package name
tangowithfoxtrot Dec 7, 2023
89aeffa
Revert "try workaround for python3"
tangowithfoxtrot Dec 7, 2023
c7829d8
update package path
tangowithfoxtrot Dec 7, 2023
bfe4d7a
update branch trigger to `main`
tangowithfoxtrot Dec 13, 2023
4e91e78
Merge branch 'main' into SM-989-create-build-and-publish-pipelines-fo…
tangowithfoxtrot Jan 8, 2024
4d3bbd9
troubleshoot schema path
tangowithfoxtrot Jan 8, 2024
db772a2
Revert "troubleshoot schema path"
tangowithfoxtrot Jan 8, 2024
a91037e
troubleshoot schema path
tangowithfoxtrot Jan 8, 2024
d954658
update download-artifact to 4.1.0
tangowithfoxtrot Jan 8, 2024
03f95fc
update var names
tangowithfoxtrot Jan 8, 2024
23d7896
format
tangowithfoxtrot Jan 8, 2024
aa4bbdc
use new var names
tangowithfoxtrot Jan 8, 2024
df2e629
remove Windows ARM comments
tangowithfoxtrot Jan 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions .github/workflows/build-python-wheels.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
---
name: Build Python Wheels

on:
pull_request:
push:
branches:
- "main"
- "rc"
- "hotfix-rc"
workflow_dispatch:

defaults:
run:
shell: bash
working-directory: languages/python

jobs:
generate_schemas:
uses: ./.github/workflows/generate_schemas.yml

setup:
name: Setup
runs-on: ubuntu-22.04
outputs:
package_version: ${{ steps.retrieve-version.outputs.package_version }}
steps:
- name: Checkout repo
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

- name: Get Package Version
id: retrieve-version
run: |
VERSION="$(grep -o '^version = ".*"' ../../crates/bitwarden-py/Cargo.toml | grep -Eo "[0-9]+\.[0-9]+\.[0-9]+")"
echo "package_version=$VERSION" >> $GITHUB_OUTPUT

build:
name: Building Python wheel for - ${{ matrix.settings.os }} - ${{ matrix.settings.target }}
runs-on: ${{ matrix.settings.os || 'ubuntu-latest' }}
needs:
- generate_schemas
- setup
env:
_PACKAGE_VERSION: ${{ needs.setup.outputs.package_version }}
strategy:
fail-fast: false
matrix:
settings:
- os: macos-12
target: x86_64-apple-darwin

- os: macos-12
target: aarch64-apple-darwin

- os: windows-2022
target: x86_64-pc-windows-msvc

- os: ubuntu-22.04
target: x86_64-unknown-linux-gnu

- os: ubuntu-22.04
target: aarch64-unknown-linux-gnu

steps:
- name: Checkout repo
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

- name: Setup Node
uses: actions/setup-node@8f152de45cc393bb48ce5d89d36b731f54556e65 # v4.0.0
with:
node-version: 18

- name: Install rust
uses: dtolnay/rust-toolchain@439cf607258077187679211f12aa6f19af4a0af7 # stable
with:
toolchain: stable
targets: ${{ matrix.settings.target }}

- name: Cache cargo registry
uses: Swatinem/rust-cache@3cf7f8cc28d1b4e7d01e3783be10a97d55d483c8 # v2.7.1
with:
key: ${{ matrix.settings.target }}-cargo-${{ matrix.settings.os }}

- name: Retrieve schemas
uses: actions/download-artifact@f44cd7b40bfd40b6aa1cc1b9b5b7bf03d3c67110 # v4.1.0
with:
name: schemas.py
path: ${{ github.workspace }}/languages/python/bitwarden_sdk

- name: Build wheels
if: ${{ matrix.settings.target != 'x86_64-unknown-linux-gnu' }}
uses: PyO3/maturin-action@b9e8f88fd4448fdecf5095864cdc7e39a544aa9f # v1.40.7
with:
target: ${{ matrix.settings.target }}
args: --release --find-interpreter --sdist
sccache: "true"
manylinux: "2_28" # https://github.com/pola-rs/polars/pull/12211
working-directory: ${{ github.workspace }}/languages/python

- name: Build wheels (Linux - x86_64)
if: ${{ matrix.settings.target == 'x86_64-unknown-linux-gnu' }}
uses: PyO3/maturin-action@b9e8f88fd4448fdecf5095864cdc7e39a544aa9f # v1.40.7
with:
target: ${{ matrix.settings.target }}
args: --release --find-interpreter --sdist
container: quay.io/pypa/manylinux_2_28_x86_64:2023-11-20-745eb52
sccache: "true"
manylinux: "2_28" # https://github.com/pola-rs/polars/pull/12211
working-directory: ${{ github.workspace }}/languages/python

- name: Upload wheels
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
with:
name: bitwarden_sdk-${{ env._PACKAGE_VERSION }}-${{ matrix.settings.target }}
path: ${{ github.workspace }}/target/wheels/bitwarden_sdk*.whl

- name: Upload sdists
if: ${{ matrix.settings.target == 'x86_64-unknown-linux-gnu' }} # we only need one sdist
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
with:
name: bitwarden_sdk-${{ env._PACKAGE_VERSION }}-sdist
path: ${{ github.workspace }}/target/wheels/bitwarden_sdk-*.tar.gz
45 changes: 23 additions & 22 deletions languages/python/README.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,32 @@
# Requirements

- Python3
- setuptools
```bash
pip install setuptools
```
- setuptools_rust
```bash
pip install setuptools_rust
```
- dateutil
```bash
pip install python-dateutil
```

# Installation

From the `languages/python/` directory,
# Build locally
## Requirements

- Python 3
- `maturin` (install with `pip install maturin[patchelf]`)
- `npm`

## Build

From the root of the repository:
```bash
python3 ./setup.py develop
npm run schemas # generate schemas.py

cd languages/python/
maturin develop
```

Rename the the resulting `.so` file to `bitwarden_py.so`, if it isn't already there.
You can now import `BitwardenClient` in your Python code.

# Use without building locally

```bash
pip install BitwardenClient
```

# Run

Set the `ORGANIZATION_ID` and `ACCESS_TOKEN` environment variables to your organization ID and access token, respectively.

```bash
python3 ./login.py
python3 ./example.py
```
13 changes: 13 additions & 0 deletions languages/python/bitwarden_sdk/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
"""The official Bitwarden client library for Python."""

__version__ = "0.1.0"

from .bitwarden_client import *
from .schemas import *

__doc__ = bitwarden_client.__doc__
if hasattr(bitwarden_client, "__all__"):
__all__ = bitwarden_client.__all__

if hasattr(schemas, "__all__"):
__all__ += schemas.__all__
57 changes: 41 additions & 16 deletions languages/python/example.py
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,29 +1,35 @@
import json
#!/usr/bin/env python3
import logging
import sys
from bitwarden_sdk.bitwarden_client import BitwardenClient
from bitwarden_sdk.schemas import client_settings_from_dict, DeviceType
import os

from bitwarden_sdk import BitwardenClient, DeviceType, client_settings_from_dict

# Create the BitwardenClient, which is used to interact with the SDK
client = BitwardenClient(client_settings_from_dict({
"apiUrl": "http://localhost:4000",
"deviceType": DeviceType.SDK,
"identityUrl": "http://localhost:33656",
"userAgent": "Python",
}))
client = BitwardenClient(
client_settings_from_dict(
{
"apiUrl": os.getenv("API_URL", "http://localhost:4000"),
"deviceType": DeviceType.SDK,
"identityUrl": os.getenv("IDENTITY_URL", "http://localhost:33656"),
"userAgent": "Python",
}
)
)

# Add some logging & set the org id
logging.basicConfig(level=logging.DEBUG)
organization_id = "org_id_here"
organization_id = os.getenv("ORGANIZATION_ID")

# Attempt to authenticate with the Secrets Manager Access Token
client.access_token_login("access_token_here")
client.access_token_login(os.getenv("ACCESS_TOKEN"))

# -- Example Project Commands --

project = client.projects().create("ProjectName", organization_id)
project2 = client.projects().create("Project - Don't Delete Me!", organization_id)
updated_project = client.projects().update(project.data.id, "Cool New Project Name", organization_id)
updated_project = client.projects().update(
project.data.id, "Cool New Project Name", organization_id
)
get_that_project = client.projects().get(project.data.id)

input("Press Enter to delete the project...")
Expand All @@ -33,9 +39,28 @@

# -- Example Secret Commands --

secret = client.secrets().create("TEST_SECRET", "This is a test secret", organization_id, "Secret1234!", [project2.data.id])
secret2 = client.secrets().create("Secret - Don't Delete Me!", "This is a test secret that will stay", organization_id, "Secret1234!", [project2.data.id])
secret_updated = client.secrets().update(secret.data.id, "TEST_SECRET_UPDATED", "This as an updated test secret", organization_id, "Secret1234!_updated", [project2.data.id])
secret = client.secrets().create(
"TEST_SECRET",
"This is a test secret",
organization_id,
"Secret1234!",
[project2.data.id],
)
secret2 = client.secrets().create(
"Secret - Don't Delete Me!",
"This is a test secret that will stay",
organization_id,
"Secret1234!",
[project2.data.id],
)
secret_updated = client.secrets().update(
secret.data.id,
"TEST_SECRET_UPDATED",
"This as an updated test secret",
organization_id,
"Secret1234!_updated",
[project2.data.id],
)
secret_retrieved = client.secrets().get(secret.data.id)

input("Press Enter to delete the secret...")
Expand Down
29 changes: 29 additions & 0 deletions languages/python/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
[build-system]
build-backend = "maturin"
requires = ["maturin>=1.0,<2.0", "setuptools_rust>=1.8.1"]

[project]
authors = [{ name = "Bitwarden", email = "support@bitwarden.com" }]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: Other/Proprietary License",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Rust",
"Topic :: Security",
]
dependencies = ["dateutils >= 0.6.6"]
description = "A Bitwarden Client for python"
name = "bitwarden_sdk"
readme = "README.md"
requires-python = ">=3.0"
version = "0.1.0"

[tool.maturin]
bindings = "pyo3"
compatibility = "2_28"
include = [
{ path = "bitwarden_sdk/*.py", format = ["sdist", "wheel"] }
]
manifest-path = "../../crates/bitwarden-py/Cargo.toml"
python-packages = ["bitwarden_sdk"]
12 changes: 0 additions & 12 deletions languages/python/setup.py

This file was deleted.