Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tr/fix author -> tr/test #17

Merged
merged 22 commits into from
Oct 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
c5bf2fb
add github related cicd from dev-profiler repo
tapishr Sep 16, 2023
2c47865
fix version change in python script
tapishr Sep 16, 2023
b4100a0
revert changes of previous commits since not working
tapishr Sep 16, 2023
8f26a9d
Merge pull request #4 from Alokit-Innovations/tr/cicd
avikalpg Sep 18, 2023
855202d
Add setup instructions and description to Readme
tapishr Sep 20, 2023
982d22f
Merge pull request #7 from Alokit-Innovations/tr/readme
MuskanPaliwal Sep 20, 2023
e831f1d
Add instructions about oauth and setup
tapishr Sep 20, 2023
18abb76
Revert "Add setup instructions and description to Readme"
tapishr Sep 20, 2023
342c6c6
Merge pull request #8 from Alokit-Innovations/revert-7-tr/readme
tapishr Sep 20, 2023
472345e
Merge branch 'main' into tr/readme
tapishr Sep 20, 2023
eb01845
Add what happens on port 3000 in instructions
tapishr Sep 20, 2023
1263cfc
Merge branch 'tr/readme' of github.com:Alokit-Innovations/vibi-dpu in…
tapishr Sep 20, 2023
8cdbcc5
Merge pull request #9 from Alokit-Innovations/tr/readme
MuskanPaliwal Sep 20, 2023
850d248
added trigger to run the bitbucket flow test script from main repository
MuskanPaliwal Oct 4, 2023
b5da7cc
renamed cloudbuild.yaml file to bitbucket-test-trigger.yaml
MuskanPaliwal Oct 4, 2023
5c91338
Merge pull request #13 from Alokit-Innovations/mkp/bitbucket-test-script
tapishr Oct 4, 2023
7923a84
Merge pull request #2 from Alokit-Innovations/tr/pubsub
avikalpg Oct 5, 2023
e436bcb
Merge branch 'main' into tr/setup
avikalpg Oct 7, 2023
d95c4d8
Merge pull request #3 from Alokit-Innovations/tr/setup
avikalpg Oct 7, 2023
923d57b
Merge branch 'main' into tr/review
tapishr Oct 13, 2023
d2c9861
add user email instead of display_name in blamevec
tapishr Oct 18, 2023
a75c548
remove unused imports
tapishr Oct 18, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Please check the action items covered in the PR -

- [ ] Build is running
- [ ] Eventing is functional and tested
- [ ] Unit or integration tests added and running
- [ ] Manual QA
81 changes: 81 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@

name: Auto-version

on:
pull_request:
types:
- closed

jobs:
build:
if: github.event.pull_request.merged == true && github.event.pull_request.base.ref == 'main' && github.event.pull_request.head.ref != 'update-cargo-toml'
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v2

- name: Run Build Script
run: |
git config user.name "GitHub Actions Bot"
git config user.email "<>"

- name: run script
run: |
python scripts/update.py

- name: Create Pull Request
uses: peter-evans/create-pull-request@v3
with:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: "Updated code"
title: "Update Cargo.toml"
body: "This pull request updates the Cargo.toml file."
branch: update-cargo-toml
reviewers: tapishr

- name: Build Rust CLI
run: |
cd vibi-dpu
cargo build --release

- name: Get version from Cargo.toml
id: get_version
run: |
cd vibi-dpu
echo "::set-output name=version::$(grep -Po '(?<=version = ")[\d.]+(?=")' Cargo.toml)"

- name: Create Release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ steps.get_version.outputs.version }}
release_name: Release ${{ steps.get_version.outputs.version }}
draft: false
prerelease: false

- name: cargo deb
run: |
cargo install cargo-deb

- name: build binary
run: |
cd vibi-dpu
cargo deb

- name: Rename Debian Package
run: mv ./vibi-dpu/target/debian/*.deb ./vibi-dpu/target/debian/vibi-dpu.deb

- name: Upload .deb Package
id: upload_deb
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./vibi-dpu/target/debian/vibi-dpu.deb
asset_name: vibi-dpu.deb
asset_content_type: application/octet-stream

26 changes: 26 additions & 0 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Rust

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

env:
CARGO_TERM_COLOR: always

jobs:
build:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Build
run: |
cd vibi-dpu
cargo build --verbose
- name: Run tests
run: |
cd vibi-dpu
cargo test --verbose
55 changes: 53 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,53 @@
# vibi-dpu
Vibinex Data Processing Unit for collecting and analyzing private data
# Vibi-DPU

Vibi-DPU is an application written in Rust and packaged as a Docker image. It runs on the users' infrastructure to analyze private Intellectual Property data, empowering users through analysis without sacrificing privacy.

The application communicates with our Next.js server, hosted on our infrastructure, which is also open source. You can find it [here](https://github.com/Alokit-Innovations/team-monitor-website/).

Currently, we analyze code in Git repositories, and are soon planning to add APM data and business events. The insights we get from these analyses are communicated through comments/actions on pull requests and through our [open-source Chrome Extension](https://chrome.google.com/webstore/detail/vibinex-code-review/jafgelpkkkopeaefadkdjcmnicgpcncc).

For more information, visit our website at https://vibinex.com/.

## Setup Instructions

To run Vibi-DPU locally:

1. Generate public url using ngrok - `ngrok http 3000`. We will run our next server locally on port 3000 in later steps.
2. Paste this in OAuth consumers in callback_url field.
3. Clone [team-monitor-webiste](https://github.com/Alokit-Innovations/team-monitor-website/) locally.
4. Paste the client id and secret in team-monitor-wesite in .env.local in root directory. Also use them in the docker command below.
5. Fire up cloud sql proxy - `./cloud-sql-proxy --port 5432 vibi-test-394606:asia-south1:test-db`
6. Change url in team-monitor-website in .env.local - `NEXTAUTH_URL=https://example.ngrok-free.app`
7. Start team-monitor-website - `npm run dev`
8. Build vibi-dpu, go to vibi-dpu/vibi-dpu and run - `cargo build`
9. Go up to the root directory of vibi-dpu - `cd ../`
10. **Build the Docker image**: In the root directory of the project, run the following command to build a Docker image with the name "dpu".

```bash
docker build \
--build-arg GCP_CREDENTIALS=/path/to/your/keyfile.json \
--build-arg TOPIC_NAME=my-topic-name \
--build-arg SUBSCRIPTION_NAME=my-subscription-name \
--build-arg BITBUCKET_CLIENT_ID=your-bitbucket-client-id \
--build-arg BITBUCKET_CLIENT_SECRET=your-bitbucket-client-secret \
--build-arg BITBUCKET_BASE_URL=your-bitbucket-base-url \
--build-arg INSTALL_ID=your-install-id \
--build-arg SERVER_URL=your-server-url \
-t dpu .
```
11. **Run the Docker container**: After building the image, you can run it using the following command.

```bash
docker run dpu
```
12. For bitbucket, replace your url in this url and paste it on your browser and visit it. If you are using ngrok, you might get a "visit site" ngrok welcome page. Click and visit site. Grant any permissions asked from your user to bitbucket. Example URL - `https://bitbucket.org/site/oauth2/authorize?response_type=code&client_id=raFykYJRvEBHPttQAm&redirect_uri=https%3A%2F%2F5bef-171-76-86-89.ngrok-free.app%2Fapi%2Fbitbucket%2Fcallbacks%2Finstall&scope=repository%20pullrequest%20pullrequest:write%20webhook%20account%20repository:write`. You only need to replace the `5bef-171-76-86-89.ngrok-free.app` part with your own ngrok url instead of generating a new formatted url.
13. This would start the "setting up" part of dpu, where it calls bitbucket apis and collects repo info, user info, workspace info and pr info.
14. Next begin your testing. For instance, if you push to a PR, you should be able to see logs in next server, in dpu and see the required actions being performed on the PR.

## Contributing

We welcome contributions from the community! Please read our contributing guidelines before submitting a pull request.

## License

This project is licensed under the terms of the GNU-GPLv3.
22 changes: 22 additions & 0 deletions bitbucket-test-trigger.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
steps:
# Clone the main repository
- name: 'gcr.io/cloud-builders/git'
args: ['clone', '--recursive', 'https://github.com/Alokit-Innovations/test-scripts.git']

# Install Python dependencies
- name: 'gcr.io/cloud-builders/python'
args: ['pip', 'install', '-r', 'requirements.txt']
dir: 'test-scripts'

# Run the script from the main repository
- name: 'gcr.io/cloud-builders/python'
args: ['on-prem-bitbucket-tests.py']
dir: 'test-scripts'
env:
- 'test_oauth_consumer_key=${_TEST_OAUTH_CONSUMER_KEY}'
- 'test_oauth_consumer_secret=${_TEST_OAUTH_CONSUMER_SECRET}'
- 'test_db_host=${_TEST_DB_HOST}'
- 'test_db_port=${_TEST_DB_PORT}'
- 'test_db_name=${_TEST_DB_NAME}'
- 'test_db_user=${_TEST_DB_USER}'
- 'test_db_password=${_TEST_DB_PASSWORD}'
23 changes: 23 additions & 0 deletions scripts/update.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@

import re
import fileinput

# Read the current version from the Cargo.toml file
current_version = ""
with open("devprofiler/Cargo.toml", "r") as cargo_file:
for line in cargo_file:
match = re.search(r'^version\s*=\s*"(.*?)"', line)
if match:
current_version = match.group(1)
break

# Generate a new version number (increment the patch version)
version_parts = current_version.split('.')
new_patch = int(version_parts[2]) + 1
new_version = f"{version_parts[0]}.{version_parts[1]}.{new_patch}"

# Update the Cargo.toml file with the new version number
for line in fileinput.input("devprofiler/Cargo.toml", inplace=True):
line = re.sub(r'^version\s*=\s*".*?"', f'version = "{new_version}"', line.rstrip())
print(line)

63 changes: 47 additions & 16 deletions vibi-dpu/src/utils/gitops.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ use serde::Serialize;
use sha256::digest;

use crate::bitbucket::auth::refresh_git_auth;
use crate::bitbucket::user::get_commit_bb;

use super::hunk::BlameItem;
use super::review::Review;
Expand Down Expand Up @@ -352,7 +351,7 @@ pub async fn generate_blame(review: &Review, linemap: &HashMap<String, Vec<Strin
if blamelines.len() == 0 {
continue;
}
let blamitems_opt = process_blameitem(path, commit, linenum, blamelines, review).await;
let blamitems_opt = process_blameitem(path, linenum, blamelines).await;
if blamitems_opt.is_some() {
let blameitems = blamitems_opt.expect("blameitem not found in blameitem_opt");
blamevec.extend(blameitems);
Expand All @@ -362,7 +361,7 @@ pub async fn generate_blame(review: &Review, linemap: &HashMap<String, Vec<Strin
return blamevec;
}

async fn process_blameitem(path: &str, commit: &str, linenum: &str, blamelines: Vec<&str>, review: &Review) -> Option<Vec<BlameItem>> {
async fn process_blameitem(path: &str, linenum: &str, blamelines: Vec<&str>) -> Option<Vec<BlameItem>> {
let linenumint_res = linenum.parse::<usize>();
let mut blamevec = Vec::<BlameItem>::new();
if linenumint_res.is_err() {
Expand All @@ -371,8 +370,7 @@ async fn process_blameitem(path: &str, commit: &str, linenum: &str, blamelines:
return None;
}
let linenumint = linenumint_res.expect("Uncaught error in linenumint_res");
let lineauthormap = process_blamelines(&blamelines, linenumint,
&review.repo_name(), &review.repo_owner()).await;
let lineauthormap = process_blamelines(&blamelines, linenumint).await;
let mut linebreak = linenumint;
for lidx in linenumint..(linenumint + blamelines.len()-1) {
if lineauthormap.contains_key(&lidx) && lineauthormap.contains_key(&(lidx+1)) {
Expand Down Expand Up @@ -405,21 +403,54 @@ async fn process_blameitem(path: &str, commit: &str, linenum: &str, blamelines:
return Some(blamevec);
}

async fn process_blamelines(blamelines: &Vec<&str>, linenum: usize,
repo_name: &str, repo_owner: &str) -> HashMap<usize, LineItem> {
async fn process_blamelines(blamelines: &Vec<&str>, linenum: usize) -> HashMap<usize, LineItem> {
let mut linemap = HashMap::<usize, LineItem>::new();
for lnum in 0..blamelines.len() {
let ln = blamelines[lnum];
let wordvec: Vec<&str> = ln.split(" ").collect();
let commit = wordvec[0];
let lineitem_opt = get_commit_bb(commit, repo_name, repo_owner).await;
if lineitem_opt.is_some() {
let lineitem = lineitem_opt.expect("Empty linemap_opt");
linemap.insert(
linenum + lnum,
lineitem
);
}
let (author, idx) = extract_author(&wordvec);
let timestamp = extract_timestamp(&wordvec, idx);
let lineitem = LineItem::new(author, timestamp);
linemap.insert(
linenum + lnum,
lineitem
);
}
return linemap;
}

fn extract_author(wordvec: &Vec<&str>) -> (String, usize) {
let mut author = wordvec[1];
let mut idx = 1;
// Check if the second value is an email address (enclosed in angle brackets)
if !author.starts_with('(') && !author.ends_with('>') {
// Shift the index to the next non-empty value
while idx < wordvec.len() && (wordvec[idx] == "" || !wordvec[idx].starts_with('(')){
idx += 1;
}
if idx < wordvec.len() {
author = wordvec[idx];
}
} else {
// Remove the angle brackets from the email address
author = author.trim_start_matches('<').trim_end_matches('>');
}
let authorstr = author.replace("(", "")
.replace("<", "")
.replace(">", "");
return (authorstr, idx)
}

fn extract_timestamp(wordvec: &Vec<&str>, mut idx: usize) -> String {
let mut timestamp = wordvec[2];
if timestamp == "" || timestamp.starts_with('(') {
idx = idx + 1;
while idx < wordvec.len() && (wordvec[idx] == "" || wordvec[idx].starts_with('(')) {
idx = idx + 1;
}
if idx < wordvec.len() {
timestamp = wordvec[idx];
}
}
return timestamp.to_string();
}