Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PrefixSum Re-enable tests. #420

Open
wants to merge 671 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
671 commits
Select commit Hold shift + click to select a range
32829a2
Changing weights on CPU copies it to GPU
l-hoang Nov 11, 2020
9527c90
GPU conv layer forward pass test
l-hoang Nov 11, 2020
818b685
When init CuBLAS, set var to true
l-hoang Nov 11, 2020
25a8d19
GPU allocation helper function on GNNLayer
l-hoang Nov 11, 2020
75719d0
Prepping GCN Layer backward phase for GPU code
l-hoang Nov 11, 2020
6af0343
GPU GCN weight gradient/layer gradient calc
l-hoang Nov 12, 2020
d1a7eff
Functions for backward/weight gradient from GPU
l-hoang Nov 12, 2020
614651f
Readded zero'ing of output matrix for aggregation
l-hoang Nov 12, 2020
068387c
GPU GCN layer unit test: simple forward/backward
l-hoang Nov 12, 2020
bbe5fe7
Softmax layer funcs prep split into CPU/GPU
l-hoang Nov 13, 2020
e735eb6
Dataset masks in GNNs now chars
l-hoang Nov 13, 2020
85c544e
Cleanup to Softmax layer to let GPU build work
l-hoang Nov 13, 2020
6139288
Copy over GNN node masks to GPU
l-hoang Nov 13, 2020
cba333b
Softmax GPU object + hook to its forward phase
l-hoang Nov 13, 2020
ed546d4
Softmax function for GPUs
l-hoang Nov 14, 2020
8017d6a
CUDA_KERNEL_LOOP and some helper calcs
l-hoang Nov 14, 2020
5eb6b4d
Definition of forward phase gpu softmax
l-hoang Nov 14, 2020
77ec2f3
Softmax forward phase fixes; unit test for it too
l-hoang Nov 17, 2020
bf1b355
Softmax: mask selection function refactoring
l-hoang Nov 17, 2020
f71b7c6
Fixed ground truth type on GPUs
l-hoang Nov 18, 2020
4562c8b
Softmax backward phase on GPU
l-hoang Nov 18, 2020
4e5266c
Structure added for GPU global accuracy function
l-hoang Nov 18, 2020
b9e3b32
Allocate GPU memory for Adam optimizer
l-hoang Nov 18, 2020
af9c72a
GradientDescent call now uses PointerWithSize
l-hoang Nov 18, 2020
599ee5f
Adam optimizer on GPU + test done; CPU fix
l-hoang Nov 19, 2020
5fd1abf
gpu epoch test; fixes to returned pointers
l-hoang Nov 20, 2020
e500cb0
GNN: Copy over norm factors from CPU to GPU
l-hoang Nov 24, 2020
0cdaaf5
GPU GCN aggregation uses norm factors
l-hoang Nov 25, 2020
982a0bc
Init function for CuRAND
l-hoang Nov 25, 2020
256c560
GNNLayer GPU: init dropout memory
l-hoang Nov 25, 2020
0fa6ea7
Added CuRAND uniform generate wrapper call
l-hoang Nov 25, 2020
b52e1cb
GNN GPU: forward phase dropout
l-hoang Nov 25, 2020
a186e58
GPU GNN: backward phase dropout derivative
l-hoang Nov 25, 2020
461d575
Gradient sync to sum, master only softmax
l-hoang Dec 16, 2020
98509eb
Old code fixing: zero out output matrices
l-hoang Dec 19, 2020
f20e473
Testing splits for ogbn datasets
l-hoang Jan 7, 2021
4e58928
Off by one for ogbn training splits
l-hoang Jan 7, 2021
2ae60a6
multilabel reading test
l-hoang Jan 9, 2021
3398c5f
F1 scoring for multi-class labels
l-hoang Jan 11, 2021
4025be2
GNN sigmoid output layer
l-hoang Jan 12, 2021
50537bc
Sigmoid and multiclass option added to gnnbench
l-hoang Jan 12, 2021
3ffe9ed
Fixing cross-entropy loss; no ReLU for last layer
l-hoang Jan 19, 2021
3bf8339
Fixing Sigmoid and GCN derivatives
l-hoang Jan 22, 2021
ef58e82
Fixing GPU code/build
l-hoang Jan 27, 2021
eca2d0d
Various comments on GCN layer
l-hoang Jan 28, 2021
0dc99b2
Add sigmoid test, fix conv test
l-hoang Jan 28, 2021
53f6140
gpuconv layer test fix based on backward phase fix
l-hoang Jan 30, 2021
f0802df
Configuration hooks for random subgraph sampling
l-hoang Feb 1, 2021
e72595a
Layers account for sampling: sampling test
l-hoang Feb 1, 2021
f573b1d
gcn app sampling option: loss/accuracy report fix
l-hoang Feb 2, 2021
c2582e9
Option for diff. sample rates; test sample fix
l-hoang Feb 2, 2021
e393c03
ogbn-proteins split
l-hoang Feb 5, 2021
78f1e86
Inductive gnn training
l-hoang Feb 5, 2021
5b95e08
Sample and inductive norm factor correction
l-hoang Feb 6, 2021
c704c5a
Implement a distributed multi-gpu GCN. (#1)
nicelhc13 Feb 9, 2021
0f99d7c
libgnn: SAINT sampling, dense layer, var renames
l-hoang Feb 11, 2021
781b0cb
GALOIS_LOG_VASSERT
l-hoang Feb 11, 2021
e0a6f8c
gcn-app allows specifying layers; var changes
l-hoang Feb 11, 2021
3112b9e
Fix libgnn tests; disable self aggregation option
l-hoang Feb 11, 2021
6607147
Partial SAGE layer + self loop norm fix in GCN
l-hoang Feb 17, 2021
a0a1427
SAGE layer: forward phase
l-hoang Feb 18, 2021
0b43a3a
train/val/test splits for non-complete masks
l-hoang Feb 19, 2021
c80ede8
GNNLayer: optimize function made virtual
l-hoang Feb 19, 2021
bd25d8d
L2 normalization layer + test
l-hoang Feb 22, 2021
0b7c2eb
SAGE Layer completed + tests
l-hoang Feb 23, 2021
8ee7093
Fixed dropout derivative bug in Dense/GCN layers
l-hoang Feb 23, 2021
95d8c07
VectorMultAdd
l-hoang Feb 24, 2021
9cba4e2
GCN layer aggregation uses fused multiply add
l-hoang Feb 24, 2021
c89ad46
Preliminary timers
l-hoang Feb 24, 2021
c279dff
SAGE aggregation: fused multiply add
l-hoang Feb 24, 2021
34199ab
Fix GNN CVC by fixing the edge master function
l-hoang Feb 26, 2021
f71e418
Support for non existent labels (-1)
l-hoang Feb 26, 2021
2543fb9
(attempted) -1 fixing for graph label reading
l-hoang Feb 27, 2021
5bff7cc
Val/test intervals added to training + timers
l-hoang Feb 28, 2021
b7a01bd
Fixed aggregate sync in GNNs
l-hoang Mar 1, 2021
c983f2c
Func to sync layer weight matrix; layer init
l-hoang Mar 1, 2021
9213478
SAGE paired matrix init
l-hoang Mar 2, 2021
e3bbbc3
SAGE distribution: sync 2nd set of weights
l-hoang Mar 2, 2021
97ca9be
Cleanup some prints in cusp and gnn; ogbn100m
l-hoang Mar 2, 2021
7b0d246
aggregate sync test for gnn runs via ctest
l-hoang Mar 3, 2021
f2960b1
GNN replication factor fix
l-hoang Mar 3, 2021
660e5ea
Timer chnages for layers and GNN
l-hoang Mar 3, 2021
89604a5
Forcefully change last layer to match label classes
l-hoang Mar 4, 2021
1bd09da
Dist. acc consistency, weight sync collective
l-hoang Mar 6, 2021
5ce1c24
Bitset now used for GNN aggregation
l-hoang Mar 6, 2021
1bf25f4
Slightly more intelligent bitset for GNN
l-hoang Mar 6, 2021
011fc14
Softmax-cross-entropy derivative merged
l-hoang Mar 7, 2021
8d80e9e
better serialization for vec of vectors in gluon
l-hoang Mar 9, 2021
6c6947d
Size usage, weight sync sub removal, fixes
l-hoang Mar 9, 2021
3c15b65
2 minor fixes: paper100M split, features size
l-hoang Mar 9, 2021
5f1f896
The big GNN space saving commit (part 1)
l-hoang Mar 11, 2021
d95af12
Backward pass call takes non-const forward input
l-hoang Mar 11, 2021
347018f
GNN space saving (part 2): no softmax output
l-hoang Mar 12, 2021
bc1b748
buffer overflow check
l-hoang Mar 18, 2021
81249b6
Remove host fence
l-hoang Mar 16, 2021
39bec32
Adam optimizer fix: epsilon outside sqrt
l-hoang Apr 1, 2021
fa1d597
GNNGraph now has edge sample bit and reverse edges
l-hoang Apr 14, 2021
386fbb8
CSR_CSC always creates in-to-out mapping
l-hoang Apr 15, 2021
fbfe895
Separate edge sample data; data also vector now
l-hoang Apr 15, 2021
7891816
GNN set sample edge via in-edge; more tests
l-hoang Apr 15, 2021
4065206
Basic edge sampling for distributed SAGE
l-hoang Apr 19, 2021
51cab96
CSR/CSC Graph: manual construction functions
l-hoang Apr 23, 2021
455159e
AVX512 for vector add/mul function
l-hoang Apr 23, 2021
4d7bc09
MKL link update: 20.0 and parallel link
l-hoang Apr 24, 2021
0d31eff
GNNSubgraph in Graph
l-hoang Apr 26, 2021
5b48172
WIP inductive training on GNN
l-hoang Apr 26, 2021
8674b90
Implement split/combine communications based on Katana's implementati…
nicelhc13 Apr 27, 2021
7970157
Inductive/sampling for training now co-exist
l-hoang Apr 28, 2021
d001e00
SAGE sync fix + bitset fix for GNNs
l-hoang Apr 28, 2021
d7d2e3d
sample-bit-test fix: call update
l-hoang Apr 28, 2021
6a7e09d
Distributed calculation of gnn node degrees
l-hoang Apr 29, 2021
bcffe89
Inductive graph uses train degrees rather
l-hoang Apr 29, 2021
42448f0
GNNSubgraph fix edge dest construction
l-hoang Apr 30, 2021
7464acf
GNN graph (S)ID conversion helper functions
l-hoang Apr 30, 2021
1217c4b
SAGE layer: distributed subgraph compatibility
l-hoang Apr 30, 2021
4f03bdf
Sampled degree sync for subgraphs, bug fix
l-hoang May 1, 2021
4b19e5c
GNN single host training minibatching
l-hoang May 3, 2021
69b5268
GPU update (#2)
nicelhc13 May 4, 2021
bd7b7cd
Add a GPU wrapper and fix libgnn CMakList error
nicelhc13 May 4, 2021
50a767a
Minibatch generator loops through masters only
l-hoang May 3, 2021
8366824
Fixes to make distributed minibatch work
l-hoang May 4, 2021
6566351
Fanout CLI, sample bitset, sampling behavior, time
l-hoang May 5, 2021
b6696aa
doInductiveTraining changed to useTrainingSubgraph
l-hoang May 5, 2021
9020522
Inductive subgraph option, fan out fix
l-hoang May 5, 2021
f56ef5c
Addition of an output rows dim for layer dims
l-hoang May 6, 2021
068437e
WIP: optimize row size as layers progress
l-hoang May 7, 2021
faf6b66
GNN resize layer affects output; reset all sampled
l-hoang May 11, 2021
9772fb9
distgraphconvert builds again
l-hoang May 11, 2021
b5a79f4
papers100M contiguous remapping related changes
l-hoang May 11, 2021
fa608ae
GNN subgraph view implementation
l-hoang May 13, 2021
27826c2
Training subgraph resizes rows in GNN; degree norm
l-hoang May 13, 2021
9a016eb
row change optimization for full batch sampling
l-hoang May 13, 2021
a39868c
Test batching
l-hoang May 14, 2021
8ff7cfc
Minibatch test interval
l-hoang May 15, 2021
6ab5a9e
MKL microbenchmark
l-hoang May 18, 2021
2455110
GNN subgraph PODResizableArray rather than vector
l-hoang May 18, 2021
154fd9c
MKL micro delete galois and OMP loop
l-hoang May 18, 2021
ee42d7e
mkl micro: use Large arrays when galois active
l-hoang May 19, 2021
27feed3
Timer disabling option in GNNs
l-hoang May 19, 2021
6e008da
GNN subgraphs always use global norm factor
l-hoang May 21, 2021
7f8779e
Fixed rows for GNN/SAGE (not GCN)
l-hoang May 21, 2021
a23cb14
Back to using subgraph degrees
l-hoang May 21, 2021
2fb3dac
GNN subgraph "choose_all" fix
l-hoang May 22, 2021
aba12dc
Dynamic resizing of intermediate/output matrices
l-hoang May 22, 2021
3e22e29
QoL layers/size CLI specification
l-hoang Jun 1, 2021
d4a8a36
GSTL vs std memory leak test
l-hoang Jun 3, 2021
05888c4
wip 2dvec
l-hoang Jun 5, 2021
8aa64e4
WIP part 2: cut off 0s
l-hoang Jun 5, 2021
2de8ab5
WIP #3: write directly to 2d vector
l-hoang Jun 5, 2021
3d33e96
Directly serialize vectors to comm buffer
l-hoang Jun 7, 2021
5b44273
Direct deserialization of GNN sync
l-hoang Jun 7, 2021
3a80d24
MKL microbenchmark additions and changes
l-hoang Jun 9, 2021
7ef6e89
mkl micro: transpose for single thread mkl
l-hoang Jun 10, 2021
fbdf833
SampledAny opt, fix end-of-execution free overhead
l-hoang Jun 16, 2021
9146c0c
DynamicBitsets for each layer's edge samples
l-hoang Jun 16, 2021
23d449b
GNN: Optimize SID map, fix subgraph master masking
l-hoang Jun 17, 2021
b8d89a1
prints for gnnsubgraph
l-hoang Jun 25, 2021
2ac6505
Fixed empty master/mirror counting in subgraph
l-hoang Jun 25, 2021
ea88e27
Remove dead mirrors from GNN sampling
l-hoang Jun 26, 2021
8f6bfde
GNN: FATAL in dead test code path
l-hoang Jun 26, 2021
2c2008c
Fixed row correction code to remove dead mirrors
l-hoang Jun 27, 2021
8c15669
Sample sync; non subgraph node needs to return vec
l-hoang Jun 29, 2021
3d820f6
Resizes layers after mirror nodes are deleted
l-hoang Jun 30, 2021
46196de
Softmax backward timer fix
l-hoang Jun 30, 2021
c99236c
Adjustable learning rate for Adam optimizer
l-hoang Jun 30, 2021
46c07d9
Timers for full forward/backward phase in GNN
l-hoang Jun 30, 2021
7124870
sage layer assertion fix
l-hoang Jul 3, 2021
daf355f
Mirror/master nodes in gluon now pointers
l-hoang Jul 4, 2021
225f3fc
Subgraph mirror creation for GNN sampling
l-hoang Jul 4, 2021
59d6dee
Argument for gnn gluon handshake to disable timers
l-hoang Jul 6, 2021
1933cbf
Added one timer for all GNN backward syncs
l-hoang Jul 7, 2021
8799bff
Optimize layer 0 backward for SAGE layer
l-hoang Jul 8, 2021
3ab8b26
Undo last commit
l-hoang Jul 8, 2021
20c9ce8
Sample flag sync can be readSource
l-hoang Jul 12, 2021
44c003d
GCN barrier before start
l-hoang Jul 13, 2021
14c6168
Parallel SID mapping for GNN subgraph
l-hoang Jul 13, 2021
7546f21
Minibatch shuffling between epochs
l-hoang Jul 17, 2021
b68fcae
WIP: timing subgraph clears and beginning to fix
l-hoang Jul 20, 2021
44e7995
Parallel bitset reset and parallel std::fill
l-hoang Jul 20, 2021
5648fd9
ogbnpapers remapping program
l-hoang Jul 20, 2021
0b8538b
GNN softmax, sync, masking correctness changes
l-hoang Jul 22, 2021
1f0dbf6
GNN: time-based randomness; parreset in subgraph
l-hoang Jul 22, 2021
a7477e8
GNN distributed minibatch shuffler
l-hoang Jul 24, 2021
1a0a94d
Updated the distributed minibatch sampler
l-hoang Jul 28, 2021
97e8632
Test: all hosts share the same minibatch shuffle
l-hoang Jul 31, 2021
c42688d
Major mirror deletion fix: don't delete masters
l-hoang Aug 11, 2021
c27153a
Revert minibatch selection to "pick locally"
l-hoang Aug 12, 2021
916d25f
Warning/crashes added to layers
l-hoang Aug 12, 2021
adbd3b5
Add design doc README, some comments to SAGE
l-hoang Aug 13, 2021
051f88b
Fix a minor bug with a file path
May 22, 2023
7bd324a
Add timers for time breakdown
nicelhc13 Jun 14, 2023
5ab5c10
WMD CSV-based graph ingestion in Galois (#3)
hochanlee-amd Jul 25, 2023
d9a1c69
Use a separate vector to specify sampled nodes instead of a node labe…
hochanlee-amd Aug 2, 2023
01cdb4c
Templatizing graphs and layers, feature construction and training/tes…
hochanlee-amd Aug 31, 2023
40609e2
Add ego graph construction to GCN
hochanlee-amd Sep 16, 2023
e9a2a03
Fixed cpuinf parsing bug (#10)
marcopolo4096 Sep 20, 2023
31e32c2
Add ReLU layer (#9)
hochanlee-amd Sep 22, 2023
9c74645
feat: Add pre-commit to the repo and contributing guidelines
patrickkenney9801 Oct 2, 2023
57618b1
chore: Run clang-format and pre-commit checks on repo
patrickkenney9801 Oct 2, 2023
ce1a079
Make linking MKL libraries in a portable way
hochanlee-amd Oct 24, 2023
80cf952
feat: Add Dockerfile to repo
patrickkenney9801 Oct 24, 2023
5fadc59
Fix for GCN minibatching (#19)
hochanlee-amd Nov 1, 2023
5f7f4d2
Use relaxed consistency in atomic helpers.
insertinterestingnamehere Aug 21, 2020
f8c3387
Fixed merging for gitignore
AdityaAtulTewari Mar 6, 2024
b3ccbfe
Makefile commit
AdityaAtulTewari Mar 6, 2024
99adf0c
merge CMakeLists.txt
AdityaAtulTewari Mar 6, 2024
243a0e6
LS_LC_CSR_64_Graph.h merge
AdityaAtulTewari Mar 6, 2024
c8898a6
Update Contributing.md
AdityaAtulTewari Mar 6, 2024
42f4ca3
chore: Resolve DistGraph conflicts by taking splitting into 2 files
patrickkenney9801 Mar 6, 2024
255dd41
chore: Fix WMD graph conflicts and network conflict
patrickkenney9801 Mar 6, 2024
dd567bc
chore: Fix last conflicts by removing debug content
patrickkenney9801 Mar 6, 2024
a051848
chore: Fix docker image
patrickkenney9801 Mar 6, 2024
ce71e4c
chore: Things compile
patrickkenney9801 Mar 6, 2024
2255c19
chore: WMD test works
patrickkenney9801 Mar 7, 2024
c122b26
chore: Run pre-commit
patrickkenney9801 Mar 7, 2024
a724c4a
add new LS_CSR and LargeVector implementations (#1)
meyerzinn Feb 27, 2024
15f9de9
fix LargeVector to avoid exhausting address space (#4)
meyerzinn Mar 4, 2024
850470f
chore: Resolve last merge conflict
patrickkenney9801 Mar 7, 2024
fd1c92b
chore: Only build libgnn if MKL is enabled
patrickkenney9801 Mar 7, 2024
ae40781
chore: Add make directive to run tests we care about
patrickkenney9801 Mar 7, 2024
7c9c072
chore: Run pre-commit on repo
patrickkenney9801 Mar 7, 2024
15f203f
ci: Enable github ci
patrickkenney9801 Mar 18, 2024
6550c3f
reduce contention on parallel insertions (#5)
meyerzinn Mar 19, 2024
86f9a71
chore: Add parallel map and pcg generator as dependencies (#3)
patrickkenney9801 Mar 19, 2024
700cc9b
reduce vertex metadata size and remove vertex locks (#8)
meyerzinn Mar 21, 2024
0db6557
add info for compaction policy (#7)
meyerzinn Mar 21, 2024
8e25f1d
improve compaction performance by reducing lock contention (#9)
meyerzinn Mar 21, 2024
c54db22
add data to the LS_CSR (#10)
meyerzinn Mar 26, 2024
12c4e18
chore fix latent merge issues (#13)
patrickkenney9801 Mar 29, 2024
89d65c3
support creation of new vertices in LSCSR (#15)
meyerzinn Apr 1, 2024
9b91847
GNN tests (#12)
nicelhc13 Apr 2, 2024
0302191
Revert "GNN tests (#12)" (#19)
meyerzinn Apr 2, 2024
5d4ca96
improve LS_CSR edge iterator performance (#18)
meyerzinn Apr 2, 2024
096977f
add features to LS_CSR (#22)
meyerzinn Apr 3, 2024
24b8db2
fix bug in LSCSR class visibility (#25)
meyerzinn Apr 3, 2024
dcafab6
add findEdge to LS_CSR (#23)
meyerzinn Apr 4, 2024
8c1fd1d
expose the LS_CSR prefix sum computation (#26)
meyerzinn Apr 4, 2024
8ee55e5
chore: Add test graph constructor (#16)
patrickkenney9801 Apr 4, 2024
2658bd2
remove tombstoning from LS_CSR (#27)
meyerzinn Apr 4, 2024
11b843e
remove explicit hugepages (#24)
meyerzinn Apr 4, 2024
cb75c39
fix latent bug in morphgraph (#30)
meyerzinn Apr 12, 2024
b9a1906
Use parallel hashmap in LS_CSR for edge data (#31)
meyerzinn Apr 12, 2024
3262256
ci: Disable debug builds (#32)
patrickkenney9801 Apr 13, 2024
2785392
DistLocalGraph switch to LS_LC_CSR graph (#28)
divija95 Apr 14, 2024
0b435d8
use work stealing in compaction (#34)
meyerzinn Apr 14, 2024
1cad580
dynamic edges support (#35)
divija95 Apr 15, 2024
73e91c6
use serial prefix sum in lscsr (#37)
meyerzinn Apr 15, 2024
2a07be1
Updated test file
AdityaAtulTewari Apr 15, 2024
79a5e03
File update
AdityaAtulTewari Apr 15, 2024
4296fc8
fixup! File update
meyerzinn Apr 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
*.csv filter=lfs diff=lfs merge=lfs -text
inputs/wmd/data.001.csv filter=lfs diff=lfs merge=lfs -text
162 changes: 162 additions & 0 deletions .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
name: Docker / Ubuntu 22.04 / Build and Test
run-name: docker-ubuntu-2204 performed by @${{ github.triggering_actor }}

on:
pull_request:
branches:
- master
- main
push:
branches:
- master
- main
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
docker-create-ubuntu-2204:
name: create
runs-on: self-hosted
permissions:
contents: read
packages: write
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Create Docker Image
timeout-minutes: 45
run: |
make ci-image

ubuntu-2204-docker:
name: gcc / ${{ matrix.build-type }} / ${{ matrix.sanitizer-type }}
runs-on: self-hosted
permissions:
contents: read
packages: write
env:
IMAGE_NAME: galois
CONTAINER_SRC_DIR: "/pando-galois"
CONTAINER_BUILD_DIR: "/pando-galois/build"
CONTAINER_WORK_DIR: "/pando-galois"
GALOIS_CONTAINER_ENV: "-e=GALOIS_BUILD_TOOL=Ninja"
INTERACTIVE: ""
defaults:
run:
shell: bash -l {0}
strategy:
matrix:
build-type: ['Release']
sanitizer-type: ['nosan', 'san']
needs: docker-create-ubuntu-2204

steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
lfs: 'true'
submodules: recursive

- name: Set up environment variables
timeout-minutes: 1
run: |
echo "UNAME=$(whoami)" >> $GITHUB_ENV
echo "UID=$(id -u)" >> $GITHUB_ENV
echo "GID=$(id -g)" >> $GITHUB_ENV
echo "SRC_DIR=$(pwd)" >> $GITHUB_ENV
echo "GALOIS_CCACHE_DIR=/var/local/$(whoami)/.ccache" >> $GITHUB_ENV
echo "IMAGE_VERSION=$(git log --pretty="%h" -1 Dockerfile)" >> $GITHUB_ENV
if [ ${{ matrix.sanitizer-type }} == 'san' ]; then
echo "GALOIS_CONTAINER_ENV=$GALOIS_CONTAINER_ENV -e=GALOIS_EXTRA_CXX_FLAGS='\"-fsanitize=address -fsanitize=undefined\"'" >> $GITHUB_ENV
fi
if [ ${{ matrix.build-type }} == 'Debug' ]; then
echo "GALOIS_CONTAINER_ENV=$GALOIS_CONTAINER_ENV -e=GALOIS_EXTRA_CXX_FLAGS='-O3'" >> $GITHUB_ENV
fi
if [ ${{ runner.name }} == 'zerberus-0' ]; then
echo "CONTAINER_CPUSET='--cpuset-cpus=0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30'" >> $GITHUB_ENV
fi
if [ ${{ runner.name }} == 'zerberus-1' ]; then
echo "CONTAINER_CPUSET='--cpuset-cpus=1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31'" >> $GITHUB_ENV
fi
if [ ${{ runner.name }} == 'languedoc-0' ]; then
echo "CONTAINER_CPUSET='--cpuset-cpus=0,1,2,3,4,5,6,7,32,33,34,35,36,37,38,39'" >> $GITHUB_ENV
fi
if [ ${{ runner.name }} == 'languedoc-1' ]; then
echo "CONTAINER_CPUSET='--cpuset-cpus=16,17,18,19,20,21,22,23,48,49,50,51,52,53,54,55'" >> $GITHUB_ENV
fi
if [ ${{ runner.name }} == 'languedoc-2' ]; then
echo "CONTAINER_CPUSET='--cpuset-cpus=8,9,10,11,12,13,14,15,40,41,42,43,44,45,46,47'" >> $GITHUB_ENV
fi
if [ ${{ runner.name }} == 'languedoc-3' ]; then
echo "CONTAINER_CPUSET='--cpuset-cpus=24,25,26,27,28,29,30,31,56,57,58,59,60,61,62,63'" >> $GITHUB_ENV
fi
cat $GITHUB_ENV

- name: Configure
timeout-minutes: 10
run: |
mkdir -p ${{ env.GALOIS_CCACHE_DIR }} -m=777
CONTAINER_CMD="bash -lc 'source /opt/intel/oneapi/setvars.sh && make setup-ci'" \
CONTAINER_OPTS="-e=BUILD_TYPE=${{ matrix.build-type }}" \
IMAGE_NAME="${{ env.IMAGE_NAME }}" \
VERSION="${{ env.IMAGE_VERSION }}" \
make docker

- name: Build
timeout-minutes: 15
run: |
CONTAINER_CMD="bash -c 'ninja -j10 || ninja || ninja'" \
IMAGE_NAME="${{ env.IMAGE_NAME }}" \
VERSION="${{ env.IMAGE_VERSION }}" \
CONTAINER_WORKDIR="${{ env.CONTAINER_BUILD_DIR }}" \
make docker

- name: Run Tests
timeout-minutes: 5
run: |
CONTAINER_CMD="make run-tests" \
IMAGE_NAME="${{ env.IMAGE_NAME }}" \
VERSION="${{ env.IMAGE_VERSION }}" \
make docker

docker-pre-commit-ubuntu-2204:
name: pre-commit
runs-on: self-hosted
permissions:
contents: read
packages: write
env:
IMAGE_NAME: galois
CONTAINER_SRC_DIR: "/pando-galois"
CONTAINER_BUILD_DIR: "/pando-galois/build"
CONTAINER_WORK_DIR: "/pando-galois"
INTERACTIVE: ""
defaults:
run:
shell: bash -l {0}
needs: docker-create-ubuntu-2204

steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up environment variables
timeout-minutes: 1
run: |
echo "SRC_DIR=$(pwd)" >> $GITHUB_ENV
echo "IMAGE_VERSION=$(git log --pretty="%h" -1 Dockerfile)" >> $GITHUB_ENV
cat $GITHUB_ENV
- name: Check pre-commit
timeout-minutes: 10
run: |
IMAGE_NAME="${{ env.IMAGE_NAME }}" \
VERSION="${{ env.IMAGE_VERSION }}" \
make docker-pre-commit
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,15 @@ cscope.out
.tags*
tags
.ycm_extra_conf.py
.ccache

# no build files
/build*
/*build*
/dockerbuild*

# no python build artifacts
*.pyc
/python/galois.egg-info
/python/galois/*.so
/_skbuild

12 changes: 6 additions & 6 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[submodule "moderngpu"]
path = external/moderngpu
url = https://github.com/moderngpu/moderngpu.git
[submodule "cub"]
path = external/cub
url = https://github.com/NVlabs/cub.git
[submodule "external/pcg-cpp"]
path = external/pcg-cpp
url = https://github.com/imneme/pcg-cpp.git
[submodule "external/parallel-hashmap"]
path = external/parallel-hashmap
url = https://github.com/greg7mdp/parallel-hashmap.git
25 changes: 25 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
files: ^libcusp|^libdeepgalois|^libdist|^libgalois|^libgluon|^libgnn|^libwmd
exclude: ^scripts|^python|^inputs
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.2.0
hooks:
- id: end-of-file-fixer
- id: mixed-line-ending
- id: trailing-whitespace
- repo: https://github.com/Lucas-C/pre-commit-hooks
rev: v1.2.0
hooks:
- id: forbid-tabs
exclude: ^scripts|^python
- id: remove-tabs
exclude: ^scripts|^python
args: [--whitespaces-count, '2']
- repo: https://github.com/pocc/pre-commit-hooks
rev: v1.3.5
hooks:
- id: clang-format
args: [-i]
# - id: clang-tidy
# args: [--fix, -p=build/compile_commands.json]
1 change: 1 addition & 0 deletions .tool-versions
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pre-commit 2.19.0
Loading