Skip to content

Commit

Permalink
Integration test cleanup workflow (#138)
Browse files Browse the repository at this point in the history
## Problem

Integration tests attempt to cleanup after themselves with `afterEach`,
but it is possible for the `afterEach` delete to fail or for someone to
add a test that does not have proper cleanup steps. If we don't have a
way to cleanup these orphaned resources, eventually all tests will fail
due to quota exhaustion.

For example, while trying to write tests for create collection I
accidentally created a bunch of these abandoned resources because the
afterEach call to deleteCollection was failing; collections can't be
deleted while they are still being created.

## Solution

Create a workflow that deletes all indexes and projects within the
sdk-node-testing project.
- Can be triggered manually
- Runs automatically once per day

## Type of Change

- [x] Infrastructure change (CI configs, etc)
  • Loading branch information
jhamon authored Oct 9, 2023
1 parent 895c434 commit d2674c8
Show file tree
Hide file tree
Showing 3 changed files with 47 additions and 0 deletions.
18 changes: 18 additions & 0 deletions .github/workflows/cleanup.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: 'Cleanup test resources'

on:
workflow_dispatch: {}
schedule:
- cron: '0 0 * * *'

jobs:
cleanup:
name: Delete all indexes and collections in sdk-node-testing project
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Setup
uses: ./.github/actions/setup
- name: Cleanup
run: npm run test:integration:cleanup
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
"repl": "npm run build && node utils/replInit.ts",
"test:integration:node": "jest src/integration/ -c jest.config.integration-node.js",
"test:integration:edge": "jest src/integration/ -c jest.config.integration-edge.js",
"test:integration:cleanup": "npm run build && node utils/cleanupResources.ts",
"test:unit": "jest src/"
},
"engines": {
Expand Down
28 changes: 28 additions & 0 deletions utils/cleanupResources.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
var dotenv = require('dotenv');
dotenv.config();

for (const envVar of ['PINECONE_ENVIRONMENT', 'PINECONE_API_KEY']) {
if (!process.env[envVar]) {
console.warn(`WARNING Missing environment variable ${envVar} in .env file`);
} else {
console.log(`INFO Found environment variable ${envVar} in .env file`);
}
}

var pinecone = require('../dist');

(async () => {
const p = new pinecone.Pinecone();

const collections = await p.listCollections();
for (const collection of collections) {
console.log(`Deleting collection ${collection.name}`);
await p.deleteCollection(collection.name);
}

const indexes = await p.listIndexes();
for (const index of indexes) {
console.log(`Deleting index ${index.name}`);
await p.deleteIndex(index.name);
}
})();

0 comments on commit d2674c8

Please sign in to comment.