-
Notifications
You must be signed in to change notification settings - Fork 23
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feature: cross language integration tests (#417)
* feat: very first pass at spinning up a mock metrics server to write tests The theory is that we'll use this, pass in the command to spin up the child server, and provide the METRICS_SERVER as an env variable that each of the sdks can look for. Once we can pass that in we can make HTTP level assertions about what the SDKs can and can't do! Lets try and write a test first against the node server and see if we can make it work 🤞 * test: add a first integration test via the new testing method Now i'm gunna try and put this into a dockerfile like httpsnippet, then I can try and add another language 🤞 * test: add docker-compose file for running node/express integration tests * test: try and get new integration tests working on gh actions * test: attempting to get integration tests working on actions * refactor: attempt to bubble up errors better * refactor: toString potential errors before rejecting * test(integration): build the node module before running tests * test(integration): try adding it as a dependent job * test(integration): remove integration test from `npm test` It's gunna be run separately via another action, so don't do it twice. * test(integration): fix incorrect eslint-config reference * test(integration): fix eslint package reference * test(integration): linty linty * test(integration): attempt to get it working * test(integration): ignore lint error in express example This dependency will be there when we run the integration tests, but it won't be when the linter gets run because the TS files haven't been built * test(integration): turning off a lint rule * feat(integration/node): move more setup to Dockerfile instead of gh action * feature: cross language integration tests (dotnet edition) (#435) * test(integration/dotnet): add super simple dotnet 6.0 test server Generated using: ``` dotnet new web --no-https ``` * test(integration/dotnet): add very basic .net 6.0 app for metrics Still outstanding: - make the sdk respect `METRICS_SERVER` env variable - try and get `readme__apiKey` better as an env variable - wire it up to an integration test * test(integration/dotnet): use regular environment variable name This is slightly confusing since if they actually wanted to set this up themselves then they would probably use appsettings.json instead. This is fine for a demo app though, and will mean it works from the integration test layer 🤞 * test(integration/node): update tests to work with new port number stuff * refactor(dotnet): stylistic and whitespace changes I installed the C# vs code plugin and it did this 🤷♂️ * refactor(dotnet): fix warning to do with null variable in test app * refactor(integration/dotnet): allow the metrics server to be set from the env Use UriBuilder to construct a URI that always has the correct path on the end * refactor(dotnet): remove the await from the metrics call so it doesnt block Right now if metrics is down, this will cause our user's server to stop responding to requests. This is probably what you'd want in the vast majority of cases - since it's actually kinda rare you'd wanna fire and forget an HTTP request. * chore(integration/dotnet): make host header lookup case insensitive * test(integration/node): fix up child process usage to use spawn When using child_process.exec, the process could not reliably be killed when the test ended, which meant the docker container never closed. This caused timeouts on the github action: https://github.com/readmeio/metrics-sdks/runs/6215430863?check_suite_focus=true * test(integration/node): this no longer needs to be executable * test(integration): remove the snapshot test for now There's way too many inconsistencies between even these two web servers to do jest snapshot testing, so opting to be much more explicit about assertions for the time being. Would rather get something up and passing than not running these tests going forward. * test(integration/dotnet): add docker file for .net test Added to docker-compose and a new dotnet.yml to be run on gh actions * refactor: rename incorrect naming of job in gh actions * test(integration/dotnet): ugh, stupid case insensitive macOS This failed on github cos it's using a real linux env: https://github.com/readmeio/metrics-sdks/runs/6226930070?check_suite_focus=true Lets see if this does the trick 🤞
- Loading branch information
1 parent
fa38ca9
commit bca2105
Showing
26 changed files
with
999 additions
and
1,095 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
node_modules |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
coverage/ | ||
dist/ | ||
node_modules/ | ||
# We just want to lint the very top level | ||
packages/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
{ | ||
"extends": [ | ||
"@readme/eslint-config" | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
name: dotnet | ||
|
||
on: [push] | ||
|
||
jobs: | ||
integration: | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- uses: actions/checkout@v3 | ||
|
||
- run: docker-compose run integration_dotnet_v6.0 | ||
|
||
- name: Cleanup | ||
if: always() | ||
run: docker-compose down |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
{ | ||
"extends": "@readme/eslint-config/testing" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
// Jest Snapshot v1, https://goo.gl/fbAQLP | ||
|
||
exports[`Metrics SDK Integration Tests should make a request to a metrics backend with a har file 1`] = ` | ||
Object { | ||
"email": "owlbert@example.com", | ||
"id": "owlbert-api-key", | ||
"label": "Owlbert", | ||
} | ||
`; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,158 @@ | ||
import http from 'http'; | ||
import { cwd } from 'process'; | ||
import { spawn } from 'child_process'; | ||
import { promisify } from 'util'; | ||
import { once } from 'events'; | ||
import getPort from 'get-port'; | ||
import caseless from 'caseless'; | ||
|
||
if (!process.env.EXAMPLE_SERVER) { | ||
// eslint-disable-next-line no-console | ||
console.error('Missing `EXAMPLE_SERVER` environment variable'); | ||
process.exit(1); | ||
} | ||
|
||
// https://gist.github.com/krnlde/797e5e0a6f12cc9bd563123756fc101f | ||
http.get[promisify.custom] = function getAsync(options) { | ||
return new Promise((resolve, reject) => { | ||
http | ||
.get(options, response => { | ||
response.end = new Promise(res => { | ||
response.on('end', res); | ||
}); | ||
resolve(response); | ||
}) | ||
.on('error', reject); | ||
}); | ||
}; | ||
|
||
const get = promisify(http.get); | ||
|
||
const randomApiKey = 'a-random-readme-api-key'; | ||
|
||
// Converts an array of headers like this: | ||
// [ | ||
// { name: 'host', value: 'localhost:49914' }, | ||
// { name: 'connection', value: 'close' }, | ||
// ]; | ||
// | ||
// To an object that can be passed in to caseless: | ||
// { | ||
// host: 'localhost:49914', | ||
// connection: 'close' | ||
// } | ||
function arrayToObject(array) { | ||
return array.reduce((prev, next) => { | ||
return Object.assign(prev, { [next.name]: next.value }); | ||
}, {}); | ||
} | ||
|
||
describe('Metrics SDK Integration Tests', () => { | ||
let metricsServer; | ||
let httpServer; | ||
let PORT; | ||
|
||
beforeAll(async () => { | ||
metricsServer = http.createServer().listen(0, 'localhost'); | ||
|
||
await once(metricsServer, 'listening'); | ||
const { address, port } = metricsServer.address(); | ||
PORT = await getPort(); | ||
|
||
// In order to use child_process.spawn, we have to provide a | ||
// command along with an array of arguments. So this is a very | ||
// rudimental way of splitting the two values provided to us | ||
// from the environment variable. | ||
// | ||
// I tried refactoring this to use child_process.exec, which just | ||
// takes in a single string to run, but that creates it's own | ||
// shell so we can't do `cp.kill()` on it later on (because that | ||
// just kills the shell, not the actual command we're running). | ||
// | ||
// Annoyingly this works under macOS, so it must be a platform | ||
// difference when running under docker/linux. | ||
const [command, ...args] = process.env.EXAMPLE_SERVER.split(' '); | ||
|
||
httpServer = spawn(command, args, { | ||
cwd: cwd(), | ||
env: { | ||
PORT, | ||
METRICS_SERVER: new URL(`http://${address}:${port}`).toString(), | ||
README_API_KEY: randomApiKey, | ||
...process.env, | ||
}, | ||
}); | ||
return new Promise((resolve, reject) => { | ||
httpServer.stderr.on('data', data => { | ||
// eslint-disable-next-line no-console | ||
console.error(`stderr: ${data}`); | ||
return reject(data.toString()); | ||
}); | ||
httpServer.on('error', err => { | ||
// eslint-disable-next-line no-console | ||
console.error('error', err); | ||
return reject(err.toString()); | ||
}); | ||
// eslint-disable-next-line consistent-return | ||
httpServer.stdout.on('data', data => { | ||
if (data.toString().match(/listening/)) return resolve(); | ||
// eslint-disable-next-line no-console | ||
console.log(`stdout: ${data}`); | ||
}); | ||
}); | ||
}); | ||
|
||
afterAll(() => { | ||
httpServer.kill(); | ||
return new Promise((resolve, reject) => { | ||
metricsServer.close(err => { | ||
if (err) return reject(err); | ||
return resolve(); | ||
}); | ||
}); | ||
}); | ||
|
||
// TODO this needs fleshing out more with more assertions and complex | ||
// test cases, along with more servers in different languages too! | ||
it('should make a request to a metrics backend with a har file', async () => { | ||
await get(`http://localhost:${PORT}`); | ||
|
||
const [req] = await once(metricsServer, 'request'); | ||
expect(req.url).toBe('/v1/request'); | ||
expect(req.headers.authorization).toBe('Basic YS1yYW5kb20tcmVhZG1lLWFwaS1rZXk6'); | ||
|
||
let body = ''; | ||
// eslint-disable-next-line no-restricted-syntax | ||
for await (const chunk of req) { | ||
body += chunk; | ||
} | ||
body = JSON.parse(body); | ||
const [har] = body; | ||
|
||
// Check for a uuid | ||
// https://uibakery.io/regex-library/uuid | ||
// eslint-disable-next-line no-underscore-dangle | ||
expect(har._id).toMatch(/^[0-9a-f]{8}-[0-9a-f]{4}-[0-5][0-9a-f]{3}-[089ab][0-9a-f]{3}-[0-9a-f]{12}$/i); | ||
expect(har.group).toMatchSnapshot(); | ||
expect(har.clientIPAddress).toBe('127.0.0.1'); | ||
|
||
const { request, response } = har.request.log.entries[0]; | ||
|
||
expect(request.url).toBe(`http://localhost:${PORT}/`); | ||
expect(request.method).toBe('GET'); | ||
expect(request.httpVersion).toBe('HTTP/1.1'); | ||
|
||
const requestHeaders = caseless(arrayToObject(request.headers)); | ||
expect(requestHeaders.get('connection')).toBe('close'); | ||
expect(requestHeaders.get('host')).toBe(`localhost:${PORT}`); | ||
|
||
expect(response.status).toBe(200); | ||
expect(response.statusText).toBe('OK'); | ||
expect(response.content.text).toBe(JSON.stringify({ message: 'hello world' })); | ||
expect(response.content.size).toBe(25); | ||
expect(response.content.mimeType).toBe('application/json; charset=utf-8'); | ||
|
||
const responseHeaders = caseless(arrayToObject(response.headers)); | ||
expect(responseHeaders.get('content-type')).toBe('application/json; charset=utf-8'); | ||
}); | ||
}); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build-env | ||
|
||
ADD packages/dotnet /src | ||
|
||
WORKDIR /src/examples/net6.0 | ||
RUN dotnet publish -o out | ||
|
||
# Build runtime image | ||
# TODO add this to base.Dockerfile? | ||
FROM node:16 | ||
WORKDIR /src | ||
ADD package*.json /src/ | ||
RUN npm ci | ||
ADD __tests__ /src/__tests__ | ||
|
||
COPY --from=build-env /src /src | ||
COPY --from=build-env /usr/share/dotnet /usr/share/dotnet | ||
|
||
# Put the dotnet executable in the path | ||
ENV PATH /usr/share/dotnet:$PATH | ||
ENV DOTNET_CLI_TELEMETRY_OPTOUT=true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
FROM node:16 | ||
|
||
ADD packages/node /src/packages/node | ||
|
||
# Build node sdk | ||
WORKDIR /src/packages/node | ||
RUN npm ci --ignore-scripts | ||
RUN npm run build | ||
|
||
# Install top level dependencies | ||
WORKDIR /src | ||
ADD package*.json /src/ | ||
RUN npm ci | ||
ADD __tests__ /src/__tests__ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
version: '3' | ||
services: | ||
integration_node_express: | ||
build: | ||
context: . | ||
dockerfile: ./__tests__/integrations/node.Dockerfile | ||
command: npm run test:integration | ||
environment: | ||
- EXAMPLE_SERVER=node ./packages/node/examples/express/index.js | ||
|
||
integration_dotnet_v6.0: | ||
build: | ||
context: . | ||
dockerfile: ./__tests__/integrations/dotnet.Dockerfile | ||
command: npm run test:integration | ||
environment: | ||
- EXAMPLE_SERVER=dotnet examples/net6.0/out/net6.0.dll |
Oops, something went wrong.