Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: wise.com connector #23

Merged
merged 55 commits into from
Oct 6, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
a946420
add structure for wise connector
altitude Jul 24, 2022
1ac6272
fix: first
Jul 25, 2022
a5589b1
fix: lint
Jul 25, 2022
1e630d6
Merge pull request #24 from numary/fix/update-workflows-taskfile-mods
flemzord Jul 25, 2022
eec30ba
fix: GET using id
gfyrag Aug 17, 2022
caf40f2
chore: first
Aug 25, 2022
ba0c92c
chore: bump go-libs
Aug 26, 2022
f522070
chore: bump go-libs
Aug 26, 2022
ecbb8d6
chore: add dependabot
Aug 26, 2022
b23fb28
chore: move message models
Aug 26, 2022
9782f03
chore: new method to create kafka message
Aug 26, 2022
2945ea5
chore: move kafka messages to core pkg
Aug 26, 2022
4aa232c
chore: update messages
Aug 26, 2022
9b83e68
Merge pull request #26 from numary/feature/fix-get-by-id
altitude Aug 27, 2022
bfc6f83
chore: revert moving files
Aug 29, 2022
0c131f0
chore: update messages
Aug 29, 2022
7c1234a
Merge pull request #27 from numary/refactor/move-models-to-sharedpubl…
Aug 29, 2022
e1f0940
chore(deps): bump github/codeql-action from 1 to 2
dependabot[bot] Aug 29, 2022
276452c
Merge pull request #28 from numary/dependabot/github_actions/github/c…
flemzord Aug 29, 2022
4729bb2
fix: bad dependencies
gfyrag Aug 29, 2022
475e1f3
fix: payments publish
gfyrag Aug 29, 2022
11a633f
Merge pull request #30 from numary/fix/missing-id
flemzord Aug 29, 2022
27ac2be
chore(deps): bump github.com/numary/go-libs from 1.0.0 to 1.0.1
dependabot[bot] Aug 29, 2022
de4db8a
Merge pull request #31 from numary/dependabot/go_modules/github.com/n…
Sep 1, 2022
afd89a8
fix: Ingestion message payload marshaling (#32)
Sep 1, 2022
da4d147
chore(deps): bump docker/build-push-action from 2 to 3
dependabot[bot] Sep 12, 2022
19e99bc
chore(deps): bump docker/setup-qemu-action from 1 to 2
dependabot[bot] Sep 12, 2022
bedb497
chore(deps): bump docker/login-action from 1 to 2
dependabot[bot] Sep 12, 2022
a8a684a
chore(deps): bump actions/checkout from 2 to 3
dependabot[bot] Sep 12, 2022
b22e237
chore(deps): bump docker/setup-buildx-action from 1 to 2
dependabot[bot] Sep 12, 2022
0ce043e
Merge pull request #38 from formancehq/dependabot/github_actions/dock…
flemzord Sep 15, 2022
06c7912
Merge pull request #39 from formancehq/dependabot/github_actions/dock…
flemzord Sep 15, 2022
f805af2
Merge pull request #40 from formancehq/dependabot/github_actions/dock…
flemzord Sep 15, 2022
3540916
Merge pull request #41 from formancehq/dependabot/github_actions/acti…
flemzord Sep 15, 2022
c3a0296
Merge pull request #42 from formancehq/dependabot/github_actions/dock…
flemzord Sep 15, 2022
c3e060c
feat: dummypay connector
darkmatterpool Sep 23, 2022
516543a
feat: dummypay connector
darkmatterpool Sep 23, 2022
e12e126
Merge branch 'feat/connector-dummypay' of formance:formancehq/payment…
darkmatterpool Sep 23, 2022
8b71be7
feat: uncommented dummypay load
darkmatterpool Sep 23, 2022
6a0f52f
feat: dummypay payments generation
darkmatterpool Sep 25, 2022
c862071
feat: dummypay connector tests
darkmatterpool Sep 25, 2022
ea398ae
fix: linter fixes
darkmatterpool Sep 25, 2022
1963c75
ci: Switch to FormanceHQ Organization
flemzord Sep 26, 2022
c412023
Merge branch 'main' into feat/connector-dummypay
flemzord Sep 26, 2022
5ad16b8
fix: drop syscall as it's not supported on windows arm
darkmatterpool Sep 26, 2022
a56969b
feat: add scheme generation
darkmatterpool Sep 26, 2022
93e2105
fix: typo in error message
darkmatterpool Sep 26, 2022
39e9ade
feat: add swagger examples for dummypay
darkmatterpool Sep 26, 2022
63be834
Merge pull request #43 from formancehq/feat/connector-dummypay
darkmatterpool Sep 27, 2022
d6e11a6
add structure for wise connector
altitude Jul 24, 2022
2c8d0fd
Merge remote-tracking branch 'origin/feat/connector-wise' into feat/c…
darkmatterpool Sep 28, 2022
b7edb34
fix: general refactoring
darkmatterpool Sep 28, 2022
d0075d8
feat: complete wise integration
darkmatterpool Oct 4, 2022
79c5cef
feat: add payment status mapping
darkmatterpool Oct 4, 2022
89baaf9
feat: add additional error checks
darkmatterpool Oct 5, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
version: 2
updates:

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "gomod"
directory: "/"
schedule:
interval: "weekly"
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Build special named branch
name: Build and push special named branch
on:
push:
branches:
Expand All @@ -7,29 +7,30 @@ on:
- 'feat/**'
- 'fix/**'
- 'hotfix/**'

jobs:
BuildDockerImage:
name: 'Build & Publish Docker'
build-push:
name: 'Build & Push Docker image'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v2
- name: Login to GitHub Container Registry
uses: docker/login-action@v1
uses: docker/login-action@v2
with:
registry: ghcr.io
username: "NumaryBot"
password: ${{ secrets.NUMARY_GITHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v2
uses: docker/build-push-action@v3
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: ghcr.io/numary/payments:${{ github.sha }}
tags: ghcr.io/formancehq/payments:${{ github.sha }}
build-args: |
APP_SHA=${{ github.sha }}
VERSION=${{ github.sha }}
8 changes: 4 additions & 4 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@ jobs:
language: [ 'go' ]
steps:
- name: Checkout repository
uses: actions/checkout@v2
uses: actions/checkout@v3
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
- name: Autobuild
uses: github/codeql-action/autobuild@v1
uses: github/codeql-action/autobuild@v2
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
uses: github/codeql-action/analyze@v2
35 changes: 19 additions & 16 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,40 +5,43 @@ on:

name: Main
jobs:
Test:
uses: numary/gh-workflows/.github/workflows/golang-test.yml@main
with:
go_version: 1.18
lint:
uses: formancehq/gh-workflows/.github/workflows/golang-lint.yml@main

Build:
uses: numary/gh-workflows/.github/workflows/goreleaser-build.yml@main
test:
uses: formancehq/gh-workflows/.github/workflows/golang-test.yml@main

build:
uses: formancehq/gh-workflows/.github/workflows/goreleaser-build.yml@main
needs:
- Test
- lint
- test

build_docker:
name: 'Build & Publish Docker'
build-push:
name: 'Build & Push Docker image'
runs-on: ubuntu-latest
needs:
- Test
- lint
- test
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v2
- name: Login to GitHub Container Registry
uses: docker/login-action@v1
uses: docker/login-action@v2
with:
registry: ghcr.io
username: "NumaryBot"
password: ${{ secrets.NUMARY_GITHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v2
uses: docker/build-push-action@v3
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: ghcr.io/numary/payments:main,ghcr.io/numary/payments:${{ github.sha }}
tags: ghcr.io/formancehq/payments:main,ghcr.io/formancehq/payments:${{ github.sha }}
build-args: |
APP_SHA=${{ github.sha }}
VERSION=develop
20 changes: 20 additions & 0 deletions .github/workflows/pr-open.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: Pull Request - Open
on:
pull_request:
types: [assigned, opened, synchronize, reopened]
jobs:
pr-style:
uses: formancehq/gh-workflows/.github/workflows/pr-style.yml@main

lint:
uses: formancehq/gh-workflows/.github/workflows/golang-lint.yml@main

test:
uses: formancehq/gh-workflows/.github/workflows/golang-test.yml@main

build:
uses: formancehq/gh-workflows/.github/workflows/goreleaser-build.yml@main
needs:
- pr-style
- lint
- test
22 changes: 0 additions & 22 deletions .github/workflows/pr_open.yml

This file was deleted.

30 changes: 16 additions & 14 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,41 +5,43 @@ on:
permissions:
contents: write
jobs:
Test:
uses: numary/gh-workflows/.github/workflows/golang-test.yml@main
with:
go_version: 1.18
lint:
uses: formancehq/gh-workflows/.github/workflows/golang-lint.yml@main

Build:
uses: numary/gh-workflows/.github/workflows/goreleaser-release.yml@main
test:
uses: formancehq/gh-workflows/.github/workflows/golang-test.yml@main

build:
uses: formancehq/gh-workflows/.github/workflows/goreleaser-release.yml@main
secrets:
NUMARY_GITHUB_TOKEN: ${{ secrets.NUMARY_GITHUB_TOKEN }}
FURY_TOKEN: ${{ secrets.FURY_TOKEN }}
needs:
- Test
- lint
- test

build_docker:
build-docker:
name: 'Build & Publish Docker'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v2
- name: Login to GitHub Container Registry
uses: docker/login-action@v1
uses: docker/login-action@v2
with:
registry: ghcr.io
username: "NumaryBot"
password: ${{ secrets.NUMARY_GITHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v2
uses: docker/build-push-action@v3
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: ghcr.io/numary/payments:latest,ghcr.io/numary/payments:${{ github.event.release.tag_name }}
tags: ghcr.io/formancehq/payments:latest,ghcr.io/formancehq/payments:${{ github.event.release.tag_name }}
build-args: |
APP_SHA=${{ github.sha }}
VERSION=${{ github.event.release.tag_name }}
24 changes: 0 additions & 24 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -1,32 +1,8 @@
linters-settings:
goimports:
local-prefixes: github.com/golangci/golangci-lint
govet:
check-shadowing: false

nolintlint:
allow-leading-space: true # don't require machine-readable nolint directives (i.e. with no leading space)
allow-unused: false # report any unused nolint directives
require-explanation: false # don't require an explanation for nolint directives
require-specific: false # don't require nolint directives to be specific about which linter is being skipped

linters:
disable-all: true
enable:
- deadcode #Default linter
- errcheck #Default linter
- gosimple #Default linter
- govet #Default linter
- ineffassign #Default linter
- staticcheck #Default linter
- structcheck #Default linter
- typecheck #Default linter
- unused #Default linter
- varcheck #Default linter
- gofmt
- gci
- goimports

run:
timeout: 5m
go: '1.18'
15 changes: 12 additions & 3 deletions Taskfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,27 @@ vars:
TIMEOUT: 10m

tasks:
tests:
default:
cmds:
- go test -v {{.FAILFAST}} -coverpkg {{.PKG}} -coverprofile {{.COVERAGE_FILE}} -covermode atomic -timeout {{.TIMEOUT}} {{.PKG}}
- task: lint
- task: tests

lint:
cmds:
- golangci-lint run -v --fix {{.PKG}}
- golangci-lint run -v --fix {{.PKG}}

tests:
cmds:
- go test -v {{.FAILFAST}} -coverpkg {{.PKG}} -coverprofile {{.COVERAGE_FILE}} -covermode atomic -timeout {{.TIMEOUT}} {{.PKG}}

build:
cmds:
- go build -o {{.BINARY_NAME}}

install:
cmds:
- go install -o {{.BINARY_NAME}}

clean:
cmds:
- go clean
Expand Down
14 changes: 12 additions & 2 deletions cmd/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,16 @@ import (
"github.com/numary/go-libs/sharedauth"
"github.com/numary/go-libs/sharedlogging"
"github.com/numary/go-libs/sharedlogging/sharedlogginglogrus"
"github.com/numary/go-libs/sharedotlp"
"github.com/numary/go-libs/sharedotlp/sharedotlptraces"
sharedotlp "github.com/numary/go-libs/sharedotlp/pkg"
"github.com/numary/go-libs/sharedotlp/pkg/sharedotlptraces"
"github.com/numary/go-libs/sharedpublish"
"github.com/numary/go-libs/sharedpublish/sharedpublishhttp"
"github.com/numary/go-libs/sharedpublish/sharedpublishkafka"
"github.com/numary/payments/pkg/api"
"github.com/numary/payments/pkg/bridge/cdi"
"github.com/numary/payments/pkg/bridge/connectors/dummypay"
"github.com/numary/payments/pkg/bridge/connectors/stripe"
"github.com/numary/payments/pkg/bridge/connectors/wise"
bridgeHttp "github.com/numary/payments/pkg/bridge/http"
"github.com/numary/payments/pkg/database"
paymentapi "github.com/numary/payments/pkg/http"
Expand Down Expand Up @@ -278,5 +280,13 @@ func HTTPModule() fx.Option {
viper.GetBool(authBearerUseScopesFlag),
stripe.NewLoader(),
),
cdi.ConnectorModule[dummypay.Config, dummypay.TaskDescriptor](
viper.GetBool(authBearerUseScopesFlag),
dummypay.NewLoader(),
),
cdi.ConnectorModule(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@darkmatterpool I don't like this way of register connectors. It would be ideal if the connectors registers themselves to the core. If you have an idea for this....

viper.GetBool(authBearerUseScopesFlag),
wise.NewLoader(),
),
)
}
14 changes: 9 additions & 5 deletions docs/tuto-connector.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Tutorial connector

_referenced in `/pkg/bridge/connectors/dummypay`_

We are going to create a fake connector which read a directory.
In this directory, a fake bank service will create files.
Each files contain a payin or a payout as a json object.
Expand Down Expand Up @@ -30,7 +32,7 @@ A connector has a name.
This name is provided by the loader by the method Name().
Also, each connector define a config object using generics which has to implement interface payments.ConnectorConfigObject.
This interface only have a method Validate() error which is used by the code to validate an external config is valid before load the connector with it.
Since, some properties of the config may have some optionnal, the loader is also in charge of configuring default values on it.
Since, some properties of the config may have some optional, the loader is also in charge of configuring default values on it.
This is done using the method ```ApplyDefaults(Config)```.

The framework provide the capabilities to run tasks.
Expand Down Expand Up @@ -188,7 +190,7 @@ Basically this context provides two things :

But, what is a task ?

A task is like a process than the framework will handle for you. It is basically a simple function.
A task is like a process that the framework will handle for you. It is basically a simple function.
When installed, a connector has the opportunity to schedule some tasks and let the system handle them for him.
A task has a descriptor.
The descriptor must be immutable and represents a specific task in the system. It can be anything.
Expand All @@ -213,7 +215,7 @@ Add some logic on our connector :
```go
...
WithInstall(func(ctx task.ConnectorContext[TaskDescriptor]) error {
return ctx.Scheduler().Schedule("directory")
return ctx.Scheduler().Schedule("directory", true)
}).
...
```
Expand Down Expand Up @@ -306,7 +308,8 @@ As expected, the task trigger an error because of non-existent /tmp/payments dir

You can see the tasks on api too :
```bash
# curl http://localhost:8080/connectors/example/tasks|jq
curl http://localhost:8080/connectors/example/tasks | jq

[
{
"provider": "example",
Expand Down Expand Up @@ -425,7 +428,8 @@ As you can see, this time the second task has been started and was terminated wi

It should have created a payment on database. Let's check :
```bash
# curl http://localhost:8080/connectors/payments|jq
curl http://localhost:8080/payments | jq

{
"data": [
{
Expand Down
Loading