Refactor internal services #915

anbraten · 2022-05-14T16:14:24Z

Preparation for #783

Changes

load secret, registry, config & environment services through service-manager
move configFetcher logic into config service
add license-header where missing
move service setup logic

server/model/repo.go

server/plugins/registry/http.go

anbraten · 2024-02-07T22:10:28Z

@qwerty287 Started updating this PR. For the first step I would just do some refactoring to load the different "services". And in a second step I would add the http service implementations for secrets & registries.

Currently we use some terms for "addons" like services, addons, plugins, extensions (what I've introduced when initially working on this). Do you have any preferences how should I call them? As the builtin implementations will also be of that ... type, sticking with services could be a quite fitting term?

What should I do with the go-plugin addon system for secrets, registries & configs?

…po-plugins

docs/docs/91-migrations.md

server/extensions/manager.go

server/extensions/permissions/admins.go

anbraten · 2024-02-11T14:30:19Z

Renamed everything as discussed. Only question now left is how the can unify the config fetcher usage. Currently we refetch the config from the http endpoint for restart, but not from the forge, making a unique interface pretty difficult.

qwerty287 · 2024-02-11T14:33:10Z

I think it's fine to refetch in any case.

anbraten · 2024-02-11T14:37:38Z

I think it's fine to refetch in any case.

So not using the stored config at all and always use a fresh config from the forge / http endpoint?

qwerty287 · 2024-02-11T14:38:54Z

Yes (config must still be stored to show in ui)

anbraten · 2024-02-11T15:11:19Z

PR should be finally ready now.

Adjusted the interface a bit again to include oldConfigData and a restart argument allowing me to handle it the same way we currently did. Could be get some refactoring and cleanup later on.

codecov · 2024-02-11T15:23:32Z

Codecov Report

Attention: 151 lines in your changes are missing coverage. Please review.

Comparison is base (e1521ef) 36.36% compared to head (f1133e0) 36.68%.

Files	Patch %	Lines
server/services/secret/db.go	33.82%	44 Missing and 1 partial ⚠️
server/api/registry.go	0.00%	17 Missing ⚠️
server/api/repo_secret.go	0.00%	17 Missing ⚠️
server/api/global_secret.go	0.00%	11 Missing ⚠️
server/api/org_secret.go	0.00%	11 Missing ⚠️
server/pipeline/restart.go	0.00%	8 Missing ⚠️
server/services/config/forge.go	92.00%	6 Missing and 2 partials ⚠️
server/pipeline/items.go	0.00%	7 Missing ⚠️
cmd/server/server.go	0.00%	5 Missing ⚠️
server/pipeline/create.go	0.00%	4 Missing ⚠️
... and 8 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #915      +/-   ##
==========================================
+ Coverage   36.36%   36.68%   +0.32%     
==========================================
  Files         224      225       +1     
  Lines       14848    14782      -66     
==========================================
+ Hits         5399     5423      +24     
+ Misses       9057     8966      -91     
- Partials      392      393       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

lafriks · 2024-02-11T17:11:59Z

Did not test but code looks good

## [2.4.0](https://github.com/woodpecker-ci/woodpecker/releases/tag/2.4.0) - 2024-03-19 ### 🔒 Security - Improve security context handling [[#3482](#3482)] - fix(deps): update module github.com/moby/moby to v24.0.9+incompatible [[#3323](#3323)] ### ✨ Features - Cli setup command [[#3384](#3384)] - Add bitbucket datacenter (server) support [[#2503](#2503)] - Cli updater [[#3382](#3382)] ### 📚 Documentation - Delete docs for v0.15.x [[#3508](#3508)] - Add deployment plugin [[#3495](#3495)] - Bump follow-redirects and fix broken anchors [[#3488](#3488)] - fix: plugin doc page not found [[#3480](#3480)] - Documentation improvements [[#3376](#3376)] - fix(deps): update docs npm deps non-major [[#3455](#3455)] - Add "Sonatype Nexus" plugin [[#3446](#3446)] - Add blog post [[#3439](#3439)] - Add "Gradle Wrapper Validation" plugin [[#3435](#3435)] - Add blog post [[#3410](#3410)] - Extend core ideas documentation [[#3405](#3405)] - docs: fix contributions link [[#3363](#3363)] - Update/fix some docs [[#3359](#3359)] - chore(deps): update dependency marked to v12 [[#3325](#3325)] ### 🐛 Bug Fixes - Fix skip setup for some general cli commands [[#3498](#3498)] - Move generic agent flags to cmd/agent/core [[#3484](#3484)] - Fix usage of WOODPECKER_DATABASE_DATASOURCE_FILE [[#3404](#3404)] - Set pull-request id and labels on pr-closed event [[#3442](#3442)] - Update org name on login [[#3409](#3409)] - Do not alter secret key upper-/lowercase [[#3375](#3375)] - fix: can't run multiple services on k8s [[#3395](#3395)] - Fix agent polling [[#3378](#3378)] - Remove empty strings from slice before parsing agent config [[#3387](#3387)] - Set correct link for commit [[#3368](#3368)] - Fix schema links [[#3369](#3369)] - Fix correctly handle gitlab pr closed events [[#3362](#3362)] - fix: update schema event_enum to remove error warning when.event [[#3357](#3357)] - Fix version check on next [[#3340](#3340)] - Ignore gitlab merge request events without code changes [[#3338](#3338)] - Ignore gitlab push events without commits [[#3339](#3339)] - Consider gitlab inherited permissions [[#3308](#3308)] - fix: agent panic when node is terminated during step execution [[#3331](#3331)] ### 📈 Enhancement - Enable golangci linter gomnd [[#3171](#3171)] - Apply "grpcnotrace" go build tag [[#3448](#3448)] - Simplify store interfaces [[#3437](#3437)] - Deprecate alternative names on secrets [[#3406](#3406)] - Store workflows/steps for blocked pipeline [[#2757](#2757)] - Parse email from Gitea webhook [[#3420](#3420)] - Replace http types on forge interface [[#3374](#3374)] - Prevent agent deletion when it's still running tasks [[#3377](#3377)] - Refactor internal services [[#915](#915)] - Lint for event filter and deprecate `exclude` [[#3222](#3222)] - Allow editing all environment variables in pipeline popups [[#3314](#3314)] - Parse backend options in backend [[#3227](#3227)] - Make agent usable for external backends [[#3270](#3270)] - Add no branches text [[#3312](#3312)] - Add loading spinner to repo list [[#3310](#3310)] ### Misc - Post on mastodon when releasing a new version [[#3509](#3509)] - chore(deps): update dependency alpine_3_18/ca-certificates to v20240226 [[#3501](#3501)] - fix(deps): update module github.com/google/go-github/v59 to v60 [[#3493](#3493)] - fix(deps): update dependency @intlify/unplugin-vue-i18n to v3 [[#3492](#3492)] - chore(deps): update dependency vue-tsc to v2 [[#3491](#3491)] - chore(deps): update dependency eslint-config-airbnb-typescript to v18 [[#3490](#3490)] - chore(deps): update web npm deps non-major [[#3489](#3489)] - fix(deps): update golang (packages) [[#3486](#3486)] - fix(deps): update module google.golang.org/protobuf to v1.33.0 [security] [[#3487](#3487)] - chore(deps): update docker.io/techknowlogick/xgo docker tag to go-1.22.1 [[#3476](#3476)] - chore(deps): update docker.io/golang docker tag to v1.22.1 [[#3475](#3475)] - Update prettier version [[#3471](#3471)] - chore(deps): update woodpeckerci/plugin-ready-release-go docker tag to v1.1.0 [[#3464](#3464)] - chore(deps): lock file maintenance [[#3465](#3465)] - chore(deps): update postgres docker tag to v16.2 [[#3461](#3461)] - chore(deps): update lycheeverse/lychee docker tag to v0.14.3 [[#3429](#3429)] - fix(deps): update golang (packages) [[#3430](#3430)] - More `when` filters [[#3407](#3407)] - Apply `documentation`/`ui` label to corresponding renovate updates [[#3400](#3400)] - chore(deps): update dependency eslint-plugin-simple-import-sort to v12 [[#3396](#3396)] - chore(deps): update typescript-eslint monorepo to v7 (major) [[#3397](#3397)] - fix(deps): update module github.com/google/go-github/v58 to v59 [[#3398](#3398)] - chore(deps): update docker.io/techknowlogick/xgo docker tag to go-1.22.0 [[#3392](#3392)] - chore(deps): update docker.io/golang docker tag [[#3391](#3391)] - fix(deps): update golang (packages) [[#3393](#3393)] - chore(deps): update docker.io/woodpeckerci/plugin-docker-buildx docker tag to v3.1.0 [[#3394](#3394)] - Add link checking [[#3371](#3371)] - Apply `dependencies` label to all PRs [[#3358](#3358)] - chore(deps): update docker.io/woodpeckerci/plugin-docker-buildx docker tag to v3.0.1 [[#3324](#3324)] --------- Co-authored-by: 6543 <m.huber@kithara.com> Co-authored-by: Anbraten <6918444+anbraten@users.noreply.github.com>

anbraten added the feature add new functionality label May 14, 2022

anbraten added this to the 1.0.0 milestone May 14, 2022

mscherer reviewed May 17, 2022

View reviewed changes

server/model/repo.go Outdated Show resolved Hide resolved

mscherer reviewed May 17, 2022

View reviewed changes

server/plugins/registry/http.go Outdated Show resolved Hide resolved

anbraten mentioned this pull request May 18, 2022

Hashicorp vault as secret storage integration #929

Open

3 tasks

anbraten mentioned this pull request Jun 2, 2022

Use asym key to sign webhooks woodpecker-ci/example-config-service#1

Closed

anbraten changed the title ~~Repo plugins~~ Repo extensions Jun 2, 2022

This comment was marked as off-topic.

Sign in to view

mscherer mentioned this pull request Aug 1, 2022

Plugin Mechanism go-gitea/gitea#20126

Open

20 tasks

anbraten mentioned this pull request Aug 11, 2022

Support shared configs #1089

Closed

3 tasks

This comment was marked as off-topic.

Sign in to view

anbraten mentioned this pull request Nov 26, 2022

Roadmap #869

Closed

31 tasks

qwerty287 removed this from the 1.0.0 milestone May 31, 2023

xoxys mentioned this pull request Jul 27, 2023

Extend Gated Feature with an Allow/Block List #336

Open

qwerty287 added this to the 3.x.x milestone Dec 25, 2023

qwerty287 mentioned this pull request Jan 12, 2024

Deprecate HTTP config service #3179

Closed

anbraten mentioned this pull request Jan 29, 2024

Rework addons (use rpc) #3268

Merged

anbraten removed this from the 3.x.x milestone Jan 30, 2024

addons

df09748

anbraten mentioned this pull request Feb 7, 2024

Allow to configure config extensions for repos #3349

Open

4 tasks

further improvements

69824f2

anbraten force-pushed the repo-plugins branch from f37c392 to 69824f2 Compare February 7, 2024 20:54

improve logic

ba2b536

anbraten added 6 commits February 7, 2024 23:23

cleanup

d824a1c

cleanup

80e8fcd

undo unrelated change

f2e43ee

rename folder

b094c9c

fixes

dc0be46

cleanup

59f237a

anbraten added 8 commits February 8, 2024 20:49

update readme

b38b722

service => extension

535ee6b

Merge branch 'main' into repo-plugins

6f5c285

add todos

a2df3a0

Merge branch 'repo-plugins' of github.com:anbraten/woodpecker into re…

fc4dfdf

…po-plugins

adjust todo

b3ef5fb

rename to ExtensionsMangager

6ec6ef8

add migration note

7d0e442

qwerty287 reviewed Feb 9, 2024

View reviewed changes

docs/docs/91-migrations.md Outdated Show resolved Hide resolved

server/extensions/manager.go Outdated Show resolved Hide resolved

server/extensions/permissions/admins.go Outdated Show resolved Hide resolved

anbraten added 5 commits February 11, 2024 14:33

Merge remote-tracking branch 'upstream/main' into repo-plugins

1a5c4e1

rename back to service

d5c574d

extension => service

5976fd4

update

6e0f1aa

cleanup

8285801

fix config fetching

f1133e0

lafriks approved these changes Feb 11, 2024

View reviewed changes

anbraten merged commit 82e1ce9 into woodpecker-ci:main Feb 11, 2024
8 checks passed

anbraten deleted the repo-plugins branch February 11, 2024 17:42

woodpecker-bot mentioned this pull request Feb 12, 2024

🎉 Release 2.4.0 #3307

Merged

1 task

qwerty287 mentioned this pull request Feb 17, 2024

Restarting pipelines does not work #3399

Closed

3 tasks

BrewTestBot mentioned this pull request Mar 19, 2024

woodpecker-cli 2.4.0 Homebrew/homebrew-core#166593

Merged

zc-devs mentioned this pull request Nov 14, 2024

Conversion extension or pipeline modification "on the fly"? #4341

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor internal services #915

Refactor internal services #915

anbraten commented May 14, 2022 •

edited

Loading

This comment was marked as off-topic.

This comment was marked as off-topic.

anbraten commented Feb 7, 2024 •

edited

Loading

anbraten commented Feb 11, 2024

qwerty287 commented Feb 11, 2024

anbraten commented Feb 11, 2024

qwerty287 commented Feb 11, 2024

anbraten commented Feb 11, 2024

codecov bot commented Feb 11, 2024

lafriks commented Feb 11, 2024

Refactor internal services #915

Refactor internal services #915

Conversation

anbraten commented May 14, 2022 • edited Loading

Changes

This comment was marked as off-topic.

This comment was marked as off-topic.

anbraten commented Feb 7, 2024 • edited Loading

anbraten commented Feb 11, 2024

qwerty287 commented Feb 11, 2024

anbraten commented Feb 11, 2024

qwerty287 commented Feb 11, 2024

anbraten commented Feb 11, 2024

codecov bot commented Feb 11, 2024

Codecov Report

lafriks commented Feb 11, 2024

anbraten commented May 14, 2022 •

edited

Loading

anbraten commented Feb 7, 2024 •

edited

Loading