Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: migrate existing tables to have autoincrementing primary keys #274

Merged
merged 7 commits into from
Jul 16, 2024

Conversation

rolznz
Copy link
Contributor

@rolznz rolznz commented Jul 15, 2024

Fixes #252

This PR adds autoincrement to the primary keys of all our tables which fixes weird bugs caused by ID reuse which is the default in SQLITE (if you delete app ID 7, then the next app you create will have ID 7 again). Only apps, app_permissions and user_configs are preserved. The other tables are dropped (payments are dropped as part of dynamic budgets, request_events and response_events are not critical)

TESTS:

  • diff select * from table-name before and after migration (no change)
  • diff schema changes (only expected changes - autoincrement, one missed index rename for scopes)
  • autoincrement
    • apps
    • app_permissions
    • user_configs
    • request_events
    • response_events
    • transactions
  • cascading delete
    • apps -> app_permissions
    • apps -> request_events
    • request_events -> response_events
  • create app with one NWC request, delete app, create new app -> new ID and bug fixed?
  • check: should index creation of request and response events be done after new app table is created?

@rolznz rolznz marked this pull request as ready for review July 15, 2024 16:36
// }
// this causes errors when concurrently saving DB entries and otherwise requires mutexes
// sqlDb.SetMaxOpenConns(1)
err = gormDB.Exec("PRAGMA busy_timeout = 5000", nil).Error
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added so that we won't get a SQL busy error thrown if there are two concurrent DB queries

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea!

// }
// this causes errors when concurrently saving DB entries and otherwise requires mutexes
// sqlDb.SetMaxOpenConns(1)
err = gormDB.Exec("PRAGMA busy_timeout = 5000", nil).Error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea!

db/migrations/202407151352_autoincrement.go Show resolved Hide resolved
db/migrations/202407151352_autoincrement.go Outdated Show resolved Hide resolved
db/migrations/202407151352_autoincrement.go Show resolved Hide resolved
@rolznz rolznz merged commit a224276 into feat/dynamic-budgets Jul 16, 2024
4 of 8 checks passed
@rolznz rolznz deleted the fix/autoincrement branch July 16, 2024 15:08
rolznz added a commit that referenced this pull request Jul 19, 2024
* feat: add transactions table

* feat: add transactions service with makeinvoice method

* feat: use internal transactions WIP

* feat: check unsettled transactions

* feat: use transactions service in nip47 package WIP

* feat: use transactions service in NIP-47 handlers

* fix: transaction list colors

* fix: tests

* chore: remove old payments table

* fix: return budget usage as sats

* feat: use internal transactions table for keysend payments

* feat: consume nwc_payment_received event in transaction service (WIP)

* feat: update existing transaction from nwc_payment_received event in transactions service

* feat: update existing transaction from nwc_payment_sent event in transactions service

* feat: handle async payment failed events in transactions service

* fix: tests

* feat: correctly implement NIP-47 NOT_FOUND error code

* feat: intercept self payments

* feat: isolated balance and visibility

* feat: validate keysend payment does not exceed app internal balance

* feat: budget check in transactions service, correctly pass payment errors to NIP-47 response

also reduce query duplication

* fix: order transactions when looking up transaction

* feat: add fee reserves to unsettled outgoing transactions

* chore: rename app permissions max amount field

* chore: rename transaction amount values to be clearly millisats

* chore: merge balance type and visibility into isolated property on apps table

* chore: move duplicated permission check from nip-47 controllers to handler

* fix: app name in transaction list

* Feat: permissions revamp v2 (#273)

* feat: revamp permissions component

* chore: changes

* chore: changes

* chore: add view mode for show app screen

* chore: further changes

* feat: new illustration for linking account (#254)

* feat: new illustration for linking account

* fix: update paths

* fix: icon props (#256)

* fix: use date from frontend

* chore: add expiryselect component

* chore: further changes

* chore: further changes

* chore: add date-fns for date picker

* chore: further changes

* chore: further changes

* fix: add central LDK gossip node to help gossip new public channels (#262)

* fix: make dialog responsive (#258)

* chore: further changes

* typo

* chore: spacing issues

* chore: use scopes from capabilities

* chore: change scope type descriptions

* chore: styling fixes

* chore: fix typings

* chore: budget renewal component

* fix: LDK mark channel as inactive and show error if counterparty forwarding info missing (#267)

fix: mark channel as inactive and show error if counterparty forwarding info missing

* chore: remove unnecessary dark classes (#255)

* fix: links to open first channel in sidebar and onboarding checklist (#268)

* feat: improve migrate node UI (#269)

* fix: migrate node copy (#270)

* fix: stop nostr when app is shutdown and use context to stop lnclient (#271)

* fix: permissions revamp WIP

* feat: basic isolated apps UI

* fix: new app connection, edit app connection, deep linking

---------

Co-authored-by: im-adithya <imadithyavardhan@gmail.com>
Co-authored-by: René Aaron <100827540+reneaaron@users.noreply.github.com>
Co-authored-by: Michael Bumann <hello@michaelbumann.com>

* fix: make transactions table ID autoincrement

* fix: do not send unrelated notifications to isolated apps

* fix: nip-47 notifications not receiving updated transaction state

* fix: migrate existing tables to have autoincrementing primary keys (#274)

* fix: migrate existing tables to have autoincrementing primary keys

* fix: recreate request and response event tables after apps

* fix: remove null from request_events app_id

* fix: do not crash when reloading app created page

* chore: update pragma commands, add busy_timeout

* fix: remove unnecessary migration of user_configs table

* chore: update comment on autoincrement migration

* feat: sqlite database config improvements

* fix: autoincrement migration to delete unlinked app permissions

* chore: update tests for dynamic budgets (WIP)

* fix: get balance tests

* chore: add extra event handler tests

* chore: add extra event handler test

* chore: add extra multi_pay_invoice tests

* chore: transactions service tests (WIP)

* chore: add transactions service payment tests

* chore: add tests for self payments

* fix: notifications tests

* chore: add notifications tests for transactions service

* chore: add fee reserve tests for transactions service

* chore: add list transactions tests

* chore: add keysend tests for transactions service

* feat: subscribe for payments and invoices (#281)

* fix: db transaction should be passed as pointer

* feat(lnd): subscribe for payments and invoices

* chore: rearrange check for settled

* chore: remove TODO

* chore: retry on error and add select

* chore: publish payment failed event

* chore: use json logging

* feat: add lnd notification types

* chore: remove sleep

* fix: incorrect key on transaction list items

* fix: test

* fix: disable isolated app type on non-supported backends

---------

Co-authored-by: im-adithya <imadithyavardhan@gmail.com>
Co-authored-by: René Aaron <100827540+reneaaron@users.noreply.github.com>
Co-authored-by: Michael Bumann <hello@michaelbumann.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants