Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sc-5933 Marshal transaction #99

Merged
merged 1 commit into from
Jun 20, 2022
Merged

sc-5933 Marshal transaction #99

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions pkg/rvasp/db/db.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -250,21 +250,21 @@ func (Account) TableName() string {
// TODO: Add a field for the transaction payload marshaled as a string.
type Transaction struct {
gorm.Model
TxID string `gorm:"not null"`
Envelope string `gorm:"not null"`
AccountID uint `gorm:"not null"`
Account Account `gorm:"foreignKey:AccountID"`
OriginatorID uint `gorm:"column:originator_id;not null"`
Originator Identity `gorm:"foreignKey:OriginatorID"`
BeneficiaryID uint `gorm:"column:beneficiary_id;not null"`
Beneficiary Identity `gorm:"foreignKey:BeneficiaryID"`
Amount decimal.Decimal `gorm:"type:numeric(15,2)"`
Amount decimal.Decimal `gorm:"type:decimal(15,8)"`
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid truncation in the database for small valued transfers.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hehe, you're right numeric(15,2) is for real money that has cents and not partial cents but we definitely need more precision for bitcoin transactions that can have a large fraction of a coin! Definitely a bit of bias we added when I first developed this model!

Debit bool `gorm:"not null"`
State pb.TransactionState `gorm:"not null;default:0"`
Timestamp time.Time `gorm:"not null"`
NotBefore time.Time `gorm:"not null"`
NotAfter time.Time `gorm:"not null"`
Identity string `gorm:"not null"`
Transaction string `gorm:"not null"`
Comment on lines 266 to +267
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that we're using postgres we could consider making these JSON types rather than raw strings -- but only if it's easy to do.

VaspID uint `gorm:"not null"`
Vasp VASP `gorm:"foreignKey:VaspID"`
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/rvasp/fixtures/wallets.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -407,7 +407,7 @@
"badnews@evilvasp.gg",
3,
"SendFull",
"SyncRequire",
"AsyncRepair",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this require us to re-migrate and update the documentation?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation is already updated but we do need to run the migration at some point.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a good stopping point that we can tag this as v1.0 to test goreleaser and the container builds then run the migrations? The postgres migrations shouldn't result in any downtime for the rVASPs, so I think we can run them when we need to; but we should still do it at off-peak hours.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do that whenever, I don't have any more rVASP work planned for the current sprint.

{
"natural_person": {
"name": {
Expand Down
10 changes: 8 additions & 2 deletions pkg/rvasp/rvasp.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -524,8 +524,7 @@ func (s *Server) respondAsync(peer *peers.Peer, payload *protocol.Payload, ident
switch xfer.State {
case pb.TransactionState_AWAITING:
// Fill the transaction with a new TxID to continue the handshake
xfer.TxID = uuid.New().String()
transaction.Txid = xfer.TxID
transaction.Txid = uuid.New().String()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch - CipherTrace has been reporting an issue where the rVASP returns an incorrect "receipt ID" than the one it was sent. I wonder if this is related. Note that they claim that receipt ID == envelope ID, but I'm not sure about that terminology. CC @DanielSollis

if payload, err = createTransferPayload(identity, transaction); err != nil {
log.Error().Err(err).Msg("could not create transfer payload")
return nil, protocol.Errorf(protocol.InternalError, "could not create transfer payload: %s", err)
Expand Down Expand Up @@ -578,6 +577,13 @@ func (s *Server) respondAsync(peer *peers.Peer, payload *protocol.Payload, ident
}
xfer.Identity = string(data)

// Update the transaction with the new generic.Transaction
if data, err = json.Marshal(transaction); err != nil {
log.Error().Err(err).Msg("could not marshal generic.Transaction")
return nil, status.Errorf(codes.Internal, "could not marshal generic.Transaction: %s", err)
}
xfer.Transaction = string(data)

// Update the Transaction in the database with the pending timestamps
if xfer.NotBefore, err = time.Parse(time.RFC3339, pending.ReplyNotBefore); err != nil {
log.Error().Err(err).Msg("TRISA protocol error: could not parse ReplyNotBefore timestamp")
Expand Down
90 changes: 54 additions & 36 deletions pkg/rvasp/trisa.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,29 @@ func (s *TRISA) handleTransaction(ctx context.Context, peer *peers.Peer, in *pro
}
}

// Repair the identity payload in a received transfer request by filling in the
// beneficiary identity information.
func (s *TRISA) repairBeneficiary(identity *ivms101.IdentityPayload, account db.Account) (err error) {
identity.BeneficiaryVasp = &ivms101.BeneficiaryVasp{}
if identity.BeneficiaryVasp.BeneficiaryVasp, err = s.parent.vasp.LoadIdentity(); err != nil {
log.Error().Err(err).Msg("could not load beneficiary vasp")
return err
}

identity.Beneficiary = &ivms101.Beneficiary{
BeneficiaryPersons: make([]*ivms101.Person, 0, 1),
AccountNumbers: []string{account.WalletAddress},
}

var beneficiary *ivms101.Person
if beneficiary, err = account.LoadIdentity(); err != nil {
log.Error().Err(err).Msg("could not load beneficiary account identity")
return err
}
identity.Beneficiary.BeneficiaryPersons = append(identity.Beneficiary.BeneficiaryPersons, beneficiary)
return nil
}

// respondTransfer responds to a transfer request from the originator by sending back
// the payload with the beneficiary identity information. If requireBeneficiary is
// true, the beneficiary identity must be filled in, or the transfer is rejected. If
Expand All @@ -381,24 +404,8 @@ func (s *TRISA) respondTransfer(in *protocol.SecureEnvelope, peer *peers.Peer, i
return nil, status.Errorf(codes.FailedPrecondition, "TRISA protocol error: missing beneficiary vasp identity")
}
} else {
// Update the identity with the beneficiary information
identity.BeneficiaryVasp = &ivms101.BeneficiaryVasp{}
if identity.BeneficiaryVasp.BeneficiaryVasp, err = s.parent.vasp.LoadIdentity(); err != nil {
log.Error().Err(err).Msg("could not load beneficiary vasp")
return nil, protocol.Errorf(protocol.InternalError, "request could not be processed")
}

identity.Beneficiary = &ivms101.Beneficiary{
BeneficiaryPersons: make([]*ivms101.Person, 0, 1),
AccountNumbers: []string{account.WalletAddress},
}

var beneficiary *ivms101.Person
if beneficiary, err = account.LoadIdentity(); err != nil {
log.Error().Err(err).Msg("could not load beneficiary")
return nil, protocol.Errorf(protocol.InternalError, "request could not be processed")
}
identity.Beneficiary.BeneficiaryPersons = append(identity.Beneficiary.BeneficiaryPersons, beneficiary)
// Fill in the beneficiary identity information for the repair policy
s.repairBeneficiary(identity, account)
}

// Update the transaction with beneficiary information
Expand Down Expand Up @@ -487,7 +494,6 @@ func (s *TRISA) respondPending(in *protocol.SecureEnvelope, peer *peers.Peer, id
log.Error().Err(err).Msg("could not construct transaction")
return nil, protocol.Errorf(protocol.InternalError, "request could not be processed")
}
xfer.TxID = transaction.Txid
xfer.Envelope = in.Id
xfer.Account = account
xfer.Amount = decimal.NewFromFloat(transaction.Amount)
Expand All @@ -511,12 +517,19 @@ func (s *TRISA) respondPending(in *protocol.SecureEnvelope, peer *peers.Peer, id
xfer.NotAfter = now.Add(s.parent.conf.AsyncNotAfter)

// Marshal the identity info into the local transaction
var identityBytes []byte
if identityBytes, err = protojson.Marshal(identity); err != nil {
var data []byte
if data, err = protojson.Marshal(identity); err != nil {
log.Error().Err(err).Msg("could not marshal identity")
return nil, protocol.Errorf(protocol.InternalError, "request could not be processed")
}
xfer.Identity = string(identityBytes)
xfer.Identity = string(data)

// Marshal the generic.Transaction into the local transaction
if data, err = protojson.Marshal(transaction); err != nil {
log.Error().Err(err).Msg("could not marshal transaction")
return nil, protocol.Errorf(protocol.InternalError, "request could not be processed")
}
xfer.Transaction = string(data)

// Save the updated transaction in the database
if err = s.parent.db.Save(&xfer).Error; err != nil {
Expand Down Expand Up @@ -582,27 +595,32 @@ func (s *TRISA) sendAsync(tx *db.Transaction) (err error) {
return fmt.Errorf("could not fetch originator address")
}

// Fetch the beneficiary address
var beneficiary *db.Identity
if beneficiary, err = tx.GetBeneficiary(s.parent.db); err != nil {
log.Error().Err(err).Msg("could not fetch beneficiary address")
return fmt.Errorf("could not fetch beneficiary address")
}

// Create the identity for the payload
identity := &ivms101.IdentityPayload{}
if err = protojson.Unmarshal([]byte(tx.Identity), identity); err != nil {
log.Error().Err(err).Msg("could not unmarshal identity from transaction")
return fmt.Errorf("could not unmarshal identity from transaction: %s", err)
}

// Create the transaction for the payload
transaction := &generic.Transaction{
Txid: tx.TxID,
Originator: originator.WalletAddress,
Beneficiary: beneficiary.WalletAddress,
Amount: float64(tx.AmountFloat()),
Timestamp: tx.Timestamp.Format(time.RFC3339),
// Repair the beneficiary information if this is the first handshake
if tx.State == pb.TransactionState_PENDING_SENT {
var account *db.Account
if account, err = tx.GetAccount(s.parent.db); err != nil {
log.Error().Err(err).Msg("could not fetch beneficiary account")
return fmt.Errorf("could not fetch beneficiary account: %s", err)
}

if err = s.repairBeneficiary(identity, *account); err != nil {
log.Error().Err(err).Msg("could not repair beneficiary information")
return fmt.Errorf("could not repair beneficiary information: %s", err)
}
}
bbengfort marked this conversation as resolved.
Show resolved Hide resolved

// Create the generic.Transaction for the payload
transaction := &generic.Transaction{}
if err = protojson.Unmarshal([]byte(tx.Transaction), transaction); err != nil {
log.Error().Err(err).Msg("could not unmarshal generic.Transaction from transaction")
return fmt.Errorf("could not unmarshal generic.Transaction from transaction: %s", err)
}

// Create the payload
Expand Down
44 changes: 26 additions & 18 deletions scripts/rvasp-test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,113 +1,121 @@
#!/bin/bash
# TODO: check codes

if [ "$1" == "--local" ]; then
ALICE_ENDPOINT=localhost:5434
BOB_ENDPOINT=localhost:6434
else
ALICE_ENDPOINT=admin.alice.vaspbot.net:443
BOB_ENDPOINT=admin.bob.vaspbot.net:443
fi
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've discovered that this script is really useful for local testing as well, thanks for adding it!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad it's helpful!


# Send from Alice to Bob
# Partial Sync Repair: success expected
echo "Alice --> Bob Partial Sync Repair"
rvasp transfer -e admin.alice.vaspbot.net:443 \
rvasp transfer -e $ALICE_ENDPOINT \
-a 1ASkqdo1hvydosVRvRv2j6eNnWpWLHucMX -d 0.0001 \
-b 18nxAxBktHZDrMoJ3N2fk9imLX8xNnYbNh

# Partial Sync Require: rejection expected
echo "Alice --> Bob Partial Sync Require"
rvasp transfer -e admin.alice.vaspbot.net:443 \
rvasp transfer -e $ALICE_ENDPOINT \
-a 1ASkqdo1hvydosVRvRv2j6eNnWpWLHucMX -d 0.0002 \
-b 1LgtLYkpaXhHDu1Ngh7x9fcBs5KuThbSzw

# Full Sync Repair: success expected
echo "Alice --> Bob Full Sync Repair"
rvasp transfer -e admin.alice.vaspbot.net:443 \
rvasp transfer -e $ALICE_ENDPOINT \
-a 1MRCxvEpBoY8qajrmNTSrcfXSZ2wsrGeha -d 0.0003 \
-b 18nxAxBktHZDrMoJ3N2fk9imLX8xNnYbNh

# Full Sync Require: success expected
echo "Alice --> Bob Full Sync Require"
rvasp transfer -e admin.alice.vaspbot.net:443 \
rvasp transfer -e $ALICE_ENDPOINT \
-a 14HmBSwec8XrcWge9Zi1ZngNia64u3Wd2v -d 0.0004 \
-b 1LgtLYkpaXhHDu1Ngh7x9fcBs5KuThbSzw

# Partial Async Repair: success expected
# TODO: how to test getting a message back?
echo "Alice --> Bob Partial Async Repair"
rvasp transfer -e admin.alice.vaspbot.net:443 \
rvasp transfer -e $ALICE_ENDPOINT \
-a 1ASkqdo1hvydosVRvRv2j6eNnWpWLHucMX -d 0.0005 \
-b 14WU745djqecaJ1gmtWQGeMCFim1W5MNp3

# Partial Async Reject: rejection expected
echo "Alice --> Bob Partial Async Reject"
rvasp transfer -e admin.alice.vaspbot.net:443 \
rvasp transfer -e $ALICE_ENDPOINT \
-a 1ASkqdo1hvydosVRvRv2j6eNnWpWLHucMX -d 0.0006 \
-b 1Hzej6a2VG7C8iCAD5DAdN72cZH5THSMt9

# Full Async Repair: success expected
echo "Alice --> Bob Full Async Repair"
rvasp transfer -e admin.alice.vaspbot.net:443 \
rvasp transfer -e $ALICE_ENDPOINT \
-a 1MRCxvEpBoY8qajrmNTSrcfXSZ2wsrGeha -d 0.0007 \
-b 14WU745djqecaJ1gmtWQGeMCFim1W5MNp3

# Full Async Require: reject expected
echo "Alice --> Bob Full Async Reject"
rvasp transfer -e admin.alice.vaspbot.net:443 \
rvasp transfer -e $ALICE_ENDPOINT \
-a 14HmBSwec8XrcWge9Zi1ZngNia64u3Wd2v -d 0.0008 \
-b 1Hzej6a2VG7C8iCAD5DAdN72cZH5THSMt9

# Send Error: rejection expected.
echo "Alice --> Bob Send Error"
rvasp transfer -e admin.alice.vaspbot.net:443 \
rvasp transfer -e $ALICE_ENDPOINT \
-a 19nFejdNSUhzkAAdwAvP3wc53o8dL326QQ -d 0.0009 \
-b 1Hzej6a2VG7C8iCAD5DAdN72cZH5THSMt9

# Send from Bob to Alice
# Partial Sync Repair: success expected
echo "Bob --> Alice Partial Sync Repair"
rvasp transfer -e admin.bob.vaspbot.net:443 \
rvasp transfer -e $BOB_ENDPOINT \
-a 18nxAxBktHZDrMoJ3N2fk9imLX8xNnYbNh -d 0.00010 \
-b 1ASkqdo1hvydosVRvRv2j6eNnWpWLHucMX

# Partial Sync Require: rejection expected
echo "Bob --> Alice Partial Sync Require"
rvasp transfer -e admin.bob.vaspbot.net:443 \
rvasp transfer -e $BOB_ENDPOINT \
-a 18nxAxBktHZDrMoJ3N2fk9imLX8xNnYbNh -d 0.00011 \
-b 1MRCxvEpBoY8qajrmNTSrcfXSZ2wsrGeha

# Full Sync Repair: success expected
echo "Bob --> Alice Full Sync Repair"
rvasp transfer -e admin.bob.vaspbot.net:443 \
rvasp transfer -e $BOB_ENDPOINT \
-a 1LgtLYkpaXhHDu1Ngh7x9fcBs5KuThbSzw -d 0.00012 \
-b 1ASkqdo1hvydosVRvRv2j6eNnWpWLHucMX

# Full Sync Require: success expected
echo "Bob --> Alice Full Sync Require"
rvasp transfer -e admin.bob.vaspbot.net:443 \
rvasp transfer -e $BOB_ENDPOINT \
-a 14WU745djqecaJ1gmtWQGeMCFim1W5MNp3 -d 0.00013 \
-b 1MRCxvEpBoY8qajrmNTSrcfXSZ2wsrGeha

# Partial Async Repair: success expected
echo "Bob --> Alice Partial Async Repair"
rvasp transfer -e admin.bob.vaspbot.net:443 \
rvasp transfer -e $BOB_ENDPOINT \
-a 18nxAxBktHZDrMoJ3N2fk9imLX8xNnYbNh -d 0.00014 \
-b 14HmBSwec8XrcWge9Zi1ZngNia64u3Wd2v

# Partial Async Require: rejection expected
echo "Bob --> Alice Partial Async Reject"
rvasp transfer -e admin.bob.vaspbot.net:443 \
rvasp transfer -e $BOB_ENDPOINT \
-a 18nxAxBktHZDrMoJ3N2fk9imLX8xNnYbNh -d 0.00015 \
-b 19nFejdNSUhzkAAdwAvP3wc53o8dL326QQ

# Full Async Repair: success expected
echo "Bob --> Alice Full Async Repair"
rvasp transfer -e admin.bob.vaspbot.net:443 \
rvasp transfer -e $BOB_ENDPOINT \
-a 1LgtLYkpaXhHDu1Ngh7x9fcBs5KuThbSzw -d 0.00016 \
-b 14HmBSwec8XrcWge9Zi1ZngNia64u3Wd2v

# Full Async Require: rejection expected
echo "Bob --> Alice Full Async Reject"
rvasp transfer -e admin.bob.vaspbot.net:443 \
rvasp transfer -e $BOB_ENDPOINT \
-a 14WU745djqecaJ1gmtWQGeMCFim1W5MNp3 -d 0.00017 \
-b 19nFejdNSUhzkAAdwAvP3wc53o8dL326QQ

# Send Error: rejection expected.
echo "Bob --> Alice Send Error"
rvasp transfer -e admin.bob.vaspbot.net:443 \
rvasp transfer -e $BOB_ENDPOINT \
-a 1Hzej6a2VG7C8iCAD5DAdN72cZH5THSMt9 -d 0.00018 \
-b 14HmBSwec8XrcWge9Zi1ZngNia64u3Wd2v