Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix getLog API to use log_cids table #92

Merged
merged 14 commits into from
Sep 16, 2021
Merged

Fix getLog API to use log_cids table #92

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
d8a5358
Remove ipld-eth-indexer dependency.
arijitAD Aug 12, 2021
b3e4fbf
Fix get logs API.
arijitAD Aug 14, 2021
c0a4600
Refactoring.
arijitAD Aug 15, 2021
3d1b308
Update go mod.
arijitAD Aug 16, 2021
a28892f
Fix get log API to use log_cids table.
arijitAD Aug 20, 2021
d0c3241
Add reciept status in log for graphql.
arijitAD Aug 30, 2021
71837c4
Self review.
arijitAD Aug 31, 2021
967c148
Add test for graphql.
arijitAD Sep 1, 2021
121c75c
Change getLogs GQL API to take nil contract address.
arijitAD Sep 2, 2021
4d9edd2
Minor change.
arijitAD Sep 2, 2021
458aae1
Address comments.
arijitAD Sep 3, 2021
04a0f9a
Add status field for pre-byzantium blocks.
arijitAD Sep 13, 2021
7f0c8fb
Add index and fix go.mod.
arijitAD Sep 14, 2021
ed4171a
Revert "Add status field for pre-byzantium blocks."
arijitAD Sep 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions db/migrations/00006_create_eth_transaction_cids_table.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ CREATE TABLE eth.transaction_cids (
src VARCHAR(66) NOT NULL,
tx_data BYTEA,
tx_type BYTEA,
gas INTEGER NOT NULL,
Copy link
Collaborator

@i-norden i-norden Sep 14, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's call this gas_limit, it's more accurate to how things are defined in EIPs/specs and more explicit about what it is/how it is used.

UNIQUE (header_id, tx_hash)
);

Expand Down
1 change: 1 addition & 0 deletions db/migrations/00007_create_eth_receipt_cids_table.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ CREATE TABLE eth.receipt_cids (
post_state VARCHAR(66),
post_status INTEGER,
log_root VARCHAR(66),
gas_used INTEGER NOT NULL,
UNIQUE (tx_id)
);

Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,6 @@ require (
github.com/vulcanize/ipfs-ethdb v0.0.4-0.20210824131459-7bb49801fc12
)

replace github.com/ethereum/go-ethereum v1.10.8 => github.com/vulcanize/go-ethereum v1.10.8-statediff-0.0.26
replace github.com/ethereum/go-ethereum v1.10.8 => /Users/arijitdas/go/src/github.com/ethereum/go-ethereum

replace github.com/vulcanize/ipfs-ethdb v0.0.2-alpha => github.com/vulcanize/pg-ipfs-ethdb v0.0.2-alpha
2 changes: 0 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -711,8 +711,6 @@ github.com/valyala/fasttemplate v1.0.1/go.mod h1:UQGH1tvbgY+Nz5t2n7tXsz52dQxojPU
github.com/valyala/fasttemplate v1.2.1/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ=
github.com/vulcanize/gap-filler v0.3.1 h1:N5d+jCJo/VTWFvBSbTD7biRhK/OqDZzi1tgA85SIBKs=
github.com/vulcanize/gap-filler v0.3.1/go.mod h1:qowG1cgshVpBqMokiWro/1xhh0uypw7oAu8FQ42JMy4=
github.com/vulcanize/go-ethereum v1.10.8-statediff-0.0.26 h1:1UBVQpeJnHkmSKxXanbNGE8w+LR0iZhfGr0QrQ62+C4=
github.com/vulcanize/go-ethereum v1.10.8-statediff-0.0.26/go.mod h1:nXs5fPBjAVzBmIGtrc0f7akQwkXI5Mi+6I1QcbD2br0=
github.com/vulcanize/ipfs-ethdb v0.0.4-0.20210824131459-7bb49801fc12 h1:IKqHA89qA+VZBYt1nZ1EInVrAgB3iA5U+klkF4l8mn4=
github.com/vulcanize/ipfs-ethdb v0.0.4-0.20210824131459-7bb49801fc12/go.mod h1:IueWysMbZu0uFmu+ia6mEnyWsTvwe2q2lbYdy2muRUM=
github.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1/go.mod h1:8UvriyWtv5Q5EOgjHaSseUEdkQfvwFv1I/In/O2M9gc=
Expand Down
10 changes: 8 additions & 2 deletions pkg/eth/api.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -524,12 +524,18 @@ func (pea *PublicEthAPI) localGetTransactionReceipt(ctx context.Context, hash co
"logsBloom": receipt.Bloom,
}

// Assign receipt status or post state.
if len(receipt.PostState) > 0 {
if blockNumber <= pea.B.Config.ChainConfig.ByzantiumBlock.Uint64() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A problem with this is that this deviates from the normal, expected behavior of this eth json-rpc endpoint. The regular endpoint only fills in the status if it is explicitly known: https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/api.go#L1650. So this breaks compatibility with the regular endpoint.

if receipt.GasUsed > tx.Gas() {
Copy link
Collaborator

@i-norden i-norden Sep 14, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm missing something. When will receipt.GasUsed ever be more than tx.Gas()? When a tx fails for whatever reason is it's receipt.GasUsed simply set to something above the tx.Gas() to indicate it failed? Seems odd but I'm not super familiar with this pre-byzantium behavior- wouldn't it need to track/report how much gas was actually used during the failed execution/reversion (because the tx sender still has to pay for whatever execution was performed prior to it reverting/failing)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an issue to track this #102

fields["status"] = hexutil.Uint(types.ReceiptStatusFailed)
} else {
fields["status"] = hexutil.Uint(types.ReceiptStatusSuccessful)
}
} else if len(receipt.PostState) > 0 {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still want to fill in the fields["root"] if receipt.PostState is present, whether or not we can also assume the status above.

fields["root"] = hexutil.Bytes(receipt.PostState)
} else {
fields["status"] = hexutil.Uint(receipt.Status)
}

if receipt.Logs == nil {
fields["logs"] = []*types.Log{}
}
Expand Down
73 changes: 53 additions & 20 deletions pkg/eth/api_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,15 +43,16 @@ import (
)

var (
randomAddr = common.HexToAddress("0x1C3ab14BBaD3D99F4203bd7a11aCB94882050E6f")
randomHash = crypto.Keccak256Hash(randomAddr.Bytes())
number = rpc.BlockNumber(test_helpers.BlockNumber.Int64())
londonBlockNum = rpc.BlockNumber(test_helpers.LondonBlockNum.Int64())
wrongNumber = number + 1
blockHash = test_helpers.MockBlock.Header().Hash()
baseFee = test_helpers.MockLondonBlock.BaseFee()
ctx = context.Background()
expectedBlock = map[string]interface{}{
randomAddr = common.HexToAddress("0x1C3ab14BBaD3D99F4203bd7a11aCB94882050E6f")
randomHash = crypto.Keccak256Hash(randomAddr.Bytes())
number = rpc.BlockNumber(test_helpers.BlockNumber.Int64())
londonBlockNum = rpc.BlockNumber(test_helpers.LondonBlockNum.Int64())
byzantiumBlockNum = rpc.BlockNumber(test_helpers.ByzantiumBlockNum.Int64())
wrongNumber = number + 1
blockHash = test_helpers.MockBlock.Header().Hash()
baseFee = test_helpers.MockLondonBlock.BaseFee()
ctx = context.Background()
expectedBlock = map[string]interface{}{
"number": (*hexutil.Big)(test_helpers.MockBlock.Number()),
"hash": test_helpers.MockBlock.Hash(),
"parentHash": test_helpers.MockBlock.ParentHash(),
Expand Down Expand Up @@ -131,14 +132,15 @@ var (
"receiptsRoot": test_helpers.MockUncles[1].ReceiptHash,
"uncles": []common.Hash{},
}
expectedTransaction = eth.NewRPCTransaction(test_helpers.MockTransactions[0], test_helpers.MockBlock.Hash(), test_helpers.MockBlock.NumberU64(), 0, test_helpers.MockBlock.BaseFee())
expectedTransaction2 = eth.NewRPCTransaction(test_helpers.MockTransactions[1], test_helpers.MockBlock.Hash(), test_helpers.MockBlock.NumberU64(), 1, test_helpers.MockBlock.BaseFee())
expectedTransaction3 = eth.NewRPCTransaction(test_helpers.MockTransactions[2], test_helpers.MockBlock.Hash(), test_helpers.MockBlock.NumberU64(), 2, test_helpers.MockBlock.BaseFee())
expectedLondonTransaction = eth.NewRPCTransaction(test_helpers.MockLondonTransactions[0], test_helpers.MockLondonBlock.Hash(), test_helpers.MockLondonBlock.NumberU64(), 0, test_helpers.MockLondonBlock.BaseFee())
expectRawTx, _ = rlp.EncodeToBytes(test_helpers.MockTransactions[0])
expectRawTx2, _ = rlp.EncodeToBytes(test_helpers.MockTransactions[1])
expectRawTx3, _ = rlp.EncodeToBytes(test_helpers.MockTransactions[2])
expectedReceipt = map[string]interface{}{
expectedTransaction = eth.NewRPCTransaction(test_helpers.MockTransactions[0], test_helpers.MockBlock.Hash(), test_helpers.MockBlock.NumberU64(), 0, test_helpers.MockBlock.BaseFee())
expectedTransaction2 = eth.NewRPCTransaction(test_helpers.MockTransactions[1], test_helpers.MockBlock.Hash(), test_helpers.MockBlock.NumberU64(), 1, test_helpers.MockBlock.BaseFee())
expectedTransaction3 = eth.NewRPCTransaction(test_helpers.MockTransactions[2], test_helpers.MockBlock.Hash(), test_helpers.MockBlock.NumberU64(), 2, test_helpers.MockBlock.BaseFee())
expectedLondonTransaction = eth.NewRPCTransaction(test_helpers.MockLondonTransactions[0], test_helpers.MockLondonBlock.Hash(), test_helpers.MockLondonBlock.NumberU64(), 0, test_helpers.MockLondonBlock.BaseFee())
expectedByzantiumTransaction = eth.NewRPCTransaction(test_helpers.MockByzantiumTransactions[0], test_helpers.MockByzantiumBlock.Hash(), test_helpers.MockByzantiumBlock.NumberU64(), 0, test_helpers.MockByzantiumBlock.BaseFee())
expectRawTx, _ = rlp.EncodeToBytes(test_helpers.MockTransactions[0])
expectRawTx2, _ = rlp.EncodeToBytes(test_helpers.MockTransactions[1])
expectRawTx3, _ = rlp.EncodeToBytes(test_helpers.MockTransactions[2])
expectedReceipt = map[string]interface{}{
"blockHash": blockHash,
"blockNumber": hexutil.Uint64(uint64(number.Int64())),
"transactionHash": expectedTransaction.Hash,
Expand Down Expand Up @@ -180,6 +182,20 @@ var (
"logsBloom": test_helpers.MockReceipts[2].Bloom,
"root": hexutil.Bytes(test_helpers.MockReceipts[2].PostState),
}
expectedByzantiumReceipt = map[string]interface{}{
"blockHash": test_helpers.MockByzantiumBlock.Hash(),
"blockNumber": hexutil.Uint64(uint64(byzantiumBlockNum)),
"transactionHash": expectedByzantiumTransaction.Hash,
"transactionIndex": hexutil.Uint64(0),
"from": expectedByzantiumTransaction.From,
"to": expectedByzantiumTransaction.To,
"gasUsed": hexutil.Uint64(test_helpers.MockByzantiumReceipts[0].GasUsed),
"cumulativeGasUsed": hexutil.Uint64(test_helpers.MockByzantiumReceipts[0].CumulativeGasUsed),
"contractAddress": nil,
"logs": test_helpers.MockByzantiumReceipts[0].Logs,
"logsBloom": test_helpers.MockByzantiumReceipts[0].Bloom,
"status": hexutil.Uint(test_helpers.MockByzantiumReceipts[0].Status),
}
)

// SetupDB is use to setup a db for watcher tests
Expand Down Expand Up @@ -210,6 +226,13 @@ var _ = Describe("API", func() {

db, err = SetupDB()
Expect(err).ToNot(HaveOccurred())

// setting chain config to for byzantium block
chainConfig.ByzantiumBlock = big.NewInt(1)

// setting chain config to for london block
chainConfig.LondonBlock = big.NewInt(3)

indexAndPublisher := indexer.NewStateDiffIndexer(chainConfig, db)
backend, err := eth.NewEthBackend(db, &eth.Config{
ChainConfig: chainConfig,
Expand Down Expand Up @@ -249,14 +272,17 @@ var _ = Describe("API", func() {
}
expectedBlock["uncles"] = uncleHashes

// setting chain config to for london block
chainConfig.LondonBlock = big.NewInt(2)
indexAndPublisher = indexer.NewStateDiffIndexer(chainConfig, db)
tx, err = indexAndPublisher.PushBlock(test_helpers.MockLondonBlock, test_helpers.MockLondonReceipts, test_helpers.MockLondonBlock.Difficulty())
Expect(err).ToNot(HaveOccurred())

err = tx.Close(err)
Expect(err).ToNot(HaveOccurred())

tx, err = indexAndPublisher.PushBlock(test_helpers.MockByzantiumBlock, test_helpers.MockByzantiumReceipts, test_helpers.MockByzantiumBlock.Difficulty())
Expect(err).ToNot(HaveOccurred())

err = tx.Close(err)
Expect(err).ToNot(HaveOccurred())
})

// Single test db tear down at end of all tests
Expand Down Expand Up @@ -644,6 +670,13 @@ var _ = Describe("API", func() {
_, err := api.GetTransactionReceipt(ctx, randomHash)
Expect(err).To(HaveOccurred())
})

It("Retrieve receipt for pre byzantium block transaction", func() {
hash := test_helpers.MockByzantiumTransactions[0].Hash()
tx, err := api.GetTransactionReceipt(ctx, hash)
Expect(err).ToNot(HaveOccurred())
Expect(tx).To(Equal(expectedByzantiumReceipt))
})
})

Describe("eth_getLogs", func() {
Expand Down
63 changes: 63 additions & 0 deletions pkg/eth/backend.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import (
"errors"
"fmt"
"math/big"
"strconv"

"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/hexutil"
Expand Down Expand Up @@ -472,6 +473,7 @@ func (b *Backend) GetTransaction(ctx context.Context, txHash common.Hash) (*type
BlockHash string `db:"block_hash"`
BlockNumber uint64 `db:"block_number"`
Index uint64 `db:"index"`
Gas uint64 `db:"gas"`
}
if err := b.DB.Get(&tempTxStruct, RetrieveRPCTransaction, txHash.String()); err != nil {
return nil, common.Hash{}, 0, 0, err
Expand Down Expand Up @@ -799,6 +801,67 @@ func (b *Backend) RPCGasCap() *big.Int {
return b.Config.RPCGasCap
}

type logsCID struct {
Log *types.Log
CID string
RctCID string
LogLeafData []byte
RctStatus uint64
}

// DecomposeGQLLogs return logs for graphql.
func (b *Backend) DecomposeGQLLogs(logCIDs []LogResult) ([]logsCID, error) {
logs := make([]logsCID, len(logCIDs))
for i, l := range logCIDs {
topics := make([]common.Hash, 0)
if l.Topic0 != "" {
topics = append(topics, common.HexToHash(l.Topic0))
}
if l.Topic1 != "" {
topics = append(topics, common.HexToHash(l.Topic1))
}
if l.Topic2 != "" {
topics = append(topics, common.HexToHash(l.Topic2))
}
if l.Topic3 != "" {
topics = append(topics, common.HexToHash(l.Topic3))
}

// TODO: should we convert string to uint ?
blockNum, err := strconv.ParseUint(l.BlockNumber, 10, 64)
if err != nil {
return nil, err
}

var status uint64
if blockNum <= b.Config.ChainConfig.ByzantiumBlock.Uint64() {
if l.RctGasUsed > l.Gas {
status = types.ReceiptStatusFailed
} else {
status = types.ReceiptStatusSuccessful
}
} else {
status = l.RctStatus
}

logs[i] = logsCID{
Log: &types.Log{
Address: common.HexToAddress(l.Address),
Topics: topics,
Data: l.Data,
Index: uint(l.Index),
TxHash: common.HexToHash(l.TxHash),
},
CID: l.LeafCID,
RctCID: l.RctCID,
LogLeafData: l.LogLeafData,
RctStatus: status,
}
}

return logs, nil
}

func (b *Backend) SubscribeNewTxsEvent(chan<- core.NewTxsEvent) event.Subscription {
panic("implement me")
}
Expand Down
5 changes: 3 additions & 2 deletions pkg/eth/cid_retriever.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -316,7 +316,8 @@ func (ecr *CIDRetriever) RetrieveFilteredGQLLogs(tx *sqlx.Tx, rctFilter ReceiptF
id := 1
pgStr := `SELECT eth.log_cids.leaf_cid, eth.log_cids.index, eth.log_cids.receipt_id,
eth.log_cids.address, eth.log_cids.topic0, eth.log_cids.topic1, eth.log_cids.topic2, eth.log_cids.topic3,
eth.log_cids.log_data, eth.transaction_cids.tx_hash, data, eth.receipt_cids.cid, eth.receipt_cids.post_status
eth.log_cids.log_data, eth.transaction_cids.tx_hash, eth.transaction_cids.gas, data, eth.receipt_cids.cid, eth.receipt_cids.post_status,
eth.receipt_cids.gas_used,header_cids.block_number
FROM eth.log_cids, eth.receipt_cids, eth.transaction_cids, eth.header_cids, public.blocks
WHERE eth.log_cids.receipt_id = receipt_cids.id
AND receipt_cids.tx_id = transaction_cids.id
Expand Down Expand Up @@ -375,7 +376,7 @@ func (ecr *CIDRetriever) RetrieveFilteredLog(tx *sqlx.Tx, rctFilter ReceiptFilte
return logCIDs, nil
}

// RetrieveRctCIDs retrieves and returns all of the rct cids at the provided blockheight or block hash that conform to the provided
// RetrieveRctCIDs retrieves and returns all the rct cIDs at the provided blockHeight or block hash that conform to the provided
// filter parameters and correspond to the provided tx ids
func (ecr *CIDRetriever) RetrieveRctCIDs(tx *sqlx.Tx, rctFilter ReceiptFilter, blockNumber int64, blockHash *common.Hash, trxIds []int64) ([]models.ReceiptModel, error) {
log.Debug("retrieving receipt cids for block ", blockNumber)
Expand Down
Loading