Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix getLog API to use log_cids table #92

Merged
merged 14 commits into from
Sep 16, 2021
1 change: 1 addition & 0 deletions db/migrations/00006_create_eth_transaction_cids_table.sql
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ CREATE TABLE eth.transaction_cids (
src VARCHAR(66) NOT NULL,
tx_data BYTEA,
tx_type BYTEA,
gas INTEGER NOT NULL,
Copy link
Collaborator

@i-norden i-norden Sep 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's call this gas_limit, it's more accurate to how things are defined in EIPs/specs and more explicit about what it is/how it is used.

UNIQUE (header_id, tx_hash)
);

Expand Down
1 change: 1 addition & 0 deletions db/migrations/00007_create_eth_receipt_cids_table.sql
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ CREATE TABLE eth.receipt_cids (
post_state VARCHAR(66),
post_status INTEGER,
log_root VARCHAR(66),
gas_used INTEGER NOT NULL,
UNIQUE (tx_id)
);

Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,6 @@ require (
github.com/vulcanize/ipfs-ethdb v0.0.4-0.20210824131459-7bb49801fc12
)

replace github.com/ethereum/go-ethereum v1.10.8 => github.com/vulcanize/go-ethereum v1.10.8-statediff-0.0.26
replace github.com/ethereum/go-ethereum v1.10.8 => /Users/arijitdas/go/src/github.com/ethereum/go-ethereum

replace github.com/vulcanize/ipfs-ethdb v0.0.2-alpha => github.com/vulcanize/pg-ipfs-ethdb v0.0.2-alpha
2 changes: 0 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -711,8 +711,6 @@ github.com/valyala/fasttemplate v1.0.1/go.mod h1:UQGH1tvbgY+Nz5t2n7tXsz52dQxojPU
github.com/valyala/fasttemplate v1.2.1/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ=
github.com/vulcanize/gap-filler v0.3.1 h1:N5d+jCJo/VTWFvBSbTD7biRhK/OqDZzi1tgA85SIBKs=
github.com/vulcanize/gap-filler v0.3.1/go.mod h1:qowG1cgshVpBqMokiWro/1xhh0uypw7oAu8FQ42JMy4=
github.com/vulcanize/go-ethereum v1.10.8-statediff-0.0.26 h1:1UBVQpeJnHkmSKxXanbNGE8w+LR0iZhfGr0QrQ62+C4=
github.com/vulcanize/go-ethereum v1.10.8-statediff-0.0.26/go.mod h1:nXs5fPBjAVzBmIGtrc0f7akQwkXI5Mi+6I1QcbD2br0=
github.com/vulcanize/ipfs-ethdb v0.0.4-0.20210824131459-7bb49801fc12 h1:IKqHA89qA+VZBYt1nZ1EInVrAgB3iA5U+klkF4l8mn4=
github.com/vulcanize/ipfs-ethdb v0.0.4-0.20210824131459-7bb49801fc12/go.mod h1:IueWysMbZu0uFmu+ia6mEnyWsTvwe2q2lbYdy2muRUM=
github.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1/go.mod h1:8UvriyWtv5Q5EOgjHaSseUEdkQfvwFv1I/In/O2M9gc=
Expand Down
10 changes: 8 additions & 2 deletions pkg/eth/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -524,12 +524,18 @@ func (pea *PublicEthAPI) localGetTransactionReceipt(ctx context.Context, hash co
"logsBloom": receipt.Bloom,
}

// Assign receipt status or post state.
if len(receipt.PostState) > 0 {
if blockNumber <= pea.B.Config.ChainConfig.ByzantiumBlock.Uint64() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A problem with this is that this deviates from the normal, expected behavior of this eth json-rpc endpoint. The regular endpoint only fills in the status if it is explicitly known: https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/api.go#L1650. So this breaks compatibility with the regular endpoint.

if receipt.GasUsed > tx.Gas() {
Copy link
Collaborator

@i-norden i-norden Sep 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm missing something. When will receipt.GasUsed ever be more than tx.Gas()? When a tx fails for whatever reason is it's receipt.GasUsed simply set to something above the tx.Gas() to indicate it failed? Seems odd but I'm not super familiar with this pre-byzantium behavior- wouldn't it need to track/report how much gas was actually used during the failed execution/reversion (because the tx sender still has to pay for whatever execution was performed prior to it reverting/failing)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an issue to track this #102

fields["status"] = hexutil.Uint(types.ReceiptStatusFailed)
} else {
fields["status"] = hexutil.Uint(types.ReceiptStatusSuccessful)
}
} else if len(receipt.PostState) > 0 {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still want to fill in the fields["root"] if receipt.PostState is present, whether or not we can also assume the status above.

fields["root"] = hexutil.Bytes(receipt.PostState)
} else {
fields["status"] = hexutil.Uint(receipt.Status)
}

if receipt.Logs == nil {
fields["logs"] = []*types.Log{}
}
Expand Down
73 changes: 53 additions & 20 deletions pkg/eth/api_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,15 +43,16 @@ import (
)

var (
randomAddr = common.HexToAddress("0x1C3ab14BBaD3D99F4203bd7a11aCB94882050E6f")
randomHash = crypto.Keccak256Hash(randomAddr.Bytes())
number = rpc.BlockNumber(test_helpers.BlockNumber.Int64())
londonBlockNum = rpc.BlockNumber(test_helpers.LondonBlockNum.Int64())
wrongNumber = number + 1
blockHash = test_helpers.MockBlock.Header().Hash()
baseFee = test_helpers.MockLondonBlock.BaseFee()
ctx = context.Background()
expectedBlock = map[string]interface{}{
randomAddr = common.HexToAddress("0x1C3ab14BBaD3D99F4203bd7a11aCB94882050E6f")
randomHash = crypto.Keccak256Hash(randomAddr.Bytes())
number = rpc.BlockNumber(test_helpers.BlockNumber.Int64())
londonBlockNum = rpc.BlockNumber(test_helpers.LondonBlockNum.Int64())
byzantiumBlockNum = rpc.BlockNumber(test_helpers.ByzantiumBlockNum.Int64())
wrongNumber = number + 1
blockHash = test_helpers.MockBlock.Header().Hash()
baseFee = test_helpers.MockLondonBlock.BaseFee()
ctx = context.Background()
expectedBlock = map[string]interface{}{
"number": (*hexutil.Big)(test_helpers.MockBlock.Number()),
"hash": test_helpers.MockBlock.Hash(),
"parentHash": test_helpers.MockBlock.ParentHash(),
Expand Down Expand Up @@ -131,14 +132,15 @@ var (
"receiptsRoot": test_helpers.MockUncles[1].ReceiptHash,
"uncles": []common.Hash{},
}
expectedTransaction = eth.NewRPCTransaction(test_helpers.MockTransactions[0], test_helpers.MockBlock.Hash(), test_helpers.MockBlock.NumberU64(), 0, test_helpers.MockBlock.BaseFee())
expectedTransaction2 = eth.NewRPCTransaction(test_helpers.MockTransactions[1], test_helpers.MockBlock.Hash(), test_helpers.MockBlock.NumberU64(), 1, test_helpers.MockBlock.BaseFee())
expectedTransaction3 = eth.NewRPCTransaction(test_helpers.MockTransactions[2], test_helpers.MockBlock.Hash(), test_helpers.MockBlock.NumberU64(), 2, test_helpers.MockBlock.BaseFee())
expectedLondonTransaction = eth.NewRPCTransaction(test_helpers.MockLondonTransactions[0], test_helpers.MockLondonBlock.Hash(), test_helpers.MockLondonBlock.NumberU64(), 0, test_helpers.MockLondonBlock.BaseFee())
expectRawTx, _ = rlp.EncodeToBytes(test_helpers.MockTransactions[0])
expectRawTx2, _ = rlp.EncodeToBytes(test_helpers.MockTransactions[1])
expectRawTx3, _ = rlp.EncodeToBytes(test_helpers.MockTransactions[2])
expectedReceipt = map[string]interface{}{
expectedTransaction = eth.NewRPCTransaction(test_helpers.MockTransactions[0], test_helpers.MockBlock.Hash(), test_helpers.MockBlock.NumberU64(), 0, test_helpers.MockBlock.BaseFee())
expectedTransaction2 = eth.NewRPCTransaction(test_helpers.MockTransactions[1], test_helpers.MockBlock.Hash(), test_helpers.MockBlock.NumberU64(), 1, test_helpers.MockBlock.BaseFee())
expectedTransaction3 = eth.NewRPCTransaction(test_helpers.MockTransactions[2], test_helpers.MockBlock.Hash(), test_helpers.MockBlock.NumberU64(), 2, test_helpers.MockBlock.BaseFee())
expectedLondonTransaction = eth.NewRPCTransaction(test_helpers.MockLondonTransactions[0], test_helpers.MockLondonBlock.Hash(), test_helpers.MockLondonBlock.NumberU64(), 0, test_helpers.MockLondonBlock.BaseFee())
expectedByzantiumTransaction = eth.NewRPCTransaction(test_helpers.MockByzantiumTransactions[0], test_helpers.MockByzantiumBlock.Hash(), test_helpers.MockByzantiumBlock.NumberU64(), 0, test_helpers.MockByzantiumBlock.BaseFee())
expectRawTx, _ = rlp.EncodeToBytes(test_helpers.MockTransactions[0])
expectRawTx2, _ = rlp.EncodeToBytes(test_helpers.MockTransactions[1])
expectRawTx3, _ = rlp.EncodeToBytes(test_helpers.MockTransactions[2])
expectedReceipt = map[string]interface{}{
"blockHash": blockHash,
"blockNumber": hexutil.Uint64(uint64(number.Int64())),
"transactionHash": expectedTransaction.Hash,
Expand Down Expand Up @@ -180,6 +182,20 @@ var (
"logsBloom": test_helpers.MockReceipts[2].Bloom,
"root": hexutil.Bytes(test_helpers.MockReceipts[2].PostState),
}
expectedByzantiumReceipt = map[string]interface{}{
"blockHash": test_helpers.MockByzantiumBlock.Hash(),
"blockNumber": hexutil.Uint64(uint64(byzantiumBlockNum)),
"transactionHash": expectedByzantiumTransaction.Hash,
"transactionIndex": hexutil.Uint64(0),
"from": expectedByzantiumTransaction.From,
"to": expectedByzantiumTransaction.To,
"gasUsed": hexutil.Uint64(test_helpers.MockByzantiumReceipts[0].GasUsed),
"cumulativeGasUsed": hexutil.Uint64(test_helpers.MockByzantiumReceipts[0].CumulativeGasUsed),
"contractAddress": nil,
"logs": test_helpers.MockByzantiumReceipts[0].Logs,
"logsBloom": test_helpers.MockByzantiumReceipts[0].Bloom,
"status": hexutil.Uint(test_helpers.MockByzantiumReceipts[0].Status),
}
)

// SetupDB is use to setup a db for watcher tests
Expand Down Expand Up @@ -210,6 +226,13 @@ var _ = Describe("API", func() {

db, err = SetupDB()
Expect(err).ToNot(HaveOccurred())

// setting chain config to for byzantium block
chainConfig.ByzantiumBlock = big.NewInt(1)

// setting chain config to for london block
chainConfig.LondonBlock = big.NewInt(3)

indexAndPublisher := indexer.NewStateDiffIndexer(chainConfig, db)
backend, err := eth.NewEthBackend(db, &eth.Config{
ChainConfig: chainConfig,
Expand Down Expand Up @@ -249,14 +272,17 @@ var _ = Describe("API", func() {
}
expectedBlock["uncles"] = uncleHashes

// setting chain config to for london block
chainConfig.LondonBlock = big.NewInt(2)
indexAndPublisher = indexer.NewStateDiffIndexer(chainConfig, db)
tx, err = indexAndPublisher.PushBlock(test_helpers.MockLondonBlock, test_helpers.MockLondonReceipts, test_helpers.MockLondonBlock.Difficulty())
Expect(err).ToNot(HaveOccurred())

err = tx.Close(err)
Expect(err).ToNot(HaveOccurred())

tx, err = indexAndPublisher.PushBlock(test_helpers.MockByzantiumBlock, test_helpers.MockByzantiumReceipts, test_helpers.MockByzantiumBlock.Difficulty())
Expect(err).ToNot(HaveOccurred())

err = tx.Close(err)
Expect(err).ToNot(HaveOccurred())
})

// Single test db tear down at end of all tests
Expand Down Expand Up @@ -644,6 +670,13 @@ var _ = Describe("API", func() {
_, err := api.GetTransactionReceipt(ctx, randomHash)
Expect(err).To(HaveOccurred())
})

It("Retrieve receipt for pre byzantium block transaction", func() {
hash := test_helpers.MockByzantiumTransactions[0].Hash()
tx, err := api.GetTransactionReceipt(ctx, hash)
Expect(err).ToNot(HaveOccurred())
Expect(tx).To(Equal(expectedByzantiumReceipt))
})
})

Describe("eth_getLogs", func() {
Expand Down
63 changes: 63 additions & 0 deletions pkg/eth/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import (
"errors"
"fmt"
"math/big"
"strconv"

"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/hexutil"
Expand Down Expand Up @@ -472,6 +473,7 @@ func (b *Backend) GetTransaction(ctx context.Context, txHash common.Hash) (*type
BlockHash string `db:"block_hash"`
BlockNumber uint64 `db:"block_number"`
Index uint64 `db:"index"`
Gas uint64 `db:"gas"`
}
if err := b.DB.Get(&tempTxStruct, RetrieveRPCTransaction, txHash.String()); err != nil {
return nil, common.Hash{}, 0, 0, err
Expand Down Expand Up @@ -799,6 +801,67 @@ func (b *Backend) RPCGasCap() *big.Int {
return b.Config.RPCGasCap
}

type logsCID struct {
Log *types.Log
CID string
RctCID string
LogLeafData []byte
RctStatus uint64
}

// DecomposeGQLLogs return logs for graphql.
func (b *Backend) DecomposeGQLLogs(logCIDs []LogResult) ([]logsCID, error) {
logs := make([]logsCID, len(logCIDs))
for i, l := range logCIDs {
topics := make([]common.Hash, 0)
if l.Topic0 != "" {
topics = append(topics, common.HexToHash(l.Topic0))
}
if l.Topic1 != "" {
topics = append(topics, common.HexToHash(l.Topic1))
}
if l.Topic2 != "" {
topics = append(topics, common.HexToHash(l.Topic2))
}
if l.Topic3 != "" {
topics = append(topics, common.HexToHash(l.Topic3))
}

// TODO: should we convert string to uint ?
blockNum, err := strconv.ParseUint(l.BlockNumber, 10, 64)
if err != nil {
return nil, err
}

var status uint64
if blockNum <= b.Config.ChainConfig.ByzantiumBlock.Uint64() {
if l.RctGasUsed > l.Gas {
status = types.ReceiptStatusFailed
} else {
status = types.ReceiptStatusSuccessful
}
} else {
status = l.RctStatus
}

logs[i] = logsCID{
Log: &types.Log{
Address: common.HexToAddress(l.Address),
Topics: topics,
Data: l.Data,
Index: uint(l.Index),
TxHash: common.HexToHash(l.TxHash),
},
CID: l.LeafCID,
RctCID: l.RctCID,
LogLeafData: l.LogLeafData,
RctStatus: status,
}
}

return logs, nil
}

func (b *Backend) SubscribeNewTxsEvent(chan<- core.NewTxsEvent) event.Subscription {
panic("implement me")
}
Expand Down
5 changes: 3 additions & 2 deletions pkg/eth/cid_retriever.go
Original file line number Diff line number Diff line change
Expand Up @@ -316,7 +316,8 @@ func (ecr *CIDRetriever) RetrieveFilteredGQLLogs(tx *sqlx.Tx, rctFilter ReceiptF
id := 1
pgStr := `SELECT eth.log_cids.leaf_cid, eth.log_cids.index, eth.log_cids.receipt_id,
eth.log_cids.address, eth.log_cids.topic0, eth.log_cids.topic1, eth.log_cids.topic2, eth.log_cids.topic3,
eth.log_cids.log_data, eth.transaction_cids.tx_hash, data, eth.receipt_cids.cid, eth.receipt_cids.post_status
eth.log_cids.log_data, eth.transaction_cids.tx_hash, eth.transaction_cids.gas, data, eth.receipt_cids.cid, eth.receipt_cids.post_status,
eth.receipt_cids.gas_used,header_cids.block_number
FROM eth.log_cids, eth.receipt_cids, eth.transaction_cids, eth.header_cids, public.blocks
WHERE eth.log_cids.receipt_id = receipt_cids.id
AND receipt_cids.tx_id = transaction_cids.id
Expand Down Expand Up @@ -375,7 +376,7 @@ func (ecr *CIDRetriever) RetrieveFilteredLog(tx *sqlx.Tx, rctFilter ReceiptFilte
return logCIDs, nil
}

// RetrieveRctCIDs retrieves and returns all of the rct cids at the provided blockheight or block hash that conform to the provided
// RetrieveRctCIDs retrieves and returns all the rct cIDs at the provided blockHeight or block hash that conform to the provided
// filter parameters and correspond to the provided tx ids
func (ecr *CIDRetriever) RetrieveRctCIDs(tx *sqlx.Tx, rctFilter ReceiptFilter, blockNumber int64, blockHash *common.Hash, trxIds []int64) ([]models.ReceiptModel, error) {
log.Debug("retrieving receipt cids for block ", blockNumber)
Expand Down
Loading