Skip to content

Latest commit

 

History

History
909 lines (580 loc) · 49.5 KB

release-notes.md

File metadata and controls

909 lines (580 loc) · 49.5 KB
Raw
description
Monthly release notes for the Spice.ai Cloud Platform

Release Notes

June 2024

The June release included a C# SDK for Spice AI and Spice OSS to allow for development using .Net Standard 2.0 and .Net 6.0 or greater.

Changes

  1. [Dotnet SDK] Released v0.1.0
  2. [Platform] Performance, reliability, and security improvements.

May 2024

The May release included EigenLayer Delegation Manager datasets. The datasets can be explored in the portal Playground and queried via SQL.

Changes

  1. [Data] SQL Datasets for Eigenlayer now include Delegation Manager datasets.
  2. [Platform] Performance, reliability, and security improvements.

April 2024

The April release included updates to the Spice SDK's to support querying a local Spice Runtime without requiring a Spice.ai API key. Details and examples can be found in the Rust, Go, Python, and Node.js SDK documentation.

Ethereum Holesky nodes were shut down and datasets were removed on Thursday, April 25th at 8pm PST.

Holesky JSON RPC and Beacon API's requests to /holesky will return a 404 Not Found response and Holesky SQL datasets are no longer available.

Changes

  1. [Rust SDK] Released v2.0.0
  2. [Node.js SDK] Released v2.0.0
  3. [Go SDK] Released v6.0
  4. [Python SDK] Released v2.0
  5. [API] Removed Holesky JSON-RPC API
  6. [Data] Removed Holesky datasets
  7. [Platform] Performance, reliability, and security improvements.

March 2024

Ethereum Goerli nodes were shut down on Friday March 22 at 5pm PST.

The Ethereum Foundation planned to stop supporting Goerli one month after Dencun was activated on Ethereum mainnet, which would be April 13th, however nearly all Goerli Validators were already shut down, effectively shutting the network down prematurely. Spice AI is shutting down nodes early, aligned to other providers decisions to also shut down, such as Alchemy.

After the shutdown, JSON RPC and Beacon requests to /goerli returned 404 Not Found and Goerli SQL datasets, while still available for query, are no longer being updated, including Firecache datasets.

Goerli SQL Tables will remain available for historical query purposes.

The Ethereum and Beacon datasets now include fields from the Cancun and Dencun network upgrade.

Changes

  1. [Rust SDK] Released v1.0.3
  2. [API] Removed Goerli JSON-RPC API
  3. [Data] SQL datasets for Ethereum and Beacon chains now include Cancun and Dencun fields.

February 2024

The February release focused on stability and performance improvements.

Changes

  1. [Platform] Performance, reliability, and security improvements.

January 2024

The January release added support for the Ethereum Holesky testnet in preview. Holesky data can be explored in the portal Playground, queried by SQL and Firecache, and made available to applications by JSON-RPC API.

Holesky Testnet APIs and datasets available in preview

New in this release

  1. [Platform] Added Ethereum Holesky testnet data and API's.
  2. [Platform] Achieved SOC 2 Type II compliance.

Changes

  1. [Platform] Performance, reliability, and security improvements.
  2. [Node JS SDK] Released v1.0.2.

December 2023

The December release focused on stability and performance improvements.

Changes

  1. [Platform] Performance, reliability, and security improvements.

November 2023

The November release focused on External Data Sources and overall performance improvements following the General Availability (GA) release in October.

You can now connect external data sources like PostgreSQL and MySQL to Spice.ai, which are then available to query with SQL including joining with any other Spice.ai dataset in the Spice.ai platform.

New Data Source dialog for connecting external data sources.

Navigation is also easier with the addition of the Command Bar. Simply use Cmd+K on Mac or Windows+K in Windows to quickly navigate the portal by keyboard! Give it a try!

The new navigation command bar.

New In this Release

  1. [Portal] Added PostgreSQL and MySQL external data sources. See the docs for details.
  2. [Portal] Added Dark Mode! Configure via Profile Settings.
  3. [Portal] Added Command Bar navigation.
  4. [Data] Added eth.wallet_lst_balances including rETH, stETH, and cbETH LSTs.
  5. [Firecache] Upgraded to DuckDB 0.92.
  6. [Platform] Performance, reliability, and security improvements.
  7. [Go SDK] Released v3.0.0, learn more.

October 2023

Spice.ai is now generally available! Read the announcement.

General Availability launches a brand-new collaborative developer experience, v1 APIs and SDKs, over 100 open-source community and EigenLayer datasets, and a removal of the waitlist. Any developer can now sign-up and use Spice.ai with one of the new pricing plans, including the $99 developer plan and the complementary Community Edition.

New In this Release

  1. [HTTP API] Add v1 API:
    1. SQL Query API
    2. Prices API
    3. Gas Fees API
    4. Predictions API
  2. [Portal] New Portal and developer experience, without waitlist limitations.
  3. [Portal] New pricing plans are available, learn more.
  4. [Portal] Datasets and Views generally available, learn more.
  5. [Firecache] Upgraded to DuckDB 0.91.
  6. [Functions] Upgraded to DuckDB 0.91.
  7. [Ethereum] Added support for eth_maxPriorityFeePerGas and trace_filter to JSON RPC API.
  8. [Ethereum] Expanded Ethereum client diversity by adding Reth support.
  9. [Ethereum] Expanded Polygon client diversity by adding Erigon support.
  10. [Python SDK] v1.0.0 released, learn more.
  11. [Go SDK] v3.0.0 released, learn more.
  12. [Node.js SDK] v1.0.0 released, learn more.
  13. [Rust SDK] v1.0.1 released, learn more.

September 2023

In the September release, Crypto/Token Prices data has been significantly expanded and improved upon.

Historical and latest prices are available via REST API and SQL Query.

The number of token pairs supported by the REST API has greatly increased. In addition, if a pair does not have swap data, the platform will attempt to calculate a synthetic price for the pair, routed via swaps across different exchanges. For example, if a price for the pair BAO-AUD does not exist, but the pairs BAO-USDT and USDT-AUD do, then the API will calculate the routed price across the two swaps to determine a synthetic price, which will be returned in the payload as the spiceai price. The API will use data cross both centralized and decentralized exchanges. This enables pricing data where it was not possible before.

Additional tables have been added for token prices and assets.

  • prices.assets list all assets tracked and available via SQL Query. 8,856 assets are currently available.
  • prices.all_pairs list all the token pairs with prices available via SQL Query. 3,342 pairs are currently available.

In addition to the existing prices.[tokenA-tokenB] a table of prices for each token pair.

Note: In both cases, the list is a subset of what is available via the REST API.

New In this Release

  1. [SQL Query] Added prices.assets table.
  2. [SQL Query] Added prices.all_pairs table.
  3. [Prices API] Added swap routed token pricing.
  4. [Prices API] Enhanced coverage of token pairs.
  5. [Portal] Bugfixes and minor improvements to the UI.

August 2023

The August release expands the functionality of Spice Functions (beta) and Datasets and Views.

Spice Functions

{% hint style="info" %} Spice Functions is in beta and initially available for Design Partners. Get in touch for more info. {% endhint %}

Developers can now author Spice Functions in Golang with the Go1.x runtime in addition to Python.

Combine Spice Functions with expanded Datasets and Views support to generate custom datasets "ETL-less" in the Spice platform that can be kept private or shared with others.

Datasets and Views

Spice now supports SQL migrations to enable creation of custom user-authored datasets and views.

New In this Release

  1. [Spice Functions] Add Go1.x runtime.
  2. [Datasets and Views] Add SQL migrations.
  3. [Datasets and Views] Add sharing and publishing.
  4. [Portal] Add basic charting for SQL and Firecache results.
  5. [Portal] Bugfixes and minor improvements to the UI.

July 2023

The July release enables new compute over data capability with Spice Functions including application data storage, the ability to define and create custom datasets, and deep integration with GitHub.

Spice Functions

{% hint style="info" %} Spice Functions is in beta and initially available for Design Partners. Get in touch for more info. {% endhint %}

Spice Functions is a hosted compute experience that enables developers to write code in their preferred language and run it on the Spice platform, co-located with Spice data.

See Broken link for full documentation on creating and deploying Spice Functions.

Spice Firecache

{% hint style="info" %} Spice Firecache is in beta and initially available for Design Partners. Get in touch for more info. {% endhint %}

Spice Firecache is a real-time SQL cache that enables developers to cache Spice datasets for high concurrency, blazing fast SQL query up to 10x the performance of general SQL query.

The Spice platform completely manages the Spice Firecache data lifecycle, ensuring datasets are real-time updated on triggers like new blocks and with stale data being automatically evicted.

See spice-firecache for full documentation on deploying datasets to the Spice Firecache.

Custom Datasets

Spice now supports the definition and creation of your own private Datasets and Views, which can then be queried with SQL, cached in Spice Firecache, and published publicly to be shared with others.

See datasets-and-views.md for full documentation on creating private Datasets and Views.

Connecting Github Repos

Connecting your Spice.xyz app to a GitHub repository allows you to create custom Datasets and Views, add datasets to Spice Firecache, author Spice Functions, and train AI Models.

See connect-github-repository.md for full documentation on connecting your Github Repo to a Spice App.

New In this Release

  1. [Platform/Portal] Support for compute over data with Spice Functions
  2. [Platform/Portal] Real-time SQL caching through Spice Firecache
  3. [Platform/Portal] Support for custom datasets and views beyond those curated by Spice
  4. [Portal] Connect your Github repo to manage all your Spice code with version control
  5. [Flow] RPCs can now be sent to the Spice Flow node using the Flow Access API
  6. [Flow] Added datasets detailing transfers of NFTs and tokens on the Flow blockchain, for more details see Broken link

Changes

  1. [Portal] Any users you add to your org that is on a Pro or Partner plan will now automatically be activated and removed from the Spice.xyz waitlist.

June 2023

The June release includes new datasets for Ethereum/Goerli wallet balances + Flow core/nft datasets, additional platform performance upgrades along with several usability and user-experience improvements.

Wallet Balance Datasets

The wallet balances datasets contains a block-level view of all Ether balance changes due to gas fees, withdrawals, and committed external and internal (contract) transactions.

{% hint style="info" %} Wallet Balances datasets do not yet have full historical data. {% endhint %}

Flow Datasets

Flow is a fast, decentralized, and developer-friendly blockchain, designed as the foundation for a new generation of games, apps, and the digital assets that power them. Supercharge your Flow apps with these datasets indexed by Spice.xyz.

{% hint style="info" %} Flow support is in preview. Contact us to let us know if Flow support is important to you! {% endhint %}

Portal

Portal UX improvements include the ability to search the dataset reference for specific datasets and inserting a sample query with a click. GitHub avatars now show up in the organization drop-down, as well as a distinction between your personal account and other organizations you are part of. Searching for members to add to an organization now properly autocompletes.

{% embed url="https://youtu.be/2qk8ZMZOJek" %} Spice.xyz SQL explorer dataset reference improvements {% endembed %}

New In this Release

  1. [Wallet Balances] Added wallet balance datasets for Ethereum and Goerli which track changes to wallet balances on every block.
  2. [Flow] Added Flow datasets, including core and enriched NFT/token datasets.
  3. [Portal] Show GitHub avatars in the organization drop-down.
  4. [Portal] The organization drop-down now distinguishes between your personal account and other organizations you are part of.

Changes

  1. [Portal] You can now search for datasets in the dataset reference.
  2. [Portal] Sample queries can be injected into the query editor by clicking in the dataset reference.
  3. [Portal] Improved error messages in the SQL editor for syntax errors.
  4. [Portal] The dialog box used for adding organization members now auto-completes for potential members to add.
  5. [Performance] Improved performance for several datasets, including eth.beacon.validators and goerli.beacon.validators.

May 2023

The May release includes significant platform performance upgrades with the rollout of Spice AI's second generation platform architecture (Gen. 2) along with several usability and user-experience improvements.

Ethereum and Polygon block latency (the time to process a block and make it available for query) is now subsecond for core data like blocks, transactions, and logs, and less than 2.5 seconds for enriched data like NFTs, ENS, Uniswap, and Sushiswap. Query execution speed is now 1.5x-10x faster!

Example Queries referenced in docs are now 1.5x-10x faster!

Portal UX improvements include the ability to collapse the sidebar, resize the SQL query input and results pane, and expand to a fullscreen focus mode. SQL query errors are now highlighted with red underline swiggles along with a more readable error pane. Finally, results render faster and app switching is faster.

{% embed url="https://youtu.be/ZJNt3paE7hA" %} Spice.xyz SQL explorer fullscreen mode {% endembed %}

New In this Release

  1. [Platform] Spice AI's Second-Generation Platform Architecture (Gen. 2) resulting in 30-50% data latency improvements, 1.5x-10x faster queries, with a target availability of 99.9%.
  2. [Portal] SQL query editor contextual error reporting.
  3. [Portal] SQL query explorer fullscreen mode.
  4. [Portal] SQL query editor and results pane resizability.
  5. [Portal] Collapsable sidebar.
  6. [Portal] Page loading progress bar.

Changes

  1. [Portal] The portal now uses a GitHub App to integrate with GitHub. This will result in a request for re-authorization on the next sign-in.
  2. [Go SDK] gospice v0.2.1 released with Prices API and Apache Arrow v12 support.
  3. [Node.js SDK] spice.js v0.3.2 released with a fix to re-support Node 16.

April 2023

This month we added new datasets including Chainlink prices data feeds, Aave v2 contract data, Ethereum Shanghai/Capella beacon and withdrawals data, and we now fully index the Goerli testnet. We also released the Goerli Beacon and Ethereum Beacon APIs.

Chainlink Data Feeds: You can now leverage oracles for various data feed on the Chainlink network by querying prices specific tables including eth.chainlink.prices and eth.chainlink.recent_prices.

This includes accessing the latest, minute-precision data on transaction_hash, oracle_address, block_timestamp, price of the asset in WEI, asset_address and more. The columns and their schema available for each table can be viewed with the describe <table> command:

/* Show the columns available */
DESCRIBE eth.chainlink.prices;

Data for for Curve, Balancer, Aave, Yearn, DAI, Tether USD and 30 other top assets is available. See the full list of supported data feeds here.


Aave V2 Dataset: In addition to the Aave token prices data feed on the Chainlink network, we have released indexed Aave V2 data for loans, collaterals and prices. You can query for the latest updates to Aave loans and collateral by querying eth.aave_v2.loan_updates and eth.aave_v2.collateral_updates.

Latest view of all Aave V2 loans query, sample output


Ethereum Shanghai/Capella Upgrade and Withdrawals Datasets: The Ethereum Shanghai/Capella upgrade is now full supported after its completion on April 12, 2023, 10:27:35pm UTC at epoch 194048.

The completion of Ethereum's Shanghai hard fork, also known as "Shapella," has authorized the withdrawal of funds for individuals who have "staked" their ETH to validate and secure blockchain transactions.

We also released the eth.withdrawals dataset to capture new withdrawals. For example, you can now query to get ETH withdrawn:

with eth_withdrawn as (
  select eth.withdrawals.block_number, SUM(amount) as amount_gwei
  from eth.withdrawals
  group by eth.withdrawals.block_number
  order by block_number asc)

select AVG(amount_gwei / 1e9) as amount_eth
from eth_withdrawn

Get ETH withdrawn after the Ethereum Shanghai upgrade, sample output

Explore detailed sample SQL queries for DEX, Cross-Chain, Ethereum, Beacon, and Bitcoin in the Spice Docs.\

Goerli Testnet: Full support for the Goerli testnet has been added, including hosted beacon and execution nodes, and fully indexed datasets, helping developers test their applications before mainnet launch.

Available with realtime and historical data on Spice.xyz, this includes Goerli base type tables, Goerli token specific tables, and Goerli Beacon Chain specific tables. You can start by querying goerli.traces, goerli.blocks or a number of sample beacon queries.

For example, you can query from goerli.beacon.validators to retrieve recently updated validators, with their balance and status:

Recently updated validators, with their balance and status by querying from goerli.beacon.validators, sample output

Goerli Beacon API: You can easily retrieve requests like "get validator balances from state" using the https://data.spiceai.io/goerli/beacon API, which is compatible with the read-only GET requests from the Beacon Node API specification.

All Goerli testnet data APIs are available at /goerli/v0.1 including the following:

Ethereum Beacon API: The Ethereum API is now updated to support Beacon requests via the https://data.spiceai.io/eth/beacon API. This is compatible with the read-only GET requests from the Beacon Node API specification.

See the full list of Beacon API requests here.

New In this Release

  • [Goerli] added goerli.blocks, goerli.recent_blocks
  • [Goerli] added goerli.transactions, goerli.recent_transactions
  • [Goerli] added goerli.logs, goerli.recent_logs
  • [Goerli] added goerli.traces, goerli.recent_traces
  • [Goerli] added goerli.withdrawals, goerli.recent_withdrawals
  • [Goerli] added goerli.contracts
  • [Goerli] added goerli.tokens
  • [Goerli] added goerli.token_transfers
  • [Goerli] added goerli.tokens_erc20
  • [Goerli] added goerli.token_transfers_erc20
  • [Goerli] added goerli.tokens_erc721
  • [Goerli] added goerli.token_transfers_erc721
  • [Goerli] added goerli.tokens_erc1155
  • [Goerli] added goerli.token_transfers_erc1155
  • [Goerli] added goerli.token_mints
  • [Goerli] added goerli.recent_token_mints
  • [Goerli] added goerli.beacon.slots, goerli.beacon.recent_slots
  • [Goerli] added goerli.beacon.voluntary_exits, goerli.beacon.recent_voluntary_exits
  • [Goerli] added goerli.beacon.attestations, goerli.beacon.recent_attestations
  • [Goerli] added goerli.beacon.attester_slashings, goerli.beacon.recent_attester_slashings
  • [Goerli] added goerli.beacon.deposits, goerli.beacon.recent_deposits
  • [Goerli] added goerli.beacon.proposer_slashings, goerli.beacon.recent_proposer_slashings
  • [Goerli] added goerli.beacon.bls_to_execution_changes, goerli.beacon.recent_bls_to_execution_changes
  • [Goerli] added goerli.beacon.withdrawals, goerli.beacon.recent_withdrawals
  • [Beacon] added Goerli Beacon HTTP API
  • [Beacon] added Ethereum Beacon HTTP API
  • [Aave V2] added eth.aave_v2.loan_updates, eth.aave_v2.loans
  • [Aave V2] added eth.aave_v2.collateral_updates, eth.aave_v2.collateral
  • [Chainlink] added eth.chainlink.prices, eth.chainlink.recent_prices
  • [Chainlink] added price feed support for the following assets:
    • USD Coin (USDC)
    • 1INCH Token (1INCH)
    • Yearn.finance (YFI)
    • Dai Stablecoin (DAI)
    • DefiPulse Index (DPI)
    • UST (UST)
    • Decentraland MANA (MANA)
    • Liquid staked Ether 2.0 (stETH)
    • ChainLink Token (LINK)
    • Balancer (BAL)
    • Enjin Coin (ENJ)
    • Tether USD (USDT)
    • Wrapped BTC (WBTC)
    • Maker (MKR)
    • Convex Token (CVX)
    • Fei USD (FEI)
    • Uniswap (UNI)
    • Pax Dollar (USDP)
    • Kyber Network Crystal (KNC)
    • Aave Token (AAVE)
    • 0x Protocol Token (ZRX)
    • Binance USD (BUSD)
    • SushiBar (xSUSHI)
    • Republic Token (REN)
    • renFIL (renFIL)
    • Basic Attention Token (BAT)
    • Synthetix Network Token (SNX)
    • Ampleforth (AMPL)
    • Frax (FRAX)
    • Rai Reflex Index (RAI)
    • Synth sUSD (sUSD)
    • TrueUSD (TUSD) \

Changes

  • [Portal] Improved performance of portal pages
  • [Performance] Improved query performance, data redundancy, and scale by migrating to new infrastructure

Resources

March 2023

If you are waitlisted, you can now instantly enable your account by signing up for a preview plan at 50% off. This month's release also includes support for the Ethereum Beacon chain dataset and improved monitoring with request logs.

Self-Service Activation from Waitlist: as a waitlisted customer, you can now enable your account instantly without manual Spice AI approval when you sign up for a Pro for Teams or Lite Plan. Begin a 7-day trial of the Pro for Team plan, then monthly subscription of $1,245 at 50% off (normally $2,490). It comes with early access (off the waitlist), dedicated support on Discord, and a bi-weekly sync with the Spice AI engineering team.

Improved monitoring: you can now track requests, their status code, and duration to Spice, in addition to the existing usage monitoring metrics dashboard. Start by going to your app, under the Monitor sidebar, select API Requests. You can then toggle between Metrics and Logs views. Within Logs, you can select to retrieve API requests from the past 1 hour, 8 hours, 24 hours, and up to the past 3 days.

Ethereum Beacon Chain Dataset (Preview): The Ethereum beacon chain is the central coordination and consensus layer for the Ethereum 2.0 upgrade, which aims to improve scalability, security, and sustainability. Now you can query and access beacon chain dataset with eth.beacon, eth.beacon.validators, eth.beacon.withdrawals and more.

You can also reference the full query list via Ethereum Beacon tables or by directly querying the following in the application:

SHOW TABLES IN eth.beacon

Example SQL queries include slots, attestations, validators, deposits, voluntary exits, attester slashings and more, all retrievable within a few seconds.

Sample Queries

  1. ENS Domain Names that map to a validator\

  2. Number of Weekly Attester Slashings \

  3. Top 10 most frequent block proposers\

By leveraging beacon chain capabilities, developers can create scalable decentralized applications that can handle a large volume of transactions, while also participating in network governance and incentivization through its Proof of Stake consensus mechanism. You can learn more about the chain here.

New In this Release

  • [Portal] Added Self-Service account activation
  • [Beacon] Added Ethereum beacon chain dataset eth.beacon
  • Beacon] Added Ethereum beacon chain dataset eth.validators
  • [Beacon] Added Ethereum beacon chain datasets eth.beacon.slots, eth.beacon.recent_slots
  • [Beacon] Added Ethereum beacon chain datasets eth.beacon.attestations, eth.beacon.recent_attestations
  • [Beacon] Added Ethereum beacon chain datasets eth.beacon.deposits, eth.beacon.recent_deposits
  • [Beacon] Added Ethereum beacon chain datasets eth.beacon.voluntary_exits, eth.beacon.recent_voluntary_exits
  • [Beacon] Added Ethereum beacon chain datasets eth.beacon.attester_slashings, eth.beacon.recent_attester_slashings
  • [Beacon] Added Ethereum beacon chain datasets eth.beacon.proposer_slashings, eth.beacon.recent_proposer_slashings
  • [Beacon] Added Ethereum beacon chain datasets eth.beacon.bls_to_execution_changes, eth.beacon.recent_bls_to_execution_changes
  • [Beacon] Added Ethereum beacon chain datasets eth.beacon.withdrawals, eth.beacon.recent_withdrawals

Changes

  • [Portal] Improved performance of portal pages

Resources

February 2023

Spice released the NFT dataset with Metadata in preview, along with the Historic Prices API in preview, and the v0.3 update to the Node.js SDK spice.js.

Historic Prices API (Preview): In addition to accessing the Spot Prices of popular token/currency pairs on top exchanges, users can now access the /v0.1/prices/[pair] API, which returns historical prices of the specified token/currency pair. You can refine your search by defining parameters like start and end, which will return all prices between the two timestamps at a provided granularity.

Please contact us if you would like to access the API.

NFT Dataset with Metadata (Preview): You can now query and leverage fungible and non-fungible token contracts under eth.tokens as well as isolated NFT specific per token information under eth.nfts. These are made available also with name and symbol for ERC721 tokens. See the full list of NFT tables and Token tables.

Node.js SDK v0.3: adds support for improved Prices APIs supporting spot/latest and historical prices for pair across Coinbase, Gemini, and tracked by Coinmarket Cap. This upgrade ensures the SDK remains the easiest way to use and query Spice.xyz with Node.js.

It uses Apache Apache Flight to efficiently stream data to the client and Apache Arrow Records as data frames which are then easily converted to JavaScript objects/arrays or JSON.

New functions available on the client are:

  • getPrice()
  • getPrices()

You'll need Node.js 16+ to get started. See the step-by-step installation guide here.

New In this Release

Changes

  • [Performance] Improved query performance and scale
  • [Portal] Improved performance of portal pages
  • [Node.js SDK] Add getPrice and getPrices API

Resources

January 2023

Sharing a packed release this month focused on a new dataset, org management, two developer querying tools, and two SDKs.

Ethereum Name Service (ENS) Events: now out of preview, it's available by querying ens.domains, which is updated in real-time to have the latest eth_addresswhenever a transfer or name-registered event happens. Both eth.ens and other on-demand events indexed under the eth.ens prefix such as eth.ens_event_nameregistered are updated in real time with the same high-performance standards of published Spice datasets. See the tables here.

Organization Management: existing users can start exploring org support and management for users, apps, and datasets by authorizing Spice with read-only access when logging into the Spice.xyz portal. This will also act as a parent container for future concepts including apps, users, teams, datasets, and metrics for streamlined management and navigation within spice.xyz.

The big game changers this month are Async Query and Flow AI Query Auto-Complete (in preview) — which supercharge the developer experience.

Async Query API: you can post a JSON payload specifying list of query completion, and notifications results can be fetched asynchronously once the query has completed. Choose function client apps such as AWS Lambda to async query Spice.xyz and receive result notifications by webhook. Results are accessible up to 20 minutes after they are posted.

Flow (AI SQL Query Auto-Complete): you can now query with plain text, hit enter, and watch as Flow autocompletes the request with the corresponding SQL lines in your spice.xyz query editor. Hit tab to accept the AI-generated completion, then accept, reject, or edit before run the query.

We've also released the new gospice Golang SDK and added improvements to our Python SDK including timeout support.

Go SDK: gospice is the easiest way to query Spice.xyz from Go. It uses Apache Apache Flight to efficiently stream data to the client and Apache Arrow Records as data frames.
GoDocs available at pkg.go.dev/github.com/spiceai/gospice.

We continue to make performance improvements for faster, more reliable queries and also recently launched our much anticipated Polygon Support, which you can read more about here.

Read the announcement post at blog.spice.ai

New In this Release

Changes

  • [Performance] Improved query performance and scale
  • [Portal] Improved performance of portal pages
  • [Polygon] Added WebSocket support for Polygon
  • [Python SDK] Added support for Python 3.11
  • [Python SDK] Added back support for Python 3.7
  • [Python SDK] Added query timeout support
  • [Python SDK] Upgraded to pyarrow 10.0.1
  • [Node.js SDK] Added Async query support
  • [Node.js SDK] Upgraded to arrow 10.0.1

Resources

December 2022

Spice supports querying Polygon data in addition to the existing Ethereum and BItcoin datasets.

Read more on the Spice AI Blog.

Changes

  • Support for fundamental Polygon datasets and NFT specific tables
    • polygon.blocks / polygon.recent_blocks
    • polygon.transactions / polygon.recent_transactions
    • polygon.nft_transfers /polygon.recent_nft_transfers
    • polygon.nft_airdrop_transfers / polygon.recent_nft_airdrop_transfers
    • polygon.nft_owners
  • View the columns and their schema available for each table with the describe <table> command

Resources

November 2022

DEX Liquidity & Events Dataset

Spice now features DEX Liquidity and Events Dataset where users can access indexed liquidity data from UniSwap [V2, V3] and SushiSwap liquidity pools, within seconds.

Data is available both historically and in real time, providing unparalleled visibility on DEX liquidity pools.

ENS (Preview)
You can now preview the new dataset for Ethereum Name Service events, available by querying eth.ens. Within the dataset, you’ll find that each ENS event will have its own table as immutable data.

This also includes the new ens.domains dataset, which is updated in real-time to have the latest eth_address whenever a transfer or name-registered event happens

Read more on the Spice AI blog.

Changes

  • Prices dataset now out of preview
  • Added export to CSV results format in addition to existing formats JSON and Apache Arrow, for easy use with apps, ML, or libraries like NumPy and Pandas
  • Added performance improvements for faster queries across existing datasets including eth.traces

Resources

October 2022

Spice now has prices dataset in preview.

  • Query historical High-Low-Open-Close (HILO) prices to minute precision
  • Join with other real-time and historical web3 data to do calculations and conversions back to USD
  • Initial data sources include Gemini, Coinbase, SushiSwap, and UniSwap (a mix of CEX, DEX, and third parties)

Read more on the Spice AI blog.

Resources

September 2022

Ethereum Traces Dataset

Spice launches the eth.traces dataset for both Ethereum and Polygon. The Ethereum internal transactions are available for query at eth.traces and are updated in real-time.

Polygon Support (Preview)

Spice launches and tests Polygon support.

Changes

  • completed real-time indexing and parquet writing for Polygon
  • incorporated asynchronous process block notifications to enable Polygon support
  • added improvements on data integrity and bug fixes
  • added transaction_hash and block_timestamp to eth.nft_owners

Resources

August 2022

Key infrastructure improvements were made for Spice's query performance, security, and data availability.

Changes

  • Improved performance for node configuration
  • Upgrade dremio infrastructure
  • Expanded node memory for broader availability

Resources

July 2022

Spice supports querying Bitcoin data in addition to the existing Ethereum datasets.

Read more on the Spice AI blog.

Changes

  • Support for fundamental Bitcoin datasets
    • btc.blocks / btc.recent_blocks
    • btc.transactions / btc.recent_transactions
    • btc.transaction_inputs / btc.recent_transaction_inputs
    • btc.transaction_outputs / btc.recent_transaction_outputs
  • example-bitcoin-queries for querying Bitcoin data.

Resources

June 2022

This update includes a new Javascript/Typescript SDK for Node.js, an improved Python SDK, and performance improvements for Ethereum tokens and NFT datasets.

Ethereum Name Service (ENS) support is also available in beta with the new ens.domains dataset.

Read more on the Spice AI blog.

Changes

  • New Node.js SDK
  • Improved Python SDK
  • ENS support in beta
    • ens.domains
  • Improved performance for eth.tokens_ and eth.nft_ prefixed tables

Resources

May 2022

Spice now has some of the best automated token standard detection for erc20, erc721, and erc1155 tokens available. Tokens are detected by their signatures and as they emit events over time, so Spice provides a probability of standards compliance to each contract. Along with this support are now token-specific tables like eth.tokens_erc1155 and eth.token_transfers_erc20. See the entire list Tokens Tables.

Performance is now even better, especially for larger queries, and for results over the HTTP API. We still recommend the Apache Arrow Flight API that's easily accessible via the Python SDK for production use though.

WebSocket support is now available in private preview to Design Partners - get in touch if you are interested in custom limits, early access features, and dedicated support.

Changes

  • Improved Ethereum standard detection for erc20, erc721, and erc115 tokens
  • Added new token specific tables
    • eth.tokens_erc20
    • eth.tokens_erc721
    • eth.tokens_erc1155
    • eth.token_transfers_erc20
    • eth.token_transfers_erc721
    • eth.token_transfers_erc1155
  • Added new columns to eth.token_transfers table
    • token_standard
    • token_id
  • Added new columns to eth.contracts table
    • erc20_confidence
    • erc721_confidence
    • erc1155_confidence
    • is_erc1155
  • eth.nft_ tables now include erc1155 tokens

Resources

Apr 2022

Spice is now hosted at spice.xyz! And with this release Spice is now 100% data complete! We also have new combined documentation at docs.spice.xyz, new NFT specific tables like nft_transfers and nft_airdrop_transfers, significantly improved erc721 detection, and higher data integrity.

Changes

  • Now hosted at spice.xyz
  • Now 100% data complete on core Ethereum tables
  • Refreshed documentation site at docs.spice.xyz
  • Added input field to the transactions table
  • Added data, topics, block_timestamp fields to the logs table
  • Added block_timestamp and block_hash to the token_transfers table

Breaking Changes

  • The receipts table fields have been moved to the transactions table with prefix receipt_. The told receipts table has been removed.
  • receipts.cumulative_gas_used → transactions.receipt_cumulative_gas_used
  • receipts.gas_used → transactions.receipt_gas_used
  • receipts.contract_address → transactions.receipt_contract_address
  • receipts.root → transactions.receipt_root
  • receipts.effective_gas_price → transactions.receipt_effective_gas_price

Resources

Mar 2022

More improvements to performance and data completeness. Now 99.99% complete for Ethereum base types.

Significantly improved erc721 detection and now include new nft specific tables including eth.nft_contracts, eth.nft_transfers and eth.nft_airdrop_transfers.

Changes

  • Improved erc721 detection, now with 28,000+ erc721 rows
  • Adds eth.nft_contracts table
  • Adds eth.nft_transfers table
  • Adds eth.nft_airdrop_transfers table
  • Adds eth.recent_blocks table which holds the latest 100 blocks
  • More improvements to query performance
  • Now at 99.99% data complete

Resources

Feb 2022

Significant upgrades in performance and data completeness. Adds the ability to query Ethereum, Sushiswap, and Uniswap data with SQL. Adds an Apache Flight endpoint.

Changes

  • Adds SQL query
  • Adds Sushiswap, Uniswap-V2, and Uniswap-V3 liquidity data
  • Adds Apache Flight endpoint
  • Adds management portal
  • Significantly improved performance with the ability to query across real-time and historical data

Resources

Jan 2022

Initial release of the Spice beta!

Changes

  • The Initial release of the Spice Private Beta!
  • Adds /eth/v0.1/gasfees API.
  • Adds /eth/v0.1/contracts API.
  • Adds /v0.1/prices API.