Skip to content

Commit

Permalink
feat: Initial merge of Web UI logic (#2352)
Browse files Browse the repository at this point in the history
* Initial merge of Web UI logic

Signed-off-by: Danny Chiao <danny@tecton.ai>

* Initial merge of Web UI logic

Signed-off-by: Danny Chiao <danny@tecton.ai>

* Initial merge of Web UI logic

Signed-off-by: Danny Chiao <danny@tecton.ai>

* Rebase

Signed-off-by: Danny Chiao <danny@tecton.ai>

* Rebase

Signed-off-by: Danny Chiao <danny@tecton.ai>

* test

Signed-off-by: Danny Chiao <danny@tecton.ai>

* test

Signed-off-by: Danny Chiao <danny@tecton.ai>

* test

Signed-off-by: Danny Chiao <danny@tecton.ai>

* add sample image

Signed-off-by: Danny Chiao <danny@tecton.ai>

* add sample image

Signed-off-by: Danny Chiao <danny@tecton.ai>

* remove comment

Signed-off-by: Danny Chiao <danny@tecton.ai>

* remove comment

Signed-off-by: Danny Chiao <danny@tecton.ai>
  • Loading branch information
adchia authored Mar 3, 2022
1 parent 1ba86fb commit ce3bc59
Show file tree
Hide file tree
Showing 170 changed files with 24,951 additions and 17 deletions.
28 changes: 24 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
### Scratch files ###
scratch*

### Feast UI deprecated folder ####
ui/

### Local Environment ###
*local*.env

Expand Down Expand Up @@ -197,4 +194,27 @@ sdk/go/protos/
**/registry.db
**/*.aof
**/*.rdb
**/nodes.conf
**/nodes.conf


# Feast UI dependencies
ui/node_modules
ui/.pnp
ui/.pnp.js
ui/coverage
ui/build
ui/feature_repo/data/online.db
ui/feature_repo/registry.db
ui/.vercel

# misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local

**/npm-debug.log*
**/yarn-debug.log*
**/yarn-error.log*

16 changes: 10 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,11 @@ cd my_feature_repo
feast apply
```

### 4. Build a training dataset
### 4. Explore your data in the web UI (experimental)

![Web UI](ui/sample.png)

### 5. Build a training dataset
```python
from feast import FeatureStore
import pandas as pd
Expand Down Expand Up @@ -86,7 +90,7 @@ print(training_df.head())
```

### 5. Load feature values into your online store
### 6. Load feature values into your online store
```commandline
CURRENT_TIME=$(date -u +"%Y-%m-%dT%H:%M:%S")
feast materialize-incremental $CURRENT_TIME
Expand All @@ -96,7 +100,7 @@ feast materialize-incremental $CURRENT_TIME
Materializing feature view driver_hourly_stats from 2021-04-14 to 2021-04-15 done!
```

### 6. Read online features at low latency
### 7. Read online features at low latency
```python
from pprint import pprint
from feast import FeatureStore
Expand Down Expand Up @@ -165,7 +169,7 @@ The list below contains the functionality that contributors are planning to deve
* [x] [Azure Cache for Redis (community plugin)](https://github.com/Azure/feast-azure)
* [x] [Postgres (community plugin)](https://github.com/nossrannug/feast-postgres)
* [x] [Custom online store support](https://docs.feast.dev/how-to-guides/adding-support-for-a-new-online-store)
* [ ] Bigtable
* [ ] Bigtable (in progress)
* [ ] Cassandra
* **Streaming**
* [x] [Custom streaming ingestion job support](https://docs.feast.dev/how-to-guides/creating-a-custom-provider)
Expand All @@ -192,16 +196,16 @@ The list below contains the functionality that contributors are planning to deve
* [ ] Feature Logging (for training)
* **Data Quality Management (See [RFC](https://docs.google.com/document/d/110F72d4NTv80p35wDSONxhhPBqWRwbZXG4f9mNEMd98/edit))**
* [x] Data profiling and validation (Great Expectations)
* [ ] Training-serving skew detection (in progress)
* [ ] Metric production
* [ ] Training-serving skew detection
* [ ] Drift detection
* **Feature Discovery and Governance**
* [x] Python SDK for browsing feature registry
* [x] CLI for browsing feature registry
* [x] Model-centric feature tracking (feature services)
* [x] Amundsen integration (see [Feast extractor](https://github.com/amundsen-io/amundsen/blob/main/databuilder/databuilder/extractor/feast_extractor.py))
* [ ] Feast Web UI (in progress)
* [ ] REST API for browsing feature registry
* [ ] Feast Web UI
* [ ] Feature versioning


Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started/third-party-integrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Don't see your offline store or online store of choice here? Check out our guide
* [x] [Azure Cache for Redis (community plugin)](https://github.com/Azure/feast-azure)
* [x] [Postgres (community plugin)](https://github.com/nossrannug/feast-postgres)
* [x] [Custom online store support](https://docs.feast.dev/how-to-guides/adding-support-for-a-new-online-store)
* [ ] Bigtable
* [ ] Bigtable (in progress)
* [ ] Cassandra

### **Deployments**
Expand Down
6 changes: 3 additions & 3 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ The list below contains the functionality that contributors are planning to deve
* [x] [Azure Cache for Redis (community plugin)](https://github.com/Azure/feast-azure)
* [x] [Postgres (community plugin)](https://github.com/nossrannug/feast-postgres)
* [x] [Custom online store support](https://docs.feast.dev/how-to-guides/adding-support-for-a-new-online-store)
* [ ] Bigtable
* [ ] Bigtable (in progress)
* [ ] Cassandra
* **Streaming**
* [x] [Custom streaming ingestion job support](https://docs.feast.dev/how-to-guides/creating-a-custom-provider)
Expand All @@ -64,14 +64,14 @@ The list below contains the functionality that contributors are planning to deve
* [ ] Feature Logging (for training)
* **Data Quality Management (See [RFC](https://docs.google.com/document/d/110F72d4NTv80p35wDSONxhhPBqWRwbZXG4f9mNEMd98/edit))**
* [x] Data profiling and validation (Great Expectations)
* [ ] Training-serving skew detection (in progress)
* [ ] Metric production
* [ ] Training-serving skew detection
* [ ] Drift detection
* **Feature Discovery and Governance**
* [x] Python SDK for browsing feature registry
* [x] CLI for browsing feature registry
* [x] Model-centric feature tracking (feature services)
* [x] Amundsen integration (see [Feast extractor](https://github.com/amundsen-io/amundsen/blob/main/databuilder/databuilder/extractor/feast_extractor.py))
* [ ] Feast Web UI (in progress)
* [ ] REST API for browsing feature registry
* [ ] Feast Web UI
* [ ] Feature versioning
10 changes: 7 additions & 3 deletions infra/templates/README.md.jinja2
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,11 @@ cd my_feature_repo
feast apply
```

### 4. Build a training dataset
### 4. Explore your data in the web UI (experimental)

![Web UI](ui/sample.png)

### 5. Build a training dataset
```python
from feast import FeatureStore
import pandas as pd
Expand Down Expand Up @@ -84,7 +88,7 @@ print(training_df.head())

```

### 5. Load feature values into your online store
### 6. Load feature values into your online store
```commandline
CURRENT_TIME=$(date -u +"%Y-%m-%dT%H:%M:%S")
feast materialize-incremental $CURRENT_TIME
Expand All @@ -94,7 +98,7 @@ feast materialize-incremental $CURRENT_TIME
Materializing feature view driver_hourly_stats from 2021-04-14 to 2021-04-15 done!
```

### 6. Read online features at low latency
### 7. Read online features at low latency
```python
from pprint import pprint
from feast import FeatureStore
Expand Down
61 changes: 61 additions & 0 deletions ui/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# [WIP] Feast Web UI

![Sample UI](sample.png)

This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).

## Project structure
The Web UI is powered by a JSON registry dump from Feast (running `feast registry-dump`). Running `yarn start` launches a UI
powered by test data.
- `public/` contains assets as well as demo data loaded by the Web UI.
- There is a `projects-list.json` which represents all Feast projects the UI shows.
- There is also a `registry.json` which is the registry dump for the feature repo.
- `feature_repo/` contains a sample Feast repo which generates the `registry.json`
- `src/` contains the Web UI source code. This parses the registry json blob in `src/parsers` to make this data
available for the rest of the UI.
- `src/custom-tabs` includes sample custom tabs. This is a WIP plugin system where users can inject their own tabs and
data to the UI.


## Available Scripts

In the project directory, you can run:

### `yarn start`

Runs the app in the development mode.\
Open [http://localhost:3000](http://localhost:3000) to view it in the browser.

The page will reload if you make edits.\
You will also see any lint errors in the console.

### `yarn test`

Launches the test runner in the interactive watch mode.\
See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.

### `yarn build`

Builds the app for production to the `build` folder.\
It correctly bundles React in production mode and optimizes the build for the best performance.

The build is minified and the filenames include the hashes.\
Your app is ready to be deployed!

See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information.

### `yarn eject`

**Note: this is a one-way operation. Once you `eject`, you can’t go back!**

If you aren’t satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project.

Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you’re on your own.

You don’t have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn’t feel obligated to use this feature. However we understand that this tool wouldn’t be useful if you couldn’t customize it when you are ready for it.

## Learn More

You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started).

To learn React, check out the [React documentation](https://reactjs.org/).
72 changes: 72 additions & 0 deletions ui/feature_repo/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Feast repo

## Overview

This pulls from the dataset used in https://github.com/feast-dev/feast-aws-credit-scoring-tutorial but adds metadata for a full set of Feast FCOs.

This also adds an on demand feature view + feature services + a saved dataset.

## Setting up Feast

Install a dev build Feast using pip

Clone a feast repo:
```bash
git clone https://github.com/feast-dev/feast.git
```

Install a dev build of feast
```bash
cd feast
pip install -e "sdk/python[dev]"
```

Then for this demo, you'll actually need to fix a bug by adding this to `type_map.py#L144`:
```python
if isinstance(value, np.bool_):
return ValueType.BOOL
```

## Test features
We have already set up a feature repository here using test data from [data](data)). Features have already been pre-materialized to a local sqlite online store. The results of `feast registry-dump` have been thrown into [registry.json](../public/registry.json)

To query against this registry, you can use run the `test_get_features.py`
```bash
python test_get_features.py
```

Output:
```
--- Historical features (from saved dataset) ---
mortgage_due credit_card_due missed_payments_1y total_wages dob_ssn event_timestamp state tax_returns_filed location_type population city zipcode
0 741165 2944 3 71067272 19781116_7723 2021-04-12 08:12:10+00:00 MI 2424 PRIMARY 4421 WEIDMAN 48893
1 91803 8419 0 534687864 19530219_5179 2021-04-12 10:59:42+00:00 GA 19583 PRIMARY 38542 DALTON 30721
2 1553523 5936 0 226748453 19500806_6783 2021-04-12 15:01:12+00:00 TX 6827 PRIMARY 12902 CLEBURNE 76031
3 976522 833 0 34796963 19931128_5771 2021-04-12 16:40:26+00:00 VA 1287 PRIMARY 2342 GLADE HILL 24092
--- Online features ---
city : ['DALTON']
credit_card_due : [8419]
dob_ssn : ['19530219_5179']
location_type : ['PRIMARY']
missed_payments_1y : [0]
mortgage_due : [91803]
population : [38542]
state : ['GA']
tax_returns_filed : [19583]
total_wages : [534687864]
zipcode : [30721]
city : ['DALTON']
credit_card_due : [8419]
dob_ssn : ['19530219_5179']
location_type : ['PRIMARY']
missed_payments_1y : [0]
mortgage_due : [91803]
population : [38542]
state : ['GA']
tax_returns_filed : [19583]
total_wages : [534687864]
transaction_gt_last_credit_card_due : [False]
zipcode : [30721]
```
Binary file added ui/feature_repo/data/credit_history.parquet
Binary file not shown.
Loading

0 comments on commit ce3bc59

Please sign in to comment.