Skip to content

Commit 8e0c1b5

Browse files
feat: Adding documentation for On Demand Feature Transformations with writes (feast-dev#4607)
* Update beta-on-demand-feature-view.md * updated docs Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * feat: Adding documentation for On Demand Feature Transformations with writes Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> --------- Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
1 parent 00eaf74 commit 8e0c1b5

File tree

1 file changed

+69
-7
lines changed

1 file changed

+69
-7
lines changed

docs/reference/beta-on-demand-feature-view.md

+69-7
Original file line numberDiff line numberDiff line change
@@ -4,20 +4,28 @@
44

55
## Overview
66

7-
On demand feature views allows data scientists to use existing features and request time data (features only available
8-
at request time) to transform and create new features at the time the data is read from the online store. Users define
9-
python transformation logic which is executed in both historical retrieval and online retrieval paths.
7+
On Demand Feature Views (ODFVs) allow data scientists to use existing features and request-time data (features only
8+
available at request time) to transform and create new features. Users define Python transformation logic which is
9+
executed during both historical retrieval and online retrieval. Additionally, ODFVs provide flexibility in
10+
applying transformations either during data ingestion (at write time) or during feature retrieval (at read time),
11+
controlled via the `write_to_online_store` parameter.
12+
13+
By setting `write_to_online_store=True`, transformations are applied during data ingestion, and the transformed
14+
features are stored in the online store. This can improve online feature retrieval performance by reducing computation
15+
during reads. Conversely, if `write_to_online_store=False` (the default if omitted), transformations are applied during
16+
feature retrieval.
1017

1118
### Why use on demand feature views?
1219

1320
This enables data scientists to easily impact the online feature retrieval path. For example, a data scientist could
1421

1522
1. Call `get_historical_features` to generate a training dataframe
1623
2. Iterate in notebook on feature engineering in Pandas/Python
17-
3. Copy transformation logic into on demand feature views and commit to a dev branch of the feature repository
24+
3. Copy transformation logic into ODFVs and commit to a development branch of the feature repository
1825
4. Verify with `get_historical_features` (on a small dataset) that the transformation gives expected output over historical data
19-
5. Verify with `get_online_features` on dev branch that the transformation correctly outputs online features
20-
6. Submit a pull request to the staging / prod branches which impact production traffic
26+
5. Decide whether to apply the transformation on writes or on reads by setting the `write_to_online_store` parameter accordingly.
27+
6. Verify with `get_online_features` on dev branch that the transformation correctly outputs online features
28+
7. Submit a pull request to the staging / prod branches which impact production traffic
2129

2230
## CLI
2331

@@ -32,10 +40,18 @@ See [https://github.com/feast-dev/on-demand-feature-views-demo](https://github.c
3240

3341
### **Registering transformations**
3442

35-
On Demand Transformations support transformations using Pandas and native Python. Note, Native Python is much faster but not yet tested for offline retrieval.
43+
On Demand Transformations support transformations using Pandas and native Python. Note, Native Python is much faster
44+
but not yet tested for offline retrieval.
45+
46+
When defining an ODFV, you can control when the transformation is applied using the write_to_online_store parameter:
47+
48+
- `write_to_online_store=True`: The transformation is applied during data ingestion (on write), and the transformed features are stored in the online store.
49+
- `write_to_online_store=False` (default when omitted): The transformation is applied during feature retrieval (on read).
3650

3751
We register `RequestSource` inputs and the transform in `on_demand_feature_view`:
3852

53+
## Example of an On Demand Transformation on Read
54+
3955
```python
4056
from feast import Field, RequestSource
4157
from feast.types import Float64, Int64
@@ -100,6 +116,52 @@ def transformed_conv_rate_python(inputs: Dict[str, Any]) -> Dict[str, Any]:
100116
return output
101117
```
102118

119+
## Example of an On Demand Transformation on Write
120+
121+
```python
122+
from feast import Field, on_demand_feature_view
123+
from feast.types import Float64
124+
import pandas as pd
125+
126+
# Existing Feature View
127+
driver_hourly_stats_view = ...
128+
129+
# Define an ODFV without RequestSource
130+
@on_demand_feature_view(
131+
sources=[driver_hourly_stats_view],
132+
schema=[
133+
Field(name='conv_rate_adjusted', dtype=Float64),
134+
],
135+
mode="pandas",
136+
write_to_online_store=True, # Apply transformation during write time
137+
)
138+
def transformed_conv_rate(features_df: pd.DataFrame) -> pd.DataFrame:
139+
df = pd.DataFrame()
140+
df['conv_rate_adjusted'] = features_df['conv_rate'] * 1.1 # Adjust conv_rate by 10%
141+
return df
142+
```
143+
Then to ingest the data with the new feature view make sure to include all of the input features required for the
144+
transformations:
145+
146+
```python
147+
from feast import FeatureStore
148+
import pandas as pd
149+
150+
store = FeatureStore(repo_path=".")
151+
152+
# Data to ingest
153+
data = pd.DataFrame({
154+
"driver_id": [1001],
155+
"event_timestamp": [pd.Timestamp.now()],
156+
"conv_rate": [0.5],
157+
"acc_rate": [0.8],
158+
"avg_daily_trips": [10],
159+
})
160+
161+
# Ingest data to the online store
162+
store.push("driver_hourly_stats_view", data)
163+
```
164+
103165
### **Feature retrieval**
104166

105167
{% hint style="info" %}

0 commit comments

Comments
 (0)