Skip to content

Commit 802de24

Browse files
committed
Add stream orchestration notes
Signed-off-by: Danny Chiao <[email protected]>
1 parent 0b8ce3e commit 802de24

File tree

2 files changed

+3
-3
lines changed

2 files changed

+3
-3
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,4 +57,4 @@ These are meant mostly to be done in order, with examples building on previous c
5757
| 30-45 | Setting up Feast projects & CI/CD + powering batch predictions | [Module 0](module_0/README.md) |
5858
| 15-20 | Streaming ingestion & online feature retrieval with Kafka, Spark, Airflow, Redis | [Module 1](module_1/README.md) |
5959
| 10-15 | Real-time feature engineering with on demand transformations | [Module 2](module_2/README.md) |
60-
| 30 | (WIP) Orchestrated batch transformations using dbt + Airflow with Feast | [Module 3](module_3/README.md) |
60+
| 30 | (WIP) Orchestrated batch/stream transformations using dbt + Airflow with Feast | [Module 3](module_3/README.md) |

module_3/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
<h1>Module 3: Orchestrated batch transformations using dbt + Airflow with Feast (Snowflake)</h1>
1+
<h1>Module 3: Orchestrated batch / stream transformations using dbt + Airflow with Feast (Snowflake)</h1>
22

33
> **Note:** This module is still WIP, and does not have a public data set to use. There is a smaller dataset visible in `data/`
44
@@ -271,7 +271,7 @@ To outline a few approaches:
271271
- **Option 4** If you're using Flink, then consider configuring a [restart strategy](https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/ops/state/task_failure_recovery/)
272272

273273
# Conclusion
274-
By the end of this module, you will have learned how to build a full feature platform, with orchestrated batch transformations (using dbt + Airflow), orchestrated materialization (with Feast + Airflow).
274+
By the end of this module, you will have learned how to build a full feature platform, with orchestrated batch transformations (using dbt + Airflow), orchestrated materialization (with Feast + Airflow), and pointers on orchestrating streaming transformations.
275275

276276
## Limitations
277277
- Feast does not itself handle orchestration of transformation or materialization, and relies on the user to configure this with tools like dbt and Airflow.

0 commit comments

Comments
 (0)