Prompt

Design scalable data pipeline architecture.

<data_sources>
{list all data sources and volumes}
</data_sources>

<transformations>
{required data transformations}
</transformations>

<destinations>
{data warehouses and consumers}
</destinations>

Design:
1. Ingestion strategy
2. Processing framework
3. Data quality checks
4. Error handling
5. Monitoring approach
6. Scaling strategy