Logo

Data Management

ETL Job Manager

AI Generated Summary

142 ETL jobs ran today with a 97.2% success rate. 4 jobs failed due to a timeout on the customer_events transform step; retries are queued. Average processing duration improved 12% after last week's optimization.

Confidence 96.1% · generated 6 min ago

Jobs Today

142

Success Rate

97.2%

Failed

4

Avg Duration

4m 12s

Job Dependency Flow

extract_events transform_events load_warehouse refresh_models
4-stage chain · 1 stage currently failing transform_events blocking downstream jobs

Success Rate Dashboard

Daily success rate · last 14 days

Processing Duration

By job type

Job Queue

Currently running & scheduled jobs

Data Transformation Summary

Rows Transformed

18.7M

Schema Changes

2

Fields Mapped

246

Rows Rejected

1,082

Dedup Merges

3,940

Avg Transform Time

4.2s

Last transform run 4m 12s Healthy

Resource Usage

Compute allocation during job runs

Failed Jobs / Retry Queue

transform_events · timeout after 10m
refresh_models · upstream dependency failed
load_warehouse · connection pool exhausted
extract_events · source schema drift
4 jobs failed today

AI Optimization Suggestions

Increase timeout threshold for transform_events to 15m and add incremental checkpointing to avoid full re-runs on failure.
Parallelize refresh_models across 3 shards to remove its dependency on a single transform_events run.
Projected duration improvement −12% avg runtime

Job History

load_warehouse completed · 4m 02s12m ago
extract_events completed · 1m 48s16m ago
refresh_models completed · 6m 30s28m ago
Runs completed today 138 of 142

Notifications

4 jobs failed · retry queued10m ago
Weekly ETL cost report generated3h ago
Optimization applied to load_warehouseYesterday
Unread notifications 1 new

Activity Timeline

transform_events timed out

10 minutes ago

load_warehouse completed successfully

12 minutes ago

Optimization applied to load_warehouse job

1 day ago