Watchdog File Pipelines quarantine layout stopped a vendor CSV from poisoning downstream merges twice already.
Automation Scripts
Robust File Pipelines with Watchdog Hooks
Detect new files, validate schemas, and route them through staged folders automatically.
20 hours · blended Blended cohort with shared office hours BRL 690 · informational
Program narrative
Ideal for teams that receive recurring partner drops. You will combine watchdog observers with validation scripts that quarantine bad files instead of halting the whole pipeline.
Inside the bundle
- Observer patterns that avoid duplicate event storms
- Schema validation using lightweight JSON or CSV checks
- Quarantine folder layout with human-readable reasons
- Graceful shutdown snippet for Windows services
- Throughput notes for SMB shares vs local disks
- Mentor checkpoint on backpressure strategies
- Sample docker-compose for optional isolated testing
Outcomes we expect you to evidence
- Stand up a watched folder with quarantine behavior
- Produce runbooks for partners who mislabel files
- Measure average processing time on a sample batch
Mentor anchor
Automation mentor focused on file-heavy logistics workflows.
Aline Rocha
Primary reviewer for this program track.
Participant questions
Labs include SMB caveats; mentors help interpret your latency numbers.
Conceptual overview only; depth stays with OS-level watchers you can reason about offline.
No dedicated appliances; you need a machine that can stay awake during tests.