Skip to content

Scope, fee and working days are published on every engagement

support@corephysixlabs.com 7702752958 Track an order
CorePhysix Labs Technology lab
Start a project
01 / WEB Web & App Engineering 5 engagements · from ₹24,999 02 / CLD Cloud & DevOps 5 engagements · from ₹12,999 03 / DAT Data & AI 5 engagements · from ₹14,999 04 / UXD Product Design & UX 5 engagements · from ₹14,999 05 / API APIs & Integrations 5 engagements · from ₹18,999 06 / SEC Security & Compliance 5 engagements · from ₹12,999 07 / QAT QA & Test Automation 5 engagements · from ₹12,999 08 / SUP Managed Support 5 engagements · from ₹8,999 09 / ADV Tech Advisory 12 engagements · from ₹100

Not sure where to start?

Book a short advisory session and let a specialist point you to the right engagement.

See advisory sessions Full catalogue

Specification DAT·102

DAT · Data & AI Defined scope Fixed fee

Airflow Data Pipeline Build

Up to eight Airflow pipelines with retries, alerts and safe backfills, replacing unmonitored cron jobs so data failures are caught fast.

Fee · GST incl.
₹34,999
Duration
18 working days
Deliverables
7

01 Overview

A cron job can fail for weeks before anyone spots the wrong month-end figure. Rebuilt in Airflow, those jobs leave a log for every run, a failure alerts a person and a backfill is a routine action rather than a lost weekend. The engagement covers up to eight pipelines. Tasks are made idempotent so a failed day can simply be run again, and dependencies are declared so no downstream job reads half-written data. Airflow runs on a managed service or inside your own Kubernetes cluster, whichever your team prefers.

02 Deliverables

7 items

What you receive at handover

  1. D01 Airflow environment on MWAA, Cloud Composer or your own Kubernetes cluster
  2. D02 Up to 8 pipelines migrated from existing scripts or built from scratch
  3. D03 Tasks made idempotent, with retries and timeouts set deliberately
  4. D04 Explicit dependencies that hold downstream tasks until data is complete
  5. D05 Slack or email alerts for failures and missed schedules
  6. D06 Backfill procedure written up and demonstrated on real data
  7. D07 Connections and variables managed through a secrets backend

03 Outcomes

What should be true once it ships

  • A failed run is flagged within minutes, not discovered at month end
  • Re-running a bad day is routine instead of a manual repair job
  • New pipelines follow an established pattern rather than starting from nothing

04 Process

How the 4 stages run

  1. Catalogue Every scheduled job is listed, including the forgotten ones
  2. Design Dependencies, schedules and failure handling are agreed per pipeline
  3. Migrate Pipelines are rebuilt and run alongside the old jobs
  4. Switch over Old cron entries are disabled once the outputs match

05 Fit & inputs

A good fit for

  • Teams whose data jobs live in a crontab on a server no one looks at
  • Analytics that break quietly and are spotted days later
  • Companies whose pipelines duplicate data whenever they are re-run

What we need from you

  • Access to the current scripts and the servers running them
  • Source and destination credentials for each job
  • A person able to confirm when an output is correct

These inputs are collected through the technical brief in your dashboard after payment. The duration is counted from when they arrive.

06 Answers

Questions about this engagement

Sometimes, yes. If the inventory shows Dagster or a simple scheduled container would serve you better, we recommend it and adjust the scope.

In most cases, after some light refactoring so they are safe to re-run. That work is included in the migration.

Your own team, working from the runbook. If you would rather hand off monitoring, a Managed Support plan covers it.

This is technology work delivered digitally — nothing is shipped. Work is delivered into systems you control within the duration stated above, and the delivery and refund policies set out what happens if a date is missed.

REL Related

Often considered alongside.

All Data & AI
DAT — Data & AI DAT·101

Postgres and dbt Analytics Warehouse

A Postgres or Snowflake warehouse with tested dbt models, so app, payment and ad data share one set of definitions everyone trusts.

Postgres or Snowflake warehouse provisioned with access roles

Fixed fee · GST incl.

₹39,999

Duration

25 working days

View scope
DAT — Data & AI DAT·105

Machine Learning Feasibility Study

A data audit and baseline model that show whether your data supports the model you want, before you commit budget to building it.

Audit of data volume, quality, labelling and class balance

Fixed fee · GST incl.

₹18,999

Duration

10 working days

View scope
DAT — Data & AI DAT·103

Metabase Reporting Setup

Four Metabase dashboards on a governed model layer, built around leadership decisions so recurring questions no longer need an analyst.

Metabase set up with a connection to your warehouse or database

Fixed fee · GST incl.

₹14,999

Duration

9 working days

View scope

GST included

₹34,999

Next step

Have a system that needs building, fixing or securing?

Explore the catalogue Describe the problem