EF
Status Report · Week 1 of 9

Data Cloud → Snowflake Migration

Foundation week is on track. Two databases provisioned in CH Snowflake, three specs landed for the build phase, and the field-metadata coverage now spans all 8 business units. Plan endorsed in today's W1 review with Rich, Mike Grabbe and Adrian — handoff team confirmed, three concrete action items lined up. Build phase opens 6 May once external dependencies clear.

W1 · Day 2 of 7 Phase 0 · Foundation & Access 14.0 h · $2,040 · 8.2% NTE On track
Plan

9-week build · 6 phases · M1–M6 workstreams

Phase plan as agreed in the 29 Apr architecture walkthrough and committed in docs/migration-plan/migration-plan-deck.html. Foundation week (W1) sets up the CH dbt-2 home and the specs the build phase consumes.

P0 · W1 Foundation
P1 · W2-3 dbt #1 normalize
P2 · W3-4 Splink IR
P3 · W4-5 Canonical + CIs
P4 · W6-7 SFMC + Segments
P5 · W8 Cutover
P6 · W9 + Jul Handoff

Receiving team (confirmed 2026-04-30 W1 review): Mike Grabbe + Adrian LeDoux (ET Data Engineering). Rich DiTieri transitioning out — weekly standups with Mike Grabbe + Adrian throughout the engagement for knowledge transfer. Hypercare through July, then steady-state under ET ownership.

Latest update

W1 review · 2026-04-30

Working session with Rich, Mike Grabbe and Adrian after the migration-plan deck presentation. Plan endorsed, scope confirmed, handoff team finalised, three concrete action items lined up for next week.

Decisions taken

Architecture
  • Normalization moves to dbt (instead of staying in Snowflake) for easier maintenance + GitHub CI/CD
  • Cloud Pages = segment-creation UI, riding on super-message impressions (zero additional cost)
  • IR QA targets parity with Data Cloud results; imperfect matching acceptable — neither system will be perfect
  • Some downtime between old and new systems is acceptable before the 29 Jun deadline
Scope & ownership
  • Completion target: 30 June — driven by the Data Cloud timeline constraint
  • Segmentation UI scope strictly limited — fewer than 5 segments routinely, < monthly creation cadence; don't over-build
  • Mike Grabbe + Adrian LeDoux take ownership post-handoff, weekly standups for KT during the engagement
  • Rich is transitioning out — knowledge-transfer is a flagged risk, mitigated by docs + standups

Action items lined up for next week

Action 1 · Jonatan
Reach out to Mike Grabbe for the replication group
Enable the Snowflake replication-group feature US→CH. Mike Grabbe has prior experience from the Language project — needs a follow-up to avoid critical-path drag.
Action 2 · Jonatan
Ping Katie for dbt + GitHub access
Critical blocker preventing build kick-off. Target provisioning today / Monday at the latest so Phase 1 starts on time.

Other coordination points raised

  • NPS data from Student Tours identified as missing from the JIRA tickets — needs to be incorporated during the dbt #1 normalization phase
  • Ygritte v2 agent available directly in Snowflake CH for data validation — better credit limits than Langdoc, with built-in semantic-layer management via Cortex Analyst
  • Cross-region replication credit cost — multipliers exist; consensus is it won't be a major concern, but tracked via Resource Monitor
  • Warehouse sizing for IR — Snowpark-optimized warehouse may need an upgrade from XS to S/M; Snowflake support request adds 2-3 day potential delay

Full session log: docs/sessions/2026-04-30-rich-w1-review.md in the repo.

Done this week

What landed in W1

  1. 1
    CH Snowflake dbt-2 home provisioned — EF_DBT_SANDBOX + EF_DBT_PROD on account HH82036 (Zurich, aws_eu_central_2). Six schemas in each (replicated_us · sfmc_engagement · canonical · ir · cis · segments), warehouse EF_DBT_TRANSFORM_WH capped at 200 cr/month via Resource Monitor, two RW roles, MCP read access verified.
    Ticket: RLT-3700 · status: Done
  2. 2
    Replication-group spec drafted (US→CH) — architecture facts verified live (both accounts in Snowflake org EF, AWS↔AWS cross-region → Database Replication primitive). Copy-paste brief for Mike (Snowflake org admin) embedded in the spec; idempotent DDL skeleton for source + target accounts; lag SLO 30 min and refresh-trigger plan documented.
    Ticket: RLT-3811 · status: On Hold — pending Mike
  3. 3
    Canonical-model spec drafted — port the 7 KQ-audited DC DMOs (Individual · Account · Contact Point Email/Phone/Address · Opportunity · Sales Order — 259 fields total) as 1:1 dbt models in EF_DBT_PROD.CANONICAL. Pattern: one staging model per (DMO × DLO) → UNION ALL final per DMO. PKs stay product-scoped; cross-product joins go via Splink IR (the unified_individual layer). Engagement DMO excluded from canonical scope.
    Ticket: RLT-3812 · status: To Do · execution window: 20 May – 2 Jun
  4. 4
    Field-metadata coverage extended to all 8 BUs — pre-existing inventory only covered 5 streams in 3 BUs. Reproducible script scripts/extend_field_manifests.py (with smart Snowflake view-name fallback per BU) generated 47 new per-stream manifests; total now 52 streams · 8/8 BUs · 993 fields with Snowflake-derived types. The hand-curated 5 originals preserved as-is.
    Supports the canonical-model build under RLT-3812
  5. 5
    Foundation hygiene — CH Snowflake MCP target documented in agent instructions; access tokens rotated and restored to least-privilege storage across the local toolchain; bidirectional cross-links established for the W1 tickets (RLT-3700, RLT-3811, RLT-3812) across all engagement systems.
Pending external action

What blocks W2 build kick-off

Snowflake org admin
Mike Grabbe to enable the replication group US→CH

Brief delivered via RLT-3811: source/target accounts, asks (org-level enablement, replication group EF_DBT_NORMALIZE_TO_CH_RG, credit cost model), SLO. Mike Grabbe has prior experience from the Language project — confirmed in the W1 review meeting.

Lead time: 1–3 days typical · unblocks the entire P1 dbt #1 normalize chain.
Access provisioning
Katie to provision dbt + GitHub access

Critical blocker preventing the build phase from starting. Target: today / Monday at the latest. Once granted, dbt project #1 (US) and #2 (CH) bootstrap can land within W2 day 1.

Action item from the W1 review meeting (2026-04-30).
Knowledge-transfer cadence
Weekly standup with Mike Grabbe + Adrian

Standing meeting to be scheduled — agreed in the W1 review as the mitigation for the dual transition (Jonatan's engagement end + Rich's role transition). Documentation cadence runs alongside.

Mitigates the engagement-wide knowledge-transfer risk.
Budget snapshot

$25 k NTE · spent vs remaining

NTE cap
$25,000
SOW Amendment #2 (signed 29 Apr)
Spent to date
$2,040
14.0 hours · 8.2% of NTE
Remaining
$22,960
≈ 143h at part-time rate
M1 phase target
$2,560
16h SOW low · 🟠 80% used in W1 day 2

NTE consumption

8.2% consumed (W1 day 2). Burn pace is intentional this week — foundation work front-loads provisioning + spec writing so the build phase (W2 onwards) executes against a settled scope.

M1 phase budget — Foundation & Access (W1)

🟠 80% used in 2 days — phase target nearly exhausted on day 2. Remaining 3 days of W1 deliberately light (external dependencies in motion); planned soft landing into W2.

Scope reality vs NTE

PathEstimate$ at $140/hStatus
SOW low + new dbt #1 normalize + canonical replication189 h$26,460+$1.5 k over NTE

Trace: at the full-time intensity rate ($140/h) the plan currently exceeds the $25 k NTE by ~$1.5 k. Modest gap, manageable via aggressive AI-assisted SQL or marginal scope trim — agreed at the W1 review meeting today.

What's next

W2 (6 May – 12 May) · dbt #1 normalize kicks off

Day-1 deliverables (assuming W1 blockers clear by Mon 5 May)
P1 ramp-up
  • dbt project #1 (US) bootstrap — repo, profiles, env config
  • dbt project #2 (CH) bootstrap — profiles point to EF_DBT_SANDBOX/EF_DBT_PROD
  • Replication group operational — first refresh validated (RLT-3811)
  • Prefect scaffolding — connections + secrets + alerting
Asks of EF / ET this week
Stakeholder asks
  • Mike Grabbe to action the replication-group brief (RLT-3811) — ideally by Mon 5 May
  • Katie to provision dbt + GitHub access — today / Monday at the latest
  • Schedule the standing weekly KT standup with Mike Grabbe + Adrian
  • Add NPS data from Student Tours (raised in the W1 review) to the dbt #1 normalize scope
Reference material

Where to dig deeper

Engagement docs
Plan + commercial baseline
This week's specs
Specs landed in W1