Data architecture · in detail
Process, deliverables, pricing and when you actually need this.
Before anyone writes a pipeline, somebody has to decide where the data will live, how it flows, and who pays for storage. This is the work that prevents technical debt from accumulating faster than the value you extract.
Process step-by-step
Architecture work follows a fixed sequence.
You can’t design storage before you know what questions the business is asking. You can’t choose tools before you’ve mapped existing sources.
Inventory & requirement gathering
We interview stakeholders across finance, operations, and IT. We document every data source currently in use—ERP, CRM, spreadsheets, external APIs. We map which reports people actually read and which KPIs they trust. This isn’t about what the data could do. It’s about what the business needs it to do next quarter.
Deliverable: Current state map showing every system, its role, and who depends on it.
Architecture design
Based on the inventory, we design the target state. We choose between cloud platforms (Azure, AWS, GCP), decide on lakehouse vs warehouse, define data zones (raw, curated, gold), and specify access control patterns. We model the key entities—customer, product, transaction—and define how they relate. Every decision is documented with a reason.
Deliverable: Target architecture diagram with layers, tool choices, and cost estimates per component.
Build sequence & governance
We create a phased implementation plan: which pipelines get built first, which reports migrate when, what runs in parallel. We define naming conventions, folder structures, and deployment patterns. We specify who approves schema changes and how data quality gets monitored. This is the blueprint that engineering will follow for the next 6-12 months.
Deliverable: Implementation roadmap with phases, dependencies, and governance rules.
Decision log & review
Throughout the engagement, we maintain a log of rejected options: why we didn’t choose Snowflake, why we skipped data mesh patterns, why certain sources stay outside the platform. This prevents future teams from re-litigating decisions. At the end, we present the full package and run a technical review session with your IT team.
Deliverable: Decision log with rationale for every major choice, plus handoff session.
Detailed deliverables
What you actually receive
These are working documents, not slide decks. Engineering teams use them to write pipelines. Finance uses them to forecast cloud spend. IT uses them to set permissions.
Current state map
A diagram showing every system that currently holds business data: ERP, CRM, spreadsheets, legacy databases, SaaS tools. Each node includes the data owner, update frequency, and which business processes depend on it. This isn’t decorative—it’s the foundation for migration planning. If a source isn’t on this map, it doesn’t exist in the target architecture.
Target architecture diagram
A layered design showing ingestion zone, transformation layers, and consumption endpoints. It specifies cloud platform (Azure Data Lake, Databricks, Snowflake), defines medallion layers (bronze/silver/gold or equivalent), and maps data flow from source to dashboard. It includes compute vs storage trade-offs and estimated monthly cost per layer. This is what you show to the CFO when they ask where the cloud bill is going.
Build sequence roadmap
A Gantt-style plan showing which pipelines get built in which order, with dependencies between them. It specifies quick wins (reports that can go live in 4 weeks) vs foundational work (dimensional models that take 3 months). Each phase includes expected business value—this is how you justify continued investment when the CTO asks if the project is on track.
Decision log & rejected options
A written record of architectural choices: why we picked Databricks over Snowflake, why we use delta tables instead of parquet, why certain sources stay in the old system. It includes the trade-offs for each decision and the conditions under which you’d reverse it. This prevents new hires from proposing solutions you already evaluated and rejected for good reasons.
Governance & naming standards
Rules for how tables get named, how schemas get versioned, who approves breaking changes, and how data quality alerts get routed. This includes folder structures in the data lake, tagging conventions for cost allocation, and access control patterns (RBAC vs ABAC). Without this, teams create their own conventions and you end up with five different naming schemes six months later.
Cost model & scaling triggers
A spreadsheet showing projected monthly cost per component at current data volume, plus scaling thresholds. It specifies when to move from serverless to provisioned clusters, when to add more storage, and what happens to the bill if data volume doubles. This lets finance plan budgets and lets IT know when to renegotiate contracts.
Related case
Industrial: SAP data warehouse on Databricks
A manufacturing client needed a scalable data platform to replace manual reporting. We designed a medallion architecture on Databricks with automated Bronze layer ingestion from SAP ECC tables, multi-environment CI/CD pipelines for dev/test/prod, and governance patterns that allowed their team to collaborate without breaking production—turning months of planning into a working architecture in 4 weeks.
Read the full case →Pricing & what affects it
Transparent pricing context
Base architecture project
€45/hour
Estimated 120–160 hours · 3–4 weeks
What makes the price go up
- Number of source systems: Integrating 15 data sources takes longer than 5. Each system requires extraction analysis, API documentation review, and schema mapping.
- Regulatory requirements: GDPR compliance, SOC2, or industry-specific auditing add governance layers. We need to define data retention policies, anonymization rules, and access logs.
- Multi-region deployment: If data needs to live in EU and US regions simultaneously, architecture complexity doubles. We define replication patterns, latency SLAs, and cost allocation per region.
- Legacy migration complexity: Moving off a 15-year-old on-prem SQL Server with undocumented tables requires reverse engineering. We allocate time for data profiling and transformation logic discovery.
What keeps the price down
- Greenfield projects: If you’re starting from zero with no legacy systems, there’s no migration analysis. We design the target state directly.
- Single cloud platform: Designing for Azure only is faster than hybrid AWS+GCP. No cross-cloud networking or IAM federation.
- Standard tools: Using Databricks + Delta Lake (widely documented patterns) is faster than custom open-source stacks that require bespoke design.
- Clear requirements: If business stakeholders already know which KPIs they need and which reports they trust, we skip discovery and go straight to design.
Team reinforcement
€42.5/hour
Embedding an architect in your team for 6+ months
Ongoing advisory
€45/hour
Quarterly architecture reviews
Questions
FAQ: Data architecture
Do I need an architect if I’m just using Power BI on top of an Excel file?
No. If your data fits in Excel and Power BI performs fine, you don’t have an architecture problem. You have a reporting problem, which is solved with BI work. Architecture becomes necessary when data volume exceeds what Excel can handle, when multiple teams need simultaneous access, or when regulatory compliance requires audit trails and access control.
How do I know if I should build on Azure, AWS, or GCP?
If you already use Microsoft 365 and have an enterprise agreement, Azure is usually cheaper due to bundled licensing. If your IT team has deep AWS experience, AWS avoids retraining costs. If you need specific ML tools only available on GCP (like BigQuery ML), that drives the choice. We evaluate existing contracts, team skills, and tool requirements—there’s no universal best answer.
What’s the difference between a data lake, a data warehouse, and a lakehouse?
A data lake stores raw files (CSV, JSON, parquet) in cloud storage. It’s cheap and flexible but requires engineering work to query. A data warehouse (Snowflake, Synapse) stores structured tables optimized for SQL queries. It’s fast for BI but expensive at scale. A lakehouse (Databricks, Fabric) combines both: raw file storage with SQL query performance. For most modern projects, lakehouse is the default unless you have a specific reason to split them.
Next steps
Architecture work comes first.
If you’re deciding between cloud platforms or need to map your current data landscape before building pipelines, this is where you start.