Data and Field Lineage

Visualizing the journey of data through complex transformations

Workday • Product Design Leadership

Background

In enterprise data platforms, understanding how data flows and transforms across systems is critical for maintaining data quality, ensuring compliance, and enabling effective troubleshooting. Users needed a way to trace the origin, transformations, and destinations of individual data fields across complex data pipelines.

The existing workflow required users to manually trace field connections across multiple disconnected interfaces, creating significant cognitive overhead and slowing down critical debugging and analysis tasks. Without clear field-level lineage visualization, users struggled to answer fundamental questions like "Where does this field come from?" and "What happens to my data downstream?"

Goals

Design an intuitive field lineage visualization system that empowers users to understand data flow at the field level.

Visual Flow Representation

Create a clear visual representation showing field origins, transformations, and dependencies in data pipelines

Impact Analysis

Enable users to quickly understand the upstream and downstream impact of field changes

Reduce Cognitive Load

Simplify complex lineage relationships into digestible, navigable views

Self-Service Debugging

Empower analysts to trace data issues independently without engineering support

My Role

Lead Product Designer

  • Led end-to-end UX design for the field lineage visualization feature
  • Conducted discovery research and competitive analysis of lineage tools
  • Facilitated design workshops with engineering and product stakeholders
  • Created interaction patterns for navigating complex lineage graphs
  • Developed high-fidelity prototypes for usability testing
  • Collaborated with data engineering team on technical feasibility

Process

Our approach combined user research, iterative design, and close collaboration with data engineering to create a solution that balanced visual clarity with technical accuracy.

Phase 1: Discovery & Research
Conducted interviews with data analysts and engineers to understand how they currently trace field relationships. Mapped mental models for how users conceptualize data flow. Analyzed competitive lineage tools including dbt, Atlan, and Collibra.
Phase 2: Concept Development
Explored multiple visualization approaches including tree views, graph layouts, and column-based representations. Created low-fidelity wireframes to test different interaction patterns.
Phase 3: Design & Validation
Developed high-fidelity prototypes and conducted usability testing. Refined the visual language and interaction patterns based on user feedback.
Phase 4: Phased Delivery
Worked with product management to define a phased rollout strategy, starting with single-level lineage and expanding to multi-hop visualization in subsequent releases.

Solution

Key Design Decisions

Interactive Graph Visualization

Designed a node-link diagram that clearly shows field relationships, with interactive zoom and pan controls for exploring complex lineages. Users can expand or collapse nodes to manage visual complexity.

Bidirectional Navigation

Implemented upstream and downstream views allowing users to trace where a field comes from (backward lineage) or where it flows to (forward lineage) with a single click.

Transformation Annotations

Added inline annotations showing the specific transformations applied between nodes, helping users understand not just the flow but the logic applied to data.

Contextual Detail Panel

Designed a slide-out panel that shows detailed field metadata, data types, and sample values when a user selects a node, without losing the overall lineage context.

Search and Filter

Implemented field search with type-ahead and filtering by field type, enabling users to quickly locate specific fields within large lineage graphs.

Outcome

The field lineage feature was successfully delivered in a phased approach, with the initial release focused on single-hop lineage visualization and subsequent releases expanding to multi-hop analysis.

  • Reduced Investigation Time: Users reported significantly faster troubleshooting, with some tasks reduced from hours to minutes
  • Increased Self-Service: Analysts were able to trace field issues independently without escalating to data engineering
  • Improved Data Governance: Clear lineage visualization supported compliance and audit requirements
  • Positive User Feedback: High satisfaction scores in post-release surveys, with users highlighting the intuitive visual design

Reflections

This project reinforced the importance of designing for progressive disclosure in complex data visualization. By allowing users to start with a simple view and gradually explore deeper relationships, we avoided overwhelming them with complexity while still providing access to detailed information when needed.

The phased delivery approach proved essential for managing both technical complexity and user adoption. Starting with core functionality allowed us to gather feedback and iterate before expanding to more advanced features.

Due to confidentiality requirements, detailed visuals and specific metrics have been omitted from this case study.