The Problem and Our Solution

Traditional approach: The connection problem at Step 3

Mesoura approach: Reads code and standardises metadata automatically

How It Works

Mesoura Metadata's workflow transforms raw code into visual data lineage in five steps:

1. Locating

Upload your code files directly to the platform. Mesoura accepts SQL scripts and Python files containing data transformations.

2. Collating

Files are organised and stored securely. The system prepares for parsing by categorising code by language and identifying transformation patterns.

3. Parsing

This is where the magic happens. Our parsers analyse SQL queries and Python DataFrame operations to extract transformation logic. Unlike traditional approaches, parsing happens automatically; no manual interpretation required.

4. Aligning

Parsed transformations are aligned into a coherent data lineage structure. Column-level connections are mapped, transformation types are identified, and relationships between tables and DataFrames are established.

5. Visualising

View your data lineage through an interactive, physics-based node graph. Click nodes to explore column details, trace transformation paths, and understand exactly how data flows through your systems.

Interactive Demo

Explore how Mesoura Metadata visualises column-level data transformations. Toggle between SQL and Python scenarios to see how different code types are parsed and displayed. Each node represents a column, and arrows show transformations between columns. Hover over edges to see transformation details, and drag nodes to explore relationships. Use scroll to zoom in/out.

-- Create transformed orders table with concatenated customer summary
CREATE TABLE transformed_orders AS
SELECT
    order_id,
    CONCAT(customer_name, '  ', DATE_FORMAT(order_date, '%Y-%m-%d')) 
        AS customer_order_summary,
    quantity,
    CAST(price AS FLOAT) AS price_float
FROM orders;

-- Create summary with aggregations
CREATE TABLE order_summary AS
SELECT
    order_id,
    SUM(quantity) AS total_quantity,
    SUM(price_float) AS total_revenue
FROM transformed_orders
GROUP BY order_id;

# Load data from Snowflake
df = pd.DataFrame(data, columns=columns)

# Data Transformation
df['salary'] = df['salary'].astype(int)
df['hire_date'] = pd.to_datetime(df['hire_date'])

# Concatenate first and last name
df['full_name'] = df['first_name'] + ' ' + df['last_name']

# Add seniority level based on salary
def determine_seniority(salary):
    if salary < 50000:
        return "Junior"
    elif 50000 <= salary < 100000:
        return "Mid"
    else:
        return "Senior"

df['seniority_level'] = df['salary'].apply(determine_seniority)

# Group by department and calculate metrics
grouped_df = df.groupby('department').agg({'salary': 'mean', 'email': 'count'}).reset_index()
grouped_df.rename(columns={'salary': 'avg_salary', 'email': 'employee_count'}, inplace=True)

# Identify duplicate names
df['name_duplicate_count'] = df.groupby('full_name')['full_name'].transform('count')

What We Visualise

Mesoura Metadata helps you answer critical questions about how data flows between your systems:

Use Cases

Metadata Resources

Learn more about data lineage challenges and how Mesoura Metadata addresses them:

Key Features

• Reads Your Code Automatically: Upload SQL and Python transformation scripts, no configuration required
• Standardises Metadata: Converts transformation logic into unified metadata structure showing clear system connections
• Column-Level Detail: See exactly which source columns feed which target columns through each transformation
• Interactive Visualisation: Physics-based node graph; drag, zoom, hover to explore data flow between systems
• Purpose-Built Views: Upstream dependencies for DQ, critical data element highlighting for compliance, redundancy identification for efficiency
• Impact Analysis: Click any node to see what feeds it and what depends on it
• Export Documentation: Generate lineage snapshots for audit documentation and stakeholder presentations
• Desktop Software: Your code stays on your machine, no cloud uploads required
• Always Current: Re-upload updated scripts anytime to refresh lineage visualisation

Ready to Get Started?

Schedule a demo or request pricing information to see how Mesoura Metadata can eliminate stitching problems and provide automated, accurate data lineage.