Senior Incorta Data Engineer

Position: Senior Incorta Data Engineer

Experience Required

6-8 Years

Job Summary

We are seeking an experienced Senior Incorta Data Engineer to design, develop, and optimize enterprise reporting and analytics solutions using the Incorta platform . The ideal candidate should possess strong expertise in Incorta schemas, dashboards, Spark SQL, PySpark, SQL, ETL, data warehousing, and ERP data integration (SAP/Oracle) .

The candidate will be responsible for building scalable data pipelines, integrating data from multiple enterprise sources, developing business dashboards, optimizing query performance, and ensuring high-quality reporting solutions in a large-scale data environment.

Key Responsibilities

Incorta Development

• Design, develop, and maintain Incorta schemas, Business Views, and Insights Dashboards.
• Build scalable and high-performing analytics solutions using the Incorta platform.
• Develop interactive dashboards and reports for business users.
• Optimize Incorta query performance and data load processes.
• Manage Incorta data models and schema design.

Data Engineering & Pipeline Development

• Design and build scalable ETL/data pipelines using PySpark .
• Develop data ingestion frameworks for structured and semi-structured data.
• Process large-scale datasets efficiently in distributed environments.
• Integrate data from databases, APIs, flat files, ERP systems, and cloud sources.
• Ensure data quality, consistency, and reliability throughout the data lifecycle.

Data Warehousing & Data Modeling

• Design enterprise data warehouse solutions.
• Develop logical and physical data models.
• Create optimized schemas for analytics and reporting.
• Apply dimensional modeling techniques and ETL best practices.
• Support enterprise reporting and business intelligence initiatives.

ERP & Enterprise Data Integration

• Integrate data from:
  • SAP
  • Oracle
  • SQL Databases
  • REST APIs
  • Flat Files
  • Enterprise Applications
• Build reliable integration pipelines for business-critical data.
• Ensure seamless synchronization between ERP systems and analytics platforms.

Performance Optimization

• Optimize SQL and Spark SQL queries.
• Improve data load performance within Incorta.
• Monitor system performance and identify bottlenecks.
• Enhance scalability and efficiency of data processing workflows.

Testing & Data Validation

• Perform data validation and reconciliation.
• Troubleshoot ETL and reporting issues.
• Validate business data accuracy and completeness.
• Support testing of dashboards and reporting solutions.

Collaboration

• Collaborate with business stakeholders to gather reporting and analytics requirements.
• Work closely with Data Architects, BI Developers, and Business Analysts.
• Participate in Agile ceremonies and project planning.
• Provide technical support and mentoring to junior developers.

Documentation

• Maintain technical documentation for:
  • Data models
  • ETL pipelines
  • Incorta schemas
  • Dashboard designs
  • Reporting solutions
• Create operational runbooks and deployment documentation.

Required Skills

Incorta

• Incorta Platform
• Schema Development
• Business Views
• Insights Dashboards
• Dashboard Development
• Spark SQL

Data Engineering

• PySpark
• Data Pipeline Development
• Data Transformation
• Data Processing
• Data Ingestion
• Data Integration

Data Warehousing

• Data Warehousing Concepts
• ETL Processes
• Data Modeling
• Schema Design
• Star Schema
• Snowflake Schema

Big Data

• SQL
• Spark SQL
• Large Dataset Processing
• Distributed Data Processing

ERP Integration

• SAP Integration
• Oracle Integration
• Database Integration
• REST APIs
• Flat File Processing

Performance & Quality

• Query Performance Optimization
• Data Load Optimization
• Data Validation
• Testing
• Troubleshooting
• Performance Monitoring
• Scalability Improvements

Methodologies

• Agile
• Scrum
• SDLC

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 6-8 years of experience in Data Engineering or Business Intelligence.
• Strong hands-on experience with the Incorta platform .
• Expertise in PySpark , Spark SQL , and SQL .
• Experience with ETL processes and enterprise data warehousing.
• Experience integrating data from SAP , Oracle , and enterprise databases.
• Strong analytical and problem-solving skills.
• Excellent communication and stakeholder management abilities.

Preferred Qualifications

• Experience working in Big Data environments.
• Knowledge of cloud-based data platforms.
• Experience with Apache Spark ecosystems.
• Exposure to Power BI or Tableau.
• Experience with CI/CD pipelines.
• Familiarity with Python scripting.