Sr Data Engineer Snowflake , DBt and Qlik
Job Description
Sr. Data Engineer - Qlik Data Engineer/Developer
ā¢ Data Architecture and Strategy - Design and implement scalable, efficient data architectures. Lead the development of data strategy aligned with business objectives. Evaluate and integrate new technologies to enhance data capabilities
ā¢ Hands-on Data Pipeline Development - Implement complex data pipelines for real-time and batch processing. Optimize data flows for high-volume, high-velocity data environments. Develop advanced ETL processes for diverse data sources.
ā¢ Data Governance and Quality Management - Establish and enforce data governance policies and best practices. Implement data quality frameworks and monitoring systems. Ensure compliance with data regulations and standards
ā¢ Performance Optimization and Troubleshooting - Analyze and optimize system performance for large-scale data operations. Troubleshoot complex data issues and implement robust solutions
ā¢ Mentorship and Knowledge Sharing - Mentor junior data engineers and provide technical guidance. Contribute to the development of best practices and standards. Collaborate with cross-functional teams to drive data literacy
ā¢ Testing & Automation ā Write unit test cases, validate the data integrity & consistency requirements, build automated data pipelines using GitLab, Github, CICD tools.
ā¢ Code Deployment & Release Management - Adopt release management processes to promote code deployment to various environments including production, disaster recovery, and support activities.
Technical/Business Skills:
ā¢ Strong hands-on experience in building robust metadata-driven, automated data pipeline solutions leveraging modern cloud-based data technologies, tools for large data platforms.
ā¢ Strong experience leveraging data security, governance methodologies meeting data compliance requirements.
ā¢ Strong hands-on experience building medallion architecture automated ELT data pipelines, snowpipe frameworks leveraging Qlik Replicate, DBT Cloud, snowflake with CICD.
ā¢ Strong hands-on experience building data pipelines, data integrity solutions across multiple data sources and targets like SQL Server, Oracle, Mainframe-DB2, files, Snowflake.
ā¢ Experience working with various structured & semi-structured data files - CSV, fixed width, JSON, XML, Excel, and mainframe VSAM.
ā¢ Experience using S3, Lambda, SQS, SNS, Glue, RDS AWS services.
ā¢ Excellent proficiency in Python, Pyspark, advanced SQL for ingestion frameworks and automation.
ā¢ Hands-on data orchestration experience using DBT cloud, Astronomer Airflow.
ā¢ Experience in implementing logging, monitoring, alerting, observability, performance tuning techniques.
ā¢ Implement and maintain sensitive data protection strategies ā tokenization, snowflake data masking policies, dynamic & conditional masking, and role based masking rules.
ā¢ Strong experience designing, implementing RBAC, data access controls, adopting governance standards across Snowflake and supporting systems.
ā¢ Strong experience in adopting release management guidelines, code deployment to various environments, implementing disaster recovery strategies, leading production activities.
ā¢ Experience implementing schema drift detection and schema evolution patterns.
ā¢ Financial banking experience is a plus.
ā¢ Must have one or more certifications in the relevant technology fields.
Apply Now
Apply Now
Sr. Data Engineer - Qlik Data Engineer/Developer
ā¢ Data Architecture and Strategy - Design and implement scalable, efficient data architectures. Lead the development of data strategy aligned with business objectives. Evaluate and integrate new technologies to enhance data capabilities
ā¢ Hands-on Data Pipeline Development - Implement complex data pipelines for real-time and batch processing. Optimize data flows for high-volume, high-velocity data environments. Develop advanced ETL processes for diverse data sources.
ā¢ Data Governance and Quality Management - Establish and enforce data governance policies and best practices. Implement data quality frameworks and monitoring systems. Ensure compliance with data regulations and standards
ā¢ Performance Optimization and Troubleshooting - Analyze and optimize system performance for large-scale data operations. Troubleshoot complex data issues and implement robust solutions
ā¢ Mentorship and Knowledge Sharing - Mentor junior data engineers and provide technical guidance. Contribute to the development of best practices and standards. Collaborate with cross-functional teams to drive data literacy
ā¢ Testing & Automation ā Write unit test cases, validate the data integrity & consistency requirements, build automated data pipelines using GitLab, Github, CICD tools.
ā¢ Code Deployment & Release Management - Adopt release management processes to promote code deployment to various environments including production, disaster recovery, and support activities.
Technical/Business Skills:
ā¢ Strong hands-on experience in building robust metadata-driven, automated data pipeline solutions leveraging modern cloud-based data technologies, tools for large data platforms.
ā¢ Strong experience leveraging data security, governance methodologies meeting data compliance requirements.
ā¢ Strong hands-on experience building medallion architecture automated ELT data pipelines, snowpipe frameworks leveraging Qlik Replicate, DBT Cloud, snowflake with CICD.
ā¢ Strong hands-on experience building data pipelines, data integrity solutions across multiple data sources and targets like SQL Server, Oracle, Mainframe-DB2, files, Snowflake.
ā¢ Experience working with various structured & semi-structured data files - CSV, fixed width, JSON, XML, Excel, and mainframe VSAM.
ā¢ Experience using S3, Lambda, SQS, SNS, Glue, RDS AWS services.
ā¢ Excellent proficiency in Python, Pyspark, advanced SQL for ingestion frameworks and automation.
ā¢ Hands-on data orchestration experience using DBT cloud, Astronomer Airflow.
ā¢ Experience in implementing logging, monitoring, alerting, observability, performance tuning techniques.
ā¢ Implement and maintain sensitive data protection strategies ā tokenization, snowflake data masking policies, dynamic & conditional masking, and role based masking rules.
ā¢ Strong experience designing, implementing RBAC, data access controls, adopting governance standards across Snowflake and supporting systems.
ā¢ Strong experience in adopting release management guidelines, code deployment to various environments, implementing disaster recovery strategies, leading production activities.
ā¢ Experience implementing schema drift detection and schema evolution patterns.
ā¢ Financial banking experience is a plus.
ā¢ Must have one or more certifications in the relevant technology fields.
Apply Now
Apply Now