(8OW) Data Engineer 1

Overview Software Mind is seeking qualified candidates to fill the role of Data Engineer . In addition to a competitive salary rate and a positive work environment, we are committed to delivering high-quality technology solutions. We also offer: Flexible schedules An authentic work-life balance Payment in US Dollars About the role: We are seeking a highly motivated Data Engineer to play a crucial role in our data transformation program. You will be instrumental in the migration of data sources to a centralized cloud environment, with Databricks as our strategic standard on AWS. As a Data Engineer, you will be responsible for designing, developing, and maintaining our data infrastructure and pipelines, utilizing a variety of technologies and tools to ensure efficient data processing, integration, and management. You will collaborate closely with data scientists, analysts, and other stakeholders to ensure the availability, reliability, and scalability of our data systems, while promoting best practices in data governance and architecture. Some of the main responsibilities for the role include: Design and implement processes to ingest data from various sources into the Databricks Lakehouse platform, ensuring alignment with architectural and engineering standards. Develop, maintain, and optimize data models and ETL pipelines that support the Medallion Architecture (Bronze, Silver, Gold layers) to enhance data processing efficiency and facilitate data transformation. Utilize Databricks to integrate, consolidate, and cleanse data, ensuring accuracy and readiness for analysis, while leveraging Delta Lake for versioned data management. Collaborate with business analysts, data scientists, and stakeholders to understand their data requirements and deliver tailored solutions that leverage the capabilities of the Databricks Lakehouse platform. Engage with clients to support data engineering requirements, address technical queries, and provide guidance on best practices for utilizing the Databricks Lakehouse platform. Stay updated on industry trends and emerging technologies in data engineering, particularly those related to Databricks and cloud data solutions, continuously enhancing your skills and knowledge. Job Skills/Requirements - +90% English written and oral (at least B2 level) with excellent communication skills - Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or a related field. - Minimum of 5+ years of experience in data engineering or a related data role. - Proven experience in designing and implementing production-grade Spark-based solutions. - Proficient in query tuning, performance tuning, troubleshooting, and debugging Spark or other big data solutions. - Familiarity with big data technologies such as Spark/Delta, Hadoop, NoSQL, MPP, and OLAP. - Experience with cloud architecture, systems, and principles, particularly in AWS. - Proficient in programming languages such as Python, R, Scala, or Java. - Expertise in scaling ETL pipelines for performance and cost-effectiveness, with a focus on tuning queries for big data. - Familiarity with big data technologies such as Spark/Delta, Hadoop, NoSQL, MPP, and OLAP. - Experience with cloud architecture, systems, and principles, particularly in AWS. - Proficient in programming languages such as Python, R, Scala, or Java. - Expertise in scaling ETL pipelines for performance and cost-effectiveness, with a focus on tuning queries for big data. - Cloud certification is highly desirable. Skills and Attributes: - Strong problem-solving and troubleshooting skills. - Strong communicator, in both verbal and written form, able to articulate concepts and ideas, break through barriers, engage people, and work effectively with others under pressure. - Neutral toward technology, vendor, and product choices, more interested in results than in personal preferences. - Unflappable in the face of opposition to ideas We are Software Mind , an awesome team of engineers who are ready to ramp up any top-notch company’s projects! Our aim? To always be one step ahead. Become part of a multicultural company in constant growth with an excellent work environment certified by Great Place To Work! Originally posted on Himalayas