Lead Data Engineer, Data Mgmt Optimization - Shopping (Remote)
About the position The Lead Data Engineer for Data Management Optimization at Capital One Shopping will play a crucial role in a dynamic remote-first engineering team. This position focuses on building technology that saves customers money by working on data platforms that handle billions of rows of data daily. The role involves collaboration with Agile teams to design, develop, and support technical solutions for data analytics and management, utilizing various programming languages and data platforms. Responsibilities • Collaborate with Agile teams to design, develop, test, implement, and support technical solutions for data analytics and management at scale. • Work with a team of developers experienced in machine learning, distributed microservices, and full stack systems. • Utilize programming languages like Java, Go, and Python along with data platforms such as Trino, Hive, and Kafka. • Stay updated on tech trends, experiment with new technologies, and mentor other engineering community members. • Collaborate with digital product managers to deliver robust cloud-based solutions that enhance customer financial empowerment. Requirements • Bachelor's Degree • At least 4 years of experience in application development (Internship experience does not apply) • At least 2 years of experience in big data technologies • At least 1 year of experience with cloud computing (AWS, Microsoft Azure, Google Cloud) Nice-to-haves • 5+ years of experience in application development including Java, Go, or Python • 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) • 4+ years of experience with distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark) • 4+ years of experience working on real-time data and streaming applications • 3+ years of experience with NoSQL implementation (Mongo, Cassandra) • 3+ years of experience with UNIX/Linux including basic commands and shell scripting • 2+ years of data warehousing experience (Redshift or Snowflake) • 2+ years of experience with Agile engineering practices Benefits • Comprehensive health benefits • Financial benefits including performance-based incentives • Inclusive workplace policies • Support for total well-being Apply tot his job