What you’ll do 

• You will be a founding data engineer responsible for finding the best cloud solution to solve data storage, processing, and orchestration while keeping the cost in check.
• You will design, build, and deploy batch and real-time data pipelines, storage, and model schemas. This will involve leveraging performance tuning techniques, conceptual schemas, and modern technology.
• You will assist the team in selecting a tool to manage automated workflows and a tool to manage the data itself.
• You will be responsible for identifying, defining, protecting, and managing data from its raw form to its consumption. You will collaborate with product teams, data analysts, data scientists, and technology teams.

Responsibility 

• Design, develop, and maintain critical data infrastructure, datasets, and pipelines.
• Ensure data is stored safely and securely, adhering to frequently changing regulations (e.g. GDPR) and best practices for user data storage and security.
• Take ownership of critical data pipelines, manage their SLAs, and constantly improve pipeline efficiency and data quality.
• Facilitate data integration and transformation requirements for moving data between applications, ensuring interoperability with database, data warehouse, and data mart environments.
• Assist in designing and managing the technology stack used for data storage and processing.

• Possess over 5 years of hands-on experience in Data Engineering, specializing in the development of scalable storage solutions and robust schema layers.
• Proficient in a programming language such as Python or Java, along with their respective standard data processing libraries.
• Demonstrated expertise in crafting and troubleshooting data pipelines utilizing distributed data frameworks like Apache Spark and Flink etc.
• Extensive background in working with relational databases (AWS, RDS, Aurora), adept in SQL, data warehousing, and proficient in designing ETL/streaming pipelines.
• Proven track record of integrating data from core platforms into a centralized warehouse or datalake.
• Adhere to rigorous standards in code quality, implement automated testing, and champion other engineering best practices.
• Well-versed in establishing secure systems and access models for handling highly sensitive data.
• Exhibit strong cross-functional communication skills, proficient in extracting requirements, and skilled in architecting shared datasets.
• Possess a genuine passion for creating exceptional tools that provide a delightful user experience.

Skills

• Data Engineering,Data pipeline,Data Integration,Python,SQL,Relational Databases