- Work closely with business stakeholders to understand data requirements and develop solutions to meet their needs.
- Develop and maintain ETL (Extract, Transform, Load) processes for large and complex data sét in both commercial cloud (AWS or Azure) and on-premise environments.
- Ensure the integrity and accuracy of our data by implementing appropriate data validation and quality control measures.
- Collaborate with the Data Engineer Lead, Solution Architect to design and implement data architectures that are scalable, reliable, and secure.
- Monitor data transformation processes and identify opportunities for optimization and improvement.
- Stay up to date with the latest data transformation technologies and methodologies and evaluate their suitability for our organization.
Mô tả công việc
Quyền lợi được hưởng
- Income: Negotiation
- REMOTE
- The company has its own bonus mechanism, including many amounts: year-end bonuses, project bonuses, hot bonuses,...
- Be trained to improve knowledge and skills in IT, soft skills and professional qualifications
Yêu cầu công việc
- Bachelor's degree in Computer Science, Engineering, or a related field
- 5+ years of experience in data engineering, with a focus on data transformation using Python, Spark, or Scala.
- Strong proficiency in Python and related data transformation libraries, such as Pandas, Numpy, and Scipy.
- Proven experience in designing and developing scalable and reliable data pipelines.
- Strong knowledge of distributed computing frameworks such as Apache Spark is a bonus.
- Knowledge of cloud-based data platforms such as AWS or Microsoft Azure.
- Familiarity with data security and governance best practices.
- Excellent communication and collaboration skills.
- Ability to work independently and as part of a team.