Loại hình
Toàn thời gian
Hình thức
Tại văn phòng
Cấp bậc
Nhân viên
Ngành nghề
Kỹ thuật / Cơ khí
Phòng ban
VinSmart Future
Mức lương
Thương lượng
Địa điểm
Thành phố Hồ Chí Minh, Hồ Chí Minh
Tổng quan
- About the Role
- We are looking for a Mid/Senior Data Engineer who wants to build and lead systems, not just pipelines. You will design and develop the core components of our Data Platform: the Spark-based processing layer, ingestion frameworks, orchestration standards, data quality and observability. The goal is a platform where analysts can onboard sources, transform and create data marts on their own, while…
- The goal is a platform where analysts onboard new sources and build data marts on their own. The Data Engineering team focuses on the capabilities that make the platform faster, safer and easier to operate. As a senior member, you will set platform standards, make key design decisions and mentor other engineers.
- Platform architecture: Lead the end-to-end design of the Data Platform. Define standards for the bronze, silver and gold layers, data contracts and failure handling, and drive technical decisions across the team.
- Processing layer: Design and build the Spark-based ETL/ELT framework: reusable Spark job templates, transformation libraries, incremental and backfill patterns.
- Ingestion framework: Build a reusable ingestion layer (batch and CDC/streaming) so that new sources (SAP, event tracking, HRM, CRM, IoT and internal apps, ...etc) are onboarded through flexible configuration rather than custom code.
- Orchestration as a product: Own Airflow as a platform: DAG templates, generators, CI/CD and deployment standards so analysts can set up data marts self-service.
- Performance and quality: Tune Spark jobs and ClickHouse data layout to keep processing time and storage costs predictable. Build quality checks, reconciliation, lineage and alerting so SLAs and failures are visible platform wide.
- APIs, security and enablement: Develop the REST API layer for internal apps and AI use cases, implement access control and PII masking, and write internal libraries and documentation that reduce effort for analysts.
- Technical leadership: Mentor engineers through design and code reviews, raise engineering standards and partner with analysts and business stakeholders on platform priorities.
Yêu cầu
- Tech Stack
- Processing: Apache Spark, Scala, Python, SQL
- Orchestration: Apache Airflow, Git, CI/CD, Docker
- Storage: ClickHouse, S3, Parquet, Delta / Iceberg
- Ingestion: Kafka / Debezium, REST API / JDBC
- 5+ years for Senior level of hands-on data engineering experience, including a track record of designing and building reusable ETL frameworks or platform components rather than only individual pipelines.
- Strong production experience with Apache Spark (Scala or Python): job design, partitioning, shuffle behavior, incremental processing, and debugging slow or failing jobs.
- Strong software engineering in Scala or Python (modular code, packaging, testing) and strong SQL, with a solid understanding of columnar storage and open table formats (Parquet, Delta or Iceberg). ClickHouse experience preferred.
- Production experience with Airflow beyond writing DAGs: templating, dynamic generation, deployment, backfills, dependencies scheduling and alerting. Comfortable with Docker, Git and CI/CD.
- Experience integrating data from SAP, event tracking systems (clickstream, app events) or external APIs, including schema evolution, late or duplicate data, and CDC patterns.
- Solid data modeling skills (dimensional modeling, star schema, SCDs) and experience structuring data by layer (bronze/silver/gold: raw, cleansed and conformed, business-ready) to serve datamarts and analytics.
- System design mindset: you think in interfaces, contracts and failure modes, and prefer building a capability once over solving the same problem ten times.
- Senior candidates are expected to prove ability to lead architecture, define platform standards and mentor engineers.
- Nice to Have
- Metadata and lineage tools (DataHub, OpenMetadata...) or dbt
- Spark Structured Streaming or Flink
- Kubernetes, with Terraform on AWS/GCP
- What Success Looks Like
- New sources are onboarded through configuration, not custom code.
- Analysts build and deploy their own datamarts on standard Airflow templates.
- Pipeline SLAs, data quality and lineage are visible across the platform, and failures are caught before users notice.
- The whole Data Platform costs and run times stay predictable as data volume grows.
- The team works to shared platform standards that you helped define.
Quyền lợi
- n/a
Tóm tắt thông tin từ tin tuyển dụng chính thức. Xem bản gốc ↗
Quan tâm đến vị trí này?
Bạn sẽ được chuyển đến trang ứng tuyển chính thức của nhà tuyển dụng.
Ứng tuyển tại trang chính thức ↗
Đây là doanh nghiệp của bạn?
Nhận quản lý trang, yêu cầu chỉnh sửa hoặc gỡ bỏ
→