3e85fd1b 6af0 466b B8f5 5fe34d897df0
Dice
- Location
- Austin, TX · On-site
- Employment type
- Contract · Mid level
- Salary
- $3 – $4/yr
- Posted
- Posted 1 month ago
Freshness: We could not reach the source listing on the last check. (last checked 1 month ago) · Source: firecrawl
Applying takes you to the employer or source website.
About this role
Seeking a Big Data Engineer to develop, maintain, and optimize scalable data solutions using Databricks, Apache Spark, Azure Data Factory, Delta Lake, and Azure Data Lake Storage. The consultant will build ETL/ELT pipelines, optimize Spark workloads, automate deployments, and maintain data quality, governance, and security. Required Experience 4+ years with ETL/ELT, Python, R, SQL, Azure, CI/CD, DevOps, data modeling, data lakes, and data warehouses. 3+ years with Databricks, Apache Spark, Spark SQL, Delta Lake, Azure Data Factory, and Unity Catalog. Strong understanding of Spark architecture, RDDs, DataFrames, clusters, notebooks, and jobs. Experience implementing data validation, metadata management, lineage, cataloging, encryption, access controls, and auditing. Strong troubleshooting, Agile delivery, and cross-functional collaboration skills. Preferred MLflow, Scikit-learn, or TensorFlow experience. Databricks Certified Associate Developer for Apache Spark. Microsoft Azure Data Engineer Associate certification. Important Requirements Selected candidates must pick up equipment in Austin on the first day and may be required to attend onsite meetings with minimal notice. --
Skills
- Python
- Sql
- Databricks
- Spark
- Azure
- Ci/Cd
- Tensorflow
- Agile
- Devops