16400 - Sr. Data Engineer.
Our client is one of largest retailers in the world. Yet their IT department is nimble and dynamic and works with cutting edge technologies. The Information Technology Department has an opening for a Data Engineer. This position is in Pleasanton, CA.
Work on various projects to optimize data collection, integration, aggregation, analysis, and data visualizations that support business objectives in a BigData Cloud environment.
Key Responsibilities include,
Work in conjunction with other augmented staff to ingest data from varying technologies including internal and external sources. Incremental and historical data are to be ingested with an emphasis on streaming and a framework – Content Integration Network (CIN) will be developed with a goal to:
- Create re-usable components
- Auditability and logging
- Governance and Security
- Standards for DevOps manageability
- Data Quality integrated
- Configuration driven
- 2-5+ years proven experience in developing and deploying in the Cloud; Azure and Snowflake experience is a plus
- 2-5+ years proven expertise in creating pipelines for real time and near real time integration
- 2+ years proven experience with Spark SQL, Spark Streaming and using Core Spark API to explore Spark features to build data pipelines
- Databricks and Delta table knowledge is a plus
- Extensive experience in data transformations for various business use cases.
- Knowledge for handling exceptions and automated re-processing and reconciling
- Passion for Data Quality with an ability to integrate these capabilities into the deliverables
- Prior use of Big Data components and the ability to rationalize and align their fit for a business case
- Experience in working with different data sources - flat files, XML files and databases
- Proficiency in techniques for slowly changing dimensions
- Knowledge of Jenkins for continuous integration and End-to-End automation for application build and deployments
- Proven experience and ability to work with people across the organization
- Proven capabilities for strong written and oral communication skills
- Ability to integrate into a project team environment and contribute to project planning activities
- Experience in developing implementation plans and schedules and preparing documentation for the jobs according to the business requirements
- W2 candidates only, Local to Bay Area, CA candidates only
- Position location: Pleasanton, CA
Since 2002 Maxonic has been at the forefront of connecting candidate strengths to client challenges. Our award winning, dedicated team of recruiting professionals are specialized by technology, are great listeners, and will seek to find a position that meets the long term career needs of our candidates. We take pride in the over 5,000 candidates that we have placed, and the repeat business that we earn from our satisfied clients.
Interested in Applying?
Please apply below with your most current resume. Feel free to email Shreya (email@example.com) or call 408-400-2230