Posted at: 23 March

Databricks/ Pyspark Developer

Company

CompanyDATAMAXIS

DATAMAXIS is a Reston, Virginia-based B2B IT services provider specializing in strategic staffing, IT modernization, and data analytics solutions for enterprise clients.

Remote Hiring Policy:

DATAMAXIS primarily operates with an on-site workspace model, hiring from various locations including India and the United States, with some roles potentially offering remote work options.

Job Type

Full-time

Allowed Applicant Locations

United States, India

Apply Here

Job Description

Job Title: Databricks/ Pyspark Developer

Location: Remote

Job Summary: Looking for an offshore Senior Developer (for PMA and supporting EID/CJP) who has experience in Databricks/PySpark, is willing to learn new technologies if needed and is able to work with team. The developer will be mainly focusing on implementing the Delete Act changes needed for PMA . Along with that he will support the optimizations/enhancement for EID/CJP.

Essential Job Functions:

  • Design and development of data ingestion pipelines (Databricks background preferred).
  • Performance tune and optimize the databricks jobs.
  • Evaluates new features and refractors existing code.
  • Develop and integrate software applications using suitable development methodologies and standards, applying standard architectural patterns, taking into account critical performance characteristics and security measures.
  • Collaborate with Business Analysts, Technical Manager, Architects and Senior Developers to establish the physical application framework (e.g. libraries, modules, execution environments).
  • Perform end to end automation of ETL process for various datasets that are being ingested into the big data platform.
  • Must be willing to flex work hours accordingly to support application launches and manage production outages if necessary.
  • Works on best practices and documenting the process code merges and releases (Bitbucket).
  • Works with architect and manager on designs and best practices. Responsible for design of application considering the cost and best practices.
  • Good data analysis skills.
  • Must be willing to self learn new technologies, become SMEs and develop high quality code in a fast paced environment.
  • Mentor junior developers and be hands on in development work.
  • Work with QA and automation team. Must have attention to detail to cover all scenarios for testing.
  • Handles and supports current production applications.
  • Performs code reviews and discussion on the changes with Technology Manager.

Minimum Qualifications and Job Requirements:

  • Must be a team player.
  • Must have at least 5 years of IT development experience.
  • Must have strong analytical and problem-solving skills.
  • Must have experience in designing solutions, performing code reviews, mentoring junior engineers.
  • Must have strong SQL and backend experience, and working on data driven projects.

Must have the following experience: Python/PySpark, SQL, Databricks, SCALA, SQL, Spark/Spark Streaming, Big Data Tool Set, Linux, Kafka

Nice to have: Azure Data Factory

Apply Here