Careers

Data Engineer

Chantilly, VA

Job Responsibilities:

Primary responsibility is to work with data providers within the IC and DoD Enterprise to identify and ingest data sets into the data broker. In this role you will:

  • Develop, optimize, and maintain data ingest flows using Apache Nifi and Python.

  • Develop within the components in the cloud platform, such as Apache Kafka, NiFi, and HBase.

  • Communicate with data owners to set up and ensure streaming and batching components are working (including configuration parameters).

  • Document SOP related to streaming configuration, batch configuration or API management depending on role requirement.

  • Document details of each data ingest activity to ensure they can be understood by the rest of the team


Desired Skills:

  • A minimum of 3 years of experience with programming and software development including analysis, design, development, implementation, testing, maintenance, quality assurance, troubleshooting and/or upgrading of software systems

  • DoD 8570 IAT Level II Certification (e.g. Security+)

  • Demonstrable CentOS command line knowledge

  • Working knowledge of web services environments, languages, and formats such as RESTful APIs, SOAP, FTP/SFTP, HTML, JavaScript, XML, and JSON

  • Understanding of foundational ETL concepts

  • Experience implementing data ignorations with in the IC DoD Enterprise

  • Experience or expertise using, managing, and/or testing API Gateway tools and Rest APIs (desired)

  • 2+ Experience in Python Development

  • Experience or expertise configuring an LDAP client to connect to IPA (desired)

  • Advanced organizational skills with the ability to handle multiple assignments

  • Strong written and oral communication skills


Years of Experience: 5 years of experience or more.

Education: Bachelor's degree in systems engineering, computer engineering, or a related technical field

Location: Chantilly, VA

Clearance: Active TS/SCI w/ ability to obtain CI Poly

If you are unable to apply using the button above, please email hiring@crystalintel.com