Data Engineer

M1 Technology

Responsibilities

Maintain and execute data extraction pipelines to transport data in and out of multiple environments.
Enrich existing datasets using external and under-represented data sources.
Integrate and manage data catalog capabilities for discoverability and governance.
Troubleshoot and resolve issues related to data ingestion, integration, and processing.
Develop and enhance scripts and tools using Python and PySpark to process structured (SQL) and unstructured (JSON, XML) data.
Interface with cloud-based storage solutions (object repositories, RDBMS) to support analytics platforms.
Manage data acquisition via secure HTTPS requests, direct DB connections, and standalone file transfers.

Bachelor’s degree in Computer Science, Engineering, or related field; or equivalent experience.
5+ years of software development experience with at least one of: Python, Java, C#, JavaScript, or Ruby (Python strongly preferred).
5+ years of experience working with databases and applications in classified IC environments.
Extensive experience designing and maintaining SQL-based data systems and crafting complex queries.
Proficiency in generating and interpreting technical documentation (e.g., ERDs, schemas, architecture diagrams).

Familiarity with IC metrics and assessments: alignment with intelligence needs, responsiveness, and collection discipline utilization.
Experience building tools to assess and visualize large-scale datasets.
Proven ability to design and implement secure APIs for data access and integration.

In-depth knowledge of IC cloud environments and C2E cloud services.
Experience integrating data systems with visualization and analytics platforms (Tableau, PowerBI, Kibana, Grafana, etc.).

M1 Technology is an equal opportunity employer and values diversity.
We do not discriminate in hiring on the basis of race, color, religion, sex, national origin, age, disability, veteran status, or any other characteristic protected by federal, state, or local law.