JOB OVERVIEWThe SIEM Data Engineer is responsible for the ingestion, parsing, normalization, and quality of data flowing into the Splunk platform across government agencies. This role owns the data pipeline - from onboarding new log and event sources to ensuring data is accurate, consistent, and normalized to a common schema so it is immediately usable for analysis and detection. The Data Engineer works closely with the Platform Engineer, Detection Engineer, and source-system owners to deliver reliable, high-quality data.
Essential Functions:- Data Ingestion and Source Onboarding
- Develop data ingestion strategies and onboard new log and event sources using inputs, forwarders, HTTP Event Collector (HEC), and API-based integrations.
- Configure and validate data inputs across diverse log and event data types.
- Parsing and Field Extraction
- Develop and maintain parsing configurations (props.conf, transforms.conf) including field extractions, event breaking, and timestamp recognition.
- Build and tune regular expressions to accurately extract fields from structured and unstructured data.
- Data Normalization
- Design and implement data normalization to the Splunk Common Information Model (CIM) for consistent, cross-source analysis.
- Ensure data conforms to standardized schemas that support downstream reporting and detection.
- Data Quality and Monitoring
- Monitor data feeds for gaps, latency, volume anomalies, and parsing errors, and remediate issues quickly.
- Establish data validation and quality checks to maintain accuracy and completeness.
- Source Integration
- Integrate data from on-premises, cloud, and third-party systems, coordinating with source owners on formats and connectivity.
- Partner with the Platform Engineer on index and sourcetype design to optimize storage and searchability.
- Data Lifecycle and Compliance
- Apply retention, masking, and handling requirements to ingested data in line with relevant standards and regulations.
- Support compliance and audit needs related to data provenance and integrity.
- Documentation
- Maintain documentation of data sources, onboarding procedures, parsing logic, and data flows.
- Create runbooks to support consistent, repeatable data onboarding.
Qualifications:- US Citizen with an active Top Secret Security Clearance.
- Bachelor's degree in Computer Science or a related field plus 6 years of related experience.
- Proven experience with Splunk data onboarding, parsing, and normalization, preferably within government or enterprise environments.
- Hands-on experience with props.conf/transforms.conf, field extractions, and the Splunk Common Information Model (CIM).
- Strong proficiency with regular expressions and a solid understanding of log formats (syslog, JSON, CSV, Windows Event Log, cloud audit logs, etc.).
- Understanding of data pipeline concepts, data quality, and log management principles.
- Excellent problem-solving skills and the ability to diagnose and resolve data quality and ingestion issues.
- Strong written and verbal communication skills.
DESIRED:- Data onboarding via inputs, universal/heavy forwarders, HEC, and API/scripted inputs
- Parsing and event processing: props.conf/transforms.conf, event breaking, timestamp recognition, line merging
- Field extraction and strong regular expression proficiency
- Data normalization to the Splunk Common Information Model (CIM)
- Sourcetype, index, and knowledge object design in coordination with the platform team
- Data quality monitoring: gap/latency detection, volume anomaly identification, and remediation
- Deep familiarity with common log formats (syslog, JSON, CSV, Windows Event Log, cloud audit logs, network/EDR telemetry)
- Data pipeline and integration experience across on-prem, cloud, and third-party sources
- Scripting (Python and/or Bash) for automation and data validation
PHYSICAL REQUIREMENTS & ENVIRONMENTAL CONDITIONS- Inside office environment.
- Working on a computer for long periods of time.
- May involve long period of sitting at a desk.
- The work environment is fast-paced and sometimes involves extreme deadline pressures.
OTHER DUTIESThis job description is not designed to cover or contain a comprehensive listing of activities, duties or responsibilities that are required of the employee for this job. Duties, responsibilities, and activities may change at any time with or without notice.
