Principal Kafka Support & Reliability Engineer

Purple Drive Technologies

• $130K — $160K *

Canton, MA 02021In-Person

Information Technology

8 - 10 years of experience

1 month ago

By clicking Apply, I agree with Ladders' Terms of Use and Privacy Policy

Job Overview by Ladders

Qualifications

5-10 years of experience in Kafka support or reliability engineering.
Proven expertise in incident management and troubleshooting for Kafka environments.
Strong understanding of Kafka performance optimization and scalability risks.
Experience with platform stability, monitoring, and alerting enhancements.
Familiarity with AWS MSK and Confluent Cloud platforms.
Prior engagement in root cause analysis (RCA) with CAPA documentation.”],
responsibilities
: [”Act as the highest escalation point for critical Kafka production incidents.”,”Perform advanced troubleshooting for broker and partition issues.”,”Analyze Kafka workloads for performance and scalability risks.”,”Diagnose and resolve Kafka stability and resilience issues.”,”Lead configuration management and changes for Kafka environments.”,”Own and document root cause analysis for major incidents.”,”Collaborate closely with various teams for continuous improvement efforts.”],
benefits
: [”Opportunities for professional growth in a senior technical role.”,”Work in a collaborative environment with cross-functional teams.”,
Surroundings that prioritize innovation and continuous improvement.”] } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } }} } } } } } } } } } } } } } } } } } } } } ] } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } ] } }} } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } ] } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } ] } } } } }) } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } ] } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } ] ) } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } ] } } e > e > e > sne > sne > sne > sne > sne > sne > sne > sne > sne > sne > sne > sne > sne > sne > sne > sne > sne<|vq_1679|>{
:
} : { }
,
: } }
,
: } }
: } }
}
: }
} }
,
: }
; }
} : } }
}
.
: .
: . }
: {
,
. 2 }
.
: . . }
: VARIES
; } . }
; } }
,
: {
;
2 #
}
: . }
. } }
. $ }
~ ;
. ; ; , .
.
. ;
;
. 2
. 2 .
;
. ,
. .
. '
.
. .
;
,
$
$
; }
; } . , } }
;
.
;
,
. .
. ;
} .
,
. }
.
. 1 .
. ;
}
|
|
|
}
.
; # #
{
. }
; .
| .
}
} ;
;
| |
} . }
;
; .
| |
| |
; | |
| |
| |
} drinking . . .
|
| |
.
. | }
| |
; -
| }
.
| |:
} | |
. |
}
;
.
;
;
. |
, | |
.
# | |
| | |
;
}
; | |
. | |
| |
. ,
|
; | | | |
| | | | |
}
. | | . |
| |
| . |
|
}
. .
.
. |
| }
| |
;
} . | }
| |
| | |
; | . | |
| | . |
| |
. |
;
| |
; | ; | . | |
| | | | | |
}
|
; ; |
; | |
| | | _: | | .
| | . | ; | . | |
. |;
; | |
| | |
| |
|
. |
| |
; | | . |
;| ; | | | ; . | | | .
| |
| | |
| | ; |
} |
| |
| . | |
| | | ; | ;
. .
| |
| | ; |
| .
| |
}
| | | |
|
| |
| | .
| |
| | ;
| . | | | |
| ; | |
| | . | . | |
| | ;
| | | | | |
| | | .
| | |
| | |
| | ; | ; |
| | | |
;
; | |
| |
| ;
|
| | | | | | .
| | |
| | ; | ; | |
| ; | | |
} | | |
| | |
| | |
| ; | |
| . ,
| | . |
; ; | ; | | | | |
| | | | | |
| | | ; | ; |
;
| | .
| ; |
| ; | |
;
} | ; | |
| |
| |
| ; ; | | | | |
| ; | ; | |
| | | ;
| | | ; |
| ;
. |
|
| |
; | | . |
; | ; | | |
|;
| | | . |
|; |
.}
| ; | | | ; |
| | |
| ;
; | |;
; ; . |
| ; |
. | | | }
; | | ;
| |
; |
; ; |
| ; ; .
| | | | | | |
| |
; | | |
; | ; | ; ; |
|
; | | ;
|
| | | ` | ; | . |
; |
| | | | | ;
| |
; | | |
. . |
| . .
; | | |
|
| | . |
| ;
; | | | | | ;
; | | } |
| ; | | | | ; | |
| ;
; |
| | | |
| |
| ||=| . |
;
| | #{ |
. } ;
| ; |
| . . }
| | } | )
; | } |
; | ; ;
| | |
; } | ;
|
| |
/| |
| | | |
| | _
| | __
| |____
| ;
;=
| | | .
/
| | | | |
/ | /
|
|
|
| . |

Responsibilities

Act as the highest escalation point for critical Kafka production incidents.
Perform advanced troubleshooting for broker and partition issues.
Analyze Kafka workloads for performance and scalability risks.
Diagnose and resolve Kafka stability and resilience issues.
Lead configuration management and changes for Kafka environments.
Own and document root cause analysis for major incidents.
Collaborate closely with various teams for continuous improvement efforts.

Benefits

Opportunities for professional growth in a senior technical role.
Work in a collaborative environment with cross-functional teams.
Surroundings that prioritize innovation and continuous improvement.

Full Job Description

Overview:

Role: Principal Kafka Support & Reliability Engineer

Location: Canton, MA

Role Descriptions: Tier 3 Incident Management Escalation SupportAct as the highest technical escalation point for Kafka production incidents Sev 1 Sev 2.Lead deep troubleshooting across 1. Broker instability| controller elections| ISR shrinkage2. Under replicated partitions and leader imbalance3. Producerconsumer failures| lag spikes| and rebalance stormsDisk| network| JVM| and request handler saturationProvide hands on remediation for complex issues| including Partition reassignment and leader rebalanceBroker configuration tuningThrottlequota strategies for noisy producers or consumersCoordinate with vendor support during service incidents| providing logs| metrics| and forensic details.Guide Tier 2 teams during major incidents and validate restoration actions.2. Kafka Performance Engineering OptimizationAnalyze Kafka workloads for performance and scalability risks Partition skew and hot partitionsInefficient producer batchingcompressionConsumer lag root cause analysisThread pool| IO| and network bottlenecksRecommend and validate Topic design (partition count| replication factor| retention| compaction)Producer and consumer configuration best practicesQuotas| quotas enforcement| and multi tenant controlsSupport onboarding of high throughput or latency sensitive workloads| ensuring Kafka is correctly sized and tuned.3. Platform Stability| Reliability ResilienceDiagnose and resolve systemic Kafka stability issues Repeated broker failures or flappingMetadatacontroller instability (Zookeeper or KRaft)Recovery issues following failovers or maintenance eventsSupport resilience initiatives Multi AZ cluster health validationReplication and DR strategies (MirrorMaker 2| Replicator| or app level DR patterns)Failover testing and validationDefine and improve Kafka SLOs for availability| durability| and latency.4. Change| Upgrade Configuration LeadershipLead medium to high risk Kafka changes| including Broker and cluster configuration changesPartition expansion or large scale reassignmentTopic policy changes impacting durability or performanceSupport and plan Kafka version upgradesMSK Confluent upgrade cyclesClient compatibility and rollout strategiesParticipate in CAB reviews| assess risk| and design rollback and validation plans.5. Root Cause Analysis Continuous ImprovementOwn RCA documentation for major incidents with clear corrective and preventive actions (CAPA).Identify recurring failure patterns and architectural gaps.Recommend platform-level improvements Automation opportunitiesGuardrails and standardsMonitoring and alerting enhancementsContribute to continuous improvement of runbooks| knowledge base articles| and operational playbooks.

Essential Skills: Role OverviewThe Kafka Tier 3 Support Engineer is a senior technical role responsible for expert level support| advanced troubleshooting| performance engineering| and platform stabilization of enterprise Apache Kafka environments. This role functions as the final technical escalation point for Kafka-related production incidents and is accountable for root cause analysis (RCA)| complex remediation| and long term prevention. The engineer works closely with Tier 2 operations| Platform Engineering| SRE teams| application teams| and vendor support (AWS MSK Confluent Cloud providers) to ensure Kafka remains a highly reliable| scalable| and secure streaming backbone.

Desirable Skills:

Keyword:

Skills: Digital : Kafka~Digital: Amazon Connect~Digital : Kubernetes Experience Required: 10 & Above

* Ladders Estimates

Similar Jobs

Principal Software Engineer- GenAI
$136K — $219K *
Duck Creek Technologies
Boston, MA 02115 (Suffolk County)
Reposted Today
Principal Software Development Engineer
$144K — $288K *
CVS Health
Hartford, CT 06106 (Capitol County)
Reposted Today
Principal Software Engineer
$120K — $160K *
pubGENIUS
Remote
Reposted Today
Eng Prin - SW
$100K — $130K *
BAE Systems
Merrimack, NH 03054 (Hillsborough County)
Reposted Today
Java Solution Architect
$100K — $130K *
System One Holdings, LLC
Dallas, PA 18612 (Luzerne County)
Today
Principal Workday HCM and Security Consultant
$120K — $150K *
ERPA
Remote
Today

Get Ready For Your
Next Interview

More Jobs at Purple Drive Technologies

Senior Robotics Engineers
$120K — $150K *
Cupertino, CA 95014 (Santa Clara County)
Today
Manufacturing & Automotive
In-Person
Salesforce Architect
$130K — $160K *
Irvine, CA 92620 (Orange County)
Today
Enterprise Technology
In-Person
Salesforce Architect
$130K — $160K *
Irvine, CA 92620 (Orange County)
Today
Enterprise Technology
In-Person
Data Modeler
$100K — $130K *
Los Angeles, CA 90011 (Los Angeles County)
Reposted Today
Finance & Insurance
In-Person
ServiceNow Architect
$120K — $150K *
Malvern, PA 19355 (Chester County)
Today
Enterprise Technology
In-Person

More Information Technology Jobs

Business Development Director
$300K — $345K + $120K bonus *
Tier1 IT Services Firm
Kansas City, MO 64116 (Clay County)
6 days ago
Client Partner / Business Developemnt - Banking
$250K — $320K + $70K bonus *
IT Services Firm (client of TechLink Systems)
New York, NY 10001 (New York County)
6 days ago
Software Engineer II, Search & Data Infrastructure -Slack
$117K — $223K *
Salesforce
Washington, DC 20011 (District Of Columbia County)
Reposted Today
Software Engineer Lead
$55K — $158K *
The PNC Financial Services Group, Inc
Dallas, TX 75217 (Dallas County)
Reposted Today
Senior R&D Engineer-17637
$130K — $180K *
Synopsys Inc
Sunnyvale, CA 94087 (Santa Clara County)
Today

Find similar Principal Kafka Support & Reliability Engineer jobs:

Nationwide Canton, MA

Principal Kafka Support & Reliability Engineer

Job Overview by Ladders

Full Job Description

Get Ready For Your Next Interview

Find similar Principal Kafka Support & Reliability Engineer jobs:

Get Ready For Your
Next Interview