Who We Are
Checksum is a high-growth, startup revolutionizing software testing with AI-powered, self-healing QA automation.Checksums platform eliminates the need for manual test maintenance by automatically generating and adapting tests based on real user behavior. Engineering teams can now release code faster without sacrificing qualityand our traction proves it: weve tripled ARR in the last 9 months. We are backed by Leap Global Partners and were formed at super{set}, whose co-founders have over $1B in exits to Microsoft and Salesforce.
Were looking for a builder who ships. You have hands on experience building with LLMs, shown through shipped features or serious prototypes. You have worked with prompts, function calling, tool use, structured outputs, evals and failure handling You turn ambiguous problems into iterative plans, wire up tools and feedback loops, and measure what matters. You are comfortable moving between prompt design, Python code, and product tradeoffs. You enjoy pairing with engineers and PMs, mentoring teammates, and owning outcomes in a dynamic startup environment.
This is a hybrid role, 4x a week, in the San Francisco Office
What Youll Do
- Design, build and operate AI agents that plan, act, observe and learn across browser, API and data tools
- Integrate models with tools such as Playwright, internal APIs, vector search, document stores and webhooks
- Create evaluation harnesses, golden datasets and automated regressions for reliability and safety
- Optimize latency, accuracy and cost using caching, routing, function calling, tool selection and streaming I/O
- Collaborate on product UX for agent feedback, interruptions, approvals and failure recovery
- Own the full lifecycle from prototype to production, including CI, experiments and on call rotations
What Were Looking For
- BSc in computer science or equivalent experience
- Ability to work independently, and solve problems based on clear goals
- Strong Python or TypeScript Skills
- Experience building evals, datasets and automated tests for ML or LLM systems
- Experience building agents that control browsers with Playwright or Selenium
- Knowledge of CI/CD, containerization and cloud deployments
- Exposure to retrieval augmented generation, planning and memory, multi agent orchestration and toolformer style designs
- Experience with Anthropic, OpenAI, Google and open source models such as Llama, plus model selection and routing
- Background in testing automation, QA engineering or developer tools
Benefits & Perks
- A strategic role in shaping the growth trajectory of a cutting-edge AI startup
- Competitive salary, equity, and benefits package
- Lunch everyday you are in the office
- Monthly company happy hours
