AI Product Engineer

About ClickHouse

Recognized on the Forbes Cloud 100 list for 2025, ClickHouse is a rapidly expanding private cloud company excelling in real-time analytics, data warehousing, observability, and AI workloads, serving over 4,000 customers with remarkable annual revenue growth exceeding 250%.

Having recently secured $400M in Series D funding, ClickHouse counts major clients like Capital One, Lovable, Decagon, Polymarket, Airwallex, Meta, Cursor, Sony, and Tesla among its user base. Our mission is to revolutionize data utilization across industries.

Role Overview

Join ClickHouse in advancing the ClickStack platform—an open-source observability tool uniting logs, metrics, traces, and session replays to empower engineers to diagnose root causes quickly. We're focusing on building intelligent agent layers capable of analyzing incidents autonomously, summarizing issues, and assisting on-call engineers efficiently.

We seek an AI Product Engineer to design and develop agent capabilities over a petabyte-scale observability platform emphasizing excellent developer experience.

Responsibilities

• Develop agents that autonomously investigate incidents by detecting anomalies, explaining production failures, and utilizing ClickStack as a foundation.
• Create reusable skill libraries that encode debugging procedures, root cause analysis, ClickHouse query crafting, and incident response workflows, enabling agents to follow established playbooks.
• Manage the entire agent stack including context engineering, tool design, evaluation processes, tracing, and operational costs to ensure reliable production performance.
• Enhance ClickStack’s capacity for AI workloads by building MCP servers, SDKs, and integrations to allow agents to process telemetry data, act upon it, and maintain observability.
• Collaborate openly with open-source contributors and customers, collaboratively troubleshooting and incorporating feedback into product improvements.
• Address technical challenges such as latency reduction, cost optimization, managing context window constraints, improving eval coverage, and minimizing hallucinations on real telemetry data.

Candidate Profile

• Extensive experience building intelligent agents with deep understanding of context engineering, tool and skill design, evaluation methodology, and framework limitations.
• Operational mindset focused on production metrics including p99 latency, cost per task, and system reliability without manual intervention.
• Rapid deployment habit with a feedback-driven approach to learning from failures.
• Strong commitment to developer experience and robust developer tooling.
• Comfortable managing ambiguity and taking end-to-end ownership.

Required Qualifications

• Minimum of 5 years software engineering experience with 1-2 years working on large language model-powered systems or production AI agents.
• Proficiency in backend development using TypeScript/Node.js and/or Python, with capability to work in both languages.
• Practical experience designing and shipping agents featuring multi-step tool integration, planning, memory management, and error recovery, including handling failure modes.
• Expertise in designing skills using Markdown workflow encodings or Anthropic-style approaches with a clear understanding of when to implement skills versus tools.
• Experience building MCP servers and designing tools considering authentication, scoping, and observability aspects of agentic systems.
• Strong evaluation practices including constructing golden datasets, LLM-judged evaluation, and regression identification.
• Ability to write advanced SQL queries specifically for ClickHouse.
• Hands-on familiarity with containerization using Docker and orchestration with Kubernetes.
• Active participation in open source and developer community activities.

Preferred Qualifications

• Experience building or managing production agents in observability, incident response, or site reliability engineering.
• Well-formed opinions on agent observability including techniques such as tracing, cost attribution, evaluation pipelines, and OpenTelemetry integration.
• Knowledge of prompt caching, context compaction, or similar methods to efficiently run agents against production telemetry volumes.
• Experience with columnar databases and processing event ingestion streams.
• Contributions to or maintenance of open source AI or agent-related projects.
• Familiarity with systems programming languages such as Go or Rust for integrations or high-throughput infrastructure tasks.

Compensation Information

Salary ranges vary by location with specified minimum and maximum values reflecting good-faith estimates at time of posting. Actual pay will depend on qualifications, experience, performance, and local market conditions. Questions regarding compensation can be addressed via direct contact.

Employee Benefits

• Flexible and globally distributed work environment with remote work options across over 20 countries.
• Employer healthcare contributions.
• Stock options awarded to all new employees.
• Flexible paid time off policies and significant leave allowances depending on country.
• $500 allowance for home office setup for remote employees.
• Company-wide in-person gatherings to encourage team bonding and collaboration.

Culture and Inclusion

As a fast-growing startup, team members will help actively shape company culture and values. We are committed to equal opportunity employment and provide a discrimination-free workplace respecting diversity and inclusion.