LangSmith is an all-in-one developer platform for every step of the LLM-powered application lifecycle, whether you’re building with LangChain or not.
Debug, collaborate, test, and monitor your LLM applications.
LLM-apps are powerful, but have peculiar characteristics. The non-determinism, coupled with unpredictable, natural language inputs, make for countless ways the system can fall short. Traditional engineering best practices need to be re-imagined for working with LLMs, and LangSmith supports all phases of the development lifecycle.
Unexpected results happen all the time with LLMs. With full visibility into the entire sequence of calls, you can spot the source of errors and performance bottlenecks in real-time with surgical precision. Debug. Experiment. Observe. Repeat. Until you’re happy with your results.
Building LLM-powered applications requires a close partnership between developers and subject matter experts.
Traces
Easily share a chain trace with colleagues, clients, or end users, bringing explainability to anyone with the shared link.
Hub
Use LangSmith Hub to craft, version, and comment on prompts. No engineering experience required.
Annotation Queues
Try out LangSmith Annotation Queues to add human labels and feedback on traces.
Datasets
Easily collect examples, and construct datasets from production data or existing sources. Datasets can be used for evaluations, few-shot prompting, and even fine-tuning.
Layer in human feedback on runs or use AI-assisted evaluation,
with off-the-shelf and custom evaluators that can check for relevance, correctness, harmfulness, insensitivity, and more.
Quickly save debugging and production traces to datasets. Datasets are collections of either exemplary or problematic inputs and outputs that should be replicated or rectified, respectively.
Use an LLM and prompt to score your application output, or write your own functional evaluation tests to record different measures of effectiveness.
See how performance of the evaluation criteria that you’ve defined is affected by changes to your application. Know that quality is moving in a positive direction by applying engineering rigor to your testing workflow.
Continuously track qualitative characteristics of any live application, and spot issues in real-time with LangSmith monitoring.
See what’s happening with your production application, so you can take action when needed or rest assured while your chains and agents do the hard work.
Yes! Many companies who don’t build with LangChain/LangGraph use LangSmith. You can log traces to LangSmith via the Python SDK, the TypeScript SDK, or the API. See here for more information.
Getting started on LangSmith requires just two environment variables in your LangChain or LangGraph code. See how to send traces from your LangGraph agent or your LangChain app.
Yes, you can log traces to LangSmith using a standard OpenTelemetry client to access all LangSmith features, including tracing, running evals, and prompt engineering. See the docs.
LangSmith traces contain the full information of all the inputs and outputs of each step of the application, giving users full visibility into their agent or LLM app behavior. LangSmith also allows users to instantly run evals to assess agent or LLM app performance — including LLM-as-Judge evaluators for auto-scoring and the ability to attach human feedback. Learn more.
Yes, we allow customers to self-host LangSmith on our enterprise plan. We deliver the software to run on your Kubernetes cluster, and data will not leave your environment. For more information, check out our documentation.
For Cloud SaaS, traces are stored in GCP us-central-1 or GCP europe-west4, depending on your plan. Learn more.
No, LangSmith does not add any latency to your application. In the LangSmith SDK, there’s a callback handler that sends traces to a LangSmith trace collector which runs as an async, distributed process. Additionally, if LangSmith experiences an incident, your application performance will not be disrupted.
We will not train on your data, and you own all rights to your data. See LangSmith Terms of Service for more information.
See our pricing page for more information, and find a plan that works for you.
Get started with LangChain, LangSmith, and LangGraph to enhance your LLM app development, from prototype to production.