Laminar

Laminar is a unified open-source platform for tracing, evaluating, and labeling LLM products. It provides visibility into execution steps of your LLM applications, helps collect data for evaluations, few-shot examples, and fine-tuning, and offers a scalable way to build and deploy LLM pipelines. It is designed to be easy to integrate with minimal overhead and fully open-source for self-hosting.

---

How Laminar works

• Start tracing your LLM app with just 2 lines of code. Traces are collected in the background and sent via gRPC with minimal performance and latency impact.
• Online evaluations let you set up evaluators to automatically label spans, enabling scalable data generation and feedback beyond manual labeling.
• Build and deploy datasets from traces for evaluations, fine-tuning, and prompt engineering. Enhance prompts by retrieving semantically similar examples from indexed datasets.
• A pipeline builder lets you prototype and iterate on prompts and LLM chains, supporting both simple and complex workflows.
• Laminar is fully open-source and can be self-hosted with a few commands.

---

Getting started

• Start tracing your LLM app with two lines of code:
• TypeScript: import { Laminar, observe } from '@lmnr-ai/lmnr'
• Laminar.initialize({ projectApiKey: "..." }) or manually trace a function: const myFunction = observe({ name: 'myFunc' }, async () => { ... })
• Run your application and observe traces in the Laminar dashboard or your preferred backend.

---

Online evaluations

• Setup LLM or Python online evaluators to process each received span.
• Evaluators automatically label spans, offering a scalable alternative to human labeling.

---

Dynamic few-shot examples & datasets

• Create datasets from traces for evaluations, fine-tuning, and prompt engineering.
• Enhance prompts by retrieving semantically similar examples from indexed datasets.

---

Serverless LLM pipelines

• Our pipeline builder is an incredible prototyping tool that lets you quickly build and iterate on both simple prompts and complex LLM chains.
• After prototyping, you can deploy LLM pipelines to production with minimal friction.

---

Open-source & self-hosted

• Laminar is fully open-source and designed for easy self-hosting.
• Get started with just a few commands and host on your own infrastructure.

---

Core Features

• Lightweight, two-line code tracing for LLM calls with minimal overhead
• Automatic tracing across common LLM frameworks and SDKs
• Online evaluations with automatic span labeling for scalable data generation
• Dataset creation from traces for evaluations, fine-tuning, and prompt engineering
• Semantic retrieval of similar examples to enhance prompts
• Serverless or self-hosted deployment options
• Open-source and easy to self-host
• Pipeline builder for rapid experimentation with prompts and LLM chains
• Observability and visibility into every execution step of LLM apps