AgentMark – Markdown for the AI Era

Published at

Dec 31, 2024

Main Article

https://github.com/puzzlet-ai/agentmark

AgentMark

Markdown for the AI Era

Discord | Docs | Puzzlet

Overview

Develop type-safe prompts and agents using readable Markdown and JSX.

Features

AgentMark supports:

Markdown: 📝
Type Safety: 🛡️
Unified model config: 🔗
JSX components, props, & plugins: 🧩
Custom Models: 🛠️
Streaming: 🌊
Loops, Conditionals, and Filter Functions: ♻️
JSON Output: 📦
Tools & Agents: 🕵️
Observability: 👀

Read our docs to learn more.

Getting Started

Below is a basic example to help you get started with AgentMark:

example.prompt.mdx

---
name: basic-prompt
metadata:
  model:
    name: gpt-4o-mini
test_settings:
  props:
    num: 3
---

<System>You are a math expert</System>

<User>What's 2 + {props.num}?</User>

Models

By default, AgentMark doesn't support any model providers. Instead, support must be added through our plugins. Here's a list of currently supported plugins you can start using.

Built-In Model Plugins

Provider	Model	Supported	`@puzzlet/all-models`
OpenAI	gpt-4o	✅ Supported	✅
OpenAI	gpt-4o-mini	✅ Supported	✅
OpenAI	gpt-4-turbo	✅ Supported	✅
OpenAI	gpt-4	✅ Supported	✅
OpenAI	o1-mini	✅ Supported	✅
OpenAI	o1-preview	✅ Supported	✅
OpenAI	gpt-3.5-turbo	✅ Supported	✅
Anthropic	claude-3-5-haiku-latest	✅ Supported	✅
Anthropic	claude-3-5-sonnet-latest	✅ Supported	✅
Anthropic	claude-3-opus-latest	✅ Supported	✅
Meta	ALL	✅ Supported	🧩 Only
Custom	any	✅ Supported	🧩 Only
Google	ALL	⚠️ Coming Soon	N/A
Grok	ALL	⚠️ Coming Soon	N/A

Want to add support for another model? Open an issue.

Custom Model Plugins

Refer to our docs to learn how to add custom model support.

Language Support

We plan on providing support for AgentMark across a variety of languages.

Language	Support Status
TypeScript	✅ Supported
Others	Need something else? Open an issue

Running AgentMark

You can run AgentMark using any of the following methods:

1. VSCode Extension

Run .prompt.mdx files directly within your VSCode editor. Note: You can test props by using test_settings in your prompts.

Download the VSCode Extension

2. FileLoader

Run AgentMark files from your file system. Below is a sample implementation:

import { ModelPluginRegistry, FileLoader, createTemplateRunner } from "@puzzlet/agentmark";
import AllModelPlugins from '@puzzlet/all-models';

// Register models
ModelPluginRegistry.registerAll(AllModelPlugins);

// Create a file loader pointing to your prompts directory
const fileLoader = new FileLoader("./prompts", createTemplateRunner);

const run = async () => {
  // Load a prompt, relative to the file loader's root
  const mathPrompt = await fileLoader.load("math/addition.prompt.mdx");
  
  const props = {
    num1: 5,
    num2: 3
  }
  // Run the prompt
  const result = await mathPrompt.run(props);
  console.log("Run result:", result.result);

  // Compile to see the AgentMark configuration
  const compiled = await mathPrompt.compile(props);
  console.log("Compiled configuration:", compiled);

  // Deserialize to see raw model parameters (i.e. whats sent to the LLM: OpenAI, Anthropic, etc.)
  const deserialized = await mathPrompt.deserialize(props);
  console.log("Model parameters:", deserialized);
}
run();

3. Puzzlet Integration

Puzzlet is a platform for managing, versioning, and monitoring your LLM prompts in production, with built-in observability, evaluations, and prompt management.

import { Puzzlet } from '@puzzlet/sdk';
import { ModelPluginRegistry, createTemplateRunner } from "@puzzlet/agentmark";
import AllModelPlugins from '@puzzlet/all-models';

ModelPluginRegistry.registerAll(AllModelPlugins);

const puzzletClient = new Puzzlet({
  apiKey: process.env.PUZZLET_API_KEY!,
  appId: process.env.PUZZLET_APP_ID!,
}, createTemplateRunner);

const run = async () => {
  // Load prompt from Puzzlet instead of local file
  const prompt = await puzzletClient.fetchPrompt('math/addition.prompt.mdx');
  
  // Run the prompt
  const result = await prompt.run({
    num1: 5,
    num2: 3
  });
  console.log("Run result:", result);

  // Compile the prompt
  const compiled = await prompt.compile({
    num1: 5,
    num2: 3
  });
  console.log("Compiled configuration:", compiled);

  // Deserialize the prompt
  const deserialized = await prompt.deserialize({
    num1: 5,
    num2: 3
  });
  console.log("Model parameters:", deserialized);
}
run();

Type Safety

AgentMark & Puzzlet supports automatic type generation from your prompt schemas. Define input (input_schema) and output (schema) types in your prompt files:

---
name: math-addition
metadata:
  model:
    name: gpt-4o
    settings:
      schema:
        type: "object"
        properties:
          sum:
            type: "number"
            description: "The sum of the two numbers"
        required: ["sum"]
input_schema:
  type: "object"
  properties:
    num1:
      type: "number"
      description: "First number to add"
    num2:
      type: "number"
      description: "Second number to add"
  required: ["num1", "num2"]
---

<System>You are a helpful math assistant that performs addition.</System>

Then generate types using the CLI:

# From local files
npx @puzzlet/cli generate-types --root-dir ./prompts > puzzlet.types.ts

# From local Puzzlet server
npx @puzzlet/cli generate-types --local 9002 > puzzlet.types.ts

Use the generated types with FileLoader:

import PuzzletTypes from './puzzlet.types';
import { FileLoader, createTemplateRunner } from "@puzzlet/agentmark";

const fileLoader = new FileLoader<PuzzletTypes>("./prompts", createTemplateRunner);

// TypeScript will enforce correct input/output types
const prompt = await fileLoader.load("math/addition.prompt.mdx");
const result = await prompt.run({
  num1: 5,   // Must be number
  num2: 3    // Must be number
});
const sum = result.result.sum;  // type-safe number

Or with Puzzlet:

import PuzzletTypes from './puzzlet.types';
import { Puzzlet } from '@puzzlet/sdk';

const puzzlet = new Puzzlet<PuzzletTypes>({
  apiKey: process.env.PUZZLET_API_KEY!,
  appId: process.env.PUZZLET_APP_ID!,
}, createTemplateRunner);

// Same type safety as FileLoader
const prompt = await puzzlet.fetchPrompt("math/addition.prompt.mdx");
const result = await prompt.run({
  num1: 5,
  num2: 3
});
const sum = result.result.sum; // type-safe number

AgentMark is also type-safe within markdown files. Read more here.

Contributing

We welcome contributions! Please check out our contribution guidelines for more information.

Community

Join our community to collaborate, ask questions, and stay updated:

License

This project is licensed under the MIT License.

gittech. site