Skip to content

Commit 3dd7669

Browse files
rabisgabhithesysAditya-thesysankit-thesysPrakhar
committed
Migrate to openui
This commit migrates the project to openui and replaces existing thesysdev/crayon with a new Language Specification (@openuidev/openui-lang), React Runtime (@openuidev/lang-react) alongwith new Headless Chat Runtime (@openuidev/react-headless) plus a DS (@openuidev/react-ui) Co-authored-by: Abhishek <abhishek@thesys.dev> Co-authored-by: Aditya <aditya@thesys.dev> Co-authored-by: Ankit <ankit@thesys.dev> Co-authored-by: Prakhar <prakhar@thesys.dev> Co-authored-by: Subham <subham@thesys.dev>
1 parent 65216f3 commit 3dd7669

1,386 files changed

Lines changed: 84148 additions & 53023 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.cursor/rules/styling-rule.mdc

Lines changed: 287 additions & 261 deletions
Large diffs are not rendered by default.

.cursor/rules/use-pnpm.mdc

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,13 @@
11
---
2-
description: Comprehensive pnpm package manager guidelines for the Crayon monorepo
3-
globs: package.json,*.json,*.toml
4-
alwaysApply: true
2+
description: Comprehensive pnpm package manager guidelines for the OpenUI monorepo
3+
alwaysApply: false
54
---
65

7-
# Crayon Package Management with pnpm
6+
# OpenUI Package Management with pnpm
87

98
## Overview
109

11-
Crayon uses **pnpm** as its package manager instead of npm. pnpm provides better performance, disk efficiency, and strict dependency management. This document covers all pnpm usage patterns for the Crayon monorepo.
10+
OpenUI uses **pnpm** as its package manager instead of npm. pnpm provides better performance, disk efficiency, and strict dependency management. This document covers all pnpm usage patterns for the OpenUI monorepo.
1211

1312
## Why pnpm?
1413

@@ -62,7 +61,7 @@ pnpm --recursive run <script-name>
6261

6362
## Workspace Management
6463

65-
Crayon uses pnpm workspaces for its monorepo structure. Here are the key workspace commands:
64+
OpenUI uses pnpm workspaces for its monorepo structure. Here are the key workspace commands:
6665

6766
### Workspace Structure
6867

@@ -162,7 +161,7 @@ poetry shell # Activate virtual environment
162161
```bash
163162
# Clone and setup the project
164163
git clone <repository-url>
165-
cd crayon
164+
cd openui
166165

167166
# Install all dependencies
168167
pnpm install

.github/workflows/deploy.yml

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,7 @@
11
name: Build and deploy docs
22

33
on:
4-
push:
5-
branches: [ main ]
6-
pull_request:
7-
branches: [ main ]
4+
workflow_dispatch:
85

96
defaults:
107
run:

.gitignore

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
# OS
22
.DS_Store
33

4-
54
# IDEs and editors
65
/.idea
76
.project
@@ -11,10 +10,26 @@
1110
.settings/
1211
*.sublime-workspace
1312

14-
# IDE - VSCode
13+
# VSCode
1514
.vscode/*
1615
!.vscode/settings.json
1716
!.vscode/tasks.json
1817
!.vscode/launch.json
1918
!.vscode/extensions.json
19+
20+
# Cursor
21+
.cursor/plans
22+
.cursor/agents
23+
24+
# Dependencies
25+
node_modules
26+
.pnpm-store/
27+
28+
# Build output
29+
dist
30+
.next
2031
*.tsbuildinfo
32+
typings
33+
34+
# Logs
35+
*storybook.log
File renamed without changes.

CONTRIBUTING.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Contributing Guidelines
22

3-
Thank you for considering contributing to *crayon*! This document provides guidelines for contributing.
3+
Thank you for considering contributing to _openui_! This document provides guidelines for contributing.
44

55
## How to Contribute
66

@@ -13,6 +13,7 @@ Thank you for considering contributing to *crayon*! This document provides guide
1313
## Bug Reports
1414

1515
Use Github Issues to report bugs. When reporting bugs, please include:
16+
1617
- A clear description of the issue
1718
- Steps to reproduce
1819
- Expected vs actual behavior

README.md

Lines changed: 0 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -1,50 +0,0 @@
1-
<a href="https://www.crayonai.org">
2-
<img src="https://crayonai.org/img/social-card.png" alt="crayonai"/>
3-
</a>
4-
5-
<p align="center">
6-
<a href="https://crayonai.org">Homepage</a> ·
7-
<a href="https://crayonai.org/docs">Documentation</a> ·
8-
<a href="https://thesys.dev/?utm_source=github&utm_medium=crayon-readme">Thesys</a>
9-
</p>
10-
11-
<p align="center">
12-
<img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License: MIT" />
13-
<a href="https://x.com/thesysdev"><img src="https://img.shields.io/twitter/url/https/twitter/follow/thesysdev?style=social&label=Follow%20%40thesys" alt="Thesys: Visit" /></a>
14-
<a href="https://discord.gg/Pbv5PsqUSv"><img src="https://img.shields.io/badge/Discord-Join-blue.svg" alt="Discord: Join" /></a>
15-
<img src="https://github.com/thesysdev/crayon/actions/workflows/deploy.yml/badge.svg" alt="Deploy Status" />
16-
<img src="https://github.com/thesysdev/crayon/actions/workflows/build-js.yml/badge.svg" alt="Build Status" />
17-
</p>
18-
19-
# Crayon
20-
**Generative UI** SDK for your AI agents. <br />
21-
22-
Crayon is a UI framework for building agentic UI interfaces beyond just text. It is a set of extensible React components, lightweight state management and hooks to help you build your own UI that seamlessly integrates with any backend.
23-
24-
## Quick Start
25-
26-
Install the Crayon package:
27-
```bash
28-
npm install @crayonai/react-core
29-
```
30-
31-
Create a new Crayon component:
32-
```tsx
33-
import { type ResponseTemplate, CrayonChat } from "@crayonai/react-core";
34-
35-
const templates: ResponseTemplate[] = [
36-
{
37-
name: "breakdown_expenses",
38-
component: BreakdownExpenses,
39-
},
40-
]
41-
42-
export default function App() {
43-
return <CrayonChat templates={templates}/>;
44-
}
45-
```
46-
47-
## Packages
48-
49-
- [react-core](./js/packages/react-core): Core framework and hooks for managing state and agents
50-
- [react-ui](./js/packages/react-ui): Modular UI components for use with Crayon

SECURITY.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,16 @@
22

33
## Reporting a Vulnerability
44

5-
We take the security of Crayon seriously. If you believe you have found a security vulnerability, please report it to us through GitHub's Security Advisory "Report a Vulnerability" tab.
5+
We take the security of OpenUI seriously. If you believe you have found a security vulnerability, please report it to us through GitHub's Security Advisory "Report a Vulnerability" tab.
66

77
**Please do not report security vulnerabilities through public GitHub issues.**
88

99
Instead:
10+
1011
1. Go to the "Security" tab of this repository
1112
2. Click "Report a vulnerability"
1213
3. Fill out the form with a description of the vulnerability
1314

1415
We will respond as quickly as possible to your report and work with you to verify and address the issue.
1516

16-
Thank you for helping keep Crayon and its users safe!
17+
Thank you for helping keep OpenUI and its users safe!

benchmarks/README.md

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
# OpenUI Lang Benchmark
2+
3+
Measures token efficiency and estimated generation latency of **OpenUI Lang** vs two JSON-based streaming formats across seven real-world UI scenarios.
4+
5+
## Formats
6+
7+
| Format | Description |
8+
| ---------------------- | -------------------------------------------------------------------------- |
9+
| **OpenUI Lang** | Line-oriented DSL generated directly by the LLM |
10+
| **Thesys C1 JSON** | Normalized component tree JSON (`component` + `props`) |
11+
| **Vercel JSON-Render** | JSONL stream of [JSON Patch (RFC 6902)](https://jsonpatch.com/) operations |
12+
13+
All three formats encode exactly the same UI. The LLM always generates OpenUI Lang — C1 JSON and Vercel JSONL are derived from the parsed AST.
14+
15+
## Methodology
16+
17+
1. Use a fixed set of seven prompts in `generate-samples.ts` (`simple-table`, `chart-with-data`, `contact-form`, `dashboard`, `pricing-page`, `settings-panel`, `e-commerce-product`).
18+
2. Generate OpenUI Lang once per prompt with `gpt-5.2` at `temperature: 0` using streaming completions.
19+
3. Parse the OpenUI Lang output with `createParser(schema)` so positional arguments map to named props via `schema.json`.
20+
4. Convert the same parsed AST into:
21+
- `*.c1.json`: Thesys C1 format via `thesys-c1-converter.ts`, which recursively maps AST elements to `{ component, props }` and removes parser metadata (`type`, `typeName`, `partial`, `__typename`).
22+
- `*.vercel.jsonl`: JSON Patch operation stream via `vercel-jsonl-converter.ts`.
23+
5. Count tokens from all saved artifacts in `samples/` using `tiktoken` with `encoding_for_model("gpt-5")`.
24+
6. Report:
25+
- Token counts and percentage reduction of OpenUI Lang vs each JSON format.
26+
- Estimated decode latency at a fixed 60 tokens/second.
27+
- Generation-time TTFT/TPS in `samples/metrics.json` captured from streaming usage data.
28+
29+
## Results
30+
31+
Measured with `tiktoken` (`gpt-5` model encoder). Generated by GPT-5.2 at temperature 0.
32+
33+
| Scenario | Vercel JSON-Render | Thesys C1 JSON | OpenUI Lang | vs Vercel | vs C1 |
34+
| ------------------ | -----------------: | -------------: | ----------: | ---------: | ---------: |
35+
| simple-table | 340 | 357 | 148 | -56.5% | -58.5% |
36+
| chart-with-data | 520 | 516 | 231 | -55.6% | -55.2% |
37+
| contact-form | 893 | 849 | 294 | -67.1% | -65.4% |
38+
| dashboard | 2247 | 2261 | 1226 | -45.4% | -45.8% |
39+
| pricing-page | 2487 | 2379 | 1195 | -52.0% | -49.8% |
40+
| settings-panel | 1244 | 1205 | 540 | -56.6% | -55.2% |
41+
| e-commerce-product | 2449 | 2381 | 1166 | -52.4% | -51.0% |
42+
| **TOTAL** | **10180** | **9948** | **4800** | **-52.8%** | **-51.7%** |
43+
44+
## Running
45+
46+
### Prerequisites
47+
48+
Export `OPENAI_API_KEY` in your shell:
49+
50+
```bash
51+
export OPENAI_API_KEY=sk-...
52+
```
53+
54+
### 1. Generate samples (calls OpenAI)
55+
56+
```bash
57+
cd js/benchmarks
58+
pnpm generate
59+
```
60+
61+
This calls OpenAI once per scenario, saves the raw `.oui` output, then converts it to `.c1.json` and `.vercel.jsonl`. All files land in `samples/`. A `metrics.json` with TTFT and TPS from the API response is also written.
62+
63+
### 2. Run the benchmark report (offline)
64+
65+
```bash
66+
pnpm bench
67+
```
68+
69+
Reads the files in `samples/`, counts tokens with `tiktoken`, and prints two tables: token counts and estimated latency at 60 tok/s.
70+
71+
## File Layout
72+
73+
```
74+
benchmarks/
75+
├── generate-samples.ts # Calls OpenAI, converts AST to all three formats
76+
├── run-benchmark.ts # Reads samples/, prints token/latency tables
77+
├── thesys-c1-converter.ts # AST -> normalized Thesys C1 JSON converter
78+
├── vercel-jsonl-converter.ts # RFC 6902-compliant AST -> JSONL converter
79+
├── schema.json # JSON Schema for the default component library
80+
│ # (auto-generated by library.toJSONSchema())
81+
├── system-prompt.txt # System prompt sent to the LLM
82+
├── .env # OPENAI_API_KEY
83+
└── samples/
84+
├── <scenario>.oui # Raw LLM output (OpenUI Lang)
85+
├── <scenario>.c1.json # Normalized Thesys C1 JSON
86+
├── <scenario>.vercel.jsonl # Vercel JSON-Render (RFC 6902 patches)
87+
└── metrics.json # TTFT + TPS recorded during generation
88+
```
89+
90+
## Updating the Schema
91+
92+
`schema.json` and `system-prompt.txt` are generated by `library.toJSONSchema()` and `library.prompt()` in `@openuidev/react-ui`. If you add or change components, regenerate them:
93+
94+
```ts
95+
import { defaultLibrary } from "@openuidev/react-ui";
96+
import { writeFileSync } from "fs";
97+
98+
writeFileSync("schema.json", JSON.stringify(defaultLibrary.toJSONSchema(), null, 2));
99+
writeFileSync("system-prompt.txt", defaultLibrary.prompt());
100+
```
101+
102+
The parser (`createParser`) reads component definitions from the `$defs` section of this file to map positional arguments to named props.

benchmarks/generate-samples.ts

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
import { OpenAI } from "openai";
2+
import { createParser } from "../packages/lang-react/src/parser/parser.js";
3+
import { writeFileSync, mkdirSync, readFileSync } from "fs";
4+
import { join } from "path";
5+
import { astToVercelJsonl } from "./vercel-jsonl-converter.js";
6+
import { astToThesysC1Json } from "./thesys-c1-converter.js";
7+
8+
const systemPrompt = readFileSync(join(process.cwd(), "system-prompt.txt"), "utf-8");
9+
10+
// Load the combined JSON Schema produced by library.toJSONSchema().
11+
// createParser reads component definitions from $defs so named props are
12+
// correctly mapped (instead of falling back to positional _args).
13+
const schemaStr = readFileSync(join(process.cwd(), "schema.json"), "utf-8");
14+
const schema = JSON.parse(schemaStr);
15+
const parser = createParser(schema);
16+
17+
// 1. Setup
18+
const openai = new OpenAI();
19+
const MODEL = "gpt-5.2";
20+
21+
const PROMPTS = {
22+
"simple-table": "Create a simple table showing 5 employees with columns for Name, Department, Salary, and YoY change.",
23+
"chart-with-data": "Create a dashboard with a metric card showing Total Revenue, and a BarChart showing 6 months of revenue data.",
24+
"contact-form": "Create a contact form with Name, Email, Phone, Subject dropdown, and Message text area. Include Submit and Cancel buttons.",
25+
"dashboard": "Create a product analytics dashboard. It should have a Header, a BarChart for Monthly Active Users, a PieChart for User Acquisition, a Table for Top Features, and a LineChart for MRR and ARR.",
26+
"pricing-page": "Create a pricing page with 3 tiers: Basic, Pro, and Enterprise. Include a feature comparison table below the pricing cards.",
27+
"settings-panel": "Create a user settings panel with tabs for Profile, Security, and Notifications. The Profile tab should have a form to update name and avatar. The Security tab should have a toggle for 2FA.",
28+
"e-commerce-product": "Create a product detail page for a pair of sneakers. Include an ImageGallery, a title, price, size selector (Select), color options (RadioGroup), and an Add to Cart button."
29+
};
30+
31+
// 2. Generation Loop
32+
async function main() {
33+
mkdirSync("samples", { recursive: true });
34+
const metrics: Record<string, any> = {};
35+
36+
for (const [name, userPrompt] of Object.entries(PROMPTS)) {
37+
console.log(`Generating ${name}...`);
38+
39+
const startTime = Date.now();
40+
let firstTokenTime = 0;
41+
let fullText = "";
42+
43+
const stream = await openai.chat.completions.create({
44+
model: MODEL,
45+
temperature: 0,
46+
messages: [
47+
{ role: "system", content: systemPrompt },
48+
{ role: "user", content: userPrompt }
49+
],
50+
stream: true,
51+
stream_options: { include_usage: true }
52+
});
53+
54+
let usage = null;
55+
56+
for await (const chunk of stream) {
57+
const content = chunk.choices[0]?.delta?.content;
58+
if (content) {
59+
// Only record TTFT on the first chunk that actually contains content
60+
if (!firstTokenTime) firstTokenTime = Date.now();
61+
fullText += content;
62+
}
63+
if (chunk.usage) {
64+
usage = chunk.usage;
65+
}
66+
}
67+
68+
const endTime = Date.now();
69+
const ttft = firstTokenTime - startTime;
70+
const totalDuration = endTime - startTime;
71+
const completionTokens = usage?.completion_tokens || 0;
72+
const tps = completionTokens / ((totalDuration - ttft) / 1000);
73+
74+
metrics[name] = {
75+
ttft_ms: ttft,
76+
total_duration_ms: totalDuration,
77+
completion_tokens: completionTokens,
78+
tps: tps
79+
};
80+
81+
// Save OpenUI Lang
82+
writeFileSync(join("samples", `${name}.oui`), fullText);
83+
84+
// Parse AST using createParser so named props are correctly mapped
85+
const ast = parser.parse(fullText).root;
86+
87+
// Save Thesys C1 JSON (normalized component/props envelope)
88+
writeFileSync(join("samples", `${name}.c1.json`), astToThesysC1Json(ast));
89+
90+
// Save Vercel JSONL
91+
writeFileSync(join("samples", `${name}.vercel.jsonl`), astToVercelJsonl(ast));
92+
93+
console.log(` Done! ${completionTokens} tokens @ ${tps.toFixed(1)} tok/s`);
94+
}
95+
96+
writeFileSync(join("samples", "metrics.json"), JSON.stringify(metrics, null, 2));
97+
console.log("All samples generated and saved to ./samples/");
98+
}
99+
100+
main().catch(console.error);

0 commit comments

Comments
 (0)