CourseApache Doris HackathonCommunity over Code 2026 Glasgow · Tue 13 Oct · 11:20–15:00 · Wee Dram Room

Track A · Task A4

Agent Trace Explorer

AI agents emit JSON that changes shape every release. Store it in VARIANT and query it like real columns.

60–120 min
Intermediate
Any language · CLI, notebook or web
AI tools welcome

Stuck for more than 10 minutes? Come to the Doris table and ask Mingyu, or post in #dev on the Apache Doris Slack.

Get set up

What you are building, why Doris makes it simple, and a running cluster.

What you'll build

Your agent's JSON is a mess. Query it anyway.

LLM calls, tool calls, errors, final answers — each with its own fields, and new fields every release. Doris VARIANT turns that JSON into typed columns automatically, so you can filter, search and aggregate it with plain SQL. No schema migrations.

A trace explorer for 1,000 AI-agent sessions (about 20,000 events):

a sessions list with event counts, errors, tokens and latency,
a session timeline: click a session to replay what the agent did, step by step,
tool and model stats: p95 latency, error rate, token usage,
search inside the JSON: find every tool error that mentions "timeout".

Then answer: did the agent's v2 release (deployed at 13:00) make things better or worse?

SketchWhat you'll build · wireframe, not a screenshot

The sketch sits at the top of the code panel

Why it's interesting on Doris

Schema-on-write, without the schema. VARIANT infers a type for every JSON path and stores frequent paths as real columnar subcolumns, so payload['latency_ms'] reads one column instead of parsing every document.
New fields just appear. When v2 events add cost_usd, the new path shows up as a subcolumn on the next load, with no ALTER TABLE.
Search inside JSON. An inverted index on the VARIANT column makes every text path searchable with MATCH_*.

Panel: the index lines of the table definition

Before you start

Docker is running and the Doris image is pulled. About 1.9 GB, once. Pull it before you arrive: the venue Wi-Fi is shared. Docker Desktop (macOS) and Docker Engine (Linux) both work.
docker pull apache/doris:all-in-one-4.1.3
Doris is running. One container runs one FE and one BE. It is ready when docker ps shows (healthy), about 30 seconds after it starts. About this image
docker run -d --name doris -p 9030:9030 -p 8030:8030 -p 8040:8040 apache/doris:all-in-one-4.1.3
You can connect. The container ships a MySQL client, so you don't need one. Any MySQL client works too: 127.0.0.1:9030, user root, no password.
docker exec -it doris mysql -uroot -h127.0.0.1 -P9030
You have the starter kit. Download and unzip doris-hackathon-glasgow-2026.zip, then run the commands on this page from inside its doris-hackathon-glasgow-2026 folder. Its doris.sh wraps them: start, load, sql, stop.
curl -LO https://github.com/morningman/demo-env/releases/download/for-hackathon/doris-hackathon-glasgow-2026.zip && unzip doris-hackathon-glasgow-2026.zip && cd doris-hackathon-glasgow-2026
Your tools are ready. Any language with a MySQL driver (Python pymysql, Node mysql2, Go go-sql-driver/mysql, Java JDBC …). AI coding assistants are welcome.

Host: 127.0.0.1
Port: 9030
User: rootno password
Database: hackathon

The data

hackathon.agent_events — one row per agent event, the event body in a VARIANT column:

Tablehackathon.agent_events · sql

CREATE TABLE agent_events (  session_id  VARCHAR(40)  NOT NULL,  ts          DATETIME(3)  NOT NULL,  event_id    BIGINT       NOT NULL,  event_type  VARCHAR(32)  NOT NULL,   -- user_message / llm_call / tool_call / error / final_answer  payload     VARIANT      NOT NULL,  INDEX idx_payload (payload) USING INVERTED PROPERTIES ("parser" = "english"))DUPLICATE KEY(session_id, ts)DISTRIBUTED BY HASH(session_id) BUCKETS 1PROPERTIES ("replication_num" = "1", "storage_format" = "V3");

Example payloads — note how the shapes differ:

Example payloadsjson

{"text": "Why did checkout fail at 11:40?", "lang": "en", "agent_version": "v1"}{"model": "model-large", "usage": {"input_tokens": 812, "output_tokens": 96}, "latency_ms": 1430, "agent_version": "v1"}{"tool": "sql_query", "args": {"sql": "SELECT ..."}, "latency_ms": 37, "status": "ok", "agent_version": "v1"}{"tool": "http_get", "args": {"url": "..."}, "latency_ms": 3012, "status": "error", "agent_version": "v1", "error": {"type": "Timeout", "message": "read timeout after 3000 ms calling https://api.internal/orders"}}

Sessions that start at 13:00 or later run agent v2: their payloads say "agent_version": "v2", and v2 LLM calls add a cache_hit flag. Milestone 5 then loads events with fields the table has never seen.

Panel → Table: the full table definition

Build with AI

Pair with your AI coding agent — that's encouraged. Doris 4.x search and AI features are newer than what most models were trained on, so give your agent the right context before it writes SQL:

Paste the context prompt below into your agent.
Point it at the reference pages listed on this page and at AGENTS.md in the starter kit. Many coding agents read that file on their own.
Make it run every SQL statement against your live cluster before wiring it into code.
Better still: connect the Doris MCP Server (see Task A2) so your agent can inspect the schema and test queries itself.

Context promptpaste into your agent

I'm building an agent trace explorer on Apache Doris 4.1 for a hackathon.Connection: MySQL protocol, 127.0.0.1:9030, user root, empty password, database `hackathon`.Use a plain MySQL driver and raw SQL.Table `agent_events`: session_id, ts DATETIME(3), event_id, event_type(user_message / llm_call / tool_call / error / final_answer), payload VARIANT(inverted index, english parser). Payload shapes differ by event_type. Every payload hasagent_version: "v1" before 13:00, "v2" from 13:00; v2 llm_calls add cache_hit.Doris SQL rules:1. Read JSON paths as payload['a']['b']. CAST to a concrete type before comparing,   sorting or aggregating: CAST(payload['latency_ms'] AS DOUBLE).2. VARIANT cannot be a key, a sort key or a join key.3. Text search inside JSON: payload['error']['message'] MATCH_ANY 'timeout'. Never LIKE.4. Percentiles: percentile_approx(expr, 0.95).5. `SET describe_extend_variant_column = true; DESC agent_events;` lists inferred subcolumns.6. To insert JSON yourself, CAST the text: CAST('{"a": 1}' AS JSON).Build: sessions list, session timeline, tool/model stats, and JSON search.Run every SQL statement against the live cluster first.

Doris traps your agent will probably fall into

Trap	What to do instead
Comparing VARIANT paths without a cast	`CAST(payload['latency_ms'] AS INT)`; VARIANT can't be a key or join column
Expecting `DESC` to list the JSON paths	Run `SET describe_extend_variant_column = true;` first; then `DESC` shows every inferred subcolumn and its type
Using `LIKE '%term%'` for text search	`col MATCH_ANY 'a b'` (OR), `MATCH_ALL` (AND), `MATCH_PHRASE 'a b'` — they use the inverted index
Using an ORM	Prefer a plain MySQL driver and raw SQL; ORMs may emit statements Doris doesn't support

Build it

6 milestones, M0 → M5. The panel follows the step you are reading.

Milestone 010 min

Load and inspect

From inside the starter-kit folder:

M0 · Loadbash

docker exec -i doris mysql -uroot -h127.0.0.1 -P9030 < seed/a4_agent_events.sql

Then, in a SQL shell:

M0 · Inspectsql

USE hackathon;SET describe_extend_variant_column = true;DESC agent_events;     -- see the subcolumns Doris inferred: payload.tool, payload.usage.input_tokens, ...

Checkpoint

The load reports 1,000 sessions and 20,401 events; DESC lists about 20 payload.* paths, each with an inferred type.

Milestone 115 min

Query JSON paths

M1 · Query JSON pathssql

SELECT CAST(payload['tool'] AS STRING) AS tool, COUNT(*) AS callsFROM agent_eventsWHERE event_type = 'tool_call'GROUP BY toolORDER BY calls DESC;

Checkpoint

Five tools; sql_query is called most (3,506 times).

Milestone 215 min

Aggregate inferred subcolumns

M2 · Aggregate inferred subcolumnssql

SELECT CAST(payload['tool'] AS STRING) AS tool,       COUNT(*) AS calls,       percentile_approx(CAST(payload['latency_ms'] AS DOUBLE), 0.95) AS p95_ms,       SUM(CASE WHEN CAST(payload['status'] AS STRING) = 'error' THEN 1 ELSE 0 END) AS errorsFROM agent_eventsWHERE event_type = 'tool_call'GROUP BY toolORDER BY p95_ms DESC;

Checkpoint

python_exec has the worst p95 (about 2.1 s); http_get fails most often (425 errors).

Milestone 310 min

Search inside the JSON

M3 · Search inside the JSONsql

SELECT session_id, ts, CAST(payload['error']['message'] AS STRING) AS error_messageFROM agent_eventsWHERE payload['error']['message'] MATCH_ANY 'timeout'ORDER BY tsLIMIT 20;

Checkpoint

398 events mention a timeout in total. Count them per hour: that is your first clue about v2.

Milestone 420 min

Session timeline

M4 · Session timelinesql

SELECT ts, event_type,       CAST(payload['tool'] AS STRING)      AS tool,       CAST(payload['status'] AS STRING)    AS status,       CAST(payload['latency_ms'] AS INT)   AS latency_ms,       CAST(payload['text'] AS STRING)      AS textFROM agent_eventsWHERE session_id = 's-0042'ORDER BY ts;

Build the explorer UI around these queries: sessions list → timeline → stats.

Checkpoint

Session s-0042 has 25 events: a question, LLM and tool calls (one failed sql_query, retried), and a final answer.

Milestone 510 min

Watch the schema evolve

M5 · Load the drift batchbash

docker exec -i doris mysql -uroot -h127.0.0.1 -P9030 < a4/drift_batch.sql   # events with new fields

M5 · Query the new fieldssql

SELECT CAST(payload['agent_version'] AS STRING) AS version,       COUNT(*) AS llm_calls,       ROUND(SUM(CAST(payload['cost_usd'] AS DOUBLE)), 2) AS cost_usdFROM agent_eventsWHERE event_type = 'llm_call'GROUP BY versionORDER BY version;

Run DESC agent_events again: the new paths are there. No ALTER TABLE happened.

Checkpoint

A third version appears, v2.1: only its LLM calls carry cost_usd, and DESC now lists payload.cost_usd and payload.retries.

Ship it

Check it off, show it, collect your badge.

Definition of Done

Your checklist

Heterogeneous events loaded into one VARIANT column (the seed plus the drift batch).
At least three JSON-path queries, including one aggregation on an inferred numeric subcolumn (for example p95 latency per tool).
A full-text search on a JSON text path.
A session timeline view (CLI, notebook or web).
A README with three insights from the data — including your verdict on the v2 release — plus how to run and one screenshot.

Submit & get your badge

Take part → submit → get your badge.

Put your work in a folder named after your GitHub ID: the code, plus a README with what it does, how to run it, one screenshot or GIF, and the Doris features you used.
Open a pull request that adds it to morningman/demo-env as doris-hackathon-glasgow-2026/<your-github-id>/. Fork and push, or use GitHub's Add file → Upload files. Nothing else to fill in.
Show it at the Doris table (a 2-minute demo is plenty), or at the show-and-tell around 14:30.
Join the Apache Doris Slack and say hi in #dev: questions, submissions and badges are all handled there.

Everyone who takes part in a task on site gets the Apache Doris Contributor badge: no merged pull request to Doris needed. Didn't finish by 15:00? Keep going — submissions are open until 31 October. Teams are fine, but not needed: list every member's GitHub ID in the README.

About the Contributor badge

Stretch goals

Bring your own traces. Export your own AI coding agent's local session logs (JSONL) and load them, one CAST('…' AS JSON) per line. It all stays on your laptop.
Schema Template. Pin hot paths with VARIANT<'latency_ms': INT, 'tool': STRING> in a copy of the table and compare DESC output and query behaviour. Fill it with INSERT … SELECT …, CAST(payload AS JSON): a VARIANT does not cast straight into a templated one.
Let an assistant explore it. Connect the Doris MCP Server (Task A2) and ask your assistant to find the worst session.
Classify errors with SQL (bring your own API key): AI_CLASSIFY(CAST(payload['error']['message'] AS STRING), ['timeout', 'auth', 'bad input', 'other']). See LLM SQL Functions.

Troubleshooting & references

Symptom	Fix
`docker: command not found` on macOS	Start Docker Desktop. If it is running, link the CLI: `sudo ln -s /Applications/Docker.app/Contents/Resources/bin/docker /usr/local/bin/docker`
`error getting credentials` when pulling images	Remove the `credsStore` field from `~/.docker/config.json` (local dev only)
`port is already allocated`	Something else uses 9030, 8030 or 8040, often another Doris. Stop it (`docker ps`), or publish other host ports, e.g. `-p 19030:9030`, and connect to 19030
`The container name "/doris" is already in use`	You started it before: `docker start doris`. To start over, `docker rm -f doris` (this deletes its data)
The container exits, or never turns `(healthy)`	Read `docker logs doris`. Usually it is memory: give Docker Desktop 6 GB or more (Settings → Resources). On Apple Silicon, don't add `--platform linux/amd64`
`permission denied: ./doris.sh`	Your unzip tool dropped the executable bit: `chmod +x doris.sh`, or run `bash doris.sh start`
`Current database is not set`	Run `USE hackathon;` first, or connect with `-Dhackathon`
`score() function requires WHERE clause with MATCH function, ORDER BY and LIMIT`	Add a `MATCH_*` predicate, `ORDER BY score() DESC` and a `LIMIT`, and keep `score()` unwrapped (see the traps above)
Stuck for more than 10 minutes	Come to the Doris table and ask Mingyu, or post in `#dev` on the Apache Doris Slack

References

VARIANT Data Type VARIANT SQL reference Full-text Search LLM SQL Functions

SketchWhat you'll build · wireframe, not a screenshot

Start Doristerminal

docker pull apache/doris:all-in-one-4.1.3   # 1.9 GB, oncedocker run -d --name doris -p 9030:9030 -p 8030:8030 -p 8040:8040 apache/doris:all-in-one-4.1.3docker ps        # wait for (healthy), about 30 sdocker exec -it doris mysql -uroot -h127.0.0.1 -P9030

Host: 127.0.0.1
Port: 9030
User: rootno password
Database: hackathon

Starter kitdownloaddoris-hackathon-glasgow-2026.zip

Route60–120 min · M0 → M5 → done

M0Load and inspect10 min
M1Query JSON paths15 min
M2Aggregate subcolumns15 min
M3Search the JSON10 min
M4Session timeline20 min
M5Schema evolves10 min
DoDDone→ submit

Tablehackathon.agent_events · sql

CREATE TABLE agent_events (  session_id  VARCHAR(40)  NOT NULL,  ts          DATETIME(3)  NOT NULL,  event_id    BIGINT       NOT NULL,  event_type  VARCHAR(32)  NOT NULL,   -- user_message / llm_call / tool_call / error / final_answer  payload     VARIANT      NOT NULL,  INDEX idx_payload (payload) USING INVERTED PROPERTIES ("parser" = "english"))DUPLICATE KEY(session_id, ts)DISTRIBUTED BY HASH(session_id) BUCKETS 1PROPERTIES ("replication_num" = "1", "storage_format" = "V3");

Example payloadsjson

{"text": "Why did checkout fail at 11:40?", "lang": "en", "agent_version": "v1"}{"model": "model-large", "usage": {"input_tokens": 812, "output_tokens": 96}, "latency_ms": 1430, "agent_version": "v1"}{"tool": "sql_query", "args": {"sql": "SELECT ..."}, "latency_ms": 37, "status": "ok", "agent_version": "v1"}{"tool": "http_get", "args": {"url": "..."}, "latency_ms": 3012, "status": "error", "agent_version": "v1", "error": {"type": "Timeout", "message": "read timeout after 3000 ms calling https://api.internal/orders"}}

M0 · Loadbash

docker exec -i doris mysql -uroot -h127.0.0.1 -P9030 < seed/a4_agent_events.sql

M0 · Inspectsql

USE hackathon;SET describe_extend_variant_column = true;DESC agent_events;     -- see the subcolumns Doris inferred: payload.tool, payload.usage.input_tokens, ...

M1 · Query JSON pathssql

SELECT CAST(payload['tool'] AS STRING) AS tool, COUNT(*) AS callsFROM agent_eventsWHERE event_type = 'tool_call'GROUP BY toolORDER BY calls DESC;

M2 · Aggregate inferred subcolumnssql

SELECT CAST(payload['tool'] AS STRING) AS tool,       COUNT(*) AS calls,       percentile_approx(CAST(payload['latency_ms'] AS DOUBLE), 0.95) AS p95_ms,       SUM(CASE WHEN CAST(payload['status'] AS STRING) = 'error' THEN 1 ELSE 0 END) AS errorsFROM agent_eventsWHERE event_type = 'tool_call'GROUP BY toolORDER BY p95_ms DESC;

M3 · Search inside the JSONsql

SELECT session_id, ts, CAST(payload['error']['message'] AS STRING) AS error_messageFROM agent_eventsWHERE payload['error']['message'] MATCH_ANY 'timeout'ORDER BY tsLIMIT 20;

M4 · Session timelinesql

SELECT ts, event_type,       CAST(payload['tool'] AS STRING)      AS tool,       CAST(payload['status'] AS STRING)    AS status,       CAST(payload['latency_ms'] AS INT)   AS latency_ms,       CAST(payload['text'] AS STRING)      AS textFROM agent_eventsWHERE session_id = 's-0042'ORDER BY ts;

M5 · Load the drift batchbash

docker exec -i doris mysql -uroot -h127.0.0.1 -P9030 < a4/drift_batch.sql   # events with new fields

M5 · Query the new fieldssql

SELECT CAST(payload['agent_version'] AS STRING) AS version,       COUNT(*) AS llm_calls,       ROUND(SUM(CAST(payload['cost_usd'] AS DOUBLE)), 2) AS cost_usdFROM agent_eventsWHERE event_type = 'llm_call'GROUP BY versionORDER BY version;

Context promptpaste into your agent

I'm building an agent trace explorer on Apache Doris 4.1 for a hackathon.Connection: MySQL protocol, 127.0.0.1:9030, user root, empty password, database `hackathon`.Use a plain MySQL driver and raw SQL.Table `agent_events`: session_id, ts DATETIME(3), event_id, event_type(user_message / llm_call / tool_call / error / final_answer), payload VARIANT(inverted index, english parser). Payload shapes differ by event_type. Every payload hasagent_version: "v1" before 13:00, "v2" from 13:00; v2 llm_calls add cache_hit.Doris SQL rules:1. Read JSON paths as payload['a']['b']. CAST to a concrete type before comparing,   sorting or aggregating: CAST(payload['latency_ms'] AS DOUBLE).2. VARIANT cannot be a key, a sort key or a join key.3. Text search inside JSON: payload['error']['message'] MATCH_ANY 'timeout'. Never LIKE.4. Percentiles: percentile_approx(expr, 0.95).5. `SET describe_extend_variant_column = true; DESC agent_events;` lists inferred subcolumns.6. To insert JSON yourself, CAST the text: CAST('{"a": 1}' AS JSON).Build: sessions list, session timeline, tool/model stats, and JSON search.Run every SQL statement against the live cluster first.

Apache Doris Hackathon

All tasks

Back to the course