CourseApache Doris HackathonCommunity over Code 2026 Glasgow · Tue 13 Oct · 11:20–15:00 · Wee Dram Room

Track A · Task A3

Log Search Explorer

A mini Kibana where everything is SQL — then use it to find what broke checkout at 11:40.

60–120 min
Beginner-friendly
Any language · CLI or web
AI tools welcome

Stuck for more than 10 minutes? Come to the Doris table and ask Mingyu, or post in #dev on the Apache Doris Slack.

Get set up

What you are building, why Doris makes it simple, and a running cluster.

What you'll build

Build a log explorer. Then solve an outage with it.

200,000 log lines, six services, one bad morning. Search them by keyword with BM25 ranking, slice them by service, level and time, and count everything as you go — all in plain SQL on one Doris table.

A small log explorer with:

a search box (keywords, phrases) with relevance ranking,
filters for service, level and time range,
facet counts next to the results (hits per service / level),
optionally a timeline of errors per 5 minutes.

Then the real test: at 11:40 checkout started failing. Which service caused it, and when did the trouble really start? Your explorer should make the answer obvious.

SketchWhat you'll build · wireframe, not a screenshot

The sketch sits at the top of the code panel

Why it's interesting on Doris

Search and analytics on the same table. The inverted index answers the keyword query; the same WHERE clause feeds a GROUP BY for facets. No second system to keep in sync.
Real ranking. score() sorts hits by BM25, so the most relevant lines come first, not just the newest.
Filters are cheap. service and level have their own inverted indexes; time ranges prune by the sort key.

Panel: the index lines of the table definition

Before you start

Docker is running and the Doris image is pulled. About 1.9 GB, once. Pull it before you arrive: the venue Wi-Fi is shared. Docker Desktop (macOS) and Docker Engine (Linux) both work.
docker pull apache/doris:all-in-one-4.1.3
Doris is running. One container runs one FE and one BE. It is ready when docker ps shows (healthy), about 30 seconds after it starts. About this image
docker run -d --name doris -p 9030:9030 -p 8030:8030 -p 8040:8040 apache/doris:all-in-one-4.1.3
You can connect. The container ships a MySQL client, so you don't need one. Any MySQL client works too: 127.0.0.1:9030, user root, no password.
docker exec -it doris mysql -uroot -h127.0.0.1 -P9030
You have the starter kit. Download and unzip doris-hackathon-glasgow-2026.zip, then run the commands on this page from inside its doris-hackathon-glasgow-2026 folder. Its doris.sh wraps them: start, load, sql, stop.
curl -LO https://github.com/morningman/demo-env/releases/download/for-hackathon/doris-hackathon-glasgow-2026.zip && unzip doris-hackathon-glasgow-2026.zip && cd doris-hackathon-glasgow-2026
Your tools are ready. Any language with a MySQL driver (Python pymysql, Node mysql2, Go go-sql-driver/mysql, Java JDBC …). AI coding assistants are welcome.

Host: 127.0.0.1
Port: 9030
User: rootno password
Database: hackathon

The data

hackathon.app_logs — one table for all six services:

Tablehackathon.app_logs · sql

CREATE TABLE app_logs (  ts          DATETIME(3)  NOT NULL,  service     VARCHAR(32)  NOT NULL,   -- api-gateway, auth, catalog, search, checkout, payment  level       VARCHAR(8)   NOT NULL,   -- DEBUG / INFO / WARN / ERROR  host        VARCHAR(32),  trace_id    VARCHAR(32),  status      INT,                     -- HTTP status, when relevant  latency_ms  INT,  message     STRING       NOT NULL,  INDEX idx_msg (message) USING INVERTED PROPERTIES ("parser" = "english", "support_phrase" = "true"),  INDEX idx_service (service) USING INVERTED,  INDEX idx_level (level) USING INVERTED)DUPLICATE KEY(ts)DISTRIBUTED BY HASH(trace_id) BUCKETS 1PROPERTIES ("replication_num" = "1");

seed/a3_app_logs.sql generates 200,000 lines for 13 October 2026 with a single INSERT … SELECT … FROM numbers("number" = "200000") — no download needed. Somewhere in there is the outage.

Panel → Table: the full table definition

Build with AI

Pair with your AI coding agent — that's encouraged. Doris 4.x search and AI features are newer than what most models were trained on, so give your agent the right context before it writes SQL:

Paste the context prompt below into your agent.
Point it at the reference pages listed on this page and at AGENTS.md in the starter kit. Many coding agents read that file on their own.
Make it run every SQL statement against your live cluster before wiring it into code.
Better still: connect the Doris MCP Server (see Task A2) so your agent can inspect the schema and test queries itself.

Context promptpaste into your agent

I'm building a log search explorer on Apache Doris 4.1 for a hackathon.Connection: MySQL protocol, 127.0.0.1:9030, user root, empty password, database `hackathon`.Use a plain MySQL driver and raw SQL.Table `app_logs`: ts DATETIME(3), service, level, host, trace_id, status INT, latency_ms INT,message STRING. Inverted indexes on message (english parser, phrase support), service, level.Doris SQL rules:1. Text search: `message MATCH_ANY 'a b'`, `MATCH_ALL`, `MATCH_PHRASE 'a b'`. Never LIKE.2. BM25: `score()` only with a MATCH_* predicate + `ORDER BY score() DESC LIMIT n`,   selected plain (`score() AS relevance`): no ROUND() or JOIN in that SELECT.   Bind the user's search text as a driver parameter.3. score() cannot be used with GROUP BY. For facet counts run a second query with the   same WHERE clause and GROUP BY service, level.4. Time buckets: `minute_floor(ts, 5)`.Build: search box, service/level/time filters, facet counts, and an error timeline.Run every SQL statement against the live cluster first.

Doris traps your agent will probably fall into

Trap	What to do instead
Using `LIKE '%term%'` for text search	`col MATCH_ANY 'a b'` (OR), `MATCH_ALL` (AND), `MATCH_PHRASE 'a b'` — they use the inverted index
Calling `score()` in an arbitrary query	`score()` needs a `MATCH_` predicate in `WHERE` and* `ORDER BY score() DESC LIMIT n`; otherwise Doris rejects the query
Rounding, joining or ranking `score()` in the same query	Keep `score() AS relevance` plain in a single-table query. Round it, join it or `ROW_NUMBER()` it in an outer query, and use a subquery, not a `WITH` CTE
`score()` inside `GROUP BY` / aggregates	Not allowed. For facet counts, reuse the same `WHERE` without `score()`
Using an ORM	Prefer a plain MySQL driver and raw SQL; ORMs may emit statements Doris doesn't support

Build it

6 milestones, M0 → M5. The panel follows the step you are reading.

M0
Milestone 05 min
Generate the logs
From inside the starter-kit folder:
M0 · Generate the logsbash
```
docker exec -i doris mysql -uroot -h127.0.0.1 -P9030 < seed/a3_app_logs.sql
```
Checkpoint
A few seconds later: 200,000 log lines, from 00:00:00 to 23:59:59 on 13 October.
M1
Milestone 115 min
Keyword search with BM25
M1 · Keyword search with BM25sql
```
USE hackathon;SELECT ts, service, level, message, score() AS relevanceFROM app_logsWHERE message MATCH_ANY 'timeout refused'ORDER BY relevance DESCLIMIT 20;
```
Wire it to a search box. Pass the user's text as a driver parameter (MATCH_ANY %s); a3/starter.py in the starter kit does exactly this.
Checkpoint
All 20 hits are connection refused lines. BM25 weighs rare terms more: refused appears in 68 lines, timeout in 347. Search timeout alone to see the others.

Milestone 215 min

Filters

M2 · Filterssql

SELECT ts, service, level, message, score() AS relevanceFROM app_logsWHERE message MATCH_ANY 'timeout'  AND level = 'ERROR'  AND service IN ('checkout', 'payment')  AND ts >= '2026-10-13 11:00:00' AND ts < '2026-10-13 12:30:00'ORDER BY relevance DESCLIMIT 20;

Checkpoint

Every hit is an ERROR line that mentions a timeout. Only one of the two services shows up.

M3
Milestone 315 min
Facet counts
score() can't be aggregated, so facets reuse the same filter without it:
M3 · Facet countssql
```
SELECT service, level, COUNT(*) AS hitsFROM app_logsWHERE message MATCH_ANY 'timeout'  AND ts >= '2026-10-13 11:00:00' AND ts < '2026-10-13 12:30:00'GROUP BY service, levelORDER BY hits DESC;
```
Checkpoint
163 matching lines in that window, all ERROR. The counts must add up to the same search without GROUP BY: check with SELECT COUNT(*) and the same WHERE.

Milestone 415 min

Error timeline

M4 · Error timelinesql

SELECT minute_floor(ts, 5) AS bucket, service, COUNT(*) AS errorsFROM app_logsWHERE level = 'ERROR'  AND ts >= '2026-10-13 11:00:00' AND ts < '2026-10-13 12:30:00'GROUP BY bucket, serviceORDER BY bucket, service;

Checkpoint

Two services jump between 11:35 and 12:00. For the rest of the day no service logs more than 4 errors in a 5-minute bucket.

Milestone 5
Solve the outage
Use your explorer to answer: Which service caused the checkout failures, and at what minute did it start? Put the answer — and the query that proves it — in your README.

Ship it

Check it off, show it, collect your badge.

Definition of Done

Your checklist

Keyword search with BM25 ranking (score()).
At least two structured filters (service, level) plus a time range.
Facet counts that follow the current search.
The outage solved: root-cause service + start time, with the supporting query.
Submitted as a pull request: your GitHub-ID folder with the code and a README (how to run + one screenshot).

Submit & get your badge

Take part → submit → get your badge.

Put your work in a folder named after your GitHub ID: the code, plus a README with what it does, how to run it, one screenshot or GIF, and the Doris features you used.
Open a pull request that adds it to morningman/demo-env as doris-hackathon-glasgow-2026/<your-github-id>/. Fork and push, or use GitHub's Add file → Upload files. Nothing else to fill in.
Show it at the Doris table (a 2-minute demo is plenty), or at the show-and-tell around 14:30.
Join the Apache Doris Slack and say hi in #dev: questions, submissions and badges are all handled there.

Everyone who takes part in a task on site gets the Apache Doris Contributor badge: no merged pull request to Doris needed. Didn't finish by 15:00? Keep going — submissions are open until 31 October. Teams are fine, but not needed: list every member's GitHub ID in the README.

About the Contributor badge

Stretch goals

Phrase and type-ahead: MATCH_PHRASE 'connection refused', MATCH_PHRASE_PREFIX for search-as-you-type.
Trace view: click a line to show every log with the same trace_id, in time order.
Query-string search: try the SEARCH() DSL (Lucene-style field:term AND …) — see SEARCH Function.
Live tail: insert a few new rows every second and auto-refresh.
Index vs. no index: compare MATCH_ANY with LIKE '%timeout%' on the full table and show the timing.

Troubleshooting & references

Symptom	Fix
`docker: command not found` on macOS	Start Docker Desktop. If it is running, link the CLI: `sudo ln -s /Applications/Docker.app/Contents/Resources/bin/docker /usr/local/bin/docker`
`error getting credentials` when pulling images	Remove the `credsStore` field from `~/.docker/config.json` (local dev only)
`port is already allocated`	Something else uses 9030, 8030 or 8040, often another Doris. Stop it (`docker ps`), or publish other host ports, e.g. `-p 19030:9030`, and connect to 19030
`The container name "/doris" is already in use`	You started it before: `docker start doris`. To start over, `docker rm -f doris` (this deletes its data)
The container exits, or never turns `(healthy)`	Read `docker logs doris`. Usually it is memory: give Docker Desktop 6 GB or more (Settings → Resources). On Apple Silicon, don't add `--platform linux/amd64`
`permission denied: ./doris.sh`	Your unzip tool dropped the executable bit: `chmod +x doris.sh`, or run `bash doris.sh start`
`Current database is not set`	Run `USE hackathon;` first, or connect with `-Dhackathon`
`score() function requires WHERE clause with MATCH function, ORDER BY and LIMIT`	Add a `MATCH_*` predicate, `ORDER BY score() DESC` and a `LIMIT`, and keep `score()` unwrapped (see the traps above)
Stuck for more than 10 minutes	Come to the Doris table and ask Mingyu, or post in `#dev` on the Apache Doris Slack

References

Full-text Search BM25 Inverted Index SEARCH Function

SketchWhat you'll build · wireframe, not a screenshot

Start Doristerminal

docker pull apache/doris:all-in-one-4.1.3   # 1.9 GB, oncedocker run -d --name doris -p 9030:9030 -p 8030:8030 -p 8040:8040 apache/doris:all-in-one-4.1.3docker ps        # wait for (healthy), about 30 sdocker exec -it doris mysql -uroot -h127.0.0.1 -P9030

Host: 127.0.0.1
Port: 9030
User: rootno password
Database: hackathon

Starter kitdownloaddoris-hackathon-glasgow-2026.zip

Route60–120 min · M0 → M5 → done

M0Generate the logs5 min
M1Keyword search15 min
M2Filters15 min
M3Facet counts15 min
M4Error timeline15 min
M5Solve the outage
DoDDone→ submit

Tablehackathon.app_logs · sql

CREATE TABLE app_logs (  ts          DATETIME(3)  NOT NULL,  service     VARCHAR(32)  NOT NULL,   -- api-gateway, auth, catalog, search, checkout, payment  level       VARCHAR(8)   NOT NULL,   -- DEBUG / INFO / WARN / ERROR  host        VARCHAR(32),  trace_id    VARCHAR(32),  status      INT,                     -- HTTP status, when relevant  latency_ms  INT,  message     STRING       NOT NULL,  INDEX idx_msg (message) USING INVERTED PROPERTIES ("parser" = "english", "support_phrase" = "true"),  INDEX idx_service (service) USING INVERTED,  INDEX idx_level (level) USING INVERTED)DUPLICATE KEY(ts)DISTRIBUTED BY HASH(trace_id) BUCKETS 1PROPERTIES ("replication_num" = "1");

M0 · Generate the logsbash

docker exec -i doris mysql -uroot -h127.0.0.1 -P9030 < seed/a3_app_logs.sql

M1 · Keyword search with BM25sql

USE hackathon;SELECT ts, service, level, message, score() AS relevanceFROM app_logsWHERE message MATCH_ANY 'timeout refused'ORDER BY relevance DESCLIMIT 20;