CourseApache Doris HackathonCommunity over Code 2026 Glasgow · Tue 13 Oct · 11:20–15:00 · Wee Dram Room

Track A · Task A1

Hybrid Search App

Keywords, meaning and filters — in one SQL query. Build a search box over the Apache Doris docs.

60–120 min
Beginner-friendly
Any language · CLI or web
AI tools welcome

Stuck for more than 10 minutes? Come to the Doris table and ask Mingyu, or post in #dev on the Apache Doris Slack.

Get set up

What you are building, why Doris makes it simple, and a running cluster.

What you'll build

Build a search box that understands both words and meaning.

Exact keyword matches ranked by BM25, semantic neighbours from a vector index, and structured filters — served by one Doris table and one SQL statement. No Elasticsearch, no separate vector database.

A small search app (CLI, notebook, or web page — your call) over 299 sections of the Apache Doris documentation. A user types a question such as "how do I store messy JSON?", picks a category, and switches between three modes:

KeywordBM25-ranked full-text matches.
Vectornearest neighbours by embedding similarity.
Hybridkeyword pre-filter, then vector ranking (and, as a stretch, Reciprocal Rank Fusion).

Every result shows the page, the section, its score (keyword) or distance (vector, hybrid), and a link to the doc. A "Show SQL" toggle reveals the exact query Doris ran.

SketchWhat you'll build · wireframe, not a screenshot

The sketch sits at the top of the code panel

Why it's interesting on Doris

One table, two indexes. An inverted index on the text and an HNSW ANN index on the vectors live in the same table. The planner pre-filters with the inverted index, then ranks the survivors with the ANN index.
Search-engine ranking in SQL. score() gives you BM25 — the same relevance function Lucene uses — inside a normal SELECT.
Filters are just SQL. Category, page, IN lists, ranges — written next to MATCH_ANY in the same WHERE clause.

Panel: the index lines of the table definition

Before you start

Docker is running and the Doris image is pulled. About 1.9 GB, once. Pull it before you arrive: the venue Wi-Fi is shared. Docker Desktop (macOS) and Docker Engine (Linux) both work.
docker pull apache/doris:all-in-one-4.1.3
Doris is running. One container runs one FE and one BE. It is ready when docker ps shows (healthy), about 30 seconds after it starts. About this image
docker run -d --name doris -p 9030:9030 -p 8030:8030 -p 8040:8040 apache/doris:all-in-one-4.1.3
You can connect. The container ships a MySQL client, so you don't need one. Any MySQL client works too: 127.0.0.1:9030, user root, no password.
docker exec -it doris mysql -uroot -h127.0.0.1 -P9030
You have the starter kit. Download and unzip doris-hackathon-glasgow-2026.zip, then run the commands on this page from inside its doris-hackathon-glasgow-2026 folder. Its doris.sh wraps them: start, load, sql, stop.
curl -LO https://github.com/morningman/demo-env/releases/download/for-hackathon/doris-hackathon-glasgow-2026.zip && unzip doris-hackathon-glasgow-2026.zip && cd doris-hackathon-glasgow-2026
Your tools are ready. Any language with a MySQL driver (Python pymysql, Node mysql2, Go go-sql-driver/mysql, Java JDBC …). AI coding assistants are welcome.

Host: 127.0.0.1
Port: 9030
User: rootno password
Database: hackathon

The data

hackathon.doc_chunks — the 47 key-features pages of the Doris 4.x docs, one row per section:

Tablehackathon.doc_chunks · sql

CREATE TABLE doc_chunks (  id          INT           NOT NULL,  page        VARCHAR(64)   NOT NULL,   -- doc slug, e.g. 'hybrid-search'  page_title  VARCHAR(128)  NOT NULL,  section     VARCHAR(256)  NOT NULL,  category    VARCHAR(32)   NOT NULL,   -- see the 7 values below  url         VARCHAR(256)  NOT NULL,  body        STRING        NOT NULL,  embedding   ARRAY<FLOAT>  NOT NULL,   -- 8-dim toy vector, unit length  INDEX idx_body (body) USING INVERTED PROPERTIES ("parser" = "english", "support_phrase" = "true"),  INDEX idx_category (category) USING INVERTED,  INDEX idx_emb (embedding) USING ANN PROPERTIES (    "index_type" = "hnsw", "metric_type" = "l2_distance", "dim" = "8"))DUPLICATE KEY(id)DISTRIBUTED BY HASH(id) BUCKETS 1PROPERTIES ("replication_num" = "1");

category values (for your filter dropdown): search, ai, lakehouse, ingestion, performance, table-design, operations.

About the vectors. To keep everything offline, the starter kit ships a tiny toy embedder, a1/toy_embed.py and a1/toy_embed.js (same output, digit for digit). It maps text onto 8 hand-defined concept dimensions — search, vectors & AI, JSON, lakehouse, ingestion, performance, data model, operations — using synonym lists, then normalizes the result to unit length. The same function produced the vectors in the table, so your query vectors live in the same space. It is crude on purpose, yet it already shows what vector search adds: "store messy nested records whose schema keeps changing" lands on the VARIANT pages without saying JSON or VARIANT. The stretch goals swap it for a real model.

Panel → Table: the full table definition

Build with AI

Pair with your AI coding agent — that's encouraged. Doris 4.x search and AI features are newer than what most models were trained on, so give your agent the right context before it writes SQL:

Paste the context prompt below into your agent.
Point it at the reference pages listed on this page and at AGENTS.md in the starter kit. Many coding agents read that file on their own.
Make it run every SQL statement against your live cluster before wiring it into code.
Better still: connect the Doris MCP Server (see Task A2) so your agent can inspect the schema and test queries itself.

Context promptpaste into your agent

I'm building a hybrid search app on Apache Doris 4.1 for a hackathon.Connection: MySQL protocol, host 127.0.0.1, port 9030, user root, empty password,database `hackathon`. Use a plain MySQL driver and raw SQL (no ORM).Table `doc_chunks` (299 rows): id, page, page_title, section, url,category (search / ai / lakehouse / ingestion / performance / table-design / operations),body (STRING, inverted index, english parser, phrase support),embedding (ARRAY<FLOAT> NOT NULL, 8 dims, unit length, HNSW index, l2_distance).Doris SQL rules — follow them exactly:1. Keyword search uses `body MATCH_ANY 'a b'` (OR), `MATCH_ALL` (AND), `MATCH_PHRASE`. Never LIKE.   The parser keeps stopwords, so pass keywords(text) from toy_embed, not the raw question.   Bind the user's text as a driver parameter.2. BM25 relevance is `score()`. It only works with a MATCH_* predicate in WHERE and   `ORDER BY score() DESC LIMIT n`, selected plain (`score() AS relevance`): no ROUND(),   no window function, no JOIN in that SELECT. Round, rank or join it in an outer query.3. Vector search: `ORDER BY l2_distance_approximate(embedding, <vector>) ASC LIMIT n`.   Pass the vector as a string and cast it: CAST(%s AS ARRAY<FLOAT>) with '[f1, ..., f8]'.   Never pass a list/array as a driver parameter.4. Hybrid = MATCH_* pre-filter + vector ORDER BY; show the distance, do not select score()   in that query. For fusion, rank two lists in subqueries and combine with RRF (k = 60);   a1/rrf.sql in the starter kit shows the pattern.5. Query vectors come from toy_embed(text) in a1/toy_embed.py (toyEmbed in a1/toy_embed.js).   Do not invent embeddings.Build: a CLI (or small web page) with keyword / vector / hybrid modes, a category filter,and a "show SQL" option. Run every SQL statement against the live cluster first.

Doris traps your agent will probably fall into

Trap	What to do instead
Using `LIKE '%term%'` for text search	`col MATCH_ANY 'a b'` (OR), `MATCH_ALL` (AND), `MATCH_PHRASE 'a b'` — they use the inverted index
Calling `score()` in an arbitrary query	`score()` needs a `MATCH_` predicate in `WHERE` and* `ORDER BY score() DESC LIMIT n`; otherwise Doris rejects the query
Rounding, joining or ranking `score()` in the same query	Keep `score() AS relevance` plain in a single-table query. Round it, join it or `ROW_NUMBER()` it in an outer query, and use a subquery, not a `WITH` CTE
Sorting by BM25 and vector distance in one `ORDER BY`, or selecting `score()` in a vector-ranked query	Not supported. Hybrid = a `MATCH_*` pre-filter + vector `ORDER BY`, showing the distance; or fuse two ranked lists (RRF)
Passing the query vector as a driver parameter	A list becomes `(…)` in pymysql and 8 separate arguments in mysql2. Pass the string `'[0.1, …]'` and write `CAST(%s AS ARRAY<FLOAT>)`, or inline the literal
Looking for a cosine ANN metric	ANN metrics are `l2_distance` and `inner_product`. Normalize vectors to unit length; then L2 order equals cosine order

Build it

5 milestones, M0 → M4. The panel follows the step you are reading.

M0
Milestone 05 min
Load the data
From inside the starter-kit folder:
M0 · Load the databash
```
docker exec -i doris mysql -uroot -h127.0.0.1 -P9030 < seed/a1_doc_chunks.sql
```
Checkpoint
The load ends with a count per category: 7 categories, 299 sections in total.
M1
Milestone 115 min
Keyword search with BM25
Open a SQL shell (./doris.sh sql, or the docker exec command above) and run:
M1 · Keyword search with BM25sql
```
USE hackathon;SELECT page_title, section, score() AS relevanceFROM doc_chunksWHERE body MATCH_ANY 'vector index recall'ORDER BY relevance DESCLIMIT 10;
```
Then try MATCH_ALL (every term must appear) and MATCH_PHRASE 'inverted index' (adjacent, in order). Wire it up: input box → SQL → result list. Pass the user's text as a driver parameter (MATCH_ANY %s), never by pasting it into the SQL: the first apostrophe breaks the query. a1/starter.py and a1/starter.js do exactly this.
Typed a whole question? The english parser keeps stopwords, so 'how do I …' matches a third of the table. Pass keywords(text) from the toy embedder instead: it drops stopwords, plus apache and doris, which nearly every section mentions.
Checkpoint
The top hits for vector index recall come from the Vector Index and Hybrid Search pages; the first is How does the Apache Doris vector index work?
M2
Milestone 210 min
Add filters
M2 · Add filterssql
```
SELECT page_title, section, category, score() AS relevanceFROM doc_chunksWHERE body MATCH_ANY 'vector index recall'  AND category = 'search'ORDER BY relevance DESCLIMIT 10;
```
Expose the category as a dropdown or a CLI flag (the 7 values are listed under The data). Extra conditions are plain SQL: category IN (…), page = ….
Checkpoint
The Vector Index sections disappear — that page is filed under ai — and Hybrid Search moves to the top.
M3
Milestone 315 min
Vector search
Embed the question with the toy embedder, from code or from the command line:
M3 · Embed the questionpython
```
from toy_embed import toy_embed, to_sql_arrayvec = to_sql_array(toy_embed("store messy nested records whose schema keeps changing"))# '[0.005740, 0.005740, 0.999885, ...]'  ·  CLI: python a1/toy_embed.py "..."
```
M3 · Vector searchsql
```
SELECT page_title, section,       l2_distance_approximate(embedding,         [0.005740, 0.005740, 0.999885, 0.005740, 0.005740, 0.005740, 0.005740, 0.005740]) AS distFROM doc_chunksORDER BY dist ASCLIMIT 10;
```
From code, keep the SQL fixed and pass the vector string as a parameter: l2_distance_approximate(embedding, CAST(%s AS ARRAY<FLOAT>)). A Python list or a JavaScript array as the parameter does not work.
Checkpoint
The first six hits are all VARIANT sections, although the question says neither JSON nor VARIANT. Keyword mode on the same words mixes in Metadata Cache and Binlog sections.
M4
Milestone 420 min
Hybrid
Question: "stream kafka changes into doris exactly once". The vector is its toy embedding; the keyword filter keeps only sections that say kafka:
M4 · Hybridsql
```
SELECT page_title, section,       l2_distance_approximate(embedding, [0.007020, 0.007020, 0.007020, 0.007020, 0.999827, 0.007020, 0.007020, 0.007020]) AS distFROM doc_chunksWHERE body MATCH_ANY 'kafka'                     -- keyword pre-filter (inverted index)  AND category IN ('ingestion', 'lakehouse')     -- structured filterORDER BY dist ASC                                -- vector ranking (ANN)LIMIT 10;
```
- L4WHERE body MATCH_ANY 'kafka'keyword pre-filter (inverted index)
- L5AND category IN ('ingestion', 'lakehouse')structured filter
- L6ORDER BY dist ASCvector ranking (ANN)
In your app, one input feeds both halves: keywords(text) for the MATCH_ANY, toy_embed(text) for the vector. MATCH_ANY on several words is a loose filter; offer a "must contain" box or MATCH_ALL when users want it strict. Don't select score() here: show the distance.
Add a mode switch (keyword / vector / hybrid) and a Show SQL toggle. Compare the three lists for the same question — where do they disagree, and why?
Checkpoint
Every hit mentions Kafka, led by How does the Apache Doris Kafka and CDC integration work? Vector mode alone ranks Pipeline Execution Engine › Overview first for the same question.

Ship it

Check it off, show it, collect your badge.

Definition of Done

Your checklist

A user can type any free-text query (CLI argument, prompt, or text box).
Three modes work: keyword (BM25 via score()), vector (ANN), hybrid (keyword pre-filter + vector ranking).
At least one structured filter (category).
Results show page, section and score or distance; the executed SQL can be displayed.
Submitted as a pull request: your GitHub-ID folder with the code and a README (how to run + one screenshot).

Submit & get your badge

Take part → submit → get your badge.

Put your work in a folder named after your GitHub ID: the code, plus a README with what it does, how to run it, one screenshot or GIF, and the Doris features you used.
Open a pull request that adds it to morningman/demo-env as doris-hackathon-glasgow-2026/<your-github-id>/. Fork and push, or use GitHub's Add file → Upload files. Nothing else to fill in.
Show it at the Doris table (a 2-minute demo is plenty), or at the show-and-tell around 14:30.
Join the Apache Doris Slack and say hi in #dev: questions, submissions and badges are all handled there.

Everyone who takes part in a task on site gets the Apache Doris Contributor badge: no merged pull request to Doris needed. Didn't finish by 15:00? Keep going — submissions are open until 31 October. Teams are fine, but not needed: list every member's GitHub ID in the README.

About the Contributor badge

Stretch goals

Reciprocal Rank Fusion. Fuse the BM25 list and the vector list with 1/(60 + rank) in one statement: a1/rrf.sql in the starter kit ranks each list with ROW_NUMBER() one level above the plain score() query and merges them with a FULL OUTER JOIN. The RRF page walks through the same pattern on a four-row example.
Real embeddings with EMBED() (bring your own API key): create an AI resource, create doc_chunks_v2 with the model's dim, fill it with INSERT INTO doc_chunks_v2 SELECT …, EMBED(body) FROM doc_chunks (the vector column is NOT NULL, so insert rather than update), and embed the query in SQL too. See Embedding.
Search-as-you-type with MATCH_PHRASE_PREFIX.
Highlight matched terms in the results (Doris returns scores, not highlights — render them in your app).
Latency badge: show how many milliseconds each mode takes.

Troubleshooting & references

Symptom	Fix
`docker: command not found` on macOS	Start Docker Desktop. If it is running, link the CLI: `sudo ln -s /Applications/Docker.app/Contents/Resources/bin/docker /usr/local/bin/docker`
`error getting credentials` when pulling images	Remove the `credsStore` field from `~/.docker/config.json` (local dev only)
`port is already allocated`	Something else uses 9030, 8030 or 8040, often another Doris. Stop it (`docker ps`), or publish other host ports, e.g. `-p 19030:9030`, and connect to 19030
`The container name "/doris" is already in use`	You started it before: `docker start doris`. To start over, `docker rm -f doris` (this deletes its data)
The container exits, or never turns `(healthy)`	Read `docker logs doris`. Usually it is memory: give Docker Desktop 6 GB or more (Settings → Resources). On Apple Silicon, don't add `--platform linux/amd64`
`permission denied: ./doris.sh`	Your unzip tool dropped the executable bit: `chmod +x doris.sh`, or run `bash doris.sh start`
`Current database is not set`	Run `USE hackathon;` first, or connect with `-Dhackathon`
`score() function requires WHERE clause with MATCH function, ORDER BY and LIMIT`	Add a `MATCH_*` predicate, `ORDER BY score() DESC` and a `LIMIT`, and keep `score()` unwrapped (see the traps above)
`Ann topn query vector cannot be empty`	The vector literal is empty or still a placeholder: paste the output of `python a1/toy_embed.py "…"`
`Can not found function 'l2_distance_approximate' which has 9 arity`	Your driver expanded an array parameter into 8 values: pass the vector as a string with `CAST(? AS ARRAY<FLOAT>)`
Stuck for more than 10 minutes	Come to the Doris table and ask Mingyu, or post in `#dev` on the Apache Doris Slack

References

Hybrid Search BM25 Full-text Search Vector Index Reciprocal Rank Fusion Embedding

SketchWhat you'll build · wireframe, not a screenshot

Start Doristerminal

docker pull apache/doris:all-in-one-4.1.3   # 1.9 GB, oncedocker run -d --name doris -p 9030:9030 -p 8030:8030 -p 8040:8040 apache/doris:all-in-one-4.1.3docker ps        # wait for (healthy), about 30 sdocker exec -it doris mysql -uroot -h127.0.0.1 -P9030

Host: 127.0.0.1
Port: 9030
User: rootno password
Database: hackathon

Starter kitdownloaddoris-hackathon-glasgow-2026.zip

Route60–120 min · M0 → M4 → done

M0Load the data5 min
M1Keyword search15 min
M2Add filters10 min
M3Vector search15 min
M4Hybrid20 min
DoDDone→ submit

Tablehackathon.doc_chunks · sql

CREATE TABLE doc_chunks (  id          INT           NOT NULL,  page        VARCHAR(64)   NOT NULL,   -- doc slug, e.g. 'hybrid-search'  page_title  VARCHAR(128)  NOT NULL,  section     VARCHAR(256)  NOT NULL,  category    VARCHAR(32)   NOT NULL,   -- see the 7 values below  url         VARCHAR(256)  NOT NULL,  body        STRING        NOT NULL,  embedding   ARRAY<FLOAT>  NOT NULL,   -- 8-dim toy vector, unit length  INDEX idx_body (body) USING INVERTED PROPERTIES ("parser" = "english", "support_phrase" = "true"),  INDEX idx_category (category) USING INVERTED,  INDEX idx_emb (embedding) USING ANN PROPERTIES (    "index_type" = "hnsw", "metric_type" = "l2_distance", "dim" = "8"))DUPLICATE KEY(id)DISTRIBUTED BY HASH(id) BUCKETS 1PROPERTIES ("replication_num" = "1");

M0 · Load the databash

docker exec -i doris mysql -uroot -h127.0.0.1 -P9030 < seed/a1_doc_chunks.sql

M1 · Keyword search with BM25sql

USE hackathon;SELECT page_title, section, score() AS relevanceFROM doc_chunksWHERE body MATCH_ANY 'vector index recall'ORDER BY relevance DESCLIMIT 10;

M2 · Add filterssql

SELECT page_title, section, category, score() AS relevanceFROM doc_chunksWHERE body MATCH_ANY 'vector index recall'  AND category = 'search'ORDER BY relevance DESCLIMIT 10;

M3 · Embed the questionpython

from toy_embed import toy_embed, to_sql_arrayvec = to_sql_array(toy_embed("store messy nested records whose schema keeps changing"))# '[0.005740, 0.005740, 0.999885, ...]'  ·  CLI: python a1/toy_embed.py "..."

M3 · Vector searchsql

SELECT page_title, section,       l2_distance_approximate(embedding,         [0.005740, 0.005740, 0.999885, 0.005740, 0.005740, 0.005740, 0.005740, 0.005740]) AS distFROM doc_chunksORDER BY dist ASCLIMIT 10;

M4 · Hybridsql

SELECT page_title, section,       l2_distance_approximate(embedding, [0.007020, 0.007020, 0.007020, 0.007020, 0.999827, 0.007020, 0.007020, 0.007020]) AS distFROM doc_chunksWHERE body MATCH_ANY 'kafka'                     -- keyword pre-filter (inverted index)  AND category IN ('ingestion', 'lakehouse')     -- structured filterORDER BY dist ASC                                -- vector ranking (ANN)LIMIT 10;

Context promptpaste into your agent

I'm building a hybrid search app on Apache Doris 4.1 for a hackathon.Connection: MySQL protocol, host 127.0.0.1, port 9030, user root, empty password,database `hackathon`. Use a plain MySQL driver and raw SQL (no ORM).Table `doc_chunks` (299 rows): id, page, page_title, section, url,category (search / ai / lakehouse / ingestion / performance / table-design / operations),body (STRING, inverted index, english parser, phrase support),embedding (ARRAY<FLOAT> NOT NULL, 8 dims, unit length, HNSW index, l2_distance).Doris SQL rules — follow them exactly:1. Keyword search uses `body MATCH_ANY 'a b'` (OR), `MATCH_ALL` (AND), `MATCH_PHRASE`. Never LIKE.   The parser keeps stopwords, so pass keywords(text) from toy_embed, not the raw question.   Bind the user's text as a driver parameter.2. BM25 relevance is `score()`. It only works with a MATCH_* predicate in WHERE and   `ORDER BY score() DESC LIMIT n`, selected plain (`score() AS relevance`): no ROUND(),   no window function, no JOIN in that SELECT. Round, rank or join it in an outer query.3. Vector search: `ORDER BY l2_distance_approximate(embedding, <vector>) ASC LIMIT n`.   Pass the vector as a string and cast it: CAST(%s AS ARRAY<FLOAT>) with '[f1, ..., f8]'.   Never pass a list/array as a driver parameter.4. Hybrid = MATCH_* pre-filter + vector ORDER BY; show the distance, do not select score()   in that query. For fusion, rank two lists in subqueries and combine with RRF (k = 60);   a1/rrf.sql in the starter kit shows the pattern.5. Query vectors come from toy_embed(text) in a1/toy_embed.py (toyEmbed in a1/toy_embed.js).   Do not invent embeddings.Build: a CLI (or small web page) with keyword / vector / hybrid modes, a category filter,and a "show SQL" option. Run every SQL statement against the live cluster first.

Apache Doris Hackathon

All tasks

Back to the course