Selected Work

Seven years of one question, arriving at its current form: not what a memory agent should remember, but when a memory should be allowed to surface in AI conversation systems.

The question began optically. A 2019 light work showed that the full spectrum sits inside a white line, entirely imperceptible until a prism separates it out — what is present but unreachable, and what has to be introduced to reach it. Everything since has been that question in a different register: educational agents that decide how much to reveal and when, a retrieval system that taught me what is not the hard part, and a memory prototype I have built and used alone for two years. That prototype is where the question took its current form. Not what an agent should remember, but when a memory should be allowed to surface — and what it costs when the timing is wrong.

ONGOING

Hearth: A Two-Year Solo Prototype for Long-Term Conversational Memory

(July 2024 – present)

A supportive conversational agent I have been building and living with since July 2024. Its memory layer separates writing a new memory from revising an existing one — a PATCH / INSERT write policy rather than an append-only log — so that what the system knows about a person accumulates as a revisable account rather than a transcript. It is where the question my PhD asks stopped being abstract.

CONTRIBUTION

Sole author: concept, backend, memory architecture, and mobile client. No collaborators, no funding, no publication deadline, and no users other than myself — maintained across two years because I kept returning to it.
FastAPI backend with a LangGraph-orchestrated conversation graph
MongoDB memory store with a PATCH / INSERT write policy distinguishing revision from accumulation
React Native mobile client, used intermittently but continuously since July 2024

RESEARCH RELEVANCE

This is the evidence of sustained devotion I would point to first, precisely because nothing external sustained it. It has one user and no audience. What two years of living with it taught me is that a memory system's hardest failures are not retrieval failures. NBChat would surface something accurate, relevant, and correctly retrieved — and it would still land badly, because it arrived at the wrong moment. Relevance optimisation has no way to represent that error; it has no action for holding, for cueing obliquely, for asking first, or for saying nothing. Naming that error — memory-timing-induced breakdown — and studying it in a population where mistiming carries real cost is the proposal this prototype produced.

UNDER REVIEW

MobiClues: Constructivist Learning through Gamified Discovery and Multi-Agent AI Tutoring

(IEEE TALE 2026)

Read preprintopen_in_new

A mobile learning application in which learners photograph real-world objects that a vision LLM turns into described virtual items, fuse them into new artifacts, and receive relevance scores as incremental clues — while a multi-agent retrieval-augmented tutoring system diagnoses prior knowledge, calibrates scaffolding to the diagnosed level, and decides per turn whether retrieval is pedagogically warranted. Every architectural component is accountable to a named constructivist principle.

CONTRIBUTION

First author. Sole author of the pedagogical framework, the system architecture, and the argument holding each component accountable to a named constructivist principle; led the mobile prototype together with a technical team headed by Martin Choy. Implemented the two-phase learning cycle over a FastAPI service with a LangGraph-orchestrated multi-agent tutoring pipeline and shared session state.

RESEARCH RELEVANCE

MobiClues contains the earliest working version of the question my PhD asks. Its Gating Agent decides not what to retrieve but whether to retrieve at all, and it decides on the basis of the learner's diagnosed state rather than the relevance of the query — retrieval serving the person rather than the answer. My research direction moves that decision from the pedagogical register to the affective one: not whether a learner needs grounding, but whether a person is ready for a memory to surface. The discovery phase is also a direct technical precedent for the proposed Stage 1 elicitation study, which likewise uses participants' own photographs as retrieval cues.

PUBLISHED

MetaClues: Constructivist Learning via Progressive Clues from Generative AI in a Metaverse Environment

(IEEE MetaCom 2025)

Read Preprintopen_in_new View on IEEEopen_in_new

A gamified Metaverse framework in which learners photograph real-world objects, convert them into 3D items through a generative AI pipeline, and fuse them to approach a hidden target artifact. A reasoning LLM scores each item's semantic proximity to that artifact — but the score is never returned as a verdict. It is converted into a progressive clue, so the system releases what it knows in graded increments. Evaluated in a pilot study with 19 university students.

CONTRIBUTION

First author. Designed and implemented the generative AI pipeline — vision-LLM captioning of learner photographs, reasoning-LLM item fusion, text-to-image synthesis, and image-to-3D generation — together with the multi-faceted relevance scoring mechanism that converts semantic proximity into progressive clues. Designed the study instruments and ran the pilot. The underlying Metaverse platform was led by me and built by Martin Choy's technical team; William K. W. Cheung supervised.

RESEARCH RELEVANCE

The scoring system in MetaClues knows the answer and deliberately does not give it. Saying everything at once would end the inquiry, so the system withholds and releases in increments — which is to say it runs a disclosure policy, arrived at for pedagogical reasons rather than affective ones. This is the earliest point in my work where an agent decides how much of what it knows to surface, and it is the same structural decision my proposed Stage 2 studies under emotional stakes rather than educational ones. The pilot (n = 19; SUS 3.72; TAM PU 3.82, PE 3.79) is also where I learned to instrument and run the kind of controlled comparative study that Stage 2 requires.

TECHNICAL

RAG & Embedding Fine-Tuning for Domain-Specific Retrieval

(HKBU MSc DAAI COMP7045 NLP course, 2024)

Read reportopen_in_new

A solo project testing how far retrieval quality in a RAG pipeline can be improved by fine-tuning the embedding model alone, under deliberately scarce data: a single research paper as the entire corpus, and 150 LLM-generated synthetic queries as the only training signal.

CONTRIBUTION

Sole author. Built a structure-aware parse of the source paper so that chunks followed the document's own headings and tables rather than arbitrary character offsets; generated synthetic query–chunk pairs and fine-tuned sentence-transformers/all-mpnet-base-v2 with MultipleNegativesRankingLoss; evaluated with both the sentence-transformers InformationRetrievalEvaluator and the RAGAS framework, and deployed the result in a LangChain / Chroma QA application. Accuracy@1 rose from 0.37 to 0.60 and NDCG@10 from 0.51 to 0.76; RAGAS context recall rose from 83.5% to 94.5%, with context relevancy the single metric that did not improve.

RESEARCH RELEVANCE

This is the least art-adjacent thing I have made, and I keep it in view for two reasons. It is where I learned that retrieval quality is a measurable and largely tractable engineering problem — which is exactly why the problem I now want to study is not a retrieval problem. Everything this project optimises, my research direction treats as already handled and beside the point: the memory can be perfectly retrieved and still arrive wrongly. It is also the technical grounding for the optional Stage 3 formalisation, where a disclosure policy would be evaluated against multi-session memory benchmarks — an engineering extension rather than a thesis contribution.

CONDITIONALLY ACCEPTED

We Can't See the Rainbow in the White

(SIGGRAPH Asia 2020, Art Gallery)

View Artworkopen_in_new

A durational light work that decomposes a projected white line through prisms and moving image. White is not an independent colour but a mixture of the visible spectrum; the work removes green, then red, so the line shifts from white to magenta to pure blue — a rainbow, then a duo-coloured rainbow, then a monochromatic one. The spectrum is present in the white throughout. It only becomes perceivable when something separates it out.

CONTRIBUTION

Sole authorship: concept, optical and mechanical design, moving image, and physical computing (Arduino, servo motor, prism, brass, acrylic). Video, no sound, 8 min 45 s; size variable.

RESEARCH RELEVANCE

This is where the question starts, seven years before I had the vocabulary for it: what is fully present but not perceivable, and what has to be introduced for it to become perceivable again. My PhD direction asks the same question of autobiographical memory in people with acquired smell loss. The memories remain encoded after the olfactory cue that once opened them is permanently gone, and the research asks what an alternative pathway to them would look like — and what it costs when that pathway opens at the wrong moment. The prism and the conversational agent occupy the same structural position. The difference is that a prism cannot mistime.