How I Built Mihir AI

Why I built it

A portfolio is usually a static brochure. Recruiters and hiring managers still have to hunt for answers: which project matches this role, which certification is current, is he available for contract work. I wanted a product that does that work for them.

Two constraints shaped every decision: run at effectively $0/month on free tiers, and treat a public AI endpoint with storage and email as a security surface from day one.

RAG in plain English

Retrieval-Augmented Generation is an open-book exam for the model. Instead of answering only from training memory or from one giant profile stuffed into every prompt, Mihir AI first searches a small library of verified portfolio passages, selects the most relevant ones, then writes an answer using that evidence.

How a question becomes an answer

1. 1 Question A visitor asks something about experience, skills, or availability.
2. →
3. 2 Search verified portfolio Semantic and keyword search run over indexed passages.
4. →
5. 3 Select evidence Fusion and optional reranking choose the top passages.
6. →
7. 4 Generate answer The model answers from those passages with citation IDs.
8. →
9. 5 Check citation IDs Server-side validation keeps only IDs that were actually retrieved.

From v1 to hybrid RAG

The first production chatbot put a structured professional profile directly into the system prompt. That was a deliberate stage-one design: small profile, fast to ship, easy to audit, and enough to build sessions, CSRF, rate limits, Turnstile, and lead capture around a real product.

As the system matured, selective context, source IDs, and measurable retrieval quality became more valuable than sending the full profile on every turn. Hybrid RAG was introduced behind a RAG_ENABLED feature flag so rollout and rollback stayed safe. When the flag is off, the legacy profile-in-prompt path remains available.

How the knowledge base is built

Mihir’s experience, skills, projects, education, and certifications live as readable Markdown files. An offline script turns their sections into searchable passages, embeds them, and stores them in D1.

1. Markdown5 source files
2. →
3. Chunk## headings
4. →
5. Heading context[Title > Heading]
6. →
7. HashSHA-256
8. →
9. EmbedBGE base
10. →
11. NormalizeL2 unit length
12. →
13. StoreD1 + FTS5

What happens when someone asks a question

Security checks run before expensive AI generation. Sessions are anonymous; there is no user login. CSRF proves the browser holds a token issued with that session; it is not authentication.

1. 1
   Validate JSON and input limits

   Body shape, required fields, 2,000-character message cap.

2. 2
   Load and validate the anonymous session

   D1 lookup; reject ended or missing sessions.

3. 3
   Verify HMAC-signed CSRF

   HMAC-SHA256 over session id and secret; constant-time compare.

4. 4
   Check D1 session message cap

   Hard ceiling so one conversation cannot drain quota.

5. 5
   Apply global and per-IP KV rate limits

   Time-bucketed counters with TTL.

6. 6
   Require Turnstile when threshold is reached

   Server-side siteverify after sustained use.

7. 7
   Screen direct prompt-injection patterns

   Regex filter before any model call; canned refusal on match.

8. 8
   Load up to six recent messages

   Short conversation history for generation context.

9. 9
   Run RAG retrieval

   When RAG_ENABLED is on: hybrid retrieve → top 5 passages.

10. 10
    Generate an answer

    Llama 3.1 8B FP8 via Workers AI.

11. 11
    Validate citation metadata & log grounding

    Filter unknown IDs; lexical overlap is observability-only.

12. 12
    Persist the turn to D1

    User message and assistant reply stored together.

13. 13
    Return JSON to the widget

    reply, answer, citations, retrieved.

How hybrid retrieval works

This is the centrepiece of the current architecture: two independent search methods, fused by rank (not raw score), then optionally reordered by an LLM before the top five passages reach generation.

1. Questionuser message
2. →
3. BGE embed768-dim query
4. →
5. Exact vectordot product scan
6. →
7. FTS5 / BM25keyword ranks
8. →
9. RRF fusionk = 60
10. →
11. LLM rerankoptional
12. →
13. Top 5to generation
14. →
15. Cited answerIDs validated

Query embedding

The question is embedded with the same BGE model used at index time and L2-normalized so that a dot product equals cosine similarity.

Exact vector search

With only 21 chunks, the Worker loads stored vectors and scores every one with a linear scan. A dedicated vector database would add operational cost without benefit at this scale. Something like Cloudflare Vectorize becomes appropriate only when the corpus grows materially larger. Mihir AI does not currently use Pinecone, pgvector, or Vectorize.

Keyword search

SQLite FTS5 with BM25 finds exact terms such as product names, company names, and certification codes that pure semantic search can miss.

Reciprocal Rank Fusion (RRF)

Vector similarity and BM25 scores live on different scales, so averaging them is meaningless. RRF combines rank positions instead:

score(chunk) = Σ  1 / (k + rank)
with k = 60

LLM reranking

Up to about 20 fused candidates can be sent to Llama 3.1 for a relevance reorder. The reranker must return up to five allowed passage IDs. If the call fails or returns invalid output, the system falls back to the RRF order, retrieval never fails solely because reranking failed.

Generation

The final five passages, up to six recent history messages, and the current question go to the generation model. The RAG system prompt requires the model to use only retrieved context, cite exact chunk IDs, avoid guessing, and refuse when evidence is insufficient. Retrieved text is framed as reference material, never as instructions.

Grounding and citation honesty

Traceability is a first-class design goal, and so is being precise about what the code actually guarantees.

What the server does today

• Extracts citation IDs matching [doc_id#chunk_index] from the model reply.
• Compares those IDs to the retrieved set for that request.
• Excludes unknown IDs from the structured citations array in the API response.
• Logs rag_hallucinated_citation when an unknown ID appears.
• Runs a deterministic lexical-overlap assessment via assessGrounding().
• Logs rag_ungrounded_claim when a citation-bearing sentence looks unsupported.

Honest limitations

• A real citation ID does not prove every claim in that sentence is true; it only confirms that the ID was among the retrieved set.
• An uncited factual statement is not deterministically blocked today.
• The frontend currently renders reply only. It does not show a source panel, even though the backend already returns citations and retrieved.

These are transparent engineering limitations and logical next improvements, not hidden failures.

Security around the AI

RAG did not replace the security model; it sits behind it. Every mutating request still has to clear the same edge controls before any retrieval or generation cost is spent.

Input validation, body size limits, and control-character stripping still apply at the API boundary. Lead email is sent only to me from Worker code after an explicit form submit, never from free-form model output.

Tools I used and why

Each tool has a narrow job. The stack stayed small on purpose: static frontend, serverless Worker, no frontend framework, no runtime npm dependencies in production Worker code.

How I built and validated it

The original product shipped in seven foundation phases. RAG did not exist in those phases; they built the protected product first.

Later stage: RAG expansion and production rollout

A separate expansion stage added the retrieval system without discarding the foundation:

• Markdown corpus and deterministic chunking
• Embedding and indexing scripts
• Hybrid retrieval, RRF, and optional LLM reranking
• Citation metadata validation and grounding observations
• Evaluation harness with a golden dataset
• RAG_ENABLED rollout flag with legacy fallback

AI-assisted development with human ownership

• Architecture and acceptance criteria remained human-owned.
• AI coding tools accelerated implementation and review.
• Changes were independently reviewed and tested (unit, integration, eval).
• Security posture and factual accuracy remained the engineer’s responsibility.

How I measured it

A controlled golden set drives retrieval and safety checks against the real retrieval code and live Workers AI (via the evaluation harness), not mock scores invented for marketing.

What these numbers mean

• Recall@5: of the passages that should be found for a question, what fraction appear in the top five results?
• MRR (Mean Reciprocal Rank): on average, how close to rank 1 is the first correct passage? Always first → 1.0; always second → 0.5.

Tradeoffs and what comes next

None of the “next” items are claimed as shipped. They are the natural follow-ons once you look honestly at the current system.