Every database project has the same hidden friction point: the data may be powerful, but the interface is not always human-friendly. A user can know exactly what they want to ask, yet still get blocked by syntax, schema structure, and query logic.
That gap is what pushed me toward this project.
I set out to build a Natural Language to MongoDB engine capable of converting plain English requests into executable MongoDB shell queries. The goal was not just to create another AI demo that generated code. The goal was to build a system that could be deployed in real-world environments: fast, lightweight, private, and capable of understanding database schemas with high accuracy. The project was designed around a 16GB CPU target, strict latency constraints, and efficient local deployment from the start.
The problem
Imagine asking:
“Find all vehicles involved in accidents with severity level above 3 during the last month.”
A human immediately understands the intent.
A database does not.
To answer that question correctly, a model must understand the user’s intent, identify the correct collection, link language concepts to schema fields, select the appropriate MongoDB operators, generate syntactically valid queries, and ensure the result remains executable. MongoDB makes this even harder because of nested documents, flexible schemas, and NoSQL-specific operators. The challenge was never just generating text. The challenge was generating correct queries.
Why not just use ChatGPT?
This was the first obvious question, and it is a fair one.
ChatGPT is excellent for general-purpose reasoning, but this project had different requirements. We needed privacy, edge deployment, low cost at scale, and full control over the system.
A cloud-hosted model introduces what I think of as the Cloud Latency Tax: every request pays for network transfer, remote inference, and the round trip back to the user. That is acceptable for casual use, but it becomes a real bottleneck when the system must behave like an interactive database assistant. It also becomes expensive when usage scales, because every query adds to API cost. In a high-traffic setting, that recurring cost grows linearly with usage. A local model avoids both the latency penalty and the scaling tax.
Privacy was another major reason. In many enterprise settings, database schemas, logs, and analytics data should not leave the organization’s infrastructure. A local model keeps the entire pipeline on-premise, without exposing schemas or user queries to third-party APIs. That matters not just for compliance, but for trust.
Edge deployment also mattered. Many industrial systems run in environments where internet access is restricted, unreliable, or intentionally disabled. A cloud-dependent assistant is simply the wrong shape for that problem. Our model had to run on a 16GB CPU machine, without assuming GPUs or always-on connectivity.
So the point was never to replace ChatGPT. The point was to build something better suited to a narrow, high-value task: private, fast, schema-aware query generation under real deployment constraints.
Building the dataset
One lesson became obvious early: model performance is often limited more by data quality than model size.
Instead of relying on public data, I curated a structured dataset of 1,000 query pairs across 10 database schemas and 5 difficulty levels. The schemas were intentionally varied, ranging from single-collection setups to multi-collection scenarios, so the model would learn both simple and complex query patterns.
A particularly useful design choice was to reverse the traditional workflow. Instead of starting from natural language and trying to infer MongoDB queries, I generated the target MongoDB queries first and then produced natural language descriptions using GPT-4o Mini, followed by an automated validation agent. That gave me a cleaner, more controlled training set with better schema-query alignment and less ambiguity.
Selecting the right model
With high-end GPUs off the table, I had to find a base model under 3 billion parameters that could execute swiftly on a 16GB CPU. I exhaustively benchmarked several compact, instruction-tuned candidates on fixed text-to-query tasks, tracking prompt processing speed (pp512) and text generation speed (tg128). The final choice was Qwen2.5-0.5B-Instruct because it gave the best balance of instruction-following ability, compactness, and runtime efficiency.
That decision mattered. Bigger models are not always better, especially when the task is specialized and the deployment environment is constrained. Sometimes the best model is the one that fits the problem rather than the one that simply dominates the benchmark charts.
Fine-tuning with QLoRA
Training was done with QLoRA, Unsloth, TRL, and 4-bit quantization, which made the fine-tuning pipeline memory-efficient enough to run within the CPU-oriented setup. The supervised fine-tuning stage gave the first major jump in performance. After SFT, the model reached about 78% accuracy.
That was a strong start, but not enough. The remaining gap came from cases where the model produced something that looked plausible but was still wrong in schema linking, field selection, or operator usage. In structured generation tasks, that is the dangerous failure mode: the output can sound correct while still being unusable.
Going beyond SFT: DPO and PPO
To push the model further, I used preference-based alignment.
DPO helped improve consistency by teaching the model which outputs were preferable when there were multiple candidates. It was a useful refinement step, but the gains were limited on their own.
The bigger improvement came from PPO, where I introduced a custom reward function focused on the things that actually matter in database query generation: syntax correctness, schema linking correctness, and correct operator usage. In other words, the model was not rewarded for sounding fluent. It was rewarded for being right.
That combination of SFT, DPO, and PPO took the system from 78% after SFT to 94% overall accuracy. The main lesson here was that supervision can teach the shape of the task, but preference optimization can help teach the judgment needed to solve it reliably.
Making inference practical
Accuracy alone does not make a product usable. Latency matters.
To keep the system practical, I integrated llama.cpp for local CPU inference and added prompt caching so repeated static schema segments would not need to be re-encoded every time. That reduced redundant computation and made the system feel much more responsive in use. The project report notes a drop in latency from 819 ms to 21.98 ms, which is exactly the kind of improvement that turns a prototype into something interactive. The architecture was also designed around a strict low-latency target for constrained environments.
That part of the project taught me something important: inference optimization is not a side quest. For real systems, it is part of the core model design.
What this project taught me
This project changed how I think about AI systems.
A general-purpose model is powerful, but a specialized model is often more useful when the task is narrow, the environment is constrained, and the cost of being wrong is high. This was not about building a bigger model. It was about building the right system: private, fast, schema-aware, and deployable where it actually needs to run.
The most satisfying part was seeing plain English turn into valid MongoDB commands in a low-latency local setup. That moment made all the dataset work, fine-tuning, reward design, and optimization effort feel worth it.
Closing thought
In the end, this was more than an NLP demo. It was a complete pipeline for turning natural language into executable database operations under real constraints.
That is what made the project meaningful to me: not just that it worked, but that it worked for the right reasons.