Problem
Manual API test design does not scale.
Execution tools like Postman and JMeter still depend on manually designed test cases. APIT demonstrates LLM-based test generation: it reads API documentation, defines test scope, generates structured test cases, executes them, and reports pass/fail outcomes.
What I built
Controlled data synthesis and small-model fine-tuning pipeline.
Manual API documents were distilled into prompts, used to generate 1,258 bilingual English/Vietnamese samples, and converted into structured test-case outputs with reasoning. Qwen-2.5-3B and Llama-3.2-3B were fine-tuned with LoRA/4-bit training under identical hyperparameters.
Evaluation
Structured output quality measured with reference-based metrics.
Generated test cases were evaluated with precision, recall, and F1 using fuzzy string similarity, numeric tolerance, and categorical matching. Qwen-2.5-3B LoRA achieved 0.655 macro-F1, outperforming the Llama-3.2-3B baseline.
Reasoning quality was assessed through logical validity, coherence, and groundedness, aligning with the agent evaluation direction.
Agent Testing Connection
APIT as evidence for the Agent Testing / Evaluation direction.
APIT demonstrates the evaluation methodology behind ATA. The same LLM-as-a-Judge patterns, structured output evaluation, and fine-tuning experiments used in APIT inform how RAG and agent workflows should be tested and validated at scale.
The APIT pipeline — document understanding → test generation → automated evaluation → regression reporting — mirrors the evaluation loops that ATA applies to RAG and agent workflows as a completed standalone validation platform.