During early platform deployments, enterprise-scale qualifications containing high volumes of Knowledge Evidence (KE) points routinely caused system errors. Because Large Language Models (LLMs) operate under strict maximum token completion constraints, extensive single-shot mapping requests resulted in truncated, incomplete JSON payloads. This structural corruption triggered parsing failures, causing application timeouts and completely blocking RTO users from processing large, complex course files.
✔ Asynchronous processing for long-running AI operations
✔ Semantic similarity analysis and duplicate mapping prevention
✔ Live status monitoring and progress polling interfaces
2
The Solution
RannLab designed and executed a production-ready, hybrid mapping architecture to dynamically scale based on course payload sizes. Standard qualifications under 100 KE points process via a low-latency, single-call path. For massive qualifications exceeding 100 KE points, the backend activates a chunked batching pipeline that flattens nested data and splits requests into segmented batches of 25 KE points each. To preserve semantic context, each subsequent batch receives a global index of previously mapped items to eliminate data duplication. Furthermore, a specialized JSON repair utility was built into the API to strip markdown fences, balance trailing punctuation, and salvage slightly truncated responses. These heavy computational tasks are handled in the background by asynchronous queue workers using BullMQ and powered by a high-performance Redis caching layer, entirely separate from main HTTP threads.
