Transforming Alteon Support with AI-Powered Support-File Analysis

Why Traditional RAG Falls Short for TSDMP Analysis

RAG has become the go-to architecture for building AI systems that need to process and analyze domain-specific documents. At its core, RAG combines the power of large language models with external knowledge retrieval, enabling systems to provide accurate, contextual responses based on specific datasets rather than just pre-trained parametric knowledge.

1. Domain Expertise Gap

However, the challenge of analyzing technical support files from specialized and proprietary networking and cyber-security devices, like the Alteons is unique. These files contain specialized jargon, proprietary software terminology, hardware-specific subsystem logs, and diagnostic patterns that have typically never appeared in any public dataset, and thus they represent a knowledge domain that diverges quite significantly from standard IT/networking troubleshooting data or data from ubiquitous vendors like Cisco, which the LLM might have been exposed to during training. Since an LLM relies on its parametric knowledge to contextualize, interpret, and reason about the retrieved content, language models tend to lack underlying conceptual patterns to interpret domain-specific terminology, relationships, and patterns and perform inadequately when there is a strong divergence between retrieved content and the model’s training distributions

2. Scale & Cost Challenge

TSDMP files vary dramatically in size, largely determined by the Alteon platform that produced them (VX, Standalone, vADC, or VA). In the field, we see anything from ≈ 6,000 to ≈ 200,000 lines per file, with a median around 140,000 lines. This amounts to a range of ~140K to ~1.3M input tokens without even considering additional prompt tokens for optimal in-context learning. Adopting best in class prompting techniques to elicit reasoning traces in LLM responses, increases the number of output tokens to roughly the number of input tokens that were consumed.

Using as an example the current Claude 3.5 Sonnet rates, this footprint translates to ≈ US $0.90 for the smallest files, up to ≈ US $30 for the largest, with the median landing near US $15.

In short, simply streaming raw TSDMP files through an LLM is both expensive and operationally unpredictable—a serious concern when these files are artefacts that support engineers rely on daily to diagnose and troubleshoot customer’s issues. If the support queue were to run dozens of these files every day, the spend would escalate quickly and make it economically impractical for regular use in a support environment—a problem we need to solve.