DevOps KB Pages

What is RAG (Retrieval-Augmented Generation)? Retrieval-Augmented Generation (RAG) is the process of optimizing the output of a large language model, so it references an authoritative knowledge base outside of its training data sources before generating a response. Large Language Models (LLMs) are trained on vast volumes of data and use billions of parameters to generate original output for tasks like answering questions, translating languages, and completing sentences. RAG extends the already powerful capabilities of LLMs to specific domains or an organization's internal knowledge base, all without the need to retrain the model. It is a cost-effective approach to improving LLM output so it remains relevant, accurate, and useful in various contexts. Related Blogs: KB-OpenAI References: What Is RAG? Best Practices in Retrieval Augmented Generation

  Embeddings: What are embeddings : OpenAI’s text embeddings measure the relatedness of text strings. Embeddings are commonly used for: Search (where results are ranked by relevance to a query string) Clustering (where text strings are grouped by similarity) Recommendations (where items with related text strings are recommended) Anomaly detection (where outliers with little relatedness are identified) Diversity measurement (where similarity distributions are analyzed) Classification (where text strings are classified by their most similar label) An embedding is a vector (list) of floating point numbers. The  distance  between two vectors measures their relatedness. Small distances suggest high relatedness and large distances suggest low relatedness. What is the difference between embedding and vector database? Embeddings encode all types of data into vectors that capture the meaning and context of an asset. This allows us to find similar assets by searching for neighborin...