KB:LLM Vectors vs Embeddings

The sequence is from "Text" to "Tokens," then "Vectors," and finally "Embeddings."

In data processing and machine learning, the creation or extraction of a vector typically precedes the embedding process. Initially, data is converted into a vector form, which is a numerical representation, and then, embeddings are generated from these vectors. Embeddings are lower-dimensional representations that capture the relationships and features of the data, making it easier to use in various machine learning models. This generalized concept reflects the idea that vectors serve as the foundation upon which embeddings are built.









When a model is trained, the initial vectorized values (those random vectors assigned at the beginning) don’t exist separately after the embeddings are created. Here’s how it works:

Training Process:

  • Initial Vectors: These are the starting points, just random numbers. They exist at the beginning of the training process but are not kept in any permanent state.

  • Transformation into Embeddings: During training, the model continuously updates these vectors based on the data it processes. As the model learns, these vectors are adjusted to become more accurate representations of the words or tokens they represent.

After Training:

  • Embeddings Replace Initial Vectors: The original, random vectors effectively "disappear" in the sense that they are overwritten by the refined, meaningful embeddings. The final embeddings are what get stored and used by the model.

  • No Separate Storage: There’s no separate storage of the initial vectors versus the embeddings. The embeddings are just the evolved version of those vectors after training. The model only keeps the embeddings since they contain the meaningful, trained information.

To Sum Up:

The initial vectorized values aren’t kept around separately. They’re transformed through the training process into embeddings, which then replace the initial vectors entirely. So, from the model’s perspective, what starts as random vectors ends up as embeddings, with no need to retain the original random values.



References: 

- https://never-stop-learning.de/embedding-vector-for-azure-ai-search/

- https://www.linkedin.com/pulse/how-do-embeddings-work-large-language-model-llm-onfinanceofficial-54awc/

- https://www.qwak.com/post/utilizing-llms-with-embedding-stores


Comments

Post a Comment

Popular posts from this blog

KB: Azure ACA Container fails to start (no User Assigned or Delegated Managed Identity found for specified ClientId)

Image
When deploying secure workloads using Azure Container Apps (ACA) , teams often face confusion between User Assigned Managed Identities (UAMI) and App Registrations . While both entities are visible in Azure Active Directory and have similar identifiers (Application ID, Object ID), they serve very different purposes . This confusion can lead to authentication failures when accessing services like Azure App Configuration or Key Vault . A common issue occurs when a container app is configured with a managed identity, but the environment variables or role assignments mistakenly reference an App Registration instead, causing errors like: "No User Assigned or Delegated Managed Identity found for specified ClientId" This article breaks down the difference between UAMI and App Registrations, explains why this issue happens, and outlines the correct approach to resolve it. Symptoms Azure Container App fails to authenticate to Azure App Configuration or other services. Logs...
Read more

Electron Process Execution Failure with FSLogix

  Technical Note: Electron Process Execution Failure with FSLogix 1. Overview When running Electron-based applications in environments using FSLogix Profile or Office Containers , users may encounter issues where the Electron process fails to launch or execute properly . This behavior has been observed in Azure Virtual Desktop (AVD) , Windows Virtual Desktop (WVD) , and other environments where FSLogix filter drivers are active. 2. Symptoms Electron-based applications (e.g., desktop apps built on Electron, CLI wrappers) do not start , remain unresponsive, or terminate silently. No visible logs or error messages are generated by the application. Standard executables run correctly when placed outside of the FSLogix-controlled profile path (e.g., copying to C:\Temp allows execution). The issue is reproducible across all Electron apps in the FSLogix-managed profile. 3. Root Cause The issue is linked to FSLogix filter drivers ( frxdrv , frxdrvvt , frxccd ) interfe...
Read more

KB:RMM VS DEX (Remote Monitoring Management vs Digital Employee Experience)

Digital Employee Experience (DEX) and Remote Monitoring and Management (RMM) tools serve different purposes and cater to distinct aspects of IT and employee management. Digital Employee Experience (DEX) Focus: Enhancing the overall experience of employees with their digital workplace tools and environment. Key Features: User Experience Monitoring: Tracks how employees interact with software and hardware, identifying issues impacting productivity. Performance Analytics: Provides insights into the performance of applications from the end-user's perspective. Feedback Mechanisms: Allows employees to report issues and give feedback on their digital tools and environment. Employee Well-being: Monitors factors that affect employee satisfaction and well-being, such as system performance and usability. Proactive Support: Identifies potential issues before they become significant problems, enabling proactive IT support. Goals: Improve employee satisfaction and productivity. Ensure a sm...
Read more