lakeFS Acquires DVC, Uniting Data Version Control Pioneers to Accelerate AI-Ready Data
Most organizations collect massive amounts of data from various sources, including customer interactions, supply networks, financial systems, and more. As a result, teams may feel overwhelmed by a flood of data while seeking key insights, and the question of data manageability becomes more pressing than ever. This is where data discovery comes in. Data discovery […]
Since 2021, we’ve published the annual State of Data Engineering Report, which includes a summary of all key categories that directly impact data engineering infrastructure. In 2025, we see five primary trends that influence the categories that will be covered in this report. Trend #1: MLOps space is slowly diminishing The MLOps space is slowly
During the 2025 Nvidia GTC conference, one of the keywords that drew a lot of attention was “AI factory.” An AI factory is Nvidia’s idea for producing large-scale AI systems. This concept aligns AI development with the industrial process, in which raw data is received, improved through computation, and converted into valuable products via data-driven
Today’s organizations don’t just use a single data storage solution – they operate across on-prem servers, multiple cloud providers, and hybrid environments. This distributed approach has become necessary, but it comes with significant costs: teams struggle with siloed tools, duplicated processes, and an endless cycle of environment management that diverts focus from delivering actual value.
A solid AI infrastructure is essential for efficiently developing and deploying AI and machine learning (ML) applications – from facial and speech recognition to text processing and computer vision. Before we dive into why AI infrastructure is crucial and how it works, let’s define it first. What is AI infrastructure? AI infrastructure, also known as
With the volumes of generated data increasing, metadata has become an essential component in organizing and comprehending massive datasets. Metadata plays a key role in any modern data strategy, especially among organizations that treat data as one of their most precious assets. This article dives into all the different metadata types, tools, and frameworks to
In today’s data-driven world, businesses don’t just rely on data – they are built on it. But as data infrastructure sprawls across on-prem systems, multiple cloud providers, and third-party platforms, a new challenge is taking center stage: distributed data management. It’s a silent bottleneck with loud consequences. Challenges in Distributed Data Management Managing data across
Many companies are modernizing their data storage infrastructure to capitalize on the opportunities of machine learning (ML) and advanced analytics. However, teams face several unique data management challenges such as the increasing time required for AI training and inference workloads, as well as the cost and scarcity and resources, particularly GPUs. Storage is a key
Supercharging Machine Learning Machine learning (ML) is essential in driving critical business decisions and innovation across various industries. To maintain competitive advantages, organizations continually refine and enhance their ML models through iterative fine-tuning and parallel experimentation. While these strategies are powerful, they come with substantial challenges related to data management, reproducibility, and resource optimization. lakeFS
Training a Large Language Model (LLM) such as ChatGPT or DeepSeek is a complicated, data-intensive process. As a young discipline, best practices and tool chains are still emerging and being formulated for LLM training. lakeFS has a lot to contribute to this field, as a highly scalable data version control system. In this article, we
This article discusses AI Agents in business and automation, focusing on building an AI Agent using lakeFS, LangChain, OpenAI, and FAISS (Facebook AI Similarity Search) to answer questions based on documents. It explains what AI Agents and LangChain are, and how lakeFS is used for data version control. The article also provides an example of
Managing complex and massive data sets is tricky but metadata management tools can help teams keep their data in shape. Metadata management has become critical in data strategies created by organizations that treat data as an important asset. In this article, we dive into metadata management and give you an overview of tools teams use
We use cookies to improve your experience and understand how our site is used.
Join our community of experts:
introduce yourself, share your knowledge and discover best practices from fellow peers
