{"id":25573,"date":"2024-12-13T05:21:06","date_gmt":"2024-12-13T13:21:06","guid":{"rendered":"http:\/\/www.palada.net\/index.php\/2024\/12\/13\/news-19302\/"},"modified":"2024-12-13T05:21:06","modified_gmt":"2024-12-13T13:21:06","slug":"news-19302","status":"publish","type":"post","link":"https:\/\/www.palada.net\/index.php\/2024\/12\/13\/news-19302\/","title":{"rendered":"DeepSpeed: a tuning tool for large language models"},"content":{"rendered":"

Credit to Author: gallagherseanm| Date: Fri, 13 Dec 2024 11:30:50 +0000<\/strong><\/p>\n

\n

Large Language Models (LLMs) have the potential to automate and reduce the workloads of many types, including those of cybersecurity analysts and incident responders. But generic LLMs lack the domain-specific knowledge to handle these tasks well. While they may have been built with training data that included some cybersecurity-related resources, that is often insufficient for taking on more specialized tasks that require more up to date and, in some cases, proprietary knowledge to perform well\u2014knowledge not available to the LLMs when they were trained.<\/span>\u00a0<\/span><\/p>\n

There are several existing solutions for tuning \u201cstock\u201d (unmodified) LLMs for specific types of tasks. But unfortunately, these solutions were insufficient for the types of applications of LLMs that Sophos X-Ops is attempting to implement. For that reason, SophosAI\u00a0 has assembled a framework that utilizes\u00a0 DeepSpeed, a library developed by Microsoft that can be used to train and tune the inference of a model with (in theory) trillions of parameters by scaling up the compute power and number of graphics processing units (GPUs) used during training. The framework is open source licensed and can be found in <\/span>our GitHub repository<\/span><\/a>.<\/span>\u00a0<\/span><\/p>\n

While many of the parts of the framework are not novel and leverage existing open-source libraries, SophosAI has synthesized several of the key components for ease of use. And we continue to work on improving the performance of the framework.<\/span>\u00a0<\/span><\/p>\n

The (inadequate) alternatives<\/span>\u00a0<\/span><\/h2>\n

There are several existing approaches to adapting stock LLMs to domain-specific knowledge. Each of them has its own advantages and limitations.\u00a0<\/span>\u00a0<\/span><\/p>\n

 <\/p>\n

\u00a0<\/span><\/p>\n

To be fully effective, a domain expert LLM requires pre-training of all its parameters to learn the proprietary knowledge of a company. That undertaking can be resource intensive and time consuming\u2014which is why we turned to DeepSpeed\u00a0 for our training framework, which we implemented in Python. The version of the framework that we are releasing as open source can be run in the Amazon Web Services SageMaker machine learning service, but it could be adapted to other environments.\u00a0<\/span>\u00a0<\/span><\/p>\n

Training frameworks (including DeepSpeed) allow you to scale up large model training tasks through parallelism. There are three main types of parallelism: data, tensor, and pipeline.<\/span>\u00a0<\/span><\/p>\n

\"\"<\/a>
Figure 1: an illustration of the three main types of model training parallelism.<\/figcaption><\/figure>\n

In data parallelism, each process working on the training task (essentially each graphics processor unit, or GPU) receives a copy of the full model\u2019s weights but only a subset of the data, called a minibatch. After the forward pass through the data (to calculate <\/span>loss<\/span><\/a> , or the amount of inaccuracy in the parameters of the model being used for training) and the backward pass (to calculate the <\/span>gradient of the loss<\/span><\/a>) are completed, the resulting gradients are synchronized.<\/span>\u00a0<\/span><\/p>\n

In Tensor parallelism, each layer of the model being used for training is split across the available processes. Each process computes a portion of the layer \u2018s operation using the full training data set. The partial outputs from each of these layers are synchronized across processes to create a single output matrix.\u00a0<\/span>\u00a0<\/span><\/p>\n

Pipeline parallelism splits up the model differently. Instead of parallelizing by splitting layers of the model, each layer of the model receives its own process. The minibatches of data are divided into micro-batches and that are sent down the \u201cpipeline\u201d sequentially. Once a process finishes a micro-batch, it receives a new one. This method may experience \u201cbubbles\u201d where a process is idling, waiting for the output of processes hosting earlier model layers.<\/span>\u00a0<\/span><\/p>\n

These three parallelism techniques can also be combined in several ways\u2014and are, in the DeepSpeed training library.<\/span>\u00a0<\/span><\/p>\n

Doing it with DeepSpeed<\/span>\u00a0<\/span><\/h2>\n

DeepSpeed performs sharded data parallelism. Every model layer is split such that each process gets a slice, and each process is given a separate mini batch as input. During the forward pass, each process shares its slice of the layer with the other processes. At the end of this communication, each process now has a copy of the full model layer.\u00a0<\/span>\u00a0<\/span><\/p>\n

Each process computes the layer output for its mini batch. After the process finishes computation for the given layer and its mini batch, the process discards the parts of the layer it was not originally holding.\u00a0<\/span>\u00a0<\/span><\/p>\n

The backwards pass through the training data is done in a similar fashion. As with data parallelism, the gradients are accumulated at the end of the backwards pass and synchronized across processes.<\/span>\u00a0<\/span><\/p>\n

Training processes are more constrained in their performance by memory than processing power\u2014and bringing on more GPUs with additional memory to handle a batch that is too large for the GPU\u2019s own memory can cause significant performance cost because of the communication speed between GPUs, as well as the cost of using more processors than would otherwise be required to run the process. One of the key elements of the DeepSpeed library is its <\/span>Zero Redundancy Optimizer (ZeRO),<\/span><\/a> a set of memory utilization techniques that can efficiently parallelize very large language model training. ZeRO can reduce the memory consumption of each GPU by partitioning the model states (optimizers, gradients, and parameters) across parallelized data processes instead of duplicating them across each process.\u00a0<\/span>\u00a0<\/span><\/p>\n

The trick is finding the right combination of training approaches and optimizations for your computational budget. There are three selectable levels of partitioning in ZeRO:<\/span>\u00a0<\/span><\/p>\n