Docs

Torch Log Service

Ship logs to TOS

Decompress Snappy-compressed files

TLS automatically applies Snappy to compress TOS files when shipping logs to TOS. After logs are shipped successfully, you can decompress TOS files using language programing tools like C++, Java, Python, and Go etc.

Decompress using the C++ Library

Download and install dependencies.

Debian/Ubuntu method

sudo apt update
sudo apt install libsnappy1v5 libsnappy-dev

CentOS method

sudo yum update
sudo yum install snappy snappy-devel

Mac method

brew install snappy

Manual installation method

Download Snappy Lib.

Install Snappy Lib.

git clone https://github.com/google/snappy
make && make install

Notice

Manually installed libraries may not be recognized by the system automatically. You need to add the library path manually.

Extract the files.

Code example:

int main() {
std::string decompressed;

// Read compressed data and uncompress the data.
std::ifstream inFile("compressed.snappy", std::ios::binary);
std::string readCompressed((std::istreambuf_iterator<char>(inFile)), std::istreambuf_iterator<char>());
inFile.close();

// Uncompress.
snappy::Uncompress(readCompressed.data(), readCompressed.size(), &decompressed);
std::cout << "Decompressed text: " << decompressed << "\n";

return 0;
}

Decompress files using Java Lib

Download the dependencies.

Description

We recommend using Java Lib version 1.1.2.6 or above. Version 1.1.2.1 has a bug that may prevent some compressed files from being extracted. However, versions 1.1.2.6 and above have resolved this issue. For more information about this bug, see bad handling of the MAGIC HEADER.

Manual method

Download snappy-java.

Maven method

<dependency>
<groupId>org.xerial.snappy</groupId>
<artifactId>snappy-java</artifactId>
<version>1.1.10.4</version>
<type>jar</type>
<scope>compile</scope>
</dependency>

Extract the files.

The code example is as follows:

package main;

import org.xerial.snappy.Snappy;

import java.io.*;
import java.nio.file.Files;
import java.nio.file.Paths;

public class SnappyCompressionDemo {

public static void main(String[] args) throws Exception {
String compressedFilePath = "compressed.snappy";

// Read compressed data from files
byte[] readCompressedData = Files.readAllBytes(Paths.get(compressedFilePath));

// Decompress data
byte[] decompressedData = Snappy.uncompress(readCompressedData);

// Output decompressed data
String decompressedString = new String(decompressedData);
System.out.println("Decompressed data: " + decompressedString);
}
}

Decompress files using Python Lib

Download the dependency package python-lib.

Install pip.

pip install python-snappy

Extract the files.

The python3 code example is as follows:

import snappy

def demo():
# Read compressed data
with open("compressed.snappy", "rb") as in_file:
read_compressed = in_file.read()

# Decompressed data
decompressed = snappy.uncompress(read_compressed)
print(f"Decompressed data: {decompressed.decode('utf-8')}")

if __name__ == "__main__":
demo()

Notice

If you encounter errors while using snappy.uncompress on Windows, such as UnicodeDecodeError: 'gbk' codec can't decode byte 0xff in position 0: illegal multibyte sequence, it may be caused by the Windows file system starting files with BOM. We recommend using a Unix-like system or setting the correct file encoding in the encoding parameter of the open function.

In command-line scenarios, only Hadoop mode (hadoop_stream_decompress) and stream mode (stream_decompress) are supported. Snappy-compressed files can not be extracted using the following command-line tool: python -m snappy -d compressed_file.snappy uncompressed_file.

Decompress using Go Lib

Download dependency package go-snappy.

Install dependencies.

go get github.com/golang/snappy
go mod tidy

Extract the files.

The code example is as follows:

package main

import (
"fmt"
"log"
"os"

"github.com/golang/snappy"
)

func main() {
// Read compressed data from files
readCompressed, err := os.ReadFile("compressed.snappy")
if err != nil {
log.Fatalf("Failed to read compressed file: %v", err)
}

// Decompressed data
decompressed, err := snappy.Decode(nil, readCompressed)
if err != nil {
log.Fatalf("Failed to decompress data: %v", err)
}
fmt.Printf("Decompressed data: %s\n", string(decompressed))
}

Last updated: 2026.04.24 16:25:19