What is Little-endian and Big-endian?
date
Jan 18, 2023
slug
what-is-little-endian-and-big-endian
status
Published
tags
Storage
Go
summary
This post explains what little-endian and big-endian is. It also explains the
binary.PutUvarint function in the standard go package.type
Post
Explanation
While browsing through the leveldb source code, I noticed a quote: All fixed-length integers are little-endian. What is little-endian and big-endian?
Little-endian is a method of ordering data in memory in which the least significant byte (the "little end") of a word is stored at the lowest memory address, and the most significant byte is stored at the highest memory address. This is in contrast to big-endian, in which the most significant byte is stored at the lowest memory address. Little-endian is used in most personal computers and some other computer systems.
For example, if a computer uses 32-bit words to represent a 4-byte integer number, and the number is represented in memory as 0x12345678, in little-endian the four bytes (78, 56, 34, 12) would be stored in memory starting at the lowest memory address, while in big-endian the four bytes (12, 34, 56, 78) would be stored starting at the lowest memory address.
binary.PutUvarint
Additionally, in the context of networking, the little-endian is used in the Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) to indicate the byte order of multi-byte fields in the header of the packet, such as the source and destination port numbers and the sequence and acknowledgement numbers.
In the table package, the lengths are serialized to little-endian using
binary.PutUvarint .func PutUvarint(buf []byte, x uint64) int {
i := 0
for x >= 0x80 {
buf[i] = byte(x) | 0x80
x >>= 7
i++
}
buf[i] = byte(x)
return i + 1
}For example, the encoding of 0x904e obtained from
binary.PutUvarint is not the binary representation of the decimal number 10000. The binary.PutUvarint function encodes integers using the uvarint format, which uses a variable number of bytes to represent the integer, where the most significant bit of each byte is used to indicate whether there are more bytes to come.The decimal number 10000 is equal to 0x2710 in hexadecimal, which is equal to 0010 0111 0001 0000 in binary. The last 7 bytes are 001 0000, the highest position being 1 because it is not the least byte. Thus the first byte is 1001,000. So the first byte is 1001 0000 and equal to hexadecimal 0x90. The remaining bytes are 0010 0111 0, again with the least 7 bits. because it's the last byte of the encoding, the highest position being 0. Thus the second byte is 0100 1110 which is equal to hexadecimal 0x4e. Connected together in little endian order is 0x904e.
Go example code
func ExamplePutUvarint() {
// Test case 1: Encode the number 0
var x uint64 = 0
dst := make([]byte, binary.MaxVarintLen64)
n := binary.PutUvarint(dst, x)
fmt.Printf("Test case 1: Number: %d, Encoded: %x, Bytes written: %d\n", x, dst[:n], n)
// Test case 2: Encode the number 100
x = 100
dst = make([]byte, binary.MaxVarintLen64)
n = binary.PutUvarint(dst, x)
fmt.Printf("Test case 2: Number: %d, Encoded: %x, Bytes written: %d\n", x, dst[:n], n)
// Test case 3: Encode the number 10000
x = 10000
dst = make([]byte, binary.MaxVarintLen64)
n = binary.PutUvarint(dst, x)
fmt.Printf("Test case 3: Number: %d, Encoded: %x, Bytes written: %d\n", x, dst[:n], n)
// Test case 4: Encode the number 1000000000
x = 1000000000
dst = make([]byte, binary.MaxVarintLen64)
n = binary.PutUvarint(dst, x)
fmt.Printf("Test case 4: Number: %d, Encoded: %x, Bytes written: %d\n", x, dst[:n], n)
// Output:
// Test case 1: Number: 0, Encoded: 00, Bytes written: 1
// Test case 2: Number: 100, Encoded: 64, Bytes written: 1
// Test case 3: Number: 10000, Encoded: 904e, Bytes written: 2
// Test case 4: Number: 1000000000, Encoded: 8094ebdc03, Bytes written: 5
}