The ln-history project works with messages of the Bitcoin Lightning network specifically the gossip messages. This page should give you a an overview about their strucuture and most important types. For detailed information see the protocol:
The Lightning Network is a distributed system, meaning there is no central server from which data can be retrieved. Instead, each node must gather data from multiple peers in order to avoid relying on a single trusted source. This creates a strong need for compact messaging, so that nodes can participate in the network without requiring significant bandwidth. Furthermore, the message format is defined at a very low level, close to the hardware — every bit in a message has a specific purpose and is carefully designed to minimize overhead.
Lightning messages follow the TLV format, which stands for Type-Length-Value. The actual positioning of the values are Length->Type->Value.
Lets understand how each part is structured.
The length determines the number of bytes of the message. It is encoded as varint a format that is used a lot in Bitcoin. It is a clever way to compress unsigned integer values. The two python functions which can be found in the lnhistoryclient/parser/common.py explain it clearly:
The type of a Lightning Message is exactly 2
bytes long and contains the information of how to parse the bytes.
Certain ranges represent the kind of message type, see:
0
to 31
- Setup & control32
to 127
- Channel128
to 255
- Commitment256
to 511
- Routing32768
to 65535
- CustomThe value of a Lightning Message contains the actual data of the message. This is most times the longest part.
A specific type of Ligtning Message are the gossip messages. They are part of the routing range between 256
and 511
.
For this tool we are interested in those three gossip messages
256
- channel_announcement
257
- node_announcement
258
- channel_update
Each of them has unique rules for which byte represents which information.
See those python functions from lnhistoryclient/parser/parser.py to get an understanding: