Linux network teaming - Revision history

Tguruswamy: /* Maximum throughput (one-to-one) */

2023-05-11T17:01:35Z

Maximum throughput (one-to-one)

← Older revision		Revision as of 17:01, 11 May 2023
Line 34:		Line 34:
	There are only two ways to get more transmission bandwidth over the bonded link than a single link (10 Gb):		There are only two ways to get more transmission bandwidth over the bonded link than a single link (10 Gb):

	* Use '''roundrobin''' mode. No application support or cooperation required. Packets can arrive out-of-order, which may increase CPU load on the receiver or cause retransmissions -- leading to the usable bandwidth being ~~more like~~ 150% of a single link.		* Use '''roundrobin''' mode. No application support or cooperation required. Packets can arrive out-of-order, which may increase CPU load on the receiver or cause retransmissions -- leading to the usable bandwidth being perhaps 150% of a single link.
	* Use '''loadbalance mode and multiple streams which hash to different interfaces'''. Requires application support to generate multiple streams and an appropriate stream definition/hashing algorithm chosen. For example, if Layer 4 hashing is chosen, the application can transmit on two separate TCP ports which hash to interface 1 and 2. The receiving application then must reconstruct the data into a single data stream, if required.		* Use '''loadbalance mode and multiple streams which hash to different interfaces'''. Requires application support to generate multiple streams and an appropriate stream definition/hashing algorithm chosen. For example, if Layer 4 hashing is chosen, the application can transmit on two separate TCP ports which hash to interface 1 and 2. The receiving application then must reconstruct the data into a single data stream, if required.

Tguruswamy: /* Maximum throughput (one-to-one) */

2023-05-11T17:01:09Z

Maximum throughput (one-to-one)

← Older revision		Revision as of 17:01, 11 May 2023
Line 27:		Line 27:
	==Examples==		==Examples==
	===Maximum throughput (one-to-one) ===		===Maximum throughput (one-to-one) ===
	Consider a simple configuration where we are trying to get maximum throughput from a single detector PC to network storage ~~'''using a single network stream'''~~. The connection from the detector PC is the bottleneck so we add an extra interface card.		Consider a simple configuration where we are trying to get maximum throughput from a single detector PC to network storage. The connection from the detector PC is the bottleneck so we add an extra interface card.
	How should we configure the two available interfaces?		How should we configure the two available interfaces?

Line 34:		Line 34:
	There are only two ways to get more transmission bandwidth over the bonded link than a single link (10 Gb):		There are only two ways to get more transmission bandwidth over the bonded link than a single link (10 Gb):

	* Use '''roundrobin''' mode. No application support required. Packets can arrive out-of-order, which may increase CPU load on the receiver or cause retransmissions.		* Use '''roundrobin''' mode. No application support or cooperation required. Packets can arrive out-of-order, which may increase CPU load on the receiver or cause retransmissions -- leading to the usable bandwidth being more like 150% of a single link.
	* Use '''loadbalance mode and multiple streams which hash to different interfaces'''. Requires application support to generate multiple streams and an appropriate stream definition/hashing algorithm chosen. For example, if Layer 4 hashing is chosen, the application can transmit on two separate TCP ports which hash to interface 1 and 2. The receiving application then must reconstruct the data into a single data stream, if required.		* Use '''loadbalance mode and multiple streams which hash to different interfaces'''. Requires application support to generate multiple streams and an appropriate stream definition/hashing algorithm chosen. For example, if Layer 4 hashing is chosen, the application can transmit on two separate TCP ports which hash to interface 1 and 2. The receiving application then must reconstruct the data into a single data stream, if required.

Tguruswamy: /* Maximum throughput (one-to-one) */

2023-05-11T17:00:03Z

Maximum throughput (one-to-one)

← Older revision		Revision as of 17:00, 11 May 2023
Line 27:		Line 27:
	==Examples==		==Examples==
	===Maximum throughput (one-to-one) ===		===Maximum throughput (one-to-one) ===
	Consider a simple configuration where we are trying to get maximum throughput from a single detector PC to network storage. The connection from the detector PC is the bottleneck so we add an extra interface card.		Consider a simple configuration where we are trying to get maximum throughput from a single detector PC to network storage '''using a single network stream'''. The connection from the detector PC is the bottleneck so we add an extra interface card.
	How should we configure the two available interfaces?		How should we configure the two available interfaces?

Tguruswamy: /* Testing */

2023-03-15T06:55:43Z

Testing

← Older revision		Revision as of 06:55, 15 March 2023
Line 41:		Line 41:
	==Testing==		==Testing==

	<code>iperf3</code> is a common tool used to test network bandwidth between two points. Run <code>iperf3 -s</code> on one computer, and <code>iperf3 -c {hostname}</code> on another. By default, iperf3 will only create a single network stream. Adding the <code>--parallel {n}</code> flag will cause the iperf3 client to use multiple ports to connect to the server and so, if ''loadbalance+Layer 4'' hashing is chosen on the bonded interfaces, allow for multi-stream testing with respect to the bonded network interfaces. Note this still runs iperf3 within a single thread and on a single CPU core.		<code>iperf3</code> is a common tool used to test network bandwidth between two points. Run <code>iperf3 -s</code> on one computer, and <code>iperf3 -c {hostname}</code> on another. By default, iperf3 will only create a single network stream. Adding the <code>--parallel {n}</code> flag will cause the iperf3 client to use multiple TCP/UDP ports to connect to the server and so, if ''loadbalance+Layer 4'' hashing is chosen on the bonded interfaces, allow for multi-stream testing with respect to the bonded network interfaces. Note this still runs iperf3 within a single thread and on a single CPU core.

	Alternatively, start up entirely separate iperf3 server instances, listening on different ports, and run several clients in parallel. This can allow the use of multiple network streams (assuming Layer 4 hashing) and CPU cores.		Alternatively, start up entirely separate iperf3 server instances, listening on different ports, and run several clients in parallel. This can allow the use of multiple network streams (assuming Layer 4 hashing) and CPU cores.

Tguruswamy: /* Maximum throughput (one-to-one) */

2023-03-15T06:55:26Z

Maximum throughput (one-to-one)

← Older revision		Revision as of 06:55, 15 March 2023
Line 35:		Line 35:

	* Use '''roundrobin''' mode. No application support required. Packets can arrive out-of-order, which may increase CPU load on the receiver or cause retransmissions.		* Use '''roundrobin''' mode. No application support required. Packets can arrive out-of-order, which may increase CPU load on the receiver or cause retransmissions.
	* Use '''loadbalance mode and multiple streams which hash to different interfaces'''. Requires application support to generate multiple streams and an appropriate stream definition/hashing algorithm chosen. For example, if Layer 4 hashing is chosen, the application can transmit on two separate ports which hash to interface 1 and 2. The receiving application then must reconstruct the data into a single data stream, if required.		* Use '''loadbalance mode and multiple streams which hash to different interfaces'''. Requires application support to generate multiple streams and an appropriate stream definition/hashing algorithm chosen. For example, if Layer 4 hashing is chosen, the application can transmit on two separate TCP ports which hash to interface 1 and 2. The receiving application then must reconstruct the data into a single data stream, if required.

	Another option is to avoid network teaming entirely, and configure each interface with separate IPs, guaranteeing both transmissions can happen in parallel.		Another option is to avoid network teaming entirely, and configure each interface with separate IPs, guaranteeing both transmissions can happen in parallel.

Tguruswamy: /* Maximum throughput (one-to-one) */

2023-03-15T06:55:08Z

Maximum throughput (one-to-one)

← Older revision		Revision as of 06:55, 15 March 2023
Line 34:		Line 34:
	There are only two ways to get more transmission bandwidth over the bonded link than a single link (10 Gb):		There are only two ways to get more transmission bandwidth over the bonded link than a single link (10 Gb):

	* Use '''roundrobin''' mode. No application support required. Packets ~~may~~ arrive out-of-order, which ~~increases~~ CPU load on the receiver or ~~causes~~ retransmissions.		* Use '''roundrobin''' mode. No application support required. Packets can arrive out-of-order, which may increase CPU load on the receiver or cause retransmissions.
	* Use '''loadbalance mode and multiple streams which hash to different interfaces'''. Requires application support to generate multiple streams and an appropriate stream definition/hashing algorithm chosen. For example, if Layer 4 hashing is chosen, the application can transmit on two separate ports which hash to interface 1 and 2. The receiving application then must reconstruct the data into a single data stream, if required.		* Use '''loadbalance mode and multiple streams which hash to different interfaces'''. Requires application support to generate multiple streams and an appropriate stream definition/hashing algorithm chosen. For example, if Layer 4 hashing is chosen, the application can transmit on two separate ports which hash to interface 1 and 2. The receiving application then must reconstruct the data into a single data stream, if required.

Tguruswamy: /* Maximum throughput (one-to-one) */

2023-03-15T06:54:48Z

Maximum throughput (one-to-one)

← Older revision		Revision as of 06:54, 15 March 2023
Line 34:		Line 34:
	There are only two ways to get more transmission bandwidth over the bonded link than a single link (10 Gb):		There are only two ways to get more transmission bandwidth over the bonded link than a single link (10 Gb):

	* Use '''roundrobin''' mode. No application support required. Packets may arrive out-of-order, which increases CPU load on the receiver or ~~cause~~ retransmissions.		* Use '''roundrobin''' mode. No application support required. Packets may arrive out-of-order, which increases CPU load on the receiver or causes retransmissions.
	* Use '''loadbalance mode and multiple streams which hash to different interfaces'''. Requires application support to generate multiple streams and an appropriate stream definition/hashing algorithm chosen. For example, if Layer 4 hashing is chosen, the application can transmit on two separate ports which hash to interface 1 and 2. The receiving application then must reconstruct the data into a single data stream, if required.		* Use '''loadbalance mode and multiple streams which hash to different interfaces'''. Requires application support to generate multiple streams and an appropriate stream definition/hashing algorithm chosen. For example, if Layer 4 hashing is chosen, the application can transmit on two separate ports which hash to interface 1 and 2. The receiving application then must reconstruct the data into a single data stream, if required.

Tguruswamy: /* Maximum throughput (one-to-one) */

2023-03-15T06:16:10Z

Maximum throughput (one-to-one)

← Older revision		Revision as of 06:16, 15 March 2023
Line 37:		Line 37:
	* Use '''loadbalance mode and multiple streams which hash to different interfaces'''. Requires application support to generate multiple streams and an appropriate stream definition/hashing algorithm chosen. For example, if Layer 4 hashing is chosen, the application can transmit on two separate ports which hash to interface 1 and 2. The receiving application then must reconstruct the data into a single data stream, if required.		* Use '''loadbalance mode and multiple streams which hash to different interfaces'''. Requires application support to generate multiple streams and an appropriate stream definition/hashing algorithm chosen. For example, if Layer 4 hashing is chosen, the application can transmit on two separate ports which hash to interface 1 and 2. The receiving application then must reconstruct the data into a single data stream, if required.

	Another option is to avoid network teaming entirely, and configure each interface with separate IPs ~~on different subnets~~, guaranteeing both transmissions can happen in parallel.		Another option is to avoid network teaming entirely, and configure each interface with separate IPs, guaranteeing both transmissions can happen in parallel.

	==Testing==		==Testing==

Tguruswamy: /* Bonding modes */

2023-03-15T06:14:21Z

Bonding modes

← Older revision		Revision as of 06:14, 15 March 2023
Line 14:		Line 14:
	The bonding mode decides how each stream is handled. Using the teamd terminology (see <code>runner.name</code>):		The bonding mode decides how each stream is handled. Using the teamd terminology (see <code>runner.name</code>):

	; broadcast : every packet of every stream is sent over all bonded interfaces.		; broadcast : every packet of every stream is sent over all bonded interfaces (duplicated).
	: aggregate bandwidth = single link bandwidth. single-stream bandwidth = single link bandwidth.		: aggregate bandwidth = single link bandwidth. single-stream bandwidth = single link bandwidth.
	; roundrobin : the packets of every stream are cycled between interfaces in order. Packet 1 -> interface 1, Packet 2 -> interface 2, Packet 3 -> interface 1, etc.		; roundrobin : the packets of every stream are cycled between interfaces in order. Packet 1 -> interface 1, Packet 2 -> interface 2, Packet 3 -> interface 1, etc.

Tguruswamy: /* Testing and troubleshooting */

2023-03-15T05:11:06Z

Testing and troubleshooting

← Older revision		Revision as of 05:11, 15 March 2023
Line 39:		Line 39:
	Another option is to avoid network teaming entirely, and configure each interface with separate IPs on different subnets, guaranteeing both transmissions can happen in parallel.		Another option is to avoid network teaming entirely, and configure each interface with separate IPs on different subnets, guaranteeing both transmissions can happen in parallel.

	==Testing ~~and troubleshooting~~==		==Testing==

	<code>iperf3</code> is a common tool used to test network bandwidth between two points. Run <code>iperf3 -s</code> on one computer, and <code>iperf3 -c {hostname}</code> on another. By default, iperf3 will only create a single network stream. Adding the <code>--parallel {n}</code> flag will cause the iperf3 client to use multiple ports to connect to the server and so, if ''loadbalance+Layer 4'' hashing is chosen on the bonded interfaces, allow for multi-stream testing with respect to the bonded network interfaces. Note this still runs iperf3 within a single thread and on a single CPU core.		<code>iperf3</code> is a common tool used to test network bandwidth between two points. Run <code>iperf3 -s</code> on one computer, and <code>iperf3 -c {hostname}</code> on another. By default, iperf3 will only create a single network stream. Adding the <code>--parallel {n}</code> flag will cause the iperf3 client to use multiple ports to connect to the server and so, if ''loadbalance+Layer 4'' hashing is chosen on the bonded interfaces, allow for multi-stream testing with respect to the bonded network interfaces. Note this still runs iperf3 within a single thread and on a single CPU core.