Difference between revisions of "SciStream at APS"

From Beam Line Controls
Jump to navigation Jump to search
(from talks with flavio)
 
 
(9 intermediate revisions by 2 users not shown)
Line 1: Line 1:
= SciStream Tunnel Setup Guide =
= SciStream Tunnel Setup Guide =


This guide provides step-by-step instructions on how to tunnel a connection through SciStream using <code>s2uc</code> running from a container. This setup involves two endpoints, each running on separate hosts, to establish a secure tunnel for forwarding packets.
This guide provides step-by-step instructions on how to tunnel a connection through SciStream using <code>s2uc</code> running from a container. This setup involves two endpoints, each running on separate hosts, to establish a secure tunnel(TLS) for forwarding packets.


== Endpoint Configuration ==
Scistream's API TLS certificates are on the test servers at <code>/local/scistream</code>. Use the folder corresponding to the server -- i.e. on server1, always use /server1; on server2, use /server2.
 
== How data flows through the tunnel ==
 
The tunnel consists of parts that work together:
 
'''Server side (Inbound Request):''' Destination that receives data. TCP server listens for connections.
'''Client side (Outbound Request):''' Source that sends data. TCP client that actively creates new connections
 
** Your client application connects to the local port (5100) on the client machine
** The SciStream outbound request forwards this connection through the secure tunnel
** The SciStream inbound request receives the forwarded connection
** Data is delivered to your server application listening on the receiver port (5300)
 
== Request Configuration ==


=== Inbound Request ("Server side": 192.168.150.1) ===
=== Inbound Request ("Server side": 192.168.150.1) ===
'''Purpose:''' Sets up the receiving end of the tunnel. This establishes a listener on the specified SciStream endpoint that waits for incoming connections from the outbound request (other SciStream endpoint) and it forwards this connection to the <remote_ip>:<receiver_ports>.


<pre>podman run --rm --net=host -v ./server1:/scistream --entrypoint=s2uc castroflaviojr/scistream:1.2.1 inbound-request --remote_ip 192.168.150.1 --s2cs 192.168.150.1:5000 --receiver_ports 5300 --num_conn 1 --server_cert=/scistream/server.crt
<pre>podman run --rm --net=host -v ./server1:/scistream --entrypoint=s2uc castroflaviojr/scistream:1.2.1 inbound-request --remote_ip 192.168.150.1 --s2cs 192.168.150.1:5000 --receiver_ports 5300 --num_conn 1 --server_cert=/scistream/server.crt
Line 33: Line 48:


=== Outbound Request ("Client side": 192.168.150.2) ===
=== Outbound Request ("Client side": 192.168.150.2) ===
Purpose: Sets up the sending end of the tunnel. This establishes a listener on the specified SciStream endpoint that waits for incoming connections from the client application and forwards that data to the other SciStream endpoint.


<pre>podman run --rm --net=host -v ./server2:/scistream --entrypoint=s2uc castroflaviojr/scistream:1.2.1 outbound-request --remote_ip 192.168.150.1 --s2cs 192.168.150.2:5000 --receiver_ports 5100 --num_conn 1 --server_cert=/scistream/server.crt bd9f1a7e-04d7-11f0-b44c-946dae415862 192.168.150.1:5100
<pre>podman run --rm --net=host -v ./server2:/scistream --entrypoint=s2uc castroflaviojr/scistream:1.2.1 outbound-request --remote_ip 192.168.150.1 --s2cs 192.168.150.2:5000 --receiver_ports 5100 --num_conn 1 --server_cert=/scistream/server.crt bd9f1a7e-04d7-11f0-b44c-946dae415862 192.168.150.1:5100
Line 40: Line 57:
** <code>--remote_ip</code>: The eventual destination IP address we want data to be forwarded to.
** <code>--remote_ip</code>: The eventual destination IP address we want data to be forwarded to.
** <code>--s2cs</code>: The IP and port for the local s2cs scistream control server.
** <code>--s2cs</code>: The IP and port for the local s2cs scistream control server.
** <code>--receiver_ports</code>: Local port which the client application will connect to.
** <code>--receiver_ports</code>: Local port which the client application will connect to (i.e. client should connect to {s2cs_ip:receiver_port}).
** <code>--num_conn</code>: Number of connections to establish.
** <code>--num_conn</code>: Number of connections to establish.
** <code>--server_cert</code>: Path to the server certificate for TLS.
** <code>--server_cert</code>: Path to the server certificate for TLS.

Latest revision as of 18:29, 14 April 2025

SciStream Tunnel Setup Guide

This guide provides step-by-step instructions on how to tunnel a connection through SciStream using s2uc running from a container. This setup involves two endpoints, each running on separate hosts, to establish a secure tunnel(TLS) for forwarding packets.

Scistream's API TLS certificates are on the test servers at /local/scistream. Use the folder corresponding to the server -- i.e. on server1, always use /server1; on server2, use /server2.

How data flows through the tunnel

The tunnel consists of parts that work together:

Server side (Inbound Request): Destination that receives data. TCP server listens for connections. Client side (Outbound Request): Source that sends data. TCP client that actively creates new connections

    • Your client application connects to the local port (5100) on the client machine
    • The SciStream outbound request forwards this connection through the secure tunnel
    • The SciStream inbound request receives the forwarded connection
    • Data is delivered to your server application listening on the receiver port (5300)

Request Configuration

Inbound Request ("Server side": 192.168.150.1)

Purpose: Sets up the receiving end of the tunnel. This establishes a listener on the specified SciStream endpoint that waits for incoming connections from the outbound request (other SciStream endpoint) and it forwards this connection to the <remote_ip>:<receiver_ports>.

podman run --rm --net=host -v ./server1:/scistream --entrypoint=s2uc castroflaviojr/scistream:1.2.1 inbound-request --remote_ip 192.168.150.1 --s2cs 192.168.150.1:5000 --receiver_ports 5300 --num_conn 1 --server_cert=/scistream/server.crt
  • Parameters:
    • --remote_ip: The address the user application is listening on.
    • --s2cs: The IP and port for the local s2cs scistream control server.
    • --receiver_ports: The port the user application is listening on (i.e. the user application is bound to {remote_ip:receiver_port}).
    • --num_conn: Number of connections to establish.
    • --server_cert: Path to the server certificate for TLS.

Note the UID:

After running the command, note the UID and listener port provided in the output. Example:

uid; s2cs; access_token; role
bd9f1a7e-04d7-11f0-b44c-946dae415862 192.168.150.1:5000 INVALID_TOKEN PROD
sending client request message
started client request
waiting for hello message
sending for hello message
Hello message sent successfully
Listeners: ['192.168.150.1:5100']

Outbound Request ("Client side": 192.168.150.2)

Purpose: Sets up the sending end of the tunnel. This establishes a listener on the specified SciStream endpoint that waits for incoming connections from the client application and forwards that data to the other SciStream endpoint.

podman run --rm --net=host -v ./server2:/scistream --entrypoint=s2uc castroflaviojr/scistream:1.2.1 outbound-request --remote_ip 192.168.150.1 --s2cs 192.168.150.2:5000 --receiver_ports 5100 --num_conn 1 --server_cert=/scistream/server.crt bd9f1a7e-04d7-11f0-b44c-946dae415862 192.168.150.1:5100
  • Parameters:
    • --remote_ip: The eventual destination IP address we want data to be forwarded to.
    • --s2cs: The IP and port for the local s2cs scistream control server.
    • --receiver_ports: Local port which the client application will connect to (i.e. client should connect to {s2cs_ip:receiver_port}).
    • --num_conn: Number of connections to establish.
    • --server_cert: Path to the server certificate for TLS.
    • UID: Use the UID noted from the inbound request.
    • IP:PORT: Use the IP and port noted from the inbound request.

Example output:

uid; s2cs; access_token; role
bd9f1a7e-04d7-11f0-b44c-946dae415862 192.168.150.2:5000 INVALID_TOKEN CONS
started client request
waiting for hello message
Hello message sent successfully
Listeners: ['192.168.150.2:5100']

Verification

  • Check Listening Ports:

On both hosts, verify that the stunnel processes are listening on the expected ports using:

ss -tlpn
  • Test the Tunnel:

Use iperf3 to test the tunnel:

  • Client Side (Host 192.168.150.2):
iperf3 -c 192.168.150.2 -p 5100
  • Server Side (Host 192.168.150.1):
iperf3 -s -p 5300

Closing the Connection

To close a connection request, use the release option with s2uc:

s2uc release <uid>