How to Use the SOS Controller
Introduction
SOS has been designed such that all configuration and interaction with the service can be done through a single point of contact -- namely the controller. The controller is the Floodlight OpenFlow controller outfitted with a special SOS module. All interaction is done over a REST API, which uses HTTP to facilitate high-level and well-defined data push and fetch.
By default, the SOS controller’s REST API runs at the controller’s IP on TCP port 8080. An HTTP client can be used to leverage the API. The popular utility curl is used throughout this document as an example HTTP client. For more information on curl, please refer to curl’s documentation. The following is a general example on how to use curl:
curl http://<controller-ip>:<tcp-port>/<uri-path> -X <http-command> -d '<data-string>'
where
- <controller-ip> is the IP address of the SOS controller
- <tcp-port> is the TCP port on which the REST server is running (default of 8080)
- <uri-path> is the URI of the API
- <http-command> is the HTTP command to use
- <data-string> is the (optional) data to send to the API
Specific examples of this generalized form will be provided for each SOS API discussed.
SOS REST API
The SOS controller has the following APIs defined to perform common SOS operations. All input and output is formatted in JSON, although, if the need arises, other formats such as XML can also be used. The following JSON APIs are defined for SOS:
| Operation | API (<uri-path>) | HTTP Command (<http-command>) | JSON Input Data (<data-string>) | JSON Data Returned | About |
|---|---|---|---|---|---|
| Configure parameters | /wm/sos/config/json | POST or PUT | '{ "parallel-connections" : "<integer>", "buffer-size" : "<integer>", "queue-capacity" : "<integer>", "hard-timeout" : "<integer>", "idle-timeout" : "<integer>" }' | '{ "code" : "<integer>", "message" : "<string> }' | Modify the running configuration of SOS. User need not supply all data – only those parameters to set/change. Input data is a JSON object of key:value pairs. <integer> should be replaced with a zero or positive value. Returns 0 code for success; all others indicate failure. |
| Retrieve parameters | /wm/sos/config/json | GET | N/A | '{ "parallel-connections" : "<integer>", "buffer-size" : "<integer>", "queue-capacity" : "<integer>", "hard-timeout" : "<integer>", "idle-timeout" : "<integer>" }' | Set of parameters as they are configured. |
| Add/remove agent | /wm/sos/agent/add/json /wm/sos/agent/remove/json | POST or PUT | '{ "ip-address" : "<string>", "data-port" : "<integer>", "control-port" : "<integer>", "feedback-port" : "<integer>" }' | '{ "code" : "<integer>", "message" : "<string> }' | Specify the same data for add or remove. Input data is a JSON object of key:value pairs. <string> should be replaced with a dotted-decimal IP address, e.g. 192.168.1.1. <integer> should be replaced with the corresponding port number. Returns 0 code for success; all others indicate failure. |
| Retrieve agents | /wm/sos/agent/json | GET | N/A | '[ { "ip-address" : "<string>", "mac-address" : "<string>", "data-port" : "<integer>", "control-port" : "<integer>", "feedback-port" : "<integer>", "active-transfers" : [ "<string>" ] } ]' | List of SOS agent objects. Includes list of active transfers. <string> is a dotted-decimal IP address, colon-delimited MAC address, and a UUID, respectively. |
| Add/remove whitelist entry | /wm/sos/whitelist/add/json /wm/sos/whitelist/remove/json | POST or PUT | '{ "server-ip-address" : "<string>", "server-tcp-port" : "<integer>", "client-ip-address" : "<string>", "start-time" : "<string>", "stop-time" : "<string>" }' | '{ "code" : "<integer>", "message" : "<string> }' | Specify the same data for add or remove. Input data is a JSON object of key:value pairs. <string> IP addresses should be replaced with a dotted-decimal IP address, e.g. 192.168.1.1. <integer> should be replaced with the corresponding port number. <string> date/time should be replaced with any valid date/time string or value. Note: Start/stop times are currently ignored. Returns 0 code for success; all others indicate failure. |
| Retrieve whitelist entries | /wm/sos/whitelist/json | GET | N/A | '[ { "server-ip-address" : "<string>", "server-tcp-port" : "<integer>", "client-ip-address" : "<string>" } ]' | List of whitelist entry objects. <string> is a dotted-decimal IP address. |
| Enable/disable module | /wm/sos/module/enable/json /wm/sos/module/disable/json | POST or PUT | '' | '{ "code" : "<integer>", "message" : "<string> }' | Specify an empty string as the data. An empty string is given by two single quotes, one after the other, e.g. ''. Do not use a double quote. GET not used for security/safety reasons. Returns 0 code for success; all others indicate failure. |
| Get module operating mode | /wm/sos/module/json | GET | N/A | '{ "enabled" : "<boolean>" }' | <boolean> is either true if enabled or false if disabled. |
| Query status | /wm/sos/status/json | GET | N/A | '{ "code" : "<integer>", "message" : "<string> }' | Omit -d parameter. Returns whether or not the agents are ready to accept another transfer. At present, each agent is allowed at most one transfer at a time. This can be used to detect when one transfer is "cleaned up" and the system is ready for another. Returns 0 code for ready; 6 for not ready; all others indicate failure. |
| Get statistics | /wm/sos/stats/json | GET | N/A | Non-trivial output. Gives agents, whitelist entries, active and past transfers, routes utilized, and more. | Omit -d parameter. Query past and present statistics. Does not include performance evaluation of transfers, although it will in the near future. |
Typical Workflow
In order to use SOS, the network must first be configured. This is an assumed prerequisite to this document.
After the network is configured, the agents should then be added to the controller, information about the transfer to conduct SOS on should be whitelisted in the controller, any SOS parameters should be tuned as desired, and lastly the controller should be checked to ensure the network is ready to handle the pending transfer. At this point, the transfer can be initiated.
In the following steps, the controller is assumed to be running at the IP address 192.168.1.1 with its REST API exposed on port 8080.
Add the SOS Agents
Once the network is configured, the SOS controller needs to know about the agents it has at its disposal. One or more agents can be added, although note that SOS must have at least two configured agents in order to function. The following is an example of adding an agent:
curl http://192.168.1.1:8080/wm/sos/agent/add/json -X POST -d '{"ip-address":"10.0.0.1", "control-port":"9998", "data-port":"9877", "feedback-port":"9997"}' | python -m json.tool
which returns:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 189 0 97 100 92 1308 1240 --:--:-- --:--:-- --:--:-- 1310
{
"code": "0",
"message": "Agent successfully added. It will be available for the next SOS session."
}
The Python module json.tool is used to format the JSON output in human-readable form. This is not strictly required, although it sure makes the JSON string look pretty 
Once an agent is added to the controller, there is no need to add it again in subsequent transfers.
Add a Whitelist Entry
To keep track of the transfers which should have SOS performed, the controller maintains a whitelist, where all transfers listed will use SOS, while all transfers absent will be handled outside of SOS. The default behavior of the controller is to emulate a L2 learning switch for all transfers that are not proactively whitelisted.
The following is an example of adding a whitelist entry:
curl http://192.168.1.1:8080/wm/sos/whitelist/add/json -X POST -d '{"server-ip-address":"10.0.0.4", "server-tcp-port":"5001", "client-ip-address":"10.0.0.2"}' | python -m json.tool
which returns:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 191 0 101 100 90 16248 14478 --:--:-- --:--:-- --:--:-- 16833
{
"code": "0",
"message": "WhitelistEntry successfully added. The entry may initiate the data transfer."
}
Once a transfer is whitelisted, it will persist until explicitly removed. The start and stop values are currently ignored (if supplied).
Tune Transfer Parameters
SOS's performance can be tuned by adjusting the number of parallel connections to use between the agents, the agent application's TCP receive buffer size (in bytes) per connection, and the reordering queue length on the receiving agent. The OpenFlow flow timeouts can also be adjusted, but it is recommended that they be left at their defaults of 0 for the hard timeout and 60 seconds for the idle timeout.
Any changes to these parameters will not impact ongoing transfers and will only apply to future transfers. Any settings will remain in effect until changed at a later time.
To adjust the number of parallel connections to use between the agents, one can do the following:
curl http://192.168.1.1:8080/wm/sos/config/json -X POST -d '{"parallel-connections":"4096"}' | python -m json.tool
which returns:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 88 0 57 100 31 9961 5417 --:--:-- --:--:-- --:--:-- 11400
{
"code": "0",
"message": "Parallel connections set to 4096"
}
It is recommended that the number of parallel connections range from 1 to 8,000. The agents will not reliably handle more than 8,000 given RAM utilization. As a general rule, the more parallel connections that are used, the better the performance will be. However, there comes a point where the CPU struggles to keep up with reading from a too-large number of these sockets.
Likewise, to adjust the TCP receive buffer size in the agent application, one can do the following:
curl http://192.168.1.1:8080/wm/sos/config/json -X POST -d '{"buffer-size":"70000"}' | python -m json.tool
which returns:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 72 0 49 100 23 8105 3804 --:--:-- --:--:-- --:--:-- 9800
{
"code": "0",
"message": "Buffer size set to 70000"
}
It is recommended that the buffer size be at least 30,000 bytes and go no higher than 500,000 bytes. Values above 500,000 have not shown improvements in data transfer speed, while values under 30,000 bytes tend to hinder performance. The "sweet spot" at this point has been determined to be around 60,000-70,000 bytes given a large ~4,000 number of parallel connections.
And lastly, to adjust the agent application's reordering queue length, one can do the following:
curl http://192.168.1.1:8080/wm/sos/config/json -X POST -d '{"queue-capacity":"5"}' | python -m json.tool
which returns:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 70 0 48 100 22 5710 2617 --:--:-- --:--:-- --:--:-- 6000
{
"code": "0",
"message": "Queue capacity set to 5"
}
The queue capacity is merely a safety feature that will only take effect if there is packet loss on a given parallel connection or if the agent CPU cannot keep up with the arrival rate of packets. Better performance with high queue capacity values indicates network instability, which implies that perhaps the buffer size and number of parallel connections parameters are either too large or too small.
If you would like to experiment with idle timeouts, you may adjust the idle timeout as follows:
curl http://192.168.1.1:8080/wm/sos/config/json -X POST -d '{"idle-timeout":"60"}' | python -m json.tool
which returns:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 68 0 47 100 21 9712 4339 --:--:-- --:--:-- --:--:-- 11750
{
"code": "0",
"message": "Idle timeout set to 60"
}
The idle timeout is by default set to 60 seconds. This means that the last packet of the transfer, on a per-flow basis, will cause the flows to expire and be automatically removed from the switches 60s later. This is a long time, but it is safe in that it allows the agents to clear their buffers and finish transferring data, which might be terminated prematurely given a combination of shorter timeouts and poor parallel connection and buffer size parameter choices. Note that the "feedback-port" key used when adding an agent (above) is going to be used to allow the agents to send explicit notification to the controller when they have completed a particular transfer; this is about 95% complete at this time. It will eliminate the need to estimate idle timeouts.
Changing the hard timeout is supported, but it does not make sense at present (thus I will not include an example ). Having hard timeouts could serve as a way to kick out transfers that have used more than their allotted share/time, but this is not a feature that is supported. As such, it is recommended that the hard timeout be left at 0 seconds (infinite).
Check Controller Readiness
Before a transfer is to be initiated, the controller should be queried to ensure all systems are ready to react to the transfer and perform SOS. This works well in a single-user environment where the user is performing sequential transfers. As such, it is the model we use at this point; however, a solution is nearing completion that allows the controller to pre-allocate resources for a user during a specific time period (which is more real-world) – these are the unused start and end times indicated in the whitelist REST API above.
To probe the controller to see if it is ready, one can use the status API as follows:
curl http://192.168.1.1:8080/wm/sos/status/json -X GET | python -m json.tool
which returns:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 68 0 47 100 21 9712 4339 --:--:-- --:--:-- --:--:-- 11750
{
"code": "0",
"message": "Ready to accept a transfer"
}
if the controller is ready. The return will appear as follows if the user should wait before proceeding:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 68 0 47 100 21 9712 4339 --:--:-- --:--:-- --:--:-- 11750
{
"code": "6",
"message": "Not ready to accept a transfer"
}
Initiate Transfer
Once the agents have been added, the transfer has been whitelisted, and the controller indicates the SOS network is ready to handle the transfer, the user can initiate the transfer using any means on the client machine. The assumption is that the underlying transport protocol used is TCP and that the client IP address, server IP address, and server TCP port are those indicated in the whitelisted entry.
 | It is important to remember that SOS is a transparent service. From a user's perspective, the transfer should be initiated from the client to the server as a normal transfer would be conducted. The transfer should not in any way be directed to/from the agent IP addresses – this will happen automatically for you. In order for SOS to properly detect and perform this redirection, the transfer must be initiated by the client IP to the server IP at the server TCP port as indicated in the whitelist entry configured above. |
|---|
Check Statistics
To get information about the transfer being performed, such as the agents chosen, the routes being used, etc. The statistics API can be used as follows:
curl http://192.168.1.1:8080/wm/sos/stats/json -X GET | python -m json.tool
which returns, as an example:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3283 0 3283 0 0 126k 0 --:--:-- --:--:-- --:--:-- 128k
{
"active-transfers": [
{
"buffer-size": "40000",
"client": {
"ip-address": "10.0.0.2",
"mac-address": "00:60:dd:43:ac:79",
"tcp-port": "51519"
},
"client-side-agent": {
"active-transfers": [
"9b3eb3ab-68f4-41ba-a102-d1498362eb63"
],
"control-port": "9998",
"data-port": "9877",
"feedback-port": "9997",
"ip-address": "10.0.0.1",
"mac-address": "00:60:dd:43:ac:72"
},
"flow-timeout": "10",
"parallel-connections": "4096",
"queue-capacity": "5",
"route-agent-to-agent": {
"destination-device": {
"active-transfers": [
"9b3eb3ab-68f4-41ba-a102-d1498362eb63"
],
"control-port": "9998",
"data-port": "9877",
"feedback-port": "9997",
"ip-address": "10.0.0.3",
"mac-address": "90:e2:ba:2b:c9:95"
},
"route": [
"[id=00:00:00:60:dd:43:ac:72, port=local]",
"[id=00:00:00:60:dd:43:ac:72, port=1]",
"[id=00:03:00:01:e8:d7:58:9b, port=25]",
"[id=00:03:00:01:e8:d7:58:9b, port=4]",
"[id=00:01:00:01:e8:8b:77:db, port=61]",
"[id=00:01:00:01:e8:8b:77:db, port=47]",
"[id=00:00:90:e2:ba:2b:c9:95, port=1]",
"[id=00:00:90:e2:ba:2b:c9:95, port=local]"
],
"route-type": "AGENT_2_AGENT",
"source-device": {
"active-transfers": [
"9b3eb3ab-68f4-41ba-a102-d1498362eb63"
],
"control-port": "9998",
"data-port": "9877",
"feedback-port": "9997",
"ip-address": "10.0.0.1",
"mac-address": "00:60:dd:43:ac:72"
}
},
"route-client-to-agent": {
"destination-device": {
"active-transfers": [
"9b3eb3ab-68f4-41ba-a102-d1498362eb63"
],
"control-port": "9998",
"data-port": "9877",
"feedback-port": "9997",
"ip-address": "10.0.0.1",
"mac-address": "00:60:dd:43:ac:72"
},
"route": [
"[id=00:03:00:01:e8:d7:58:9b, port=23]",
"[id=00:03:00:01:e8:d7:58:9b, port=25]",
"[id=00:00:00:60:dd:43:ac:72, port=1]",
"[id=00:00:00:60:dd:43:ac:72, port=local]"
],
"route-type": "CLIENT_2_AGENT",
"source-device": {
"ip-address": "10.0.0.2",
"mac-address": "00:60:dd:43:ac:79",
"tcp-port": "51519"
}
},
"route-server-to-agent": {
"destination-device": {
"active-transfers": [
"9b3eb3ab-68f4-41ba-a102-d1498362eb63"
],
"control-port": "9998",
"data-port": "9877",
"feedback-port": "9997",
"ip-address": "10.0.0.3",
"mac-address": "90:e2:ba:2b:c9:95"
},
"route": [
"[id=00:01:00:01:e8:8b:77:db, port=48]",
"[id=00:01:00:01:e8:8b:77:db, port=47]",
"[id=00:00:90:e2:ba:2b:c9:95, port=1]",
"[id=00:00:90:e2:ba:2b:c9:95, port=local]"
],
"route-type": "SERVER_2_AGENT",
"source-device": {
"ip-address": "10.0.0.4",
"mac-address": "02:63:1e:4f:ab:f1",
"tcp-port": "5001"
}
},
"server": {
"ip-address": "10.0.0.4",
"mac-address": "02:63:1e:4f:ab:f1",
"tcp-port": "5001"
},
"server-side-agent": {
"active-transfers": [
"9b3eb3ab-68f4-41ba-a102-d1498362eb63"
],
"control-port": "9998",
"data-port": "9877",
"feedback-port": "9997",
"ip-address": "10.0.0.3",
"mac-address": "90:e2:ba:2b:c9:95"
},
"server-side-agent-tcp-port": {
"length": "2",
"port": "37296"
},
"time-init": "Mon Oct 26 14:39:44 EDT 2015",
"time-start": "Mon Oct 26 14:39:45 EDT 2015",
"time-stop": "none",
"transfer-id": "9b3eb3ab-68f4-41ba-a102-d1498362eb63"
}
],
"agents": [
{
"active-transfers": [
"9b3eb3ab-68f4-41ba-a102-d1498362eb63"
],
"control-port": "9998",
"data-port": "9877",
"feedback-port": "9997",
"ip-address": "10.0.0.1",
"mac-address": "00:60:dd:43:ac:72"
},
{
"active-transfers": [
"9b3eb3ab-68f4-41ba-a102-d1498362eb63"
],
"control-port": "9998",
"data-port": "9877",
"feedback-port": "9997",
"ip-address": "10.0.0.3",
"mac-address": "90:e2:ba:2b:c9:95"
}
],
"terminated-connections": [],
"whitelist-entries": [
{
"client-ip-address": "10.0.0.2",
"server-ip-address": "10.0.0.4",
"server-tcp-port": "5001"
}
]
}