CCTools Network Options

When working in a cloud or HPC environment, you may find complex network conditions such as multiple network interfaces, firewalls, and other conditions. The following environment variables can be used to shape the network behavior of Makeflow, Work Queue, Parrot, Chirp, and other tools, so as to work correctly in these environments.

IPV6 Support

IPV6 is supported by all of the CCTools components, however it is not turned on by default because IPV6 is not reliably deployed at all sites. You can enable IPV6 support with the CCTOOLS_IP_MODE environment variable.

To enable both IPV4 and IPV6 support according to the local system configuration: (recommended use)

export CCTOOLS_IP_MODE=AUTO

To enable only IPV4 support: (the default)

export CCTOOLS_IP_MODE=IPV4

To enable only IPV6 support: (not recommended; use only for testing IPV6)

export CCTOOLS_IP_MODE=IPV6

Where it is necessary to combine an address and port together into a single string, an IPV4 combination looks like this:

192.168.0.1:9094

But an IPV6 combination looks like this:

[1234::abcd]:9094

TCP Port Ranges

When creating a listening TCP port, the CCTools will, by default, pick any port available on the machine. However, some computing sites set up firewall rules that only permit connections within certain port ranges. To accommodate this, set the TCP_LOW_PORT and TCP_HIGH_PORT environment variables, and the CCTools will only use ports within that range.

For example, if your site firewall only allows ports 8000-9000, do this:

export TCP_LOW_PORT=8000
export TCP_HIGH_PORT=9000

TCP Window Size

The performance of TCP connections over wide area links can be significantly affected by the "window size" used within the kernel. Ideally, the window size is set to the product of the network bandwidth and latency, and is managed automatically by the kernel. In certain cases, you may wish to set it manually with the TCP_WINDOW_SIZE environment variable, which gives the window size in bytes.

For example, to se the window size to 1MB:

export TCP_WINDOW_SIZE=1048576

HTTP Proxies

if your computing site requires all HTTP requests to be routed through a proxy, specify that proxy with the HTTP_PROXY environment variable. The value should be a semi-colon separated list of proxy URLs, in order of preference. The final entry may be DIRECT indicating that a direct connection should be attempted if all proxy connections fail.

For example:

export HTTP_PROXY=http://proxy01.nd.edu:3128;http://proxy02.nd.edu:3129;DIRECT