Networking and connection modes
Enterprise module
Connection modes are part of the GPU Fleet module (gpu-fleet) of your Cognipeer package.
The Connection tab of a machine's page decides how the console reaches the models on that machine. How the agent connects to the console (heartbeat, commands) is the same in every mode: the agent always connects outbound to the console over HTTPS. What changes is the traffic from the console to the model.
| Mode | The console connects to | What must be open on the machine | When to use it |
|---|---|---|---|
| Direct | http://<service address>:<model port> | The port of every model | The console and the machine are on the same network (VNet, VPN). |
| Agent gateway | The Endpoint URL, or http://<service address>:<gateway port> | Only the gateway port (default 7480), or a tunnel | Machines behind a tunnel, an HTTPS proxy or a VPN. |
| Relay | The connection the agent opened to the console | Nothing | Machines behind NAT that accept no inbound connections (for example a laptop Mac). |
Direct
The default. The console connects to each model's port on the service address. The service address is an IP or a DNS name (10.0.0.4, host.tailnet.ts.net), not a URL.
On a Mac, MLX models listen on 127.0.0.1 only; in direct mode they are reached through the gateway port on the same service address.
Agent gateway
The agent puts one authenticated port (default 7480) in front of all models. The console adds a token valid only for that deployment to each request; the gateway checks it and forwards the request to the model on the machine. In this mode Docker models also listen on 127.0.0.1 only.
You can put a tunnel or an HTTPS name in front of the gateway and enter it in Endpoint URL:
# ngrok
ngrok http 7480
# Endpoint URL: https://<your-name>.ngrok.app
# Cloudflare Tunnel
cloudflared tunnel --url http://localhost:7480
# Tailscale Funnel
tailscale funnel 7480The Endpoint URL must start with http:// or https:// and cannot contain a user name, password, query or fragment. A path prefix (https://proxy.example.com/gpu-01) is accepted. The console connects to the Endpoint URL over public addresses only.
If a tunnel on the same machine is the only way in, you do not need to expose the gateway: set COGNIPEER_GATEWAY_BIND=127.0.0.1 on the agent. The gateway port is set with COGNIPEER_GATEWAY_PORT (0 or off turns the gateway off; direct mode then cannot reach MLX models).
Relay
The agent opens a persistent WebSocket to the console outward, and model requests travel over it. No port, tunnel or public IP is needed. When the console runs on several nodes, requests are routed between the nodes through Redis.
Relay also carries streaming responses. A single request body can be at most 32 MB.
Testing the connection
Test connection on the card checks:
- in relay mode, whether the agent is connected;
- in gateway mode, whether the gateway answers;
- a
GET /v1/modelscall for up to five deployments on the machine.
Each step is listed with its reason when it fails (connection refused, name not resolved, timeout).
Changing the mode
- After a change, the inference monitoring records point to the new address. In relay mode these records are removed (monitoring does not go through the relay); Model Hub and pool traffic keep working through the relay.
- Docker models started while the machine was in gateway or relay mode listen on
127.0.0.1. When you go back to Direct, restart these models so their ports are reachable from outside.
See also Mac (Apple Silicon) and MLX.

