Life of a Knative Request (data plane)
(This page explains the ingress gateway used by Knative is Istio, which is the default.)
Assume a client just made a
GET / to the domain name of a Knative Service.
Here's how the traffic bytes flow through:
Domain name resolution
If you have an external domain pointing to a Knative Service, it means you've configured public DNS records for it, so public DNS handles that.
Traffic that comes to the internal domain uses Kubernetes service
For example, if you have a KService named
hello, a Kubernetes Service (of
type:ExternalName) is created to point to the cluster-local gateway is
$ kubectl describe service hello Name: hello Namespace: default Type: ExternalName IP: External Name: cluster-local-gateway.gke-system.svc.cluster.local
So, when a name lookup is made to kube-dns for
hello.default.svc.cluster.local, it returns a CNAME record pointing to the
cluster-local gateway (and conveniently, its IP address in an
$ dig hello.default.svc.cluster.local [...] ;; ANSWER SECTION: hello.default.svc.cluster.local. 30 IN CNAME cluster-local-gateway.gke-system.svc.cluster.local. cluster-local-gateway.gke-system.svc.cluster.local. 30 IN A 10.4.12.131
Request comes to one of the gateways
As part of creating the KService,
Knative creates Istio
VirtualService object for the KService and registers
it to these gateways:
$ kubectl get virtualservice hello -o=yaml apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: hello [...] spec: gateways: - knative-serving/cluster-local-gateway # <- internal gateway - knative-serving/gke-system-gateway # <- external gateway [...]
Gateway routes the traffic to Revisions
VirtualService object of the
route: section specifies
traffic routing rules. This is where traffic splitting configuration takes effect:
spec: [...] http: route: - weight: 100 destination: host: hello-lbsxh-3.default.svc.cluster.local port: number: 80 [...]
In this case, each
host: entry will point to a Kubernetes Service
pointing to a Knative Revision.
Istio configures Envoy proxy
Envoy routing to Revision
Since Knative configures the traffic as
http: on Istio, Istio will tell Envoy
to do Layer-7 (application-layer) traffic load balancing.
Envoy will read the incoming TCP connection, and parse out each request, and
each request will be load-balanced to one of the entries in
field, choosen using their
This way, Envoy routes the traffic to Revision’s IP address.
Dynamic Revision endpoint
If a Knative Revision has no active Pods running (i.e. scaled to zero), its corresponding Kubernetes
(which has the same name as the
Revision) will point to IP of the
activator component to wake up the
KService Pods on first request:
$ kubectl describe service hello-lbsxh-3 Name: hello-lbsxh-3 Namespace: default Labels: serving.knative.dev/revision=hello-lbsxh-3 [...] Type: ClusterIP Endpoints: 10.0.2.11:8012 # <- IP of knative activator Port: http 80/TCP
Activation via request (when scaled-to-zero)
When first request comes to the activator, it will hold onto the request, and will scale up the Knative Service to >0. Once a Pod becomes ready,
- it will proxy the traffic to Pod
- it will update the Endpoints of the Kubernetes Service so that the subsequent
requests to the
Revisiondirectly resolve to the app's Pod IPs.
Activation is explained here in detail.
Routing traffic to Pod (when Revision has Pods running)
Revision has Pods running, its Kubernetes Service will no longer
be pointing to the
$ kubectl describe service hello-lbsxh-3 Name: hello-lbsxh-3 Namespace: default Type: ClusterIP Labels: serving.knative.dev/revision=hello-lbsxh-3 [...] Endpoints: 10.0.0.43:8013,10.0.1.33:8013,10.0.3.39:8013 + 1 more...
From here on, normal Kubernetes ClusterIP load balancing takes place to forward traffic to Pod IPs.
Traffic reaching to Pod
Envoy proxy contains the
Service on port
Endpoints (as seen above), list the backend at
:8013. In the
Knative Pod, the
queue-proxy component listens on this port number (or
queue-proxy is responsible for making sure the Pod receives only the desired
amount of concurrent requests, and it
also reports concurrency metrics for autoscaling.
queue-proxy sends the traffic to the actual application by
establishing connection to the app pod over loopback interface (
the default port number (
8080) or the custom port.
TODO: add what happens when queue-proxy reaches the hard concurrency limit and how/where the traffic is sent to.
Traffic reaches to the user app
This way, the traffic makes its way to the app process.
When the user application sends a response back, the packets traverse the same components in the reverse direction, and make their way to the user.