Question:
This is a Feature Request What would you like to be High level documentation for Gateway API with links out to https://gateway- This should include the following: new "Gateway API" page between Service and Ingress in This should provide a high level overview of Gateway API, with a focus on content that is unlikely to become stale. This can likely take some content from both https://gateway- and https://gateway- and would help increase the discoverability of Gateway API. the "Ingress" page to note that the API is effectively frozen and all new features are being to Gateway API (with a corresponding link to the page by 1). the section on the "Service" page that references Ingress and Gateway API to give more prominence to Gateway API. Why is this needed With the upcoming graduation of Gateway API to GA (x-ref ), it's past time to cover it in Kubernetes docs. Comments I'd previously talked with about this at a high level, but definitely let me know if you've got any suggestions for how we can scope/adjust this differently. /sig network /assign
checkout the gateway API in the section and look out for the bugs
/triage accepted /priority important-longterm
How about adding a short link eg that redirects (302) to https://gateway- ?
I'd put Service, Ingress, then Gateway - and leave it in that order until most new clusters include Gateway as an add-on.
Also see

<ins> --- title: Gateway API content_type: concept description: >- Gateway API is a family of API kinds that provide dynamic infrastructure provisioning and advanced traffic routing. weight: 55 --- <!-- overview --> Make network services available by using an extensible, role-oriented, protocol-aware configuration mechanism. [Gateway API](https://gateway-api.sigs.k8s.io/) is an {{<glossary_tooltip text="add-on" term_id="addons">}} containing API [kinds](https://gateway-api.sigs.k8s.io/references/spec/) that provide dynamic infrastructure provisioning and advanced traffic routing. <!-- body --> ## Design principles The following principles shaped the design and architecture of Gateway API: * __Role-oriented:__ Gateway API kinds are modeled after organizational roles that are responsible for managing Kubernetes service networking: * __Infrastructure Provider:__ Manages infrastructure that allows multiple isolated clusters to serve multiple tenants, e.g. a cloud provider. * __Cluster Operator:__ Manages clusters and is typically concerned with policies, network access, application permissions, etc. * __Application Developer:__ Manages an application running in a cluster and is typically concerned with application-level configuration and [Service](/docs/concepts/services-networking/service/) composition. * __Portable:__ Gateway API specifications are defined as [custom resources](docs/concepts/extend-kubernetes/api-extension/custom-resources) and are supported by many [implementations](https://gateway-api.sigs.k8s.io/implementations/). * __Expressive:__ Gateway API kinds support functionality for common traffic routing use cases such as header-based matching, traffic weighting, and others that were only possible in [Ingress](/docs/concepts/services-networking/ingress/) by using custom annotations. * __Extensible:__ Gateway allows for custom resources to be linked at various layers of the API. This makes granular customization possible at the appropriate places within the API structure. ## Resource model Gateway API has three stable API kinds: * __GatewayClass:__ Defines a set of gateways with common configuration and managed by a controller that implements the class. * __Gateway:__ Defines an instance of traffic handling infrastructure, such as cloud load balancer. * __HTTPRoute:__ Defines HTTP-specific rules for mapping traffic from a Gateway listener to a representation of backend network endpoints. These endpoints are often represented as a {{<glossary_tooltip text="Service" term_id="service">}}. Gateway API is organized into different API kinds that have interdependent relationships to support the role-oriented nature of organizations. A Gateway object is associated with exactly one GatewayClass; the GatewayClass describes the gateway controller responsible for managing Gateways of this class. One or more route kinds such as HTTPRoute, are then associated to Gateways. A Gateway can filter the routes that may be attached to its `listeners`, forming a bidirectional trust model with routes. The following figure illustrates the relationships of the three stable Gateway API kinds: {{< figure src="/docs/images/gateway-kind-relationships.svg" alt="A figure illustrating the relationships of the three stable Gateway API kinds" class="diagram-medium" >}} ### GatewayClass {#api-kind-gateway-class} Gateways can be implemented by different controllers, often with different configurations. A Gateway must reference a GatewayClass that contains the name of the controller that implements the class. A minimal GatewayClass example: ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: GatewayClass metadata: name: example-class spec: controllerName: example.com/gateway-controller ``` In this example, a controller that has implemented Gateway API is configured to manage GatewayClasses with the controller name `example.com/gateway-controller`. Gateways of this class will be managed by the implementation's controller. See the [GatewayClass](https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1.GatewayClass) reference for a full definition of this API kind. ### Gateway {#api-kind-gateway} A Gateway describes an instance of traffic handling infrastructure. It defines a network endpoint that can be used for processing traffic, i.e. filtering, balancing, splitting, etc. for backends such as a Service. For example, a Gateway may represent a cloud load balancer or an in-cluster proxy server that is configured to accept HTTP traffic. A minimal Gateway resource example: ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: example-gateway spec: gatewayClassName: example-class listeners: - name: http protocol: HTTP port: 80 ``` In this example, an instance of traffic handling infrastructure is programmed to listen for HTTP traffic on port 80. Since the `addresses` field is unspecified, an address or hostname is assigned to the Gateway by the implementation's controller. This address is used as a network endpoint for processing traffic of backend network endpoints defined in routes. See the [Gateway](https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1.Gateway) reference for a full definition of this API kind. ### HTTPRoute {#api-kind-httproute} The HTTPRoute kind specifies routing behavior of HTTP requests from a Gateway listener to backend network endpoints. For a Service backend, an implementation may represent the backend network endpoint as a Service IP or the backing Endpoints of the Service. An HTTPRoute represents configuration that is applied to the underlying Gateway implementation. For example, defining a new HTTPRoute may result in configuring additional traffic routes in a cloud load balancer or in-cluster proxy server. A minimal HTTPRoute example: ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: example-httproute spec: parentRefs: - name: example-gateway hostnames: - "www.example.com" rules: - matches: - path: type: PathPrefix value: /login backendRefs: - name: example-svc port: 8080 ``` In this example, HTTP traffic from Gateway `example-gateway` with the Host: header set to `www.example.com` and the request path specified as `/login` will be routed to Service `example-svc` on port `8080`. See the [HTTPRoute](https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1.HTTPRoute) reference for a full definition of this API kind. ## Request flow Here is a simple example of HTTP traffic being routed to a Service by using a Gateway and an HTTPRoute: {{< figure src="/docs/images/gateway-request-flow.svg" alt="A diagram that provides an example of HTTP traffic being routed to a Service by using a Gateway and an HTTPRoute" class="diagram-medium" >}} In this example, the request flow for a Gateway implemented as a reverse proxy is: 1. The client starts to prepare an HTTP request for the URL `http://www.example.com` 2. The client's DNS resolver queries for the destination name and learns a mapping to one or more IP addresses associated with the Gateway. 3. The client sends a request to the Gateway IP address; the reverse proxy receives the HTTP request and uses the Host: header to match a configuration that was derived from the Gateway and attached HTTPRoute. 4. Optionally, the reverse proxy can perform request header and/or path matching based on match rules of the HTTPRoute. 5. Optionally, the reverse proxy can modify the request; for example, to add or emove headers, based on filter rules of the HTTPRoute. 6. Lastly, the reverse proxy forwards the request to one or more backends. ## Conformance Gateway API covers a broad set of features and is widely implemented. This combination requires clear conformance definitions and tests to ensure that the API provides a consistent experience wherever it is used. See the [conformance](https://gateway-api.sigs.k8s.io/concepts/conformance/) documentation to understand details such as release channels, support levels, and running conformance tests. ## Migrating from Ingress Gateway API is the successor to the [Ingress](/docs/concepts/services-networking/ingress/) API. However, it does not include the Ingress kind. As a result, a one-time conversion from your existing Ingress resources to Gateway API resources is necessary. Refer to the [ingress migration](https://gateway-api.sigs.k8s.io/guides/migrating-from-ingress/#migrating-from-ingress) guide for details on migrating Ingress resources to Gateway API resources. ## {{% heading "whatsnext" %}} Instead of Gateway API resources being natively implemented by Kubernetes, the specifications are defined as [Custom Resources](docs/concepts/extend-kubernetes/api-extension/custom-resources) supported by a wide range of [implementations](https://gateway-api.sigs.k8s.io/implementations/). [Install](https://gateway-api.sigs.k8s.io/guides/#installing-gateway-api) the Gateway API CRDs or follow the installation instructions of your selected implementation. After installing an implementation, use the [Getting Started](https://gateway-api.sigs.k8s.io/guides/) guide to help you quickly start working with Gateway API. {{< note >}} Make sure to review the documentation of your selected implementation to understand any caveats. {{< /note >}} Refer to the [API specification](https://gateway-api.sigs.k8s.io/reference/spec/) for additional details of all Gateway API kinds. </ins>


Question:
This is a Feature Request What would you like to be High level documentation for Gateway API with links out to https://gateway- This should include the following: new "Gateway API" page between Service and Ingress in This should provide a high level overview of Gateway API, with a focus on content that is unlikely to become stale. This can likely take some content from both https://gateway- and https://gateway- and would help increase the discoverability of Gateway API. the "Ingress" page to note that the API is effectively frozen and all new features are being to Gateway API (with a corresponding link to the page by 1). the section on the "Service" page that references Ingress and Gateway API to give more prominence to Gateway API. Why is this needed With the upcoming graduation of Gateway API to GA (x-ref ), it's past time to cover it in Kubernetes docs. Comments I'd previously talked with about this at a high level, but definitely let me know if you've got any suggestions for how we can scope/adjust this differently. /sig network /assign
checkout the gateway API in the section and look out for the bugs
/triage accepted /priority important-longterm
How about adding a short link eg that redirects (302) to https://gateway- ?
I'd put Service, Ingress, then Gateway - and leave it in that order until most new clusters include Gateway as an add-on.
Also see

{{< feature-state for_k8s_version="v1.19" state="stable" >}} {{< glossary_definition term_id="ingress" length="all" >}} <ins> {{< note >}} Ingress is frozen. New features are being added to the [Gateway API](/docs/concepts/services-networking/gateway/). {{< /note >}} </ins> <!-- body --> ## Terminology


Question:
This is a Feature Request What would you like to be High level documentation for Gateway API with links out to https://gateway- This should include the following: new "Gateway API" page between Service and Ingress in This should provide a high level overview of Gateway API, with a focus on content that is unlikely to become stale. This can likely take some content from both https://gateway- and https://gateway- and would help increase the discoverability of Gateway API. the "Ingress" page to note that the API is effectively frozen and all new features are being to Gateway API (with a corresponding link to the page by 1). the section on the "Service" page that references Ingress and Gateway API to give more prominence to Gateway API. Why is this needed With the upcoming graduation of Gateway API to GA (x-ref ), it's past time to cover it in Kubernetes docs. Comments I'd previously talked with about this at a high level, but definitely let me know if you've got any suggestions for how we can scope/adjust this differently. /sig network /assign
checkout the gateway API in the section and look out for the bugs
/triage accepted /priority important-longterm
How about adding a short link eg that redirects (302) to https://gateway- ?
I'd put Service, Ingress, then Gateway - and leave it in that order until most new clusters include Gateway as an add-on.
Also see

* Read about [Ingress](/docs/concepts/services-networking/ingress/), which exposes HTTP and HTTPS routes from outside the cluster to Services within your cluster. <del> * Read about [Gateway](https://gateway-api.sigs.k8s.io/), an extension to </del> <ins> * Read about [Gateway](/docs/concepts/services-networking/gateway/), an extension to </ins> Kubernetes that provides more flexibility than Ingress. For more context, read the following:


Question:
This is a Feature Request What would you like to be High level documentation for Gateway API with links out to https://gateway- This should include the following: new "Gateway API" page between Service and Ingress in This should provide a high level overview of Gateway API, with a focus on content that is unlikely to become stale. This can likely take some content from both https://gateway- and https://gateway- and would help increase the discoverability of Gateway API. the "Ingress" page to note that the API is effectively frozen and all new features are being to Gateway API (with a corresponding link to the page by 1). the section on the "Service" page that references Ingress and Gateway API to give more prominence to Gateway API. Why is this needed With the upcoming graduation of Gateway API to GA (x-ref ), it's past time to cover it in Kubernetes docs. Comments I'd previously talked with about this at a high level, but definitely let me know if you've got any suggestions for how we can scope/adjust this differently. /sig network /assign
checkout the gateway API in the section and look out for the bugs
/triage accepted /priority important-longterm
How about adding a short link eg that redirects (302) to https://gateway- ?
I'd put Service, Ingress, then Gateway - and leave it in that order until most new clusters include Gateway as an add-on.
Also see

 <svg id="graph-div" width="100%" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" height="130" style="max-width: 465.6875px;" viewBox="0 0 465.6875 130"><g><g class="output"><g class="clusters"><g class="cluster" id="flowchart-cluster-721" transform="translate(232.84375,65)" style="opacity: 1;"><rect width="449.6875" height="114" x="-224.84375" y="-57"></rect><g class="label" transform="translate(0, -43)" id="graph-divText"><g transform="translate(-24.578125,-12)"><foreignObject width="49.15625" height="24">
cluster
</foreignObject></g></g></g></g><g class="edgePaths"><g class="edgePath LS-gateway LE-gatewayclass" id="L-gateway-gatewayclass" style="opacity: 1;"><path class="path" d="M264.53125,65L268.6979166666667,65C272.8645833333333,65,281.1979166666667,65,289.53125,65C297.8645833333333,65,306.1979166666667,65,310.3645833333333,65L314.53125,65" marker-end="url(#arrowhead1117)" style="fill:none"></path><defs><marker id="arrowhead1117" viewBox="0 0 10 10" refX="9" refY="5" markerUnits="strokeWidth" markerWidth="8" markerHeight="6" orient="auto"><path d="M 0 0 L 10 5 L 0 10 z" class="arrowheadPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"></path></marker></defs></g><g class="edgePath LS-httproute1 LE-gateway" id="L-httproute1-gateway" style="opacity: 1;"><path class="path" d="M132.03125,65L136.19791666666666,65C140.36458333333334,65,148.69791666666666,65,157.03125,65C165.36458333333334,65,173.69791666666666,65,177.86458333333334,65L182.03125,65" marker-end="url(#arrowhead1118)" style="fill:none"></path><defs><marker id="arrowhead1118" viewBox="0 0 10 10" refX="9" refY="5" markerUnits="strokeWidth" markerWidth="8" markerHeight="6" orient="auto"><path d="M 0 0 L 10 5 L 0 10 z" class="arrowheadPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"></path></marker></defs></g></g><g class="edgeLabels"><g class="edgeLabel" transform="" style="opacity: 1;"><g transform="translate(0,0)" class="label"><rect rx="0" ry="0" width="0" height="0"></rect><foreignObject width="0" height="0">

</foreignObject></g></g><g class="edgeLabel" transform="" style="opacity: 1;"><g transform="translate(0,0)" class="label"><rect rx="0" ry="0" width="0" height="0"></rect><foreignObject width="0" height="0">

</foreignObject></g></g></g><g class="nodes"><g class="node k8s" id="flowchart-gatewayclass-717" transform="translate(373.609375,65)" style="opacity: 1;"><rect rx="0" ry="0" x="-59.078125" y="-22" width="118.15625" height="44" class="label-container"></rect><g class="label" transform="translate(0,0)"><g transform="translate(-49.078125,-12)"><foreignObject width="98.15625" height="24">
GatewayClass
</foreignObject></g></g></g><g class="node k8s" id="flowchart-gateway-715" transform="translate(223.28125,65)" style="opacity: 1;"><rect rx="0" ry="0" x="-41.25" y="-22" width="82.5" height="44" class="label-container"></rect><g class="label" transform="translate(0,0)"><g transform="translate(-31.25,-12)"><foreignObject width="62.5" height="24">
Gateway
</foreignObject></g></g></g><g class="node k8s" id="flowchart-httproute1-719" transform="translate(82.515625,65)" style="opacity: 1;"><rect rx="0" ry="0" x="-49.515625" y="-22" width="99.03125" height="44" class="label-container"></rect><g class="label" transform="translate(0,0)"><g transform="translate(-39.515625,-12)"><foreignObject width="79.03125" height="24">
HTTPRoute
</foreignObject></g></g></g></g></g></g></svg>   No newline at end of file


Question:
This is a Feature Request What would you like to be High level documentation for Gateway API with links out to https://gateway- This should include the following: new "Gateway API" page between Service and Ingress in This should provide a high level overview of Gateway API, with a focus on content that is unlikely to become stale. This can likely take some content from both https://gateway- and https://gateway- and would help increase the discoverability of Gateway API. the "Ingress" page to note that the API is effectively frozen and all new features are being to Gateway API (with a corresponding link to the page by 1). the section on the "Service" page that references Ingress and Gateway API to give more prominence to Gateway API. Why is this needed With the upcoming graduation of Gateway API to GA (x-ref ), it's past time to cover it in Kubernetes docs. Comments I'd previously talked with about this at a high level, but definitely let me know if you've got any suggestions for how we can scope/adjust this differently. /sig network /assign
checkout the gateway API in the section and look out for the bugs
/triage accepted /priority important-longterm
How about adding a short link eg that redirects (302) to https://gateway- ?
I'd put Service, Ingress, then Gateway - and leave it in that order until most new clusters include Gateway as an add-on.
Also see

<ins> <svg version="1.1" viewBox="0.0 0.0 821.0 113.0" fill="none" stroke="none" stroke-linecap="square" stroke-miterlimit="10" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg"><clipPath id="p.0"><path d="m0 0l821.0 0l0 113.0l-821.0 0l0 -113.0z" clip-rule="nonzero"/></clipPath><g clip-path="url(#p.0)"><path fill="#000000" fill-opacity="0.0" d="m0 0l821.0 0l0 113.0l-821.0 0z" fill-rule="evenodd"/><path fill="#1155cc" d="m199.1693 28.55643l103.30708 0l0 66.4252l-103.30708 0z" fill-rule="evenodd"/><path stroke="#1155cc" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m199.1693 28.55643l103.30708 0l0 66.4252l-103.30708 0z" fill-rule="evenodd"/><path fill="#ffffff" d="m220.01613 63.78278l0 -2.25l5.8125 0l0 5.312496q-0.84375 0.828125 -2.453125 1.453125q-1.609375 0.625 -3.25 0.625q-2.09375 0 -3.65625 -0.875q-1.5625 -0.890625 -2.34375 -2.515625q-0.78125 -1.6406212 -0.78125 -3.5624962q0 -2.09375 0.875 -3.703125q0.875 -1.625 2.5625 -2.5q1.28125 -0.65625 3.203125 -0.65625q2.484375 0 3.875 1.046875q1.40625 1.03125 1.796875 2.875l-2.671875 0.5q-0.28125 -0.984375 -1.0625 -1.546875q-0.78125 -0.578125 -1.9375 -0.578125q-1.78125 0 -2.828125 1.125q-1.03125 1.125 -1.03125 3.328125q0 2.3749962 1.046875 3.5624962q1.0625 1.1875 2.78125 1.1875q0.84375 0 1.6875 -0.328125q0.859375 -0.328125 1.46875 -0.8125l0 -1.6874962l-3.09375 0zm10.198929 -1.8125l-2.328125 -0.421875q0.40625 -1.40625 1.359375 -2.078125q0.953125 -0.671875 2.84375 -0.671875q1.703125 0 2.546875 0.40625q0.84375 0.40625 1.171875 1.03125q0.34375 0.625 0.34375 2.28125l-0.015625 2.9999962q0 1.265625 0.109375 1.875q0.125 0.609375 0.46875 1.296875l-2.53125 0q-0.109375 -0.25 -0.25 -0.75q-0.0625 -0.234375 -0.09375 -0.3125q-0.65625 0.640625 -1.40625 0.96875q-0.734375 0.3125 -1.59375 0.3125q-1.484375 0 -2.34375 -0.8125q-0.859375 -0.8125 -0.859375 -2.046875q0 -0.828125 0.390625 -1.46875q0.390625 -0.6406212 1.09375 -0.9687462q0.703125 -0.34375 2.03125 -0.609375q1.796875 -0.328125 2.484375 -0.625l0 -0.25q0 -0.75 -0.359375 -1.0625q-0.359375 -0.3125 -1.375 -0.3125q-0.6875 0 -1.078125 0.28125q-0.375 0.265625 -0.609375 0.9375zm3.421875 2.0781212q-0.484375 0.15625 -1.5625 0.390625q-1.0625 0.21875 -1.390625 0.4375q-0.5 0.359375 -0.5 0.90625q0 0.53125 0.40625 0.9375q0.40625 0.390625 1.015625 0.390625q0.703125 0 1.328125 -0.46875q0.46875 -0.34375 0.609375 -0.84375q0.09375 -0.328125 0.09375 -1.25l0 -0.5zm9.485092 -5.031246l0 2.03125l-1.75 0l0 3.9062462q0 1.1875 0.046875 1.390625q0.046875 0.1875 0.21875 0.3125q0.1875 0.125 0.4375 0.125q0.359375 0 1.03125 -0.25l0.21875 2.0q-0.890625 0.375 -2.015625 0.375q-0.703125 0 -1.265625 -0.234375q-0.546875 -0.234375 -0.8125 -0.59375q-0.25 -0.375 -0.34375 -1.0q-0.09375 -0.453125 -0.09375 -1.8125l0 -4.218746l-1.171875 0l0 -2.03125l1.171875 0l0 -1.921875l2.578125 -1.5l0 3.421875l1.75 0zm7.368927 6.593746l2.546875 0.421875q-0.484375 1.40625 -1.546875 2.140625q-1.0625 0.734375 -2.65625 0.734375q-2.515625 0 -3.734375 -1.65625q-0.953125 -1.3125 -0.953125 -3.3281212q0 -2.40625 1.25 -3.765625q1.265625 -1.359375 3.1875 -1.359375q2.15625 0 3.40625 1.421875q1.25 1.421875 1.1875 4.374996l-6.40625 0q0.03125 1.140625 0.609375 1.78125q0.59375 0.625 1.484375 0.625q0.59375 0 1.0 -0.328125q0.421875 -0.328125 0.625 -1.0625zm0.15625 -2.5937462q-0.03125 -1.109375 -0.578125 -1.6875q-0.546875 -0.578125 -1.328125 -0.578125q-0.84375 0 -1.390625 0.609375q-0.546875 0.609375 -0.53125 1.65625l3.828125 0zm6.422592 5.671871l-3.0625 -9.671871l2.484375 0l1.8125 6.343746l1.671875 -6.343746l2.46875 0l1.609375 6.343746l1.859375 -6.343746l2.515625 0l-3.109375 9.671871l-2.453125 0l-1.671875 -6.218746l-1.640625 6.218746l-2.484375 0zm14.620819 -6.718746l-2.328125 -0.421875q0.40625 -1.40625 1.359375 -2.078125q0.953125 -0.671875 2.84375 -0.671875q1.703125 0 2.546875 0.40625q0.84375 0.40625 1.171875 1.03125q0.34375 0.625 0.34375 2.28125l-0.015625 2.9999962q0 1.265625 0.109375 1.875q0.125 0.609375 0.46875 1.296875l-2.53125 0q-0.109375 -0.25 -0.25 -0.75q-0.0625 -0.234375 -0.09375 -0.3125q-0.65625 0.640625 -1.40625 0.96875q-0.734375 0.3125 -1.59375 0.3125q-1.484375 0 -2.34375 -0.8125q-0.859375 -0.8125 -0.859375 -2.046875q0 -0.828125 0.390625 -1.46875q0.390625 -0.6406212 1.09375 -0.9687462q0.703125 -0.34375 2.03125 -0.609375q1.796875 -0.328125 2.484375 -0.625l0 -0.25q0 -0.75 -0.359375 -1.0625q-0.359375 -0.3125 -1.375 -0.3125q-0.6875 0 -1.078125 0.28125q-0.375 0.265625 -0.609375 0.9375zm3.421875 2.0781212q-0.484375 0.15625 -1.5625 0.390625q-1.0625 0.21875 -1.390625 0.4375q-0.5 0.359375 -0.5 0.90625q0 0.53125 0.40625 0.9375q0.40625 0.390625 1.015625 0.390625q0.703125 0 1.328125 -0.46875q0.46875 -0.34375 0.609375 -0.84375q0.09375 -0.328125 0.09375 -1.25l0 -0.5zm3.828827 -5.031246l2.71875 0l2.328125 6.859371l2.25 -6.859371l2.65625 0l-3.421875 9.312496l-0.609375 1.6875q-0.328125 0.84375 -0.640625 1.28125q-0.3125 0.453125 -0.703125 0.71875q-0.390625 0.28125 -0.96875 0.4375q-0.578125 0.15625 -1.3125 0.15625q-0.734375 0 -1.453125 -0.15625l-0.21875 -2.0q0.59375 0.125 1.078125 0.125q0.890625 0 1.3125 -0.53125q0.4375 -0.515625 0.671875 -1.328125l-3.6875 -9.703121z" fill-rule="nonzero"/><path fill="#000000" fill-opacity="0.0" d="m85.531494 61.769028l113.6378 0" fill-rule="evenodd"/><path stroke="#000000" stroke-width="2.0" stroke-linejoin="round" stroke-linecap="butt" stroke-dasharray="2.0,6.0" d="m85.531494 61.769028l101.6378 0" fill-rule="evenodd"/><path fill="#000000" stroke="#000000" stroke-width="2.0" stroke-linecap="butt" d="m187.1693 65.072495l9.076187 -3.3034668l-9.076187 -3.303463z" fill-rule="evenodd"/><path fill="#000000" fill-opacity="0.0" d="m302.47638 61.769028l43.338593 0" fill-rule="evenodd"/><path stroke="#000000" stroke-width="2.0" stroke-linejoin="round" stroke-linecap="butt" d="m302.47638 61.769028l31.338593 0" fill-rule="evenodd"/><path fill="#000000" stroke="#000000" stroke-width="2.0" stroke-linecap="butt" d="m333.81497 65.072495l9.076172 -3.3034668l-9.076172 -3.303463z" fill-rule="evenodd"/><path fill="#d9d9d9" d="m27.899902 43.32703l51.09812 0l0 0c6.6939926 0 12.120544 8.256859 12.120544 18.442207c0 10.185345 -5.426552 18.442204 -12.120544 18.442204l-51.09812 0l0 0c-6.6939926 0 -12.120544 -8.256859 -12.120544 -18.442204c0 -10.1853485 5.426552 -18.442207 12.120544 -18.442207z" fill-rule="evenodd"/><path stroke="#ffffff" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m27.899902 43.32703l51.09812 0l0 0c6.6939926 0 12.120544 8.256859 12.120544 18.442207c0 10.185345 -5.426552 18.442204 -12.120544 18.442204l-51.09812 0l0 0c-6.6939926 0 -12.120544 -8.256859 -12.120544 -18.442204c0 -10.1853485 5.426552 -18.442207 12.120544 -18.442207z" fill-rule="evenodd"/><path fill="#000000" d="m39.219585 65.14236l1.609375 0.21875q-0.265625 1.65625 -1.359375 2.609375q-1.078125 0.9375 -2.671875 0.9375q-1.984375 0 -3.1875 -1.296875q-1.203125 -1.296875 -1.203125 -3.71875q0 -1.578125 0.515625 -2.75q0.515625 -1.171875 1.578125 -1.75q1.0625 -0.59375 2.3125 -0.59375q1.578125 0 2.578125 0.796875q1.0 0.796875 1.28125 2.265625l-1.59375 0.234375q-0.234375 -0.96875 -0.8125 -1.453125q-0.578125 -0.5 -1.390625 -0.5q-1.234375 0 -2.015625 0.890625q-0.78125 0.890625 -0.78125 2.8125q0 1.953125 0.75 2.84375q0.75 0.875 1.953125 0.875q0.96875 0 1.609375 -0.59375q0.65625 -0.59375 0.828125 -1.828125zm2.96875 3.546875l0 -13.359375l1.640625 0l0 13.359375l-1.640625 0zm4.191696 -11.46875l0 -1.890625l1.640625 0l0 1.890625l-1.640625 0zm0 11.46875l0 -9.671875l1.640625 0l0 9.671875l-1.640625 0zm10.769821 -3.109375l1.6875 0.203125q-0.40625 1.484375 -1.484375 2.3125q-1.078125 0.8125 -2.765625 0.8125q-2.125 0 -3.375 -1.296875q-1.234375 -1.3125 -1.234375 -3.671875q0 -2.453125 1.25 -3.796875q1.265625 -1.34375 3.265625 -1.34375q1.9375 0 3.15625 1.328125q1.234375 1.3125 1.234375 3.703125q0 0.15625 0 0.4375l-7.21875 0q0.09375 1.59375 0.90625 2.453125q0.8125 0.84375 2.015625 0.84375q0.90625 0 1.546875 -0.46875q0.640625 -0.484375 1.015625 -1.515625zm-5.390625 -2.65625l5.40625 0q-0.109375 -1.21875 -0.625 -1.828125q-0.78125 -0.953125 -2.03125 -0.953125q-1.125 0 -1.90625 0.765625q-0.765625 0.75 -0.84375 2.015625zm9.141342 5.765625l0 -9.671875l1.46875 0l0 1.375q1.0625 -1.59375 3.0781212 -1.59375q0.875 0 1.609375 0.3125q0.734375 0.3125 1.09375 0.828125q0.375 0.5 0.515625 1.203125q0.09375 0.453125 0.09375 1.59375l0 5.953125l-1.640625 0l0 -5.890625q0 -1.0 -0.203125 -1.484375q-0.1875 -0.5 -0.671875 -0.796875q-0.484375 -0.296875 -1.140625 -0.296875q-1.046875 0 -1.8124962 0.671875q-0.75 0.65625 -0.75 2.515625l0 5.28125l-1.640625 0zm13.953838 -1.46875l0.234375 1.453125q-0.6875 0.140625 -1.234375 0.140625q-0.890625 0 -1.390625 -0.28125q-0.484375 -0.28125 -0.6875 -0.734375q-0.203125 -0.46875 -0.203125 -1.9375l0 -5.578125l-1.203125 0l0 -1.265625l1.203125 0l0 -2.390625l1.625 -0.984375l0 3.375l1.65625 0l0 1.265625l-1.65625 0l0 5.671875q0 0.6875 0.078125 0.890625q0.09375 0.203125 0.28125 0.328125q0.203125 0.109375 0.578125 0.109375q0.265625 0 0.71875 -0.0625z" fill-rule="nonzero"/><path fill="#1155cc" d="m345.82153 28.55643l121.73227 0l0 66.4252l-121.73227 0z" fill-rule="evenodd"/><path stroke="#1155cc" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m345.82153 28.55643l121.73227 0l0 66.4252l-121.73227 0z" fill-rule="evenodd"/><path fill="#ffffff" d="m357.26614 68.689026l0 -13.359371l2.703125 0l0 5.265625l5.28125 0l0 -5.265625l2.703125 0l0 13.359371l-2.703125 0l0 -5.843746l-5.28125 0l0 5.843746l-2.703125 0zm16.472961 0l0 -11.093746l-3.953125 0l0 -2.265625l10.609375 0l0 2.265625l-3.953125 0l0 11.093746l-2.703125 0zm11.395966 0l0 -11.093746l-3.953125 0l0 -2.265625l10.609375 0l0 2.265625l-3.953125 0l0 11.093746l-2.703125 0zm8.395996 0l0 -13.359371l4.328125 0q2.453125 0 3.203125 0.203125q1.140625 0.296875 1.921875 1.3125q0.78125 1.0 0.78125 2.59375q0 1.234375 -0.453125 2.078125q-0.453125 0.828125 -1.140625 1.3125q-0.6875 0.46875 -1.390625 0.625q-0.96875 0.203125 -2.796875 0.203125l-1.765625 0l0 5.031246l-2.6875 0zm2.6875 -11.093746l0 3.78125l1.484375 0q1.59375 0 2.125 -0.203125q0.546875 -0.203125 0.84375 -0.65625q0.3125 -0.453125 0.3125 -1.03125q0 -0.734375 -0.4375 -1.203125q-0.421875 -0.484375 -1.078125 -0.59375q-0.484375 -0.09375 -1.9375 -0.09375l-1.3125 0zm9.756073 11.093746l0 -13.359371l5.6875 0q2.140625 0 3.109375 0.359375q0.96875 0.359375 1.546875 1.28125q0.578125 0.921875 0.578125 2.109375q0 1.5 -0.890625 2.484375q-0.875 0.96875 -2.625 1.234375q0.875 0.5 1.4375 1.125q0.578125 0.6093712 1.53125 2.1562462l1.640625 2.609375l-3.234375 0l-1.9375 -2.90625q-1.046875 -1.5625 -1.4375 -1.9687462q-0.375 -0.40625 -0.8125 -0.546875q-0.421875 -0.15625 -1.34375 -0.15625l-0.546875 0l0 5.578121l-2.703125 0zm2.703125 -7.703121l2.0 0q1.9375 0 2.421875 -0.15625q0.484375 -0.171875 0.75 -0.578125q0.28125 -0.40625 0.28125 -1.0q0 -0.671875 -0.359375 -1.078125q-0.359375 -0.421875 -1.015625 -0.53125q-0.328125 -0.046875 -1.96875 -0.046875l-2.109375 0l0 3.390625zm10.160431 2.734375q0 -1.28125 0.625 -2.46875q0.625 -1.203125 1.78125 -1.828125q1.15625 -0.625 2.578125 -0.625q2.1875 0 3.59375 1.421875q1.40625 1.421875 1.40625 3.609375q0 2.1874962 -1.421875 3.6406212q-1.421875 1.4375 -3.5625 1.4375q-1.328125 0 -2.546875 -0.59375q-1.203125 -0.609375 -1.828125 -1.765625q-0.625 -1.171875 -0.625 -2.8281212zm2.625 0.125q0 1.4531212 0.671875 2.2187462q0.6875 0.75 1.6875 0.75q1.0 0 1.671875 -0.75q0.6875 -0.765625 0.6875 -2.2343712q0 -1.421875 -0.6875 -2.1875q-0.671875 -0.765625 -1.671875 -0.765625q-1.0 0 -1.6875 0.765625q-0.671875 0.765625 -0.671875 2.203125zm15.724121 4.843746l0 -1.453125q-0.53125 0.78125 -1.390625 1.234375q-0.859375 0.4375 -1.8125 0.4375q-0.96875 0 -1.75 -0.421875q-0.765625 -0.4375 -1.125 -1.203125q-0.34375 -0.78125 -0.34375 -2.140625l0 -6.124996l2.5625 0l0 4.4375q0 2.0468712 0.140625 2.5156212q0.140625 0.453125 0.515625 0.71875q0.375 0.265625 0.953125 0.265625q0.65625 0 1.171875 -0.359375q0.515625 -0.359375 0.703125 -0.890625q0.203125 -0.53125 0.203125 -2.6093712l0 -4.078125l2.546875 0l0 9.671871l-2.375 0zm9.474091 -9.671871l0 2.03125l-1.75 0l0 3.9062462q0 1.1875 0.046875 1.390625q0.046875 0.1875 0.21875 0.3125q0.1875 0.125 0.4375 0.125q0.359375 0 1.03125 -0.25l0.21875 2.0q-0.890625 0.375 -2.015625 0.375q-0.703125 0 -1.265625 -0.234375q-0.546875 -0.234375 -0.8125 -0.59375q-0.25 -0.375 -0.34375 -1.0q-0.09375 -0.453125 -0.09375 -1.8125l0 -4.218746l-1.171875 0l0 -2.03125l1.171875 0l0 -1.921875l2.578125 -1.5l0 3.421875l1.75 0zm7.368927 6.593746l2.546875 0.421875q-0.484375 1.40625 -1.546875 2.140625q-1.0625 0.734375 -2.65625 0.734375q-2.515625 0 -3.734375 -1.65625q-0.953125 -1.3125 -0.953125 -3.3281212q0 -2.40625 1.25 -3.765625q1.265625 -1.359375 3.1875 -1.359375q2.15625 0 3.40625 1.421875q1.25 1.421875 1.1875 4.374996l-6.40625 0q0.03125 1.140625 0.609375 1.78125q0.59375 0.625 1.484375 0.625q0.59375 0 1.0 -0.328125q0.421875 -0.328125 0.625 -1.0625zm0.15625 -2.5937462q-0.03125 -1.109375 -0.578125 -1.6875q-0.546875 -0.578125 -1.328125 -0.578125q-0.84375 0 -1.390625 0.609375q-0.546875 0.609375 -0.53125 1.65625l3.828125 0z" fill-rule="nonzero"/><path fill="#d9d9d9" d="m109.42257 42.04987l63.244102 0l0 18.456692l-63.244102 0z" fill-rule="evenodd"/><path stroke="#ffffff" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m109.42257 42.04987l63.244102 0l0 18.456692l-63.244102 0z" fill-rule="evenodd"/><path fill="#000000" d="m127.99311 55.018215l0 -7.625l1.0000076 0l0 3.125l3.96875 0l0 -3.125l1.015625 0l0 7.625l-1.015625 0l0 -3.59375l-3.96875 0l0 3.59375l-1.0000076 0zm9.60186 0l0 -6.734375l-2.515625 0l0 -0.890625l6.046875 0l0 0.890625l-2.53125 0l0 6.734375l-1.0 0zm6.509262 0l0 -6.734375l-2.515625 0l0 -0.890625l6.046875 0l0 0.890625l-2.53125 0l0 6.734375l-1.0 0zm4.571762 0l0 -7.625l2.875 0q0.75 0 1.15625 0.0625q0.5625 0.09375 0.9375 0.359375q0.390625 0.265625 0.609375 0.75q0.234375 0.46875 0.234375 1.03125q0 0.96875 -0.625 1.65625q-0.609375 0.671875 -2.234375 0.671875l-1.953125 0l0 3.09375l-1.0 0zm1.0 -4.0l1.96875 0q0.984375 0 1.390625 -0.359375q0.421875 -0.375 0.421875 -1.03125q0 -0.484375 -0.25 -0.8125q-0.234375 -0.34375 -0.640625 -0.453125q-0.25 -0.078125 -0.9375 -0.078125l-1.953125 0l0 2.734375z" fill-rule="nonzero"/><path fill="#d9d9d9" d="m109.42126 63.026245l63.244102 0l0 18.456696l-63.244102 0z" fill-rule="evenodd"/><path stroke="#ffffff" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m109.42126 63.026245l63.244102 0l0 18.456696l-63.244102 0z" fill-rule="evenodd"/><path fill="#000000" d="m123.959114 75.99459l0 -5.53125l0.84375 0l0 0.84375q0.328125 -0.59375 0.59375 -0.78125q0.28125 -0.1875 0.609375 -0.1875q0.46875 0 0.953125 0.3125l-0.3125 0.859375q-0.34375 -0.203125 -0.6875 -0.203125q-0.3125 0 -0.5625 0.1875q-0.234375 0.1875 -0.34375 0.515625q-0.15625 0.5 -0.15625 1.09375l0 2.890625l-0.9375 0zm7.345497 -1.78125l0.96875 0.125q-0.234375 0.84375 -0.859375 1.3125q-0.609375 0.46875 -1.578125 0.46875q-1.203125 0 -1.9218826 -0.75q-0.703125 -0.75 -0.703125 -2.09375q0 -1.390625 0.71875 -2.15625q0.7187576 -0.78125 1.8593826 -0.78125q1.109375 0 1.8125 0.765625q0.703125 0.75 0.703125 2.125q0 0.078125 0 0.234375l-4.125 0q0.046875 0.921875 0.515625 1.40625q0.46875 0.484375 1.15625 0.484375q0.515625 0 0.875 -0.265625q0.359375 -0.28125 0.578125 -0.875zm-3.078125 -1.515625l3.09375 0q-0.0625 -0.6875 -0.359375 -1.046875q-0.453125 -0.53125 -1.15625 -0.53125q-0.640625 0 -1.09375 0.4375q-0.4375 0.421875 -0.484375 1.140625zm8.738998 5.421875l0 -2.71875q-0.21875 0.3125 -0.609375 0.515625q-0.390625 0.203125 -0.828125 0.203125q-0.984375 0 -1.703125 -0.78125q-0.703125 -0.796875 -0.703125 -2.15625q0 -0.828125 0.28125 -1.484375q0.296875 -0.671875 0.84375 -1.015625q0.546875 -0.34375 1.203125 -0.34375q1.03125 0 1.609375 0.875l0 -0.75l0.84375 0l0 7.65625l-0.9375 0zm-2.875 -4.90625q0 1.0625 0.4375 1.609375q0.453125 0.53125 1.078125 0.53125q0.59375 0 1.015625 -0.5q0.4375 -0.515625 0.4375 -1.546875q0 -1.109375 -0.453125 -1.65625q-0.453125 -0.5625 -1.0625 -0.5625q-0.609375 0 -1.03125 0.515625q-0.421875 0.515625 -0.421875 1.609375zm8.910873 2.78125l0 -0.8125q-0.65625 0.9375 -1.75 0.9375q-0.5 0 -0.921875 -0.1875q-0.421875 -0.1875 -0.625 -0.46875q-0.203125 -0.28125 -0.296875 -0.703125q-0.046875 -0.265625 -0.046875 -0.875l0 -3.421875l0.9375 0l0 3.0625q0 0.734375 0.046875 1.0q0.09375 0.359375 0.375 0.578125q0.296875 0.203125 0.703125 0.203125q0.421875 0 0.796875 -0.203125q0.375 -0.21875 0.515625 -0.59375q0.15625 -0.375 0.15625 -1.078125l0 -2.96875l0.9375 0l0 5.53125l-0.828125 0zm6.0827484 -1.78125l0.96875 0.125q-0.234375 0.84375 -0.859375 1.3125q-0.609375 0.46875 -1.578125 0.46875q-1.203125 0 -1.921875 -0.75q-0.703125 -0.75 -0.703125 -2.09375q0 -1.390625 0.71875 -2.15625q0.71875 -0.78125 1.859375 -0.78125q1.109375 0 1.8125 0.765625q0.703125 0.75 0.703125 2.125q0 0.078125 0 0.234375l-4.125 0q0.046875 0.921875 0.515625 1.40625q0.46875 0.484375 1.15625 0.484375q0.515625 0 0.875 -0.265625q0.359375 -0.28125 0.578125 -0.875zm-3.078125 -1.515625l3.09375 0q-0.0625 -0.6875 -0.359375 -1.046875q-0.453125 -0.53125 -1.15625 -0.53125q-0.640625 0 -1.09375 0.4375q-0.4375 0.421875 -0.484375 1.140625zm4.8483734 1.640625l0.921875 -0.140625q0.078125 0.5625 0.4375 0.859375q0.359375 0.296875 1.0 0.296875q0.640625 0 0.953125 -0.265625q0.3125 -0.265625 0.3125 -0.625q0 -0.3125 -0.28125 -0.5q-0.1875 -0.125 -0.953125 -0.3125q-1.03125 -0.265625 -1.4375 -0.453125q-0.390625 -0.1875 -0.59375 -0.515625q-0.203125 -0.34375 -0.203125 -0.75q0 -0.359375 0.171875 -0.671875q0.171875 -0.328125 0.453125 -0.53125q0.21875 -0.15625 0.59375 -0.265625q0.390625 -0.125 0.8125 -0.125q0.65625 0 1.140625 0.1875q0.5 0.1875 0.734375 0.515625q0.234375 0.3125 0.3125 0.859375l-0.90625 0.125q-0.0625 -0.4375 -0.375 -0.671875q-0.296875 -0.234375 -0.828125 -0.234375q-0.65625 0 -0.9375 0.21875q-0.265625 0.203125 -0.265625 0.484375q0 0.1875 0.109375 0.328125q0.125 0.15625 0.359375 0.25q0.140625 0.0625 0.828125 0.25q1.0 0.265625 1.390625 0.4375q0.390625 0.15625 0.609375 0.484375q0.234375 0.3125 0.234375 0.796875q0 0.46875 -0.28125 0.890625q-0.265625 0.40625 -0.78125 0.640625q-0.515625 0.21875 -1.171875 0.21875q-1.078125 0 -1.640625 -0.4375q-0.5625 -0.453125 -0.71875 -1.34375zm7.75 0.8125l0.125 0.828125q-0.390625 0.09375 -0.703125 0.09375q-0.5 0 -0.78125 -0.15625q-0.28125 -0.171875 -0.40625 -0.4375q-0.109375 -0.265625 -0.109375 -1.109375l0 -3.171875l-0.6875 0l0 -0.734375l0.6875 0l0 -1.359375l0.9375 -0.5625l0 1.921875l0.9375 0l0 0.734375l-0.9375 0l0 3.234375q0 0.390625 0.046875 0.515625q0.046875 0.109375 0.15625 0.1875q0.109375 0.0625 0.328125 0.0625q0.15625 0 0.40625 -0.046875z" fill-rule="nonzero"/><path fill="#000000" fill-opacity="0.0" d="m467.5538 61.769028l116.75589 -0.8503914" fill-rule="evenodd"/><path stroke="#000000" stroke-width="2.0" stroke-linejoin="round" stroke-linecap="butt" d="m467.5538 61.769028l104.756256 -0.76299286" fill-rule="evenodd"/><path fill="#000000" stroke="#000000" stroke-width="2.0" stroke-linecap="butt" d="m572.3341 64.30941l9.05188 -3.3694801l-9.100037 -3.2372704z" fill-rule="evenodd"/><path fill="#d9d9d9" d="m473.99738 52.540684l88.8819 0l0 18.456692l-88.8819 0z" fill-rule="evenodd"/><path stroke="#ffffff" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m473.99738 52.540684l88.8819 0l0 18.456692l-88.8819 0z" fill-rule="evenodd"/><path fill="#000000" d="m490.55234 65.509026l0 -7.624996l3.375 0q1.015625 0 1.546875 0.203125q0.53125 0.203125 0.84375 0.734375q0.328125 0.515625 0.328125 1.140625q0 0.8125 -0.53125 1.375q-0.515625 0.546875 -1.625 0.703125q0.40625 0.1875 0.609375 0.375q0.453125 0.40625 0.84375 1.015625l1.328125 2.0781212l-1.265625 0l-1.015625 -1.5937462q-0.4375 -0.6875 -0.734375 -1.046875q-0.28125 -0.359375 -0.515625 -0.5q-0.21875 -0.15625 -0.453125 -0.203125q-0.171875 -0.046875 -0.5625 -0.046875l-1.171875 0l0 3.3906212l-1.0 0zm1.0 -4.265621l2.171875 0q0.6875 0 1.078125 -0.140625q0.390625 -0.140625 0.59375 -0.453125q0.203125 -0.3125 0.203125 -0.6875q0 -0.546875 -0.390625 -0.890625q-0.390625 -0.34375 -1.234375 -0.34375l-2.421875 0l0 2.515625zm6.2112427 1.5q0 -1.53125 0.84375 -2.265625q0.71875 -0.625 1.734375 -0.625q1.140625 0 1.859375 0.75q0.734375 0.75 0.734375 2.0625q0 1.0625 -0.328125 1.6874962q-0.3125 0.609375 -0.921875 0.953125q-0.609375 0.328125 -1.34375 0.328125q-1.15625 0 -1.875 -0.734375q-0.703125 -0.75 -0.703125 -2.1562462zm0.953125 0q0 1.0625 0.46875 1.5937462q0.46875 0.53125 1.15625 0.53125q0.703125 0 1.15625 -0.53125q0.46875 -0.5312462 0.46875 -1.6249962q0 -1.015625 -0.46875 -1.546875q-0.453125 -0.53125 -1.15625 -0.53125q-0.6875 0 -1.15625 0.53125q-0.46875 0.515625 -0.46875 1.578125zm8.942108 2.7656212l0 -0.8125q-0.65625 0.9375 -1.75 0.9375q-0.5 0 -0.921875 -0.1875q-0.421875 -0.1875 -0.625 -0.46875q-0.203125 -0.28125 -0.296875 -0.703125q-0.046875 -0.265625 -0.046875 -0.8749962l0 -3.421875l0.9375 0l0 3.0625q0 0.734375 0.046875 0.9999962q0.09375 0.359375 0.375 0.578125q0.296875 0.203125 0.703125 0.203125q0.421875 0 0.796875 -0.203125q0.375 -0.21875 0.515625 -0.59375q0.15625 -0.3749962 0.15625 -1.0781212l0 -2.96875l0.9375 0l0 5.531246l-0.828125 0zm4.3483887 -0.84375l0.125 0.828125q-0.390625 0.09375 -0.703125 0.09375q-0.5 0 -0.78125 -0.15625q-0.28125 -0.171875 -0.40625 -0.4375q-0.109375 -0.265625 -0.109375 -1.1093712l0 -3.171875l-0.6875 0l0 -0.734375l0.6875 0l0 -1.359375l0.9375 -0.5625l0 1.921875l0.9375 0l0 0.734375l-0.9375 0l0 3.234375q0 0.3906212 0.046875 0.5156212q0.046875 0.109375 0.15625 0.1875q0.109375 0.0625 0.328125 0.0625q0.15625 0 0.40625 -0.046875zm0.9137573 -5.703121l0 -1.078125l0.9375 0l0 1.078125l-0.9375 0zm0 6.546871l0 -5.531246l0.9375 0l0 5.531246l-0.9375 0zm2.3674927 0l0 -5.531246l0.84375 0l0 0.796875q0.609375 -0.921875 1.75 -0.921875q0.5 0 0.921875 0.1875q0.421875 0.171875 0.625 0.46875q0.21875 0.296875 0.296875 0.6875q0.046875 0.265625 0.046875 0.921875l0 3.3906212l-0.9375 0l0 -3.3593712q0 -0.578125 -0.109375 -0.859375q-0.109375 -0.28125 -0.390625 -0.453125q-0.265625 -0.171875 -0.640625 -0.171875q-0.59375 0 -1.03125 0.390625q-0.4375 0.375 -0.4375 1.4375l0 3.0156212l-0.9375 0zm5.7546387 0.453125l0.90625 0.140625q0.0625 0.421875 0.328125 0.609375q0.34375 0.265625 0.9375 0.265625q0.65625 0 1.0 -0.265625q0.359375 -0.25 0.484375 -0.71875q0.078125 -0.296875 0.078125 -1.203125q-0.625 0.71875 -1.53125 0.71875q-1.140625 0 -1.765625 -0.8125q-0.625 -0.8281212 -0.625 -1.9843712q0 -0.78125 0.28125 -1.453125q0.296875 -0.671875 0.828125 -1.03125q0.546875 -0.375 1.28125 -0.375q0.984375 0 1.609375 0.796875l0 -0.671875l0.875 0l0 4.781246q0 1.296875 -0.265625 1.828125q-0.265625 0.546875 -0.84375 0.859375q-0.5625 0.3125 -1.390625 0.3125q-0.984375 0 -1.59375 -0.453125q-0.609375 -0.4375 -0.59375 -1.34375zm0.78125 -3.3124962q0 1.078125 0.421875 1.5781212q0.4375 0.5 1.078125 0.5q0.65625 0 1.09375 -0.484375q0.4375 -0.4999962 0.4375 -1.5624962q0 -1.015625 -0.453125 -1.53125q-0.453125 -0.515625 -1.09375 -0.515625q-0.609375 0 -1.046875 0.515625q-0.4375 0.5 -0.4375 1.5zm8.262146 2.8593712l0 -5.531246l0.84375 0l0 0.84375q0.328125 -0.59375 0.59375 -0.78125q0.28125 -0.1875 0.609375 -0.1875q0.46875 0 0.953125 0.3125l-0.3125 0.859375q-0.34375 -0.203125 -0.6875 -0.203125q-0.3125 0 -0.5625 0.1875q-0.234375 0.1875 -0.34375 0.515625q-0.15625 0.5 -0.15625 1.09375l0 2.8906212l-0.9375 0zm7.189209 0l0 -0.8125q-0.65625 0.9375 -1.75 0.9375q-0.5 0 -0.921875 -0.1875q-0.421875 -0.1875 -0.625 -0.46875q-0.203125 -0.28125 -0.296875 -0.703125q-0.046875 -0.265625 -0.046875 -0.8749962l0 -3.421875l0.9375 0l0 3.0625q0 0.734375 0.046875 0.9999962q0.09375 0.359375 0.375 0.578125q0.296875 0.203125 0.703125 0.203125q0.421875 0 0.796875 -0.203125q0.375 -0.21875 0.515625 -0.59375q0.15625 -0.3749962 0.15625 -1.0781212l0 -2.96875l0.9375 0l0 5.531246l-0.828125 0zm2.2858887 0l0 -7.624996l0.9375 0l0 7.624996l-0.9375 0zm6.1643066 -1.7812462l0.96875 0.125q-0.234375 0.8437462 -0.859375 1.3124962q-0.609375 0.46875 -1.578125 0.46875q-1.203125 0 -1.921875 -0.75q-0.703125 -0.75 -0.703125 -2.0937462q0 -1.390625 0.71875 -2.15625q0.71875 -0.78125 1.859375 -0.78125q1.109375 0 1.8125 0.765625q0.703125 0.75 0.703125 2.125q0 0.078125 0 0.234375l-4.125 0q0.046875 0.921875 0.515625 1.4062462q0.46875 0.484375 1.15625 0.484375q0.515625 0 0.875 -0.265625q0.359375 -0.28125 0.578125 -0.8749962zm-3.078125 -1.515625l3.09375 0q-0.0625 -0.6875 -0.359375 -1.046875q-0.453125 -0.53125 -1.15625 -0.53125q-0.640625 0 -1.09375 0.4375q-0.4375 0.421875 -0.484375 1.140625z" fill-rule="nonzero"/><path fill="#1155cc" d="m584.3202 35.097115l100.409485 0l0 51.65354l-100.409485 0z" fill-rule="evenodd"/><path stroke="#1155cc" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m584.3202 35.097115l100.409485 0l0 51.65354l-100.409485 0z" fill-rule="evenodd"/><path fill="#ffffff" d="m602.00183 63.500134l2.625 -0.25q0.234375 1.3124962 0.953125 1.9374962q0.734375 0.609375 1.96875 0.609375q1.296875 0 1.953125 -0.546875q0.671875 -0.546875 0.671875 -1.2812462q0 -0.484375 -0.28125 -0.8125q-0.28125 -0.328125 -0.96875 -0.578125q-0.484375 -0.15625 -2.171875 -0.578125q-2.15625 -0.546875 -3.03125 -1.328125q-1.234375 -1.09375 -1.234375 -2.6875q0 -1.015625 0.578125 -1.90625q0.578125 -0.890625 1.65625 -1.34375q1.09375 -0.46875 2.640625 -0.46875q2.515625 0 3.78125 1.109375q1.28125 1.09375 1.34375 2.9375l-2.703125 0.109375q-0.171875 -1.03125 -0.75 -1.46875q-0.5625 -0.453125 -1.703125 -0.453125q-1.171875 0 -1.84375 0.46875q-0.421875 0.3125 -0.421875 0.84375q0 0.46875 0.40625 0.796875q0.5 0.4375 2.46875 0.90625q1.96875 0.453125 2.90625 0.953125q0.953125 0.5 1.484375 1.359375q0.53125 0.859375 0.53125 2.125q0 1.1562462 -0.640625 2.1562462q-0.640625 1.0 -1.8125 1.484375q-1.15625 0.484375 -2.890625 0.484375q-2.53125 0 -3.890625 -1.171875q-1.359375 -1.171875 -1.625 -3.4062462zm18.709167 1.2656212l2.546875 0.421875q-0.484375 1.40625 -1.546875 2.140625q-1.0625 0.734375 -2.65625 0.734375q-2.515625 0 -3.734375 -1.65625q-0.953125 -1.3125 -0.953125 -3.3281212q0 -2.40625 1.25 -3.765625q1.265625 -1.359375 3.1875 -1.359375q2.15625 0 3.40625 1.421875q1.25 1.421875 1.1875 4.375l-6.40625 0q0.03125 1.1406212 0.609375 1.7812462q0.59375 0.625 1.484375 0.625q0.59375 0 1.0 -0.328125q0.421875 -0.328125 0.625 -1.0625zm0.15625 -2.5937462q-0.03125 -1.109375 -0.578125 -1.6875q-0.546875 -0.578125 -1.328125 -0.578125q-0.84375 0 -1.390625 0.609375q-0.546875 0.609375 -0.53125 1.65625l3.828125 0zm7.0788574 5.671871l-2.5625 0l0 -9.671871l2.375 0l0 1.375q0.609375 -0.984375 1.09375 -1.28125q0.484375 -0.3125 1.109375 -0.3125q0.875 0 1.6875 0.484375l-0.796875 2.234375q-0.640625 -0.421875 -1.203125 -0.421875q-0.53125 0 -0.90625 0.296875q-0.375 0.296875 -0.59375 1.078125q-0.203125 0.765625 -0.203125 3.2343712l0 2.984375zm7.463379 0l-3.90625 -9.671871l2.6875 0l1.828125 4.9375l0.53125 1.6406212q0.203125 -0.625 0.265625 -0.8281212q0.125 -0.40625 0.265625 -0.8125l1.84375 -4.9375l2.625 0l-3.84375 9.671871l-2.296875 0zm7.7194824 -10.984371l0 -2.375l2.5625 0l0 2.375l-2.5625 0zm0 10.984371l0 -9.671871l2.5625 0l0 9.671871l-2.5625 0zm13.620789 -6.812496l-2.53125 0.453125q-0.125 -0.75 -0.578125 -1.125q-0.453125 -0.390625 -1.171875 -0.390625q-0.953125 0 -1.53125 0.65625q-0.5625 0.65625 -0.5625 2.203125q0 1.7343712 0.578125 2.4374962q0.578125 0.703125 1.546875 0.703125q0.734375 0 1.203125 -0.40625q0.46875 -0.421875 0.65625 -1.421875l2.515625 0.421875q-0.390625 1.734375 -1.515625 2.625q-1.109375 0.875 -2.96875 0.875q-2.125 0 -3.390625 -1.328125q-1.25 -1.34375 -1.25 -3.7187462q0 -2.390625 1.265625 -3.71875q1.265625 -1.34375 3.421875 -1.34375q1.765625 0 2.796875 0.765625q1.046875 0.75 1.515625 2.3125zm7.5319824 3.7343712l2.546875 0.421875q-0.484375 1.40625 -1.546875 2.140625q-1.0625 0.734375 -2.65625 0.734375q-2.515625 0 -3.734375 -1.65625q-0.953125 -1.3125 -0.953125 -3.3281212q0 -2.40625 1.25 -3.765625q1.265625 -1.359375 3.1875 -1.359375q2.15625 0 3.40625 1.421875q1.25 1.421875 1.1875 4.375l-6.40625 0q0.03125 1.1406212 0.609375 1.7812462q0.59375 0.625 1.484375 0.625q0.59375 0 1.0 -0.328125q0.421875 -0.328125 0.625 -1.0625zm0.15625 -2.5937462q-0.03125 -1.109375 -0.578125 -1.6875q-0.546875 -0.578125 -1.328125 -0.578125q-0.84375 0 -1.390625 0.609375q-0.546875 0.609375 -0.53125 1.65625l3.828125 0z" fill-rule="nonzero"/><path fill="#000000" fill-opacity="0.0" d="m684.7297 60.923885l56.661377 -26.834648" fill-rule="evenodd"/><path stroke="#000000" stroke-width="2.0" stroke-linejoin="round" stroke-linecap="butt" d="m684.7297 60.923885l45.816162 -21.698391" fill-rule="evenodd"/><path fill="#000000" stroke="#000000" stroke-width="2.0" stroke-linecap="butt" d="m731.95984 42.211063l6.7888184 -6.870373l-9.61676 0.8992386z" fill-rule="evenodd"/><path fill="#1155cc" d="m741.3963 15.661417l63.24414 0l0 36.88189l-63.24414 0z" fill-rule="evenodd"/><path stroke="#1155cc" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m741.3963 15.661417l63.24414 0l0 36.88189l-63.24414 0z" fill-rule="evenodd"/><path fill="#ffffff" d="m756.76 41.02236l0 -13.359377l4.328125 0q2.453125 0 3.203125 0.203125q1.140625 0.296875 1.921875 1.3125q0.78125 1.0 0.78125 2.59375q0 1.2343769 -0.453125 2.078127q-0.453125 0.828125 -1.140625 1.3125q-0.6875 0.46875 -1.390625 0.625q-0.96875 0.203125 -2.796875 0.203125l-1.765625 0l0 5.03125l-2.6875 0zm2.6875 -11.093752l0 3.781252l1.484375 0q1.59375 0 2.125 -0.203125q0.546875 -0.203125 0.84375 -0.65625q0.3125 -0.453125 0.3125 -1.0312519q0 -0.734375 -0.4375 -1.203125q-0.421875 -0.484375 -1.078125 -0.59375q-0.484375 -0.09375 -1.9375 -0.09375l-1.3125 0zm9.1466675 6.125002q0 -1.28125 0.625 -2.46875q0.625 -1.203125 1.78125 -1.8281269q1.15625 -0.625 2.578125 -0.625q2.1875 0 3.59375 1.4218769q1.40625 1.421875 1.40625 3.609375q0 2.1875 -1.421875 3.640625q-1.421875 1.4375 -3.5625 1.4375q-1.328125 0 -2.546875 -0.59375q-1.203125 -0.609375 -1.828125 -1.765625q-0.625 -1.171875 -0.625 -2.828125zm2.625 0.125q0 1.453125 0.671875 2.21875q0.6875 0.75 1.6875 0.75q1.0 0 1.671875 -0.75q0.6875 -0.765625 0.6875 -2.234375q0 -1.421875 -0.6875 -2.1875q-0.671875 -0.765625 -1.671875 -0.765625q-1.0 0 -1.6875 0.765625q-0.671875 0.765625 -0.671875 2.203125zm18.239746 4.84375l-2.390625 0l0 -1.421875q-0.59375 0.828125 -1.40625 1.234375q-0.796875 0.40625 -1.609375 0.40625q-1.671875 0 -2.859375 -1.34375q-1.1875 -1.34375 -1.1875 -3.75q0 -2.453125 1.15625 -3.734375q1.15625 -1.2812519 2.921875 -1.2812519q1.625 0 2.8125 1.3437519l0 -4.812502l2.5625 0l0 13.359377zm-6.84375 -5.046875q0 1.546875 0.4375 2.234375q0.609375 1.015625 1.71875 1.015625q0.890625 0 1.5 -0.75q0.625 -0.765625 0.625 -2.25q0 -1.671875 -0.609375 -2.40625q-0.59375 -0.734375 -1.53125 -0.734375q-0.90625 0 -1.53125 0.734375q-0.609375 0.71875 -0.609375 2.15625z" fill-rule="nonzero"/><path fill="#1155cc" d="m741.3963 63.026245l63.24414 0l0 36.88189l-63.24414 0z" fill-rule="evenodd"/><path stroke="#1155cc" stroke-width="1.0" stroke-linejoin="round" stroke-linecap="butt" d="m741.3963 63.026245l63.24414 0l0 36.88189l-63.24414 0z" fill-rule="evenodd"/><path fill="#ffffff" d="m756.76 88.38719l0 -13.359375l4.328125 0q2.453125 0 3.203125 0.203125q1.140625 0.296875 1.921875 1.3125q0.78125 1.0 0.78125 2.59375q0 1.234375 -0.453125 2.078125q-0.453125 0.828125 -1.140625 1.3125q-0.6875 0.46875 -1.390625 0.625q-0.96875 0.203125 -2.796875 0.203125l-1.765625 0l0 5.03125l-2.6875 0zm2.6875 -11.09375l0 3.78125l1.484375 0q1.59375 0 2.125 -0.203125q0.546875 -0.203125 0.84375 -0.65625q0.3125 -0.453125 0.3125 -1.03125q0 -0.734375 -0.4375 -1.203125q-0.421875 -0.484375 -1.078125 -0.59375q-0.484375 -0.09375 -1.9375 -0.09375l-1.3125 0zm9.1466675 6.125q0 -1.28125 0.625 -2.46875q0.625 -1.203125 1.78125 -1.828125q1.15625 -0.625 2.578125 -0.625q2.1875 0 3.59375 1.421875q1.40625 1.421875 1.40625 3.609375q0 2.1875 -1.421875 3.640625q-1.421875 1.4375 -3.5625 1.4375q-1.328125 0 -2.546875 -0.59375q-1.203125 -0.609375 -1.828125 -1.765625q-0.625 -1.171875 -0.625 -2.828125zm2.625 0.125q0 1.453125 0.671875 2.21875q0.6875 0.75 1.6875 0.75q1.0 0 1.671875 -0.75q0.6875 -0.765625 0.6875 -2.234375q0 -1.421875 -0.6875 -2.1875q-0.671875 -0.765625 -1.671875 -0.765625q-1.0 0 -1.6875 0.765625q-0.671875 0.765625 -0.671875 2.203125zm18.239746 4.84375l-2.390625 0l0 -1.421875q-0.59375 0.828125 -1.40625 1.234375q-0.796875 0.40625 -1.609375 0.40625q-1.671875 0 -2.859375 -1.34375q-1.1875 -1.34375 -1.1875 -3.75q0 -2.453125 1.15625 -3.734375q1.15625 -1.28125 2.921875 -1.28125q1.625 0 2.8125 1.34375l0 -4.8125l2.5625 0l0 13.359375zm-6.84375 -5.046875q0 1.546875 0.4375 2.234375q0.609375 1.015625 1.71875 1.015625q0.890625 0 1.5 -0.75q0.625 -0.765625 0.625 -2.25q0 -1.671875 -0.609375 -2.40625q-0.59375 -0.734375 -1.53125 -0.734375q-0.90625 0 -1.53125 0.734375q-0.609375 0.71875 -0.609375 2.15625z" fill-rule="nonzero"/><path fill="#000000" fill-opacity="0.0" d="m684.7297 60.923885l56.661377 20.535435" fill-rule="evenodd"/><path stroke="#000000" stroke-width="2.0" stroke-linejoin="round" stroke-linecap="butt" d="m684.7297 60.923885l45.379517 16.446606" fill-rule="evenodd"/><path fill="#000000" stroke="#000000" stroke-width="2.0" stroke-linecap="butt" d="m728.9836 80.47627l9.65863 -0.013198853l-7.4074097 -6.198372z" fill-rule="evenodd"/></g></svg> </ins>  No newline at end of file


Question:
This is a Feature Request What would you like to be High level documentation for Gateway API with links out to https://gateway- This should include the following: new "Gateway API" page between Service and Ingress in This should provide a high level overview of Gateway API, with a focus on content that is unlikely to become stale. This can likely take some content from both https://gateway- and https://gateway- and would help increase the discoverability of Gateway API. the "Ingress" page to note that the API is effectively frozen and all new features are being to Gateway API (with a corresponding link to the page by 1). the section on the "Service" page that references Ingress and Gateway API to give more prominence to Gateway API. Why is this needed With the upcoming graduation of Gateway API to GA (x-ref ), it's past time to cover it in Kubernetes docs. Comments I'd previously talked with about this at a high level, but definitely let me know if you've got any suggestions for how we can scope/adjust this differently. /sig network /assign
checkout the gateway API in the section and look out for the bugs
/triage accepted /priority important-longterm
How about adding a short link eg that redirects (302) to https://gateway- ?
I'd put Service, Ingress, then Gateway - and leave it in that order until most new clusters include Gateway as an add-on.
Also see

<ins> --- title: Gateway API id: gateway-api date: 2023-10-19 full_link: /docs/concepts/services-networking/gateway/ short_description: > An API for modeling service networking in Kubernetes. aka: tags: - networking - architecture - extension --- A family of API kinds for modeling service networking in Kubernetes. <!--more--> Gateway API provides a family of extensible, role-oriented, protocol-aware API kinds for modeling service networking in Kubernetes. </ins>


Question:
This is a Feature Request <!-- Please only use this template for submitting feature/enhancement requests --<!-- See for guidance on writing an actionable issue description. --What would you like to be Within , add a new and brief page that has an overview of autoscaling. This means adding a new Markdown file. The new page should mention: (aim to make it clear that this is a thing that autoscales nodes, not Pods) (aim to make it clear that this is a thing that autoscales nodes, not Pods) Optionally, also mention: - marked as third party software as it's a separate project from Kubernetes - marked as third party software as it's a separate project from Kubernetes For this issue, all that's needed is a few words about each item. Later PRs could add more detail. Why is this needed our documentation on autoscaling is incomplete readers would like to have their workloads automatically sized to match demand. Comments This helps with /language en /kind feature PS. I didn't mark this as a good first issue. However, if you're looking for something to work on, and haven't already contributed to Kubernetes' docs, there may be some issues available marked as good first issues. See this . If you have already worked on an issue, please leave these for people who are new to contributing.
/triage accepted /priority important-longterm /help
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /triage accepted /priority important-longterm /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I would like to work on this issue /assign
Should I Make a new file inside Workloads!! In 1 PR I will be creating 2 topics like (HorizontalPodAutoscaler, VerticalPodAutoscaler) and will it be sufficient, because It will be taking 2-3 PR's!
For advice on how to contribute to the Kubernetes docs, please ask in #sig-docs on the Kubernetes Slack workspace. If you're not already a member of that workspace, you can visit for an invitation. Anyone is welcome to work on this issue. it's slightly unhelpful to assign an issue to yourself and also to ask questions about how to work on it; ideally, people only assign an issue once they are confident that they know how to resolve it.
Our no cookie-licking policy means that anyone is welcome to open a PR that closes an open issue. It's still friendly to reach out to people who might already be working on the same thing. See for an explanation of the jargon.
Thanks I was just asking it. I can fix this issue!
Hey This has been pending for more than a month. So, May I go ahead and create a PR? I have a basic markdown structure for file is ready and I can add more details as needed after raising a PR.
PRs are welcome, and because this project is open source, nobody needs permission.
In light of our policy on cookie licking: /unassign I think we'll make progress better without any assignee. If anyone wants to share that they have started work, feel free to open a draft pull request that mentions this issue.

 --- title: Autoscaling Workloads description: >- With autoscaling, you can automatically update your workloads in one way or another. This allows your cluster to react to changes in resource demand more elastically and efficiently. content_type: concept weight: 40 ---  In Kubernetes, you can _scale_ a workload depending on the current demand of resources. This allows your cluster to react to changes in resource demand more elastically and efficiently. When you scale a workload, you can either increase or decrease the number of replicas managed by the workload, or adjust the resources available to the replicas in-place. The first approach is referred to as _horizontal scaling_, while the second is referred to as _vertical scaling_. There are manual and automatic ways to scale your workloads, depending on your use case.  ## Scaling workloads manually Kubernetes supports _manual scaling_ of workloads. Horizontal scaling can be done using the `kubectl` CLI. For vertical scaling, you need to _patch_ the resource definition of your workload. See below for examples of both strategies. - **Horizontal scaling**: [Running multiple instances of your app](/docs/tutorials/kubernetes-basics/scale/scale-intro/) - **Vertical scaling**: [Resizing CPU and memory resources assigned to containers](/docs/tasks/configure-pod-container/resize-container-resources) ## Scaling workloads automatically Kubernetes also supports _automatic scaling_ of workloads, which is the focus of this page. The concept of _Autoscaling_ in Kubernetes refers to the ability to automatically update an object that manages a set of Pods (for example a {{< glossary_tooltip text="Deployment" term_id="deployment" >}}. ### Scaling workloads horizontally In Kubernetes, you can automatically scale a workload horizontally using a _HorizontalPodAutoscaler_ (HPA). It is implemented as a Kubernetes API resource and a {{< glossary_tooltip text="controller" term_id="controller" >}} and periodically adjusts the number of {{< glossary_tooltip text="replicas" term_id="replica" >}} in a workload to match observed resource utilization such as CPU or memory usage. There is a [walkthrough tutorial](/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough) of configuring a HorizontalPodAutoscaler for a Deployment. ### Scaling workloads vertically {{< feature-state for_k8s_version="v1.25" state="stable" >}} You can automatically scale a workload vertically using a _VerticalPodAutoscaler_ (VPA). Different to the HPA, the VPA doesn't come with Kubernetes by default, but is a separate project that can be found [on GitHub](https://github.com/kubernetes/autoscaler/tree/9f87b78df0f1d6e142234bb32e8acbd71295585a/vertical-pod-autoscaler). Once installed, it allows you to create {{< glossary_tooltip text="CustomResourceDefinitions" term_id="customresourcedefinition" >}} (CRDs) for your workloads which define _how_ and _when_ to scale the resources of the managed replicas. {{< note >}} You will need to have the [Metrics Server](https://github.com/kubernetes-sigs/metrics-server) installed to your cluster for the HPA to work. {{< /note >}} At the moment, the VPA can operate in four different modes: {{
}} Mode | Description :----|:----------- `Auto` | Currently `Recreate`, might change to in-place updates in the future `Recreate` | The VPA assigns resource requests on pod creation as well as updates them on existing pods by evicting them when the requested resources differ significantly from the new recommendation `Initial` | The VPA only assigns resource requests on pod creation and never changes them later. `Off` | The VPA does not automatically change the resource requirements of the pods. The recommendations are calculated and can be inspected in the VPA object. {{
}} #### Requirements for in-place resizing {{< feature-state for_k8s_version="v1.27" state="alpha" >}} Resizing a workload in-place **without** restarting the {{< glossary_tooltip text="Pods" term_id="pod" >}} or its {{< glossary_tooltip text="Containers" term_id="container" >}} requires Kubernetes version 1.27 or later.
 Additionally, the `InPlaceVerticalScaling` feature gate needs to be enabled. {{< feature-gate-description name="InPlacePodVerticalScaling" >}} ### Autoscaling based on cluster size For workloads that need to be scaled based on the size of the cluster (for example `cluster-dns` or other system components), you can use the _Cluster Proportional Autoscaler_.
 Just like the VPA, it is not part of the Kubernetes core, but hosted in its own repository [on GitHub](https://github.com/kubernetes-sigs/cluster-proportional-autoscaler). The Cluster Proportional Autoscaler watches the number of schedulable {{< glossary_tooltip text="nodes" term_id="node" >}} and cores and scales the number of replicas of the target workload accordingly. ### Event driven Autoscaling It is also possible to scale workloads based on events, for example using the [_Kubernetes Event Driven Autoscaler_ (**KEDA**)](https://keda.sh/). KEDA is a CNCF graduated enabling you to scale your workloads based on the number of events to be processed, for example the amount of messages in a queue. There exists a wide range of adapters for different event sources to choose from. ### Autoscaling based on schedules Another strategy for scaling your workloads is to **schedule** the scaling operations, for example in order to reduce resource consumption during off-peak hours. Similar to event driven autoscaling, such behavior can be achieved using KEDA in conjunction with its [`Cron` scaler](https://keda.sh/docs/2.13/scalers/cron/). The `Cron` scaler allows you to define schedules (and time zones) for scaling your workloads in or out. ## Scaling cluster infrastructure If scaling workloads isn't enough to meet your needs, you can also scale your cluster infrastructure itself. Scaling the cluster infrastructure normally means adding or removing {{< glossary_tooltip text="nodes" term_id="node" >}}. This can be done using one of two available autoscalers: - [**Cluster Autoscaler**](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler) - [**Karpenter**](https://github.com/kubernetes-sigs/karpenter?tab=readme-ov-file) Both scalers work by watching for pods marked as _unschedulable_ or _underutilized_ nodes and then adding or removing nodes as needed. ## {{% heading "whatsnext" %}} - Learn more about scaling horizontally - [Scale a StatefulSet](/docs/tasks/run-application/scale-stateful-set/) - [HorizontalPodAutoscaler Walkthrough](/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/) - [Resize Container Resources In-Place](/docs/tasks/configure-pod-container/resize-container-resources/) - [Autoscale the DNS Service in a Cluster](/docs/tasks/administer-cluster/dns-horizontal-autoscaling/)


Question:
This is a Bug Report Problem: The link for in  is incorrect. It is redirecting to but it should redirect to Proposed Solution: should redirect to instead of
/sig docs /kind bug
Page related to the issue:
/language en
I would to like to solve this issue if triage gets accepted.
/triage accepted
/assign
The issue is fixed here:

You can use {{< glossary_tooltip term_id="containerd" text="ContainerD" >}} 1.4.0+ as the container runtime for Kubernetes nodes that run Windows. <del> Learn how to [install ContainerD on a Windows node](/docs/setup/production-environment/container-runtimes/#install-containerd). </del> <ins> Learn how to [install ContainerD on a Windows node](/docs/setup/production-environment/container-runtimes/#containerd). </ins> {{< note >}} There is a [known limitation](/docs/tasks/configure-pod-container/configure-gmsa/#gmsa-limitations) when using GMSA with containerd to access Windows network shares, which requires a


Question:
This is a Bug Report <!-- Thanks for filing an issue! Before submitting, please fill in the following information. --<!-- See for guidance on writing an actionable issue description. --<!--Required Information--Problem: The code block is not being processed correctly, resulting in the page rendering in an unexpected way. <img width="625" alt="Screenshot 2023-12-06 at 16 52 11" src=""Proposed Solution: fix L84 (to yaml) Page to Update: <!--Optional Information (remove the comment tags around information you would like to include)--<!--Kubernetes Version:--<!--Additional Information:--
This issue is currently awaiting triage. SIG Docs takes a lead on issue triage for this website, but any Kubernetes member can accept issues by applying the label. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/language ja /assign
/area localization

- `kubectl edit cm kubelet-config-x.yy -n kube-system`を呼び出します(`x.yy`はKubernetesのバージョンに置き換えてください)。 - 既存の`cgroupDriver`の値を修正するか、以下のような新しいフィールドを追加します。 <del> ``yaml </del> <ins> ```yaml </ins> cgroupDriver: systemd ```


Question:
Please note: This issue is not suitable for new contributors since you need some familiarity with how Kubernetes works. The Kubernetes website currently lacks a guide on creating Windows/Linux nodes. See for more details. SIG Windows opted to remove the guide that did exist for creating Windows nodes, in PR Write a new task page about creating nodes for all operating systems. This could, potentially, be in a tabbed format as mentions in the issue referenced above. We would also need the entry reflected on the creates the Linux task page by moving / refactoring the existing "join node" contents from the "create a cluster" page: creates the Windows task page by taking the content from at the end of the Windows page we add note/disclaimer whatever that says that CNI on Windows needs special handling and we link to the various CNI choices at: adds tabs in the to choose between OS for joining nodes. /priority backlog /help
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : Problem Statement: /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/triage accepted Thank you for filing this
/sig cluster-lifecycle
If you know how to set up new nodes (with, at least, the tool): you can work on this issue. You don't need anyone's permission to open a pull request. All we ask is that you read our and that you abide by the project's code of conduct.
/assign
Relevant to
Are you still working on this issue?
/assign sending PR soon.
/priority important-longterm
Relevant to
/retitle Write new task pages about creating Kubernetes nodes
/cc
Done in PR /close
Closing this issue. <detailsIn response to : Done in PR Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

communicates with). 1. (Recommended) If you have plans to upgrade this single control-plane `kubeadm` cluster <del> to high availability you should specify the `--control-plane-endpoint` to set the shared endpoint for all control-plane nodes. Such an endpoint can be either a DNS name or an IP address of a load-balancer. </del> <ins> to [high availability](/docs/setup/production-environment/tools/kubeadm/high-availability/) you should specify the `--control-plane-endpoint` to set the shared endpoint for all control-plane nodes. Such an endpoint can be either a DNS name or an IP address of a load-balancer. </ins> 1. Choose a Pod network add-on, and verify whether it requires any arguments to be passed to `kubeadm init`. Depending on which third-party provider you choose, you might need to set the `--pod-network-cidr` to


Question:
Please note: This issue is not suitable for new contributors since you need some familiarity with how Kubernetes works. The Kubernetes website currently lacks a guide on creating Windows/Linux nodes. See for more details. SIG Windows opted to remove the guide that did exist for creating Windows nodes, in PR Write a new task page about creating nodes for all operating systems. This could, potentially, be in a tabbed format as mentions in the issue referenced above. We would also need the entry reflected on the creates the Linux task page by moving / refactoring the existing "join node" contents from the "create a cluster" page: creates the Windows task page by taking the content from at the end of the Windows page we add note/disclaimer whatever that says that CNI on Windows needs special handling and we link to the various CNI choices at: adds tabs in the to choose between OS for joining nodes. /priority backlog /help
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : Problem Statement: /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/triage accepted Thank you for filing this
/sig cluster-lifecycle
If you know how to set up new nodes (with, at least, the tool): you can work on this issue. You don't need anyone's permission to open a pull request. All we ask is that you read our and that you abide by the project's code of conduct.
/assign
Relevant to
Are you still working on this issue?
/assign sending PR soon.
/priority important-longterm
Relevant to
/retitle Write new task pages about creating Kubernetes nodes
/cc
Done in PR /close
Closing this issue. <detailsIn response to : Done in PR Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

kubectl apply -f <add-on.yaml> ``` <ins> {{< note >}} Only a few CNI plugins support Windows. More details and setup instructions can be found in [Adding Windows worker nodes](/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes/#network-config). {{< /note >}} </ins> You can install only one Pod network per cluster. Once a Pod network has been installed, you can confirm that it is working by


Question:
Please note: This issue is not suitable for new contributors since you need some familiarity with how Kubernetes works. The Kubernetes website currently lacks a guide on creating Windows/Linux nodes. See for more details. SIG Windows opted to remove the guide that did exist for creating Windows nodes, in PR Write a new task page about creating nodes for all operating systems. This could, potentially, be in a tabbed format as mentions in the issue referenced above. We would also need the entry reflected on the creates the Linux task page by moving / refactoring the existing "join node" contents from the "create a cluster" page: creates the Windows task page by taking the content from at the end of the Windows page we add note/disclaimer whatever that says that CNI on Windows needs special handling and we link to the various CNI choices at: adds tabs in the to choose between OS for joining nodes. /priority backlog /help
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : Problem Statement: /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/triage accepted Thank you for filing this
/sig cluster-lifecycle
If you know how to set up new nodes (with, at least, the tool): you can work on this issue. You don't need anyone's permission to open a pull request. All we ask is that you read our and that you abide by the project's code of conduct.
/assign
Relevant to
Are you still working on this issue?
/assign sending PR soon.
/priority important-longterm
Relevant to
/retitle Write new task pages about creating Kubernetes nodes
/cc
Done in PR /close
Closing this issue. <detailsIn response to : Done in PR Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

kubectl label nodes --all node.kubernetes.io/exclude-from-external-load-balancers- ``` <del> ### Joining your nodes {#join-nodes} The nodes are where your workloads (containers and Pods, etc) run. To add new nodes to your cluster do the following for each machine: * SSH to the machine * Become root (e.g. `sudo su -`) * [Install a runtime](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-runtime) if needed * Run the command that was output by `kubeadm init`. For example: ```bash kubeadm join --token <token> <control-plane-host>:<control-plane-port> --discovery-token-ca-cert-hash sha256:<hash> ``` If you do not have the token, you can get it by running the following command on the control-plane node: ```bash kubeadm token list ``` The output is similar to this: ```console TOKEN                    TTL  EXPIRES              USAGES           DESCRIPTION            EXTRA GROUPS 8ewj1p.9r9hcjoqgajrj4gi  23h  2018-06-12T02:51:28Z authentication,  The default bootstrap  system: signing          token generated by     bootstrappers: 'kubeadm init'.        kubeadm: default-node-token ``` By default, tokens expire after 24 hours. If you are joining a node to the cluster after the current token has expired, you can create a new token by running the following command on the control-plane node: ```bash kubeadm token create ``` The output is similar to this: </del> <ins> ### Adding more control plane nodes </ins> <del> ```console 5didvk.d09sbcov8ph2amjw ``` If you don't have the value of `--discovery-token-ca-cert-hash`, you can get it by running the following command chain on the control-plane node: ```bash openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null |  openssl dgst -sha256 -hex | sed 's/^.* //' ``` The output is similar to: ```console 8cb2de97839780a412b93877f8507ad6c94f73add17d5d7058e91741c9d5ec78 ``` {{< note >}} To specify an IPv6 tuple for `<control-plane-host>:<control-plane-port>`, IPv6 address must be enclosed in square brackets, for example: `[2001:db8::101]:2073`. {{< /note >}} </del> <ins> See [Creating Highly Available Clusters with kubeadm](/docs/setup/production-environment/tools/kubeadm/high-availability/) for steps on creating a high availability kubeadm cluster by adding more control plane nodes. </ins> <del> The output should look something like: </del> <ins> ### Adding worker nodes {#join-nodes} </ins> <del> ``` [preflight] Running pre-flight checks </del> <ins> The worker nodes are where your workloads run. </ins> <del> ... (log output of join workflow) ... </del> <ins> The following pages show how to add Linux and Windows worker nodes to the cluster by using the `kubeadm join` command: </ins> <del> Node join complete: * Certificate signing request sent to control-plane and response received. * Kubelet informed of new secure connection details. Run 'kubectl get nodes' on control-plane to see this machine join. ``` A few seconds later, you should notice this node in the output from `kubectl get nodes` when run on the control-plane node. {{< note >}} As the cluster nodes are usually initialized sequentially, the CoreDNS Pods are likely to all run on the first control-plane node. To provide higher availability, please rebalance the CoreDNS Pods with `kubectl -n kube-system rollout restart deployment coredns` after at least one new node is joined. {{< /note >}} </del> <ins> * [Adding Linux worker nodes](/docs/tasks/administer-cluster/kubeadm/adding-linux-nodes/) * [Adding Windows worker nodes](/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes/) </ins> ### (Optional) Controlling your cluster from machines other than the control-plane node


Question:
Please note: This issue is not suitable for new contributors since you need some familiarity with how Kubernetes works. The Kubernetes website currently lacks a guide on creating Windows/Linux nodes. See for more details. SIG Windows opted to remove the guide that did exist for creating Windows nodes, in PR Write a new task page about creating nodes for all operating systems. This could, potentially, be in a tabbed format as mentions in the issue referenced above. We would also need the entry reflected on the creates the Linux task page by moving / refactoring the existing "join node" contents from the "create a cluster" page: creates the Windows task page by taking the content from at the end of the Windows page we add note/disclaimer whatever that says that CNI on Windows needs special handling and we link to the various CNI choices at: adds tabs in the to choose between OS for joining nodes. /priority backlog /help
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : Problem Statement: /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/triage accepted Thank you for filing this
/sig cluster-lifecycle
If you know how to set up new nodes (with, at least, the tool): you can work on this issue. You don't need anyone's permission to open a pull request. All we ask is that you read our and that you abide by the project's code of conduct.
/assign
Relevant to
Are you still working on this issue?
/assign sending PR soon.
/priority important-longterm
Relevant to
/retitle Write new task pages about creating Kubernetes nodes
/cc
Done in PR /close
Closing this issue. <detailsIn response to : Done in PR Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

* A compatible Linux host. The Kubernetes project provides generic instructions for Linux distributions based on Debian and Red Hat, and those distributions without a package manager. * 2 GB or more of RAM per machine (any less will leave little room for your apps). <del> * 2 CPUs or more. </del> <ins> * 2 CPUs or more for control plane machines. </ins> * Full network connectivity between all machines in the cluster (public or private network is fine). * Unique hostname, MAC address, and product_uuid for every node. See [here](#verify-mac-address) for more details. * Certain ports are open on your machines. See [here](#check-required-ports) for more details.


Question:
Please note: This issue is not suitable for new contributors since you need some familiarity with how Kubernetes works. The Kubernetes website currently lacks a guide on creating Windows/Linux nodes. See for more details. SIG Windows opted to remove the guide that did exist for creating Windows nodes, in PR Write a new task page about creating nodes for all operating systems. This could, potentially, be in a tabbed format as mentions in the issue referenced above. We would also need the entry reflected on the creates the Linux task page by moving / refactoring the existing "join node" contents from the "create a cluster" page: creates the Windows task page by taking the content from at the end of the Windows page we add note/disclaimer whatever that says that CNI on Windows needs special handling and we link to the various CNI choices at: adds tabs in the to choose between OS for joining nodes. /priority backlog /help
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : Problem Statement: /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/triage accepted Thank you for filing this
/sig cluster-lifecycle
If you know how to set up new nodes (with, at least, the tool): you can work on this issue. You don't need anyone's permission to open a pull request. All we ask is that you read our and that you abide by the project's code of conduct.
/assign
Relevant to
Are you still working on this issue?
/assign sending PR soon.
/priority important-longterm
Relevant to
/retitle Write new task pages about creating Kubernetes nodes
/cc
Done in PR /close
Closing this issue. <detailsIn response to : Done in PR Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

<ins> --- title: Adding Linux worker nodes content_type: task weight: 50 --- <!-- overview --> This page explains how to add Linux worker nodes to a kubeadm cluster. ## {{% heading "prerequisites" %}} * Each joining worker node has installed the required components from [Installing kubeadm](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/), such as, kubeadm, the kubelet and a {{< glossary_tooltip term_id="container-runtime" text="container runtime" >}}. * A running kubeadm cluster created by `kubeadm init` and following the steps in the document [Creating a cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/). * You need superuser access to the node. <!-- steps --> ## Adding Linux worker nodes To add new Linux worker nodes to your cluster do the following for each machine: 1. Connect to the machine by using SSH or another method. 1. Run the command that was output by `kubeadm init`. For example: ```bash sudo kubeadm join --token <token> <control-plane-host>:<control-plane-port> --discovery-token-ca-cert-hash sha256:<hash> ``` ### Additional information for kubeadm join {{< note >}} To specify an IPv6 tuple for `<control-plane-host>:<control-plane-port>`, IPv6 address must be enclosed in square brackets, for example: `[2001:db8::101]:2073`. {{< /note >}} If you do not have the token, you can get it by running the following command on the control plane node: ```bash sudo kubeadm token list ``` The output is similar to this: ```console TOKEN                    TTL  EXPIRES              USAGES           DESCRIPTION            EXTRA GROUPS 8ewj1p.9r9hcjoqgajrj4gi  23h  2018-06-12T02:51:28Z authentication,  The default bootstrap  system: signing          token generated by     bootstrappers: 'kubeadm init'.        kubeadm: default-node-token ``` By default, node join tokens expire after 24 hours. If you are joining a node to the cluster after the current token has expired, you can create a new token by running the following command on the control plane node: ```bash sudo kubeadm token create ``` The output is similar to this: ```console 5didvk.d09sbcov8ph2amjw ``` If you don't have the value of `--discovery-token-ca-cert-hash`, you can get it by running the following commands on the control plane node: ```bash sudo cat /etc/kubernetes/pki/ca.crt | openssl x509 -pubkey  | openssl rsa -pubin -outform der 2>/dev/null |  openssl dgst -sha256 -hex | sed 's/^.* //' ``` The output is similar to: ```console 8cb2de97839780a412b93877f8507ad6c94f73add17d5d7058e91741c9d5ec78 ``` The output of the `kubeadm join` command should look something like: ``` [preflight] Running pre-flight checks ... (log output of join workflow) ... Node join complete: * Certificate signing request sent to control-plane and response received. * Kubelet informed of new secure connection details. Run 'kubectl get nodes' on control-plane to see this machine join. ``` A few seconds later, you should notice this node in the output from `kubectl get nodes`. (for example, run `kubectl` on a  control plane node). {{< note >}} As the cluster nodes are usually initialized sequentially, the CoreDNS Pods are likely to all run on the first control plane node. To provide higher availability, please rebalance the CoreDNS Pods with `kubectl -n kube-system rollout restart deployment coredns` after at least one new node is joined. {{< /note >}} ## {{% heading "whatsnext" %}} * See how to [add Windows worker nodes](/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes/). </ins>


Question:
Please note: This issue is not suitable for new contributors since you need some familiarity with how Kubernetes works. The Kubernetes website currently lacks a guide on creating Windows/Linux nodes. See for more details. SIG Windows opted to remove the guide that did exist for creating Windows nodes, in PR Write a new task page about creating nodes for all operating systems. This could, potentially, be in a tabbed format as mentions in the issue referenced above. We would also need the entry reflected on the creates the Linux task page by moving / refactoring the existing "join node" contents from the "create a cluster" page: creates the Windows task page by taking the content from at the end of the Windows page we add note/disclaimer whatever that says that CNI on Windows needs special handling and we link to the various CNI choices at: adds tabs in the to choose between OS for joining nodes. /priority backlog /help
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : Problem Statement: /help Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/triage accepted Thank you for filing this
/sig cluster-lifecycle
If you know how to set up new nodes (with, at least, the tool): you can work on this issue. You don't need anyone's permission to open a pull request. All we ask is that you read our and that you abide by the project's code of conduct.
/assign
Relevant to
Are you still working on this issue?
/assign sending PR soon.
/priority important-longterm
Relevant to
/retitle Write new task pages about creating Kubernetes nodes
/cc
Done in PR /close
Closing this issue. <detailsIn response to : Done in PR Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

<ins> --- title: Adding Windows worker nodes content_type: task weight: 50 --- <!-- overview --> This page explains how to add Windows worker nodes to a kubeadm cluster. ## {{% heading "prerequisites" %}} * A running [Windows Server 2022](https://www.microsoft.com/cloud-platform/windows-server-pricing) (or higher) instance with administrative access. * A running kubeadm cluster created by `kubeadm init` and following the steps in the document [Creating a cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/). <!-- steps --> ## Adding Windows worker nodes {{< note >}} To facilitate the addition of Windows worker nodes to a cluster, PowerShell scripts from the repository https://sigs.k8s.io/sig-windows-tools are used. {{< /note >}} Do the following for each machine: 1. Open a PowerShell session on the machine. 1. Make sure you are Administrator or a privileged user. Then proceed with the steps outlined below. ### Install containerd {{% thirdparty-content %}} To install containerd, first run the following command: ```PowerShell curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/Install-Containerd.ps1 `````` Then run the following command, but first replace `CONTAINERD_VERSION` with a recent release from the [containerd repository](https://github.com/containerd/containerd/releases). The version must not have a `v` prefix. For example, use `1.7.22` instead of `v1.7.22`: ```PowerShell .Install-Containerd.ps1 -ContainerDVersion CONTAINERD_VERSION ``` * Adjust any other parameters for `Install-Containerd.ps1` such as `netAdapterName` as you need them. * Set `skipHypervisorSupportCheck` if your machine does not support Hyper-V and cannot host Hyper-V isolated containers. * If you change the `Install-Containerd.ps1` optional parameters `CNIBinPath` and/or `CNIConfigPath` you will need to configure the installed Windows CNI plugin with matching values. ### Install kubeadm and kubelet Run the following commands to install kubeadm and the kubelet: ```PowerShell curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/PrepareNode.ps1 .PrepareNode.ps1 -KubernetesVersion v{{< skew currentVersion >}} ``` * Adjust the parameter `KubernetesVersion` of `PrepareNode.ps1` if needed. ### Run `kubeadm join` Run the command that was output by `kubeadm init`. For example: ```bash kubeadm join --token <token> <control-plane-host>:<control-plane-port> --discovery-token-ca-cert-hash sha256:<hash> ``` #### Additional information about kubeadm join {{< note >}} To specify an IPv6 tuple for `<control-plane-host>:<control-plane-port>`, IPv6 address must be enclosed in square brackets, for example: `[2001:db8::101]:2073`. {{< /note >}} If you do not have the token, you can get it by running the following command on the control plane node: ```bash kubeadm token list ``` The output is similar to this: ```console TOKEN                    TTL  EXPIRES              USAGES           DESCRIPTION            EXTRA GROUPS 8ewj1p.9r9hcjoqgajrj4gi  23h  2018-06-12T02:51:28Z authentication,  The default bootstrap  system: signing          token generated by     bootstrappers: 'kubeadm init'.        kubeadm: default-node-token ``` By default, node join tokens expire after 24 hours. If you are joining a node to the cluster after the current token has expired, you can create a new token by running the following command on the control plane node: ```bash kubeadm token create ``` The output is similar to this: ```console 5didvk.d09sbcov8ph2amjw ``` If you don't have the value of `--discovery-token-ca-cert-hash`, you can get it by running the following commands on the control plane node: ```bash sudo cat /etc/kubernetes/pki/ca.crt | openssl x509 -pubkey  | openssl rsa -pubin -outform der 2>/dev/null |  openssl dgst -sha256 -hex | sed 's/^.* //' ``` The output is similar to: ```console 8cb2de97839780a412b93877f8507ad6c94f73add17d5d7058e91741c9d5ec78 ``` The output of the `kubeadm join` command should look something like: ``` [preflight] Running pre-flight checks ... (log output of join workflow) ... Node join complete: * Certificate signing request sent to control-plane and response received. * Kubelet informed of new secure connection details. Run 'kubectl get nodes' on control-plane to see this machine join. ``` A few seconds later, you should notice this node in the output from `kubectl get nodes`. (for example, run `kubectl` on a  control plane node). ### Network configuration CNI setup on clusters mixed with Linux and Windows nodes requires more steps than just running `kubectl apply` on a manifest file. Additionally, the CNI plugin running on control plane nodes must be prepared to support the CNI plugin running on Windows worker nodes. {{% thirdparty-content %}} Only a few CNI plugins currently support Windows. Below you can find individual setup instructions for them: * [Flannel](https://sigs.k8s.io/sig-windows-tools/guides/flannel.md) * [Calico](https://docs.tigera.io/calico/latest/getting-started/kubernetes/windows-calico/) ### Install kubectl for Windows (optional) {#install-kubectl} See [Install and Set Up kubectl on Windows](/docs/tasks/tools/install-kubectl-windows/). ## {{% heading "whatsnext" %}} * See how to [add Linux worker nodes](/docs/tasks/administer-cluster/kubeadm/adding-linux-nodes/). </ins>


Question:
This is a Bug Report Problem: text is no longer needed as sidecar containers are now entered into beta state and they are enable by default. reference link of sidecar containers present in page is wrongly redirecting to page. Proposed Solution: the text below from sidecar containers documentation. reference link present in page from page to page. Page to Update:
/sig docs /language en
/assign
Beta APIs are off by default with
is not a beta API though /triage accepted

## Enabling sidecar containers <del> Starting with Kubernetes 1.28, a </del> <ins> Enabled by default with Kubernetes 1.29, a </ins> [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) named `SidecarContainers` allows you to specify a `restartPolicy` for containers listed in a Pod's `initContainers` field. These restartable _sidecar_ containers are independent with


Question:
This is a Bug Report Problem: text is no longer needed as sidecar containers are now entered into beta state and they are enable by default. reference link of sidecar containers present in page is wrongly redirecting to page. Proposed Solution: the text below from sidecar containers documentation. reference link present in page from page to page. Page to Update:
/sig docs /language en
/assign
Beta APIs are off by default with
is not a beta API though /triage accepted

Here's an example of a Job with two containers, one of which is a sidecar: {{% code_sample language="yaml" file="application/job/job-sidecar.yaml" %}} <del> By default, this feature is not available in Kubernetes. To avail this feature, you need to enable the [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) named `SidecarContainers`. </del> ## Differences from regular containers