Skip to main content

Kubernetes 版本

  • Feature Gates
  • kubernetes/enhancements
  • 每年 3 版本，约 15 周一个版本
  • Kubernetes 1.19+ 一般提供约 1 年 patch support；进入 maintenance 后仅接受关键安全修复
  • Version Skew Policy
    • kube-apiserver 通常先升级，kubelet/kube-proxy 再逐步升级
    • Kubernetes 1.28 起，kubelet 可比 kube-apiserver 旧最多 3 个 minor（n-3）
    • 实务上仍建议一个 minor 一个 minor 升级，并至少半年评估一次
versionreleaselatest patchstatusEOL
Kubernetes 1.362026-04-221.36.1Active2027-06-28
Kubernetes 1.352025-12-171.35.5Active2027-02-28
Kubernetes 1.342025-08-271.34.8Active2026-10-27
Kubernetes 1.332025-04-231.33.12Maintenance2026-06-28
Kubernetes 1.322024-12-111.32.13EOL2026-02-28
Kubernetes 1.312024-08-131.31.14EOL2025-11-11
Kubernetes 1.302024-04-171.30.14EOL2025-07-15
Kubernetes 1.292023-12-131.29.14EOL2025-02-28
Kubernetes 1.282023-08-151.28.15EOL2024-10-22
Kubernetes 1.272023-04-111.27.16EOL2024-07-16
Kubernetes 1.262022-12-09EOL
Kubernetes 1.252022-08-23EOL
Kubernetes 1.242022-05-03EOL
Kubernetes 1.232021-12-07EOL
Kubernetes 1.222021-08-04EOL
Kubernetes 1.212021-04-08EOL
Kubernetes 1.202020-12-08EOL
Kubernetes 1.192020-08-26EOL
APIfromto
networking.k8s.io/v1beta1v1.22
networking.k8s.io/v1v1.19

特性分组说明

feature groupsdesc
api-machinery接口
apps执行
auth授权
cli命令行 - kubectl
cluster-lifecycle集群生命周期
cluster-provider集群供应商
instrumentationkubelet 自身相关
network网络
node节点
scheduling调度 - Pod
storage存储
windowsWindows 相关
特性阶段默认
  • Stable 一般默认开启,
  • Beta 可能开启
  • Alpha 一般为关闭
版本关注点
  • breaking-change
    • 移除 depracated - 例如 Ingress 升级过程
  • 默认启用的特性 - Stable, Beta
  • distro 开启的特性 - K3S, K0S
  • 对使用影响大的特性

Kubernetes 1.36​

  • Release: 2026-04-22
  • Theme: Haru
  • 70 enhancements: 18 Stable, 20 Beta, 32 Alpha
  • 重点
    • Fine-grained API authorization
    • Resource health status
    • Workload Aware Scheduling（WAS）
  • Stable
    • Volume group snapshots
    • Mutable volume attach limits
    • External signing of ServiceAccount tokens
    • DRA 部分能力 GA
    • Mutating admission policies
    • Declarative validation for Kubernetes native types
    • 移除 Kubernetes API types 对 gogo protobuf 的依赖
    • Node log query
    • User Namespaces in Pods
    • PSI based on cgroup v2
  • Beta
    • Staleness mitigation for controllers
    • IP/CIDR validation improvements
    • .kuberc user preferences 从 kubeconfig 拆分
    • Suspended Job 的 mutable container resources
    • Constrained impersonation
    • DRA 更多能力 beta
    • /statusz、/flagz 结构化输出
    • Mixed version proxy / unknown version interoperability proxy
    • Memory QoS with cgroups v2
  • Alpha
    • Workload-aware scheduling / gang scheduling 相关能力继续推进
  • 升级关注
    • 1.33 即将 EOL，1.34+ 为主要升级目标；如果要保留更长窗口，优先看 1.35/1.36。
    • DRA、WAS、GPU/AI 相关调度能力变化较多，设备插件和调度扩展需要跟随验证。

Kubernetes 1.35​

  • Release: 2025-12-17
  • Theme: Timbernetes / The World Tree Release
  • 44 enhancements: 14 Stable, 18 Beta, 12 Alpha
  • 重点
    • In-place update of Pod resources GA
      • 可不重启 Pod/Container 调整 CPU/Memory requests/limits
    • Pod certificates for workload identity and security Beta
    • Node declared features before scheduling Alpha
  • Stable
    • PreferSameNode / PreferSameZone traffic distribution
    • Job API managedBy
    • Pod .metadata.generation、.status.observedGeneration
    • Topology Manager configurable NUMA node limit
  • Beta
    • Node topology labels via Downward API
    • Native storage version migration
    • Mutable CSI volume attach limits
    • Scheduler opportunistic batching
    • StatefulSet maxUnavailable
    • .kuberc credential plugin policy
    • KYAML 默认启用
    • HPA configurable tolerance
    • User namespaces in Pods
    • OCI artifact / image volumes
    • Kubelet credential verification for cached images
    • Fine-grained container restart rules
    • CSI driver ServiceAccount token via secrets field
    • Deployment terminatingReplicas
  • Alpha
    • Gang scheduling / Workload API / PodGroup
    • Constrained impersonation
    • /flagz、/statusz JSON 输出
    • CCM route controller watch-based reconciliation
  • 升级关注
    • In-place Pod resize GA 后，VPA/自研控制器可以减少重建 Pod，但要验证运行时、指标和回滚语义。
    • KYAML、kuberc、credential plugin policy 会影响 CLI/运维体验，注意团队工具链兼容。

Kubernetes 1.34​

  • Release: 2025-08-27
  • Theme: Of Wind & Will (O' WaW)
  • 58 enhancements: 23 Stable, 22 Beta, 13 Alpha
  • 重点
    • Dynamic Resource Allocation（DRA）core GA
      • resource.k8s.io/v1、ResourceClaim、DeviceClass、ResourceSlice 等 API 稳定
    • Projected ServiceAccount tokens for kubelet image credential providers Beta
    • KYAML Alpha
  • Stable
    • Job .spec.podReplacementPolicy
    • Recovery from volume expansion failure
    • VolumeAttributesClass
    • Structured authentication configuration
    • Selector-aware fine-grained authorization
    • Anonymous request endpoint allowlist
    • Scheduler plugin-specific requeue callbacks
    • Ordered namespace deletion
    • Streaming list responses
    • Resilient watch cache initialization
    • Relaxed DNS search path validation
    • Windows kube-proxy DSR
    • Container lifecycle hook Sleep action
    • Linux node swap support
    • Special characters in env var names
    • Taint manager 与 Node lifecycle 解耦
  • Beta
    • Pod-level resource requests/limits
    • .kuberc user preferences
    • External ServiceAccount token signing
    • DRA admin access、prioritized alternatives、kubelet DRA resource reporting
    • kube-scheduler non-blocking API calls
    • Mutating admission policies
    • Snapshottable API server cache
  • 升级关注
    • 大规模集群可重点关注 streaming list、watch cache、scheduler requeue、snapshottable cache。
    • GPU/TPU/NIC 等设备管理应评估 DRA API 与现有 device plugin 迁移路径。

Kubernetes 1.33​

  • Release: 2025-04-23
  • Theme: Octarine / The Color of Magic
  • 64 enhancements: 18 Stable, 20 Beta, 24 Alpha, 2 Deprecated/Withdrawn
  • 重点
    • Sidecar containers GA
    • In-place Pod resource resize Beta
    • .kuberc Alpha
  • Stable
    • Indexed Job backoff limits per index
    • Job success policy
    • Bound ServiceAccount token security improvements
    • kubectl subresource support
    • Multiple Service CIDRs
    • nftables backend for kube-proxy
    • SMT alignment rejection options
    • Pod affinity/anti-affinity matchLabelKeys / mismatchLabelKeys
    • Pod topology spread skew considers taints/tolerations
    • Volume populators
    • Honor PersistentVolume reclaim policy
  • Beta
    • Windows kube-proxy DSR
    • DRA structured parameters
    • DRA for network interfaces
    • Scheduler asynchronous preemption
    • ClusterTrustBundles
    • Fine-grained SupplementalGroups control
    • Image volumes
    • User namespaces in Linux Pods
  • Alpha
    • .kuberc user preferences
    • Ordered namespace deletion
    • Declarative validation for native Kubernetes types
  • 升级关注
    • Sidecar containers GA 后可替代大量 init/sidecar 生命周期 hack。
    • 1.33 当前已进入 maintenance，EOL 为 2026-06-28，应尽快规划到 1.34+。

Kubernetes 1.32​

  • Release: 2024-12-11
  • Theme: Penelope
  • 44 enhancements: 13 Stable, 12 Beta, 19 Alpha
  • 更新重点
    • DRA 持续推进
    • Node 与 sidecar container 体验改善
  • Stable
    • CustomResource field selectors
    • Memory-backed volume sizing
    • Bound ServiceAccount token improvements
    • Structured authorization configuration
    • StatefulSet-created PVC auto removal
  • Beta
    • Job API managedBy
    • Anonymous auth endpoint allowlist
    • Scheduler plugin callback based requeue
    • Volume expansion failure recovery
    • Volume group snapshot
    • DRA structured parameters
    • Label and field selector authorization
  • Alpha
    • Scheduler asynchronous preemption
    • CEL mutating admission policies
    • Pod-level resources
    • PreStop sleep action zero value
    • Standardized network interface data in ResourceClaim status
  • 升级关注
    • 1.32 已 EOL；不要作为新集群目标版本。
    • StatefulSet PVC 自动删除、CRD field selector、structured authz 对平台控制面有实际价值。

Kubernetes 1.31​

  • Release: 2024-08-13
  • Theme: Elli
  • 45 enhancements: 11 Stable, 22 Beta, 12 Alpha
  • Stable
    • AppArmor support
    • kube-proxy ingress connectivity reliability
    • PersistentVolume last phase transition time
  • Beta
    • kube-proxy nftables backend
    • PersistentVolume reclaim policy changes
    • Bound ServiceAccount token improvements
    • Multiple Service CIDRs
    • Service trafficDistribution
    • VolumeAttributesClass / ModifyVolume
  • Alpha
    • New DRA APIs
    • Image volumes
    • Device health in Pod status
    • Selector-based authorization
    • Anonymous API access restrictions
  • Deprecations / Removals
    • cgroup v1 进入 maintenance mode
    • SHA-1 signature support 需要关注
    • Node .status.nodeInfo.kubeProxyVersion deprecated
  • 升级关注
    • Service trafficDistribution 从 1.31 开始进入演进，后续 1.34/1.35 变得更明确。
    • 1.31 已 EOL；仅作为历史参考，不建议继续运行。

Kubernetes 1.30​

  • Service
    • Alpha spec.trafficDistribution

Kubernetes 1.29​

  • Alpha
    • nftables backend for kube-proxy

Kubernetes 1.28​

  • CP 组件版本 <-> 节点组件版本 从 n-2 -> n-3
    • CP 可升级更快
    • CP 组件 - kube-apiserver, kube-scheduler, kube-controller-manager, cloud-controller-manager
    • 节点组件 - kubelet, kube-proxy
  • Alpha
    • init container restartPolicy

Kubernetes 1.27​

  • k8s.gcr.io -> registry.k8s.io
  • Beta
    • ReadWriteOncePod for PV & PVC

Kubernetes 1.26​

  • Stable
    • KubeletCredentialProviders
    • CPUManager
    • ServiceInternalTrafficPolicy
      • Service.spec.trafficPolicy=Cluster - Topology,PreferLocal,Local
    • MixedProtocolLBService
  • Beta
    • NodeOutOfServiceVolumeDetach
      • Non-graceful node shutdown
    • ProxyTerminatingEndpoints
      • 避免滚动升级时流量丢失
  • Alpha
    • MinimizeIPTablesRestore=false
      • iptables-restore 只传递修改内容
  • https://sysdig.com/blog/kubernetes-1-26-whats-new/

Kubernetes 1.25​

  • PodSecurityPolicy
  • Stable
    • Pod Security Admission
    • Ephemeral Containers
    • cgroups v2
  • k8s.gcr.io -> registry.k8s.io
  • KMS v2

Kubernetes 1.24​

groupstagefeaturedefaultalphabeta
appsAlphaCronJobTimeZone
appsStableIndexedJob
appsStableJobReadyPods
authStableCSRDuration
cloud-providerStableServiceLoadBalancerClass
networkAlphaNetworkPolicyStatus
networkAlphaServiceIPStaticSubrange
networkBetaMixedProtocolLBService
networkBetaServiceInternalTrafficPolicy
nodeBetaDisableKubeletCloudCredentialProviders
nodeBetaGracefulNodeShutdownBasedOnPodPriority
nodeBetaGRPCContainerProbe
nodeStablePodOverhead
schedulingStableNonPreemptingPriority
schedulingStablePodAffinityNamespaceSelector
storageAlphaCSIVolumeHealth
storageAlphaNonGracefulFailover
storageBetaAnyVolumeDataSource
storageBetaHonorPVReclaimPolicy
storageStableCSIStorageCapacity
storageStableExpandCSIVolumes1.81.11
storageStableExpandInUsePersistentVolumes1.81.11
storageStableExpandPersistentVolumes1.81.11
windowsBetaIdentifyPodOS

Kubernetes 1.23​

tip
  • Kubernetes 会逐步把 CSI 移出核心
stagegroupfeaturedefaultalphabetastable
Alphaapi-machineryCustomResourceValidationExpressions
Alphaapi-machineryOpenAPIEnum
Alphaapi-machineryOpenApiv3
Alphaapi-machineryServerSideFieldValidation
AlphaappsStatefulSetAutoDeletePVC
Alphacluster-lifecycleUnversionedKubeletConfigMap
AlphanodeCPUManagerPolicyExperimentalOptions
AlphanodeGracefulNodeShutdownBasedOnPodPriority
AlphanodeGRPCContainerProbe
AlphanodePodAndContainerStatsFromCRI
AlphastorageCSIMigrationPortworx
AlphastorageCSIMigrationRBD
AlphastorageDelegateFSGroupToCSIDriver
AlphastorageHonorPVReclaimPolicy
AlphastorageRecoverVolumeExpansionFailure
AlphawindowsIdentifyPodOS
AlphawindowsWindowsHostProcessContainerstrue
BetaauthPodSecuritytrue
BetanodeCPUManagerPolicyOptionsfalse
BetanodeEphemeralContainerstrue
BetanodeKubeletPodResourcesGetAllocatabletrue
BetaschedulingJobMutableNodeSchedulingDirectivestrue
StableappsJobReadyPods
StableappsJobTrackingWithFinalizers
StableappsStatefulSetMinReadySeconds
StableappsTTLAfterFinished
StablenetworkIngressClassNamespacedParams
StablenetworkIPv6DualStack
StablenetworkTopologyAwareHints
StablestorageConfigurableFSGroupPolicy
StablestorageCSIMigrationAWSfalse
StablestorageCSIVolumeFSGroupPolicy
StablestorageGenericEphemeralVolume

Kubernetes 1.22​

caution
  • ingress 移除了 networking.k8s.io/v1beta1 , 建议等下一个大版本再升级
    • 因为很多周边配套服务还没有升级使用 networking.k8s.io/v1
  • 新的 PodSecurity admission 控制器 作为 PSP 替代
  • Rootless 模式容器
    • KubeletInUserNamespace
  • Seccomp
    • SeccompDefault - 支持默认开启
  • 支持 swap
    • NodeMemorySwap
    • --fail-on-swap
    • kubelet MemorySwap.SwapBehavior=UnlimitedSwap
  • cgroupsv2
    • Memory QoS - min,max,low,high
  • 稳定 API
    • Server-side Apply - kubectl apply 功能迁移到服务端 - 1.14+
    • 废弃接口告警机制 - 1.19+
    • namespace 不可变 label - kubernetes.io/metadata.name - 1.21+
    • CronJobs - 1.4+
    • PodDisruptionBudget - 1.4+
    • EndpointSlice - 解决 pod 多 IP 问题
      • 1.22 会为超过 1000 的 Endpoints 添加 endpoints.kubernetes.io/over-capacity: truncated
        • 之前是 endpoints.kubernetes.io/over-capacity: warning
    • AppProtocol - Services 和 Endpoints 自定义协议
    • HugePageStorageMediumSize - 1.18+
    • Pod hostnameFQDN: true - hotsname 设置为 FQDN - 1.19+
    • CSIServiceAccountToken
    • Windows CSI 插件 - 1.16+
  • Beta API
    • APIPriorityAndFairness
    • Job .spec.suspend
    • CRS 有效期 - CertificateSigningRequestSpec.ExpirationSeconds
    • Service LoadBalancer Class - 支持多 LB
    • .spec.egress.ports.endPort - 配置端口段
    • 临时容器 - kubectl debug
    • emptyDir.sizeLimit - 限制 tmpfs 内存使用
    • 存储 volume DataSource - 支持预先包含数据
  • Alpha
    • MaxDNSSearchPathsExpanded - 扩展 DNS 长度
      • MaxDNSSearchPaths 6 -> 32
      • MaxDNSSearchListChars 256 -> 2048
    • 存储 ReadWriteOncePod - 单几点 RWM，只允许一个节点
  • 移除废弃 Beta API
    • Ingress
    • CustomResourceDefinition
    • ValidatingWebhookConfiguration
    • MutatingWebhookConfiguration
    • CertificateSigningRequest
  • 功能废弃
    • StreamingProxyRedirects
    • ServiceTopology - 使用 Topology Aware Hints / 1.17+
    • DynamicKubeletConfig
  • 参考
stagefeaturedefaultalphabetastable
AlphaAPIServerTracing1.22
AlphaCPUManagerPolicyOptions1.22
AlphaDelegateFSGroupToCSIDriver1.22
AlphaDisableCloudProviders1.22
AlphaExpandedDNSConfig1.22
AlphaJobTrackingWithFinalizers1.22
AlphaKubeletInUserNamespace1.22
AlphaMemoryQoS1.22
AlphaNodeSwap1.22
AlphaPodSecurity1.22
AlphaProxyTerminatingEndpoints1.22
AlphaReadWriteOncePod1.22
AlphaSeccompDefault1.22
AlphaStatefulSetMinReadySeconds1.22
AlphaWindowsHostProcessContainers1.22
AlphaWindowsHostProcessContainers1.22
BetaControllerManagerLeaderMigrationtrue1.211.22
BetaCSRDurationtrue1.22
BetaDaemonSetUpdateSurgetrue1.211.22
BetaEndpointSliceTerminatingConditiontrue1.201.22
BetaIndexedJobtrue1.211.22
BetaIngressClassNamespacedParamstrue1.211.22
BetaLogarithmicScaleDowntrue1.211.22
BetaMemoryManagertrue1.211.22
BetaNetworkPolicyEndPorttrue1.211.22
BetaPodAffinityNamespaceSelectortrue1.211.22
BetaPodDeletionCosttrue1.211.22
BetaPreferNominatedNodetrue1.211.22
BetaProbeTerminationGracePeriodfalse1.211.22
BetaServiceInternalTrafficPolicytrue1.211.22
BetaServiceLBNodePortControltrue1.201.22
BetaServiceLoadBalancerClasstrue1.211.22
BetaSizeMemoryBackedVolumestrue1.201.22
BetaSuspendJobtrue1.211.22
DeprecatedBalanceAttachedNodeVolumesfalse1.11
DeprecatedCSIMigrationvSphereComplete1.19
DeprecatedDynamicKubeletConfigfalse1.41.11
DeprecatedStreamingProxyRedirectsfalse1.5
DeprecatedValidateProxyRedirectstrue1.121.14
StableBoundServiceAccountTokenVolume1.131.211.22
StableCronJobControllerV21.201.211.22
StableCSIServiceAccountToken1.201.211.22
StableEndpointSliceProxying1.181.191.22
StableHugePageStorageMediumSize1.181.191.22
StableHugePageStorageMediumSize1.181.191.22
StableNamespaceDefaultLabelName1.211.22
StableServerSideApply1.141.161.22
StableSetHostnameAsFQDN1.191.201.22
StableWarningHeaders1.191.22
StableWindowsEndpointSliceProxying1.191.211.22
# 当前使用的 ingress 版本
kubectl get ing --all-namespaces -o $'go-template={{range $k,$v := .items}}{{.apiVersion}}\n{{end}}' | sort -n -u
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
  - name: PodSecurity
    configuration:
      defaults: # Defaults applied when a mode label is not set.
        enforce: <default enforce policy level>
        enforce-version: <default enforce policy version>
        audit: <default audit policy level>
        audit-version: <default audit policy version>
        warn: <default warn policy level>
        warn-version: <default warn policy version>
      exemptions:
        usernames: [<array of authenticated usernames to exempt>]
        runtimeClassNames: [<array of runtime class names to exempt>]
        namespaces: [<array of namespaces to exempt>]

Kubernetes 1.21​

  • 13 stable, 16 beta, 20 alpha, 2 废弃
  • Kustomize 从 v2.0.3 升级到 v4.0.5 - 之前因为依赖问题被 block
  • kubelet 支持结构化日志
  • TokenRequest, TokenRequestProjection - 默认开启
  • Stable/GA
    • CronJobs - 1.8 ScheduledJobs beta
    • 不可变的 Secrets, ConfigMaps
      • kubelet 不 watch 变化，减少 apiserver 负载
    • ServiceNodeExclusion, NodeDisruptionExclusion, LegacyNodeRoleBehavior
    • KEP-85 policy/v1/PodDisruptionBudget - 1.5 beta
    • KEP-23 支持 sysctl - 自 1.4
  • Beta
    • IPv4/IPv6 dual-stack
    • CSI Service Account Token
    • Generic ephemeral volumes - 1.19 alpha
    • Storage Capacity - 1.19 alpha
      • 之前调度不会关心存储空间问题
      • 用于后续支持基于节点存储空间调度功能
    • Graceful node shutdown
      • 1.21 默认开启
        • ShutdownGracePeriod, ShutdownGracePeriodCriticalPods 默认为 0
        • 设置为非 0 激活节点优雅关机
      • 依赖 systemd
    • KEP-592 Job ttlSecondsAfterFinished
      • Job 完成后，在 TTL 时间过后被删除
  • Alpha
    • CSI Health Monitoring
    • Topology Aware Hints
      • 在 EndpointSlice 和 Endpoints 添加元信息 - hints
      • KEP-2433
    • kEP-2255 Pod label controller.kubernetes.io/pod-deletion-cost 标识 删除成本
      • 默认 0
      • 影响 ReplicaSet 缩容调度 逻辑
    • KEP-2214 Job 索引
    • KEP-2232 停止 Job
      • .spec.suspend=true 停止，设置为 false 恢复调度
    • KEP_2227 Pod 默认容器标签 kubectl.kubernetes.io/default-container
      • 设置后 kubectl exec 可以不在指定容器 - 使用上会方便很多
  • 废弃
  • 参考
stagefeaturedefaultalphabetastable
AlphaBalanceAttachedNodeVolumes1.11
AlphaControllerManagerLeaderMigration1.211.22
AlphaControllerManagerLeaderMigration1.211.22
AlphaCSIVolumeHealth1.21
AlphaDaemonSetUpdateSurge1.211.22
AlphaEndpointSliceTerminatingCondition1.201.22
AlphaIndexedJob1.211.22
AlphaIngressClassNamespacedParams1.211.22
AlphaInTreePluginAWSUnregister1.21
AlphaInTreePluginAzureDiskUnregister1.21
AlphaInTreePluginAzureFileUnregister1.21
AlphaInTreePluginGCEUnregister1.21
AlphaInTreePluginOpenStackUnregister1.21
AlphaInTreePluginvSphereUnregister1.21
AlphaKubeletPodResourcesGetAllocatable1.21
AlphaLogarithmicScaleDown1.211.22
AlphaMemoryManager1.211.22
AlphaNetworkPolicyEndPort1.211.22
AlphaPodAffinityNamespaceSelector1.211.22
AlphaPodDeletionCost1.211.22
AlphaPreferNominatedNode1.211.22
AlphaProbeTerminationGracePeriod1.211.22
AlphaServiceInternalTrafficPolicy1.211.22
AlphaServiceLBNodePortControl1.201.22
AlphaServiceLoadBalancerClass1.211.22
AlphaSizeMemoryBackedVolumes1.201.22
AlphaSuspendJob1.211.22
AlphaTopologyAwareHints1.21
AlphaVolumeCapacityPriority1.21
BetaBoundServiceAccountTokenVolumetrue1.131.211.22
BetaCronJobControllerV2true1.201.211.22
BetaCSIMigrationAzureFilefalse1.151.21
BetaCSIMigrationvSphereCompletefalse1.19
BetaCSIServiceAccountTokentrue1.201.211.22
BetaCSIStorageCapacitytrue1.191.21
BetaDownwardAPIHugePagesfalse1.201.21
BetaDynamicKubeletConfigtrue1.41.11
BetaEfficientWatchResumptiontrue1.201.21
BetaEndpointSliceProxyingtrue1.181.191.22
BetaGenericEphemeralVolumetrue1.191.21
BetaGracefulNodeShutdowntrue1.201.21
BetaHugePageStorageMediumSizetrue1.181.191.22
BetaHugePageStorageMediumSizetrue1.181.191.22
BetaIPv6DualStacktrue1.151.21
BetaNamespaceDefaultLabelNametrue1.211.22
BetaServerSideApplytrue1.141.161.22
BetaSetHostnameAsFQDNtrue1.191.201.22
BetaTTLAfterFinishedtrue1.121.21
BetaValidateProxyRedirectstrue1.121.14
BetaWarningHeaderstrue1.191.22
BetaWindowsEndpointSliceProxyingtrue1.191.211.22
DeprecatedCSIMigrationAWSComplete1.17
DeprecatedCSIMigrationAzureDiskComplete1.17
DeprecatedCSIMigrationAzureFileComplete1.17
DeprecatedCSIMigrationGCEComplete1.17
DeprecatedCSIMigrationOpenStackComplete1.17
DeprecatedStreamingProxyRedirectstrue1.5
StableCRIContainerLogRotation1.101.111.21
StableEndpointSlice1.161.171.21
StableEndpointSliceNodeName1.201.21
StableImmutableEphemeralVolumes1.181.191.21
StableLegacyNodeRoleBehaviorfalse1.161.191.21
StableNodeDisruptionExclusion1.161.191.21
StablePodDisruptionBudget1.31.51.21
StableRootCAConfigMap1.131.201.21
StableRunAsGroup1.141.21
StableServiceAccountIssuerDiscovery1.181.201.21
StableServiceNodeExclusion1.81.191.21
StableSysctls1.111.21

POD

apiVersion: v1
kind: Pod
metadata:
  name: sysctl-example
  annotations:
    # 旧的 sysctl 控制
    security.alpha.kubernetes.io/sysctls: kernel.shm_rmid_forced=1
    # 定义 kubectl 的默认 容器
    kubectl.kubernetes.io/default-container: 'test'
    # 删除成本 Alpha
    # 影响调度
    controller.kubernetes.io/pod-deletion-cost: '10'
spec:
  securityContext:
    # 新的 sysctl 控制
    sysctls:
      - name: kernel.shm_rmid_forced
        value: 1
  # 优雅停止 - Alpha
  # Pod 级别
  terminationGracePeriodSeconds: 3600
  containers:
    containers:
      - name: test
        image: alpine
        annotations:
        livenessProbe:
          # 容器覆盖 Pod 级别配置
          terminationGracePeriodSeconds: 60

JOB

apiVersion: batch/v1
 kind: Job
 metadata:
   name: 'indexed-job'
 spec:
  # 停止的 Job
  suspend: true

  completions: 5
  parallelism: 3
  # 索引 Job
  # 每个 Pod 的索引为 0 到 completions-1
  # 默认 NonIndexed
  completionMode: Indexed

  # TTLAfterFinished
  ttlSecondsAfterFinished: 100

Kubernetes 1.20​

  • 11 stable, 15 beta, 16 alpha
  • 废弃 dockershim
    • 只是废弃 dockershim - 暂无移除计划
    • 使用 CRI 标准接口 - 目前 docker 不支持, containerd 默认实现
    • docker 底层使用 containerd, docker 依然能正常使用, 只是移除了特殊中间层支持: docker -> shim -> containerd
    • 非 Linux 平台 containerd 无支持，需要 docker
    • Mirantis 会通过 Mirantis/cri-dockerd 添加 docker 的 cri 支持
    • 结论
      • 维持现状
  • go 1.15.5
  • 不再默认包含云平台控制器 - 由云平台自行提供
  • Stable/GA
    • CSI 卷快照
    • PID 限制 - SupportNodePidsLimit
    • TokenRequest/TokenRequestProjection - 目前还需要打开特性，1.21 会默认打开
    • node.k8s.io API 进入 v1 废弃 v1beta1
  • Beta
    • API 优先级处理 - 对 API 进行优先级分层，优先处理重要 API 请求
    • 非递归 卷所归属关系/fsgroup - 可理解为 chown $GROUP:$USER $VOLUME
    • FSGroup CSIDriver 策略
    • RootCAConfigMap - 所有 NS 可访问，用于校验 kube-apiserver 链接
    • kubectl debug
    • SetHostnameAsFQDN
  • Alpha
    • CronJob v2
    • 从新实现 IPv4/IPv6 双栈
    • CSI 安全增强 - CSIServiceAccountToken - 使用其他 SA 而非当前
    • 优雅的节点停止 - GracefulNodeShutdown
    • 移除日志敏感信息 - --experimental-logging-sanitization
      • 移除字段 passwords, keys, tokens
      • 支持组件 - 不支持用户 Pod 维度相关日志
        • kube-controller-manager
        • kube-apiserver
        • kube-scheduler
        • kubelet
    • 所有 Pod 资源指标 /metrics/resources - --show-hidden-metrics-for-version=1.20
  • 可能遇到的问题
    • exec probe 不会超时，现在 ExecProbeTimeout 会生效，没有配置的默认 1s
  • CHANGELOG-1.20
# IPv4/IPv6 双栈
apiVersion: v1
kind: Service
metadata:
  name: my-service
  labels:
    app: MyApp
spec:
  # 可集群维度配置
  # ipFamilyPolicy: SingleStack # 单栈
  ipFamilyPolicy: PreferDualStack
  # 可选
  ipFamilies:
    - IPv6
    - IPv4
  selector:
    app: MyApp
  ports:
    - protocol: TCP
      port: 80

Kubernetes 1.19​

  • EndpointSlices 默认启用
  • Ingress GA
  • seccomp GA
  • KubeSchedulerConfiguration Beta
  • 存储容量跟踪 - Alpha
    • 之前调度时都不会考虑节点存储容量
  • 通用临时存储卷
  • 不可变的 Secrets 和 ConfigMaps - Beta
  • kubernetes/dashboard v2
  • 通用 ephemeral volumes
    • 生命周期与 pod 绑定
    • 支持使用动态 provisioning 存储作为临时卷 - ALPHA
  • CHANGELOG-1.19

Script​

extract-features.js
const version = '1.21';
function build(ver = version) {
  const all = Array.prototype.map.call($$('table tbody tr'), ($tr) =>
    Array.prototype.map.call($tr.querySelectorAll('td'), ($td) => $td.innerText),
  );
  const byName = all.reduce((o, [name, _, stage, s]) => {
    const v = (o[name] ||= {});
    v[stage] ??= s;
    return o;
  }, {});

  let rows = all
    .filter((v) => v[3] === ver || v[4] === ver)
    .map(([name, def, stage, s, u]) => {
      const o = byName[name];
      stage = stage.replace('GA', 'Stable');
      def = def.replace('-', '');
      switch (stage) {
        case 'Alpha':
          def = def.replace('false', '');
          break;
        case 'Stable':
          def = def.replace('true', '');
          break;
      }
      return [stage, name, def, o.Alpha, o.Beta, o.GA];
    });

  copy(rows.map((v) => v.join('|')).join('\n'));
}