Amazon Kinesis

Amazon Kinesis is a managed platform from AWS for collecting, processing and analyzing real-time streaming data at scaleAmazon Kinesis 是 AWS 推出的托管平台，用于大规模收集、处理与分析实时流数据。

It bundles several services for different stages of stream processing, from ingestion to analysis and delivery into storage它把多个服务捆绑进流处理的不同阶段，从摄取、分析到把结果送入存储。

Features

• Amazon Kinesis Data Streams captures records from producers and makes them available to consumers in ordered, replayable shardsAmazon Kinesis Data Streams 从生产者捕获记录，以有序、可重放的分片提供给消费者。
• Shards scale throughput linearly, and on-demand mode adjusts capacity automatically when traffic grows beyond provisioned limits分片让吞吐线性扩展，按需模式在流量超出预置上限时自动调整容量。
• Records are retained for a configurable window, defaulting to one day and extendable to a year, enabling replay and reprocessing记录保留可配置，默认一天、可延至一年，支持重放与再处理。
• Enhanced fan-out gives each consumer a dedicated throughput pipe, so many applications can read the same stream concurrently增强扇出为每个消费者提供专用吞吐管道，许多应用可并发读取同一流。
• Amazon Kinesis Data Firehose delivers streaming records to destinations such as S3, Redshift, OpenSearch and Splunk without running serversAmazon Kinesis Data Firehose 无需运行服务器即可把流记录送达 S3、Redshift、OpenSearch 与 Splunk 等目标。
• Firehose batches, compresses and encrypts data during delivery, and it can convert format on the fly for analytics storesFirehose 在投递中分批、压缩并加密数据，还能为分析存储即时转换格式。
• Amazon Kinesis Video Streams ingests video and audio from cameras and devices, with playback and machine learning integrationsAmazon Kinesis Video Streams 从摄像头与设备摄取音视频，提供回放与机器学习集成。
• The Kinesis Client Library handles checkpointing and lease management for consumer applications, distributing shard work among workersKinesis 客户端库为消费者应用处理检查点与租约管理，把分片工作分配给各工作进程。
• IAM policies, encryption and VPC endpoints secure data in transit and at rest within the AWS perimeterIAM 策略、加密与 VPC 端点保护 AWS 边界内传输与静止的数据。
• Integrations with Lambda and glue transform streams in place, letting serverless functions react to every record与 Lambda 和 Glue 的集成就地变换流，让无服务器函数对每条记录做出反应。
• CloudWatch metrics expose stream health, lag and provisioning, giving operators the signals to scale capacityCloudWatch 指标暴露流健康、滞后与预置情况，给运维人员伸缩容量所需的信号。
• Enhanced fan-out pairs with consumer groups so applications track progress independently and at different rates over the same stream增强扇出与消费者组配合，让应用在同一流上以不同速率独立跟踪进度。
• Producers can batch records through the client library to improve write efficiency for high-frequency sources生产者可经客户端库批量发送记录，提高高频源的写入效率。
• The platform handles shard splitting and merging during scaling, so consumers observe continuity with bounded rebalancing平台在伸缩时处理分片拆分与合并，消费者看到的是带有限再平衡的连续性。
• Stream analytics can route results back into other Kinesis streams, forming multi-stage pipelines without external orchestration流分析可把结果送回其他 Kinesis 流，无需外部编排即可形成多级管道。
• The service integrates with identity providers so access policies follow the organization's existing directory and role structure服务与身份提供方集成，让访问策略跟随组织既有目录与角色结构。
• CloudWatch alarms can trigger scaling actions on streams, closing the loop between traffic signals and capacityCloudWatch 告警可触发流的扩缩容动作，把流量信号与容量之间的回路闭合。
• Streams and Firehose can be combined in one pipeline, letting raw ingestion and delivery be governed by a single configurationStreams 与 Firehose 可在同一条管道中组合，让原始摄取与投递由单一配置治理。

History

• AWS introduced Kinesis in November 2013 at its annual re:Invent conference, the same period when stream processing was moving into the mainstreamAWS 于 2013 年 11 月在年度 re:Invent 大会推出 Kinesis，正值流处理走向主流之际。
• Data Streams was the founding service, and its shard model became the mental model for many engineers new to streamingData Streams 是创始服务，其分片模型成为许多初识流处理的工程师的心智模型。
• Firehose arrived in 2015 to remove the operational burden of delivering streams into storage and search systemsFirehose 于 2015 年推出，免除把流送入存储与搜索系统的运维负担。
• A SQL-based analytics service followed, letting users query streams continuously without writing application code随后推出的基于 SQL 的分析服务让用户无需编写应用代码即可持续查询流。
• Video Streams launched in 2017, extending the brand from data records to continuous media streamsVideo Streams 于 2017 年发布，把品牌从数据记录扩展到连续媒体流。
• The on-demand mode in 2021 removed manual capacity planning, complementing the provisioned-shard model of earlier years2021 年的按需模式去掉手动容量规划，与早年预置分片模型互补。
• AWS announced the retirement of the SQL analytics service in 2025, directing users toward alternatives as maintenance costs outweighed adoptionAWS 于 2025 年宣布退役 SQL 分析服务，引导用户转向替代品，因为维护成本已超过采用收益。
• Streaming tables introduced in 2026 map streams to Apache Iceberg tables on S3 Tables, blending streaming and lakehouse storage2026 年引入的流式表把流映射到 S3 Tables 上的 Apache Iceberg 表，融合流式与湖仓存储。
• The service grew from a single product into a family, with the Kinesis brand later applied to video and delivery offerings服务从单一产品成长为家族，Kinesis 品牌日后用于视频与投递产品。
• Capacity innovations such as the on-demand advantage program aimed to keep throughput costs predictable for growing workloads按需优势计划等容量创新旨在让增长负载的吞吐成本可预测。
• Documentation and workshops taught a generation of AWS users the shard model, shaping how the wider industry discussed streams文档与工作坊教会一代 AWS 用户分片模型，塑造了整个行业谈论流的方式。
• Regional expansion placed the service in every AWS region, so latency-sensitive workloads rarely leave the local footprint区域扩张把服务放进每个 AWS 区域，延迟敏感负载几乎不必离开本地边界。

Reception

• Kinesis is widely used inside AWS accounts as the default managed stream option, especially where teams already operate on the platformKinesis 在 AWS 账户内被广泛用作默认托管流选项，尤其对已在平台上运营的团队。
• Adopters value the absence of cluster management and the deep integration with Lambda, S3 and other AWS services采用者看重无需集群管理以及与 Lambda、S3 和其他 AWS 服务的深度集成。
• Critics note the per-shard pricing can grow expensive at high throughput, pushing heavy users to compare with self-managed Kafka批评者指出高吞吐下按分片定价可能变贵，促使重负载用户与自管 Kafka 比较。
• Lag metrics and the client library are praised, while some users report that shard management still requires attention during traffic spikes滞后指标与客户端库受称赞，而一些用户反馈流量尖峰期间分片管理仍需留意。
• Benchmarks and surveys commonly list Kinesis alongside Kafka as the leading stream platforms, split between managed and open-source camps基准与调查常把 Kinesis 与 Kafka 并列为主要流平台，分属托管与开源两派。
• The Firehose service earns credit for simplifying the delivery pipeline, with batch conversion saving downstream schema workFirehose 服务因简化投递管道而获好评，批量转换省去下游模式工作。
• Cost comparisons against self-run Kafka frequently favor the managed model once staffing and operations are factored in把人力与运维算入后，与自管 Kafka 的成本对比常倾向托管模式。
• Some large consumers describe migration paths toward Kafka for open-source consistency, while others move in the opposite direction for simplicity一些大客户为开源一致性描述迁往 Kafka 的路径，另一些则为了简单反向迁移。
• Analysts note that Kinesis lowered the barrier to streaming for traditional enterprises, many of which had no stream team before分析师指出 Kinesis 降低传统企业接触流的门槛，其中许多企业此前没有流团队。

Community

• AWS publishes extensive documentation, workshops and sample applications for every Kinesis serviceAWS 为每个 Kinesis 服务发布大量文档、工作坊与示例应用。
• Training and certification paths cover Kinesis within the broader AWS analytics curriculum培训与认证路径在更广的 AWS 分析课程中覆盖 Kinesis。
• Open-source client libraries and connectors for Java, Python, Go and Node.js are maintained by AWS on GitHubAWS 在 GitHub 上维护面向 Java、Python、Go 与 Node.js 的开源客户端库与连接器。
• An active ecosystem of third-party connectors links Kinesis to observability, messaging and machine-learning platforms活跃的第三方连接器生态把 Kinesis 接到可观测性、消息与机器学习平台。
• Community forums and Stack Overflow carry a steady stream of operational questions answered by practitioners and AWS staff社区论坛与 Stack Overflow 承载着由实践者与 AWS 员工回答的稳定运维问答流。
• Free digital training and a public analytics roadmap keep community expectations aligned with AWS's direction免费数字培训与公开分析路线图让社区预期与 AWS 方向保持一致。
• Conference sessions and community blogs regularly share production patterns for sizing, security and cost control大会演讲与社区博客定期分享规模设计、安全与成本控制的实战模式。

Related software

• Apache Kafka is the leading open-source alternative, and teams often weigh its self-managed control against Kinesis's operational simplicityApache Kafka 是领先的开源替代品，团队常在它的自管控制与 Kinesis 的运维简单之间权衡。
• Amazon Managed Streaming for Apache Kafka offers a managed Kafka on AWS, standing midway between self-managed clusters and Kinesis「Amazon Managed Streaming for Apache Kafka」在 AWS 上提供托管 Kafka，处在自管集群与 Kinesis 之间。
• Apache Flink processes streams with strong state and exactly-once semantics, and it commonly consumes Kinesis as its sourceApache Flink 以强状态与恰好一次语义处理流，常以 Kinesis 为数据源。
• AWS Lambda consumes Kinesis streams natively for event-driven workloads, replacing dedicated stream processors in many pipelinesAWS Lambda 原生消费 Kinesis 流处理事件驱动负载，在许多管道中取代专用流处理器。
• Apache Pulsar competes as a multi-tenant messaging and stream platform, offering an open alternative to the AWS native stackApache Pulsar 以多租户消息与流平台竞争，为 AWS 原生栈提供开源替代。
• Google Cloud Pub/Sub and Azure Event Hubs are the cloud rivals outside AWS, each offering managed ingestion for stream workloads谷歌云 Pub/Sub 与 Azure 事件中心是 AWS 之外的云对手，各自为流负载提供托管摄取。
• Amazon S3 serves as the primary landing destination for Firehose deliveries, and S3 Tables now host the streaming tables featureAmazon S3 充当 Firehose 投递的主要落点，S3 Tables 如今承载流式表功能。
• Amazon Timestream covers time-series database needs that overlap with the outputs of Kinesis analyticsAmazon Timestream 覆盖与 Kinesis 分析输出重叠的时序数据库需求。