Photo: Sena

Is Kafka TCP or UDP?

Kafka uses a binary protocol over TCP. The protocol defines all APIs as request response message pairs.

kafka.apache.org - Kafka protocol guide - Apache Kafka

What are the 4 types of design?

The Four types of design Business design. Enterprise design. Products design. Execution design. Aug 16, 2018

Last updated Apr 21, 2022

How big is $1 billion?

The USA meaning of a billion is a thousand million, or one followed by nine noughts (1,000,000,000).

Last updated Dec 2, 2019

Promotion

Kafka protocol guide

This document covers the wire protocol implemented in Kafka. It is meant to give a readable guide to the protocol that covers the available requests, their binary format, and the proper way to make use of them to implement a client. This document assumes you understand the basic design and terminology described here Kafka uses a binary protocol over TCP. The protocol defines all APIs as request response message pairs. All messages are size delimited and are made up of the following primitive types. The client initiates a socket connection and then writes a sequence of request messages and reads back the corresponding response message. No handshake is required on connection or disconnection. TCP is happier if you maintain persistent connections used for many requests to amortize the cost of the TCP handshake, but beyond this penalty connecting is pretty cheap. The client will likely need to maintain a connection to multiple brokers, as data is partitioned and the clients will need to talk to the server that has their data. However it should not generally be necessary to maintain multiple connections to a single broker from a single client instance (i.e. connection pooling). The server guarantees that on a single TCP connection, requests will be processed in the order they are sent and responses will return in that order as well. The broker's request processing allows only a single in-flight request per connection in order to guarantee this ordering. Note that clients can (and ideally should) use non-blocking IO to implement request pipelining and achieve higher throughput. i.e., clients can send requests even while awaiting responses for preceding requests since the outstanding requests will be buffered in the underlying OS socket buffer. All requests are initiated by the client, and result in a corresponding response message from the server except where noted. The server has a configurable maximum limit on request size and any request that exceeds this limit will result in the socket being disconnected. Kafka is a partitioned system so not all servers have the complete data set. Instead recall that topics are split into a pre-defined number of partitions, P, and each partition is replicated with some replication factor, N. Topic partitions themselves are just ordered "commit logs" numbered 0, 1, ..., P-1. All systems of this nature have the question of how a particular piece of data is assigned to a particular partition. Kafka clients directly control this assignment, the brokers themselves enforce no particular semantics of which messages should be published to a particular partition. Rather, to publish messages the client directly addresses messages to a particular partition, and when fetching messages, fetches from a particular partition. If two clients want to use the same partitioning scheme they must use the same method to compute the mapping of key to partition. These requests to publish or fetch data must be sent to the broker that is currently acting as the leader for a given partition. This condition is enforced by the broker, so a request for a particular partition to the wrong broker will result in an the NotLeaderForPartition error code (described below). How can the client find out which topics exist, what partitions they have, and which brokers currently host those partitions so that it can direct its requests to the right hosts? This information is dynamic, so you can't just configure each client with some static mapping file. Instead all Kafka brokers can answer a metadata request that describes the current state of the cluster: what topics there are, which partitions those topics have, which broker is the leader for those partitions, and the host and port information for these brokers. In other words, the client needs to somehow find one broker and that broker will tell the client about all the other brokers that exist and what partitions they host. This first broker may itself go down so the best practice for a client implementation is to take a list of two or three URLs to bootstrap from. The user can then choose to use a load balancer or just statically configure two or three of their Kafka hosts in the clients. The client does not need to keep polling to see if the cluster has changed; it can fetch metadata once when it is instantiated cache that metadata until it receives an error indicating that the metadata is out of date. This error can come in two forms: (1) a socket error indicating the client cannot communicate with a particular broker, (2) an error code in the response to a request indicating that this broker no longer hosts the partition for which data was requested. Cycle through a list of "bootstrap" Kafka URLs until we find one we can connect to. Fetch cluster metadata. Process fetch or produce requests, directing them to the appropriate broker based on the topic/partitions they send to or fetch from. If we get an appropriate error, refresh the metadata and try again.

As mentioned above the assignment of messages to partitions is something the producing client controls. That said, how should this functionality be exposed to the end-user?

Partitioning really serves two purposes in Kafka:

It balances data and request load over brokers It serves as a way to divvy up processing among consumer processes while allowing local state and preserving order within the partition. We call this semantic partitioning. For a given use case you may care about only one of these or both. To accomplish simple load balancing a simple approach would be for the client to just round robin requests over all brokers. Another alternative, in an environment where there are many more producers than brokers, would be to have each client chose a single partition at random and publish to that. This later strategy will result in far fewer TCP connections. Semantic partitioning means using some key in the message to assign messages to partitions. For example if you were processing a click message stream you might want to partition the stream by the user id so that all data for a particular user would go to a single consumer. To accomplish this the client can take a key associated with the message and use some hash of this key to choose the partition to which to deliver the message. Our APIs encourage batching small things together for efficiency. We have found this is a very significant performance win. Both our API to send messages and our API to fetch messages always work with a sequence of messages not a single message to encourage this. A clever client can make use of this and support an "asynchronous" mode in which it batches together messages sent individually and sends them in larger clumps. We go even further with this and allow the batching across multiple topics and partitions, so a produce request may contain data to append to many partitions and a fetch request may pull data from many partitions all at once. The client implementer can choose to ignore this and send everything one at a time if they like. Kafka has a "bidirectional" client compatibility policy. In other words, new clients can talk to old servers, and old clients can talk to new servers. This allows users to upgrade either clients or servers without experiencing any downtime. Since the Kafka protocol has changed over time, clients and servers need to agree on the schema of the message that they are sending over the wire. This is done through API versioning. Before each request is sent, the client sends the API key and the API version. These two 16-bit numbers, when taken together, uniquely identify the schema of the message to follow. The intention is that clients will support a range of API versions. When communicating with a particular broker, a given client should use the highest API version supported by both and indicate this version in their requests. The server will reject requests with a version it does not support, and will always respond to the client with exactly the protocol format it expects based on the version it included in its request. The intended upgrade path is that new features would first be rolled out on the server (with the older clients not making use of them) and then as newer clients are deployed these new features would gradually be taken advantage of. Note that KIP-482 tagged fields can be added to a request without incrementing the version number. This offers an additional way of evolving the message schema without breaking compatibility. Tagged fields do not take up any space when the field is not set. Therefore, if a field is rarely used, it is more efficient to make it a tagged field than to put it in the mandatory schema. However, tagged fields are ignored by recipients that don't know about them, which could pose a challenge if this is not the behavior that the sender wants. In such cases, a version bump may be more appropriate. In order to work against multiple broker versions, clients need to know what versions of various APIs a broker supports. The broker exposes this information since 0.10.0.0 as described in KIP-35. Clients should use the supported API versions information to choose the highest API version supported by both client and broker. If no such version exists, an error should be reported to the user. The following sequence may be used by a client to obtain supported API versions from a broker. Client sends ApiVersionsRequest to a broker after connection has been established with the broker. If SSL is enabled, this happens after SSL connection has been established. On receiving ApiVersionsRequest , a broker returns its full list of supported ApiKeys and versions regardless of current authentication state (e.g., before SASL authentication on an SASL listener, do note that no Kafka protocol requests may take place on an SSL listener before the SSL handshake is finished). If this is considered to leak information about the broker version a workaround is to use SSL with client authentication which is performed at an earlier stage of the connection where the ApiVersionRequest is not available. Also, note that broker versions older than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request. If multiple versions of an API are supported by broker and client, clients are recommended to use the latest version supported by the broker and itself. Deprecation of a protocol version is done by marking an API version as deprecated in the protocol documentation. Supported API versions obtained from a broker are only valid for the connection on which that information is obtained. In the event of disconnection, the client should obtain the information from the broker again, as the broker might have been upgraded/downgraded in the mean time.

The following sequence is used for SASL authentication:

Kafka ApiVersionsRequest may be sent by the client to obtain the version ranges of requests supported by the broker. This is optional. Kafka SaslHandshakeRequest containing the SASL mechanism for authentication is sent by the client. If the requested mechanism is not enabled in the server, the server responds with the list of supported mechanisms and closes the client connection. If the mechanism is enabled in the server, the server sends a successful response and continues with SASL authentication. The actual SASL authentication is now performed. If SaslHandshakeRequest version is v0, a series of SASL client and server tokens corresponding to the mechanism are sent as opaque packets without wrapping the messages with Kafka protocol headers. If SaslHandshakeRequest version is v1, the SaslAuthenticate request/response are used, where the actual SASL tokens are wrapped in the Kafka protocol. The error code in the final message from the broker will indicate if authentication succeeded or failed. If authentication succeeds, subsequent packets are handled as Kafka API requests. Otherwise, the client connection is closed. For interoperability with 0.9.0.x clients, the first packet received by the server is handled as a SASL/GSSAPI client token if it is not a valid Kafka request. SASL/GSSAPI authentication is performed starting with this packet, skipping the first two steps above.

The protocol is built out of the following primitive types.

Type Description BOOLEAN Represents a boolean value in a byte. Values 0 and 1 are used to represent false and true respectively. When reading a boolean value, any non-zero value is considered true. INT8 Represents an integer between -27 and 27-1 inclusive. INT16 Represents an integer between -215 and 215-1 inclusive. The values are encoded using two bytes in network byte order (big-endian). INT32 Represents an integer between -231 and 231-1 inclusive. The values are encoded using four bytes in network byte order (big-endian). INT64 Represents an integer between -263 and 263-1 inclusive. The values are encoded using eight bytes in network byte order (big-endian). UINT32 Represents an integer between 0 and 232-1 inclusive. The values are encoded using four bytes in network byte order (big-endian). VARINT Represents an integer between -231 and 231-1 inclusive. Encoding follows the variable-length zig-zag encoding from Google Protocol Buffers. VARLONG Represents an integer between -263 and 263-1 inclusive. Encoding follows the variable-length zig-zag encoding from Google Protocol Buffers. UUID Represents a type 4 immutable universally unique identifier (Uuid). The values are encoded using sixteen bytes in network byte order (big-endian). FLOAT64 Represents a double-precision 64-bit format IEEE 754 value. The values are encoded using eight bytes in network byte order (big-endian). STRING Represents a sequence of characters. First the length N is given as an INT16. Then N bytes follow which are the UTF-8 encoding of the character sequence. Length must not be negative. COMPACT_STRING Represents a sequence of characters. First the length N + 1 is given as an UNSIGNED_VARINT . Then N bytes follow which are the UTF-8 encoding of the character sequence. NULLABLE_STRING Represents a sequence of characters or null. For non-null strings, first the length N is given as an INT16. Then N bytes follow which are the UTF-8 encoding of the character sequence. A null value is encoded with length of -1 and there are no following bytes. COMPACT_NULLABLE_STRING Represents a sequence of characters. First the length N + 1 is given as an UNSIGNED_VARINT . Then N bytes follow which are the UTF-8 encoding of the character sequence. A null string is represented with a length of 0. BYTES Represents a raw sequence of bytes. First the length N is given as an INT32. Then N bytes follow. COMPACT_BYTES Represents a raw sequence of bytes. First the length N+1 is given as an UNSIGNED_VARINT.Then N bytes follow. NULLABLE_BYTES Represents a raw sequence of bytes or null. For non-null values, first the length N is given as an INT32. Then N bytes follow. A null value is encoded with length of -1 and there are no following bytes. COMPACT_NULLABLE_BYTES Represents a raw sequence of bytes. First the length N+1 is given as an UNSIGNED_VARINT.Then N bytes follow. A null object is represented with a length of 0. RECORDS Represents a sequence of Kafka records as NULLABLE_BYTES. For a detailed description of records see Message Sets. ARRAY Represents a sequence of objects of a given type T. Type T can be either a primitive type (e.g. STRING) or a structure. First, the length N is given as an INT32. Then N instances of type T follow. A null array is represented with a length of -1. In protocol documentation an array of T instances is referred to as [T]. COMPACT_ARRAY Represents a sequence of objects of a given type T. Type T can be either a primitive type (e.g. STRING) or a structure. First, the length N + 1 is given as an UNSIGNED_VARINT. Then N instances of type T follow. A null array is represented with a length of 0. In protocol documentation an array of T instances is referred to as [T]. The BNFs below give an exact context free grammar for the request and response binary format. The BNF is intentionally not compact in order to give human-readable name. As always in a BNF a sequence of productions indicates concatenation. When there are multiple possible productions these are separated with '|' and may be enclosed in parenthesis for grouping. The top-level definition is always given first and subsequent sub-parts are indented. All requests and responses originate from the following grammar which will be incrementally describe through the rest of this document:

RequestOrResponse => Size (RequestMessage | ResponseMessage) Size => int32

Field Description message_size The message_size field gives the size of the subsequent request or response message in bytes. The client can read requests by first reading this 4 byte size as an integer N, and then reading and parsing the subsequent N bytes of the request.

A description of the record batch format can be found here.

We use numeric codes to indicate what problem occurred on the server. These can be translated by the client into exceptions or whatever the appropriate error handling mechanism in the client language. Here is a table of the error codes currently in use: Error Code Retriable Description UNKNOWN_SERVER_ERROR -1 False The server experienced an unexpected error when processing the request. NONE 0 False OFFSET_OUT_OF_RANGE 1 False The requested offset is not within the range of offsets maintained by the server. CORRUPT_MESSAGE 2 True This message has failed its CRC checksum, exceeds the valid size, has a null key for a compacted topic, or is otherwise corrupt. UNKNOWN_TOPIC_OR_PARTITION 3 True This server does not host this topic-partition. INVALID_FETCH_SIZE 4 False The requested fetch size is invalid. LEADER_NOT_AVAILABLE 5 True There is no leader for this topic-partition as we are in the middle of a leadership election. NOT_LEADER_OR_FOLLOWER 6 True For requests intended only for the leader, this error indicates that the broker is not the current leader. For requests intended for any replica, this error indicates that the broker is not a replica of the topic partition. REQUEST_TIMED_OUT 7 True The request timed out. BROKER_NOT_AVAILABLE 8 False The broker is not available. REPLICA_NOT_AVAILABLE 9 True The replica is not available for the requested topic-partition. Produce/Fetch requests and other requests intended only for the leader or follower return NOT_LEADER_OR_FOLLOWER if the broker is not a replica of the topic-partition. MESSAGE_TOO_LARGE 10 False The request included a message larger than the max message size the server will accept. STALE_CONTROLLER_EPOCH 11 False The controller moved to another broker. OFFSET_METADATA_TOO_LARGE 12 False The metadata field of the offset request was too large. NETWORK_EXCEPTION 13 True The server disconnected before a response was received. COORDINATOR_LOAD_IN_PROGRESS 14 True The coordinator is loading and hence can't process requests. COORDINATOR_NOT_AVAILABLE 15 True The coordinator is not available. NOT_COORDINATOR 16 True This is not the correct coordinator. INVALID_TOPIC_EXCEPTION 17 False The request attempted to perform an operation on an invalid topic. RECORD_LIST_TOO_LARGE 18 False The request included message batch larger than the configured segment size on the server. NOT_ENOUGH_REPLICAS 19 True Messages are rejected since there are fewer in-sync replicas than required. NOT_ENOUGH_REPLICAS_AFTER_APPEND 20 True Messages are written to the log, but to fewer in-sync replicas than required. INVALID_REQUIRED_ACKS 21 False Produce request specified an invalid value for required acks. ILLEGAL_GENERATION 22 False Specified group generation id is not valid. INCONSISTENT_GROUP_PROTOCOL 23 False The group member's supported protocols are incompatible with those of existing members or first group member tried to join with empty protocol type or empty protocol list. INVALID_GROUP_ID 24 False The configured groupId is invalid. UNKNOWN_MEMBER_ID 25 False The coordinator is not aware of this member. INVALID_SESSION_TIMEOUT 26 False The session timeout is not within the range allowed by the broker (as configured by group.min.session.timeout.ms and group.max.session.timeout.ms). REBALANCE_IN_PROGRESS 27 False The group is rebalancing, so a rejoin is needed. INVALID_COMMIT_OFFSET_SIZE 28 False The committing offset data size is not valid. TOPIC_AUTHORIZATION_FAILED 29 False Topic authorization failed. GROUP_AUTHORIZATION_FAILED 30 False Group authorization failed. CLUSTER_AUTHORIZATION_FAILED 31 False Cluster authorization failed. INVALID_TIMESTAMP 32 False The timestamp of the message is out of acceptable range. UNSUPPORTED_SASL_MECHANISM 33 False The broker does not support the requested SASL mechanism. ILLEGAL_SASL_STATE 34 False Request is not valid given the current SASL state. UNSUPPORTED_VERSION 35 False The version of API is not supported. TOPIC_ALREADY_EXISTS 36 False Topic with this name already exists. INVALID_PARTITIONS 37 False Number of partitions is below 1. INVALID_REPLICATION_FACTOR 38 False Replication factor is below 1 or larger than the number of available brokers. INVALID_REPLICA_ASSIGNMENT 39 False Replica assignment is invalid. INVALID_CONFIG 40 False Configuration is invalid. NOT_CONTROLLER 41 True This is not the correct controller for this cluster. INVALID_REQUEST 42 False This most likely occurs because of a request being malformed by the client library or the message was sent to an incompatible broker. See the broker logs for more details. UNSUPPORTED_FOR_MESSAGE_FORMAT 43 False The message format version on the broker does not support the request. POLICY_VIOLATION 44 False Request parameters do not satisfy the configured policy. OUT_OF_ORDER_SEQUENCE_NUMBER 45 False The broker received an out of order sequence number. DUPLICATE_SEQUENCE_NUMBER 46 False The broker received a duplicate sequence number. INVALID_PRODUCER_EPOCH 47 False Producer attempted to produce with an old epoch. INVALID_TXN_STATE 48 False The producer attempted a transactional operation in an invalid state. INVALID_PRODUCER_ID_MAPPING 49 False The producer attempted to use a producer id which is not currently assigned to its transactional id. INVALID_TRANSACTION_TIMEOUT 50 False The transaction timeout is larger than the maximum value allowed by the broker (as configured by transaction.max.timeout.ms). CONCURRENT_TRANSACTIONS 51 False The producer attempted to update a transaction while another concurrent operation on the same transaction was ongoing. TRANSACTION_COORDINATOR_FENCED 52 False Indicates that the transaction coordinator sending a WriteTxnMarker is no longer the current coordinator for a given producer. TRANSACTIONAL_ID_AUTHORIZATION_FAILED 53 False Transactional Id authorization failed. SECURITY_DISABLED 54 False Security features are disabled. OPERATION_NOT_ATTEMPTED 55 False The broker did not attempt to execute this operation. This may happen for batched RPCs where some operations in the batch failed, causing the broker to respond without trying the rest. KAFKA_STORAGE_ERROR 56 True Disk error when trying to access log file on the disk. LOG_DIR_NOT_FOUND 57 False The user-specified log directory is not found in the broker config. SASL_AUTHENTICATION_FAILED 58 False SASL Authentication failed. UNKNOWN_PRODUCER_ID 59 False This exception is raised by the broker if it could not locate the producer metadata associated with the producerId in question. This could happen if, for instance, the producer's records were deleted because their retention time had elapsed. Once the last records of the producerId are removed, the producer's metadata is removed from the broker, and future appends by the producer will return this exception. REASSIGNMENT_IN_PROGRESS 60 False A partition reassignment is in progress. DELEGATION_TOKEN_AUTH_DISABLED 61 False Delegation Token feature is not enabled. DELEGATION_TOKEN_NOT_FOUND 62 False Delegation Token is not found on server. DELEGATION_TOKEN_OWNER_MISMATCH 63 False Specified Principal is not valid Owner/Renewer. DELEGATION_TOKEN_REQUEST_NOT_ALLOWED 64 False Delegation Token requests are not allowed on PLAINTEXT/1-way SSL channels and on delegation token authenticated channels. DELEGATION_TOKEN_AUTHORIZATION_FAILED 65 False Delegation Token authorization failed. DELEGATION_TOKEN_EXPIRED 66 False Delegation Token is expired. INVALID_PRINCIPAL_TYPE 67 False Supplied principalType is not supported. NON_EMPTY_GROUP 68 False The group is not empty. GROUP_ID_NOT_FOUND 69 False The group id does not exist. FETCH_SESSION_ID_NOT_FOUND 70 True The fetch session ID was not found. INVALID_FETCH_SESSION_EPOCH 71 True The fetch session epoch is invalid. LISTENER_NOT_FOUND 72 True There is no listener on the leader broker that matches the listener on which metadata request was processed. TOPIC_DELETION_DISABLED 73 False Topic deletion is disabled. FENCED_LEADER_EPOCH 74 True The leader epoch in the request is older than the epoch on the broker. UNKNOWN_LEADER_EPOCH 75 True The leader epoch in the request is newer than the epoch on the broker. UNSUPPORTED_COMPRESSION_TYPE 76 False The requesting client does not support the compression type of given partition. STALE_BROKER_EPOCH 77 False Broker epoch has changed. OFFSET_NOT_AVAILABLE 78 True The leader high watermark has not caught up from a recent leader election so the offsets cannot be guaranteed to be monotonically increasing. MEMBER_ID_REQUIRED 79 False The group member needs to have a valid member id before actually entering a consumer group. PREFERRED_LEADER_NOT_AVAILABLE 80 True The preferred leader was not available. GROUP_MAX_SIZE_REACHED 81 False The consumer group has reached its max size. FENCED_INSTANCE_ID 82 False The broker rejected this static consumer since another consumer with the same group.instance.id has registered with a different member.id. ELIGIBLE_LEADERS_NOT_AVAILABLE 83 True Eligible topic partition leaders are not available. ELECTION_NOT_NEEDED 84 True Leader election not needed for topic partition. NO_REASSIGNMENT_IN_PROGRESS 85 False No partition reassignment is in progress. GROUP_SUBSCRIBED_TO_TOPIC 86 False Deleting offsets of a topic is forbidden while the consumer group is actively subscribed to it. INVALID_RECORD 87 False This record has failed the validation on broker and hence will be rejected. UNSTABLE_OFFSET_COMMIT 88 True There are unstable offsets that need to be cleared. THROTTLING_QUOTA_EXCEEDED 89 True The throttling quota has been exceeded. PRODUCER_FENCED 90 False There is a newer producer with the same transactionalId which fences the current one. RESOURCE_NOT_FOUND 91 False A request illegally referred to a resource that does not exist. DUPLICATE_RESOURCE 92 False A request illegally referred to the same resource twice. UNACCEPTABLE_CREDENTIAL 93 False Requested credential would not meet criteria for acceptability. INCONSISTENT_VOTER_SET 94 False Indicates that the either the sender or recipient of a voter-only request is not one of the expected voters INVALID_UPDATE_VERSION 95 False The given update version was invalid. FEATURE_UPDATE_FAILED 96 False Unable to update finalized features due to an unexpected server error. PRINCIPAL_DESERIALIZATION_FAILURE 97 False Request principal deserialization failed during forwarding. This indicates an internal error on the broker cluster security setup. SNAPSHOT_NOT_FOUND 98 False Requested snapshot was not found POSITION_OUT_OF_RANGE 99 False Requested position is not greater than or equal to zero, and less than the size of the snapshot. UNKNOWN_TOPIC_ID 100 True This server does not host this topic ID. DUPLICATE_BROKER_REGISTRATION 101 False This broker ID is already in use. BROKER_ID_NOT_REGISTERED 102 False The given broker ID was not registered. INCONSISTENT_TOPIC_ID 103 True The log's topic ID did not match the topic ID in the request INCONSISTENT_CLUSTER_ID 104 False The clusterId in the request does not match that found on the server TRANSACTIONAL_ID_NOT_FOUND 105 False The transactionalId could not be found FETCH_SESSION_TOPIC_ID_ERROR 106 True The fetch session encountered inconsistent topic ID usage INELIGIBLE_REPLICA 107 False The new ISR contains at least one ineligible replica. NEW_LEADER_ELECTED 108 False The AlterPartition request successfully updated the partition state but the leader has changed. The following are the numeric codes that the ApiKey in the request can take for each of the below request types. This section gives details on each of the individual API Messages, their usage, their binary format, and the meaning of their fields.

Headers:

Request Header v0 => request_api_key request_api_version correlation_id request_api_key => INT16 request_api_version => INT16 correlation_id => INT32 Field Description request_api_key The API key of this request. request_api_version The API version of this request. correlation_id The correlation ID of this request. Request Header v1 => request_api_key request_api_version correlation_id client_id request_api_key => INT16 request_api_version => INT16 correlation_id => INT32 client_id => NULLABLE_STRING Field Description request_api_key The API key of this request. request_api_version The API version of this request. correlation_id The correlation ID of this request. client_id The client ID string. Request Header v2 => request_api_key request_api_version correlation_id client_id TAG_BUFFER request_api_key => INT16 request_api_version => INT16 correlation_id => INT32 client_id => NULLABLE_STRING Field Description request_api_key The API key of this request. request_api_version The API version of this request. correlation_id The correlation ID of this request. client_id The client ID string. _tagged_fields The tagged fields

Response Header v0 => correlation_id correlation_id => INT32

Field Description correlation_id The correlation ID of this response.

Response Header v1 => correlation_id TAG_BUFFER correlation_id => INT32

Field Description correlation_id The correlation ID of this response. _tagged_fields The tagged fields Produce Request (Version: 0) => acks timeout_ms [topic_data] acks => INT16 timeout_ms => INT32 topic_data => name [partition_data] name => STRING partition_data => index records index => INT32 records => RECORDS Field Description acks The number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR. timeout_ms The timeout to await a response in milliseconds. topic_data Each topic to produce to. name The topic name. partition_data Each partition to produce to. index The partition index. records The record data to be produced. Produce Request (Version: 1) => acks timeout_ms [topic_data] acks => INT16 timeout_ms => INT32 topic_data => name [partition_data] name => STRING partition_data => index records index => INT32 records => RECORDS Field Description acks The number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR. timeout_ms The timeout to await a response in milliseconds. topic_data Each topic to produce to. name The topic name. partition_data Each partition to produce to. index The partition index. records The record data to be produced. Produce Request (Version: 2) => acks timeout_ms [topic_data] acks => INT16 timeout_ms => INT32 topic_data => name [partition_data] name => STRING partition_data => index records index => INT32 records => RECORDS Field Description acks The number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR. timeout_ms The timeout to await a response in milliseconds. topic_data Each topic to produce to. name The topic name. partition_data Each partition to produce to. index The partition index. records The record data to be produced.

What is the richest job for a woman?

Top 25 Best Paying Jobs For Women Chief Executive Officer (CEO) Average Annual Salary: $207,000. ... Pharmacist. Average Annual Salary: $125,000....

Last updated Nov 30, 2021

What should I study for social media marketing?

Top 12 Social Media Skills Excellent Communication. The first social media skill you must have is - communication. ... Creativity. One of the most...

Last updated Sep 4, 2018

Promotion

Produce Request (Version: 3) => transactional_id acks timeout_ms [topic_data] transactional_id => NULLABLE_STRING acks => INT16 timeout_ms => INT32 topic_data => name [partition_data] name => STRING partition_data => index records index => INT32 records => RECORDS Field Description transactional_id The transactional ID, or null if the producer is not transactional. acks The number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR. timeout_ms The timeout to await a response in milliseconds. topic_data Each topic to produce to. name The topic name. partition_data Each partition to produce to. index The partition index. records The record data to be produced. Produce Request (Version: 4) => transactional_id acks timeout_ms [topic_data] transactional_id => NULLABLE_STRING acks => INT16 timeout_ms => INT32 topic_data => name [partition_data] name => STRING partition_data => index records index => INT32 records => RECORDS Field Description transactional_id The transactional ID, or null if the producer is not transactional. acks The number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR. timeout_ms The timeout to await a response in milliseconds. topic_data Each topic to produce to. name The topic name. partition_data Each partition to produce to. index The partition index. records The record data to be produced. Produce Request (Version: 5) => transactional_id acks timeout_ms [topic_data] transactional_id => NULLABLE_STRING acks => INT16 timeout_ms => INT32 topic_data => name [partition_data] name => STRING partition_data => index records index => INT32 records => RECORDS Field Description transactional_id The transactional ID, or null if the producer is not transactional. acks The number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR. timeout_ms The timeout to await a response in milliseconds. topic_data Each topic to produce to. name The topic name. partition_data Each partition to produce to. index The partition index. records The record data to be produced. Produce Request (Version: 6) => transactional_id acks timeout_ms [topic_data] transactional_id => NULLABLE_STRING acks => INT16 timeout_ms => INT32 topic_data => name [partition_data] name => STRING partition_data => index records index => INT32 records => RECORDS Field Description transactional_id The transactional ID, or null if the producer is not transactional. acks The number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR. timeout_ms The timeout to await a response in milliseconds. topic_data Each topic to produce to. name The topic name. partition_data Each partition to produce to. index The partition index. records The record data to be produced. Produce Request (Version: 7) => transactional_id acks timeout_ms [topic_data] transactional_id => NULLABLE_STRING acks => INT16 timeout_ms => INT32 topic_data => name [partition_data] name => STRING partition_data => index records index => INT32 records => RECORDS Field Description transactional_id The transactional ID, or null if the producer is not transactional. acks The number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR. timeout_ms The timeout to await a response in milliseconds. topic_data Each topic to produce to. name The topic name. partition_data Each partition to produce to. index The partition index. records The record data to be produced. Produce Request (Version: 8) => transactional_id acks timeout_ms [topic_data] transactional_id => NULLABLE_STRING acks => INT16 timeout_ms => INT32 topic_data => name [partition_data] name => STRING partition_data => index records index => INT32 records => RECORDS Field Description transactional_id The transactional ID, or null if the producer is not transactional. acks The number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR. timeout_ms The timeout to await a response in milliseconds. topic_data Each topic to produce to. name The topic name. partition_data Each partition to produce to. index The partition index. records The record data to be produced. Produce Request (Version: 9) => transactional_id acks timeout_ms [topic_data] TAG_BUFFER transactional_id => COMPACT_NULLABLE_STRING acks => INT16 timeout_ms => INT32 topic_data => name [partition_data] TAG_BUFFER name => COMPACT_STRING partition_data => index records TAG_BUFFER index => INT32 records => COMPACT_RECORDS Field Description transactional_id The transactional ID, or null if the producer is not transactional. acks The number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR. timeout_ms The timeout to await a response in milliseconds. topic_data Each topic to produce to. name The topic name. partition_data Each partition to produce to. index The partition index. records The record data to be produced. _tagged_fields The tagged fields _tagged_fields The tagged fields _tagged_fields The tagged fields Produce Response (Version: 0) => [responses] responses => name [partition_responses] name => STRING partition_responses => index error_code base_offset index => INT32 error_code => INT16 base_offset => INT64 Field Description responses Each produce response name The topic name partition_responses Each partition that we produced to within the topic. index The partition index. error_code The error code, or 0 if there was no error. base_offset The base offset. Produce Response (Version: 1) => [responses] throttle_time_ms responses => name [partition_responses] name => STRING partition_responses => index error_code base_offset index => INT32 error_code => INT16 base_offset => INT64 throttle_time_ms => INT32 Field Description responses Each produce response name The topic name partition_responses Each partition that we produced to within the topic. index The partition index. error_code The error code, or 0 if there was no error. base_offset The base offset. throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. Produce Response (Version: 2) => [responses] throttle_time_ms responses => name [partition_responses] name => STRING partition_responses => index error_code base_offset log_append_time_ms index => INT32 error_code => INT16 base_offset => INT64 log_append_time_ms => INT64 throttle_time_ms => INT32 Field Description responses Each produce response name The topic name partition_responses Each partition that we produced to within the topic. index The partition index. error_code The error code, or 0 if there was no error. base_offset The base offset. log_append_time_ms The timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended. throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. Produce Response (Version: 3) => [responses] throttle_time_ms responses => name [partition_responses] name => STRING partition_responses => index error_code base_offset log_append_time_ms index => INT32 error_code => INT16 base_offset => INT64 log_append_time_ms => INT64 throttle_time_ms => INT32 Field Description responses Each produce response name The topic name partition_responses Each partition that we produced to within the topic. index The partition index. error_code The error code, or 0 if there was no error. base_offset The base offset. log_append_time_ms The timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended. throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. Produce Response (Version: 4) => [responses] throttle_time_ms responses => name [partition_responses] name => STRING partition_responses => index error_code base_offset log_append_time_ms index => INT32 error_code => INT16 base_offset => INT64 log_append_time_ms => INT64 throttle_time_ms => INT32 Field Description responses Each produce response name The topic name partition_responses Each partition that we produced to within the topic. index The partition index. error_code The error code, or 0 if there was no error. base_offset The base offset. log_append_time_ms The timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended. throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. Produce Response (Version: 5) => [responses] throttle_time_ms responses => name [partition_responses] name => STRING partition_responses => index error_code base_offset log_append_time_ms log_start_offset index => INT32 error_code => INT16 base_offset => INT64 log_append_time_ms => INT64 log_start_offset => INT64 throttle_time_ms => INT32 Field Description responses Each produce response name The topic name partition_responses Each partition that we produced to within the topic. index The partition index. error_code The error code, or 0 if there was no error. base_offset The base offset. log_append_time_ms The timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended. log_start_offset The log start offset. throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. Produce Response (Version: 6) => [responses] throttle_time_ms responses => name [partition_responses] name => STRING partition_responses => index error_code base_offset log_append_time_ms log_start_offset index => INT32 error_code => INT16 base_offset => INT64 log_append_time_ms => INT64 log_start_offset => INT64 throttle_time_ms => INT32 Field Description responses Each produce response name The topic name partition_responses Each partition that we produced to within the topic. index The partition index. error_code The error code, or 0 if there was no error. base_offset The base offset. log_append_time_ms The timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended. log_start_offset The log start offset. throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. Produce Response (Version: 7) => [responses] throttle_time_ms responses => name [partition_responses] name => STRING partition_responses => index error_code base_offset log_append_time_ms log_start_offset index => INT32 error_code => INT16 base_offset => INT64 log_append_time_ms => INT64 log_start_offset => INT64 throttle_time_ms => INT32 Field Description responses Each produce response name The topic name partition_responses Each partition that we produced to within the topic. index The partition index. error_code The error code, or 0 if there was no error. base_offset The base offset. log_append_time_ms The timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended. log_start_offset The log start offset. throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. Produce Response (Version: 8) => [responses] throttle_time_ms responses => name [partition_responses] name => STRING partition_responses => index error_code base_offset log_append_time_ms log_start_offset [record_errors] error_message index => INT32 error_code => INT16 base_offset => INT64 log_append_time_ms => INT64 log_start_offset => INT64 record_errors => batch_index batch_index_error_message batch_index => INT32 batch_index_error_message => NULLABLE_STRING error_message => NULLABLE_STRING throttle_time_ms => INT32 Field Description responses Each produce response name The topic name partition_responses Each partition that we produced to within the topic. index The partition index. error_code The error code, or 0 if there was no error. base_offset The base offset. log_append_time_ms The timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended. log_start_offset The log start offset. record_errors The batch indices of records that caused the batch to be dropped batch_index The batch index of the record that cause the batch to be dropped batch_index_error_message The error message of the record that caused the batch to be dropped error_message The global error message summarizing the common root cause of the records that caused the batch to be dropped throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. Produce Response (Version: 9) => [responses] throttle_time_ms TAG_BUFFER responses => name [partition_responses] TAG_BUFFER name => COMPACT_STRING partition_responses => index error_code base_offset log_append_time_ms log_start_offset [record_errors] error_message TAG_BUFFER index => INT32 error_code => INT16 base_offset => INT64 log_append_time_ms => INT64 log_start_offset => INT64 record_errors => batch_index batch_index_error_message TAG_BUFFER batch_index => INT32 batch_index_error_message => COMPACT_NULLABLE_STRING error_message => COMPACT_NULLABLE_STRING throttle_time_ms => INT32 Field Description responses Each produce response name The topic name partition_responses Each partition that we produced to within the topic. index The partition index. error_code The error code, or 0 if there was no error. base_offset The base offset. log_append_time_ms The timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended. log_start_offset The log start offset. record_errors The batch indices of records that caused the batch to be dropped batch_index The batch index of the record that cause the batch to be dropped batch_index_error_message The error message of the record that caused the batch to be dropped _tagged_fields The tagged fields error_message The global error message summarizing the common root cause of the records that caused the batch to be dropped _tagged_fields The tagged fields _tagged_fields The tagged fields throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. _tagged_fields The tagged fields Fetch Request (Version: 0) => replica_id max_wait_ms min_bytes [topics] replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 topics => topic [partitions] topic => STRING partitions => partition fetch_offset partition_max_bytes partition => INT32 fetch_offset => INT64 partition_max_bytes => INT32 Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. topics The topics to fetch. topic The name of the topic to fetch. partitions The partitions to fetch. partition The partition index. fetch_offset The message offset. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. Fetch Request (Version: 1) => replica_id max_wait_ms min_bytes [topics] replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 topics => topic [partitions] topic => STRING partitions => partition fetch_offset partition_max_bytes partition => INT32 fetch_offset => INT64 partition_max_bytes => INT32 Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. topics The topics to fetch. topic The name of the topic to fetch. partitions The partitions to fetch. partition The partition index. fetch_offset The message offset. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. Fetch Request (Version: 2) => replica_id max_wait_ms min_bytes [topics] replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 topics => topic [partitions] topic => STRING partitions => partition fetch_offset partition_max_bytes partition => INT32 fetch_offset => INT64 partition_max_bytes => INT32 Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. topics The topics to fetch. topic The name of the topic to fetch. partitions The partitions to fetch. partition The partition index. fetch_offset The message offset. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. Fetch Request (Version: 3) => replica_id max_wait_ms min_bytes max_bytes [topics] replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 max_bytes => INT32 topics => topic [partitions] topic => STRING partitions => partition fetch_offset partition_max_bytes partition => INT32 fetch_offset => INT64 partition_max_bytes => INT32 Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. max_bytes The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored. topics The topics to fetch. topic The name of the topic to fetch. partitions The partitions to fetch. partition The partition index. fetch_offset The message offset. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. Fetch Request (Version: 4) => replica_id max_wait_ms min_bytes max_bytes isolation_level [topics] replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 max_bytes => INT32 isolation_level => INT8 topics => topic [partitions] topic => STRING partitions => partition fetch_offset partition_max_bytes partition => INT32 fetch_offset => INT64 partition_max_bytes => INT32 Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. max_bytes The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records topics The topics to fetch. topic The name of the topic to fetch. partitions The partitions to fetch. partition The partition index. fetch_offset The message offset. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. Fetch Request (Version: 5) => replica_id max_wait_ms min_bytes max_bytes isolation_level [topics] replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 max_bytes => INT32 isolation_level => INT8 topics => topic [partitions] topic => STRING partitions => partition fetch_offset log_start_offset partition_max_bytes partition => INT32 fetch_offset => INT64 log_start_offset => INT64 partition_max_bytes => INT32 Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. max_bytes The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records topics The topics to fetch. topic The name of the topic to fetch. partitions The partitions to fetch. partition The partition index. fetch_offset The message offset. log_start_offset The earliest available offset of the follower replica. The field is only used when the request is sent by the follower. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. Fetch Request (Version: 6) => replica_id max_wait_ms min_bytes max_bytes isolation_level [topics] replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 max_bytes => INT32 isolation_level => INT8 topics => topic [partitions] topic => STRING partitions => partition fetch_offset log_start_offset partition_max_bytes partition => INT32 fetch_offset => INT64 log_start_offset => INT64 partition_max_bytes => INT32 Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. max_bytes The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records topics The topics to fetch. topic The name of the topic to fetch. partitions The partitions to fetch. partition The partition index. fetch_offset The message offset. log_start_offset The earliest available offset of the follower replica. The field is only used when the request is sent by the follower. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. Fetch Request (Version: 7) => replica_id max_wait_ms min_bytes max_bytes isolation_level session_id session_epoch [topics] [forgotten_topics_data] replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 max_bytes => INT32 isolation_level => INT8 session_id => INT32 session_epoch => INT32 topics => topic [partitions] topic => STRING partitions => partition fetch_offset log_start_offset partition_max_bytes partition => INT32 fetch_offset => INT64 log_start_offset => INT64 partition_max_bytes => INT32 forgotten_topics_data => topic [partitions] topic => STRING partitions => INT32 Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. max_bytes The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records session_id The fetch session ID. session_epoch The fetch session epoch, which is used for ordering requests in a session. topics The topics to fetch. topic The name of the topic to fetch. partitions The partitions to fetch. partition The partition index. fetch_offset The message offset. log_start_offset The earliest available offset of the follower replica. The field is only used when the request is sent by the follower. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. forgotten_topics_data In an incremental fetch request, the partitions to remove. topic The topic name. partitions The partitions indexes to forget. Fetch Request (Version: 8) => replica_id max_wait_ms min_bytes max_bytes isolation_level session_id session_epoch [topics] [forgotten_topics_data] replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 max_bytes => INT32 isolation_level => INT8 session_id => INT32 session_epoch => INT32 topics => topic [partitions] topic => STRING partitions => partition fetch_offset log_start_offset partition_max_bytes partition => INT32 fetch_offset => INT64 log_start_offset => INT64 partition_max_bytes => INT32 forgotten_topics_data => topic [partitions] topic => STRING partitions => INT32 Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. max_bytes The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records session_id The fetch session ID. session_epoch The fetch session epoch, which is used for ordering requests in a session. topics The topics to fetch. topic The name of the topic to fetch. partitions The partitions to fetch. partition The partition index. fetch_offset The message offset. log_start_offset The earliest available offset of the follower replica. The field is only used when the request is sent by the follower. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. forgotten_topics_data In an incremental fetch request, the partitions to remove. topic The topic name. partitions The partitions indexes to forget. Fetch Request (Version: 9) => replica_id max_wait_ms min_bytes max_bytes isolation_level session_id session_epoch [topics] [forgotten_topics_data] replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 max_bytes => INT32 isolation_level => INT8 session_id => INT32 session_epoch => INT32 topics => topic [partitions] topic => STRING partitions => partition current_leader_epoch fetch_offset log_start_offset partition_max_bytes partition => INT32 current_leader_epoch => INT32 fetch_offset => INT64 log_start_offset => INT64 partition_max_bytes => INT32 forgotten_topics_data => topic [partitions] topic => STRING partitions => INT32 Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. max_bytes The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records session_id The fetch session ID. session_epoch The fetch session epoch, which is used for ordering requests in a session. topics The topics to fetch. topic The name of the topic to fetch. partitions The partitions to fetch. partition The partition index. current_leader_epoch The current leader epoch of the partition. fetch_offset The message offset. log_start_offset The earliest available offset of the follower replica. The field is only used when the request is sent by the follower. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. forgotten_topics_data In an incremental fetch request, the partitions to remove. topic The topic name. partitions The partitions indexes to forget. Fetch Request (Version: 10) => replica_id max_wait_ms min_bytes max_bytes isolation_level session_id session_epoch [topics] [forgotten_topics_data] replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 max_bytes => INT32 isolation_level => INT8 session_id => INT32 session_epoch => INT32 topics => topic [partitions] topic => STRING partitions => partition current_leader_epoch fetch_offset log_start_offset partition_max_bytes partition => INT32 current_leader_epoch => INT32 fetch_offset => INT64 log_start_offset => INT64 partition_max_bytes => INT32 forgotten_topics_data => topic [partitions] topic => STRING partitions => INT32 Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. max_bytes The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records session_id The fetch session ID. session_epoch The fetch session epoch, which is used for ordering requests in a session. topics The topics to fetch. topic The name of the topic to fetch. partitions The partitions to fetch. partition The partition index. current_leader_epoch The current leader epoch of the partition. fetch_offset The message offset. log_start_offset The earliest available offset of the follower replica. The field is only used when the request is sent by the follower. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. forgotten_topics_data In an incremental fetch request, the partitions to remove. topic The topic name. partitions The partitions indexes to forget. Fetch Request (Version: 11) => replica_id max_wait_ms min_bytes max_bytes isolation_level session_id session_epoch [topics] [forgotten_topics_data] rack_id replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 max_bytes => INT32 isolation_level => INT8 session_id => INT32 session_epoch => INT32 topics => topic [partitions] topic => STRING partitions => partition current_leader_epoch fetch_offset log_start_offset partition_max_bytes partition => INT32 current_leader_epoch => INT32 fetch_offset => INT64 log_start_offset => INT64 partition_max_bytes => INT32 forgotten_topics_data => topic [partitions] topic => STRING partitions => INT32 rack_id => STRING Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. max_bytes The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records session_id The fetch session ID. session_epoch The fetch session epoch, which is used for ordering requests in a session. topics The topics to fetch. topic The name of the topic to fetch. partitions The partitions to fetch. partition The partition index. current_leader_epoch The current leader epoch of the partition. fetch_offset The message offset. log_start_offset The earliest available offset of the follower replica. The field is only used when the request is sent by the follower. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. forgotten_topics_data In an incremental fetch request, the partitions to remove. topic The topic name. partitions The partitions indexes to forget. rack_id Rack ID of the consumer making this request

What platforms can you self-publish on?

Here's a list of the best self-publishing companies for authors: Kindle Direct Publishing (KDP) Barnes and Noble Press. Kobo. Apple Books. Self-...

Last updated Feb 1, 2021

Which is the best field in media?

media jobs offer a decent occupational outlook and a living wage. ... Interpreter/Translator. ... Film/Video Editor. ... Technical Writer. ......

Last updated Dec 3, 2021

Promotion

Fetch Request (Version: 12) => replica_id max_wait_ms min_bytes max_bytes isolation_level session_id session_epoch [topics] [forgotten_topics_data] rack_id TAG_BUFFER replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 max_bytes => INT32 isolation_level => INT8 session_id => INT32 session_epoch => INT32 topics => topic [partitions] TAG_BUFFER topic => COMPACT_STRING partitions => partition current_leader_epoch fetch_offset last_fetched_epoch log_start_offset partition_max_bytes TAG_BUFFER partition => INT32 current_leader_epoch => INT32 fetch_offset => INT64 last_fetched_epoch => INT32 log_start_offset => INT64 partition_max_bytes => INT32 forgotten_topics_data => topic [partitions] TAG_BUFFER topic => COMPACT_STRING partitions => INT32 rack_id => COMPACT_STRING Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. max_bytes The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records session_id The fetch session ID. session_epoch The fetch session epoch, which is used for ordering requests in a session. topics The topics to fetch. topic The name of the topic to fetch. partitions The partitions to fetch. partition The partition index. current_leader_epoch The current leader epoch of the partition. fetch_offset The message offset. last_fetched_epoch The epoch of the last fetched record or -1 if there is none log_start_offset The earliest available offset of the follower replica. The field is only used when the request is sent by the follower. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. _tagged_fields The tagged fields _tagged_fields The tagged fields forgotten_topics_data In an incremental fetch request, the partitions to remove. topic The topic name. partitions The partitions indexes to forget. _tagged_fields The tagged fields rack_id Rack ID of the consumer making this request _tagged_fields The tagged fields Fetch Request (Version: 13) => replica_id max_wait_ms min_bytes max_bytes isolation_level session_id session_epoch [topics] [forgotten_topics_data] rack_id TAG_BUFFER replica_id => INT32 max_wait_ms => INT32 min_bytes => INT32 max_bytes => INT32 isolation_level => INT8 session_id => INT32 session_epoch => INT32 topics => topic_id [partitions] TAG_BUFFER topic_id => UUID partitions => partition current_leader_epoch fetch_offset last_fetched_epoch log_start_offset partition_max_bytes TAG_BUFFER partition => INT32 current_leader_epoch => INT32 fetch_offset => INT64 last_fetched_epoch => INT32 log_start_offset => INT64 partition_max_bytes => INT32 forgotten_topics_data => topic_id [partitions] TAG_BUFFER topic_id => UUID partitions => INT32 rack_id => COMPACT_STRING Field Description replica_id The broker ID of the follower, of -1 if this request is from a consumer. max_wait_ms The maximum time in milliseconds to wait for the response. min_bytes The minimum bytes to accumulate in the response. max_bytes The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records session_id The fetch session ID. session_epoch The fetch session epoch, which is used for ordering requests in a session. topics The topics to fetch. topic_id The unique topic ID partitions The partitions to fetch. partition The partition index. current_leader_epoch The current leader epoch of the partition. fetch_offset The message offset. last_fetched_epoch The epoch of the last fetched record or -1 if there is none log_start_offset The earliest available offset of the follower replica. The field is only used when the request is sent by the follower. partition_max_bytes The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored. _tagged_fields The tagged fields _tagged_fields The tagged fields forgotten_topics_data In an incremental fetch request, the partitions to remove. topic_id The unique topic ID partitions The partitions indexes to forget. _tagged_fields The tagged fields rack_id Rack ID of the consumer making this request _tagged_fields The tagged fields Fetch Response (Version: 0) => [responses] responses => topic [partitions] topic => STRING partitions => partition_index error_code high_watermark records partition_index => INT32 error_code => INT16 high_watermark => INT64 records => RECORDS Field Description responses The response topics. topic The topic name. partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. records The record data. Fetch Response (Version: 1) => throttle_time_ms [responses] throttle_time_ms => INT32 responses => topic [partitions] topic => STRING partitions => partition_index error_code high_watermark records partition_index => INT32 error_code => INT16 high_watermark => INT64 records => RECORDS Field Description throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. responses The response topics. topic The topic name. partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. records The record data. Fetch Response (Version: 2) => throttle_time_ms [responses] throttle_time_ms => INT32 responses => topic [partitions] topic => STRING partitions => partition_index error_code high_watermark records partition_index => INT32 error_code => INT16 high_watermark => INT64 records => RECORDS Field Description throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. responses The response topics. topic The topic name. partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. records The record data. Fetch Response (Version: 3) => throttle_time_ms [responses] throttle_time_ms => INT32 responses => topic [partitions] topic => STRING partitions => partition_index error_code high_watermark records partition_index => INT32 error_code => INT16 high_watermark => INT64 records => RECORDS Field Description throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. responses The response topics. topic The topic name. partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. records The record data. Fetch Response (Version: 4) => throttle_time_ms [responses] throttle_time_ms => INT32 responses => topic [partitions] topic => STRING partitions => partition_index error_code high_watermark last_stable_offset [aborted_transactions] records partition_index => INT32 error_code => INT16 high_watermark => INT64 last_stable_offset => INT64 aborted_transactions => producer_id first_offset producer_id => INT64 first_offset => INT64 records => RECORDS Field Description throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. responses The response topics. topic The topic name. partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. last_stable_offset The last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED) aborted_transactions The aborted transactions. producer_id The producer id associated with the aborted transaction. first_offset The first offset in the aborted transaction. records The record data. Fetch Response (Version: 5) => throttle_time_ms [responses] throttle_time_ms => INT32 responses => topic [partitions] topic => STRING partitions => partition_index error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] records partition_index => INT32 error_code => INT16 high_watermark => INT64 last_stable_offset => INT64 log_start_offset => INT64 aborted_transactions => producer_id first_offset producer_id => INT64 first_offset => INT64 records => RECORDS Field Description throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. responses The response topics. topic The topic name. partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. last_stable_offset The last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED) log_start_offset The current log start offset. aborted_transactions The aborted transactions. producer_id The producer id associated with the aborted transaction. first_offset The first offset in the aborted transaction. records The record data. Fetch Response (Version: 6) => throttle_time_ms [responses] throttle_time_ms => INT32 responses => topic [partitions] topic => STRING partitions => partition_index error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] records partition_index => INT32 error_code => INT16 high_watermark => INT64 last_stable_offset => INT64 log_start_offset => INT64 aborted_transactions => producer_id first_offset producer_id => INT64 first_offset => INT64 records => RECORDS Field Description throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. responses The response topics. topic The topic name. partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. last_stable_offset The last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED) log_start_offset The current log start offset. aborted_transactions The aborted transactions. producer_id The producer id associated with the aborted transaction. first_offset The first offset in the aborted transaction. records The record data. Fetch Response (Version: 7) => throttle_time_ms error_code session_id [responses] throttle_time_ms => INT32 error_code => INT16 session_id => INT32 responses => topic [partitions] topic => STRING partitions => partition_index error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] records partition_index => INT32 error_code => INT16 high_watermark => INT64 last_stable_offset => INT64 log_start_offset => INT64 aborted_transactions => producer_id first_offset producer_id => INT64 first_offset => INT64 records => RECORDS Field Description throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. error_code The top level response error code. session_id The fetch session ID, or 0 if this is not part of a fetch session. responses The response topics. topic The topic name. partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. last_stable_offset The last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED) log_start_offset The current log start offset. aborted_transactions The aborted transactions. producer_id The producer id associated with the aborted transaction. first_offset The first offset in the aborted transaction. records The record data. Fetch Response (Version: 8) => throttle_time_ms error_code session_id [responses] throttle_time_ms => INT32 error_code => INT16 session_id => INT32 responses => topic [partitions] topic => STRING partitions => partition_index error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] records partition_index => INT32 error_code => INT16 high_watermark => INT64 last_stable_offset => INT64 log_start_offset => INT64 aborted_transactions => producer_id first_offset producer_id => INT64 first_offset => INT64 records => RECORDS Field Description throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. error_code The top level response error code. session_id The fetch session ID, or 0 if this is not part of a fetch session. responses The response topics. topic The topic name. partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. last_stable_offset The last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED) log_start_offset The current log start offset. aborted_transactions The aborted transactions. producer_id The producer id associated with the aborted transaction. first_offset The first offset in the aborted transaction. records The record data. Fetch Response (Version: 9) => throttle_time_ms error_code session_id [responses] throttle_time_ms => INT32 error_code => INT16 session_id => INT32 responses => topic [partitions] topic => STRING partitions => partition_index error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] records partition_index => INT32 error_code => INT16 high_watermark => INT64 last_stable_offset => INT64 log_start_offset => INT64 aborted_transactions => producer_id first_offset producer_id => INT64 first_offset => INT64 records => RECORDS Field Description throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. error_code The top level response error code. session_id The fetch session ID, or 0 if this is not part of a fetch session. responses The response topics. topic The topic name. partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. last_stable_offset The last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED) log_start_offset The current log start offset. aborted_transactions The aborted transactions. producer_id The producer id associated with the aborted transaction. first_offset The first offset in the aborted transaction. records The record data. Fetch Response (Version: 10) => throttle_time_ms error_code session_id [responses] throttle_time_ms => INT32 error_code => INT16 session_id => INT32 responses => topic [partitions] topic => STRING partitions => partition_index error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] records partition_index => INT32 error_code => INT16 high_watermark => INT64 last_stable_offset => INT64 log_start_offset => INT64 aborted_transactions => producer_id first_offset producer_id => INT64 first_offset => INT64 records => RECORDS Field Description throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. error_code The top level response error code. session_id The fetch session ID, or 0 if this is not part of a fetch session. responses The response topics. topic The topic name. partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. last_stable_offset The last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED) log_start_offset The current log start offset. aborted_transactions The aborted transactions. producer_id The producer id associated with the aborted transaction. first_offset The first offset in the aborted transaction. records The record data. Fetch Response (Version: 11) => throttle_time_ms error_code session_id [responses] throttle_time_ms => INT32 error_code => INT16 session_id => INT32 responses => topic [partitions] topic => STRING partitions => partition_index error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] preferred_read_replica records partition_index => INT32 error_code => INT16 high_watermark => INT64 last_stable_offset => INT64 log_start_offset => INT64 aborted_transactions => producer_id first_offset producer_id => INT64 first_offset => INT64 preferred_read_replica => INT32 records => RECORDS Field Description throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. error_code The top level response error code. session_id The fetch session ID, or 0 if this is not part of a fetch session. responses The response topics. topic The topic name. partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. last_stable_offset The last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED) log_start_offset The current log start offset. aborted_transactions The aborted transactions. producer_id The producer id associated with the aborted transaction. first_offset The first offset in the aborted transaction. preferred_read_replica The preferred read replica for the consumer to use on its next fetch request records The record data. Fetch Response (Version: 12) => throttle_time_ms error_code session_id [responses] TAG_BUFFER throttle_time_ms => INT32 error_code => INT16 session_id => INT32 responses => topic [partitions] TAG_BUFFER topic => COMPACT_STRING partitions => partition_index error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] preferred_read_replica records TAG_BUFFER partition_index => INT32 error_code => INT16 high_watermark => INT64 last_stable_offset => INT64 log_start_offset => INT64 aborted_transactions => producer_id first_offset TAG_BUFFER producer_id => INT64 first_offset => INT64 preferred_read_replica => INT32 records => COMPACT_RECORDS Field Description throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. error_code The top level response error code. session_id The fetch session ID, or 0 if this is not part of a fetch session. responses The response topics. topic The topic name. partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. last_stable_offset The last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED) log_start_offset The current log start offset. aborted_transactions The aborted transactions. producer_id The producer id associated with the aborted transaction. first_offset The first offset in the aborted transaction. _tagged_fields The tagged fields preferred_read_replica The preferred read replica for the consumer to use on its next fetch request records The record data. _tagged_fields The tagged fields _tagged_fields The tagged fields _tagged_fields The tagged fields Fetch Response (Version: 13) => throttle_time_ms error_code session_id [responses] TAG_BUFFER throttle_time_ms => INT32 error_code => INT16 session_id => INT32 responses => topic_id [partitions] TAG_BUFFER topic_id => UUID partitions => partition_index error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] preferred_read_replica records TAG_BUFFER partition_index => INT32 error_code => INT16 high_watermark => INT64 last_stable_offset => INT64 log_start_offset => INT64 aborted_transactions => producer_id first_offset TAG_BUFFER producer_id => INT64 first_offset => INT64 preferred_read_replica => INT32 records => COMPACT_RECORDS Field Description throttle_time_ms The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota. error_code The top level response error code. session_id The fetch session ID, or 0 if this is not part of a fetch session. responses The response topics. topic_id The unique topic ID partitions The topic partitions. partition_index The partition index. error_code The error code, or 0 if there was no fetch error. high_watermark The current high water mark. last_stable_offset The last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED) log_start_offset The current log start offset. aborted_transactions The aborted transactions. producer_id The producer id associated with the aborted transaction. first_offset The first offset in the aborted transaction. _tagged_fields The tagged fields preferred_read_replica The preferred read replica for the consumer to use on its next fetch request records The record data. _tagged_fields The tagged fields _tagged_fields The tagged fields _tagged_fields The tagged fields ListOffsets Request (Version: 0) => replica_id [topics] replica_id => INT32 topics => name [partitions] name => STRING partitions => partition_index timestamp max_num_offsets partition_index => INT32 timestamp => INT64 max_num_offsets => INT32 Field Description replica_id The broker ID of the requestor, or -1 if this request is being made by a normal consumer. topics Each topic in the request. name The topic name. partitions Each partition in the request. partition_index The partition index. timestamp The current timestamp. max_num_offsets The maximum number of offsets to report. ListOffsets Request (Version: 1) => replica_id [topics] replica_id => INT32 topics => name [partitions] name => STRING partitions => partition_index timestamp partition_index => INT32 timestamp => INT64 Field Description replica_id The broker ID of the requestor, or -1 if this request is being made by a normal consumer. topics Each topic in the request. name The topic name. partitions Each partition in the request. partition_index The partition index. timestamp The current timestamp. ListOffsets Request (Version: 2) => replica_id isolation_level [topics] replica_id => INT32 isolation_level => INT8 topics => name [partitions] name => STRING partitions => partition_index timestamp partition_index => INT32 timestamp => INT64 Field Description replica_id The broker ID of the requestor, or -1 if this request is being made by a normal consumer. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records topics Each topic in the request. name The topic name. partitions Each partition in the request. partition_index The partition index. timestamp The current timestamp. ListOffsets Request (Version: 3) => replica_id isolation_level [topics] replica_id => INT32 isolation_level => INT8 topics => name [partitions] name => STRING partitions => partition_index timestamp partition_index => INT32 timestamp => INT64 Field Description replica_id The broker ID of the requestor, or -1 if this request is being made by a normal consumer. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records topics Each topic in the request. name The topic name. partitions Each partition in the request. partition_index The partition index. timestamp The current timestamp. ListOffsets Request (Version: 4) => replica_id isolation_level [topics] replica_id => INT32 isolation_level => INT8 topics => name [partitions] name => STRING partitions => partition_index current_leader_epoch timestamp partition_index => INT32 current_leader_epoch => INT32 timestamp => INT64 Field Description replica_id The broker ID of the requestor, or -1 if this request is being made by a normal consumer. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records topics Each topic in the request. name The topic name. partitions Each partition in the request. partition_index The partition index. current_leader_epoch The current leader epoch. timestamp The current timestamp. ListOffsets Request (Version: 5) => replica_id isolation_level [topics] replica_id => INT32 isolation_level => INT8 topics => name [partitions] name => STRING partitions => partition_index current_leader_epoch timestamp partition_index => INT32 current_leader_epoch => INT32 timestamp => INT64 Field Description replica_id The broker ID of the requestor, or -1 if this request is being made by a normal consumer. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records topics Each topic in the request. name The topic name. partitions Each partition in the request. partition_index The partition index. current_leader_epoch The current leader epoch. timestamp The current timestamp. ListOffsets Request (Version: 6) => replica_id isolation_level [topics] TAG_BUFFER replica_id => INT32 isolation_level => INT8 topics => name [partitions] TAG_BUFFER name => COMPACT_STRING partitions => partition_index current_leader_epoch timestamp TAG_BUFFER partition_index => INT32 current_leader_epoch => INT32 timestamp => INT64 Field Description replica_id The broker ID of the requestor, or -1 if this request is being made by a normal consumer. isolation_level This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records topics Each topic in the request. name The topic name. partitions Each partition in the request. partition_index The partition index. current_leader_epoch The current leader epoch. timestamp The current timestamp. _tagged_fields The tagged fields _tagged_fields The tagged fields _tagged_fields The tagged fields ListOffsets Request (Version: 7) => replica_id isolation_level [topics] TAG_BUFFER replica_id => INT32 isolation_level => INT8 topics => name [partitions] TAG_BUFFER name => COMPACT_STRING partitions => partition_index current_leader_epoch timestamp TAG_BUFFER partition_index => INT32 current_leader_epoch => INT32 timestamp => INT64 Field Description replica_id The broker ID of the requestor, or -1 if this request is being made by a normal co

kafka.apache.org - Kafka protocol guide - Apache Kafka

What is the easiest job to get that pays well?

21 Steps To Landing A Higher Paying Job Utilize job boards. Find a career coach. Grow your network. Do your research. Prepare to give references....

Last updated Dec 22, 2022

How can a housewife earn money online?

How can a Housewife Earn Money at Home in India? Affiliate Marketing. Become an Insurance POSP. Sell Homemade Items. Become a Translator. Blogging....

Last updated Nov 15, 2019

Promotion

What are referral links?

A referral link is a unique URL used in referral marketing programs companies provide to their existing customers to promote their brand and...

Last updated Jul 19, 2020

Promotion

What hashtags are most popular on TikTok?

What is the most used hashtag on TikTok? Unsurprisingly the most used hashtag on TikTok is #tiktok. This is followed by two other TikTok-specific...

Last updated Mar 6, 2019