Aerospike Connect for Kafka - Outbound Release Notes

• ATTENTION: In future releases, Aerospike Kafka Outbound Connector will be renamed to aerospike-source-connector.
• The *.deb and *.rpm files support multiple architectures (ARM64 & AMD64).
• The connector is Java JVM based. Install the correct JVM for the desired architecture.
• This connector requires Java 11 or later.

Security

• [CONNECTOR-1160] - Fixed security vulnerability - Authentication Bypass in krb5-libs.

• ATTENTION: In future releases, Aerospike Kafka Outbound Connector will be renamed to aerospike-source-connector.
• The *.deb and *.rpm files support multiple architectures (ARM64 & AMD64).
• The connector is Java JVM based. Install the correct JVM for the desired architecture.
• This connector requires Java 11 or later.

Security

• [CONNECTOR-1149] - Fixed Deserialization of Untrusted Data in org.apache.avro:avro:1.11.3.
• [CONNECTOR-1149] - Fixed Denial of Service (DoS) in com.fasterxml.jackson.core:jackson-core.
• [CONNECTOR-1149] - Fixed Race Condition in io.undertow:undertow-core.
• [CONNECTOR-1149] - Fixed Uncontrolled Resource Consumption in commons-io:commons-io.

• ATTENTION: In future releases, Aerospike Kafka Outbound Connector will be renamed to aerospike-source-connector.
• The *.deb and *.rpm files support multiple architectures (ARM64 & AMD64).
• The connector is Java JVM based. Install the correct JVM for the desired architecture.
• This connector requires Java 11 or later.

New Features

• [CONNECTOR-1127] - Netty buffer customization changes in connectors and Kafka Outbound Await Metrics for better monitoring.
• [CONNECTOR-1137] - Logging improvement - Outbound Await Metrics for all applicable connectors.

Security

• [CONNECTOR-1135] - Fix Arbitrary Code Injection affecting platform-python-setuptools & python3-setuptools-wheel.
• Upgraded Pulsar version.

• ATTENTION: In future releases, Aerospike Kafka Outbound Connector will be renamed to aerospike-source-connector.
• The *.deb and *.rpm files support multiple architectures (ARM64 & AMD64).
• The connector is Java JVM based. Install the correct JVM for the desired architecture.
• This connector requires Java 11 or later.

Security

• [CONNECTOR-1105] - Fix Race Condition.
• [CONNECTOR-1108] - Fix Uncontrolled Resource Consumption ('Resource Exhaustion').

• ATTENTION: In future releases, Aerospike Kafka Outbound Connector will be renamed to aerospike-source-connector.
• The *.deb and *.rpm files support multiple architectures (ARM64 & AMD64).
• The connector is Java JVM based. Install the correct JVM for the desired architecture.
• This connector requires Java 11 or later.

Security

• [CONNECTOR-982] - Fix Improper Input Validation.
• [CONNECTOR-983] - Fix Infinite loop.
• [CONNECTOR-984] - Fix Uncontrolled Resource Consumption.
• [CONNECTOR-1022] - Fix Out-of-bounds write vulnerabilities.
• [CONNECTOR-1023] - Fix Out-of-bounds write vulnerabilities.
• [CONNECTOR-1024] - Fix Out-of-bounds write vulnerabilities.
• [CONNECTOR-1025] - Fix Out-of-bounds write vulnerabilities.

• ATTENTION: In future releases, Aerospike Kafka Outbound Connector will be renamed to aerospike-source-connector.
• The *.deb and *.rpm files support multiple architectures (ARM64 & AMD64).
• The connector is Java JVM based. Install the correct JVM for the desired architecture.
• This connector requires Java 11 or later.

New Features

• [CONNECTOR-922] - Make batching of ack configurable as a toggle.

Bug Fixes

• [CONNECTOR-927] - Kafka outbound connector hits schema registry for each XDR request.

Security

• [CONNECTOR-928] - Fix Allocation of Resources Without Limits or Throttling. CVE-2024-26308
• [CONNECTOR-929] - Fix Infinite loop. CVE-2024-25710

• ATTENTION: In future releases, Aerospike Kafka Outbound Connector will be renamed to aerospike-source-connector.
• The *.deb and *.rpm files support multiple architectures (ARM64 & AMD64).
• The connector is Java JVM based. Install the correct JVM for the desired architecture.
• This connector requires Java 11 or later.

Security

• [CONNECTOR-825] - Upgrade netty and gRPC for CVE-2023-44487.

• The *.deb and *.rpm files support multiple architectures (ARM64 & AMD64).
• The connector is Java JVM based. Install the correct JVM for the desired architecture.
• This connector requires Java 11 or later.

Security

• [CONNECTOR-775] - Address CVE-2023-36480 - upgrade to aerospike-java-client version 7.0.0

• The *.deb and *.rpm files support multiple architectures (ARM64 & AMD64).
• The connector is Java JVM based. Install the correct JVM for the desired architecture.
• This connector requires Java 11 or later.

New Features

• [CONNECTOR-373] - Batching support.
• [CONNECTOR-431] - Batching support for inbuilt formats.
• [CONNECTOR-584] - New JSON logging format.
• [CONNECTOR-705] - Bin include/exclude option in bin transformer.
• [CONNECTOR-752] - Configurable message key for Kafka messages.

Security

• [CONNECTOR-716] - Upgrade vulnerable dependencies.

• The *.deb and *.rpm files support multiple architectures (ARM64 & AMD64).
• The connector is Java JVM based. Install the correct JVM for the desired architecture.
• This connector requires Java 11 or later.

Improvements

• [CONNECTOR-562] - Multi-arch container image supporting linux/arm64 and linux/amd64 platforms.
• [CONNECTOR-679] - Upgrade vulnerable dependencies.
• [CONNECTOR-680] - Formatting option stringify-map-keys now defaults to true.

Bug Fixes

• [CONNECTOR-653] - Fixed handling of sorted map JSON conversion.

Improvements

• [CONNECTOR-445] Fix CVE-2022-34917 in Kafka Connectors.
  • Upgraded Kafka client from 2.3.0 to 3.3.1.
  • Upgraded a few internal dependencies.

Bug Fixes

• [CONNECTOR-506] Outbound connector fails to parse BigInteger value type in Change Notification Record maps.

Improvements

• [CONNECTOR-341] - Performance and stability improvements.
• [CONNECTOR-342] - Vulnerability scanning and dependency upgrades.
• [CONNECTOR-343] - Single installer for HTTP and XDR 5.0 wire protocol connector.
• [CONNECTOR-348] - Add support for only metadata, no-bins schemas in AvroMap and AvroRecord formats. The no-bins bin policy was introduced in Aerospike server version 6.0.0.0.

New Features

• [CONNECTOR-319] - Add support to plugin custom code through the Custom transformer API.
• [CONNECTOR-321] - Add support for port-based configuration.

Improvements

• [CONNECTOR-320] - Support Java 17 runtime.
• [CONNECTOR-323] - Add a routing config to skip dispatching of records to the destination.

• If you are using Aerospike Database Enterprise Edition version 5.6 or later, ensure that you use this version of the Kafka outbound connector or a later one.
• ATTENTION: HTTP connectors renamed from aerospike-kafka-outbound-legacy to aerospike-kafka-outbound-http. Refer to documentation for installation and starting instructions.
• ATTENTION: Outbound format breaking changes.
  • For Aerospike Server versions <= 4.9 that do not ship generation, expiry, and lut (last update time)
    • these fields are omitted from the payload in Json, FlatJson, and Avro-Map format.
    • these fields are shipped as nil in MessagePack format.
    • these fields are shipped as null in Avro-Record and Kafka-Avro format.
  • Last update time - lut has been upscaled to a Long from Int. Avro-Record and Kafka-Avro schemas specified in the config should change lut type from int to long.
  • Deletes written with AerospikeOutboundMetadata schema instead of AerospikeOutboundDelete schema in Kafka-Avro format.

New Features

• Prometheus integration support.
• Pre-packaged Grafana dashboards.
• PEM file format support in TLS configuration.
• Support for Boolean particle type introduced in Aerospike Server version 5.6.
• Official container image available on Docker Hub.

Bug Fixes

• Handling of new particle types introduced by the Aerospike Server not recognized by the connector.
• Possible incorrect handling of byte array and extension types in lists and maps in rare cases.

• ATTENTION Upgrade to aerospike-kafka-outbound 4.0.0 or later when using Aerospike Server 5.6 or later.
• New Kafka Outbound Connector for Aerospike Server Enterprise Edition version 5.0 and above.
• Legacy support for versions of the Aerospike Server Enterprise Edition version 4.9 and below.
  • If using Aerospike Server version 4.9 or below the Legacy connector must be used.
  • WARNING: A serious flaw has been discovered for http v2 in the library used by Aerospike Server Enterprise Edition < 5.0 change notification. http-version MUST be set to v1 to prevent xdr and change notification from being blocked.

Improvements

• Bundle a default configuration file with the installation package.

Bug Fixes

• Fail to convert Aerospike record to MessagePack format when setname is missing.

Known Issues

• JDK-8 connector crashes when consuming records from Aerospike Server 5.6 or later.
• JDK-11 connector errors on every record consumed from Aerospike Server 5.6 or later.

• ATTENTION Upgrade to aerospike-kafka-outbound 4.0.0 or later when using Aerospike Server 5.6 or later.
• New Kafka Outbound Connector for Aerospike Server Enterprise Edition version 5.0 and above.
• Legacy support for versions of the Aerospike Server Enterprise Edition version 4.9 and below.
  • If using Aerospike Server version 4.9 or below the Legacy connector must be used.
  • WARNING: A serious flaw has been discovered for http v2 in the library used by Aerospike Server Enterprise Edition < 5.0 change notification. http-version MUST be set to v1 to prevent xdr and change notification from being blocked.

Known Issues

• JDK-8 connector crashes when consuming records from Aerospike Server 5.6 or later.
• JDK-11 connector errors on every record consumed from Aerospike Server 5.6 or later.

• ATTENTION Upgrade to aerospike-kafka-outbound 4.0.0 or later when using Aerospike Server 5.6 or later.
• HyperLogLog (HLL) bins are handled and forwarded as BLOBs.
• Fixed BLOB bin conversion for Flat JSON format.

Known Issues

• JDK-8 connector crashes when consuming records from Aerospike Server 5.6 or later.
• JDK-11 connector errors on every record consumed from Aerospike Server 5.6 or later.

• ATTENTION Upgrade to aerospike-kafka-outbound 4.0.0 or later when using Aerospike Server 5.6 or later.
• Support for numeric map keys for JSON, Flat JSON and Avro.
• Fixed incorrect re-logging of parsing errors.
• Log key of the Aerospike record on parsing error.
• Added logging for the skipped records.

Known Issues

• JDK-8 connector crashes when consuming records from Aerospike Server 5.6 or later.
• JDK-11 connector errors on every record consumed from Aerospike Server 5.6 or later.

• ATTENTION Upgrade to aerospike-kafka-outbound 4.0.0 or later when using Aerospike Server 5.6 or later.
• Support for Avro, Flat JSON, Kafka Avro (with schema registry) serialization.
• Support for configuring Aerospike record bin mapping.
• Support to specify bin-mapping, routing and format at set, namespace level.

Known Issues

• JDK-8 connector crashes when consuming records from Aerospike Server 5.6 or later.
• JDK-11 connector errors on every record consumed from Aerospike Server 5.6 or later.

• ATTENTION Upgrade to aerospike-kafka-outbound 4.0.0 or later when using Aerospike Server 5.6 or later.
• Aerospike Kafka Connector (Outbound)
  • No changes except version number bump.
  • WARNING: A serious flaw has been discovered for http v2 in the library used by change notification. http-version MUST be set to v1 to prevent xdr and change notification from being blocked.

Known Issues

• JDK-8 connector crashes when consuming records from Aerospike Server 5.6 or later.
• JDK-11 connector errors on every record consumed from Aerospike Server 5.6 or later.

• ATTENTION Upgrade to aerospike-kafka-outbound 4.0.0 or later when using Aerospike Server 5.6 or later.
• Outbound connector
  • Removed feature key check.
  • Fixed json conversion exceptions with record bins having empty lists and maps.
  • WARNING: A serious flaw has been discovered for http v2 in the library used by change notification. http-version MUST be set to v1 to prevent xdr and change notification from being blocked.

Known Issues

• JDK-8 connector crashes when consuming records from Aerospike Server 5.6 or later.
• JDK-11 connector errors on every record consumed from Aerospike Server 5.6 or later.

• ATTENTION Upgrade to aerospike-kafka-outbound 4.0.0 or later when using Aerospike Server 5.6 or later.
• Initial GA release
• WARNING: A serious flaw has been discovered for http v2 in the library used by change notification. http-version MUST be set to v1 to prevent xdr and change notification from being blocked.

Known Issues

• JDK-8 connector crashes when consuming records from Aerospike Server 5.6 or later.
• JDK-11 connector errors on every record consumed from Aerospike Server 5.6 or later.