The "Schema not found; error code: 40403" error means the Kafka Connect sink connector tried to look up a schema in Schema Registry but the schema does not exist under the expected subject name. The most common causes are producing messages without registering a schema first, a subject naming strategy mismatch between the producer and the connector, or pointing to the wrong Schema Registry instance.

How Schema Registry Works with Kafka Connect

When a Kafka producer serializes a message using the Avro, Protobuf, or JSON Schema serializer, it registers the schema with Schema Registry and embeds the schema ID in the message payload (as a 5-byte header: magic byte + 4-byte schema ID).

When the sink connector deserializes the message, it reads the schema ID from the payload and fetches the schema from Schema Registry. If Schema Registry has no schema for that ID, or the subject the connector expects does not exist, you get error 40403.

org.apache.kafka.connect.errors.DataException:
  my-topic-value - Schema not found; error code: 40403

The error format <subject>-value tells you which subject the connector was looking for.

Cause 1: Schema Was Never Registered

If the producer sent messages without using a schema-aware serializer, no schema was registered. This happens when:

• The producer uses StringSerializer or ByteArraySerializer instead of KafkaAvroSerializer
• The producer was configured with auto.register.schemas=false and no schema was pre-registered
• The topic received data from a source that does not interact with Schema Registry

Fix

Register the schema manually or configure the producer to use a schema-aware serializer:

1# Check if any schemas exist for the topic
2curl -s http://localhost:8081/subjects | jq .
3
4# Check specific subject
5curl -s http://localhost:8081/subjects/my-topic-value/versions | jq .

If the subject does not exist, register the schema:

1curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
2  --data '{
3    "schema": "{\"type\":\"record\",\"name\":\"Order\",\"fields\":[{\"name\":\"id\",\"type\":\"long\"},{\"name\":\"amount\",\"type\":\"double\"}]}"
4  }' \
5  http://localhost:8081/subjects/my-topic-value/versions

Cause 2: Subject Naming Strategy Mismatch

Schema Registry uses a subject naming strategy to determine which subject name to look up for a given topic. The default is TopicNameStrategy, which constructs the subject as <topic>-key or <topic>-value.

If the producer registered the schema under a different strategy, the connector will look for a subject that does not exist.

Fix

Ensure the producer and consumer use the same strategy:

1# Producer configuration
2value.subject.name.strategy=io.confluent.kafka.serializers.subject.TopicNameStrategy
3
4# Sink connector configuration
5"value.subject.name.strategy": "io.confluent.kafka.serializers.subject.TopicNameStrategy"

Check which subjects actually exist:

1# List all subjects
2curl -s http://localhost:8081/subjects | jq .
3
4# If you see "Order" instead of "my-topic-value",
5# the producer is using RecordNameStrategy

Cause 3: Wrong Schema Registry URL

The sink connector is pointing to a different Schema Registry instance than the one the producer used.

1{
2  "name": "my-sink-connector",
3  "config": {
4    "connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector",
5    "value.converter": "io.confluent.connect.avro.AvroConverter",
6    "value.converter.schema.registry.url": "http://wrong-host:8081",
7    "topics": "my-topic"
8  }
9}

Fix

Verify the Schema Registry URL in the connector configuration and in the Kafka Connect worker properties:

1# Check worker-level config
2grep schema.registry /etc/kafka/connect-distributed.properties
3
4# Check connector-level config
5curl -s http://localhost:8083/connectors/my-sink-connector/config | jq '.["value.converter.schema.registry.url"]'
6
7# Verify Schema Registry is reachable from the Connect worker
8curl -s http://schema-registry-host:8081/subjects

Remember that the converter-level URL (value.converter.schema.registry.url) overrides the worker-level URL.

Cause 4: Schema Deleted or Soft-Deleted

If someone deleted the schema (either hard or soft delete), the lookup fails:

1# Check if the subject exists but has been soft-deleted
2curl -s "http://localhost:8081/subjects/my-topic-value?deleted=true" | jq .
3
4# Restore a soft-deleted subject
5curl -X POST "http://localhost:8081/subjects/my-topic-value/versions" \
6  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
7  --data '{"schema": "..."}'

Cause 5: Using value.converter Without Schema Registry

If the connector uses AvroConverter but the messages are plain JSON (not Avro-encoded), the converter tries to read the 5-byte schema ID header and fails:

1{
2  "value.converter": "io.confluent.connect.avro.AvroConverter",
3  "value.converter.schema.registry.url": "http://localhost:8081"
4}

Fix

Match the converter to the actual message format:

1{
2  "value.converter": "org.apache.kafka.connect.json.JsonConverter",
3  "value.converter.schemas.enable": false
4}

Diagnostic Workflow

When you encounter the 40403 error, follow this sequence:

1# 1. Identify the subject the connector is looking for (from error message)
2# Example: "my-topic-value - Schema not found"
3
4# 2. List all subjects in Schema Registry
5curl -s http://localhost:8081/subjects | jq .
6
7# 3. Check if the expected subject exists
8curl -s http://localhost:8081/subjects/my-topic-value/versions | jq .
9
10# 4. If it does not exist, inspect a message from the topic
11kafka-console-consumer --bootstrap-server localhost:9092 \
12  --topic my-topic --from-beginning --max-messages 1 \
13  --property print.headers=true
14
15# 5. Check the schema ID embedded in the message
16kafka-avro-console-consumer --bootstrap-server localhost:9092 \
17  --topic my-topic --from-beginning --max-messages 1 \
18  --property schema.registry.url=http://localhost:8081
19
20# 6. Verify the connector configuration
21curl -s http://localhost:8083/connectors/my-sink-connector/config | jq .
22
23# 7. Check connector status for detailed error
24curl -s http://localhost:8083/connectors/my-sink-connector/status | jq .

Complete Working Connector Configuration

Here is a correctly configured JDBC sink connector with all schema-related settings:

1{
2  "name": "orders-jdbc-sink",
3  "config": {
4    "connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector",
5    "tasks.max": "1",
6    "topics": "orders",
7    "connection.url": "jdbc:postgresql://localhost:5432/mydb",
8    "connection.user": "dbuser",
9    "connection.password": "dbpass",
10    "auto.create": "true",
11    "insert.mode": "upsert",
12    "pk.mode": "record_value",
13    "pk.fields": "id",
14    "value.converter": "io.confluent.connect.avro.AvroConverter",
15    "value.converter.schema.registry.url": "http://schema-registry:8081",
16    "value.subject.name.strategy": "io.confluent.kafka.serializers.subject.TopicNameStrategy",
17    "key.converter": "org.apache.kafka.connect.storage.StringConverter"
18  }
19}

Common Pitfalls

Mixing serialization formats. Producing as JSON but consuming with AvroConverter (or vice versa) causes deserialization failures. The converter must match the actual format of messages on the topic.

Forgetting the converter prefix. The schema registry URL for the connector must be set as value.converter.schema.registry.url, not just schema.registry.url. The latter is for the worker-level config. Omitting the prefix means the converter does not know where to find schemas.

Multiple environments sharing a Schema Registry. If dev and staging topics use the same Schema Registry, subjects from one environment may not exist in another. Use separate Schema Registry instances or a prefix-based naming convention.

Not checking the connector status API. The Connect REST API at /connectors/<name>/status often contains the full stack trace with the exact subject name that failed. This is more informative than the Kafka Connect worker logs.

Assuming the schema auto-registers on the consumer side. auto.register.schemas is a producer-side setting. The consumer (sink connector) never registers schemas. It only reads them. The schema must already exist before the connector reads the message.

Ignoring key vs. value schema. The error message includes -key or -value in the subject name. Make sure you are investigating the correct side. Key schemas and value schemas are separate subjects.

Summary

Error 40403 means Schema Registry does not have the schema the sink connector needs. Check whether the schema was registered, whether the subject naming strategy matches between producer and consumer, and whether the connector points to the correct Schema Registry URL. Use the diagnostic curl commands to list subjects and inspect connector configuration. Match the converter type to the actual message serialization format on the topic. When in doubt, consume a message with kafka-avro-console-consumer to verify the message format and schema ID independently of the connector.