SingleStore: Sign Up Log In Cannot get source metadata for pipeline. Failed to create consumer: ssl.ca.location failed: No further error info

All we need is an easy explanation of the problem, so here it is.

below is the command that i am using, but when i create the kafka pipeline command i get the below error:
ERROR 1933 ER_EXTRACTOR_EXTRACTOR_GET_LATEST_OFFSETS: Cannot get source metadata for pipeline. Failed to create consumer: ssl.ca.location failed: No further error information available

Command used:

CREATE PIPELINE ticketmaster_pipeline AS

LOAD DATA KAFKA ‘b-3.etainmentnonprod.z2xjta.c25.kafka.us-east-1.amazonaws.com:9094,b-1.etainmentnonprod.z2xjta.c25.kafka.us-east-1.amazonaws.com:9094,b-2.etainmentnonprod.z2xjta.c25.kafka.us-east-1.amazonaws.com:9094/ticketmaster’

CONFIG '{“sasl.username”: “AWS_ACCESS_KEY”,

     "sasl.mechanism": "PLAIN",

     "security.protocol": "SASL_SSL",

     "ssl.ca.location": "/etc/pki/ca-trust/extracted/java/cacerts"}'
CREDENTIALS ‘{“sasl.password”: “AWS_PSWD/Z”}’

INTO TABLE ticketmaster_kafka

FORMAT JSON (event_url ← event_url,event_id ← event_id,timestamp ← timestamp,event_name ← event_name,venue ← venue,event_datetime ← event_datetime,city ← city,state ← state,section ← section,row ← row,qty ← qty,seat_ids ← seat_ids,seat_numbers ← seat_numbers,inventory_type ← inventory_type,price ← price);

My kafka setup in on AWS MSK insatnce and i have data in the topic also

How to solve :

I know you bored from this bug, So we are here to help you! Take a deep breath and look at the explanation of your problem. We have many solutions to this problem, But we recommend you to use the first method because it is tested & true method that will 100% work for you.

Method 1

This is an authentication issue we can resolve it by removing everything besides CONFIG ‘{“security.protocol”: “ssl”}’

Please try this:

CREATE PIPELINE ticketmaster_pipeline AS
LOAD DATA KAFKA ‘b-3.etainmentnonprod.z2xjta.c25.kafka.us-east-1.amazonaws.com:9094,b-1.etainmentnonprod.z2xjta.c25.kafka.us-east-1.amazonaws.com:9094,b-2.etainmentnonprod.z2xjta.c25.kafka.us-east-1.amazonaws.com:9094/ticketmaster’
CONFIG ‘{"security.protocol": "SSL"}’

INTO TABLE ticketmaster_kafka
FORMAT JSON (event_url ← event_url,
event_id ← event_id,
timestamp ← timestamp,
event_name ← event_name,
venue ← venue,
event_datetime ← event_datetime,
city ← city,
state ← state,
section ← section,
row ← row,
qty ← qty,
seat_ids ← seat_ids,
seat_numbers ← seat_numbers,
inventory_type ← inventory_type,
price ← price);

Note: Use and implement method 1 because this method fully tested our system.
Thank you 🙂

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply