I'm trying to write data into a OpenSearch managed service in AWS when running my code locally via IntelliJ. However everything I've tried and I can't seem to get past:
24/05/08 15:38:00 ERROR NetworkClient: Node [...] failed (javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target); no other nodes left - aborting...Which whlie the error is self explanatory, how do I solve it?
So far:The code in spark I'm using to write looks like
object Spark { var uri = "" var username = "" var password = "" System.setProperty("javax.net.ssl.trustStore", "common/src/main/resources/rds-truststore.jks") System.setProperty("javax.net.ssl.trustStorePassword", "changeit") val getOrCreate: SparkSession = SparkSession.builder() .master("local[*]") .config("opensearch.nodes", uri) .config("opensearch.port", "443") .config("opensearch.nodes.wan.only", "true") .config("opensearch.net.http.auth.user", username) .config("opensearch.net.http.auth.pass", password) .config("opensearch.net.ssl", "true") .config("opensearch.batch.size.bytes", "1kb") .config("opensearch.net.ssl.cert.allow.self.signed", "true") .getOrCreate()}I've tried a number of different combinations with the config options (setting ssl false, removing ssl config etc) but none seem to change anything.
I'm not sure the truststore is setup correctly, but I used the AWS article for how to create this, it should be the same?
The code which generates the trust store from the linked article:
mydir=./truststore=${mydir}/rds-truststore.jksstorepassword=changeitcurl -sS "https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem" > ${mydir}/global-bundle.pemsplit -p "-----BEGIN CERTIFICATE-----" ${mydir}/global-bundle.pem rds-ca-for CERT in rds-ca-*; do alias=$(openssl x509 -noout -text -in $CERT | perl -ne 'next unless /Subject:/; s/.*(CN=|CN = )//; print') echo "Importing $alias" keytool -import -file ${CERT} -alias "${alias}" -storepass ${storepassword} -keystore ${truststore} -noprompt rm $CERTdonerm ${mydir}/global-bundle.pemecho "Trust store content is: "keytool -list -v -keystore "$truststore" -storepass ${storepassword} | grep Alias | cut -d " " -f3- | while read aliasdo expiry=`keytool -list -v -keystore "$truststore" -storepass ${storepassword} -alias "${alias}" | grep Valid | perl -ne 'if(/until: (.*?)\n/) { print "$1\n"; }'` echo " Certificate ${alias} expires in '$expiry'"doneAs far as I can tell in the AWS Console there is no ability to set custom certificates and whatnot with OpenSearch, so I'm not sure what else I should be looking at.
Edit:I should note that I have no problems calling OpenSearch via Postman, or even the Java REST client in other applications without setting any SSL configuration. Its just Spark I can't seem to get a connection with.
I'm currently using the "org.opensearch.client" % "opensearch-spark-20_2.12" % "1.0.1" package in my Spark app.
