Schlagwort: Spark SQL

How to query a „regular“ databases (DB2 in this example) via JDBC in Apache Spark 2.0.

final Dataset result = this.sparkSession.read().format("jdbc").options(this.options).load();

Where „this.options“ is filled like

this.options.put("driver", "com.ibm.db2.jcc.DB2Driver");
this.options.put("url", "jdbc:db2://host:port/db2_database");
this.options.put("user",USER_STRING);
this.options.put("password",PWD_STRING);
this.options.put("dbtable", sql_statement");

Important to notice is to wrap sql_statement like this:
(select * from table) as tmp_table

because Spark converts this to

select * from (select * from table) as tmp_table

Using the select without brackets does not work.