{"id":30,"date":"2016-08-07T11:30:05","date_gmt":"2016-08-07T11:30:05","guid":{"rendered":"http:\/\/buki.io\/?p=30"},"modified":"2016-10-04T13:28:48","modified_gmt":"2016-10-04T13:28:48","slug":"apache-spark-2-0-select-from","status":"publish","type":"post","link":"http:\/\/buki.io\/?p=30","title":{"rendered":"Apache Spark 2.0: select * from"},"content":{"rendered":"

How to query a „regular“ databases (DB2 in this example) via JDBC in Apache Spark 2.0.<\/p>\n

final Dataset result = this.sparkSession.read().format(\"jdbc\").options(this.options).load();<\/code><\/p>\n

Where „this.options“ is filled like<\/p>\n

this.options.put(\"driver\", \"com.ibm.db2.jcc.DB2Driver\");
\nthis.options.put(\"url\", \"jdbc:db2:\/\/host:port\/db2_database\");
\nthis.options.put(\"user\",USER_STRING);
\nthis.options.put(\"password\",PWD_STRING);
\nthis.options.put(\"dbtable\", sql_statement\");<\/code><\/p>\n

Important to notice is to wrap sql_statement<\/code> like this:
\n(select * from table) as tmp_table<\/code><\/p>\n

because Spark converts this to <\/p>\n

select * from (select * from table) as tmp_table<\/code><\/p>\n

Using the select without brackets does not work.<\/p>\n","protected":false},"excerpt":{"rendered":"

How to query a „regular“ databases (DB2 in this example) via JDBC in Apache Spark 2.0. final Dataset result = this.sparkSession.read().format(„jdbc“).options(this.options).load(); Where „this.options“ is filled like this.options.put(„driver“, „com.ibm.db2.jcc.DB2Driver“); this.options.put(„url“, „jdbc:db2:\/\/host:port\/db2_database“); this.options.put(„user“,USER_STRING); this.options.put(„password“,PWD_STRING); this.options.put(„dbtable“, sql_statement“); Important to notice is to wrap sql_statement like this: (select * from table) as tmp_table because Spark converts this to select … <\/p>\n

Continue reading „Apache Spark 2.0: select * from“<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[3],"tags":[13,15,14],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p7N2pi-u","_links":{"self":[{"href":"http:\/\/buki.io\/index.php?rest_route=\/wp\/v2\/posts\/30"}],"collection":[{"href":"http:\/\/buki.io\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/buki.io\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/buki.io\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/buki.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=30"}],"version-history":[{"count":4,"href":"http:\/\/buki.io\/index.php?rest_route=\/wp\/v2\/posts\/30\/revisions"}],"predecessor-version":[{"id":54,"href":"http:\/\/buki.io\/index.php?rest_route=\/wp\/v2\/posts\/30\/revisions\/54"}],"wp:attachment":[{"href":"http:\/\/buki.io\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=30"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/buki.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=30"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/buki.io\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=30"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}