R Graphical Manual

Browse All

Last data update: 2014.03.03

R: Write ffdf data to a database table by using a JDBC...

write.jdbc.ffdf

R Documentation

Write ffdf data to a database table by using a JDBC connection.

Description

Write ffdf data to a database table by using a JDBC connection. This can for example be used to store large ffdf datasets from R in Oracle, SQLite, MySQL, PostgreSQL, Hive or other SQL databases.
Mark that for very large datasets, these SQL databases might have tools to speed up by bulk loading. You might also consider that as an alternative to using this procedure.

Usage

write.jdbc.ffdf(x, name, dbConnect.args = list(drv = NULL, dbname = NULL,
  username = "", password = ""), RECORDBYTES = sum(.rambytes[vmode(x)]),
  BATCHBYTES = getOption("ffbatchbytes"), by = NULL, VERBOSE = FALSE, ...)

Arguments

`x`	the `ffdf` to write to the database
`name`	character string with the name of the table to store the data in. Passed on to `dbWriteTable`.
`dbConnect.args`	a list of arguments to pass to JDBC's `RJDBC::dbConnect` (like drv, dbname, username, password). See the examples.
`RECORDBYTES`	optional integer scalar representing the bytes needed to process a single row of the ffdf
`BATCHBYTES`	integer: bytes allowed for the size of the data.frame storing the result of reading one chunk. See documentation in `read.table.ffdf` for more details.
`by`	integer passed on to `chunk` indicating to write to the database in chunks of this size. Overwrites the behaviour of BATCHBYTES and RECORDBYTES.
`VERBOSE`	logical: TRUE to verbose timings for each processed chunk (default FALSE).
`...`	optional parameters passed on to `dbWriteTable`

Details

Opens up the JDBC connection using RJDBC::dbConnect, writes data to the SQL table using RJDBC::dbWriteTable by extracting the data in batches from the ffdf and appending them to the table.

Value

invisible()

Examples

## Not run: 
require(ff)

##
## Example query using data in sqlite
##
require(RJDBC)
dbfile <- system.file("smalldb.sqlite3", package="ETLUtils")
drv <- JDBC(driverClass = "org.sqlite.JDBC", classPath = "/usr/local/lib/sqlite-jdbc-3.7.2.jar")
query <- "select * from testdata limit 10000"
x <- read.jdbc.ffdf(query = query,
 dbConnect.args = list(drv = drv, url = sprintf("jdbc:sqlite:%s", dbfile)),
 first.rows = 100, next.rows = 1000, VERBOSE=TRUE)

write.jdbc.ffdf(x = x, name = "helloworld", row.names = FALSE, overwrite = TRUE,
  dbConnect.args = list(drv = drv, url = sprintf("jdbc:sqlite:%s", dbfile)),
  by = 1000, VERBOSE=TRUE)
test <- read.jdbc.ffdf(query = "select * from helloworld",
  dbConnect.args = list(drv = drv, url = sprintf("jdbc:sqlite:%s", dbfile)))

## End(Not run)