Producer classes to read file connections

Description

ConnectionProducer classes include ScanProducer, ReadLinesProducer, and ReadTableProducer, providing Streamer interfaces to scan, readLines, and read.table.

Usage

ScanProducer(file, ..., fileArgs=list(), scanArgs=list(...))
ReadLinesProducer(con, ..., conArgs=list(), readLinesArgs=list(...))
ReadTableProducer(file, ..., fileArgs=list(), readTableArgs=list(...))

## S3 method for class 'ConnectionProducer'
close(con, ...)

Arguments

Methods

See Producer Methods.

Internal Class Fields and Methods

Internal fields of this class are are described with, e.g., getRefClass("ReadLinesProducer")$fields.

Internal methods of this class are described with getRefClass("ReadLinesProducer")$methods() and getRefClass("ReadLinesProducer")$help().

Author(s)

Martin Morgan mtmorgan@fhcrc.org

See Also

Streamer-package, Producer-class, Streamer-class.

Examples

fl <- system.file(package="Rsamtools", "extdata", "ex1.sam")

p <- ReadLinesProducer(fl, n = 1000)  # read 1000 lines at a time
while (length(y <- yield(p)))
    print(length(y))
close(p)

p <- ReadTableProducer(fl, quote="", fill=TRUE, nrows=1000)
while (length(y <- yield(p)))
    print(dim(y))

reset(p)
dim(yield(p))

## connections opened 'under the hood' are closed, with warnings
rm(p); gc() 

## avoid warnings by managing connections
p <- ScanProducer(file(fl, "r"), verbose=TRUE,
                  scanArgs=list(what=character()))
length(yield(p))
close(p)
rm(p); gc()

Results

R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(Streamer)
> png(filename="/home/ddbj/snapshot/RGM3/R_BC/result/Streamer/ConnectionProducer-classes.Rd_%03d_medium.png", width=480, height=480)
> ### Name: ConnectionProducer
> ### Title: Producer classes to read file connections
> ### Aliases: ConnectionProducer ConnectionProducer-classes
> ###   ConnectionProducer-class close.ConnectionProducer ScanProducer-class
> ###   ReadLinesProducer-class ReadTableProducer-class ScanProducer
> ###   ReadLinesProducer ReadTableProducer
> ### Keywords: classes
> 
> ### ** Examples
> 
> fl <- system.file(package="Rsamtools", "extdata", "ex1.sam")
> 
> p <- ReadLinesProducer(fl, n = 1000)  # read 1000 lines at a time
> while (length(y <- yield(p)))
+     print(length(y))
[1] 1000
[1] 1000
[1] 1000
[1] 307
> close(p)
> 
> p <- ReadTableProducer(fl, quote="", fill=TRUE, nrows=1000)
> while (length(y <- yield(p)))
+     print(dim(y))
[1] 1000   17
[1] 1000   17
[1] 1000   17
[1] 307  17
> 
> reset(p)
class: ReadTableProducer 
> dim(yield(p))
[1] 1000   17
> 
> ## connections opened 'under the hood' are closed, with warnings
> rm(p); gc() 
         used (Mb) gc trigger (Mb) max used (Mb)
Ncells 496274 26.6     750400 40.1   592000 31.7
Vcells 587276  4.5    1308461 10.0  1023462  7.9
Warning message:
closing unused connection 3 (/home/ddbj/local/lib64/R/library/Rsamtools/extdata/ex1.sam) 
> 
> ## avoid warnings by managing connections
> p <- ScanProducer(file(fl, "r"), verbose=TRUE,
+                   scanArgs=list(what=character()))
> length(yield(p))
ConnectionProducer$reset
Read 51431 items
[1] 51431
> close(p)
> rm(p); gc()
         used (Mb) gc trigger (Mb) max used (Mb)
Ncells 499435 26.7     940480 50.3   592000 31.7
Vcells 582825  4.5    1308461 10.0  1023462  7.9
> 
> 
> 
> 
> 
> dev.off()
null device 
          1 
>