Vinayak Machan • about 5 years ago
Unable to transform bigr.frame to bigr.matrix using bigr.transform
Thanks to Bluemix support, I finally was able to download the updated bigr package which has the features to actually create linear models using bigr.
These model functions though need a bigr.matrix and do not accept a bigr.frame. So I tried using the bigr.transform function that transforms a bigr.frame to a bigr.matrix. This ran for about 10 minutes (the data frame has about 2 million records) and then errored out. I have listed below the steps I am doing and the corresponding error. The error by itself does not help much in narrowing down the cause.
Has anyone been able to successfully do this? Would appreciate some help.
# Code snippet below
bdf <- bigr.frame(
dataPath = "/user/data/part-00000"
,dataSource = "DEL"
, header = T
bmtrx <- bigr.transform(
,outData = "/user/data/matrix/outdata"
,transformPath = "/user/data/matrix/transform"
modLM <- bigr.lm(Y ~ X1 + X2 + X3 + X4, data=bmtrx) # This statement throws the below error
ERROR LISTED BELOW
Error: BigR[.bigr.jdbc.query.helper]: Error code : 0, SQLState : 08003
Caused by :
The stack trace :java.sql.SQLException
Look forward to some responses.
Comments are closed.