Friday, 13 September 2013

filter and change a column in pig

filter and change a column in pig

suppose my data looks like
row1 cats val12 val13
row2 dogs val22 val23
row3 cats val32 val33
...
data = load 'file' AS (row:chararry, pets:charray, val2:charray,
val3:charray);
filter data to only save 'cats' rows
felines = filter data by (pets matches 'cats');
now change 'cats' to 'lions'
lions = foreach felines generate replace (pets, 'cats', 'lions');
dump lions;
(lions)
(lions)
...
my goal is to create new rows to add to my table
newFelines = foreach lions generate rows, lions, val1, val2;
Error ^^^^^
"Error during parsing. Scalars can be only used with projections"
how can I obtain a set with the following new rows?
row1 lions val11 val12
row3 lions val31 val32
TIA,

No comments:

Post a Comment