Can't merge tables with error of weird array size limitation, help?
2 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Konstantin Maier
el 15 de Mayo de 2021
Comentada: Konstantin Maier
el 16 de Mayo de 2021
I got two tables, one with training data and one with created features that I want to implement.
Table A has the size 307511x241 and is like so:
>> head(app_train, 3)
ans =
3×241 table
SK_ID_CURR NAME_CONTRACT_TYPE CODE_GENDER_M CODE_GENDER_F FLAG_OWN_CAR etc.
__________ __________________ _____________ _____________ ____________
100002 0 1 0 0 etc.
100003 0 0 1 0 etc.
100004 1 1 0 1 etc.
Table B has the size 307511x36 and is like so:
>> head(polyFeaturesTrain, 3)
ans =
3×36 table
Polynomial constant EXT_SOURCE_1 EXT_SOURCE_1^2 EXT_SOURCE_1^3 etc.
___________________ __________________ ___________________ __________________
1 0.0830369673913226 0.00689513795354757 0.0005725513454074 etc.
1 0.311267311381222 0.0968873391344949 0.0301578615592749 etc.
1 0.505997930505754 0.256033905676106 0.129552626411415 etc.
5 variables in Table B are the same as in Table A (EXT_SOURCE_1, EXT_SOURCE_2, EXT_SOURCE_3, DAYS_BIRTH and TARGET). The values for TARGET are the same in both tables, while for the other variables there are some missing values in Table A but not in Table B.
I want to merge Table B into Table A (rows stay the same, only add the variables) and for identical variables want to keep the values from Table B. I tried innerjoin, outerjoin, join and some different Name-Value Pair Arguments for those functions, nothing works. Also trying TableC = [TableA, TableB] doesn't work because of the identical variables.
When using the function I often get this error message:
Error using tabular/outerjoin (line 228)
Requested 1x80527655221 (600.0GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become
unresponsive.
The table I want would be 307511x272, how can it be 600GB? I don't understand what Matlab is trying to tell me.
If someone knows what I am doing wrong, please help me out, I spent the last few hours making no progress because of this.
0 comentarios
Respuesta aceptada
Stephen23
el 15 de Mayo de 2021
"I want to merge Table B into Table A (rows stay the same, only add the variables) and for identical variables want to keep the values from Table B"
x = randi(9,7,1);
y = randi(9,7,1);
z = randi(9,7,1);
A = table(x,y)
y = round(y/2);
B = table(y,z)
V = setdiff(A.Properties.VariableNames,B.Properties.VariableNames);
C = [A(:,V),B]
Más respuestas (0)
Ver también
Categorías
Más información sobre Tables en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!