ӆhJSSKrSSKrSSKrSSKrSSKJr 0q Sr Sr Sr Sr Sr0qSrS rS rS rS rS rSrSSjrSSjrSrg)N)tqdmc &[R"S5 XX4Xg4n Sn S[;aO[S[ [SU 555(a'[ R"[SU:H5(aSn [ X5upURSURS:XdeU"5n [ R"UR5n [ R"UR5n[ R"UR5n[URS5S-n[RS S 5n[RS S 5n[[[U55S 5HnU (aUUUU:Xa UUUU'M UUS :XaURUUUS-5S UU'MHXS S &X>S S &[ R"UUS S 24*U-5nUS S 2US UU4R!5U S S 2US UU4'US S 2US UU4R!5UUUS UU4'U R#X5 U RUUUS-5S UU'M U[S 'U[S 'U[S'U [S'U"XO5$)aThe model is retrained for each test sample with the important features set to a constant. If you want to know how important a set of features is you can ask how the model would be different if those features had never existed. To determine this we can mask those features across the entire training and test datasets, then retrain the model. If we apply compare the output of this retrained model to the original model we can see the effect produced by knowning the features we masked. Since for individualized explanation methods each test sample has a different set of most important features we need to retrain the model for every test sample to get the change in model performance when a specified fraction of the most important features are withheld. IThe retrain based measures can incorrectly evaluate models in some cases!Fargsc3,# UH upXLv M g7fN.0abs jC:\Users\julio\OneDrive\Documentos\Trabajo\Ideas Frescas\venv\Lib\site-packages\shap/benchmark/measures.py !remove_retrain.. sC"B$!qv"B attr_testTư>nmaskNyp_masked_testz"Retraining for the 'remove' metricr)warningswarn _remove_cacheallzipnpto_arrayshapezeros const_randgetrrangelenpredictargsortmeanfit)rX_trainy_trainX_testy_testrmodel_generatormetric trained_model random_stater cache_match model_masked X_train_tmp X_test_tmprtie_breaking_noise last_nmasklast_yp_masked_testiorderings rremove_retrainr9 s MM]^ fo FDK  C#mF&;T"BC C C}]hOimvOvHwHwKw/OG == v||A .. .#$L((7==)K&,,'JXXfll+N#GMM!$45<""7D1J'++,R;R3S3X3X3ZK8JeAh// 029!Xja=Q:Q2R2W2W2YJq(:U1X.. /   [ 2 , 4 4ZAE5J KA NN1 L$#M'&4M"#!*M+ M& & ))c [X5upURSURS:XdeUR5n [URSU 5S-n UR S5n [ [ U55H?n X S:dM [R"X]SS24*U -5nXSX XUSX 4'MA URU 5nU"XO5$)zKEach test sample is masked by setting the important features to a constant.rrrN) rrcopyr r&r"r#rr%r$)rr(r)r*r+rr,r-r.r/r3r4 mean_valsr7r8rs r remove_maskr>Lsw/OG == v||A .. .J#GMM!$4lCdJ QI 3v;  8a<zz9T?"25G"GHH2;ZuxCCAFN &. ))r:c [R"S5 [X$5up$URSURS:XdeUR 5n UR S5n [ URS5S-n [[U55H?n X S:dM [R"XmSS24*U -5nXSX XUSX 4'MA UR 5n[[U55H?n XS:dM [R"X}SS24*U -5nXSXXUSX4'MA U"5nURX5 URU5nU "UU5$)arAn approximation of holdout that only retraines the model once. This is also called ROAR (RemOve And Retrain) in work by Google. It is much more computationally efficient that the holdout method because it masks the most important features in every sample and then retrains the model once, instead of retraining the model for every test sample like the holdout metric. rrrrN rrrrr<r&r r"r#rr%r'r$) nmask_train nmask_testr(r)r*r+ attr_trainrr,r-r2 X_train_meanr4r7r8r3r1yp_test_maskeds rbatch_remove_retrainrdsp MM]^w/OG == v||A .. .,,.K<<?L#GMM!$45< 3w<  >A zz:d#3"36H"HIH9EO_Q\Q_F`9aK8$4kn55 6!J 3v;  =1 zz9T?"25G"GHH7C_z}D]7^J(?Z]33 4 #$L[*!))*5N &. ))r:c B[R"S5 XX4Xg4n Sn S[;aO[S[ [SU 555(a'[ R"[SU:H5(aSn [ X5upURSURS:XdeU"5n [ R"UR5n [ R"UR5n[ R"UR5n[URS5S-n[RS S 5n[RS S 5n[[[U55S 5HnU (aUUUU:Xa UUUU'M UUURS:XaURUUUS-5S UU'MUXS S &X>S S &[ R"UUS S 24*U-5nUS S 2UUUS 4R!5U S S 2UUUS 4'US S 2UUUS 4R!5UUUUUS 4'U R#X5 U RUUUS-5S UU'GM U[S 'U[S 'U[S'U [S'U"XO5$)aThe model is retrained for each test sample with the non-important features set to a constant. If you want to know how important a set of features is you can ask how the model would be different if only those features had existed. To determine this we can mask the other features across the entire training and test datasets, then retrain the model. If we apply compare the output of this retrained model to the original model we can see the effect produced by only knowning the important features. Since for individualized explanation methods each test sample has a different set of most important features we need to retrain the model for every test sample to get the change in model performance when a specified fraction of the most important features are retained. rFrc3,# UH upXLv M g7frr r s rrkeep_retrain..sA"@$!qv"@rrTrrnkeepNrz Retraining for the 'keep' metricr)rr _keep_cacherrrrrrr r!rr"r#r$r%r&r')rhr(r)r*r+rr,r-r.r/rr0r1r2r3rr4 last_nkeepr6r7r8s r keep_retrainrks MM]^ fo FDK  A#k&&94"@A A Abff[YdMeirMrFsFsKw/OG == v||A .. .#$L((7==)K&,,'JXXfll+N#GMM!$45<$/J%//*:DA %F $&H I :a=E!H4 3A 6N1  1X+ + - 5 5fQQ6G H KN1 %N"qMzz9QT?"25G"GHH3:1huQxz>R;R3S3X3X3ZK8E!HJ// 029!XeAhj=Q:Q2R2W2W2YJq(58:.. /   [ 2 , 4 4ZAE5J KA NN1 J$!K$2K !(K K & ))r:c [X5upURSURS:XdeUR5n [R"UR5n [ URSU 5S-n UR S5n [[U55HLnXURS:dM[R"X^SS24*U -5nXXSXXUS4'MN URU 5n U"XK5$)z`The model is reevaluated for each test sample with the non-important features set to their mean.rrrN) rrr<rrr r&r"r#r%r$)rhr(r)r*r+rr,r-r.r/r3rr4r=r7r8s r keep_maskrmsw/OG == v||A .. .JXXfll+N#GMM!$4lCdJ QI 3v;  8fll1o %zz9T?"25G"GHH2;UXZCCAFN &. ))r:c [R"S5 [X$5up$URSURS:XdeUR 5n UR S5n [ URS5S-n [[U55HLn X URS:dM[R"XmSS24*U -5nXX SXXU S4'MN UR 5n[[U55HLn XURS:dM[R"X}SS24*U -5nXXSXXU S4'MN U"5nURX5 URU5nU "UU5$)acAn approximation of keep that only retraines the model once. This is also called KAR (Keep And Retrain) in work by Google. It is much more computationally efficient that the keep method because it masks the unimportant features in every sample and then retrains the model once, instead of retraining the model for every test sample like the keep metric. rrrrNr^) nkeep_train nkeep_testr(r)r*r+rarr,r-r2rbr4r7r8r3r1rcs rbatch_keep_retrainruns MM]^w/OG == v||A .. .,,.K<<?L#GMM!$45< 3w<  >GMM!, ,zz:d#3"36H"HIH9E{~O_F`9aK8N$455 6!J 3v;  =6<<? *zz9T?"25G"GHH7CZ]_D]7^J(a=?33 4 #$L[*!))*5N &. ))r:c[X5upURSURS:XdeURU5nU"U[U5R S55$)zRThe how well do the features plus a constant base rate sum up to the model output.r)rrr$ strip_listsum) r(r)r*r+rr,r-r.yp_tests rlocal_accuracyrzs^w/OG == v||A .. .##F+G ':i044Q7 88r:cUVs/sH0n[U[R5(a UROUPM2 sn$s snfr) isinstancepd DataFramevalues)rr s rrrs3DH IDq 1bll33AHH :D II Is7?c[RR5n[RRU5 [RRU5n[RRU5 U$)z*Generate a random array with a fixed seed.)rrandomseedrand)sizerold_seedouts rr r sIyy~~HIINN4 )).. CIINN8 Jr:c[RR5n[RRU5 [RRU5 [RRU5 g)z,Shuffle an array in-place with a fixed seed.N)rrrshuffle)arrrrs r const_shufflersDyy~~HIINN4IIcIINN8r:c:[U[5(aUS$U$)zqThis assumes that if you have a list of outputs you just want the second one (the second class is the '1' class).r)r|list)attrss rrwrws%Qx r:)i])rnumpyrpandasr} sklearn.utilsrT tqdm.autorrr9r>rLr\rdrirkrmrorqrurzrr rrwr r:rrsr >*B*.$*N*<$*N >*B*,$*N*<$*N 9Jr: