From a660d6d5d980f174922e174aae76ec683668ff0d Mon Sep 17 00:00:00 2001 From: Shine wOng <1551885@tongji.edu.cn> Date: Fri, 10 Jan 2020 11:33:54 +0800 Subject: [PATCH] add conclusion on logistic regression, not finished yet. --- ml/logistic regression/images/fig1.png | Bin 0 -> 9077 bytes ml/logistic regression/images/fig2.png | Bin 0 -> 7535 bytes ml/logistic regression/images/sigmoid.png | Bin 0 -> 15049 bytes ml/logistic regression/logistic regression.md | 86 +++++++++++++++++- 4 files changed, 83 insertions(+), 3 deletions(-) create mode 100644 ml/logistic regression/images/fig1.png create mode 100644 ml/logistic regression/images/fig2.png create mode 100644 ml/logistic regression/images/sigmoid.png diff --git a/ml/logistic regression/images/fig1.png b/ml/logistic regression/images/fig1.png new file mode 100644 index 0000000000000000000000000000000000000000..5115b69a5f17f66a41b47f628a72ec2a981e879c GIT binary patch literal 9077 zcmeHtc{r4B-}gm!VbDrKD#;!qQ%n*eQ?~3&`dK1NVMvUTwFODCWSENVWZ#)m$xim2 z$~J~*jAapZXT`CUKX<@DqxQk#oIfCB&kE}d(dh5*3G z2mlO^*;v3QFNJcSfWH{L47D|Yf{tVJ;2%s*m-R0LKv4|m_HAbHZ+6dX=3W56=S}-& z7cB~qM36djlY~YA#(Pn=JdNWCm!%#jop$y`63j?#?C&)_Yx)oV+(yDse_j7 zoVmT9_u6E$gSu0Yy6a-u+K%uc*N8gq**9KmLt93m0I^ru004G76l9QB>%aZ+P%Ry{WY3O8PWy-%(I)Jh(epX-ul3;l5oW3%fXbjMb`=d zKzOmY!x?)rZ>YPneQ9dS)s8t%m}qEV;XamCTYGWq`~1_VPaijggwejfHceC4Cn^e^ z^P8T*uIcLj`0|9`#KIycLoXpmFF|H4wx`T>V2QuAq*JYx?`PPZ-vggl{(4L$h6P<9 z`}??<*ZG$Pnz#ztgd(nB;l8)--!B@YByO@=71@W4J?3eX4^daK*_le*%#gNe(hy|y z@$tziDB$kT)PM6qC+;-!LRD22p*P!T+3!jyCTQbEi|jGkUC*{{4Ypp+;OEcx`z#JC znaN5>up>7ApQ2`B+5a(g(L6C#6g_DB2F%lQA35iULCAdWr( zk%37+d3cJTcspdt&nNE3+Uw!$2D%9}n{dqV)r=<7h3PV8)K7J~A&UNlNMEgX`RAQ~ z`{27k?OdYbc^G85NNd&@liW_YIb~d7LD84Q0FRQU+t9w}ReD{u_qCLrb3rvOE(=7( zwY^uQ&9_>;2VNfi_XgRvAXf6h;>k6-Z!Fu6b~IADD-Za* zN&b+()k;(ppmVyh;OB;AX`3i0BVqFA5&1neT3DRX^a4ACVB)lS;C`XO$v_eR|Vm%8Hxs#Thho@#VDrBT1P`Nk-QIq01hEjOt%WkxBo^!@oB%i3_C z6`YB`#9KVl^X=aLROTdBa^ZWvzF+y$!$yX8Gd(XuVTqeQm>@i`ucUajvvI%$HH?N| z^`d)MSFz2F@!Y~oRa(^Xkq?AqRd7WVa-^`SJH44CQ2`G2Chw_Vha@YBj4Y$@7*%@$U zNfRGE8mPaH$jKH@Y^ARLd?sGe_>Gt_)LClydNnTL_ls*@cy|Td!@C?Je2tac zd2`L)diS&c7}dWTbxcZq(@%=B+{j<(DQ@!<`4#g7BU$jG-2GCOkYOl7?8A*TBpcv1 zWxRitcyx(PW!iXuA~ni_DdoXgmYs!$$81@H6}CK`BOg4tqXq;ewur({?`PUiesK-q z;kAP?Mmb9{4VSsDKVe2FG`GE~UuK@4d9x_;yA7R8RoW6V8LQDt+kEa=UvD-MsCqdl zvBc!m!KTF!-WB{l@jH|8rbLkdfqHjd##ua^5g~Yid>>EhOxuyaY=)(%;E5j881{F+ zo5kaIf2C#5#{D5Aj~kWF)iG@$LQ^OzlhX+|z9AaDSTrJinFKC_QrI;`_o#C_`*k1& z6)%cU_BJui%tk)t=as7;X{3C{$AbvBBi?y5@^Sbqlj21uk#x3TEf{$2VCBkh4`XuD z;h|WuMXm3NR2M%aW+;f{rHN=+dQTa`t$UADo^(?w+EKs0U&`EG`2;=`JecwN{@TqC zV{GD36p6HiOSN<#7Pib-aYc==2V8+-MYH>JUP$#!)k!MKKm0FG1aX*KMHWI6#H&OP zU=>@Uw4sHE3$Bky>?J+Tu&P6N!nKt)lI4d- zBq+weNhs;PHhzjNp_p4hA^+`#T1W=8#R1EgSSvoqT z>FbWZmsRJ@)K(vSTdfM$OsB)P+OVPD^@vd5HN-ZCEi6s!M5D4(E5Np&mgJj z?~sZWX`z#`%^SXNrXFy=uDYSzeB@~h@qmFgq_|LP6a9QjU4_1F(1R!tRa`UiI&{Gj z+lNbS6L@TMCpyP-8mF5U4{RKvD?*D|`>fBORq1Mf@0V&Vs?M2sFhXgC!)*o8dcx0i5-o zRU9{rkZ-1t-yCg`F8FAKn}>wJl;5;4f3My#meU%3YHqPt1J2w*&&^%KA3O##JuV$A zj2LgfWo{CMWjwq`O!`7Z3pw}heD%`iKhh(&?qy;GAv@!gYCo#l9==p;7-U)8ip0#& zF}F1ROD&&I>Wdt>%~*PAT?UFnQJB6zW5316E4KatWi~dJcH#&Uy<|!rVYPhf-S=IS zoLqEuEBe-uMAtbwvojB}@+<1p$G=cj`4`y@3#K_Vq z6VXa1X7$gatWibw-Rk>RPk*A0mmA^8bYmYT3{6*2jCZF6s!j|bG5vJYrO1-r?8bb{ zVEV&!Z_L&)6EQ;14#@;RH)~IBylF%pdj~u48ADeq2T2+rxdEhnfV2pRNYT0U8nQGz zXvKr0W__8Y0Ej{Qc}@!QZ$&E4P5$CU4YR}4=xkq^Nd+nzswFD55r#HTu#8ihkOTEP zEpcxQX`8b6axw)Vphrhq%MUJ8&&H9LAC0{7${IxmFGB(%O5;uX=Duu z&b$q&9W$34Ymr5>>Zhuqv-x>NMtFMgMy8Jb3>PQg5ubC0!h<-?!0@2 z+WMR>mvzb=am)lH4UDUK^HeoC!r&*@dN=zjhTz zcta=PZC5U6ul`yQ^qzgC=q7OyG+rrvLEXpahnVoafrPZkp5I?Cp*S@8p;*~I2cp7o zlv(DMqhrpTSs9kG`zoYt^#jNA4gXXq_)LG-tV=WaCljEHj4sR@d9TM(*ndZO=_5ao zNtdkl$GVWwfX*9l(uA_}Vu2WnW3OTg?dj3;omU7le=DE+m&k-qqVKo5bx*dRESm=* zq7ZBA;sN{+edXBM=mEmi!;#hYw=4c)4ry3M=lu|;!{cpHVU(pFc1mMP{@H95_e>!9 zF=RA1@m{=tLs)xUpx$*EoE?6*@{zwly2Wxs0^5!m%l7y}ia{;*f&bjceDs!2Vb`Z@~2CO&r*059gPNF=2X4270q6 z9*wM)i4V+y(ShO+$*bKT{9Zf8WsNvQ=zbruR7*nYV^vMAUcI{3{Y^jhqWQ0;7@mGf zNlBl@vH_L_^4F#&)}Wx>9jcP&Bw~6RyXmH{0c;EsOR3cX8y1)3z_y_^W<&xbM>@xZ z%+Aj4PPyRK`}PW}wNIbPCy(VCloI;x4_+}X=keNyC_S);jyro6*-&=Z(D3DHWJ=bL zABNJ_)yCG<-bFDf9ozfL!7jd(IWqaNSUfHAtuLwSfIt8`<42c`>S=4Xlj7p8b6syg zaaKNDNmCC(FF&pe*nBXC8gW~@fgKBF=YRY)D<_AegW37|YzNlFYP(qww;T<;bJqw6 zreREwb!86{*!a@w%d}MUr+yNPdHIqLzL~h0h?AF)kRbGAc8d2e5D3QB)#zOD+>>RWhlH5 zQaUY?2D9wh)nEswJ7VdQ3D`rk2t=MA*y9DxukE> zw)Wb1NCJ}u<+kO%x5X~xhvIss7fWt=DGPrz0t}!CoTRm!KJzbBxDp$zmCDj;z0YIj z=&$JX@0_DGtAb4#XWVB&V1_wBfm45^>V#<;p~0$Crn8J65MfH;Wn_bs-ZVU9#nJ%AG*Aczl;#0S6upaUv5M6S2_0BvNT z&4;0l%mCozp@VL({2hKg*@6iKb5kO0n{|4OG|8PjsX7{pW09f$`DBe2vt?yqb~vr&&SvBU_d^n{MgchGWq zDselDHB$k)?i-$K&V~~b173Eoz*D^r* ztq|^d`c!UL9#4$VXpJcB%hLw|8L(iby@Gi*5VuZPxKUuxuwI^>^PwNfuvhk2MY>&e zAnXh~QkTEdds#-r0)(F`w(m`6Xua%Ql0y}W5pav3VI(e_*2^~iQl9IOxC@rYAe`ac zpzN&pp#TZ>vLLX2Vtx#W&~4b^U_)V`ww5e-PqbYZI+I^+1hDbYU>Kt?WNVQz?%_n( z5X!Srlqv#Eh=K>pZ?79r ziPKS^?E4u+`e`XGc+dg8wfp%_?qlD5fXFSH!h#1&oEP3sf;qJ|i8tjiIBkXohF`cz zTp0DXA%OrfyP0ob`HdNb`WpKXS}PUtjTOwPDzuym+oCaznE=~jo&pZ%0&eF5KDNC` z-FEn|lnNL?PPSblQ*pBL%4EZh;RB=cFyI|@*v51bk8c(!l*=ES9e|p4hNMFKslUHw z8hzautawcPN?VW&08P&#qfetvX#JCmvORVtgN6%u3234AwD?jtt(XFfDpPB$5F>zu z0OPZG$#UXS%F7RLzGa?Kn~lUW0v(|=FA8mq1Z(Y8%t3*&6P-u^nAM<_S)XtBMK@O8 zCi?pXxs`w;mLZ9~=*GGINB#qx&(fTz3+0%(I56EfKnONBSG7Wqg1pW`L#xvI|2Opj zlNyjI`pJeGP!0)EAHFLN0EcNZ;C${$o$2eeg&HW62WYVN2CN`6y;k(MWRi@>Z!)HJFD*H zEjcbA9(5VgSH52`F4zqavmUsVmX|RUL0kC&S-0l!t=WZ=n1bGk*TiCe0O1M+$YB|Y zLQSlAI$_OJS;EaD8|2jjs$eVSVq^!iWiQoR@bhyJnSicCG%EPbO*_5V;TT=x!lRa@ z73nmCx`ol~px6#VJ#AzNLH!TzK~UrQkksGw^@nq5Q$_|-1#RdE9zpMMQpWnJ4utbc z`sxA+?<%nopnR~?G!0I+X%&3$%cRH-l(c#+G666x=&ARN@-CPa!2Stryg=|on1S?b zIf6?S@AjR&JD~=5X3>sj3Ubo_>rZvqL0PMwL>j$Z8)m)YywD%F%d)wNg4sI?x7Q~J zvS1Eru%$F6%HQ!Gu=brFfAr_lO3>dVTa{Y2;8Cmj*OuwK8$9zL-lXfv#GckK=21;l z(xbGPu1*Q<-k8|xu_AA_vk6_lUd9uC{r*aGTEin>aK4iP4vsPpH3_9<`=!CG2M3L72Mpmq5pr0u^f8vH0 zwKW*ozZ+wFa;GUhXoEGo^pnH$rs#P8^>RT``?Y4GWuW6kd#4@x5a6r@9`XE$yas8e zy~NTm6Nxo8Ft}Zz=D5Q!Qs(MA%y)LALT_ z;>5;I>JE9>Bzm3u_^W!75XjM!!5-mVXJw5q#JHh~@HXfe&)#L9XE2ozcK*{$&kRgV zGQcY&mIbbdE8Un)>AqiZ=Rl>o^>NlEzQj1 z`!krFN1v^K_Ugp0^rh9V=D03Gkj;3c!^l(G)d1~^pp0Z!g-x|XI&V=1GU@|w?2H|@ zJ}GVGEWAIdtu0P{J%mS>O3@KgO$p$vuO8kBhRJXJv(`?0KV(|cNz)<#v_FTbL>Ro+ z$~G;u`BdB*(2r)Z3B?$g`TdpJn7$7Idy>Dl?E7}Z)JLeXV|{>!w|d>uLQ4une@ zbpdd}L(~&2B+%-Qa%F_;ZwHhj^LJGWt0K1Xj~sZZ_?L0ECE2%ulhQ% zkAIxwt@i<7Z8>8JUVGi#yV3dIYq0;l0(2QD0NwMbSKA~{bV87dBl_;MfuftC`2P#a zP0;9iEIO~e{FV?L)hk^w$|psokV|4xqG8KCyE>a;zovzwlWb2MWdwUu!cd9R$J7@u za9e*G)XfSDaX3Wa)>@gRlQ!E%#BTaFcxBAf)FvCdKe#zg+;w|n<&(dILqzca+vMRrDW_7vyWWPdJ2L}(&9Ec# z_MtQFZ=5R9kyzF2cke#CW5HYD^e6n{Ii;n>;I*|$f{Zm$EK^|@!8Xu3Cn3#O30;1+et4f|e zd$xE804i^~?L;GVb)QmJ=Sfe<)zzvdrlzb5JsHGp>a;f<3>Y zL(730AHL3tWf#9%zxuF*3+F>vxPD-L$FoN~8m#qROBKbGE_q3v*1$=aifj5Ql67O9 zAI&h0uPu(Gt$qGXlCJIWroA}u^XoJ6torT+Gg&FAUM}TPTyk>qt)S?yVA(NE2xW`; zU?agdB^O9p?XfWf_ggmp`U<}Mqo*f6KK@{yusYhRVhm*oW6Y->d-eq8+|tL+dtATH z@Z{&WQ=FWg6`rmTe`!@}9e>d4@H$cl{1eJmy~+Lrlgw|BK2O1cqwZX`d!+kd;2TM_ zK|?Ik!Xg=4xfdxLT{%y_3z0HeiqL=Mpy!Nb!J<`*HW z!`NqR3`M~(pz93jpdyJ%s;pEQ^Iymxu6uln4`` z*!=vcf;W{sz>cfCE^NYLeueG4CGzPl4UENyez!LBu_9V@g~VH)YIsA|DJS){&W+ zGc(0m;NXNGX6ZkK`IlU;S9clO_}_3UQf1gd P?*eqLA~g#%>>mFgMzp6h literal 0 HcmV?d00001 diff --git a/ml/logistic regression/images/fig2.png b/ml/logistic regression/images/fig2.png new file mode 100644 index 0000000000000000000000000000000000000000..c4438e2bbb7b5e1049ae6f1a3fefaeef784b3a7a GIT binary patch literal 7535 zcmds6c|6qH`~SFZ(~YUF76zf4iV|j`?1tMy%Z*ZzeN0i=$-a!Pk{DYH-K-Cwpv0? z0)n8`hs^h%fS?~x5F{4*6Zi|c=p1nRkm`9hh z+HRfo=7rndND^uw`dsUpzr>~TOV3K|DaLI*?3do}($Jh9`{(xLM7e?;+BTcyt3U3! zcc<^eSSHu~CfAHO^6oLsyO%NYBUECuS_V~6 zP=NF5F7b~swYIk2W&2FEuwvpc09DTWr56|cg;w7ri=vz^hPI2%#w>gwWcD)_lW6w@@}&2~TLge}jy zkDAr~IBWWA?OJQ9Yp!NFR@KG1E&q&f?P{I6E34MOTlQTBkb*VekAaev3CXcoXtP<(W+?(>k_d4A8y4Q5JZZf!-6HJs zEM6q$+swTmxiqy(9Lmd8t~coG@qS)U-SuG|Y2pFhCR|k8gEKe9K+O$nHt9(Ph4-29 zA8zZtp3a!3@)wnq)HxBiU=8I+RZh)Y5c%zq81ItlF5kxW_*f;zjOdA3*|L@s?vLFT z$tx|D8}Zp)OK#mW(=vSn`|2rX^XpL2-@SBX^xb1C+SwO3?%;?AkEZWc-|N|V(&%j( zyg|6P=+o@SG|{%m;XY2?2YIKsXuQ!GqusiMW(#+Bce1<=FQ4$#w~uX85$s!v_xWgY zG(#7sZ(z_kY72}OI^Ps8QY0KW7#fq#R3-p=A$QOO}}B#Tw1-Nfjj0)Po=@k zm-kjYi!4J~tA|H{KWLLl-IFAj&f4Cj)N>s6Bhhg0la_nyQPH~An5mBPAn#Y{I+_5M zKXRX%nwlym37t4oJgLIJmffS+XKkpIWGrG*TJ8;tLbsvk^_(ys9Z&BqG!O=xGFxAm z^xz<8?Qq{NHNKLlUcjBO@QkN2d~)&ye+vDK zg2>OZXsq#yAoo}HMOs<&pQ=YvQj`&&C*N8k#xuJj^Fq(|FI!7y)PoprwHz24@8;Oj zkCvkY-ELX`oo@B{PG@kgCaNDhgoKW6dc1;N#tkHXntNGfbw%~%=t)XZ{5GC^(ZlyJ zGK$@s!0zDQLITU#L-sX|wF42Ms>3qr8*SoG8T`hRqqnE#J6)>%_B*Yo&HYAi_D(RQ z>uy6rx4~-7x06Iw^lN_~-k11&n0;ILZDkH$yYV~itKIdTUb-9bO=3&@YvXS)mz{IC}8ZPjCjRV&Ui|#)t+!)?n=Hot712 z$d=YYA!oqI5##0K5rt-?(^v`1dPcc<_!8H{xhsl6R&R<%<}Yc{bv5Dj&CX?IPgY}X z!OTr;iOFbNLHr)~wv*Q4dro1(=Ub%K;_V$y?)6TzDS29!cRa&r`eeZIkp7g|+iH~f zN!6?_5h)LE~nJQVGqSopOEb1bG9(Q=(ZNRW$#HuIis32O~9H#yWYUwvVEB^vtrgd@U=Q zMV-_G9=$yg5ys^~W)*!3Zrq&M-*uRJTs5a*hH-XcV(eopPMxXLiZn+#-*$%&?Iyd~ z(r#Wo6|wGAqxC4IS(Cbjxz^}hm98MPV%QfNZ9VF0mhSA38Aox2Q7AVjnCJ6nj+w=# z1(p+9QzCS#&h&WMpU{on^pY1DiP55f`RUOIEfmG$mghUM=Pz6sF1lMYh*#ok;n(xs zNT(Gt_VL?D{BBMpT_5vMGyE6sYVRS94$0#c)6=QN`|ShI1VH4i#Q5|hW23D}q#3uq zubLvi>4u@nL{p7WV`Cde#dy6$^zhtlPO#Tly~MuONq+UBK(J^-$OF@}bt?0b`H;Of zKSa zy$O=3>)_&Y=8u_<2ce&gZzAs{Bu8_ZZeZmGU!U>H)&mJbp8Ra)?_1WKrWR^6^-XaC zs)8iqbV~-aM!1H7Jo0S63MKxq)Ji2$aDW*48jVG)vxs~vTzfJi(z@pD11T!Ar&4L# z`H+elVe)tzSkC?0>t3Sje9xL|;jg^srcw|CD zhf9+goEf4H5Im z!A`dH&E>?R{JO*C;IhjrvjBM$*-NIe!)3%2*-P@A2W}ckT2($6yQ8_tUe2mO!Z*kU zJ{oN4mOLcrFk~7_TOX18qO9Ff4P;OiEb>Aq_FiKAi`y`MRmWKmTqzQA7(0M;20s=6?+G;RT)Mugxj(8bfWipAS-=Xb?rP@qlOKV9Zr|JiyvUek_3m7|e&1P8w zG>4^Qnh-(HhqI+?O^`g002p-*5Jon@%*aMC`v7aqF&HyviL4&;2@xu9EiwKRcUdS6 zVm#sOPtlR5^KGQG+?vSmi`?p$7KM}+El?5rX1ctS+zU&$yS}<3)m7|GRUAHosTu;9 z2UD?mG-7??9WA$GrM+bV?z?QB=5)7rR)qL>hfls9h-@!(@Lim4PZKU(5xWm`eE%2p z#eBy~{nLf^XLw0Uc*VZ@Bu&Bd!0{dhot|J~#rz0`7C6>fGLUNb1g`bTF*H^(ya~86 z+At#{0yg48Ci-;)zt{w{j@4aX{|4j^t4GJwYd^hr%PvqHraX9XuIdMk46y2$s9lX< zxg+vtBRqKvwEgHEOP>42BtZqmXQ=t!Zbf~4{g38x~JW%Y&@xo zL7`9tEavRxjTE1+AFJ?=3sAw z-4&ZmV>Qnrj?n~EJArt%_Za&-a6RDgZNga@E#NL6UKO=Dr{2x=*SQbnAh6zradh@`DgjK5s1 z?TM)nRE{Tv@k&)XJ3F7;e6tOG{v=ykeDLivf=_=#icNW-XZY7>C9 zZz%KiVNCORvN|f}&td%Gp)X&^0i$iKu%5}RlKvYMF>;CBqNXUj|96 zQ8KpA%s3vNpL`b!=LVF^kJ^Z-5%HZxQkqU^yWLXOa`(nC8F1 zM!*~b`{yF|z^sBvWV8eW$muPxY3zRjdI&{=CZ@K-CFeMzr+vV2n$IJs{y=%>Pr}Nr zat0g7Wx^(D2CSP^;K3*wYxyzY{AD*C)9?p8OEv=VuT9|n+2GD`*%jI8H3z`UuKI^S zWMl<%uTSV%J4FN-3cCGUFF4iYgvi>VwY3PAAOaO|ZcQ0}OrT>MP+Lo>(=XMibph=T zG71U`d~UpE>SZj3&>RkgWB&jaOpmnE{03h7krrn87<^=SDfKxByWsv5vnQ0I^{CfC zbmv`Cs6DCbcG9EmRBGXt?G{;^z&4b^GNY;HeKem>H6=5h?#L67P#ZIn!mzcqMeUjZ zUWPqAMXFqfy3JkrX0le-m%IPy9qhgCc3VbxNVjjj!obqkPqn`9&!w%NZ;A>+Pk^j| zO%18fVbeXkT$UE*#Ah5F90r0UBrJ^Q&UOcOc@8W!-`nTQ+Gu@C>P!b}Xu8KAUvC*6cYh4>; zXEBrtY}AAo9WGJ+uu-GgK=)CH~yH9j>TM<2C|FP`>2GM}C48R3;& zR8+uOfX-~IRIy+^Q&J@Bc!uv_d*gpD&RFyNr-g{sbd8zuX{+cAfOKZiu_!C# zAm%>ywDDn}gZ6mw8dPvBs`6S zr>|TFDkOs2?N$L6HkAj+bNzBCSc{%Dn^U@LsybXC2|(G`eDv>)cn>~>0wg&-w7x|E z%L!kJ?+F=iyHo1wW;k6N-Kq?c0|4_ubTs^yz3e?6?{ubY+fx4#PyUCQePFs76 z1WN;Y$*eUnkghzCLK%?)4Ls_87NL0t_z7f>0pmtBIc^?&&4>kAz*({x0KY*;t7|HN zppTLQ=9Epq4$0arTs8{=fm;;|YWID#IAApgx#z|fG;o>CdkD!5<}vUg$Mz=h3x>0pvb0s75<@5|h$#;=Cg=dJvw957C$P}HQ=Z8umuh781BGsV$o;m3ME z$3q9mb6n-0D)C~0?em>&g%0xmgG~xFY@UK$c#E~D6=-qfSIj#by2qJ(CnlpT4o=XU z3%`+WWMIYuz56uxWbbYt{s|4e) z?rr%cpJL=+*2n*=kArD4Kyem^df`>S1phr1u!1DuLLE_%vY;vk$z8?{xy?-sfU0j> z_`iy;JLpH;UA{s7mo{PkV~wW&?^L1e$5xMAUfkG*BeKbN_IlcSbUu3=dLQ%);i{FO zU=UcBq38c{v8}+4SpEBo0Y)2mtr7WL*1%HLWpSJ@w+@u|T~ycxpe^?A?sve2cL-E0 zvdBBE9)bqi-I#yA)wP4x9*YQp~5HG!Yo^PtYd?W@)otFv^_jIUQ?&lGKm?9im2-qMNC(1W-JWE;^%XDgH7o&{DsL@@eP0L(^g~0I%vuuL0a*O zy*|iEO7Bt?EM|g@rXIxl_HEhg85cY^5Ni>!ldCWui@_Und&0^&aWdm-g&4y%OLN)f z*-Y2p1pe*RamLa$QeKv2AH^wQHnaEoEL*26yZqv*I)h%>H*Kel#F#zy&2O5{2GpEA!^c_IiQZ43?dbC-Ul7ab_mngxX6i{UP$lRa5GXQ2&0O= zPC1RfH(j#zCkc!DUN|kV#0H50(@e9|L`{Jjbs%!j(nv_b$Qf37-}DU!%!ZAvRwq=B zWyG#pzx|ewAdJSx29G=*`O~lUlvJAi35S!8p*y&q-V}|C-IuKDA_S*oZZj9Vcp~zW zM&&ah@rvz{)cw0`PdY#*#0CvCxb;a-4sw*9D$xT*xAZK{@HAI2zW!SGA;@f@&HDi8 zn$(%$J0oVMsb&?U4sauCG5un=NGRnU>0h@f2*L{{vy*~H?UeU=GY4#dG!0Mri8y$kCCY#HaKm_FenyEUL(z=eHdS}$q>W8ZgDC5 zLoiV=Zm*l}P+lf_%nbJ8Z)%e=+rSsVZ+GEV_Q@-PJ?SC)bb&D5IS{U;8hd327=0}2g zrGrnb1Qe-&z(Dp`aj#;#z5j1>?%WxdAG{wp3Mj)h&UrVb9%m=W>#(`3_{PP{;wxRa zb=L<(dfnyh*4EaI@zUyXYSfCqbkN;Y_SFBq{A=!7A)w$qdn)w27;~TfnC|U`B@7Ie zWNS2T-Ey_sQUyp|9yFe46gG292po%jYlqcKtPha+$tenG!#&3Z@w$7kK1?|)-)u$D z_ygaV0Kkn*#h&241pWwxnkwL>+Ij8ZjD|4pYM^8EWmD$6{?_iE=&W=)(jtAJ&{I<4MXs;o<6L<0saSX(-tx?!1o->Qd%R)3-w_W+hQT`vc{Ez@XC=7x^ek+6cF>=pCmF)1{ zUgcwSKc|Xjd+_iWkc6$-HC}-dOn)Ul9!H`SqL6y|M6Rf~G;wV9F1?db)nP7{AAR8{hZTrf1q zoXsUjvXr7uk?E0R&)Z+R=ji<;?ra{17b~1CT6fXtYnAvCR`LAq8VDNNhK8aX0g<)yiP^zE^MoNN6Q>qYJ2oRNGL3-~h0@4xz5d#EN zgg^*HdJifjbPz%)0mAPAcHP8%|MKb9pPsW67F2qk2U^lk0i@qO=i!p+&Fmh!C$*~$LZF-Hh6S!zs;Yv4}^Ei zrQs=Bi#Xy&X4FTY#ijX7s7Fh8@RKJ`UYx-{ee&evZ=Y3? zN=i4Y7t2tMT}2kt?1b@2cyqEmHm#|tsZ?7U_C17aM1cNXz?OuIUwpFquB=Q~N9W0x zFJF?%lsD9$>=h4HiB9C?am5sWbQP-W_unskqxJ zi2feRGa{zB@>Ff@XD!*%$*DjEHA)j-9NJhPB4IV@#r|t^*!lKMx(R(eldcPakvJ*T zKCw)HK}}j~B))S&2C(-MX)M1`=~l-heg+974f~CK zoLyE~oEda#*%&uM$W69rW>v>*wD?pD4K3Fz+txkfBvamr=qDu2QX@B(Bjx-S?&iID z6J_Csq5jCM@n4ze*BmC%r9m`^RuFe>TEivJuVX)xH4)X47Viz{i*6e?1%-qvgS#7J z#Yi&rA2PNBAKXiP=kBbGBIpiKB=j09-?@%8ZUhyCu%kamLq~DbI?N^m`0|EjPXT2smPE%ku*a$_?XPPEA&2mW=RZt+0b-*XJl19fQ z@QB6I%;b%@v2S!*ca5vY>{x3S*};DzFV~hj(K+vfSn`p(*CtHyqB$*OW_i)MS80_C zRLqmTs=%o}cJ6=Q9307-6@h(19mJ*AP68AuQPR?QDT*&&zO)}yIaTg9shE|6rsV^@*uJA|s35CTpQ0@DjvIF3B^v`(xqLXDZjyfLXXOr6=K~>zB}@%Q(j&knfP!9 z5$I-3>>tWT&0bdM0jx$%>+xuL&AN+OxobK)r6UrmxiS(9{;a<*He=tOJ&jYnr7}e7 zpv5(aG(J*5({(1LE3$kN)f$;MiLIj!lG1BH*i0XS`cgv`j50DrpB|E?bEkW;^$Tf7 zBvId7&%$5YNh5Sk5(Dd7cEZ1X+$q1&>=tOlKiTb5YLubYtM+LIwLwFXUOwiRGkQC@ zCcjAa0gRRXkoTq`otBnXI$5;KT=S@g$A}I`T6(R1J1C1*Y^AAL6K14WnIhZ0VAH0# z_MMG1yFs6A9v)AG|B)8x3_;bb4$k6|&e&9}ar0F?MhD(KKO}<6jPh=hDqo-pW1999 z(p9LjoeLb_7M74mq!KpQAo)&2ky&XDmf(x7X^OArYQ0@AD>nbKt<9Jse0F`ejLYZX z{i4h2r-g)STb|r~aXX1)4wYf8!u2}l$(H|>a<<%lWSDwFhc>09)*xGR95>0w4 z??!G)WiOyos(oyNe`-(*HyNg8da&)lNEm3Pn4W~~BU=n(VyrzmmBuLGB zBuL?V1R5tPAe_^6Z)CmG#IjSgK%>OO2kwc1g6jf}P0AnH>pee7rO_GV z;qLaM`2+hu{yYm{imz)1R8m+Ir!JSNL{{t-l z`IDb@-W#WcjfIU6Va^WRL%iOK9L7>fG@(%p8)n-Z?f>*yQ)2I`rQBqbGt9v#Ld%~N zv*L~}S-$pAQuXEu0kI>E6&!UXb0&HIffWYcd8nbp>!*v&;t$W}$HbyLV~lS}+`BwV zvb=}+AU-LA%D?{1DfM;L`^)*07O#=TrYYuq!LwTZni;q=p(?N=wL4iRNs@};QXV%$ z##*QJdNa&VPI83VHomk>#tezLP2*yE;o$k2K&q-4tm>MEzH~I-Jtl+@9k+<8Tx1Sc z!A}UhM8`Y7fGjkjr;;^N6_?>wV%3=PlKY5tph(U7v>P}1c1cOe)bqdILWtLIwV&Eo zS|D;|&(fn0mdRy8`Wk0VPVgMox83L!_c9HDkIV^=M*8a%jwMmNAE^fVTjZw?` z;UC7^oR^>|_ROvYl8EmejWIg>ta*6A%#ONPiag{L8e)x<#m~Lne0o4z)697^*~9D& zvZ~f+^vzFc1tUp`Jd7t7TJ^2{tz+Z^wY+RRhmLkl(#9$YJ~^g(CYBla{!9rAqeNZ5 zIT*Z4{+7JtL1>6|#N+V2ER$X?5$Jr|mEwy}m&LV}L=#gSUcgzHcBG-+=UoPeBsQ(W zJ#5FL{Z$eyTnN5gOwS+*qfF088`V5$svkL=Tqa@h z{&k$*<$zN4VT(fUqoL@}ve?=yK_S$M^uSLEZwJlh&5U(x5F7m#Or&<28V81J)sdUD zu?IwI`pSKBtyr0Nc;n_6?NQ@+HOEI!WRnRLxldIOgatpUpXcNcSdL z+D4w`VZh{`qV}a3J3dm&Zs-LKr@Ke~i?!DYqc@pi+xkfjP$`{>)M8>%D*PZzt)+F+ zuyxiaxnq-F6=PNSzIc)THry1uOU=$@-O4)cQN zd=P(&STB@e-lTtN4F==?T zzw=|gdgvY{Jjda~cs5jBU7haTyKjIao*hik{2F0Ul(^ z@#i=G+CSiS`~m`Q$cjore}8{A8r_R;Z)=k#HfJOyodP0EgpYM;$4zs zOl~x||xn`eqbkf2_?i?f8@N`xhM&1%(h%>}!44V93xDZakxvoT8>+2qLM5 z@qL}&0PXnYKrg)Mj!tu2X@3v(zRaMlM}z$^?}ha5lASG;@Jp;2P-fl#ZH!yZKDAN>*MqSj^meoo-KoP9+Em-p=4Up zG%?LS!6+DaN6gJSu9WWExfzU{7#4!!E5w0ZM&9^VyKUq(2S&2Rx|}ZEDfW+%^&R4z z)?-2|NURn|kV`#J|1e##V{~b0H&G<)0ox1C8Q|+~Oepyv8~s$Z0wfM9ete33;t+$A zsMFo_N#X+h-!0N0`qARNA}&x(cJ(k9@=E7>?$4Oj6U$VxmA! zpbjoZp>a%j$>y5VKZw*K$C!C}(><^*-5U+D2YHGxvTxNbk@e0{G$6$l)FHqOaJuaO z687@ggeKz-GLxNO2Si^w8R_dvSyfOS??+lcWTiZL^5^~o2V&-iv0+W55^b4#E-vxh z^7ai4(L#0|Z?1c+ExW^WB)0u_bzwJGZj*?!vYQMj4De{qNn*mnNdl0tu&~&#U$4o} zIxo!(%2|KjEpMc!7vnx$pA7TazJL79Q1>k4N3=8$eAAA=c|1Hs85_I%^!alIN`3v6 zxuqp#N=r;^>}G}&^W(=IvzkDRL5f0S#`v?Xn>q|N5ub#B(BD3-iSXXM5I%;&Zrw@( zY7ss0I_J+nP*Cs;7!JiV$xHYu`GD%-2~2)IZg(IQT59G~ zF9B7fg&LgozmCeM-P{jO&DU5Qpm^E=QooJD4-XFm&h9m~3WHg{bDIE^P!1NodUY{y zF3c+F+ib)(2Qm|7Gqz9h#BR#d;ttJC#9B)hHa2CStjSjQT76yJ=#L-I%cJu0q8&0| z^=iXlfqpbJSB1#89nn~IOp;tig^8v%AkQ*D0Oud0E4QPfId6>d0fK%*pGoB_r%w#l z4gdrfD{KRdi@~jBjCSZ-3UP4}G(s80R^?xT2P5||YHljEFa7#S88=M_kY^F2fe;J3 z3;N~l2)ze@ed-wZxrsU8lp^d2&VN_lzss|k2`Rrj^vj(UN+8`8tqLgIuYjb=6qT3S zqH|Gt$fS;s3F8gOu@MH779B*6`7v5<)%8uw6%m*2{x3iql&j;Xn+|TJGWtm03dmz< z{z#-nOCKkC7bBDHRC8ZhqgQO4J^uk5pfAVSf)suS#8S_t+>mPb<*MvwBudQ;^;2D? zWYIPn99_xC9{~7TmKS3RaRZHPUJbuOQ=`twYldY810T7b&FyA1&)f+`pBnwHv~S?4 z3`Wm5XM(AhG&;(aSvSMZv{1+7?VDw5?)ss?)=V^AM>Xt1&$phbzV zsh$7PN+!Sn7{viABbyF2KpBr>2HE}U60Bt5&)GXm!i=O=PAb~NV$#Ib0Os~H(&W=6 zQS{pIVs~~QC+q+q@G;rQCr_eYyg0$F8CbPhBV33HVd281`w(I7YYVxW@z0-U)Zn%Q z;JS3G`H@61iVbDznOeyI@GdpHe2W~S25$@g4uU?ELT)IhNADWV|AnBft*vis`#M=! zN$FWlO--^@%WK_OVO78jSKQeOS$`n?NT!Ootb!FuZ@0}*fro5uZGD)7!v-X{bLY;s zuU}W2&dm&gdiBVBe{1l)t3Y0^Ypi-KeL%}es?HJ-i7UWQ~i^u4r-6}&Jj*(GYNQaTv|?&q7+1gP>@Q5XocKscv%d|e8xSG?!AG9?TgZG= zzttv5Gx)0>WJ5dCVZQ-=ZPRKb!xQ0*S0Bl5(o3-BW(G0J9yD@@;j9xTi@K)*q|N@q zf&wU_4BF8|?Q;4|Q&5SyVe>mi9?1G?mRgB;9kRiKaqsRMC?I$nq4E`$mosWIG8Y0k zBC$--IZ#_wI70~Sp-%O?d$}a81L9|@JrBf-{|4=4T@uozdZK~AtvGU(kzS{w?zLTL zIXD6dj#g&cr3}KM^ji0#G(x1Ai-{~_)Bx@Zccox2vZ4Ps03k+yS^Ay^W9EWR@WWK7 z-?(G)BG!%3?2y?In?y0)>sMpJ(ah|rpQpHhD3DNq7eVi+smiJFuNphT^u8hmU`~p(I(I_Ueav*MqWD$u- z&ffo`j7{9|$azOZHh>uUUM~Xqxu>gr5Cje|kr~jPzkD&dm|6&qmSL1F%>p$p-azH} z*0A*3^XpBAn^8H@vzwOipD3h~HdRW%j~*1zuw)g`aE|oePF=D!sn41UW1`CPyf0V( z&$Q>}ty`ynqIf)+Y_9@eEZk2eZeNn)Mi@c15ggoVPQ)_((Ux=h-B&-xe4?RrysfF9^ZJaK6jo5$p25!q0rO02Rhmk58T*S>YsB9)rKC5ZBAA0mj{m{ zb_3N{=}SXVDru^BY;54_E#L9(*?uC~>9Lig0tt9LkpT5Y0Nw-cNsv_kJNzih%hL@# z$bE;L9`g8c6FynKxvHYVJdjSw)DH+CaAkdqSK$w^5#O}=?ExT3&!kG0ZFyBs zkn-=w?bFnIFX3^3!6x;ssOCN;w;;rXd!Ugcj5b+4L3PG+bBv8Jl@6*Wya+L|F{Ul$ zYF@_~)@dS5#GCs=_#t%wK#&n09zb9!Gk ztYwIa#vv$jm-qfB=cGqd&8OKt7}?NY@|z|U#%Y5xOC&rP*#H2|gR?O{i?;ooDvaqw zHXH*uV9|XoroF!bpj$~OaZKx;Q|Z12CLYs))UC`UBTTnk!b6;q!%;sBTK>jSethYF z0O}=jmm^3wf$cdNd`NZ{M5L^lJm=w|!MKh#WX$>nd-%j`}W;7B%vj zVsF?2ID@)eYkIFkrY;b$gS|TY{Jtxu#>T`5D=K#4)wfgjFeC_K7c_8AhRA<1cYEJi zS^wqxz3*L8!Qh`Ccd_*(k8YFLN`0m)9Y3m^UaZ|P4ls9C5IFMjQa*}~A#^EDJZ0SPZJkV3E z`h9(Uwhd7yJx0@gQqFiynEcAFDj;kDaf0I_UWtOr-c6}>a(8z>eEhg0Ih{lzC5yF!_ivkDfS9~(hcP`+=`YXWUCxd57HV`F2n!{1SooSQ>Q)!XD6tB7Fu z!7!uRs+t;)uCHHjJ0u2&gm|qOodz2#ucMqU?z3r6#EZLP@Jvpvrv zcp!Ru&vx(Kn}QBhxwXFgLu;0Hs#x#%*V#i`hY-OQd8C0dd3bvbcdaCpRZTT^`A3*c zmAB1y2s*b)%`5R;0`B}xSIEmwrgh_7E2ou`RiHU)t5c;OK=&<4!ZE-HjvXAy__9^A z8%Bog_Yfwff!Y7ovSJ%-3fHso7qhO2`9`tkVG^7ALX`T1%2u|0>@bTXDLVdiA=inn zgJD9S*FZJ2_;>H#^>l_Po2^h$2p2N#>HUoQt)(w$Lg+0syj?7RIfd4-Thr;QvrQaE zY9})hbTl_Dc^v3+Cph=Ky&!6kbX&6tx)or`sx6sSIO|I}V0IMSZ?jOhu$nwMq>32N zPPCV>z2dBAJ{Pe*82E#Z= z2-~{kgNq?g{;jCj_u9`w8#6n{J~Hj#zyeMbx5_J^ure=#OsT)gjUIXZ&daaTx{HPF z-}?F4Zsfv&BUSK$?r47W4V+ZdOAgz$;wgowr|7@Uj~e0GUT!1uTopuy^c05T`3JW zP$Ki*uoCeJQWqLKoiD-61)<{0ovrxItBy$^qcB*%%8EkX7a)Ve{QS4a5>9yzNf1Wj zu2fDuW9A~p^Jq5i_j`cG?GQXX%}7{7&mUgMb9%7UVzc%XixYiJmu z-p09UQsJeWDXz1TL}@|Btt}<#ku?IQIy9?}a&d`mP9qS(aCoIbhMF1RDR8m^=X>s^ ziHy|>=M}mY|BDj&yZ3Yb^k`2B5b?o)DH3k-u0MIgp;LsysIOjm-kqvjvH^enLv01um+pd7Okj5tOvLvO+z^m>teWf|NyZO^yrRNE<-zL5fiN zScXNEH9se(U1(QFhw!D)gJ&o$o)3nAp@{i!L2M}$zl`*B7Xsw-c7dry_ItYm)QiZq zO~4{fO+y26@!~~4Jk@5Txw+Y%$USQ>a0}q@<*Vw;GaEnWYht?omgfJ=41MZlw;s2? zp?44E2J7S7Qj=f#;P64WDSVurHp|q+{I+oa&KrggMgl=z?yYuCQ?# zCiPtIs;n%v3v@<;!Q4!W?yp}{jQdpRSXm**j9_R6E>(N0){CS)W~MaD>a!Pl)L}Xw z!OXBK%2`#+$?!<8=?1gO`0zvG1-Y}mygN0SB@zDmkvUKbbOfzqnVGwFEK!CX4o66Q zuV!9KS=3EUsFGmPW4gi2^=dwkxh;O)_TD|0a&1~96UD@Gl4r8Pke}DDqyDFeMqFA< z>@8Wl`w$G~W8&U-$VMPDVd~l5%8GN$G$UhI6z9LBw0nC|nb`-oY9AOW0{8wG5Np%Y z@N!DLvcivfsf-_eqWdj9+K|XYOh;1XK*=+PjfXj{#=m3sqH`IE<>|3fxwxr`K4!td zXP&4@+l{79{{#3R?XW|K4#~Oy(1Lw?xNn3ApN${JMgob}R=DP??Q9>fRx=Lx;9sWu zTR-f*ee>~TMW=*h0d+qUe%J`R{B~isUX?fiB*DWhz}DMm^46N4k54K(I@+Gt({qYW zr`r?F0t2h#;^TF};B@D~3eyY(9rjPuHtV|<+a3ckb^vupY-~?i(o(q0k9Jb4oi<$ErwlMHp2B2`_R_$kgOoRK zPRYLm3ftUrwoj6MZdzJ8Bm2rGCLFoXdA@m9zU04p1(H16s%$%7m<_mBBHgmGLSy@! zrs84#{#C)j!3#-~Z(Uuhf8zUp{=C1L;3jh&Inu!AGhj^~+!i0P_cqyya^0M%RJCJTF;erH2`a7vMTerin4w#w~_)c-~ zboHY0&GBu){G+uQR7-=3DRql`pl)VG&$t2UmINv#`WC6`-}W*qTN+RwB2CK-`fyb; zdlCZ7MLWk?Js`lXux;D9{nKtz%gTA#sTPJo=5_MBvoYI8{o*^1QBaCYdn_9<2fR%Z5pQ|CO#8=7j?2RRmz%Ld2WId_$qVNnXu|-zi)_Zr(S=PWVSJ z)dU4EK6>=Xc5Qj?$3J=4NFA3|-1V$0jZfHxY)4zt7Y#sL(Q3DA^V2(KzOn>l)%HCH zw5xtgy{3DLFKy;kYdrTIU>p3|Qlw-FtQ?z$`c}Dzlm1E^R>lTwhAn~6LyS|}e zLPv**yxYBd(Ur5+sn#0j-?mz&+MpHX0>T`IOtO{#F~r9qA-VC$YC zeQbKV)aK}(?Q~z+q7H~A@QFY`HGNgKWgU&^>FI$)LKoDDeQBqnRb%1mqm7^Zz8~ts z_h!NE$9MkTQg;Pn^~1cpl7PQj)G?}(!)CeRJ-r-{Ti6X~HyU?-y%XunwxvAVJ$0Fr zCr_%Ar>Cde1_qqTl|Hixz)vbmfS#YO?h%p)uNn0#O)|@yQA)kte`Sft7{h8 z^ga4r%9F7;)hj?ouhPxE8R#{4!W{$hg*ns8C1$rn&0o$0*|OC%uZ5$QyO@m}8NoX5 za?j426u_i4<26;dE6F8iz~=pZ)9ka`I`DT3=IVyjlcvN@EXy=bnMtGCHLps!$XQ>i z_)xWQGdHo8(V9}|Pj#iE*{asgO<|ZU;Od3}P2uWc7N;qk57TN;JNIPM`I!vo%6Tbf zRqu7`mnYqgm_^~3jelVpXbf%LVk`Vi21`3y+$~bs)-;VreqkBJ8_f!0ZU4` z0nB_Gl`d6`d_?P2dK_h9&(rHvr5+L5P=KL^GQ~0Akbq}3Oc-1WO684B{VllUwsY_=$;`Ze3XQ#7Li3FVt8g(1OG~b4(zd?qU|yk9v36P* z(}IFf)(^ID^JK6J4+G9#cvi0uQqNvxs@a70`S!sQjrjSsiC^M6>Va@`qp?AXE0{O&(7%UvTLCFiJY{lq>~Bfvg_%365|MG4(L_PS0M zmFkZketH0;Y(NKU|Ni~t$K5}E{18@9=)hO906pwfx9VhTjgJTGue1ZBLYqN)7Sa*E zdtcB_Aocto{sdm?*eM>mvP1qDPjj2l9-txl6l`rR`wd8*|3}r@6tun{`dfWk;~KW2 z=Z4&RI|nA?(y^c-$S&)*_ALT~P!>6S)~i?euPG@hn{_LDA`c{)4d{le-8y2T)UZI1 z7skBq;{PO{G@*D6$PbzO;oy|i)aI&Pwf^6@4Z`h-fr-xg66F#G2lsAf`#rMVKwYJo zV_<>7JNVpAVp9vsZY@+blLUqlrw?##^L!3J2vqSLWED~ZM#V?c)m6x5Q)AnL-W*f+b>DD!wr_S>PtUdOC^ zcO7F@RxL|d*7my%o;mg_&n299Q3zulZE|>;?X7rUmJ#5r&=kqpfW;OQ&u7;#(V>_U zkA2p0>RrJ@rd^D)HizgE1o%E&Kr(G{PeFiRI?fck&pXO#TQFNmSP=29W2*V)qzWvg z{L=#lf}?n@x?T>_@{gCYFpBR5rm(gov_0kXLMbH{rZ`_8A=!&S;`q6F8A#h#+zJ$J z63cz;bUiMFtIPQ^FvMn@1q{1Jvqc_lh`9ta4y>-@=d)zQ&2Iftncvyz``%eq@h7Bn z+K~Y{vYqFuT|k(>$19yzcPJv5*Dd^fRSd}3#&Gg^`)pWD|82kbw0K5L4xX#V)_NzLOdV|{8F+qe{f`~E@$Ks1sLJMS7L#JLsJB&-s=N% zJ%xJkIKNs38?JU#C@K;JTycr4w9%`MvuQV62-mC_Y3mOsTk58N5sugCNmMDJ-HJ{V z*Ki|TLoiaQd_26M1oA#N0avvytvGMnove3Sb;y~})_MD1E#o&*OlhwxD>iQdr0Rqon2;gFHI)d3%r7RIl`PzUXb#d*LX_c)7$_SxKMtHT z1{jW&0}=IENw`9pQqnG}yT0C2;iDZo0XGWP4Lp?p@kw(OicL~btK#I<8~UlRLe-Nf zbMI_WL*Exc3Z7E#W8VlgKR4$TrW)VG$%~9;qq7GCIbnUIiOVf@Db{l)5h|j1Awfam zQ>S$5BL!rn6Px&_f#I8%*=1myW9sIVvi~SzCT88Cz}87G5Ab`Z!dS`rUx>UgxKDys z9SH|C0DBY6MYW*?b*98G-xkkoQP%W;bu)B=b_e|`*YMR2z!&17UYEk+I+UIkywYvS zzilX8sQ9wLs@>nm5AAFy9d8u-%(in*(9A=YV0pc!RLtVDw?os^E@?CWH|JeHP_FmJ zS>8dG40m?hCv@pvR?oaQW3r_b2|M}7t*xzxxw-F=!^6XW%+9)z2Uk|suPGs_!MFlP z`12AR5-OkX*zqR54R>5|0n38b6L-wk;*-DB9M*~b@$C5NiDM+kq6p6XNIoG{mEOF* z)9V*i=JTBm5_!=BCE|@zMM|}%<{~O0rZGjK^rRccr_JgdkC^J(Uk^!>$d7w|QSkt- z^G(e2R~{-iZb{!;;q=f8Z(kwBq*CRm=Zh@H2iH2%?>lq55G_&`TPqEQ5? zF+U4k^5d<3!WYt;pI(ih19}gmWp0qCn&W=T5P3rY&gD@wo3Ji`lgSi>y z;8(@G;v*$EBGR}5m@2IO^{#TpM~=Ek9#M&t@LsYY+Kzn9Eq`HLi?);Zs8!yR23YobY`Gj@yDKASx z(88Pp=?WD#)RCz=&%LS=JCgH}g~CJCR&=fMc{Om>1EikX1 zZwZGOJa>^k$yGa|U9%j&Hr7=+g4L|K6gZDSoK_y1XIUAK%52=Ox! zX~9jemB(YV67|t;bwF1xD^f$8pt(_Hkf|{!BolXj=979z2%IHMS?-`5esyal?Fimz zVA@B|#ieMfuS_3!L&u&F)qdZ|`uq0YmPshH(kq+OyqleB^)`yYBh;}`OH3!K_Up?_ z*{@!GMmFev$bIYXrjiOQJC9349eZpIj1!mGHXfEIw6rK~tU9}mTBMz0-=b8K*dW^= z+r9kEy#)jYyJWnU2j0{}&sMME!xPfN;0N|C9X;x1drT{dgPpyV^xM0T|9v~Mq3U=% zol{JGX0z|7;r~JXlGdNc$XcG)fj1FhT)8g&?|X%`f*$}<=W0$-)F;A5`6>kkj{t9E z$-Q^(y+eGeZVjy5?(AeDEG4e0=!@yUKai?#LEcUx4?&J6U_Tz*g nFMuW}c>n+F^RD2Az$eL_tjb(E&w$rYcj#U=)Iwgo{pkMy05t-~ literal 0 HcmV?d00001 diff --git a/ml/logistic regression/logistic regression.md b/ml/logistic regression/logistic regression.md index 3c3c124..e1dd9f2 100644 --- a/ml/logistic regression/logistic regression.md +++ b/ml/logistic regression/logistic regression.md @@ -4,11 +4,9 @@ ## 目录 + 摘要 + 逻辑回归概述 - - 假设函数 + - 逻辑回归的假设函数 + 假设函数的意义 + 决策边界 - - 线性边界 - - 非线性边界 - 损失函数 - 为什么不使用平方损失函数 - 对数损失函数 @@ -18,3 +16,85 @@ - 对数损失函数是凸函数 + 过拟合 - 正则化的损失函数 + +## 逻辑回归概述 + +在 [线性回归总结](../linear\ regression/linear\ regression.md) 中,简单介绍了线性回归问题,这里将要阐述的逻辑回归,却并不是回归问题,而是一个分类问题(`classification problem`)。分类问题与回归问题的主要界限,在于分类问题的输出是离散值,而回归问题的输出是连续的。根据分类问题输出取值数量的不同,又可以分为**二分类问题**和**多分类问题**,这里主要阐述二分类问题。 + +二分类问题的输出一般为0或者1,表示某一特定的事件不发生或者发生,比如辅助诊断系统判断患者患病或者不患病,邮件分类系统判断邮件是否是垃圾邮件,都是典型的二分类问题。 + +为了训练二分类问题,一种直觉上的方法是直接套用此前的线性回归模型,这样就可以得到模型的假设函数为 + +$$ +h_\theta(x) = \theta^Tx +$$ + +仍然使用此前的平方损失函数以及梯度下降法(或者规范方程法)来对模型进行求解,对于某个简单的数据集,可以得到下面的结果 + +![fig1](images/fig1.png) + +此时$h_\theta(x)$的输出值仍然是连续值,而二分类问题要求模型的输出是0或者1,因此需要对假设函数的输出进行进一步处理。这里可以简单地令$\hat{y} = 1$当且仅当$h_\theta(x) >= 0/5$,令$\hat{y} = 0$当且仅当$h_\theta(x) < 0.5$。可以看到,采用这种策略时,对上图的数据具有很高的预测正确率。 + +然而这种方法却也有局限性,考虑在训练集中增加一个样本点$X = 80, y = 1$,此时线性回归模型的结果如下图所示: + +![fig2](images/fig2.png) + +可以看到,此时回归直线相对此前向右偏移,模型对训练集的预测能力下降了不少。可以想象,如果训练集中还有更多这样的极端数据,回归的直线将继续向右偏移,模型的预测能力将继续下降。 + +此外,线性回归模型的假设函数,其输出应该是表示$Y = 1$的概率,然而其值域是$(-\infty, +\infty)$,当$h_\theta(x) > 1$或者$h_\theta(x) < 0$时不具有意义。可见,简单套用线性回归的方法来解决分类问题是行不通的。 + +### 逻辑回归的假设函数 + +可以做一些简单的修改,使得假设函数的值域限制在$[0, 1]$。这里引入**sigmoid函数**,它的表达式$g(z)$满足 + +$$ +g(z) = \frac{1}{1 + e^{-z}} +$$ + +可以看出,它的取值范围恰好在$(0, 1)$之间,实际上,它的图像如下所示: + +![sigmoid](images/sigmoid.png) + +因此,可以把 sigmoid 函数作用到此前的线性回归模型上,就得到了新的假设函数 + +$$ +h_\theta(x) = sigmoid(\theta^Tx) +$$ + +此时,它的函数值,就可以被理解成在输入为$x$的条件下,$y = 1$的概率,即 + +$$ +P(y = 1| x) = sigmoid(\thetax) +$$ + +> 决策边界 + +以下对 sigmoid 函数进行进一步的讨论。前面已经指出,假设函数的返回值,表示的是预测样本为正(`positive`)的概率。一般地,当$h_\theta(x) >= 0.5$时,预测$\hat{y} = 1$;当$h_\theta(y) < 0.5$时,预测$\hat{y} = 0$。这样,$h_\theta(x) = 0.5$就成为正负样本的边界,称为**决策边界**(`decision boundary`)。 + +由 + +$$ +h_\theta(x) = sigmoid(\theta^Tx) = 0.5 +$$ + +根据 sigmoid 函数的图像,恰好可以得到 + +$$ +\theta^Tx = \theta_0x_0 + \theta_1x_1 + \theta_2x_2 + \cdots + theta_nx_n = 0 +$$ + +容易看出,这是一个(高维)平面的方程,预测为正负的样本,分别分布在该平面的两侧。因此,逻辑回归的假设函数,本质上就是找到这样一个高维平面对样本点进行划分,达到尽可能高的划分正确率。 + +利用**多项式回归**,可以得到更加复杂的决策边界。比如令 + +$$ +h_\theta(x) = sigmoid(\theta_0 + \theta_1x_1 + \theta_2x_2 + \theta_3x_1^2 + \theta_4x_2^2) +$$ + +则得到的决策边界为 + +$$ +\theta_0 + \theta_1x_1 + \theta_2x_2 + \theta_3x_1^2 + \theta_4x_4^2 = 0 +$$ + +容易看出,这是一个圆锥曲线。因此,通过构造更高阶的多项式,可以得到相当复杂的决策边界,从而可以将逻辑回归应用到更加复杂的分类问题当中。