第二章信源及信源熵2-12<2\〔1)=--1(^,5)=-Log—=4.17山丿1f1\⑵Fll=^7^11=-Log—=517任丿GJ)H(k)=S—Log(36)4-15—Lag!—北(4)f12(3H(x)=2杰Lo或购+—Log-6\3丫-+—Logl窮36I；3)364.3375T[36+—Leg—號AJ32742-22-32-42-52-6100；卯100L°=io.Old10J；1006.64438-Log-83601.906=0J75-Log(037^=14151-Log(4)4114.361-Log(4)41-Log(8)81.906(1,2)(2,1)共两种Log364.172(1,6)(6,1)(2,5)(5,2)(3,4)(4,3)0—14个1---13个2----12个P=-匕丿I=-Log—t丿S7.S1136共六种Log362.5853---6个2-7Log(2)1Log(4)2Log(8)32-8”用三个脉冲“•”用一个脉冲2.9(1)1(•二Log(4)241(—)=Log30.41513(2)H=-Log(4)-Log44Cl)丄旳+4-0.81130.?1«P（黑/黑）=—P（八、、/八、、丿P(白/黑)=]0142-10(1)2-11(1)H(XY)二24Log247H(Y/黑)=P(黑/白)=H(Y/白)=P(黑)=H(Y)二乜24⑵P=1241(\4\1Cf⑷—Lag——十—Log-0.38114<10)M而P(白/白)=S*1415514Log[—1+—I514=0.94P(白)=H(色)=2Tr起、is—Log—+—Log3S1?丿33i(38（3P（色数）=t-Log|—H(色数)='11-■■H（数/色）=H（色数）-H（色）=丄Log(24)2424124=1.2455.24S-1.245=4003丄Log(24)-Log(4)一Log(24)24241724Log(24)Log—242472.301124得到旳-丄J+丄丿24424317I=0+—424243H(Y)二(3)H(X/Y)=H(XY)-H(Y)二「「11>'2-12(1)H(X)1H(Y)13>H(Xr)=；Lag(8)+;0c>-k-Lagl--b-Log(ar)=1311H(Y/XJ=HpCy)-IH(Y)=1.511-1=0.511H(Y/X)=H0CY}H(Y)二1£11-1-0^11(3)i<■■:ri!■.■■■:】」2-13PM=43101516143]01d]◎rP(i)=P(ij)=1丿匕丿Iis1361To2-14H(IJ)=(1)iIS1121151101361i34科娜+陀吨町+计B（啣+吃L。&遡+亍嗔卬36_11S71'V-179、一S骼泊(2)方法1:i"i-■■:-.1'.'I=7n“□“——0.40?+—C.236=03111616方法2:+—LoeL67,十亦呢=03112-15P(j/i)=I2丿I2『1—巳巳、]1巴1-E；pc?l)=P(t2)=-p(a1/b1>FWP(bld)_2(1_0)1设最后平稳概率为W1,W2(0.5143DjO初了］P:*IC.20.8JPTW?W得Wi=07W2=0.3W］十WQ=1H(Y/黑)=0.9143Log(0.9143)0.0857Log(0.08500.422H(Y/白)=0.2Log(0.2)0.8Log(0.8)0.722H(Y/X)=W1H(Y/黑)+W2H(Y/白)=1:'2-17(1)2-240.80.914=3^105Log(12g)-2J>IO'*1<1000ri1C'00=!OOOLc0；lOOOq)-L33>xIQ4(1)⑵1585m+41500_：II：bI:；■「:—■■II'■:r-ir■'-I2-25'0.25DM)050.5J解方程组pTW二WW1W20.250.50.750.5W1W1W2解得W1=0.4W2=0.62-26P(j/i)=解方程组<113310223020k3丿=\3/1求得W=2-27求平稳概率010D0110110C<0.800仏0.200N符号条件概率0：0505状态转移概率01000.50-5100.505100.50.50D11<0.22丿111D00,20J&J解方程组5少Qp1w=W0.143得到W=W1+W2十十W4=1ihRJvV0.1432知丿2-28(1)求平稳概率P(j/i)=『2133<1Oj解方程组得到I忆丿H(S/e1)=|lc^H(S/s2)=0信源熵为:H(S)=WdH(S/s1)+W2H(S/s2)=2q51S+-0=01583442-29P(j/i)=i311133130解方程组Q1「332£££332110V3丿(Wl'W2W1凹丿得到W1=一，W2=-,W3=nHpQ/a>=1一鬼5H(X2/b)=Log©=1J8；HpGTc)==1H®(X)=W1H(X2/^+W2H(X2/b)+W3H(X3/c)=-*Lo^十-Lo^十-LoO=1.439§g42-30P(i/j)二
H阿=(1-p)iog®■匕门咱E2H*»(X)=-(1-p)Logfl-p)-pLogH(X)=log(3)=1.584In-⑷—H(p)=切-p)-j十p(2丿!_如In⑵-■0-p>ln(2)~In⑵~ki©P=—厂3当p=-时---达到最大值1.58当p=0时H(X)二0当p"时H(X)二12-33(1)fl-F°卩)P1-F01°F—J解方程组：(1-p)W14p硕=W1(1-P)W2+PW3=W2p-WlH-(1-p)W3=W3W1-kW2+W3=11得p(0)=p(1)=p(2)=-⑵H(X/0)=HpLogHpOO)+^H[X/1|+扌H凶2戶-(1-P)Loe(1-p)-pLc^p)⑶当p=0或p=1时信源熵为0第二章无失真信源编码3-1C1.C2.C3.C6Clr.C3?C6⑶HpQ=IlobQO+iLog；-^+4—Logflfi)-224UK1-3R1-巴理=1K3心—■4+R2=H㈤=2=Q.9412416L61616K22125-2+4—■3^16丄4十16丄”1616R>H㈤-22.125■0941KS=爲+1.3+1-34■1.31+—■3■25R6=H(対_2—■=0.324161616L6K6253-2(1)因为A,B,C,D四个字母，每个字母用两个码，每个码为0.5ms,所以每个字母用10ms当信源等概率分布时，信源熵为H(X)=log(4)=2平均信息传递速率为普二02bit/ms=200bit/s(2)信源熵为111t3(1IJ>H(X)=古曲+汁嗔母+二“対+—Lcg^yJ=1P85r=0.198bit/ms=198bit/s3-3与上题相同3-5丄］1丄丄丄__L248163264128128H(U)=1.984Log(2)1Log(4)1Log(8)丄Log(16)丄Log(32)丄Log(64)—Log(128)—Log(128)48163264128128(2)每个信源使用3个二进制符号，出现0的次数为冷十時十2丄十丄十216出现1的次数为1-十1丄十2-—4-14区16T23P(0)=2.3PE…—n冥仃』—U1jOLIH-2.39S+05E6P(1)=丄*口12^■+0526(4)相应的香农编码信源符号Xi符号概率pi累加概率Pi-Logp(xi)码长Ki码字x11/20110x21/40.52210x31/80.7533110x41/160.875441110x51/320.9385511110x61/640.96966111110x71/1280.984771111110x81/1280.9927711111110相应的费诺码信源符号xi符号概率Pi第一次分组第二次分组第三次分组第四次分组第五次分组第六次分组第七次分组二元码x11/200x21/41010x31/810110x41/16101110x51/321011110x61/6410111110x71/128101111110x81/128111111110(5)香农码和费诺码相同平均码长为K——1-+—-2-I—-34-—4-I5+—-S+7-+7=1.58424S16326412S1221L-1(1)Pi=21累加概率为Pi=YEi»0编码效率为:_1.咖1.9343-7累加概率分别为符号x1x1x2x3x4x5x6x7・・・概1/1/1/81/161/321/641/1281/256・・・率24累加概00.50.750.8750.9380.9690.9840.992・・・率码长12345678元010110111011110111110111111011111110・・・码(2)信源的信息量为TOC\o"1-5"\h\zH(X)=丄Lug(石I+丄Lo+-Lag(8J+丄Log(16)+■+—424EId'平均码长为：™*1+"2+—■r3+*4+|„||„4|+一a十24g162i码字的平均信息传输率为R=〒-{bii/码(3)编码效率R^—100%3-10(1)H(X)=-0.37kjg(0.37)-0.25Log(0.25)4-0.18Los(0.1S)-O.lLog(O.l)-0.07Log(0.07)-U.03U]g(0.03)二L338(2)信源符号Xi符号概率pi编码过程编码码长X10.370.->0.X0.0U100237373862x20.25*0.0.、0.0U0.01225253738x30.18卒0.0U0.112K=037-2+0252*018-2++0074+0.03,4=233-11(1)信源熵HpC)=-OULogCPJ^-022Log(0.2Z)-0.18Log(0.18f)-016LogQ31^-00&Log(O.D8r)-0.04Loy(0.04)■2352(2)香农编码：信源符号Xi符号概率pi累加概率Pi-Logp(xi)码长Ki码字X10.3201.644200x20.220.322.1843010x30.180.542.4743100x40.160.722.6443101x50.080.883.64441110x60.040.964.644511110平均码长：K=032-2+0223+0183+0163+0J0844-D.01>5-2.84编码效率为DH(X52.352R.■―——0^828K234(3)费诺编码为信源符号xi符号概率Pi1234编码码长x10.3200002x20.221012x30.1810102x40.16101103x50.081011104x60.04111114平均码长为：K=0322十0^2-2+■0.18-2+■016-3+0084+■0J044■2.4编码效率：K3B4(4)哈夫曼编码信源符号Xi符号概率pi编码过程编码码长X10.3240.f0.X。.0101232/38/4060x20.22*0.\0.40.QU0.10222323840x30.18曽.01]0.112182232x40.16*0.0.00031618x50.08t*0.0014120x60.0400141平均码长为：K=0322十0223十0.13-2+016-3+0.084十OJ044-2.4编码效率：吐驴企卜血无3-12(1)信源熵H旳=-OlLugjOl)-D.L3L..^0.13)-D.4Lo^0.4j-DWLogfDJO》-匚KLu^OD^-11"<0门-OMLa如专-。幕5创吟-2551信息传输速率2.552bit/s⑵信源符号xi符号概率Pi编码过程编码码长x10.4**0.40.**0.0.*0.4、0.曲11444x20.18•*0.10.丫0.%.•丄|0丫0.2刁0.40013x30.1x40.1x50.07x60.06x70.05x80.040.090.07o0.0610.0919180.130.23190.2oI]0.180113000040100401014000150000151K=0.44+D.1S-3+0.13+0.1-4+0.07'4+OJ06-4+0.05^+0.04'5=2j61.2血TaT0.978(3)香农编码信源符号Xi符号概率pi累加概率Pi-Logp(xi)码长Ki码字X10.401.322200x20.180.42.4743011x30.10.583.32241001x40.10.683.32241010x50.070.783.83741100x60.060.854.059511011x70.050.914.322511101x80.040.964.644511110平均码长:K：=LI.42^O.IS-3+O.l4+111-4+0.D74+0.06'5+0JO5-5+0.04-5=3.17口H〔勺2552Rs亠工=0.80JK3,17⑷费诺编码：信源符号xi符号概率pi码码长x10.400002x20.181012x30.11001003x40.111013x50.0710011004x60.06111014x70.051011104x80.04111114K=04-2+013J+013-F0.13+007-4+006^4+0.0^4+00J-4=2.64"2~264=0.9673-14XIx1x2x3x4x5x6x7pi1/31/31/91/91/2701/27J1/27编码过程1/1/1/1/1/1/1/f、1/1/1/271/27H2/1/1/o1]1/1/H凶二2-Log(3r)+3—Log(27)=2^3927K=^2+-3+^3+—4+—4+—4=2M339927272722^9nmr7=CI.9372.44431/编码码长0020121003101311131100411014第四章限失真信源编码4-101114-21011d二11011110信源熵为H(x)Log(4)23333Dmax=min{;，4，;,;}R(Dmax)=0Dmin=0R(Dmin)二R(0)=H(X)=log(4)=2p(y1)p(y2)p(y3)p(y4)只要满足p(y1)+p(y2)+p(y3)+p(y4)=1在[0,1]区间可以任意取值第五章信道编码5-1①p(>D)=3Mp(x1)=1/451、p(於沪；；p佃y沪U3>H(XY)=iLo^2)4--Lo^+—LogflZ)+-"取①=1.7341""2KHptfY)二HE>HCQ=1.73-0.98-0.75H(Y/>q=H(X¥)-Hpg=173-0811=0515lg：V)二Hp^H何尸0.811-0.75-OJ061OlogfQJ+SpjjLogtp沪LO0(2)4|“呂.0825-2接收端的不确定度为H(Y)=-a+-|-Log<1-a+114丿(1-tx)Lcig-(1-x)44.宁站“)■宁说-)ln(2)3CL：=—5C=mailKY>百卜=0161H(Y/X)二-cc十—十Oi-I--(1-qc)+—[1-«)十丄(1-x)Log(関222口431二三三三22⑶l(X;¥)-H(V)-H(Y/X)^f--Log(l+胡-]—^L。贰1—-1JU44丿匕2丿P(榻k)J^-l+QGf10g(l+Qu)1-cz1心贰1-m))n1、===G严\24loeC&41口曲丿U2丿=0得到1-Lti(l十侃)十ln(l-cc)+2ln(Z)m.:«II45-32C=LogfQ)+V(Pyj-LflgtPjj)=i+OJ9Lo0；D^+OOlLoaJJjaC=0919j=l0919*1000=919bit/s5-5(1)Lo卸功十ViF^Log^pyI=5戌令十5-6(1)条件概率ft.3空、联合概率f111'61015211TsTolo10—<3010丿后验概率1P(yo)11PM)2,p(y2)6H(Y)=十|-U)gj(2)-f-Lo^=1.4JP326(2)H(Y/X)二io1.17513一+——LogflOQ+—Log—3C10A?当接收为y2，发为x1时正确，如果发的是x1和x3为错误,各自的概率为：TOC\o"1-5"\h\z113P(x1/y2)=5,P(x2/y2)=5,P(x3/y2)=?其中错误概率为：HYPERLINK\l"bookmark91"\o"CurrentDocument"13HYPERLINK\l"bookmark319"\o"CurrentDocument"Pe=P(x1/y2)+P(x3/y2)=55°8(4)平均错误概率为711311—+—+一+—+—+—=373315?0101011510仍为0.733此信道不好原因是信源等概率分布，从转移信道来看正确发送的概率x1-y1的概率0.5有一半失真x2-y2的概率0.3有失真严重x3-y3的概率0完全失真(7)H(X)=Lost?=13&5H(X/Y)=-Log(2)丄Log(5)—Log-—Log-丄Log(5)丄Log-丄Log(10)3Log-1.3016101521521010330103课外习题设某信道，其信道矩阵为alr0.50.3a20.20.30.5£<0,30,30.4；blb2b3若信道的输入符号a1,a2,a3先验等概,(1)若使平均错误译码概率最小，请选择译码函数。(2)求出此错误译码概率Pemin解：(1)因为先验等概，所以选择最大似然译码准则F(b1)=a1F(b2)=a3F(b3)=a2Pemin二-(OJ十0.3十0.3+Q3十C.2十g=有二进制对称信道p=0.01p=0.99(1)采用最大似然译码准则确定译码函数(2)求出最小平均错误译码概率对该信道进行扩展，采用简单重复编码，000,111,采用最大似然译码准则确定译码规则求出扩展后的最小平均错误译码概率。求出扩展后的信道传输率解：(1)P(j/i)=匀卩却阿］译码函数为F(b1)=a1，F(b2)=a2(2)Pemin=(0.01+0.01)/2=0.01⑶olioU01Olli阿=000r_5-2pFPpp7==1113Ip-2PP-21U0011110101111译码函数F(p1)=F(F(由)二F(血)=F(⑹二F([34)=000=a1%)=F(3)=F(3)=000=aa平均错误最小概率为Log(蛆)Log；2)_N=(5)R=3.ai，场是两个码符号{0,1}组成的符号序列叫"叫仙011,卩严⑹阳皿1求ai之间的汉明距离N—解：D(ai，闭二工也勺计7k=1W:{000,001,010,100,011,110,101,111}的最小汉明距离解：Dmin=1设有一离散信道，其信道矩阵为坷巧Z42/(1)当信源X的概率分布为p(ai)=2/3,p(a2)=p(a3)=1/6时,按最大后验概率准则选择译码函数，并计算其平均错误译码概率Pemin(2)当信源是等概率是分布时，选择最大似然译码准则选择译码函数，并计算其平均错误译码概率Pemin。解：(1)联合概率：后验概率Sb2b；1077io根据最大后验概率准则F(bi)=ai,F(b2)=ai,F(b3)=a1最小错误译码概率为Pemin=1111—■!■!+1 24241224丄+丄.24120333(2)当信源是等概率分布时采用最大似然译码准则F(bi)=ai，F(b2)=a2，F(b3)=a3设离散无记忆信道的输入符号集X:{0,1}，输出符号集Y:{0,1,2},信道矩阵为012P,罚142J若某信源输出两个等该消息X1,X2,现在用信道输入符号集对X1,X2进行编码，Wl=00,W2=11代表X1,X2。按最大似然准则写出译码函数，并求出最小平均错误译码概率Pemin。解:^2対(1)选择译码函数F(bi)=F(b2)=F(b3)=F(b4)=F(b7)=xiF(b5)=F(b6)=F(b8)=F(b9)=X2晒吟丄J+丄J+丄亠丄丄+丄十丄=询16516S161616161(5设分组码(n,k)中，n=6,n=3，并按下列方程选取字中的码字。°2-勺Gg=哲lC4=al+a2吁■aI+乜陌二乜十电求信息序列(aia2a3)变换成六位的八个码字，并求出编码效率。解：信息序列码字(aia2a3)(C1C2C3C4C5C6)000000000001001011010010101100100110011011110101101101110110011111111000编码效率nfi1■二一51
浙公网安备 33021202002483