1. exercise in the previous class (1)
Consider an “odd” parity check code C whose codewords are
(x1, …, xk, p) with p = x1+…+xk+1. Is C a linear code?
No.
x1=1, x2=x3=...=xk=0 ⇒ p = 0, and is a codeword
x2=1, x1=x3=...=xk=0 ⇒ p = 0, and is a codeword
the sum of the two codewords = , not a codeword

2. exercise in the previous class (2)
Construct a 2D code for 6-bit information (a1, ..., a6) as follows.
determine the generator and parity check matrices
encode using the generator matrix
correct an error in the sequence a1 a2 a3 a4 a5 a6 p1 p2 q1 q2 q3 r
(a1, ..., a6)
→ (a1, ..., a6, p1, p2, q1, q2, q3, r)
p1 = x1 + x2 + x3
p2 = x4 + x5 + x6
q1 = x x4
q2 = x x5
q3 = x x6
r = x1 + x2 + x3 + x4 + x5 + x6
parity symbols:

3. exercise in the previous class (3)
p1 = x1 + x2 + x3
p2 = x4 + x5 + x6
q1 = x x4
q2 = x x5
q3 = x x6
r = x1 + x2 + x3 + x4 + x5 + x6
111000
000111
100100
010010
001001
111111
coefficients
(係数) G: H:
as is
transpose
H( )T
= ( )T = the 4-th column 
( )G
= ( )

4. in the previous class... linear codes: definition, encoding, decoding
one-bit error at the i-th symbol position
⇔　syndrome equals the i-th vector of H
if several column vectors in H are the same,
then we cannot correct one-bit errors in a codeword.
if all column vectors in H are different,
then we can correct all one-bit errors in a codeword.

5. design of error correcting codes
Construct a parity check matrix with all column vectors differ, then we have a one-bit error correcting code.
OK examples:
101100
110010
011001 H= H=
101001
010011
110100
010101 H=
NG examples:
110100
101010
010001 H= H=
C = {v | HvT = 0 mod 2},
the discussion is easier
if the right-submatrix of H is an identity matrix...

6. construction of a code coefficients H= G= 101 100 110 010 011 001 101
as is
transpose
p1 = x x3
p2 = x1 + x2
p3 = x2 + x3
000000,
001101,
010011,
011110,
100110,
101011,
110101,
codewords

7. the “shape” of a check matrix
a parity check matrix with m rows and n columns  a code with...
length
# of information symbols
# of parity symbols
= n
= n – m (= k)
= m
n = 9
m = 5
code length 9
= 4 information symbols
+ 5 parity symbols
“vertically longer” H means more parity symbols in a codeword
 less number of information symbols
 not efficient, not favorable... NG
good

8. Hamming code
To design a one-bit error correcting code with small redundancy,
construct a horizontally longest check matrix (all columns differ).
Hamming code
determine m, # of parity check symbols
list up all nonzero vectors with length m
use the vectors as columns of H
(any order is OK, but let the right-submatrix be an identity)
Richard Hamming
m = 3:
length 7
= 4 information
+ 3 parity

9. Parameters of Hamming code
determine m, # of parity check symbols
design H to have 2m – 1 different column vectors
H has m rows and 2m – 1 columns
length
# of information symbols
# of parity symbols
n = 2m – 1
k = 2m – 1 – m m m 2 3 4 5 6 7 n 15 31 63
127 k 1 11 26 57
120
(n, k) code:
code with length n, and
k information symbols

10. “shorter is the better”
comparison of codes
two codes which can correct one-bit errors:
(7, 4) Hamming code
(9, 4) 2D code
which is the “better”?
Hamming code is more efficient (small redundancy)
Hamming code is more reliable
correct data transmission with BSC with error prob. p:
Hamming code: (1－p)7 + 7p(1－p)6
2D code：(1－p)9 + 9p(1－p)8
“shorter is the better”
= 0.85
= 0.77
if p=0.1

11. codes better than Hamming code?
3 parity bits are added to correct possible one-bit errors
Is there a one-bit error correcting (6, 4) code, with only 2 parities?
No. Assume that such a code exists, then...
there are 24 = 16 codewords
# of vectors decoded to a given codeword = 1+6=7
# of vectors decoded to any one of codewords = 7×16 = 112
# of vectors with length 6 = 26 = 64, which is < 112
contradiction! (矛盾)
{0, 1}6

12. Hamming code is perfect
there are 24 = 16 codewords
# of vectors decoded to a given codeword = 1+7=8
# of vectors decoded to any one of codewords = 8×16 = 128
# of vectors with length 7 = 27 = 128
all of 128 vectors are exactly
partitioned to 16 classes with 8 vectors
{0, 1}7
Hamming code is a perfect one-bit error correcting code:
2 𝑘 𝑛 𝑛 1 = 2 𝑛

13. advanced topic: multi-bit errors?
Hamming code is a perfect one-bit error correcting code.
Are there codes which correct two or more errors?
Yes, there are many...
one-bit error: syndrome = one column vector of H
two-bits error: syndrome = sum of two column vectors of H
different combinations of t columns in H results in different sums,
 the code corrects t-bits errors.

14. advanced topic: two-bits error correcting code
ℎ 𝑖 + ℎ 𝑗 ≠ ℎ 𝑖′ + ℎ 𝑗′ if 𝑖,𝑗 ≠{ 𝑖 ′ , 𝑗 ′ }
 different two-bits errors results in different syndromes
received ⇒ the syndrome is T= h2 + h6
⇒ errors at the 2nd and 6th bits
⇒ should be transmitted

15. ability of the code
Error-correcting capability of a code is determined by
the relation of column vectors of a parity check matrix.
It is not easy to consider all the combinations of columns.
More handy and easy means is needed.
For linear codes, we can use
minimum distance, or minimum weight

16. similarity of vectors
a codeword u is sent, errors occur, and v is received:
In a practical channel, the distance between u and v is small.
u = 000
v = 000, with probability 0.729
BSC with error prob. 0.1
v = 001, with probability 0.081
v = 011, with probability 0.009
v = 111, with probability 0.001
If there is another codeword u’ near u,
then v = u’ occurs with notable probability.
safe
not safe

17. Hamming distance dH(0100,1101) = 2
a=(a1, a2,..., an), b=(b1, b2,..., bn): binary vectors
the Hamming distance between a and b, dH(a, b)
= the number of symbols which are differ between a and b
dH(0100,1101) = 2
dH(000, 011) = 2
dH(000, 111) = 3
dH(011, 011) = 0
dH(a, b) = dH(a + b, 0)
If a vector u with length n is sent over BSC with error prob. p,
then a vector v with dH(u, v) = i is received with prob. (1 – p)n–ipi.
inverse correlation between the distance and the probability
(逆相関)

18. Hamming distance between codewords
code with length 4 ...
vectors = vertices of a 4-dimensional hyper-cube
codewords = subset of vertices
C1={0000, 1100, 0011, 1111}
C2={0000, 0001, 1110, 1111}
two or more edges
between codewords
some codewords are
side-by-side
good
bad

19. minimum distance the minimum Hamming distance of a code C:
dmin = 2
dmin = 1

20. computation of the minimum distance
consider a linear code whose generator matrix is
000000
001011
010101
011110
100110
101101
110011
111000 3 4
dmin = 3 for this code
Do we need to consider all of 2k×2k combinations?

21. minimum Hamming weight
the Hamming weight of a vector u: wH(u) =dH(u, 0)...# of 1s
the minimum Hamming weight of a code C:
wmin=min{wH(u) : uC, u  0}
Lemma: if C is linear, then wmin = dmin.
proof of wmin≤dmin:
let u and v be codewords with dmin = dH(u, v).
u + v C, and wmin ≤ wH(u + v) = dH(u + v, 0) = dH(u, v) = dmin.
proof of dmin≤wmin:
let u be the codeword with wmin = wH(u).
dmin ≤ dH(u, 0) = wH(u) = wmin. u v
u + v

22. examples of minimum Hamming weight
(9, 4) 2D code：the minimum Hamming weight is 4 4 6
(7, 4) Hamming code: the minimum Hamming weight is 3 3 4 7

23. general case of Hamming code
lemma: The minimum Hamming weight of a Hamming code is 3.
proof sketch:
Let H = (h1, ..., hn) be a parity check matrix:
{h1, ..., hn} = the set of all nonzero vectors
if codeword u with weight 1, then HuT = hi = 0...contradiction
if codeword u with weight 2, then HuT = hi + hj= 0
...this means that hi = hj, contradiction
no codewords with weight 1 or 2

24. proof (cnt’d)
lemma: The minimum Hamming weight of a Hamming code is 3.
proof sketch:
Let H = (h1, ..., hn) be a parity check matrix:
{h1, ..., hn} = the set of all nonzero vectors
Choose x, y as you like, and choose z so that hx + hy = hz.
Let u be a vector having 1 at the x-th, y-th and z-th positions,
then HuT= hx + hy + hz = 0, meaning that uC.
codewords with weight 3 are constructible

25. minimum distance and error correction
What does dmin=3 mean?
Any two codewords are differ at three or more symbols.
At least three-bits errors are needed to change
a codeword to a different codeword. u’ v’ u v u v
error
error
{u’ | dH(u, u’)=1, uC}  {v’ | dH(v, v’)=1, vC} =
We can distinguish
a result of one-bit error from a codeword u, and
a result of one-bit error from other codeword v.

26. decoding territory dmin=3：define territories around codewords
radius = 1... territories do not overlap
radius = 2... territories do overlap
rule of the error correction:
if a received vector r falls in the territory of
a codeword u, then r is decoded to u.
if dmin=3, then the maximum radius of the territory is at most 1.
 the code can correct up to one-bit errors

27. general discussion
define 𝑡 𝑚𝑎𝑥 =⌊ 𝑑 𝑚𝑖𝑛 −1 2 ⌋ (⌊𝑥⌋ is the largest integer ≤ x)
dmin=7, tmax=3
dmin=8, tmax=3
tmax
tmax
tmax
tmax
territories do not overlap if the radius ≤ tmax
⇒ C can correct up to tmax bits errors in a codeword.

28. examples
dmin
tmax 3 1 4 1 5 2 6 2 7 3 8 3

29. about tmax tmax is the maximum radius that is allowed
we can consider smaller territories with radius < tmax
tmax t
vectors which do not belong to any territory
detect errors, but do not correct them

30. advantage and disadvantage
sent codeword t
received
decoded to the
correct codeword
error detection
only
decoded to a
wrong codeword
radius
correct
detect
wrong
large
large
small
large
small
small
large
small
The radius should be controlled according to applications.

31. familiar image? A: award of 10,000 Yen
B, C, D: penalty of 1,000,000 Yen A A B B C C D D
P(A) …large
P(B), P(C), P(D) …large
P(miss) …small
P(A) …small
P(B), P(C), P(D) …small
P(miss) …large

32. summary of today’s class
Hamming code
one-bit error correcting
perfect code
the minimum distance and minimum weight
handy measure of the error correcting capability
large minimum distance means more power

33. exercise Define (one of) (15, 11) Hamming code:
construct a parity check matrix, and
determine the corresponding generator matrix
Let C be a linear code with the following parity check matrix. Show that the minimum distance of C is 4.