Download presentation
Presentation is loading. Please wait.
1
Communication, data transfer
1. Selected parts of information theory coding, transfer rates, enthropy, compression, encryption under continuous construction...
2
Units of information b, bit („binary digit”): basic unit of information in binary number system, with value of 0 or 1; can be used equivalently in digital technics as True (1) or False (0 or -1 or any other number if not in binary representation) B, byte: 1B=8b also called „octet” (eg. in French), partly because byte is not trivially 8b (historically other formats existed) Modern computers, processors and controllers are typically based on n*8b representation of data (eg. memory cell size, ALU size or data bus size etc.)
3
Binary prefixes Binary storage devices typically have sizes that are powers of two (question: why?) Thus was the K=1024 prefix developed long ago They agreed that k=1000 and K=1024, but it was not universal of course Larger prefixes (M,G,T) are already capital letters so problems arose Especially with hard disks (1TB=?B)
4
Binary prefixes 1999: IEC 60027-2 Amendment 2
to make prefixes unambigous traditional SI prefixes are decimal new binary prefixes: Ki,Mi,Gi,Ti etc. KiB: kibibyte, MiB: mebibyet etc. („kilo binary byte”)
5
ASCII American Standard Code for Information Interchange
Original version: 1963, actual: 1986 7 bit symbols, 128 characters: lowercase and uppercase letters,numbers, special characters, control characters control chars: eg. new line, carriage return, tabulator, end-of-file etc. – some were made for printers Numbers’ format: to (notice the code contains the number) A: , a: (A+32) Originally the 8.bit was used as parity Later it was extended. Using the 8.bit another 128 characters were added, but these are not standard. More than 200 „code pages” exist.
6
Units of information transfer
Bit rate: number of bits transferred in unit time [b/s] informal unit notations: bps (bit per second) Bps (byte per second) CASE SENSITIVE! 1Bps=8bps
7
Units of information transfer
Gross bit rate (raw bit rate, data signaling rate): Counts all ("physical") bits Net bit rate (information rate): counts only bits containing the message (information), discounts bits for header, error correction, etc. net bit rate <= gross bit rate eg. 4B5B coding (4bit over 5bit) (Ethernet): 100Mb/s net, 125Mb/s gross bit rate
8
Units of information transfer
Symbol rate: symbols transferred per unit time unit: baud (Bd) = symbol per second only in binary coding!: 1baud=1b/s non binary: one symbol can be decoded to binary with more than one bits used eg. in modulations such as QAM,QPSK coding efficiency: bit/s per baud
9
Units of information transfer
10
Channel capacity (C) the upper limit of the gross bit rate of a noisy channel, while keeping an arbitrarily small bit error rate (without using error correction codes) in principle, a noiseless channel could have infinite bit rate, but as noise increases, the useful bit rate decreases see: Shannon-Hartley (later)
11
Bandwidth More meanings in vernacular:
actual transfer rate of information (bitrate R) channel capacity C (maximum of R) spectral bandwidth (B) (we'll use this) the bandwidth of the frequency spectrum of the signal transmitted can be -3dB bandwidth, but also can be measured btw zero points.. These are related, but not the same! Eg. for same bit rate, B will be dependent on coding/modulation
12
Bandwidth Nyquist-Hartley-Shannon
13
Bandwidth Eg. binary baseband signal
N=1; max 2 (b/s) / Hz7 For double side band (DSB) modulations, only half of bandwidth counts eg. 16QAM: N=4; max 4 (b/s) / Hz
14
Examples for baseband coding spectra
(more about it in next chapter
15
Noise, interference, error
often: additive white gaussian noise (AWGN) interference EMI (electromagnetic int.) crosstalk (between wires/signals) ISI (inter-symbol int.) distortion SNR: Signal to Noise Ratio SINR: Signal to Noise and Interference Ratio
16
Distortion "linear distortion" "nonlinear distortion":
doesn't introduce new frequency (spectral) components modifies existing spectral components (of course the relation of input and output signal is still nonlinear, but it can be achieved by linear circuits, eg. filters) "nonlinear distortion": introduces new spectral components made by circuits with nonlinear characteristics eg. diode/transistor, limiting circuit
17
Bit Error Ratio (BER) Bit Error Probability (theoretical probability of a bit error) BER is converging to BEP (see prob.calculus) BER used in practice is max 50% for a totally random channel question: why?
18
Bit Error Ratio (BER) BER is made worse by: Modify by (possibly):
noise, interference, distortion Modify by (possibly): amplitude, bit time, baseband coding, modulation error correcting codes
19
Eb/N0: energy per bit to noise power spectral density ratio
additív gaussi fehérzajjal definiálva Eb/N0: energy per bit to noise power spectral density ratio Ratio of the power in one bit/s, divided by the noise power in 1Hz bandwidth for different modulations
20
Number of symbols Number of symbols available (in "alphabet"): M (which is often a power of two, ie. M=2^N) remember: one symbol is transmitted in a "unit time" If M (N) is increased: spectral efficiency (R/B) is increased, ie. a larger bit rate can be achieved on the same spectral bandwidth the distance between levels (be it amplitude or frequency or phase) will be smaller, thus it will be more sensitive to noise see the BER graph: if M is smaller, a lower Eb/N0 is needed to achieve the same BER
21
Eye diagram (In this case):
a binary signal is seen on the oscilloscope, with the different bit values superimposed if the noise is small, the "eye" is open, the levels can be easily distinguished
22
Constellation diagram
Complex plane representation of a QAM (amplitude and phase modulation) Each spot is a level or symbol– ie. we have 4 symbols (M=4) If there is no noise, the spots are point-like. If the noise is too large, the spots will merge/overlap
23
Shannon-Hartley theorem
Relationship between channel capacity,bandwidth and signal-to-noise ratio
24
Shannon-Hartley theorem
If no noise: S/N is infinite => capacity is infinite even if bandwidth is finite remember: if B constant, bit rate can be increased by using larger M (alphabet) If noise increased: capacity decreased as in earlier, we mean that the bit error rate is to be kept arbitrarily low
25
Shannon-Hartley theorem
In noisy channel: Combined with Hartley's theorem, we can get an estimation on the max number of levels (symbol) usable (ideally):
26
Entropy Entropy according to Shannon:
X: stochastic variable with possible outcomes {x1...xn}, with probability distribution P(X) with values P(xi). Here the xi are the codewords (symbols) with P(xi) probability of occuring in the message. Entropy is practically about the information content of a message. If all symbols have the same probability, then entropy is maximal.
27
Entropy Entropy tells the randomness, the unexpectedness of a message.
Eg. N=1, (M=2 symbols), with equal probability (eg. generated by coin flipping).
28
Entropy Eg. N=2, (M=4 symbols), equal probability.
Entropy is 2 shannon (bit). This is correlating with the fact that in an alphabet of 4 symbols, each symbol can be converted to a 2bit binary representation. In a binary code with equal prob., the entropy is 1 (see prev.page). With equal probabilities: H=N
29
Entropy Eg. N=2, (M=4 symbols), not equal probability (one symbol more probable than the others). The entropy is 1,36 shannon (bit), which is smaller than the previous case. This can interpreted as saying, that if symbol x1 arrives, it is the most "expected", thus conveys less information than the others. For example if one symbol has probability=1 (certain outcome) then thus the others have to be 0, then entropy is 0, because we are certain that we'll always receive the one symbol with P=1, thus no news is gained.
30
Question? Randomness as a measure of information can be problematic in practice Eg. how random is the following series? How much can it be compressed?
31
Lossless compression If entropy is smaller than N, there is a possibility of lossless compression. The new code created can not have an average code word length longer than the entropy of the message. (Shannon's source coding theorem, 1948) There exists no such compression algorithm, that would work on all messages (ie. make them smaller) a simple proof: try compressing an already compressed message There are different methods used for different message types, eg. text, image, executable files, video, music etc.
32
Lossless compression using binary representation of codes:
variable length code: we associate target symbols (codewords) of different length (number of bits) to each source symbol prefix-free: no codeword starts with another codeword eg. {0;01;10} set is not prefix-free, because 01 starts with 0 but {0;10;110;111} is prefix-free
33
Compression - Huffman coding
prefix-free, variable length code it has the minimal average code word length given: the source symbols with their probability (or relative frequency of occurance) (this is also called weight here)
34
Compression - Huffman coding
Write down the source symbols and their weigths, in order of weigths Pair the two smallest weigths and connect them to a new node in a tree structure Repeat until there is only one node in the top row (the root) Starting from the root, new codewords are assigned to the nodes (see next page)
35
Compression - Huffman coding
36
Question For example we have a binary code with: P(0)=0,9 P(1)=0,1
Entropy is 0,47 so we feel we could achieve good compression Huffman coding, as shown previously, can not be directly applied What to do?
37
Compression LZ algorithm
38
Sound/voice coding base: lossless: lossy: PCM, ADPCM FLAC, MLP
MPEG Layer III (MP3) GSM
39
Voice coding FLAC: Free Lossless Audio Codec Linear predictor based
The next sample is approximated as a linear function of previous samples Transmits the coefficients of the linear functions and the error (difference of approx. and real sample) Compression ratio of 40-50% from: dr. Tóth Zoltán: Médiakommunikáció: Digitális rádiós információ és műsorszórás diasor
40
Voice coding Lossy compression:
utilizes the knowledge about human hearing (psychoacoustics) eg. a louder sound can "mask" another one eg. two slightly different frequencies can not always be perceived to be separate also the stereo channels are often very similar, allowing to encode the difference only for one channel
41
Redundancy Eg. human languages are redundant
Hwo stnarge we cna raed tihs. Evn tis txt cn b undrstod, if hrdly. Redundancy also means possibility of compression. If we keep redundancy, it gives: some protection against noise, interference / the ability to correct errors
42
Error detection and correction
Error detecting codes message has to be sent again need a two-way channel! Error correcting codes (ECC) message doesn't have to be sent again Forward Error Correction (FEC): called forward, because no talk-back (re-send) is needed
43
Error detection and correction
Generally, methods for detecting or correcting errors will increase the total amount of data transmitted for the same amount of actual information. That is, the gross bit rate is greater than the net bit rate.
44
Error detection: parity
Simplest error detection method: parity bit (for binary codes) Plus one bit at the end of the message (or frame, usually a byte) will make the total number of ones either odd or even. We can detect the error (in binary: inversion) of an odd number of bits. Can not correct, the message/frame/byte has to be re-sent. Eg. ASCII was originally 7b code and 8.bit was the parity. Other apps: SCSI,PCI,cache, UART, etc.
45
Error correction: repetition code
Send each bit (or message/frame) n times. If not all received are the same, make a majority decision (it helps if n is odd). Net bit rate is 1/n times the gross bit rate. Eg. 5b version: FlexRay automobile bus system. Also used in space probes. Similar method for hardware: Triple Modular Redundancy, TMR storages (eg. some ECC RAM) – total storage capacity changes on board computer and other hw multiplied (eg. spacecraft) homework: create a majority logic gate (it creates the majority decision)
46
Error correction: Hamming code
For every n bit of message, create an m+n bit long code word eg. (7,4) code: for every 4 bits, add +3 bits detects 1 or 2 bits of error or corrects 1 bit of error used eg. in ECC RAM
47
Error correction – burst error
Burst error: if a larger group of neighbouring bits are involved esp. in radio or optical comms. Most methods can only correct a few bits of error in a larger frame. =>interleaving: the message is broken into smaller parts, which are re-ordered before transmission. After reception, order is restored, thus groups errors are broken into more but smaller error groups.
48
Encryption Coding can be thought of as a one-to-one relation btw two sets of symbols If someone doesn’t know the second set of symbols or the relationship, then it is also encryption.
49
Encryption terms Encryption: the act of making a message secret
Cryptography: the science of encryption Plaintext: the source message, not encrypted Cleartext: info stored/sent without encryption Ciphertext: encrypted message
50
Encryption Possible methods Encrypting the contents of the message
this will presented first Hiding the message or the act of communication
51
Encryption In practice, we have to keep in mind that most methods can be decyphered if given enough time. We have to choose a method such that the decyphering time is longer than the actuality of the message, or the cost of decyphering is larger than it is worth. For electronic communication, usage of special characters, symbols is not applicable, so we use methods that relate eg. latin characters to latin characters, or more correctly, binary numbers to binary numbers.
52
Encryption Suppose encrypting human text.
Simple relation of latin-to-latin characters: shift all characters by same number: A->C; B->D; Z->B, etc almost trivial to decypher relate randomly just a little bit harder
53
Encryption Decyphering simple codes:
Find out the language (eg. from knowing the sender and receiver, or trying – it’s a finite set) Need a table of character relative frequency of that language Need a dictionary of that language Often start with 1..2 letter words.
54
Encryption Make the previous methods a little bit more complicated:
code the „space” character also into something else use multi-language text, make some spelling errors
55
Encryption If we use the same method for a longer time (for many messages), or use longer messages, the chance of decyphering will be close to 1. Therefore we need a method where the encryption process is split into two: a fix algorithm and a changing part (the cryptographic key)
56
Encryption Security through obscurity: when we trust the algorithm to be unknown – don’t! With keyed techniques, the knowledge of the algorithm without the key is not enough to decypher.
57
Encryption To defeat frequency analysis:
Instead of 1-to-1 relation, each letter (symbol) in the source will be related to another based on its position in the text. Thus the same letter can be coded to different letters each time. This relation is accomplished using the key, which can be a series of letters or numbers.
58
Encryption Example: Write the message in a row, under it the key, then use some algorithm on letters (or numbers) in one column to form a third one, write these in the third row to create the cyphertext. If the key is shorter than the message, copy it enough times after itself. This of course weakens the method.
59
Encryption Example: Source: „ez a szoveg titkositando”
Key: „a kulcsmondat” Method: sum of letters modulo 27 (space=1, a=2, b=3, etc) Result: „g lwmwsbktlbnkuwjemmocst”
60
Encryption One time pad (OTP): key used only onces
G.S.Vernam : XOR method one time pad, 1919. check: XOR can be used both ways (to encrypt and decrypt) OTP can be theoretically unbreakable if: key is really only used once (for one message) key is at least as long as the message key is random Please note that purely with software you can’t create true random series you need some hw source (eg. thermal noise, tunnel effect, radioactive decay)
61
Part of a key book from Germany WWII, using Enigma
62
Encryption Decyphering OTP code book is acquired keys are re-used
key is not random see: Venona project, Lorenz, Enigma
63
Public key cryptography
A big problem with OTP is the sending of the pair of the code book to the recipient. Especially hard if you want to send message to someone you didn’t meet before. To solve this, Public-Key Cryptography was invented.
64
Public key cryptography
Everyone has two keys: a public and a private. Aims: to identify the sender (digital signature) to check if message was tampered with to encrypt message these two can be done alone or together solve problem of sending keys public key can be sent on unencrypted channel
65
Public key cryptography
Method of sending message: Use receiver’s public key to encrypt message. Receiver can decrypt it using her private key. This is called asymmetric encryption (different key need to encrypt and decrypt) hybrid cryptosystems: in practice often a traditional symmetric key is encrypted using the above method and rest of message is traditional (because asymm. encryption needs more calc.resources) Keep your private key secret. Public key is found in „phone books”.
66
Public key cryptography
To check identity of sender or whether message was altered by third party: sender creates a „hash” (a short block made from the message using special algorithm), which is appended to the end of the message (encrypted of course) receiver also calculates hash and compares it to the sender’s (decrypted using the sender’s public key)
67
Public key cryptography
Problem: Identity: we can only know that it was sent by someone with a given public key; but we can’t be sure of her real identity solution trials: public-key infrastructure (PKI), web of trust (decentralization)
68
Public key cryptography
Mathematical foundation (most usual): calculation of prime factors It is very easy (quick) to multiply two very large prime numbers. It is very hard (very slow) to find the prime factors from the result. Ie. it is an asymmetric algorithm
69
Public key cryptography
Calculation resources needed Let „n” be the „size” of the input. (Think about the number of cities in the travelling salesman problem.) If the algorithm to solve the problem needs running time proportional to a polinomial function of n then this problem is member of P class of problems. It is supposed that there exists a class of problems that can only be solved much slower (eg. exponential – polinomial is very slow, but exp is much slower). If n is large, these problems can take millions of years with the best computers. This class is the NP class. Such a problem is prime factorization. (Interesting fact: if one NP full problem is solved in P time, then all can be solved.) If a faster way of solving NP problems is found, the world’s cryptography (think banking, military, company data etc) would crumble? Quantum computers might give a solution... Millennium Prize Problems - 1millio dollár jutalom, közte a NP=P probléma
70
Public key cryptography
a method :RSA m: message; n,e: public key of receiver, d: private key of receiver; c: encrypted message coding: decoding:
71
Encryption Other methods using code words hiding of message
steganography communication below noise level (spread spectrum) misguiding – hide the true message by sending a more visible false message
72
Encryption code words – these have previously agreed meaning
ie. Navajo codespeakers in WWII: language is known by only few people, and used codewords (ie. names of animals for ships etc) hard to decode if sparingly used, esp. if more codewords have similar meaning Often the codewords are put into a sentence, the rest of the sentence having no real meaning, just for misguiding.
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.