New Kind of Students?

NKS in Information Theory Course

Yehuda Ben-Shimol

Ben Gurion University of the Negev

The traditional way of teaching information theory to electrical and communication engineering students is based on the Shannon theory and his followers. Pre-existing knowledge of probability theory, random variables, and stochastic processes is assumed. In spite of the sophisticated mathematics and ideas presented in such courses (and in most textbooks on the subject) students do not acquire true understanding of the basic concepts underlying this beautiful theory.

The author had a one-time chance to teach the course Introduction to Information Theory and used this opportunity to introduce NKS to the students as an integral part of the course. The course started with a traditional way of teaching information theory with basic concepts and terms of random variables, stochastic processes, entropy, typical sequences, shannon theorems, compression, and related issues for half a semester. As is common in such courses, “toy examples” were used.

The second half of the semester started with a short introduction to NKS findings, ideas, concepts, and research methodologies. One- and two-dimensional cellular automata were presented as data generators having various degrees of complexity. The notion of randomness was introduced, making the students to actually think of its meaning for the first time. Later complexity was introduced with examples taken form the NKS book which was used as a reading material. The course covered topics from the NKS book (Chapters 1–3, part of 5, 6, and the first part of 10, which is directly related to information theory)

The students were asked to do an extensive research-oriented assignment in which they were asked to
• generate data from 1D and 2D two- and three-color CA
• develop automatic classifiers based on information theory concepts (and others) to classify the CA to the four classes determined by S. Wolfram
• use randomized and aggressive search techniques and develop automatic filters for efficient implementations
• implement various compression algorithms using the compression ratio as another classification method
• find new interesting complex CA  

During the semester various aspects of the assignments were discussed in class, increasing the level of understanding of the students to highly surprising levels. Most of the students were enthusiastic from the subject in a way that I did not see before (and is also uncommon). Needless to mention, Mathematica was used for the programming and presentations.

Many students achieved a high level of understanding, innovative and independent thinking, and gained trust in their intuition. They also found NKS interesting. Most of the students presented beautiful results. This talk will represent in detail this experimentation. From this successful experimentation and from many discussions with the students, the author suggests that NKS should be included in ordinary curricula to let the students experience a different and refreshing way of dealing with scientific issues.