Entropy

In statistical mechanics, entropy is an extensive property of a thermodynamic system. It is related to the number Ω of microscopic configurations that are consistent with the macroscopic quantities that characterize the system. Under the assumption that each microstate is probable, the entropy S is the natural logarithm of the number of microstates, multiplied by the Boltzmann constant kB. Formally, S = k B ln Ω. Macroscopic systems have a large number Ω of possible microscopic configurations. For example, the entropy of an ideal gas is proportional to the number of gas molecules N. Twenty liters of gas at room temperature and atmospheric pressure has N ≈ 6×1023. At equilibrium, each of the Ω ≈ eN configurations can be regarded as random and likely; the second law of thermodynamics states. Such systems spontaneously evolve towards the state with maximum entropy. Non-isolated systems may lose entropy, provided their environment's entropy increases by at least that amount so that the total entropy increases.

Entropy is a function of the state of the system, so the change in entropy of a system is determined by its initial and final states. In the idealization that a process is reversible, the entropy does not change, while irreversible processes always increase the total entropy; because it is determined by the number of random microstates, entropy is related to the amount of additional information needed to specify the exact physical state of a system, given its macroscopic specification. For this reason, it is said that entropy is an expression of the disorder, or randomness of a system, or of the lack of information about it; the concept of entropy plays a central role in information theory. Boltzmann's constant, therefore entropy, have dimensions of energy divided by temperature, which has a unit of joules per kelvin in the International System of Units; the entropy of a substance is given as an intensive property—either entropy per unit mass or entropy per unit amount of substance. The French mathematician Lazare Carnot proposed in his 1803 paper Fundamental Principles of Equilibrium and Movement that in any machine the accelerations and shocks of the moving parts represent losses of moment of activity.

In other words, in any natural process there exists an inherent tendency towards the dissipation of useful energy. Building on this work, in 1824 Lazare's son Sadi Carnot published Reflections on the Motive Power of Fire which posited that in all heat-engines, whenever "caloric" falls through a temperature difference, work or motive power can be produced from the actions of its fall from a hot to cold body, he made the analogy with that of. This was an early insight into the second law of thermodynamics. Carnot based his views of heat on the early 18th century "Newtonian hypothesis" that both heat and light were types of indestructible forms of matter, which are attracted and repelled by other matter, on the contemporary views of Count Rumford who showed that heat could be created by friction as when cannon bores are machined. Carnot reasoned that if the body of the working substance, such as a body of steam, is returned to its original state at the end of a complete engine cycle, that "no change occurs in the condition of the working body".

The first law of thermodynamics, deduced from the heat-friction experiments of James Joule in 1843, expresses the concept of energy, its conservation in all processes. In the 1850s and 1860s, German physicist Rudolf Clausius objected to the supposition that no change occurs in the working body, gave this "change" a mathematical interpretation by questioning the nature of the inherent loss of usable heat when work is done, e.g. heat produced by friction. Clausius described entropy as the transformation-content, i.e. dissipative energy use, of a thermodynamic system or working body of chemical species during a change of state. This was in contrast to earlier views, based on the theories of Isaac Newton, that heat was an indestructible particle that had mass. Scientists such as Ludwig Boltzmann, Josiah Willard Gibbs, James Clerk Maxwell gave entropy a statistical basis. In 1877 Boltzmann visualized a probabilistic way to measure the entropy of an ensemble of ideal gas particles, in which he defined entropy to be proportional to the natural logarithm of the number of microstates such a gas could occupy.

Henceforth, the essential problem in statistical thermodynamics, i.e. according to Erwin Schrödinger, has been to determine the distribution of a given amount of energy E over N identical systems. Carathéodory linked entropy with a mathematical definition of irreversibility, in terms of trajectories and integrability. There are two related definitions of entropy: the thermodynamic definition and the statistical mechanics definition; the classical thermodynamics definition developed first. In the classical thermodynamics viewpoint, the system is composed of large numbers of constituents and the state of the system is described by the average thermodynamic properties of those constituents.

Information

Information is the resolution of uncertainty. Information is associated with data and knowledge, as data is meaningful information and represents the values attributed to parameters, knowledge signifies understanding of an abstract or concrete concept; the existence of information can be uncoupled from an observer, which refers to that which accesses information to discern that which it specifies. In the case of knowledge, the information itself requires a cognitive observer to be accessed. In terms of communication, information is expressed either as the content of a message or through direct or indirect observation. That, perceived can be construed as a message in its own right, in that sense, information is always conveyed as the content of a message. Information can be encoded into various forms for interpretation, it can be encrypted for safe storage and communication. Information reduces uncertainty; the uncertainty of an event is measured by its probability of occurrence and is inversely proportional to that.

The more uncertain an event, the more information is required to resolve uncertainty of that event. The bit is a typical unit of information. For example, the information encoded in one "fair" coin flip is log2 = 1 bit, in two fair coin flips is log2 = 2 bits; the concept of information has different meanings in different contexts. Thus the concept becomes related to notions of constraint, control, form, knowledge, understanding, mental stimuli, perception and entropy; the English word derives from the Latin stem of the nominative: this noun derives from the verb informare in the sense of "to give form to the mind", "to discipline", "instruct", "teach". Inform itself comes from the Latin verb informare, which means to form an idea of. Furthermore, Latin itself contained the word informatio meaning concept or idea, but the extent to which this may have influenced the development of the word information in English is not clear; the ancient Greek word for form was μορφή and εἶδος "kind, shape, set", the latter word was famously used in a technical philosophical sense by Plato to denote the ideal identity or essence of something.'Eidos' can be associated with thought, proposition, or concept.

The ancient Greek word for information is πληροφορία, which transliterates from πλήρης "fully" and φέρω frequentative of to carry through. It means "bears fully" or "conveys fully". In modern Greek the word Πληροφορία is still in daily use and has the same meaning as the word information in English. In addition to its primary meaning, the word Πληροφορία as a symbol has deep roots in Aristotle's semiotic triangle. In this regard it can be interpreted to communicate information to the one decoding that specific type of sign; this is something that occurs with the etymology of many words in ancient and modern Greek where there is a strong denotative relationship between the signifier, e.g. the word symbol that conveys a specific encoded interpretation, the signified, e.g. a concept whose meaning the interpreter attempts to decode. In English, “information” is an uncountable mass noun. In information theory, information is taken as an ordered sequence of symbols from an alphabet, say an input alphabet χ, an output alphabet ϒ.

Information processing consists of an input-output function that maps any input sequence from χ into an output sequence from ϒ. The mapping may be deterministic, it may be memoryless. Information can be viewed as a type of input to an organism or system. Inputs are of two kinds. In his book Sensory Ecology Dusenbery called these causal inputs. Other inputs are important only because they are associated with causal inputs and can be used to predict the occurrence of a causal input at a time; some information is important because of association with other information but there must be a connection to a causal input. In practice, information is carried by weak stimuli that must be detected by specialized sensory systems and amplified by energy inputs before they can be functional to the organism or system. For example, light is a causal input to plants but for animals it only provides information; the colored light reflected from a flower is too weak to do much photosynthetic work but the visual system of the bee detects it and the bee's nervous system uses the information to guide the bee to the flower, where the bee finds nectar or pollen, which are causal inputs, serving a nutritional function.

The cognitive scientist and applied mathematician Ronaldo Vigo argues that information is a concept that requires at least two related entities to make quantitative sense. These are, any dimensionally defined category of objects S, any of its subsets R. R, in essence, is a representation of S, or, in other words, conveys representational information about S. Vigo defines the amount of information that R conveys a

Randomness

Randomness is the lack of pattern or predictability in events. A random sequence of events, symbols or steps has no order and does not follow an intelligible pattern or combination. Individual random events are by definition unpredictable, but in many cases the frequency of different outcomes over a large number of events is predictable. For example, when throwing two dice, the outcome of any particular roll is unpredictable, but a sum of 7 will occur twice as as 4. In this view, randomness is a measure of uncertainty of an outcome, rather than haphazardness, applies to concepts of chance and information entropy; the fields of mathematics and statistics use formal definitions of randomness. In statistics, a random variable is an assignment of a numerical value to each possible outcome of an event space; this association facilitates the calculation of probabilities of the events. Random variables can appear in random sequences. A random process is a sequence of random variables whose outcomes do not follow a deterministic pattern, but follow an evolution described by probability distributions.

These and other constructs are useful in probability theory and the various applications of randomness. Randomness is most used in statistics to signify well-defined statistical properties. Monte Carlo methods, which rely on random input, are important techniques in science, as, for instance, in computational science. By analogy, quasi-Monte Carlo methods use quasirandom number generators. Random selection, when narrowly associated with a simple random sample, is a method of selecting items from a population where the probability of choosing a specific item is the proportion of those items in the population. For example, with a bowl containing just 10 red marbles and 90 blue marbles, a random selection mechanism would choose a red marble with probability 1/10. Note that a random selection mechanism that selected 10 marbles from this bowl would not result in 1 red and 9 blue. In situations where a population consists of items that are distinguishable, a random selection mechanism requires equal probabilities for any item to be chosen.

That is, if the selection process is such that each member of a population, of say research subjects, has the same probability of being chosen we can say the selection process is random. In ancient history, the concepts of chance and randomness were intertwined with that of fate. Many ancient peoples threw dice to determine fate, this evolved into games of chance. Most ancient cultures used various methods of divination to attempt to circumvent randomness and fate; the Chinese of 3000 years ago were the earliest people to formalize odds and chance. The Greek philosophers discussed randomness at length, but only in non-quantitative forms, it was only in the 16th century that Italian mathematicians began to formalize the odds associated with various games of chance. The invention of the calculus had a positive impact on the formal study of randomness. In the 1888 edition of his book The Logic of Chance John Venn wrote a chapter on The conception of randomness that included his view of the randomness of the digits of the number pi by using them to construct a random walk in two dimensions.

The early part of the 20th century saw a rapid growth in the formal analysis of randomness, as various approaches to the mathematical foundations of probability were introduced. In the mid- to late-20th century, ideas of algorithmic information theory introduced new dimensions to the field via the concept of algorithmic randomness. Although randomness had been viewed as an obstacle and a nuisance for many centuries, in the 20th century computer scientists began to realize that the deliberate introduction of randomness into computations can be an effective tool for designing better algorithms. In some cases such randomized algorithms outperform the best deterministic methods. Many scientific fields are concerned with randomness: In the 19th century, scientists used the idea of random motions of molecules in the development of statistical mechanics to explain phenomena in thermodynamics and the properties of gases. According to several standard interpretations of quantum mechanics, microscopic phenomena are objectively random.

That is, in an experiment that controls all causally relevant parameters, some aspects of the outcome still vary randomly. For example, if a single unstable atom is placed in a controlled environment, it cannot be predicted how long it will take for the atom to decay—only the probability of decay in a given time. Thus, quantum mechanics does not specify the outcome of individual experiments but only the probabilities. Hidden variable theories reject the view that nature contains irreducible randomness: such theories posit that in the processes that appear random, properties with a certain statistical distribution are at work behind the scenes, determining the outcome in each case; the modern evolutionary synthesis ascribes the observed diversity of life to random genetic mutations followed by natural selection. The latter retains some random mutations in the gene pool due to the systematically improved chance for survival and reproduction that those mutated genes confer on individuals who possess them.

Several authors claim that evolution and sometimes development require a specific form of randomness, namely the introduction of qualitatively new behaviors. Instead of the choice of one possibility among several pre-given ones, this randomness corresponds to the formation of new possibilities; the characteristics of an organism arise to some extent deterministically and to som

Statistics

Statistics is a branch of mathematics dealing with data collection, analysis and presentation. In applying statistics to, for example, a scientific, industrial, or social problem, it is conventional to begin with a statistical population or a statistical model process to be studied. Populations can be diverse topics such as "all people living in a country" or "every atom composing a crystal". Statistics deals with every aspect of data, including the planning of data collection in terms of the design of surveys and experiments. See glossary of probability and statistics; when census data cannot be collected, statisticians collect data by developing specific experiment designs and survey samples. Representative sampling assures that inferences and conclusions can reasonably extend from the sample to the population as a whole. An experimental study involves taking measurements of the system under study, manipulating the system, taking additional measurements using the same procedure to determine if the manipulation has modified the values of the measurements.

In contrast, an observational study does not involve experimental manipulation. Two main statistical methods are used in data analysis: descriptive statistics, which summarize data from a sample using indexes such as the mean or standard deviation, inferential statistics, which draw conclusions from data that are subject to random variation. Descriptive statistics are most concerned with two sets of properties of a distribution: central tendency seeks to characterize the distribution's central or typical value, while dispersion characterizes the extent to which members of the distribution depart from its center and each other. Inferences on mathematical statistics are made under the framework of probability theory, which deals with the analysis of random phenomena. A standard statistical procedure involves the test of the relationship between two statistical data sets, or a data set and synthetic data drawn from an idealized model. A hypothesis is proposed for the statistical relationship between the two data sets, this is compared as an alternative to an idealized null hypothesis of no relationship between two data sets.

Rejecting or disproving the null hypothesis is done using statistical tests that quantify the sense in which the null can be proven false, given the data that are used in the test. Working from a null hypothesis, two basic forms of error are recognized: Type I errors and Type II errors. Multiple problems have come to be associated with this framework: ranging from obtaining a sufficient sample size to specifying an adequate null hypothesis. Measurement processes that generate statistical data are subject to error. Many of these errors are classified as random or systematic, but other types of errors can be important; the presence of missing data or censoring may result in biased estimates and specific techniques have been developed to address these problems. Statistics can be said to have begun in ancient civilization, going back at least to the 5th century BC, but it was not until the 18th century that it started to draw more from calculus and probability theory. In more recent years statistics has relied more on statistical software to produce tests such as descriptive analysis.

Some definitions are: Merriam-Webster dictionary defines statistics as "a branch of mathematics dealing with the collection, analysis and presentation of masses of numerical data." Statistician Arthur Lyon Bowley defines statistics as "Numerical statements of facts in any department of inquiry placed in relation to each other."Statistics is a mathematical body of science that pertains to the collection, interpretation or explanation, presentation of data, or as a branch of mathematics. Some consider statistics to be a distinct mathematical science rather than a branch of mathematics. While many scientific investigations make use of data, statistics is concerned with the use of data in the context of uncertainty and decision making in the face of uncertainty. Mathematical statistics is the application of mathematics to statistics. Mathematical techniques used for this include mathematical analysis, linear algebra, stochastic analysis, differential equations, measure-theoretic probability theory.

In applying statistics to a problem, it is common practice to start with a population or process to be studied. Populations can be diverse topics such as "all people living in a country" or "every atom composing a crystal". Ideally, statisticians compile data about the entire population; this may be organized by governmental statistical institutes. Descriptive statistics can be used to summarize the population data. Numerical descriptors include mean and standard deviation for continuous data types, while frequency and percentage are more useful in terms of describing categorical data; when a census is not feasible, a chosen subset of the population called. Once a sample, representative of the population is determined, data is collected for the sample members in an observational or experimental setting. Again, descriptive statistics can be used to summarize the sample data. However, the drawing of the sample has been subject to an element of randomness, hence the established numerical descriptors from the sample are due to uncertainty.

To still draw meaningful conclusions about the entire population, in

Telecommunication

Telecommunication is the transmission of signs, messages, writings and sounds or information of any nature by wire, optical or other electromagnetic systems. Telecommunication occurs when the exchange of information between communication participants includes the use of technology, it is transmitted either electrically over physical media, such as cables, or via electromagnetic radiation. Such transmission paths are divided into communication channels which afford the advantages of multiplexing. Since the Latin term communicatio is considered the social process of information exchange, the term telecommunications is used in its plural form because it involves many different technologies. Early means of communicating over a distance included visual signals, such as beacons, smoke signals, semaphore telegraphs, signal flags, optical heliographs. Other examples of pre-modern long-distance communication included audio messages such as coded drumbeats, lung-blown horns, loud whistles. 20th- and 21st-century technologies for long-distance communication involve electrical and electromagnetic technologies, such as telegraph and teleprinter, radio, microwave transmission, fiber optics, communications satellites.

A revolution in wireless communication began in the first decade of the 20th century with the pioneering developments in radio communications by Guglielmo Marconi, who won the Nobel Prize in Physics in 1909, other notable pioneering inventors and developers in the field of electrical and electronic telecommunications. These included Charles Wheatstone and Samuel Morse, Alexander Graham Bell, Edwin Armstrong and Lee de Forest, as well as Vladimir K. Zworykin, John Logie Baird and Philo Farnsworth; the word telecommunication is a compound of the Greek prefix tele, meaning distant, far off, or afar, the Latin communicare, meaning to share. Its modern use is adapted from the French, because its written use was recorded in 1904 by the French engineer and novelist Édouard Estaunié. Communication was first used as an English word in the late 14th century, it comes from Old French comunicacion, from Latin communicationem, noun of action from past participle stem of communicare "to share, divide out.

Homing pigeons have been used throughout history by different cultures. Pigeon post had Persian roots, was used by the Romans to aid their military. Frontinus said; the Greeks conveyed the names of the victors at the Olympic Games to various cities using homing pigeons. In the early 19th century, the Dutch government used the system in Sumatra, and in 1849, Paul Julius Reuter started a pigeon service to fly stock prices between Aachen and Brussels, a service that operated for a year until the gap in the telegraph link was closed. In the Middle Ages, chains of beacons were used on hilltops as a means of relaying a signal. Beacon chains suffered the drawback that they could only pass a single bit of information, so the meaning of the message such as "the enemy has been sighted" had to be agreed upon in advance. One notable instance of their use was during the Spanish Armada, when a beacon chain relayed a signal from Plymouth to London. In 1792, Claude Chappe, a French engineer, built the first fixed visual telegraphy system between Lille and Paris.

However semaphore suffered from the need for skilled operators and expensive towers at intervals of ten to thirty kilometres. As a result of competition from the electrical telegraph, the last commercial line was abandoned in 1880. On 25 July 1837 the first commercial electrical telegraph was demonstrated by English inventor Sir William Fothergill Cooke, English scientist Sir Charles Wheatstone. Both inventors viewed their device as "an improvement to the electromagnetic telegraph" not as a new device. Samuel Morse independently developed a version of the electrical telegraph that he unsuccessfully demonstrated on 2 September 1837, his code was an important advance over Wheatstone's signaling method. The first transatlantic telegraph cable was completed on 27 July 1866, allowing transatlantic telecommunication for the first time; the conventional telephone was invented independently by Alexander Bell and Elisha Gray in 1876. Antonio Meucci invented the first device that allowed the electrical transmission of voice over a line in 1849.

However Meucci's device was of little practical value because it relied upon the electrophonic effect and thus required users to place the receiver in their mouth to "hear" what was being said. The first commercial telephone services were set-up in 1878 and 1879 on both sides of the Atlantic in the cities of New Haven and London. Starting in 1894, Italian inventor Guglielmo Marconi began developing a wireless communication using the newly discovered phenomenon of radio waves, showing by 1901 that they could be transmitted across the Atlantic Ocean; this was the start of wireless telegraphy by radio. Voice and music had little early success. World War I accelerated the development of radio for military communications. After the war, commercial radio AM broadcasting began in the 1920s and became an important mass medium for entertainment and news. World War II again accelerated development of radio for the wartime purposes of aircraft and land communication, radio navigation and radar. Development of stereo FM broadcasting of radio

Modulation

In electronics and telecommunications, modulation is the process of varying one or more properties of a periodic waveform, called the carrier signal, with a modulating signal that contains information to be transmitted. Most radio systems in the 20th century used frequency modulation or amplitude modulation for radio broadcast. A modulator is a device. A demodulator is a device that performs the inverse of modulation. A modem can perform both operations; the aim of analog modulation is to transfer an analog baseband signal, for example an audio signal or TV signal, over an analog bandpass channel at a different frequency, for example over a limited radio frequency band or a cable TV network channel. The aim of digital modulation is to transfer a digital bit stream over an analog communication channel, for example over the public switched telephone network or over a limited radio frequency band. Analog and digital modulation facilitate frequency division multiplexing, where several low pass information signals are transferred over the same shared physical medium, using separate passband channels.

The aim of digital baseband modulation methods known as line coding, is to transfer a digital bit stream over a baseband channel a non-filtered copper wire such as a serial bus or a wired local area network. The aim of pulse modulation methods is to transfer a narrowband analog signal, for example, a phone call over a wideband baseband channel or, in some of the schemes, as a bit stream over another digital transmission system. In music synthesizers, modulation may be used to synthesize waveforms with an extensive overtone spectrum using a small number of oscillators. In this case, the carrier frequency is in the same order or much lower than the modulating waveform. In analog modulation, the modulation is applied continuously in response to the analog information signal. Common analog modulation techniques include: Amplitude modulation Double-sideband modulation Double-sideband modulation with carrier Double-sideband suppressed-carrier transmission Double-sideband reduced carrier transmission Single-sideband modulation Single-sideband modulation with carrier Single-sideband modulation suppressed carrier modulation Vestigial sideband modulation Quadrature amplitude modulation Angle modulation, constant envelope Frequency modulation Phase modulation Transpositional Modulation, in which the waveform inflection is modified resulting in a signal where each quarter cycle is transposed in the modulation process.

TM is a pseudo-analog modulation. Where an AM carrier carries a phase variable phase f. TM is f. Digital modulation methods can be considered as digital-to-analog conversion and the corresponding demodulation or detection as analog-to-digital conversion; the changes in the carrier signal are chosen from a finite number of M alternative symbols. A simple example: A telephone line is designed for transferring audible sounds, for example and not digital bits. Computers may, communicate over a telephone line by means of modems, which are representing the digital bits by tones, called symbols. If there are four alternative symbols, the first symbol may represent the bit sequence 00, the second 01, the third 10 and the fourth 11. If the modem plays a melody consisting of 1000 tones per second, the symbol rate is 1000 symbols/second, or 1000 baud. Since each tone represents a message consisting of two digital bits in this example, the bit rate is twice the symbol rate, i.e. 2000 bits per second. This is similar to the technique used by dial-up modems as opposed to DSL modems.

According to one definition of digital signal, the modulated signal is a digital signal. According to another definition, the modulation is a form of digital-to-analog conversion. Most textbooks would consider digital modulation schemes as a form of digital transmission, synonymous to data transmission; the most fundamental digital modulation techniques are based on keying: PSK: a finite number of phases are used. FSK: a finite number of frequencies are used. ASK: a finite number of amplitudes are used. QAM: a finite number of at least two phases and at least two amplitudes are used. In QAM, an in-phase signal and a quadrature phase signal are amplitude modulated with a finite number of amplitudes and summed, it can be seen as a two-channel system, each channel using ASK. The resulting signal is equivalent to a combination of PSK and ASK. In all of the above methods, each of these phases, frequencies or amplitudes are assigned a u