1.
Probabilistic algorithm
–
A randomized algorithm is an algorithm that employs a degree of randomness as part of its logic. The algorithm typically uses uniformly random bits as an input to guide its behavior. Formally, the performance will be a random variable determined by the random bits, thus either the running time. In the second case, random performance and random output, the algorithm for a procedure is somewhat questionable. In the case of output, it is no longer formally effective. However, in cases, probabilistic algorithms are the only practical means of solving a problem. As a motivating example, consider the problem of finding an ‘a’ in an array of n elements, input, An array of n≥2 elements, in which half are ‘a’s and the other half are ‘b’s. Output, Find an ‘a’ in the array and we give two versions of the algorithm, one Las Vegas algorithm and one Monte Carlo algorithm. Las Vegas algorithm, This algorithm succeeds with probability 1, after k iterations, the probability of finding an ‘a’ is, This algorithm does not guarantee success, but the run time is bounded. The number of iterations is less than or equal to k. Taking k to be constant the run time is Θ, Randomized algorithms are particularly useful when faced with a malicious adversary or attacker who deliberately tries to feed a bad input to the algorithm such as in the Prisoners dilemma. It is for this reason that randomness is ubiquitous in cryptography, in cryptographic applications, pseudo-random numbers cannot be used, since the adversary can predict them, making the algorithm effectively deterministic. Therefore, either a source of random numbers or a cryptographically secure pseudo-random number generator is required. Another area in which randomness is inherent is quantum computing, in the example above, the Las Vegas algorithm always outputs the correct answer, but its running time is a random variable. The Monte Carlo algorithm is guaranteed to complete in an amount of time that can be bounded by a function the size and its parameter k. Observe that any Las Vegas algorithm can be converted into a Monte Carlo algorithm, by having it output an arbitrary, Computational complexity theory models randomized algorithms as probabilistic Turing machines. Both Las Vegas and Monte Carlo algorithms are considered, and several complexity classes are studied, the complement class for RP is co-RP. Problem classes having algorithms with polynomial time average case running time whose output is always correct are said to be in ZPP, the class of problems for which both YES and NO-instances are allowed to be identified with some error is called BPP
2.
Maxima and minima
–
Pierre de Fermat was one of the first mathematicians to propose a general technique, adequality, for finding the maxima and minima of functions. As defined in set theory, the maximum and minimum of a set are the greatest and least elements in the set, unbounded infinite sets, such as the set of real numbers, have no minimum or maximum. The value of the function at a point is called the maximum value of the function. Similarly, the function has a minimum point at x∗ if f ≤ f for all x in X within distance ε of x∗. A similar definition can be used when X is a space, since the definition just given can be rephrased in terms of neighbourhoods. Note that a maximum point is always a local maximum point. In both the global and local cases, the concept of a strict extremum can be defined, note that a point is a strict global maximum point if and only if it is the unique global maximum point, and similarly for minimum points. A continuous real-valued function with a compact domain always has a maximum point, an important example is a function whose domain is a closed interval of real numbers. Finding global maxima and minima is the goal of mathematical optimization, if a function is continuous on a closed interval, then by the extreme value theorem global maxima and minima exist. Furthermore, a global maximum either must be a maximum in the interior of the domain. So a method of finding a maximum is to look at all the local maxima in the interior, and also look at the maxima of the points on the boundary. Local extrema of functions can be found by Fermats theorem. For any function that is defined piecewise, one finds a maximum by finding the maximum of each piece separately, the function x2 has a unique global minimum at x =0. The function x3 has no global minima or maxima, although the first derivative is 0 at x =0, this is an inflection point. The function x x has a global maximum at x = e. The function x−x has a global maximum over the positive real numbers at x = 1/e. The function x3/3 − x has first derivative x2 −1, setting the first derivative to 0 and solving for x gives stationary points at −1 and +1. From the sign of the second derivative we can see that −1 is a local maximum, note that this function has no global maximum or minimum
3.
Function (mathematics)
–
In mathematics, a function is a relation between a set of inputs and a set of permissible outputs with the property that each input is related to exactly one output. An example is the function that each real number x to its square x2. The output of a function f corresponding to a x is denoted by f. In this example, if the input is −3, then the output is 9, likewise, if the input is 3, then the output is also 9, and we may write f =9. The input variable are sometimes referred to as the argument of the function, Functions of various kinds are the central objects of investigation in most fields of modern mathematics. There are many ways to describe or represent a function, some functions may be defined by a formula or algorithm that tells how to compute the output for a given input. Others are given by a picture, called the graph of the function, in science, functions are sometimes defined by a table that gives the outputs for selected inputs. A function could be described implicitly, for example as the inverse to another function or as a solution of a differential equation, sometimes the codomain is called the functions range, but more commonly the word range is used to mean, instead, specifically the set of outputs. For example, we could define a function using the rule f = x2 by saying that the domain and codomain are the numbers. The image of this function is the set of real numbers. In analogy with arithmetic, it is possible to define addition, subtraction, multiplication, another important operation defined on functions is function composition, where the output from one function becomes the input to another function. Linking each shape to its color is a function from X to Y, each shape is linked to a color, there is no shape that lacks a color and no shape that has more than one color. This function will be referred to as the color-of-the-shape function, the input to a function is called the argument and the output is called the value. The set of all permitted inputs to a function is called the domain of the function. Thus, the domain of the function is the set of the four shapes. The concept of a function does not require that every possible output is the value of some argument, a second example of a function is the following, the domain is chosen to be the set of natural numbers, and the codomain is the set of integers. The function associates to any number n the number 4−n. For example, to 1 it associates 3 and to 10 it associates −6, a third example of a function has the set of polygons as domain and the set of natural numbers as codomain
4.
Metaheuristic
–
Metaheuristics sample a set of solutions which is too large to be completely sampled. Metaheuristics may make few assumptions about the problem being solved. Compared to optimization algorithms and iterative methods, metaheuristics do not guarantee that an optimal solution can be found on some class of problems. Many metaheuristics implement some form of optimization, so that the solution found is dependent on the set of random variables generated. As such, they are useful approaches for optimization problems, several books and survey papers have been published on the subject. Most literature on metaheuristics is experimental in nature, describing empirical results based on experiments with the algorithms. But some formal theoretical results are available, often on convergence. Many metaheuristic methods have been published with claims of novelty and practical efficacy and these are properties that characterize most metaheuristics, Metaheuristics are strategies that guide the search process. The goal is to explore the search space in order to find near–optimal solutions. Techniques which constitute metaheuristic algorithms range from simple local search procedures to complex learning processes, metaheuristic algorithms are approximate and usually non-deterministic. There are a variety of metaheuristics and a number of properties along which to classify them. One approach is to characterize the type of search strategy, one type of search strategy is an improvement on simple local search algorithms. A well known local search algorithm is the hill climbing method which is used to find local optimums, however, hill climbing does not guarantee finding global optimum solutions. Many metaheuristic ideas were proposed to improve local search heuristic in order to better solutions. Such metaheuristics include simulated annealing, tabu search, iterated local search, variable neighborhood search and these metaheuristics can both be classified as local search-based or global search metaheuristics. Other global search metaheuristic that are not local search-based are usually population-based metaheuristics, such metaheuristics include ant colony optimization, evolutionary computation, particle swarm optimization, and genetic algorithms. Another classification dimension is single solution vs population-based searches, another category of metaheuristics is Swarm intelligence which is a collective behavior of decentralized, self-organized agents in a population or swarm. Ant colony optimization, particle swarm optimization, social cognitive optimization, penguins search optimization algorithm, a hybrid metaheuristic is one which combines a metaheuristic with other optimization approaches, such as algorithms from mathematical programming, constraint programming, and machine learning
5.
Mathematical optimization
–
In mathematics, computer science and operations research, mathematical optimization, also spelled mathematical optimisation, is the selection of a best element from some set of available alternatives. The generalization of optimization theory and techniques to other formulations comprises an area of applied mathematics. Such a formulation is called a problem or a mathematical programming problem. Many real-world and theoretical problems may be modeled in this general framework, typically, A is some subset of the Euclidean space Rn, often specified by a set of constraints, equalities or inequalities that the members of A have to satisfy. The domain A of f is called the space or the choice set. The function f is called, variously, a function, a loss function or cost function, a utility function or fitness function, or, in certain fields. A feasible solution that minimizes the objective function is called an optimal solution, in mathematics, conventional optimization problems are usually stated in terms of minimization. Generally, unless both the function and the feasible region are convex in a minimization problem, there may be several local minima. While a local minimum is at least as good as any nearby points, a global minimum is at least as good as every feasible point. In a convex problem, if there is a minimum that is interior, it is also the global minimum. Optimization problems are often expressed with special notation, consider the following notation, min x ∈ R This denotes the minimum value of the objective function x 2 +1, when choosing x from the set of real numbers R. The minimum value in case is 1, occurring at x =0. Similarly, the notation max x ∈ R2 x asks for the value of the objective function 2x. In this case, there is no such maximum as the function is unbounded. This represents the value of the argument x in the interval, John Wiley & Sons, Ltd. pp. xxviii+489. (2008 Second ed. in French, Programmation mathématique, Théorie et algorithmes, Editions Tec & Doc, Paris,2008. Nemhauser, G. L. Rinnooy Kan, A. H. G. Todd, handbooks in Operations Research and Management Science. Amsterdam, North-Holland Publishing Co. pp. xiv+709, J. E. Dennis, Jr. and Robert B
6.
Gradient descent
–
Gradient descent is a first-order iterative optimization algorithm. To find a minimum of a function using gradient descent. If instead one takes steps proportional to the positive of the gradient, one approaches a maximum of that function. Gradient descent is known as steepest descent, or the method of steepest descent. Gradient descent should not be confused with the method of steepest descent for approximating integrals and it follows that, if a n +1 = a n − γ ∇ F for γ small enough, then F ≥ F. In other words, the term γ ∇ F is subtracted from a because we want to move against the gradient and we have F ≥ F ≥ F ≥ ⋯, so hopefully the sequence converges to the desired local minimum. Note that the value of the step size γ is allowed to change at every iteration. With certain assumptions on the function F and particular choices of γ, γ n = T | | ∇ F − ∇ F | |2 convergence to a minimum can be guaranteed. When the function F is convex, all local minima are also global minima and this process is illustrated in the adjacent picture. Here F is assumed to be defined on the plane, the blue curves are the contour lines, that is, the regions on which the value of F is constant. A red arrow originating at a point shows the direction of the gradient at that point. Note that the gradient at a point is orthogonal to the line going through that point. We see that gradient descent leads us to the bottom of the bowl, Gradient descent has problems with pathological functions such as the Rosenbrock function shown here. The Rosenbrock function has a curved valley which contains the minimum. The bottom of the valley is very flat, because of the curved flat valley the optimization is zig-zagging slowly with small stepsizes towards the minimum. The Zig-Zagging nature of the method is also evident below, where the gradient descent method is applied to F = sin cos . For some of the examples, gradient descent is relatively slow close to the minimum, technically. For poorly conditioned convex problems, gradient descent increasingly zigzags as the point nearly orthogonally to the shortest direction to a minimum point
7.
Annealing (metallurgy)
–
It involves heating a material to above its recrystallization temperature, maintaining a suitable temperature, and then cooling. In annealing, atoms migrate in the lattice and the number of dislocations decreases, leading to the change in ductility. In the cases of copper, steel, silver, and brass, copper, silver and brass can be cooled slowly in air, or quickly by quenching in water, unlike ferrous metals, such as steel, which must be cooled slowly to anneal. In this fashion, the metal is softened and prepared for further work—such as shaping, stamping, annealing occurs by the diffusion of atoms within a solid material, so that the material progresses towards its equilibrium state. Heat increases the rate of diffusion by providing the energy needed to break bonds, the movement of atoms has the effect of redistributing and eradicating the dislocations in metals and in ceramics. This alteration to existing dislocations allows an object to deform more easily. The amount of process-initiating Gibbs free energy in a metal is also reduced by the annealing process. In practice and industry, this reduction of Gibbs free energy is termed stress relief, the relief of internal stresses is a thermodynamically spontaneous process, however, at room temperatures, it is a very slow process. The high temperatures at which annealing occurs serve to accelerate this process, the creation of lattice vacancies is governed by the Arrhenius equation, and the migration/diffusion of lattice vacancies are governed by Fick’s laws of diffusion. The three stages of the process that proceed as the temperature of the material is increased are, recovery, recrystallization. The first stage is recovery, and it results in softening of the metal through removal of primarily linear defects called dislocations, recovery occurs at the lower temperature stage of all annealing processes and before the appearance of new strain-free grains. The grain size and shape do not change, the second stage is recrystallization, where new strain-free grains nucleate and grow to replace those deformed by internal stresses. If annealing is allowed to continue once recrystallization has completed, then grain growth occurs, in grain growth, the microstructure starts to coarsen and may cause the metal to lose a substantial part of its original strength. This can however be regained with hardening, the high temperature of annealing may result in oxidation of the metal’s surface, resulting in scale. If scale must be avoided, annealing is carried out in a special atmosphere, annealing is also done in forming gas, a mixture of hydrogen and nitrogen. The magnetic properties of mu-metal are introduced by annealing the alloy in a hydrogen atmosphere, typically, large ovens are used for the annealing process. The inside of the oven is large enough to place the workpiece in a position to receive exposure to the circulating heated air. For high volume process annealing, gas fired conveyor furnaces are often used, for large workpieces or high quantity parts, car-bottom furnaces are used so workers can easily move the parts in and out
8.
Monte Carlo method
–
Monte Carlo methods are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results. Their essential idea is using randomness to solve problems that might be deterministic in principle and they are often used in physical and mathematical problems and are most useful when it is difficult or impossible to use other approaches. Monte Carlo methods are used in three distinct problem classes, optimization, numerical integration, and generating draws from a probability distribution. In principle, Monte Carlo methods can be used to any problem having a probabilistic interpretation. By the law of numbers, integrals described by the expected value of some random variable can be approximated by taking the empirical mean of independent samples of the variable. When the probability distribution of the variable is parametrized, mathematicians often use a Markov Chain Monte Carlo sampler, the central idea is to design a judicious Markov chain model with a prescribed stationary probability distribution. That is, in the limit, the samples being generated by the MCMC method will be samples from the desired distribution, by the ergodic theorem, the stationary distribution is approximated by the empirical measures of the random states of the MCMC sampler. In other problems, the objective is generating draws from a sequence of probability distributions satisfying a nonlinear evolution equation, in other instances we are given a flow of probability distributions with an increasing level of sampling complexity. These models can also be seen as the evolution of the law of the states of a nonlinear Markov chain. In contrast with traditional Monte Carlo and Markov chain Monte Carlo methodologies these mean field particle techniques rely on sequential interacting samples, the terminology mean field reflects the fact that each of the samples interacts with the empirical measures of the process. Monte Carlo methods vary, but tend to follow a particular pattern, generate inputs randomly from a probability distribution over the domain. Perform a deterministic computation on the inputs, for example, consider a circle inscribed in a unit square. Given that the circle and the square have a ratio of areas that is π/4, uniformly scatter objects of uniform size over the square. Count the number of objects inside the circle and the number of objects. The ratio of the two counts is an estimate of the ratio of the two areas, which is π/4, multiply the result by 4 to estimate π. In this procedure the domain of inputs is the square that circumscribes our circle and we generate random inputs by scattering grains over the square then perform a computation on each input. Finally, we aggregate the results to obtain our final result, there are two important points to consider here, Firstly, if the grains are not uniformly distributed, then our approximation will be poor. Secondly, there should be a number of inputs
9.
Marshall Rosenbluth
–
Marshall Nicholas Rosenbluth was an American plasma physicist and member of the National Academy of Sciences. In 1997 he was awarded the National Medal of Science for discoveries in controlled thermonuclear fusion, contributions to plasma physics, and work in computational statistical mechanics. He was also a recipient of the E. O. Lawrence Prize, the Albert Einstein Award, the James Clerk Maxwell Prize in Plasma Physics, the Enrico Fermi Award, and the Hannes Alfvén Prize. Hofstadter refers to this in his 1961 Nobel Lecture, This behavior can be understood in terms of the theoretical scattering law developed by M. Rosenbluth in 1950. Although he continued to work on physics for the remainder of his career. For example, around 1980, he and coworkers produced an analysis of the free electron laser. He maintained a high productivity rate throughout his entire career, indeed, only a few years before his death, Rosenbluth discovered the existence of residual flows, a key result for understanding turbulence in tokamaks. Rosenbluth graduated from Stuyvesant High School in 1942 and he did his undergraduate study at Harvard, graduating in 1946, despite also serving in the U. S. Navy during this period. He received his Ph. D. in 1949 from the University of Chicago, in 1950, Edward Teller, considered the father of the hydrogen bomb, recruited Rosenbluth to work at Los Alamos. Rosenbluth maintained this position until 1956, the research he conducted at Los Alamos led to the development of the H-bomb. Upon his retirement, he took on the responsibility of chief scientist of the Central Team for the International Thermonuclear Experimental Reactor until 1999, Rosenbluth also served as a member of the JASON Defense Advisory Group. Rosenbluth was affectionately known as the Pope of Plasma Physics in reference to his understanding of the field. J. W. Van Dam, From Particles to Plasmas, Lectures Honoring Marshall N. Rosenbluth, beichl and F. Sullivan, The Metropolis Algorithm, Computing in Science and Engineering, Volume 2, Number 1, Jan/Feb 2000. J. E. Gubernatis, Marshall Rosenbluth and the Metropolis Algorithm, equations of State Calculations by Fast Computing Machines. 1994 Audio Interview with Marshall Rosenbluth by Richard Rhodes Voices of the Manhattan Project University of Texas Memorial Text Marshall N. Rosenbluth Papers MSS670, special Collections & Archives, UC San Diego Library
10.
Nicholas Metropolis
–
Nicholas Constantine Metropolis was a Greek-American physicist. Metropolis received his BSc and PhD degrees in physics at the University of Chicago and he arrived in Los Alamos in April 1943, as a member of the original staff of fifty scientists. After World War II, he returned to the faculty of the University of Chicago as an assistant professor. He came back to Los Alamos in 1948 to lead the group in the Theoretical Division that designed and built the MANIAC I computer in 1952 that was modeled on the IAS machine, and the MANIAC II in 1957. From 1957 to 1965 he was Professor of Physics at the University of Chicago and was the founding Director of its Institute for Computer Research, in 1965 he returned to Los Alamos where he was made a Laboratory Senior Fellow in 1980. At Los Alamos, in the 1950s, a group of researchers led by Metropolis, including John von Neumann and Stanislaw Ulam, generally speaking, the Monte Carlo method is a statistical approach to solve deterministic many-body problems. In 1953 Metropolis co-authored the first paper on a technique that was central to the now known as simulated annealing. This landmark paper showed the first numerical simulations of a liquid, the algorithm for generating samples from the Boltzmann distribution was later generalized by W. K. Hastings to become the Metropolis-Hastings algorithm. He is credited as part of the team came up with the name Monte Carlo method in reference to a colleagues relatives love for the casinos of Monte Carlo. Monte Carlo methods are a class of algorithms that rely on repeated random sampling to compute their results. In 1987 he became the first Los Alamos employee honored with the title emeritus by the University of California, Metropolis was also awarded the Pioneer Medal by the Institute of Electrical and Electronics Engineers, and was a fellow of the American Physical Society. The Nicholas Metropolis Award for Outstanding Doctoral Thesis Work in Computational Physics is awarded annually by the American Physical Society, Metropolis played the part of a scientist in the Woody Allen film Husbands and Wives. Metropolis had a son, Christopher, and two daughters, Penelope and Katharine and he was an avid skier and tennis player until his mid-seventies. He died at a home in Los Alamos, New Mexico. They played for small sums, but, Metropolis once described what a triumph it was to win ten dollars from John von Neumann. He then bought his book for five dollars and pasted the other five inside the cover as a symbol of his victory, in another passage of his book, Ulam describes Metropolis as a Greek-American with a wonderful personality. Stochastics ENIAC Colossus computer von Neumann paradox 11, ioannis A. Daglis, Από τον ENIAC στα laptop και από τον Nicholas Metropolis στον Nicholas Negroponte –60 χρόνια από την επανάσταση των κομπιούτερ, Geotropio, December 2,2006. 1993 Audio Interview with Nicholas Metropolis by Richard Rhodes Voices of the Manhattan Project Oral history interview with Nicholas C, Metropolis, Conducted by William Aspray at Charles Babbage Institute, University of Minnesota
11.
Thermodynamic state
–
Once such a set of values of thermodynamic variables has been specified for a system, the values of all thermodynamic properties of the system are uniquely determined. Usually, by default, a state is taken to be one of thermodynamic equilibrium. This means that the state is not merely the condition of the system at a specific time, Thermodynamics sets up an idealized formalism that can be summarized by a system of postulates of thermodynamics. A thermodynamic system is not simply a physical system, a thermodynamic system is a macroscopic object, the microscopic details of which are not explicitly considered in its thermodynamic description. The number of state variables required to specify the state depends on the system. Always the number is two or more, usually it is not more than some dozen, the choice is usually made on the basis of the walls and surroundings that are relevant for the thermodynamic processes that are to be considered for the system. For Planck, the characteristic of a thermodynamic state of a system that consists of a single phase. Such non-equilibrium identifying state variables indicate that some non-zero flow may be occurring within the system or between system and surroundings and they are uniquely determined by the thermodynamic state as it has been identified by the original state variables. For an idealized continuous or quasi-static process, this means that infinitesimal incremental changes in such variables are exact differentials, together, the incremental changes throughout the process, and the initial and final states, fully determine the idealized process. In the most commonly cited example, an ideal gas. Thus the thermodynamic state would range over a state space. The remaining variable, as well as other such as the internal energy. The state functions satisfy certain constraints, expressed in the laws of thermodynamics. Various thermodynamic diagrams have been developed to model the transitions between thermodynamic states, physical systems found in nature are practically always dynamic and complex, but in many cases, macroscopic physical systems are amenable to description based on proximity to ideal conditions. One such ideal condition is that of an equilibrium state. Such a state is an object of classical or equilibrium thermodynamics. Based on many observations, thermodynamics postulates that all systems that are isolated from the environment will evolve so as to approach unique stable equilibrium states. A few different types of equilibrium are listed below, thermal Equilibrium, When the temperature throughout a system is uniform, the system is in thermal equilibrium
12.
Internal energy
–
It keeps account of the gains and losses of energy of the system that are due to changes in its internal state. The internal energy of a system can be changed by transfers of matter or heat or by doing work, when matter transfer is prevented by impermeable containing walls, the system is said to be closed. Then the first law of thermodynamics states that the increase in energy is equal to the total heat added plus the work done on the system by its surroundings. If the containing walls pass neither matter nor energy, the system is said to be isolated, the first law of thermodynamics may be regarded as establishing the existence of the internal energy. The internal energy is one of the two cardinal state functions of the variables of a thermodynamic system. The internal energy of a state of a system cannot be directly measured. Such a chain, or path, can be described by certain extensive state variables of the system, namely, its entropy, S, its volume, V. The internal energy, U, is a function of those, sometimes, to that list are appended other extensive state variables, for example electric dipole moment. Customarily, thermodynamic descriptions include only items relevant to the processes under study, Thermodynamics is chiefly concerned only with changes in the internal energy, not with its absolute value. The internal energy is a function of a system, because its value depends only on the current state of the system. It is the one and only cardinal thermodynamic potential, through it, by use of Legendre transforms, are mathematically constructed the other thermodynamic potentials. These are functions of variable lists in which some extensive variables are replaced by their conjugate intensive variables, Legendre transformation is necessary because mere substitutive replacement of extensive variables by intensive variables does not lead to thermodynamic potentials. Mere substitution leads to a less informative formula, an equation of state, though it is a macroscopic quantity, internal energy can be explained in microscopic terms by two theoretical virtual components. One is the kinetic energy due to the microscopic motion of the systems particles. The other is the energy associated with the microscopic forces, including the chemical bonds. If thermonuclear reactions are specified as a topic of concern, then the static rest mass energy of the constituents of matter is also counted. There is no simple relation between these quantities of microscopic energy and the quantities of energy gained or lost by the system in work, heat. The SI unit of energy is the joule, sometimes it is convenient to use a corresponding density called specific internal energy which is internal energy per unit of mass of the system in question
13.
Hill climbing
–
In computer science, hill climbing is a mathematical optimization technique which belongs to the family of local search. It is an algorithm that starts with an arbitrary solution to a problem. If the change produces a solution, an incremental change is made to the new solution. For example, hill climbing can be applied to the travelling salesman problem and it is easy to find an initial solution that visits all the cities but will be very poor compared to the optimal solution. The algorithm starts with such a solution and makes small improvements to it, eventually, a much shorter route is likely to be obtained. Hill-climbing is optimal in convex problems – otherwise it will find only local optima, examples of algorithms that solve convex problems by hill-climbing include the simplex algorithm for linear programming and binary search. To attempt overcoming being stuck in local optima, one could use restarts, or more complex based on iterations, or on memory. The relative simplicity of the algorithm makes it a popular first choice amongst optimizing algorithms and it is used widely in artificial intelligence, for reaching a goal state from a starting node. Choice of next node and starting node can be varied to give a list of related algorithms, although more advanced algorithms such as simulated annealing or tabu search may give better results, in some situations hill climbing works just as well. Hill climbing can often produce a result than other algorithms when the amount of time available to perform a search is limited. It is an algorithm, it can return a valid solution even if its interrupted at any time before it ends. Hill climbing attempts to maximize a target function f, where x is a vector of continuous and/or discrete values, at each iteration, hill climbing will adjust a single element in x and determine whether the change improves the value of f. With hill climbing, any change that improves f is accepted, then x is said to be locally optimal. In discrete vector spaces, each value for x may be visualized as a vertex in a graph. Hill climbing will follow the graph from vertex to vertex, always locally increasing the value of f, in simple hill climbing, the first closer node is chosen, whereas in steepest ascent hill climbing all successors are compared and the closest to the solution is chosen. Both forms fail if there is no closer node, which may happen if there are local maxima in the space which are not solutions. Steepest ascent hill climbing is similar to best-first search, which all possible extensions of the current path instead of only one. Stochastic hill climbing does not examine all neighbors before deciding how to move, rather, it selects a neighbor at random, and decides whether to move to that neighbor or to examine another
14.
Travelling salesman problem
–
It is an NP-hard problem in combinatorial optimization, important in operations research and theoretical computer science. TSP is a case of the travelling purchaser problem and the vehicle routing problem. In the theory of computational complexity, the version of the TSP belongs to the class of NP-complete problems. Thus, it is possible that the running time for any algorithm for the TSP increases superpolynomially with the number of cities. The problem was first formulated in 1930 and is one of the most intensively studied problems in optimization and it is used as a benchmark for many optimization methods. The TSP has several applications even in its purest formulation, such as planning, logistics, slightly modified, it appears as a sub-problem in many areas, such as DNA sequencing. The TSP also appears in astronomy, as observing many sources will want to minimize the time spent moving the telescope between the sources. In many applications, additional constraints such as limited resources or time windows may be imposed, the origins of the travelling salesperson problem are unclear. A handbook for travelling salesmen from 1832 mentions the problem and includes example tours through Germany and Switzerland, the travelling salesperson problem was mathematically formulated in the 1800s by the Irish mathematician W. R. Hamilton and by the British mathematician Thomas Kirkman. Hamilton’s Icosian Game was a puzzle based on finding a Hamiltonian cycle. Hassler Whitney at Princeton University introduced the name travelling salesman problem soon after, Dantzig, Fulkerson and Johnson, however, speculated that given a near optimal solution we may be able to find optimality or prove optimality by adding a small amount of extra inequalities. They used this idea to solve their initial 49 city problem using a string model and they found they only needed 26 cuts to come to a solution for their 49 city problem. As well as cutting plane methods, Dantzig, Fulkerson and Johnson used branch, in the following decades, the problem was studied by many researchers from mathematics, computer science, chemistry, physics, and other sciences. Christofides made a big advance in this approach of giving an approach for which we know the worst-case scenario and his algorithm given in 1976, at worst is 1.5 times longer than the optimal solution. As the algorithm was so simple and quick, many hoped it would give way to a optimal solution method. However, until 2011 when it was beaten by less than a billionth of a percent, Richard M. Karp showed in 1972 that the Hamiltonian cycle problem was NP-complete, which implies the NP-hardness of TSP. This supplied a mathematical explanation for the apparent computational difficulty of finding optimal tours, great progress was made in the late 1970s and 1980, when Grötschel, Padberg, Rinaldi and others managed to exactly solve instances with up to 2392 cities, using cutting planes and branch-and-bound. In the 1990s, Applegate, Bixby, Chvátal, and Cook developed the program Concorde that has used in many recent record solutions
15.
Probability
–
Probability is the measure of the likelihood that an event will occur. Probability is quantified as a number between 0 and 1, the higher the probability of an event, the more certain that the event will occur. A simple example is the tossing of a fair coin, since the coin is unbiased, the two outcomes are both equally probable, the probability of head equals the probability of tail. Since no other outcomes are possible, the probability is 1/2 and this type of probability is also called a priori probability. Probability theory is used to describe the underlying mechanics and regularities of complex systems. For example, tossing a coin twice will yield head-head, head-tail, tail-head. The probability of getting an outcome of head-head is 1 out of 4 outcomes or 1/4 or 0.25 and this interpretation considers probability to be the relative frequency in the long run of outcomes. A modification of this is propensity probability, which interprets probability as the tendency of some experiment to yield a certain outcome, subjectivists assign numbers per subjective probability, i. e. as a degree of belief. The degree of belief has been interpreted as, the price at which you would buy or sell a bet that pays 1 unit of utility if E,0 if not E. The most popular version of subjective probability is Bayesian probability, which includes expert knowledge as well as data to produce probabilities. The expert knowledge is represented by some prior probability distribution and these data are incorporated in a likelihood function. The product of the prior and the likelihood, normalized, results in a probability distribution that incorporates all the information known to date. The scientific study of probability is a development of mathematics. Gambling shows that there has been an interest in quantifying the ideas of probability for millennia, there are reasons of course, for the slow development of the mathematics of probability. Whereas games of chance provided the impetus for the study of probability. According to Richard Jeffrey, Before the middle of the century, the term probable meant approvable. A probable action or opinion was one such as people would undertake or hold. However, in legal contexts especially, probable could also apply to propositions for which there was good evidence, the sixteenth century Italian polymath Gerolamo Cardano demonstrated the efficacy of defining odds as the ratio of favourable to unfavourable outcomes
16.
Permutation
–
These differ from combinations, which are selections of some members of a set where order is disregarded. For example, written as tuples, there are six permutations of the set, namely and these are all the possible orderings of this three element set. As another example, an anagram of a word, all of whose letters are different, is a permutation of its letters, in this example, the letters are already ordered in the original word and the anagram is a reordering of the letters. The study of permutations of finite sets is a topic in the field of combinatorics, Permutations occur, in more or less prominent ways, in almost every area of mathematics. For similar reasons permutations arise in the study of sorting algorithms in computer science, the number of permutations of n distinct objects is n factorial, usually written as n. which means the product of all positive integers less than or equal to n. In algebra and particularly in group theory, a permutation of a set S is defined as a bijection from S to itself and that is, it is a function from S to S for which every element occurs exactly once as an image value. This is related to the rearrangement of the elements of S in which each element s is replaced by the corresponding f, the collection of such permutations form a group called the symmetric group of S. The key to this structure is the fact that the composition of two permutations results in another rearrangement. Permutations may act on structured objects by rearranging their components, or by certain replacements of symbols, in elementary combinatorics, the k-permutations, or partial permutations, are the ordered arrangements of k distinct elements selected from a set. When k is equal to the size of the set, these are the permutations of the set, fabian Stedman in 1677 described factorials when explaining the number of permutations of bells in change ringing. Starting from two bells, first, two must be admitted to be varied in two ways which he illustrates by showing 12 and 21 and he then explains that with three bells there are three times two figures to be produced out of three which again is illustrated. His explanation involves cast away 3, and 1.2 will remain, cast away 2, and 1.3 will remain, cast away 1, and 2.3 will remain. He then moves on to four bells and repeats the casting away argument showing that there will be four different sets of three, effectively this is an recursive process. He continues with five bells using the casting method and tabulates the resulting 120 combinations. At this point he gives up and remarks, Now the nature of these methods is such, in modern mathematics there are many similar situations in which understanding a problem requires studying certain permutations related to it. There are two equivalent common ways of regarding permutations, sometimes called the active and passive forms, or in older terminology substitutions and permutations, which form is preferable depends on the type of questions being asked in a given discipline. The active way to regard permutations of a set S is to them as the bijections from S to itself. Thus, the permutations are thought of as functions which can be composed with each other, forming groups of permutations
17.
Local optimum
–
In applied mathematics and computer science, a local optimum of an optimization problem is a solution that is optimal within a neighboring set of candidate solutions. This is in contrast to an optimum, which is the optimal solution among all possible solutions. When the function to be optimized is continuous, it may be possible to employ calculus to find local optima. If the first derivative exists everywhere, it can be equated to zero, if the function has an unbounded domain, then the second derivative test provides a sufficient condition for the point to be a local maximum or local minimum. Local search or hill climbing methods for solving optimization problems start from an initial configuration, a trajectory in search space is generated, which maps an initial point to a local optimum, where local search is stuck. The search space is subdivided into basins of attraction, each consisting of all initial points which have a given local optimum as the final point of the local search trajectory. A local optimum can be isolated or part of a plateau, the locality of the optimum is dependent on the neighborhood structure as defined by the local search method that is used for optimizing the function
18.
State transition table
–
A state table is essentially a truth table in which some of the inputs are the current state, and the outputs include the next state, along with other outputs. A state table is one of many ways to specify a state machine, other ways being a state diagram, also called characteristic tables, single-dimension state tables are much more like truth tables than the two-dimensional versions. Inputs are usually placed on the left, and separated from the outputs, the outputs will represent the next state of the machine. A simple example of a machine with two states and two combinational inputs follows, S1 and S2 would most likely represent the single bits 0 and 1. State transition tables are typically two-dimensional tables, there are two common forms for arranging them. One of the dimensions indicates current states, while the other indicates events, the row/column intersections indicate the next state for a particular event, and an incidental action associated with this state transition. One of the dimensions indicates current states, while the other indicates next states, the row/column intersections indicate the event which will lead to a particular next state. Simultaneous transitions in multiple finite state machines can be shown in what is effectively an n-dimensional state transition table in which pairs of rows map current states to next states and this is an alternative to representing communication between separate, interdependent state machines. An example of a state table for a machine M together with the corresponding state diagram is given below. All the possible inputs to the machine are enumerated across the columns of the table, all the possible states are enumerated across the rows. From the state transition table given above, it is easy to see if the machine is in S1, and the next input is character 1. If a character 0 arrives, the machine will transition to S2 as can be seen from the second column, in the diagram this is denoted by the arrow from S1 to S2 labeled with a 0. For a nondeterministic finite automaton, a new input may cause the machine to be in more than one state and this is denoted in a state transition table by a pair of curly braces with the set of all target states between them. Here, a machine in the state S1 reading an input of 0 will cause it to be in two states at the same time, the states S2 and S3. The last column defines the transition of states of the special character. This special character allows the NFA to move to a different state when given no input, in state S3, the NFA may move to S1 without consuming an input character. The two cases above make the finite automaton described non-deterministic and it is possible to draw a state diagram from the table. A sequence of easy to follow steps is given below, Draw the circles to represent the states given, for each of the states, scan across the corresponding row and draw an arrow to the destination state
19.
Greedy algorithm
–
A greedy algorithm is an algorithmic paradigm that follows the problem solving heuristic of making the locally optimal choice at each stage with the hope of finding a global optimum. For example, a strategy for the traveling salesman problem is the following heuristic. This heuristic need not find a best solution, but terminates in a number of steps. In mathematical optimization, greedy algorithms solve combinatorial problems having the properties of matroids, most problems for which they work will have two properties, Greedy choice property We can make whatever choice seems best at the moment and then solve the subproblems that arise later. The choice made by an algorithm may depend on choices made so far. It iteratively makes one greedy choice after another, reducing each given problem into a smaller one, in other words, a greedy algorithm never reconsiders its choices. This is the difference from dynamic programming, which is exhaustive and is guaranteed to find the solution. After every stage, dynamic programming makes decisions based on all the decisions made in the previous stage, optimal substructure A problem exhibits optimal substructure if an optimal solution to the problem contains optimal solutions to the sub-problems. For many other problems, greedy algorithms fail to produce the optimal solution, Greedy algorithms can be characterized as being short sighted, and also as non-recoverable. They are ideal only for problems which have optimal substructure, despite this, for many simple problems, the best suited algorithms are greedy algorithms. It is important, however, to note that the algorithm can be used as a selection algorithm to prioritize options within a search. They can make commitments to certain choices too early which prevent them from finding the best overall solution later, for example, all known greedy coloring algorithms for the graph coloring problem and all other NP-complete problems do not consistently find optimum solutions. Nevertheless, they are useful because they are quick to think up, examples of such greedy algorithms are Kruskals algorithm and Prims algorithm for finding minimum spanning trees, and the algorithm for finding optimum Huffman trees. The theory of matroids, and the general theory of greedoids. Greedy algorithms appear in network routing as well, using greedy routing, a message is forwarded to the neighboring node which is closest to the destination. The notion of a location may be determined by its physical location. Location may also be an artificial construct as in small world routing. The activity selection problem is characteristic to this class of problems, in the Macintosh computer game Crystal Quest the objective is to collect crystals, in a fashion similar to the travelling salesman problem
20.
Pixel
–
The address of a pixel corresponds to its physical coordinates. LCD pixels are manufactured in a grid, and are often represented using dots or squares. Each pixel is a sample of an image, more samples typically provide more accurate representations of the original. The intensity of each pixel is variable, in color imaging systems, a color is typically represented by three or four component intensities such as red, green, and blue, or cyan, magenta, yellow, and black. The word pixel is based on a contraction of pix and el, the word pixel was first published in 1965 by Frederic C. Billingsley of JPL, to describe the elements of video images from space probes to the Moon. Billingsley had learned the word from Keith E. McFarland, at the Link Division of General Precision in Palo Alto, McFarland said simply it was in use at the time. The word is a combination of pix, for picture, the word pix appeared in Variety magazine headlines in 1932, as an abbreviation for the word pictures, in reference to movies. By 1938, pix was being used in reference to pictures by photojournalists. The concept of a picture element dates to the earliest days of television, some authors explain pixel as picture cell, as early as 1972. In graphics and in image and video processing, pel is often used instead of pixel, for example, IBM used it in their Technical Reference for the original PC. A pixel is generally thought of as the smallest single component of a digital image, however, the definition is highly context-sensitive. For example, there can be printed pixels in a page, or pixels carried by electronic signals, or represented by digital values, or pixels on a display device, or pixels in a digital camera. This list is not exhaustive and, depending on context, synonyms include pel, sample, byte, bit, dot, Pixels can be used as a unit of measure such as,2400 pixels per inch,640 pixels per line, or spaced 10 pixels apart. For example, a high-quality photographic image may be printed with 600 ppi on a 1200 dpi inkjet printer, even higher dpi numbers, such as the 4800 dpi quoted by printer manufacturers since 2002, do not mean much in terms of achievable resolution. The more pixels used to represent an image, the closer the result can resemble the original, the number of pixels in an image is sometimes called the resolution, though resolution has a more specific definition.3 megapixels. The pixels, or color samples, that form an image may or may not be in one-to-one correspondence with screen pixels. In computing, a composed of pixels is known as a bitmapped image or a raster image
21.
Potential energy
–
In physics, potential energy is energy possessed by a body by virtue of its position relative to others, stresses within itself, electric charge, and other factors. The unit for energy in the International System of Units is the joule, the term potential energy was introduced by the 19th century Scottish engineer and physicist William Rankine, although it has links to Greek philosopher Aristotles concept of potentiality. Potential energy is associated with forces that act on a body in a way that the work done by these forces on the body depends only on the initial and final positions of the body in space. These forces, that are called potential forces, can be represented at every point in space by vectors expressed as gradients of a scalar function called potential. Potential energy is the energy of an object. It is the energy by virtue of a position relative to other objects. Potential energy is associated with restoring forces such as a spring or the force of gravity. The action of stretching the spring or lifting the mass is performed by a force that works against the force field of the potential. This work is stored in the field, which is said to be stored as potential energy. If the external force is removed the field acts on the body to perform the work as it moves the body back to the initial position. Suppose a ball which mass is m, and it is in h position in height, if the acceleration of free fall is g, the weight of the ball is mg. There are various types of energy, each associated with a particular type of force. Chemical potential energy, such as the energy stored in fossil fuels, is the work of the Coulomb force during rearrangement of mutual positions of electrons and nuclei in atoms and molecules. Thermal energy usually has two components, the energy of random motions of particles and the potential energy of their mutual positions. Forces derivable from a potential are also called conservative forces, the work done by a conservative force is W = − Δ U where Δ U is the change in the potential energy associated with the force. The negative sign provides the convention that work done against a force field increases potential energy, common notations for potential energy are U, V, also Ep. Potential energy is closely linked with forces, in this case, the force can be defined as the negative of the vector gradient of the potential field. If the work for a force is independent of the path, then the work done by the force is evaluated at the start
22.
Color
–
Color or colour is the characteristic of human visual perception described through color categories, with names such as red, yellow, purple, or blue. This perception of color derives from the stimulation of cells in the human eye by electromagnetic radiation in the spectrum of light. Color categories and physical specifications of color are associated with objects through the wavelength of the light that is reflected from them and this reflection is governed by the objects physical properties such as light absorption, emission spectra, etc. By defining a color space, colors can be identified numerically by coordinates, there may also be more than three color dimensions in other color spaces, such as in the CMYK color model, wherein one of the dimensions relates to a colours colorfulness). The photo-receptivity of the eyes of species also varies considerably from our own. Honeybees and bumblebees for instance have trichromatic color vision sensitive to ultraviolet but is insensitive to red, papilio butterflies possess six types of photoreceptors and may have pentachromatic vision. The most complex color vision system in the kingdom has been found in stomatopods with up to 12 spectral receptor types thought to work as multiple dichromatic units. The science of color is sometimes called chromatics, colorimetry, or simply color science and it includes the perception of color by the human eye and brain, the origin of color in materials, color theory in art, and the physics of electromagnetic radiation in the visible range. Electromagnetic radiation is characterized by its wavelength and its intensity, when the wavelength is within the visible spectrum, it is known as visible light. Most light sources emit light at different wavelengths, a sources spectrum is a distribution giving its intensity at each wavelength. Although the spectrum of light arriving at the eye from a given direction determines the color sensation in that direction, in each such class the members are called metamers of the color in question. The table at right shows approximate frequencies and wavelengths for various pure spectral colors, the wavelengths listed are as measured in air or vacuum. A common list identifies six main bands, red, orange, yellow, green, blue, Newtons conception included a seventh color, indigo, between blue and violet. It is possible that what Newton referred to as blue is nearer to what today is known as cyan, the color of an object depends on both the physics of the object in its environment and the characteristics of the perceiving eye and brain. Some objects not only light, but also transmit light or emit light themselves. This effect is known as color constancy, opaque objects that do not reflect specularly have their color determined by which wavelengths of light they scatter strongly. If objects scatter all wavelengths with roughly equal strength, they appear white, if they absorb all wavelengths, they appear black. Opaque objects that reflect light of different wavelengths with different efficiencies look like mirrors tinted with colors determined by those differences
23.
Amorphous solid
–
In condensed matter physics and materials science, an amorphous or non-crystalline solid is a solid that lacks the long-range order that is characteristic of a crystal. In some older books, the term has been used synonymously with glass, nowadays, glassy solid or amorphous solid is considered to be the overarching concept, and glass the more special case, A glass is an amorphous solid that exhibits a glass transition. Other types of solids include gels, thin films. Amorphous materials have a structure made of interconnected structural blocks. Even amorphous materials have some shortrange order at the length scale due to the nature of chemical bonding. Furthermore, in small crystals a large fraction of the atoms are the crystal, relaxation of the surface and interfacial effects distort the atomic positions. Amorphous phases are important constituents of thin films, which are layers of a few nm to some tens of µm thickness deposited upon a substrate. For higher values, the diffusion of deposited atomic species would allow for the formation of crystallites with long range atomic order. Regarding their applications, amorphous metallic layers played an important role in the discussion of a suspected superconductivity in amorphous metals, today, optical coatings made from TiO2, SiO2, Ta2O5 etc. and combinations of them in most cases consist of amorphous phases of these compounds. Much research is carried out into thin films as a gas separating membrane layer. Also, hydrogenated amorphous silicon, a-Si, H in short, is of significance for thin film solar cells. In case of a-Si, H the missing long-range order between silicon atoms is partly induced by the presence by hydrogen in the percent range, the occurrence of amorphous phases turned out as a phenomenon of particular interest for studying thin film growth. Remarkably, the growth of polycrystalline films is used and preceded by an initial amorphous layer. The most investigated example is represented by thin multicrystalline silicon films, an initial amorphous layer was observed in many studies. The phenomenon has been interpreted in the framework of Ostwalds rule of stages that predicts the formation of phases to proceed with increasing condensation time towards increasing stability. Experimental studies of the phenomenon require a clearly defined state of the substrate surface, the Physics of Structurally Disordered Matter, An Introduction. Amorphous Solids and the Liquid State
24.
Crystalline solid
–
A crystal or crystalline solid is a solid material whose constituents are arranged in a highly ordered microscopic structure, forming a crystal lattice that extends in all directions. In addition, macroscopic single crystals are usually identifiable by their geometrical shape, the scientific study of crystals and crystal formation is known as crystallography. The process of crystal formation via mechanisms of crystal growth is called crystallization or solidification, the word crystal derives from the Ancient Greek word κρύσταλλος, meaning both ice and rock crystal, from κρύος, icy cold, frost. Examples of large crystals include snowflakes, diamonds, and table salt, most inorganic solids are not crystals but polycrystals, i. e. many microscopic crystals fused together into a single solid. Examples of polycrystals include most metals, rocks, ceramics, a third category of solids is amorphous solids, where the atoms have no periodic structure whatsoever. Examples of amorphous solids include glass, wax, and many plastics, Crystals are often used in pseudoscientific practices such as crystal therapy, and, along with gemstones, are sometimes associated with spellwork in Wiccan beliefs and related religious movements. The scientific definition of a crystal is based on the arrangement of atoms inside it. A crystal is a solid where the form a periodic arrangement. For example, when liquid water starts freezing, the change begins with small ice crystals that grow until they fuse. Most macroscopic inorganic solids are polycrystalline, including almost all metals, ceramics, ice, rocks, solids that are neither crystalline nor polycrystalline, such as glass, are called amorphous solids, also called glassy, vitreous, or noncrystalline. These have no periodic order, even microscopically, there are distinct differences between crystalline solids and amorphous solids, most notably, the process of forming a glass does not release the latent heat of fusion, but forming a crystal does. A crystal structure is characterized by its cell, a small imaginary box containing one or more atoms in a specific spatial arrangement. The unit cells are stacked in three-dimensional space to form the crystal, the symmetry of a crystal is constrained by the requirement that the unit cells stack perfectly with no gaps. There are 219 possible crystal symmetries, called space groups. These are grouped into 7 crystal systems, such as cubic crystal system or hexagonal crystal system, Crystals are commonly recognized by their shape, consisting of flat faces with sharp angles. Euhedral crystals are those with obvious, well-formed flat faces, anhedral crystals do not, usually because the crystal is one grain in a polycrystalline solid. The flat faces of a crystal are oriented in a specific way relative to the underlying atomic arrangement of the crystal. This occurs because some surface orientations are more stable than others, as a crystal grows, new atoms attach easily to the rougher and less stable parts of the surface, but less easily to the flat, stable surfaces
25.
Feasible region
–
This is the initial set of candidate solutions to the problem, before the set of candidates has been narrowed down. For example, consider the problem Minimize x 2 + y 4 with respect to the x and y. Here the feasible set is the set of pairs in which the value of x is at least 1 and at most 10 and the value of y is at least 5 and at most 12. Note that the set of the problem is separate from the objective function, which states the criterion to be optimized. In many problems, the set reflects a constraint that one or more variables must be non-negative. In pure integer programming problems, the set is the set of integers. In linear programming problems, the set is a convex polytope. Constraint satisfaction is the process of finding a point in the feasible region, a convex feasible set is one in which a line segment connecting any two feasible points goes through only other feasible points, and not through any points outside the feasible set. If the constraints of a problem are mutually contradictory, there are no points that satisfy all the constraints. In this case the problem has no solution and is said to be infeasible, feasible sets may be bounded or unbounded. For example, the set defined by the constraint set is unbounded because in some directions there is no limit on how far one can go. In contrast, the set formed by the constraint set is bounded because the extent of movement in any direction is limited by the constraints. In linear programming problems with n variables, a necessary but not sufficient condition for the set to be bounded is that the number of constraints be at least n +1. If the feasible set is unbounded, there may or may not be an optimum, in optimization and other branches of mathematics, and in search algorithms, a candidate solution is a member of the set of possible solutions in the feasible region of a given problem. A candidate solution does not have to be a likely or reasonable solution to the problem—it is simply in the set that satisfies all constraints, that is, it is in the set of feasible solutions. The space of all solutions, before any feasible points have been excluded, is called the feasible region, feasible set, search space. This is the set of all possible solutions that satisfy the problems constraints, constraint satisfaction is the process of finding a point in the feasible set. In the case of the algorithm, the candidate solutions are the individuals in the population being evolved by the algorithm
26.
Uniform distribution (continuous)
–
The support is defined by the two parameters, a and b, which are its minimum and maximum values. The distribution is often abbreviated U and it is the maximum entropy probability distribution for a random variate X under no constraint other than that it is contained in the distributions support. Sometimes they are chosen to be zero, and sometimes chosen to be 1/, the latter is appropriate in the context of estimation by the method of maximum likelihood. e. Except on a set of points with zero measure, also, it is consistent with the sign function which has no such ambiguity. For a random variable following this distribution, the value is then m1 = /2. For n ≥2, the nth cumulant of the distribution on the interval is bn/n. The mean of the distribution is, E =12, the variance is, V =1122 Let X1. Let X be the kth order statistic from this sample, then the probability distribution of X is a Beta distribution with parameters k and n − k +1. The expected value is E = k n +1 and this fact is useful when making Q–Q plots. The variances are V = k 2, to see this, if X ~ U and is a subinterval of with fixed d >0, then P = ∫ x x + d d y b − a = d b − a which is independent of x. This fact motivates the distributions name and this distribution can be generalized to more complicated sets than intervals. Restricting a =0 and b =1, the resulting distribution U is called a uniform distribution. One interesting property of the uniform distribution is that if u1 has a standard uniform distribution. This property can be used for generating antithetic variates, among other things, If X has a standard uniform distribution, then by the inverse transform sampling method, Y = − λ−1 ln has an exponential distribution with parameter λ. If X has a uniform distribution, then Y = Xn has a beta distribution with parameters. As such, If X has a uniform distribution, then Y = X is also a special case of the beta distribution with parameters. The Irwin–Hall distribution is the sum of n i. i. d, the sum of two independent, equally distributed, uniform distributions yields a symmetric triangular distribution. Uniform random variables also has a distribution, although not symmetric
27.
Drainage basin
–
A drainage basin or catchment area is any area of land where precipitation collects and drains off into a common outlet, such as into a river, bay, or other body of water. Drainage basins connect into other drainage basins at elevations in a hierarchical pattern, with smaller sub-drainage basins. Other terms used to describe drainage basins are catchment, catchment basin, drainage area, river basin and water basin. In closed drainage basins the water converges to a point inside the basin, known as a sink, which may be a permanent lake. The drainage basin acts as a funnel by collecting all the water within the covered by the basin. Each drainage basin is separated topographically from adjacent basins by a perimeter, drainage basins are similar but not identical to hydrologic units, which are drainage areas delineated so as to nest into a multi-level hierarchical drainage system. Hydrologic units are defined to allow multiple inlets, outlets, or sinks, in a strict sense, all drainage basins are hydrologic units but not all hydrologic units are drainage basins. Drainage basins of the oceans and seas of the world. Grey areas are endorheic basins that do not drain to the oceans, the following is a list of the major ocean basins, About 48. 7% of the worlds land drains to the Atlantic Ocean. The two major mediterranean seas of the world also flow to the Atlantic, The Caribbean Sea and Gulf of Mexico basin includes most of the U. S. The Mediterranean Sea basin includes much of North Africa, east-central Africa, Southern, Central, and Eastern Europe, Turkey, and the areas of Israel, Lebanon. Just over 13% of the land in the world drains to the Pacific Ocean, the Indian Oceans drainage basin also comprises about 13% of Earths land. It drains the eastern coast of Africa, the coasts of the Red Sea and the Persian Gulf, the Indian subcontinent, Burma, antarctica comprises approximately eight percent of the Earths land. The five largest river basins, from largest to smallest, are the basins of the Amazon, the Río de la Plata, the Congo, the Nile, and the Mississippi. The three rivers that drain the most water, from most to least, are the Amazon, Ganga, endorheic drainage basins are inland basins that do not drain to an ocean. Around 18% of all land drains to endorheic lakes or seas or sinks, the largest of these consists of much of the interior of Asia, which drains into the Caspian Sea, the Aral Sea, and numerous smaller lakes. Some of these, such as the Great Basin, are not single drainage basins but collections of separate, in endorheic bodies of standing water where evaporation is the primary means of water loss, the water is typically more saline than the oceans. An extreme example of this is the Dead Sea, drainage basins have been historically important for determining territorial boundaries, particularly in regions where trade by water has been important
28.
Quantum annealing
–
Quantum annealing is a metaheuristic for finding the global minimum of a given objective function over a given set of candidate solutions, by a process using quantum fluctuations. Quantum annealing is used mainly for problems where the space is discrete with many local minima. It was formulated in its present form by T. Kadowaki, sebenik and J. D. Doll, in Quantum annealing, A new method for minimizing multidimensional functions. Quantum annealing starts from a superposition of all possible states with equal weights. Then the system following the time-dependent Schrödinger equation, a natural quantum-mechanical evolution of physical systems. The amplitudes of all candidate states keep changing, realizing a quantum parallelism, according to the time-dependent strength of the transverse field, which causes quantum tunneling between states. If the rate of change of the transverse-field is slow enough, an experimental demonstration of the success of quantum annealing for random magnets was reported immediately after the initial theoretical proposal. Quantum annealing can be compared to simulated annealing, whose temperature parameter plays a role to QAs tunneling field strength. In simulated annealing, the temperature determines the probability of moving to a state of energy from a single current state. In quantum annealing, the strength of transverse field determines the probability to change the amplitudes of all states in parallel. Analytical and numerical evidence suggests that quantum annealing outperforms simulated annealing under certain conditions, the tunneling field is basically a kinetic energy term that does not commute with the classical potential energy part of the original glass. The whole process can be simulated in a computer using quantum Monte Carlo, then a suitable term consisting of non-commuting variable has to be introduced artificially in the Hamiltonian to play the role of the tunneling field. Then one may carry out the simulation with the quantum Hamiltonian thus constructed just as described above, here, there is a choice in selecting the non-commuting term and the efficiency of annealing may depend on that. If the barriers are thin enough, quantum fluctuations can surely bring the system out of the shallow local minima. For N -spin glasses, Δ is proportional to N, and with a linear annealing schedule for the transverse field and this O advantage in quantum search is well established. Moreover, it may be able to do this without the tight error controls needed to harness the quantum entanglement used in traditional quantum algorithms. In 2011, D-Wave Systems announced the first commercial quantum annealer on the market by the name D-Wave One, the company claims this system uses a 128 qubit processor chipset. On May 25,2011 D-Wave announced that Lockheed Martin Corporation entered into an agreement to purchase a D-Wave One system, on October 28,2011 USCs Information Sciences Institute took delivery of Lockheeds D-Wave One
29.
Stochastic tunneling
–
Easier tunneling allows for faster exploration of sample space and faster convergence to a good solution. Monte Carlo method-based optimization techniques sample the function by randomly hopping from the current solution vector to another with a difference in the function value of Δ E. The acceptance probability of such a jump is in most cases chosen to be min with an appropriate parameter β. The general idea of STUN is to circumvent the slow dynamics of ill-shaped energy functions that one encounters for example in spin glasses by tunneling through such barriers and this goal is achieved by Monte Carlo sampling of a transformed function that lacks this slow dynamics. In the standard-form the transformation reads f S T U N, =1 − exp where E o is the lowest function value found so far and this transformation preserves the loci of the minima. F S T U N is then used in place of E in the original algorithm giving a new probability of min The effect of such a transformation is shown in the graph. A variation on always tunneling is to do so only when trapped at a local minimum, γ is then adjusted to tunnel out of the minimum and peruse a more globally optimum solution. Detrended fluctuation analysis is the way of determining if trapped at a local minimum. Simulated annealing Parallel tempering Genetic algorithm Differential evolution K. Hamacher, adaptation in Stochastic Tunneling Global Optimization of Complex Potential Energy Landscapes. The Scaling Behaviour of Stochastic Minimization Algorithms in a Perfect Funnel Landscape, a Stochastic tunneling approach for global minimization. Nicholas Metropolis, Arianna W. Rosenbluth, Marshall N. Rosenbluth, equation of State Calculations by Fast Computing Machines. CS1 maint, Multiple names, authors list Mingjie Lin, improving FPGA Placement with Dynamically Adaptive Stochastic Tunneling. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
30.
Tabu search
–
Tabu search, created by Fred W. Glover in 1986 and formalized in 1989, is a metaheuristic search method employing local search methods used for mathematical optimization. Local searches take a potential solution to a problem and check its immediate neighbors in the hope of finding an improved solution, local search methods have a tendency to become stuck in suboptimal regions or on plateaus where many solutions are equally fit. Tabu search enhances the performance of local search by relaxing its basic rule, first, at each step worsening moves can be accepted if no improving move is available. In addition, prohibitions are introduced to discourage the search from coming back to previously-visited solutions, the implementation of tabu search uses memory structures that describe the visited solutions or user-provided sets of rules. The word tabu comes from Tongan, a language of Polynesia, Tabu search is a metaheuristic algorithm that can be used for solving combinatorial optimization problems. Local search procedures often become stuck in poor-scoring areas or areas where scores plateau, the solutions admitted to the new neighborhood, N ∗, are determined through the use of memory structures. Using these memory structures, the search progresses by iteratively moving from the current solution x to a solution x ′ in N ∗. These memory structures form what is known as the tabu list, in its simplest form, a tabu list is a short-term set of the solutions that have been visited in the recent past. More commonly, a tabu list consists of solutions that have changed by the process of moving from one solution to another and it is convenient, for ease of description, to understand a “solution” to be coded and represented by such attributes. The memory structures used in tabu search can roughly be divided into three categories, Short-term, The list of solutions recently considered, if a potential solution appears on the tabu list, it cannot be revisited until it reaches an expiration point. Intermediate-term, Intensification rules intended to bias the search towards promising areas of the search space, long-term, Diversification rules that drive the search into new regions. Short-term, intermediate-term and long-term memories can overlap in practice, within these categories, memory can further be differentiated by measures such as frequency and impact of changes made. One example of a memory structure is one that prohibits or encourages solutions that contain certain attributes or a memory structure that prevents or induces certain moves. In short-term memory, selected attributes in solutions recently visited are labeled tabu-active, solutions that contain tabu-active elements are banned. Aspiration criteria are employed that override a solutions tabu state, thereby including the otherwise-excluded solution in the allowed set, a simple and commonly used aspiration criterion is to allow solutions which are better than the currently-known best solution. In addition, tabu search is sometimes combined with other metaheuristics to create hybrid methods, the following pseudocode presents a simplified version of the tabu search algorithm as described above. This implementation has a rudimentary short-term memory, but contains no intermediate or long-term memory structures, the term fitness refers to an evaluation of the candidate solution, as embodied in an objective function for mathematical optimization. Lines 1-3 represent some initial setup, respectively creating a solution, setting that initial solution as the best seen to date
31.
Stochastic gradient descent
–
In other words, SGD tries to find minima or maxima by iteration. Each summand function Q i is associated with the i -th observation in the data set. In classical statistics, sum-minimization problems arise in least squares and in maximum-likelihood estimation, the general class of estimators that arise as minimizers of sums are called M-estimators. However, in statistics, it has long recognized that requiring even local minimization is too restrictive for some problems of maximum-likelihood estimation. Therefore, contemporary statistical theorists often consider stationary points of the likelihood function, the sum-minimization problem also arises for empirical risk minimization, In this case, Q i is the value of the loss function at i -th example, and Q is the empirical risk. In many cases, the functions have a simple form that enables inexpensive evaluations of the sum-function. For example, in statistics, one-parameter exponential families allow economical function-evaluations and gradient-evaluations, however, in other cases, evaluating the sum-gradient may require expensive evaluations of the gradients from all summand functions. To economize on the computational cost at every iteration, stochastic gradient descent samples a subset of functions at every step. This is very effective in the case of machine learning problems. In stochastic gradient descent, the gradient of Q is approximated by a gradient at a single example. As the algorithm sweeps through the set, it performs the above update for each training example. Several passes can be made over the set until the algorithm converges. If this is done, the data can be shuffled for each pass to prevent cycles, typical implementations may use an adaptive learning rate so that the algorithm converges. This can perform better than true stochastic gradient descent because the code can make use of vectorization libraries rather than computing each step separately. It may also result in smoother convergence, as the gradient computed at each step uses more training examples, the convergence of stochastic gradient descent has been analyzed using the theories of convex minimization and of stochastic approximation. This is in fact a consequence of the Robbins-Siegmund theorem, lets suppose we want to fit a straight line y = w 1 + w 2 x to a training set of two-dimensional points, …, using least squares. The objective function to be minimized is, Q = ∑ i =1 n Q i = ∑ i =1 n 2, the last line in the above pseudocode for this specific problem will become, = − η. Stochastic gradient descent is an algorithm for training a wide range of models in machine learning, including support vector machines, logistic regression
32.
Genetic algorithm
–
In computer science and operations research, a genetic algorithm is a metaheuristic inspired by the process of natural selection that belongs to the larger class of evolutionary algorithms. Genetic algorithms are used to generate high-quality solutions to optimization and search problems by relying on bio-inspired operators such as mutation, crossover. In a genetic algorithm, a population of solutions to an optimization problem is evolved toward better solutions. Each candidate solution has a set of properties which can be mutated and altered, traditionally, solutions are represented in binary as strings of 0s and 1s, but other encodings are also possible. The evolution usually starts from a population of randomly generated individuals, in each generation, the fitness of every individual in the population is evaluated, the fitness is usually the value of the objective function in the optimization problem being solved. The more fit individuals are selected from the current population. The new generation of candidate solutions is used in the next iteration of the algorithm. Commonly, the algorithm terminates when either a number of generations has been produced. A typical genetic algorithm requires, a representation of the solution domain. A standard representation of each solution is as an array of bits. Arrays of other types and structures can be used in essentially the same way, the main property that makes these genetic representations convenient is that their parts are easily aligned due to their fixed size, which facilitates simple crossover operations. Variable length representations may also be used, but crossover implementation is complex in this case. The population size depends on the nature of the problem, often, the initial population is generated randomly, allowing the entire range of possible solutions. Occasionally, the solutions may be seeded in areas where optimal solutions are likely to be found, during each successive generation, a portion of the existing population is selected to breed a new generation. Individual solutions are selected through a process, where fitter solutions are typically more likely to be selected. Certain selection methods rate the fitness of each solution and preferentially select the best solutions, other methods rate only a random sample of the population, as the former process may be very time-consuming. The fitness function is defined over the representation and measures the quality of the represented solution. The fitness function is always problem dependent, for instance, in the knapsack problem one wants to maximize the total value of objects that can be put in a knapsack of some fixed capacity
33.
Graduated optimization
–
Graduated optimization is an improvement to hill climbing that enables a hill climber to avoid settling into local optima. Often, graduated optimization gives better results than simple hill climbing, further, when certain conditions exist, it can be shown to find an optimal solution to the final problem in the sequence. These conditions are, The first optimization problem in the sequence can be solved given the starting point. The locally convex region around the optimum of each problem in the sequence includes the point that corresponds to the global optimum of the previous problem in the sequence. It can be shown inductively that if conditions are met. Unfortunately, it can be difficult to find a sequence of problems that meet these conditions. Often, graduated optimization yields good results even when the sequence of problems cannot be proven to meet all of these conditions. Graduated optimization is commonly used in processing for locating objects within a larger image. This problem can be made to be convex by blurring the images. The proper choice of the blurring operator depends on the geometric transformation relating the object in one image to the other, graduated optimization can be used in manifold learning. The Manifold Sculpting algorithm, for example, uses graduated optimization to seek an embedding for non-linear dimensionality reduction. It gradually scales variance out of extra dimensions within a set while optimizing in the remaining dimensions. It has also used to calculate conditions for fractionation with tumors, for object tracking in computer vision. A thorough review of the method and its applications can be found in, simulated annealing is closely related to graduated optimization. Instead of smoothing the function over which it is optimizing, simulated annealing randomly perturbs the current solution by a decaying amount, because simulated annealing relies on random sampling to find improvements, however, its computation complexity is exponential in the number of dimensions being optimized. By contrast, graduated optimization smooths the function being optimized, so local optimization techniques that are efficient in high-dimensional space may still be used
34.
Ant colony optimization algorithms
–
This algorithm is a member of the ant colony algorithms family, in swarm intelligence methods, and it constitutes some metaheuristic optimizations. The original idea has since diversified to solve a wider class of numerical problems, from a broader perspective, ACO performs a model-based search and share some similarities with Estimation of Distribution Algorithms. In the natural world, ants of some species wander randomly, if other ants find such a path, they are likely not to keep travelling at random, but instead to follow the trail, returning and reinforcing it if they eventually find food. Over time, however, the trail starts to evaporate. The more time it takes for an ant to travel down the path and back again, a short path, by comparison, gets marched over more frequently, and thus the pheromone density becomes higher on shorter paths than longer ones. Pheromone evaporation also has the advantage of avoiding the convergence to a optimal solution. If there were no evaporation at all, the chosen by the first ants would tend to be excessively attractive to the following ones. In that case, the exploration of the space would be constrained. The influence of pheromone evaporation in real ant systems is unclear, the idea of the ant colony algorithm is to mimic this behavior with simulated ants walking around the graph representing the problem to solve. Here are some of the most popular variations of ACO algorithms, the global best solution deposits pheromone on every iteration along with all the other ants. Added maximum and minimum pheromone amounts, only global best or iteration best tour deposited pheromone <MAZ>. All edges are initialized to τmin and reinitialized to τmax when nearing stagnation, all solutions are ranked according to their length. The amount of pheromone deposited is then weighted for each solution, the pheromone deposit mechanism of COAC is to enable ants to search for solutions collaboratively and effectively. By using a design method, ants in the feasible domain can explore their chosen regions rapidly and efficiently, with enhanced global search capability. The orthogonal design method and the adaptive radius adjustment method can also be extended to other algorithms for delivering wider advantages in solving practical problems. It is a form of ant system which divides the whole search domain into several sub-domains. The results from all the subdomains are compared and the best few of them are promoted for the next level, the subdomains corresponding to the selected results are further subdivided and the process is repeated until an output of desired precision is obtained. This method has been tested on ill-posed geophysical inversion problems and works well, for some versions of the algorithm, it is possible to prove that it is convergent
35.
Particle swarm optimization
–
In computer science, particle swarm optimization is a computational method that optimizes a problem by iteratively trying to improve a candidate solution with regard to a given measure of quality. This is expected to move the swarm toward the best solutions, the algorithm was simplified and it was observed to be performing optimization. The book by Kennedy and Eberhart describes many aspects of PSO. An extensive survey of PSO applications is made by Poli, recently, a comprehensive review on theoretical and experimental works on PSO has been published by Bonyadi and Michalewicz. PSO is a metaheuristic as it makes few or no assumptions about the problem being optimized, however, metaheuristics such as PSO do not guarantee an optimal solution is ever found. A basic variant of the PSO algorithm works by having a population of candidate solutions and these particles are moved around in the search-space according to a few simple formulae. The movements of the particles are guided by their own best known position in the search-space as well as the entire swarms best known position, when improved positions are being discovered these will then come to guide the movements of the swarm. The process is repeated and by doing so it is hoped, but not guaranteed, formally, let f, ℝn → ℝ be the cost function which must be minimized. The gradient of f is not known, the goal is to find a solution a for which f ≤ f for all b in the search-space, which would mean a is the global minimum. Maximization can be performed by considering the function h = -f instead, let S be the number of particles in the swarm, each having a position xi ∈ ℝn in the search-space and a velocity vi ∈ ℝn. Let pi be the best known position of i and let g be the best known position of the entire swarm. A basic PSO algorithm is then, for particle i =1. S do for each dimension d =1, the termination criterion can be number of iterations performed, or a solution with adequate objective function value is found. The parameters ω, φp, and φg are selected by the practitioner and control the behaviour and efficacy of the PSO method, the choice of PSO parameters can have a large impact on optimization performance. Selecting PSO parameters that yield good performance has therefore been the subject of much research, the PSO parameters can also be tuned by using another overlaying optimizer, a concept known as meta-optimization, or even fine-tuned during the optimization, e. g. by means of fuzzy logic. Parameters have also been tuned for various optimization scenarios, the topology of the swarm defines the subset of particles with which each particle can exchange information. The basic version of the uses the global topology as the swarm communication structure. This topology allows all particles to communicate all the other particles
. doi:10.1111/j.1467-9868.2006.00553.x. Retrieved 2015-06-11.
