The Expanding Medium: The Future of Computer Art

Herbert W Franke

Translated by Ingrid Kurz

The term 'computer art' refers neither to a specific style nor to a particular quality, it merely characterizes the instrumentarium. The computer proves to be a universal data-processing system, the invention and development of which came to be linked with technological considerations solely for historical reasons. Under different circumstances, the computer might equally well have been invented as an instrument of art, e.g. in the socio-political environment of ancient Greece, where art as well as science and technology was considered a noble pastime.

In the most general sense of the term, art can be regarded as a special form of communication. It is the task of the artist to provide a message, which in this particular case is also subject to certain esthetic considerations, however they may be defined. Formerly, colors, sounds and tones were regarded as the raw material of art; today they would be considered information carriers. The elements of art are data, i.e. immaterial components. Even though this statement may sound rather sober, it does imply that art is not a material but rather an intellectual process.

The means of expression of an artist are adapted to the faculties of human perception. Complex patterns are perceptible only via the eye or the ear, thus making music, poetry and the visual arts in all their combinations and variations the most sophisticated areas of artistic activity.

It is interesting to note that the arts, despite highly different results, have one common origin in the imagination of the artists: chemo-electrical engrams in the artist's brain. It is only the choice of the specific means of expression - language, music, images - that leads to outwardly different results. The functioning of a computer suggests an interesting analogy:

during processing, the data are encoded in electronic impulses; it is up to the programmer to choose the output device - printer, sound generator, plotter.

Not only is this analogy remarkable from the viewpoint of history, it also entails practical consequences. The use of computers in art leads to a compatibility of the instrumentarium - to a closer link between the different art forms which, owing to the different classical methods and instruments, have been separated and taught in different institutions. It is one of the decisive aspects of the new situation brought about by the introduction of the computer that there is no longer a reason for dividing art into different forms, be they classical or modern.

Most art historians will probably agree that esthetic quality depends neither on style nor on the instrumentarium. What counts is the creativity and sensitivity of the artist and the form and content of the message presented to the public. These aspects will not be considered here. This article will focus only on the small segment that, in view of the new media (in particular the computer), is becoming increasingly important: the new possibilities of expression provided by the instrumentarium.

To use the language of computer science, the problem is related to that of encoding. It could be defined as follows: the given quantity is the artist's message that requires an appropriate means of expression (an optimal encoding system). Depending on what the artist wants to express, different ways of realization will offer themselves. Most likely, language will be better suited for expressing drama, music for expressing an abstract emotional mood and a picture for representing a visionary imagination. Thanks to the computer, which can be used in many different ways, artists are no longer bound to those means of expression that they were taught in the course of their conventional education; rather they are free to choose and even to combine different media. Although this article is mainly about visual arts, this aspect should not be neglected.

In contrast to classical art, which looks back on thousands of years of history, computer art is a young activity still in its early stages of development. It cannot be mature because its instrumentarium is not yet mature. The artist using a computer depends on the hardware and software accessible for technological, scientific and commercial uses. Fortunately, many solutions to graphic problems are designed so universally that they can also be used for esthetic applications.

The same holds true for the output devices the artist uses today. They, too, are borrowed from practical applications. Although computers and programs are extremely useful, the output devices currently available are inadequate. Images of all styles can be displayed equally well on a monitor, but this form of realization does not always correspond to the quality of conventional forms of artistic presentation. It can be expected that output devices specially designed for esthetic productions will be developed in the future. Even though it is not yet possible to describe this future instrumentarium in any great detail, its general outlines can be anticipated; the consequences arising therefrom seem fantastic in several respects. Nevertheless, it seems important to obtain an idea of this future instrumentarium as early as possible since, seen from a future perspective, the activities occurring in this field today can be regarded as paving the way for future forms of expression.

A statement on computer-assisted visual arts that is often meant as a criticism is true: basically every static image presented on the monitor could also have been created in a conventional manner, the only difference being the time needed to produce it. Therefore, the significance of computer graphics lies only in animation. This, too, is an important aspect. While conventionally produced images can hardly be put into motion, the computer lends itself almost naturally to this possibility. This is largely due to the possibility of constructing single frames within seconds, which is easy when high resolution is not required. By putting single frames together - e.g. 25 frames/sec as in movies - a kinetic sequence is obtained. This is made all the easier, since the majority of computer-generated images are made up of variable parameters. All that is needed to put a static image into motion is a slow change of these parameters.

Another important step towards new ways of creating images is three dimensional simulation, i.e. three-dimensional, photorealistic images in true perspective. This method originated from computer applications in engineering and architecture, where it is desirable to show objects in perspective. The development of methods to solve this fairly simple task was almost comparable to that taken by artists in the Middle Ages when they learned to use the technique of perspective and had to solve such problems as perspective transformations and overlapping (what is in front, what is in back) and the question of how to cope with shadows. With the knowledge currently available in these areas it is possible to fulfil the requirements for producing technical drawings, for which photorealistic images are not needed.

Nevertheless, computer-generated images showing objects and landscapes in a photorealistic manner have drawn considerable attention in recent years. This required extensive research work, which was undertaken partly by universities and partly by companies specializing in computer animation. Among the problems solved in the course of research was the correct distribution of light. Today, routine methods such as ray tracing are available; rays of light originating from a fictitious source of light are traced until after several reflections on the objects they arrive at the viewpoint of the spectator. This method is highly complex and time consuming. Other software routines relate to special surface characteristics; dull or shiny - even metallic - or with an embossed relief. Since transparent or reflecting objects cause particular problems, they frequently occur in images designed to demonstrate the high degree of quality that can be achieved. This technique produces top-quality images and is used mostly in television and movie productions, for at present, among interested parties, only the movie and advertising industries are financially strong enough to afford the use of the big computer systems required for photorealistic animated sequences today.

It is true that the images shown in movies and commercials are not art, just as the three-dimensional presentations produced by engineers or architects are not art either. But again, this reflects the universal character of computers. Accomplishments made in the commercial field may well be of future use for other applications. This includes the visualization of mathematical and scientific contexts for didactic purposes as well as the use of computers for purposes of esthetic design. While formerly it was the graphic software of scientists and engineers that was a pacesetter for art, this function is now increasingly being taken over by movies, television and advertising.

Again, the special character of the socio-political situation has had a decisive influence on developments in the fields of science, technology and art. In a capitalist society, material and immaterial goods are priced according to their utility value. This is why computer graphics and animation are developed with a view to commercial applications and not for artistic reasons. Whether one welcomes or regrets this fact, the artist is always free to use methods supplied from other sources.

Photorealistic images require extremely large and expensive computer systems, which are hardly ever available for artistic purposes. Concerning future developments in electronics that will become increasingly relevant for art, however, it is likely that the situation will change quickly. It will be possible further to increase computing speeds and storage capacities by several orders of magnitude. As has been the case so far prices for systems will continue to drop which means that the performance of today s supercomputers will soon be equaled by medium-sized computers or perhaps even by home computers. The realization that this change could well take place within one generation should have an impact on the training of artists and teachers of art.

The improvement of computer systems has consequences that go far beyond the general availability of digital animation methods. The resulting higher resolution is the basis for efforts to find alternatives to the monitor, e.g. large-screen projections. One step in this direction is the introduction of a high-resolution television standard anticipated for 1990. The size of an image is a decisive factor in terms of its realism. The trend towards a perfect simulation of reality is linked only marginally to the problems of art; it is related rather to problems of leisure time and entertainment. The history of film and stage techniques can be seen as a continuous effort to get closer to reality an 'illusion technology' that allows the spectator to participate more actively and intensively. Examples are systems like IMAX and OMNIMAX or the dome projections used in planetariums. Ultimately, these methods might be expected to produce a kind of 'experience space' with 360� projections including optical stereo effects as well as acoustic components whose degree of perfection is already very high.

In the future, a combination of computer graphics and holography can also be expected. While holographic images of larger ensembles (scenes or landscapes) hardly seem feasible for technical reasons, any three-dimensionally programmed computer image can, in principle, be transformed into a hologram basically by way of Fourier transformations. These, however, presuppose large computer capacities. While static images can already be produced in this manner, holographic computer animation would still require the invention of special holographic displays.

There is no doubt that at first the presentation of these types of technological facilities has something of a 'show' character. However, this does not preclude their future use in art. The motion picture, too, was originally show-booth attraction. What is of interest to artists is the fact that they have instruments at their disposal that enable them to go beyond the small, limiting frame of a picture and to present instead comprehensive image of an environment or the world. Unlike motion pictures, however, these images will not be merely reproductions of reality but creative works of artists exploring far greater possibilities than traditional graphic artists, painters, stage designers or camera operators.

The new instrumentarium, the first signs of which are already visible, will certainly have an impact on the methodology and mode of thought of the designer. Painters working with conventional methods are limited to depicting faces; when painting in a naturalistic style they have to observe the rules of perspective. They have to show the systematic foreshortening of objects as they recede into the distance to ensure that the visual impression is correct.

Today, these routines can be handled by computer software. Artists working with computers focus their attention on entirely different things. They construct their objects in three dimensions which, although more difficult than confining themselves to depicting the surface, is the prerequisite for depicting an object from all angles. A landscape that has been mapped and whose data have been stored can be shown from any desired perspective - comparable to a travelling shot. As a result, artists will start to consider structural problems, e.g. the shapes of a mountain range, the distribution of trees in a forest. Their task becomes even more difficult when they work with moving objects, such as people or animals. In such cases, they have to study motion patterns as well. In designing the body of an animal, it would make sense to start with its skeleton, as the critical angles of the joints determine which movements can be performed. Again although a great deal of time-consuming preparatory work is needed, it pays off in the end, because having the animal perform any desired movement will become a routine task. Similar efforts are required for the human face. Several years of university-level research are necessary to arrive at a precise description of the different facial expressions and their variations. In order to obtain a realistic model, all muscles in the face must be assigned parameters that include all the variants of human facial expression. A future task will be to co-ordinate lip movements and speech and later maybe even to co-ordinate the meanings of words with the expression on the face.

These tasks pose entirely new problems which although they first have to be solved by professional computer scientists, lead again and again to esthetic questions. Once the appropriate routines have been developed, they will be generally available. The user will then be able to use software packages to design landscapes, plants and human beings without needing a technological background. The results will be a reproduction of our environment - a task comparable to designing a stage set or preparing an animation model for a movie.

Mere reproduction, irrespective of the type of projection (e.g. large-screen or 360o), is a routine task not requiring any special artistic ability. However, esthetic questions undoubtedly are involved; the more the scene depicted deviates from reality, the more important these questions are; for example, reconstructing a scene from ancient Greece may well become a task requiring artistic creativity. Artistic creativity will become even more important when imaginary landscapes or utopian visions are to be created. Such a design, however, could also be regarded as a task for the artist with no conventional or realistic constraints whatsoever. If the artist wished, a world of imaginary beings in which the laws of physics are no longer applicable could be created. It could be a kind of Surrealism put into motion. Through the use of sophisticated illusion technology, the spectator, being part of the scene, would partake in a genuine experience. That this can be done has already been demonstrated; examples are the dancing dinosaurs in a sequence presented at SIOGRAPH '85 computer animations in which objects penetrate each other, various commercials, or scenes from the movie TRON depicting a fantasy world.

The use of computer technology entails another element, not yet mentioned, that might be of interest to art: the possibility of interactive communication. Dialog systems offer users a chance to carry on a conversation with the computer. The computer answers questions, makes suggestions, corrects mistakes, etc. In the case of artistic applications, e.g. cybernetic objects, the users influence the esthetic experience. By clapping their hands or by casting a shadow, they can make the object respond. Another interesting application of interaction is 'variable pictures': the artist does not use the computer to create a specific image or sequence; rather he or she supplies a program that permits the activation of different image variants and sequences. It is up to the users to find their own way through these many latent images. This principle need not be confined to abstract images but can also be applied to imaginary landscapes or spaces. The user embarks upon something like a journey of exploration through a world an artist has designed. Here, too, examples already exist, even though at present they are technologically unsophisticated. At M.I.T., a town with streets, buildings and other details has been documented in a computer program that permits users to familiarize themselves with the place by walking through the streets and by zooming in on details which interest them.

In future applications - including art - there will be no need to limit the scope of action. Although the storage capacity of even the biggest computer is limited, a programming trick makes it possible to create an infinitely big world. This is achieved by not defining this world in detail but merely by giving the computer instructions on the structure and distribution of objects, which are not activated until the user approaches them. This enables the user to venture as far as desired, all the time moving in environments that are different from the ones previously 'walked' through. In order to achieve this great diversity, the processes will require random control, e.g. the commonly used method of fractals.

A final expansion of this technique will provide a link with fiction. Of all forms of art, literature has so far succeeded best at remaining out of the reach of programmers. This is due to the fact that the use of language is a creative process that presupposes an understanding of semantic context. It can be expected, however, that literature, too, will be included in the digital electronic system, particularly through the use of artificial intelligence. Present-day linguists are interested mainly in the detailed structures of texts as well as in the determining courses of action. In the former case, the rules of syntax have to be observed, while in the latter case what counts is the conception of conflict situations from which plots and dialogs evolve. Here, too, the computer might open up new perspectives. It is quite possible to have a plot that instead of being linear is ramified-the actual plot is determined by the reader. This method, which is already used in computer games, will lead to a new way of dealing with literary subjects which will involve not only plot narration but also the creation of actual images that replace conventional descriptions of landscapes and environments. Thus, the user is assigned a part in the plot, which through the use of images becomes very realistic. Although it is most likely that these interactive games will initially serve entertainment purposes, there is no reason they should not be of high artistic quality. The connection thus established between visual arts, music and literature will demand a new type of artist, one who becomes the designer of a 'sphere of action' that includes image and sound as well as all the emotional elements of classical drama. The artist does not need to be confined to realistic events; the opposite is true. As in other artforms, a work that creates a sense of alienation can be very effective, and abstract forms designed to produce multi-faceted perceptions are of considerable interest.

The reflections and considerations outlined here refer only to developments whose beginnings are already visible today. Certainly, extrapolation does not guarantee that results will be as predicted; however, it provides a basis for assessing what is and what is not possible. Although most of the developments that can be anticipated will first be used for entertainment purposes, they involve some remarkable esthetic aspects requiring the attention of creative designers. Whether the developers of the new media will be painters and graphic artists trained at academies and universities, stage designers and camera operators coming from the theater and films, mathematicians and programmers involved in computer graphics and animation, or people exercising a new, as-yet-unknown profession, there is only one thing that counts: the new media should not be used for commercial or entertainment purposes alone; rather their enormous artistic potential should be exploited fully.