Published in Digital Creativity, Vol. 10, No. 4, 1999, Swets and Zeitlinger, p. 215 - 227. ISSN 1462-6268. This paper was presented to the CADE'99 Conference, Teeside University, April 1999.

7,000 words

Abstract

3D computer graphics has been under-used by fine artists. This paper explores some of the possible reasons for this, and also some of the possible attractions of 3D. The rigid depiction of three-space, as performed so accurately by the camera, had been a goal of painters since the Renaissance, but in the twentieth century a host of factors led to its subversion. Only in the sixties and in the US did a photorealist painting become popular. The author suggests that linear perspective has been seen as a tyranny, but the time has come to re-evaluate and liberate it through 3D computer graphics as a medium for fine art. The author's own work using ray-tracing software is described.

Keywords

3D, linear perspective, aerial perspective, 20th century art, computer graphics, ray tracing.

Introduction

Linear perspective plays a key role in our perception of the outside world, and the discovery of its mathematical basis was one of the triumphs of the Renaissance. Why then do contemporary artists, when engaging with the new media technologies, generally shun 3D in favour of 2D or interactive works? As an artist working with 3D computer graphics myself, I have attempted to tackle this question from a number of angles. Some of the philosophical and scientific issues related to dimensionality give us clues as to why 20th century artists generally abandoned perspective, often also raising interesting metaphysical points about three dimensions, which inform my own work. As an artist/programmer I am in the fortunate position of writing my own 3D software, but this adds to the questions surrounding the medium. One of these questions, briefly tackled here, is whether possible theoretical limits to computing power will also limit our ability to create virtual 3D worlds. However, perhaps the most interesting questions that the medium raises are ones of perception and cognition.


Issues of Dimensionality

Imagine a toddler just old enough to stand at the edge of a cot or playpen, looking directly at a door. The visual system provides a flat image on the retina, the image of which is somehow brought into the child's consciousness, but as yet there are no concepts of space. In the child's retina (and therefore we assume, consciousness) the rectangular shape of the door then distorts into a thinner shape with the verticals still parallel but the other two sides making acute angles. An amorphous shape is revealed in the still rectangular door frame; the shape wobbles from side to side and grows; the top part of it then moves down and grows in size to fill the entire field of view; a loud wet smack is heard as vision is almost obliterated, and then the shape metamorphoses from a generally pink one to a generally black one, diminishes in size, wobbles, and shrinks small enough to pass again through the rectangular frame. The other four-sided figure changes shape to fit the frame once more with an accompanying noise. Mother has just checked on her child, kissed it, and gone out again, pulling the door too.

The Scottish philosopher David Hume (1711 - 1776) was interested in how we construct a coherent world in our mind from a stream of separate and possibly chaotic sense-impressions, such as the baby received in the above example. This is a problem of how the inner and outer worlds relate, a problem formulated by Descartes in his dualistic concept of res cogitans, and res extensans. In his Treatise on Human Nature, Hume noted that our sensations of the world are private, and cannot be directly corroborated, yet when we communicate with each other we recognise that we have a common description of the outside world. With empiricism a growing intellectual force in Europe (due to the rise of science) there was a growing interest in perception, but accompanied by a distrust of the senses. Hume solved this problem by deploying novel kinds of imagination. The first kind of imagination, for Hume, was the ability to bring an image of something into one's mind, more simply a memory. The second type of imagination was an artistic one, that we might call fantasy, but his important contribution was to postulate a third kind of imagination, one that maintained a coherent picture of the outside world from a kaleidoscope of sense impressions. The contemporary philosopher Mary Warnock [1] has made an interesting study of Hume's' ideas on imagination, asking how imagination leads to truth.

Kant (1724 - 1804) refined Hume's categories of imagination calling the fiction-making power that varies from individual to individual an empirical imagination, while calling the imagination that is fundamental to our perception of the world a transcendental imagination. Kant struggled with the problem of an outside world that we have no direct access to other than through our sense impressions; he called it the problem of the thing-in-itself.

We are more interested here as to how Kant's transcendental faculty of imagination can maintain a spatial sense of the outside world, and in particular the nature of three dimensions. Our sense of perspective derives from the observation that objects get smaller as they move further away, though the only data the eye receives is a flat image on the retina, with no depth information. The child has to develop the 'transcendental imagination' to construct a fixed size and nature for objects despite the clear sensory data of variable size and shape, as in the example of the door and mother, and to construct depth values for each object in a scene. To enter the child's view of the world as an adult is then seen as an act of counter-imagination, an act of unlearning the faculty of constructing objects as fixed in size and shape, when the sensory data tells us otherwise. This is only done as an act of enquiry (as in Hume's case), as an artistic endeavour, or, in the East, as part of a meditation practice such as Zen.

New developments in science also make us question our interpretation of sensory data. The 20th century saw a revolution in science that profoundly challenged the Newtonian world-view of the previous two centuries, casting doubt on the fundamentals of space, light, and causality. Hume, standing at the beginning of the Newtonian period of science, was sceptical about causality (once again the faculty of imagination was needed to maintain this illusion for Hume), but Newton, or more vigorously, Laplace postulated a universe that was completely based on cause and effect. We call this a deterministic view, but the Laplacian determinism has since been seriously challenged by quantum theory. Hume would have been intrigued by a scientific echo to his scepticism on causality (where he would have least expected to find one), and perhaps also intrigued by Einstein's adoption of the invariance of the speed of light as fundamental to the structure of the universe. This simple but revolutionary principle led to the Special and General theories of relativity, but also points to a subjective view of perception: the speed of light is fixed to the subject, regardless of relative motion of the subject to the source of light.

However, it is the spatial implications of the new science that interests us here. By taking the speed of light to be an absolute, Einstein had to reconsider the structure of space. In his youth Einstein had speculated on what he would see if he could 'ride' a ray of light, but only after a training in science and mathematics, and the discovery of the null-results of Michelson and Morely in attempting to measure the effects of trying to run towards or away from a ray of light, did he reach his revolutionary understanding of light and space. It is now accepted that it is a non-Euclidean four-dimensional space, with time as the fourth dimension, collectively known as space-time, a theory also set out by the Russian mathematician Minkowsky. Distortions of space-time caused by massive bodies such as planets result in the phenomenon we know as gravity. When travelling at speeds approaching that of light, space-time appears curved, or rather, objects perceived by the fast-moving observer seem distorted. This relates to Hume's problem (and ours of understanding the child's view) in that not only is a mental faculty required to overcome the apparently plastic nature of objects as apprehended by our senses, at high speeds objects cannot even maintain rigidity according to science! (We'll return to this point in connection with art later). For now it is worth mentioning an interesting perceptual oddity brought about by the film camera: the shot used by Hitchcock in 'Vertigo' and later in the first 'Jaws' movie. This involves a distortion of perspective brought about by dollying the film camera towards an object, while zooming out. The aim is to keep the object (in the case of 'Jaws' the head of an actor) a fixed size in the viewframe, so that it appears to distort, i.e. bring a plasticity to a solid object. In the case of a head the ears seem to move round to the sides, see fig. 1.

Three virtual photographs of a virtual character's head. In the leftmost image the virtual camera is far away with high zoom, resulting in an almost perspective-free (or orthogonal) view, in which the ears are clearly visible. The middle image shows the ears almost disappearing, due to the camera placed near to the head. The rightmost view is the result of an extreme closeup.

Fig. 1 Apparent distortion of an object through simultaneous dolly and zoom

Intriguingly, it seems that the general theory of relativity predicts similar distortions as the observer travels faster: apparently one would be able to see not just the front of objects but more and more of the sides, until, travelling at the speed of light, one would simultaneously see all sides of a three-dimensional object.

The mathematicians of the late 19th century were fertile in their production of what Mandelbrot has called 'mathematical monsters', ideas that were counter-intuitive at the time, and often had to wait many years for application. This included the mathematics of fractals and n-dimensional space. In this atmosphere a school headmaster published a dimensional novel under the pseudonym Edwin A. Abbott. His second edition of Flatland - A Romance of Many Dimensions appeared in 1884, and at the centenary celebrations at Brown University in the US, one of today's best-known four-dimensional artists, Tony Robbin, was present. Robbin uses computers in his four-dimensional work today, but four-dimensional space was a also key concept amongst the artists developing cubism and other new styles, including Duchamp in France and Malevich in Russia, at the birth of modernism some eighty years ago. The art historian Linda Dalrymple Henderson has made a study of the impact of four dimensional theory on the birth of modernism, but had assumed that artistic interest in the subject had died out in the thirties, until meeting Robbin in 1979. [2]

Returning to Flatland, which mainly explored an imaginary world of two dimensions, we find a social satire and some intriguing thinking on dimensionality. There are a few anomalies in Abbott's account however, which are useful pointers to thinking in three or more dimensions. Abbott assumed that the perspective of three dimensions would not translate into two, and had to invent two methods to create the sense of distance, firstly colour/texture (which becomes outlawed in his story, parodying the sartorial laws of the middle ages where only royalty and bishops were allowed bright colours such as purple, and the peasants restricted to browns) and secondly a kind of depth-cueing reliant on the often misty weather in Flatland. (This is also known as aerial perspective, i.e. appearance of depth due to effects of the air.) However, in fact, perspective would work as well for a two-dimensional baby as for a three-dimensional one, as soon as it had learned the 'transcendental imagination' to translate the perception of a shape diminishing in size into a shape of fixed size receding. The change of texture with depth, and all other conventional aerial perspective or depth cues would translate perfectly well into two dimensions, as Abbott postulated.

In the same intellectual climate as Flatland and Minkowsky's and Einstein's theories, there appeared P.D. Ouspensky's Tertium Organum, subtitled: A Key to the Enigmas of the World, 1st edition 1911. The title derived from two previous works: Francis Bacon's Novum Organum (1620) which influenced the acceptance of accurate observation and experimentation in science, and Aristotle's Organon, 4th c BC. The frontispiece to Ouspensky's book states that the "third existed earlier than the first", an echo of Jesus' statement that 'before Abraham was, I was". Clearly Ouspensky did not suffer from undue modesty, but his work has had little or no impact on modern philosophy. However, he did have an influence on Russian artists such as Malevich, particularly in their thinking on dimension, and, it seems independently of Abbott, Ouspensky also considered the implications of two-dimensional worlds.

Ouspensky's interest in the fourth dimension was part of the intellectual climate of the time, but what is interesting here is that he saw it as a solution to Kant's problem of the thing-in-itself. He argued that we could not apprehend the thing-in-itself directly because we were constrained to three dimensions, and so could not see all round it. Part of Ouspensky's struggle with dimensions was whether the fourth dimension was to be of space or time. Linda Dalrymple Henderson asserts that most of the artistic efforts in the early part of the century were focussed on a spatial fourth dimension rather than one of time. However, a cubist work like Duchamp's Nude Descending Staircase (1912) might be seen to play more on the time element.


Fine Art and 3D

Although every child through history gradually learns that mother does not actually shrink to leave the bedroom (or hut or cave) entrance, a mathematical understanding of perspective was not formalised until the Renaissance in Europe. We also know that one of the contributions of Islamic culture to the Indian sub-continent in the Mogul period was the introduction of perspective into Indian painting. The Italian Filippo Brunelleschi developed perspective in the period 1417 to 1420, and in 1425 Leon Batista Alberti published an explanation of Brunelleschi's method, the basis of all later uses of perspective in Europe. The rules of perspective that allow a three-dimensional space to be projected onto a two-dimensional surface are known as linear perspective, while other distance cues are collectively known as non-linear perspective, or aerial perspective, the development of which has been credited to Dutch and Flemish masters, and is notable in the landscapes of Jan van Eyck. 3D computer graphics systems have a range of such techniques, known collectively as depth cueing, a subject I shall return to in connection with my own software system. Edwin Abbot, as we saw, incorrectly assumed that only non-linear perspective would be available in two dimensions.

Early Renaissance painters explored perspective in different ways, often assuming more than one viewpoint, in order to emphasise certain subject matter. Over the centuries artists became less playful with perspective, until in the late nineteenth century it became one of many constraining forces to be overcome by the painters of the time. The sculptor Constantin Brancusi (1876 - 1957) was very much concerned for a freedom in which to explore his art - a freedom from the obligations of previous centuries to represent the 'religious'. Brancusi wanted 'an art of our own', a modern contemporary art that belonged to artists, and not to the church. Roger Lipsey [3] , in a survey of the spiritual in 20th century art, concluded that the birth of modernism and abstract art was in fact an expression of the spiritual, rather than the rational that some modernists proposed (he took Brancusi's theme of an 'art of our own' as part of the title to his book). He also believed that there was a relationship between modern art and the new physics, beyond the mere coincidence of abstraction.

In 1905 a Fauvist exhibition brought together the work of Matisse and other artists, declaring that colour superseded all the other elements of painting, and that colour could violate the integrity of objects, composition, subject and line. Where colour had been subordinated to the task of sculpting three-dimensional objects in linear perspective, and to enhancing the sense of depth with all the cues of aerial or non-linear perspective, it was now free to play a direct role in its impact on the senses and emotions. The impact of the Fauvists, and of other art movements that favoured abstraction and the completely abstract, was a drift towards two dimensional art, or 'art as pancake' as Leonard Schlain puts it in his book Art and Physics. The cubists, including Picasso, Braque, and the early work of Duchamp, chose to undermine linear perspective in a way that moved their work to four dimensions rather than two. Cubist paintings broke the rules of linear perspective by showing objects as if they were seen from more than one vantage point, all simultaneously, anticipating in an artistic sense the four-space of Minkowsky and Einstein. Where Einstein showed through mathematics that one could see the sides and back of an object at the same time as its front, the cubists demonstrated the idea in paint.

The contention of art historian Roger Lipsey that prevailing spiritual interests influenced the birth of modernism is contested, but the artists of the time lived in a period of intellectual ferment that is hard to empathise with now. Science competed head-on with the occult for the de-churched intelligentsia at the turn of the century, with the victory for science only assured after two world-wars. The public lecture (whose popularity is only recently returning) was the key medium for disseminating new ideas about science and the occult, and the two were jumbled up in a way we could not understand today. Rudolf Steiner, founder of Anthroposophy, was one of the key speakers on the occult of the first two decades, and influenced many artists of the period. He was against what he called spatial perspective, calling for artists to leave it behind and focus on colour perspective, even calling our physical space a 'joke of three dimensions'.[4] Steiner reached this conclusion independently of the Fauvists and in rejection of science. While it is almost impossible for a liberal Western-educated intellectual to enter Steiner's thought today, the fact remains that through the network of Steiner schools in Britain and Europe his ideas on art are still propagated. Ouspensky's thought, later to be aligned with the that of the Armenian teacher G.I.Gurdjieff, had less impact in Western Europe, but was widely disseminated amongst Russian artists. The Russian Cubo-Futurist Matiushin interwove extracts from Tertium Organum with passages from Gleizes and Metzinger's 'Du Cubisme'.[5]

Leonard Schlain finds a number of reason for the flight from a rigid 3D in modern art: including the anticipation by the modernists of relativity. In his fascinating book Art and Physics [6] he shows how the dimensional flexibility of modern artists such as Giacometti anticipated and paralleled the distortions of relativity. According to Schlain, Linda Dalrymple Henderson's The Fourth Dimension and Non-Euclidean Geometry in Modern Art concludes that modern artists who used the concept of four dimension in their work and wrote on it were all unaware of Minkowski and Einstein, and so there could be no correlation between the endeavours.[7] Schlain sees this as confirmation of his overall thesis that artists have always anticipated scientific advances, though in an allegorical rather than explicit manner. However, the Russian Cubo-Futurists and Supremacists such as Malevich worked in the 1920s by which time the ideas of Ouspensky, Minkowski and Einstein were more widely known.

Another significant development that changed artists' attitudes to dimensionality was the invention and rapid adoption of the camera. Amid fears that some of the traditional roles of the painter were to be undermined by the new discovery, the camera became an additional impetus to free modern art from the naturalistic. This included a freeing from linear perspective, as this was now the job of a mechanical device.

The exception proves the rule, and the exception to the general abandonment of realism and perspective in 20th century are art styles including realism, photorealism and superrealism. These movements include artists like Audrey Flack, Richard Estes, and Chuck Close, known respectively for portraits, landscape, and still life. In an age of high-quality colour photography and mechanical reproduction these artists have chosen to make paintings that vie with the camera for naturalistic rendition of the world, mainly city views, or of the human body. Frank Goodyear provides a good overview of this art movement in the US in the sixties and seventies [8], debating the motivations and influences on the artists, and whether it is to be seen as a movement or a set of related ventures. While many of the photorealists do work from photographs, they are not necessarily direct renditions of the image in emulsion to an image in paint. Richard Estes re-arranges elements, perspective, colour and detail to reach some inner essence of the scene (usually urban views) that he is rendering. He is also quite specific that, unlike the image recorded by his camera, everything is in focus in his paintings:

"When I look at things, some are out of focus. But I don't like to have some things out of focus and others in focus because it makes it very specific what you are supposed to look at, and I try to avoid saying that. I want you to look at all. Everything is in focus." [9]

Whether the photorealists are a reaction to Modernism or an extension of it is up for debate. Estes' idea that everything in his picture, itself already a depiction of the ordinary, should have equal weight is a modern idea, though his painter-hero is Vermeer. That photorealism developed mainly in the US is perhaps due to a diminished need to rebel against the tyranny of perspective, after all the US did not have its own Vermeers.

The flight from perspective and 3d representation in modernism had many roots, as we have seen. While the development of perspective in Renaissance times is seen as a part of the rationalistic Enlightenment project, by the start of the 20th century it was associated with the subjugation of art to church, and was part of the 'baggage' (Brancusi's term) that had to be thrown off. The work of 19th century mathematicians on dimensionality, and the discovery of relativity by Einstein opened up the fourth dimension and multiple view-points. The birth of modernism, associated with Theosophy and Anthroposophy through leading practitioners like Mondrian and Kandinsky, also led to an emphasis on two dimensional work, though how much this may have been due to Steiner's influence is not clear. The later development of fascination with primitive or outsider art, which could be called pre-perspective, was yet another reason to abandon perspective in art schools, focusing instead on the intuitive, gestural, and spontaneous. And the camera, first mistrusted by the art community, had, in the hands of the great photographers of the 20th century proved to be a fine art medium in its own right, and made it even more pointless for artists to reproduce what they saw in perspective form. While the high prices that photorealist paintings may command today show a deep popular interest in conventional three space, the photorealist movement taken as a whole is a blip in modern art.


3D Computer Graphics as a Medium

Let us turn now to the computer as an artistic medium. While the computer paint system arrived later than 3D computer-aided design systems, they were developed with and for artists right from the start. Realist painter Philip Perlstein, known for his penetrating depictions of the human body, was persuaded by a television producer in 1983 to work with an early paint system and soon found it a "medium unique unto itself".[10] In 1986 the BBC ran a series called 'Painting with Light', where it introduced a number of contemporary painters to the Quantel Paintbox (already a sophisticated broadcast paint system, though its price put it beyond most individuals to own). One of the artists in the series, Richard Hamilton, has continued to use the paintbox up to the time of writing. Hamilton's best known piece is probably his 1956 collage, Just what is it that makes today's homes so different, so appealing?, depicting the 3D interior of a room with elements cut from advertisements and photographs, each with a slightly different perspective. He re-worked this theme with the paintbox, using the 2D system's limited perspective function, but does not seem to have been tempted into using a dedicated 3D system.

Computer-Aided Design (CAD) was developed for the motor industry and other product design fields, and one of the fundamental techniques of the 3D illusion, Gouraud shading, is named after an engineer at Renault, Henri Gouraud. In fact many of the techniques in 3D are named after engineers or computer scientists, Phong shading for example named in honour of a PhD student who developed it and died tragically young. Hardly the ambience for artists however, in a century where art was almost identical with revolt. It is only in the last years of the 20th century that art is converging with science and technology, but there seems no comparable meeting of artists with 3D systems as there has been with 2D, exemplified by Perlstein and the Painting with Light series. There was a notable exception: David Em, who worked at the Jet Propulsion Laboratory [11], and there have been other collaborations with the corporate sector - John Whitney Sr. with IBM [12] in the sixties, Harold Cohen through the seventies with DEC [13], and William Latham in the eighties with IBM [14] again.

Em's work, using the 3D software of James Blinn and others at NASA Jet Propulsion Laboratory is catalogued in one of the few monographs devoted to a computer artist. What may be significant about the work is more the 2D aspects than the 3D: Blinn developed texture- and bump-mapping for simulating the surfaces of Mars, Jupiter and Saturn by the Voyager space probe, all of which Em used and modified to give a painterly quality to his 3D work, or to use in a purely 2D way. (It may be that the public in fact better remembers the computer graphics of the period than the actual photographs of the planets!) Em's best known piece is Transjovian Pipeline from 1979, which makes strong use of perspective, but his later works are more 2D. Latham's work at IBM in the late 80's and early 90's was strictly 3D, but animated. Although Latham made large photographic prints of his work for exhibition, it is the animation side that he is best known for.

So what in the nature of the 3D medium might be alien to the way that a painter works? I would suggest that in part it is the indirect working methods in colour and form that are alienating. Most systems, because of their CAD inheritance, use orthogonal views for design, while the final image is rendered in a perspective view. Hence a compositional intention in the mind of the artist is always a struggle to realise, because the position and orientation of 3D volumes in the Cartesian threespace yield a perspective view that is always just beyond the capacity of the human mind to predict, however 'good' at 3D. For the contemporary artist used to an intuitive gestural art, where a mark on the canvas remains unaltered by any arcane mathematical transformation, the lack of immediacy in 3D is frustrating. For a studio photographer it comes more naturally, because their art is similar in some respects: positioning and orienting 3D volumes to create a perspective composition. However the photographer can still see the likely perspective composition through the viewfinder, whereas 3D systems are too slow to provide more than crude previews.

After meta-design the second problem is meta-colour, again an issue that a studio photographer will know better than a painter. When a painter squeezes a colour out of a tube and spreads it on canvas the final colour is relatively predictable (more so for opaque paints than water-colours). In 3D the final colour of an object depend on the virtual lighting conditions as well as an object's nominal colour (unless fully self-illuminating, but using this option flattens the 3D illusion). This difficulty is compounded by the poverty of virtual lighting found in most 3D systems. Because of the enormous processing power required to light a virtual scene using the laws of physics or optics, empirical methods have been employed which are a good first approximation and are relatively fast. I call this a trade-off between 'physics' and 'faking it,' [15] where higher quality work demands more and more 'physics'. Ray-tracing and radiosity techniques are well-known for delivering high-quality lighting effects, but require a great deal of processing power.

Most 3D systems offer a progression of image quality so that during the design stage one can estimate the final result from low-resolution low-quality previews. As the final ray-traced or radiosity image may take hours to render, one is usually confined to the low-quality previews as one builds a scene or image. The iterative nature of the process arises from this lack of real-time rendering, and is yet another obstacle for the artist. In my own work I estimate that the fastest commercially-available PC or Macintosh is of the order of two hundred million times too slow for the real-time rendering of my scenes.

Given that a painter finds 3D graphics problematic because of meta-composition and meta-colour, might the studio photographer, familiar to some degree with both these issues, be more attracted to the medium? The problem than arises, in comparing a 3D synthesised scene with a photograph of a real scene, in the sheer visual poverty of the result. This is partly a result of the 'two cultures' divide, in that scientists are interested in the minimum description of a computer system, and are notorious for their complacency in saying that more complex descriptions are 'in principle' available, or even 'an exercise for the reader'. This means that 'in principle' we can describe a house-brick as a cuboid with a repeating texture wrapped around it, and 'in principle', it can be lit in 3D with a few point source of light. To the artist or photographer a used-house brick is a human artefact with a history and a rich visual appearance in form and texture which no 3D system can generate. Or, rather a 3D system could generate a house brick to meet the photographer's sensibilities, probably using all its resources, after which the virtual artist/photographer might say, fine, I want a hundred of these, all different, scattered around a yard, with three hundred rusty cars and a dirt track, and by the way it's dusk. This scrapyard scene, that could be photographed for a few pence and in a few minutes in any derelict inner city area, would 'in principle' be generatable by a 3D programme, but not in practice, because no machine is powerful enough at this point. Even more interesting is the possibility that it could never be, as there are many signs now that there will be a limit to computing power, set by the size of the smallest memory devices at atomic level, and processing speeds, limited by the speed of light.

As a spin-off from a direct engagement with the problems of rendering realistic imagery one develops a healthy respect for the real world and a sense of proportion about the computer power needed to synthesise such imagery. The physicists Frank Tipler and David Deutsch have based recent books on the premise that computing power will be effectively unlimited in the future, and that virtual worlds can be constructed indistinguishable from the real one. Tipler uses this scenario to 'prove' the existence of an 'Omega Point' that has the characteristics of God, while Deutsch's interest lies with speculations about parallel universes. Deutsch is a little more cautious than Tipler, stating that while memory may be effectively unlimited, processing power may not. Deutsch's solution is to slow down the 'clock speed' of the brain of the virtual world inhabitant to allow time for the processor to deliver the image data; the cost of this is that five minutes in VR may take five years in the physical body.[16] But Deutsch has plucked figures out of the hat: his ratio of five minutes to five years is a factor of roughly half a million. Exchanging this for my estimate of two hundred million brings the cost of the 5 minute virtual experience up to two thousand years! Another oversight in Deutsch's calculations lies in the problem of constructing a virtual world with an interesting behaviour: if we grant infinite memory (which is of course impossible) and a team of world-builders (probably thousands strong) prepared to build the virtual space to sufficient detail, what about generating the unfolding of this world in real time? Deutsch assumes more realistically that audio can be synthesised in real time, but again this misses the point about the auditory behaviour of a world: the caw of a crow takes place because of a complex social structure and as a reaction to all kinds of events; the song of a blackbird would defy any algorithmic synthesis!

Frank Tipler, in his Physics of Immortality [17] simply assumes that in the far future computing power will be infinite, based on the present growth curve. This issue needs some careful research, but with processor miniaturisation reaching the atomic level, and communication speeds limited by the speed of light, it seems that we could approach a theoretical limit to computing power quite soon. My own hunch, for what it's worth, is that it takes a computer the size of a universe to model a universe, let alone the multiple universes of David Deutsch.

The major aesthetic difficulty with 3D graphics is undoubtedly the rigid perspective. In a century where perspective had been subverted in every possible way in painting, and where the camera had become the major artistic tool for the exploration of 'real' light and space, the 3D programme is simply not expressive enough. It does not have the fluidity of the artistic space now permitted in art, nor does it have the subtlety of a photograph. The other problem for 3D as a fine art medium is the wealth of 3D imagery that is generated for non-artistic reasons, such as in computer games, science, and advertising. A fine artist may simply assume that the medium prescribes the aesthetics demonstrated by non-artistic use, and that it cannot be used otherwise.


Raysculpt

Bearing in mind the discussion so far on the problems and potentials of 3D, how have I developed my own system, Raysculpt? I originally wrote the system as a simple 3D modelling programme in 1985 as part of research in user-interface design, and expanded it later to include a ray-tracer, the source code of which was kindly supplied by computer artist Richard Wright. A ray-tracer is a way of 'photographing' a virtual 3D world, and uses the concept of a ray of light, but in reverse, as though travelling from the eye of the observer into the 3D database. Although computationally expensive, ray-tracing is closer to the physics of light than more empirical methods, and allows for accurate shadows, reflections and refractions. It can also generate soft shadows, blurry reflections (glossiness) and blurry refractions, but this increases rendering times by an order of magnitude again. The RaySculpt system is limited to spheres and flat places as modelling primitives, and is documented in previous papers. [18] Fig. 2 shows the system running under Windows '95.

Fig. 2 The RaySculpt system running under Windows '95

The limited 3D palette of the system was originally an accident (allowing me to focus on the interface rather than the modelling), but now is kept on purpose. While sympathetic to the goals of the photorealists, and attracted to the photographic image, I want to abstract a certain set of qualities from the 3D world I perceive, and the limitations of the system help me focus on volumes and textures in a fairly abstract way. An important part of the Raysculpt system is the direct recognition of the iterative process of adjustment, render, and re-adjustment, facilitated by the innovation of 'hot-buttons' recording the last eight user inputs. These allow for a quick return to dialogue boxes for adjusting parameters, and then re-rendering, speeding up the interaction cycle [19].

Depth mapping is a more recent addition to the system, allowing me to use the full scope of non-linear or aerial perspective effects. This is simply the generation of a grey-scale image, effectively a form of z-buffer, with ranges set relative to the camera location as concentric spheres. Although RaySculpt provides a basic depth-cueing it is the use of the depth-map as a layer mask in Photoshop that has allowed me a wider range of effects. Unlike Richard Estes, I am interested in the use of out-of-focus elements in my composition. However unlike with photography 3D computer-generated images are in perfect focus and the indication of depth-of-field by blurring is very compute-expensive, especially with a ray-tracer. Hence the use of the depth-map for this purpose, faking the depth of field by using Gaussian Blur in Photoshop. Fig. 3 shows a typical depth-map, fig. 4 its use in Photoshop, and fig. 5 the end result.

Fig. 3 A typical greyscale depth-map generated by RaySculpt

The layers palette in Photoshop, showing the use of the depth map as a layer mask. Dark areas of the layer mask reveal the background image, which is a copy of the foreground image, but heavily blurred. Note that an adjustment layer has also been applied to the background layer. This offers a wide scope of adjustments for creating aerial perspective effects.

Fig. 4 Use of the depth-map as a layer mask in Photoshop

Fig. 5 A RaySculpt image with depth-of-field simulated using the depth-map and Photoshop

RaySculpt is an on-going project, and I plan to develop more atmospheric effects, and also to include area lights and better soft shadows. It is in the area of texturing that I will really push the system, particularly as a tool for a better understanding of how texture relates to scale. Because I have been able to develop my own software, I have been able to create a more personal aesthetic with the system, focusing partly on sculptural forms and partly on qualities of depth and space such as in landscape. (The current series of images can be accessed via the Internet [20].)


Conclusions: The Tyranny and Liberation of Three-Space

Given the flight from linear perspective (the tyranny of 3D) by artists of the 20th century, the problems of object and ray poverty, and the aesthetic difficulties, why persist with 3D as an artistic medium? I can only answer for myself, that the acceptance of the constraint of 3D is in fact a liberation. Although a direct comparison would be absurd, I like to take the 3D structure of experience as a 'given' in the same way that Einstein took the invariance of the speed of light as a 'given'. After an intensely secular century of artistic activity, we don't need to rebel against the 'baggage' that Brancusi referred to, and our immediate experience of the world on a human scale is three-dimensional, a 'prior given' of space and light. We have no direct access to relativistic perceptions, so we can only pursue four dimensions as an abstract concept. Neither do we live in a two-dimensional world. The camera of course can record the outside world, so there is no need to aim at a slavish re-creation of it in virtual space, rather, when we discipline our sculptural and spatial fantasies through linear perspective, we can explore something about the deep structure of ordinary human experience in the world. The child in its playpen is exploring threespace with delight, fascination, and concentration. I am that child.

References