Numerous phenomena may cause a temporal variation of intensity. They include change of illumination of the scene e.g. cloud shadow: change of reflectance, transmittance or self illumination of objects, e.g. cooling or heating of objects, change of properties due to chemical change, the effect of rainfall: change of optics or sensor response, e.g. variation of focal length of lens system, variation of intensity range of sensor: motion of the sensor, e.g. camera mounted on a moving platform: motion of objects within the scene, e.g. moving vehicles, moving machinery, motion due to flow of liquid or gas. The motion may be rigid; when it may be expressed as a combination of translation and rotation; flexible, such as the motion of draperies; elastically deformable, such as an extensible tube, or even contain intrinsic growth patterns, such as sediment dispersal, cloud dispersion or vegetation growth. In all cases there exists a continuity in time of perceptual structures of attributes such as shape, texture, spectrum, connectivity, which enables continuity of object identification to be maintained.
Due to local intensity conservation, a field of optic flow may be defined which consists of regions of continuous and smooth variation separated by discontinuities at boundaries corresponding to occluding physical edges. The problem of determining the flow therefore becomes one of segmenting a piecewise continuous vector field consistently with the observed data.
Approach
Three possible approaches to motion estimation are as follows.- A dense image velocity field may be recovered, the
field of optic flow. The problem of flow field recovery
is under-constrained
- it follows that additional constraints, such as smoothness must be imposed. The optic flow field must then be related to the true three dimensional motions of objects within the observed scene [ullman1] . The requirement that the estimated flow be consistent with that which would be observed from the motions of rigid bodies may be utilised to constrain the solution [bk:hildreth] . This may be regarded as the canonical or traditional approach. - The motion of point or corner features may be
accurately tracked within the image and the motion of
rigid bodies then recovered from an interpretation of
these two-dimensional motions. Algorithms have been
demonstrated which utilise this approach successfully in
certain situations .
- The true three-dimensional motion of bodies in the
scene may be attempted to be recovered directly
[faugeras1] . Such methods have been
formulated but not, as yet, applied in a convincing
manner.
It may be anticipated that the motion observed in an image sequence will contain three contributions;
- Extrinsic uniform overall motion due to the motion of
the observer, including expansion due to approach, while
subject to flow discontinuities at occluding edges of
objects whenever parallax is significant
- Irregular overall motion due, for example, to
turbulence or vibration
- Intrinsic differential motion of regions relative to
their surroundings, due, for instance, to the self-motion
of vehicles, or the parallax of upright extended
structures relative to the background.
Proposed method
The chosen method of solution is to apply co-occurrence segmentation techniques [HB90] , developed previously for intensity based spatial segmentation, to the optic flow field by extending the segmentation of a scalar field to that of a vector field. The algorithm proceeds in two phases. An initial estimate of the optic flow field is obtained from the spatial and temporal derivatives; the estimate is then relaxed to provide contextual consistency. Since discontinuities of flow can occur, the relaxation must take account of flow boundaries by allowing flow discontinuities across them. Once the optic flow relaxation is running, the results of the relaxation of the flow for one frame, may be used to provide an initial estimate for the next frame in the sequence, augmented by the intensity data of that frame.Initial estimates of flow are reliable only in the vicinity of strong spatial edges; the algorithm is initiated by segmenting each image using a method based on the spatial co-occurrence matrices of intensities and relaxation [HB90] . This technique results in a robust segmentation into closed regions separated by distinct edges which lie on an inter-pixel lattice. An initial value of flow is then obtained from ratios of spatial and temporal derivatives which satisfy the motion constraint equation. A Gaussian probability distribution of flow at each image location is formed, having standard deviation proportional to the edge strength.
For pixels which are interior to a segmented region the initial value of flow is a weighted average of the local estimate and those obtained at the region boundaries in the four cardinal directions; the five values being weighted proportionately to their reliability and their distance from the considered pixel. This form of estimate reduces the effect of random noise whilst allowing smooth variation of flow within a region.
Local consistency of flow is then implemented by a relaxation algorithm which minimises the cross-entropy between the flow distribution at a point and that estimated from its immediate neighbours. The relaxation decouples pixels which are separated by a flow boundary. Flow boundaries are in turn located by making a segmentation of the image using the temporal co-occurrence matrices of intensities.
Data
Unclassified datasets used for this work have been supplied by DRA, as well as simple test data generated by hand, and data from Stanford Research Institute, which has been used in previously in the Literature by Barron et al [bfb] . Results are presented on the following data.- Square data, which is a translating square
on a blank background. This data is of size 40 by 40, and
has no noise in it.
- Yosemite data, provided by Lynn Quam at the
Stanford Research Institute. This is synthetic but highly
textured data with no noise, but very high local
variance. The images are of size 316 by 252.
- Bridge1 data, provided by DRA, consisting of
20 frames of FLIR data, from a set of some 300 frames.
These images were of size 458x380, and are of high
quality 8 bit data.
- Bridge2 data, provided by DRA, consisting of
20 frames of FLIR data, from a set of 100 frames. These
images are of size 600 by 512, and are of slightly lesser
quality, with more noise in the image.
- Filton data, provided by DRA, consisting of
25 frames of FLIR data. This data is of low quality, with
the top half of the image being useless. The original
size was 1200x512, but we used the centre 600 pixels of
the lower half of the image.
| ← Contents | Segmentation → |