Video Object Segmentation without Temporal Information

Video Object Segmentation, and video processing in general, has been historically dominated by methods that rely on the temporal consistency and redundancy in consecutive video frames. When the temporal smoothness is suddenly broken, such as when an object is occluded, or some frames are missing in a sequence, the result of these methods can deteriorate significantly. This paper explores the orthogonal approach of processing each frame independently, i.e., disregarding the temporal information. In particular, it tackles the task of semi-supervised video object segmentation: the separation of an object from the background in a video, given its mask in the first frame. We present Semantic One-Shot Video Object Segmentation (OSVOS$^\mathrm {S}$S), based on a fully-convolutional neural network architecture that is able to successively transfer generic semantic information, learned on ImageNet, to the task of foreground segmentation, and finally to learning the appearance of a single annotated object of the test sequence (hence one shot). We show that instance-level semantic information, when combined effectively, can dramatically improve the results of our previous method, OSVOS. We perform experiments on two recent single-object video segmentation databases, which show that OSVOS$^\mathrm {S}$S is both the fastest and most accurate method in the state of the art. Experiments on multi-object video segmentation show that OSVOS$^\mathrm {S}$S obtains competitive results.

Symmetry From Shape and Shape From Symmetry

This article discusses the detection and use of symmetry in planar shapes. The methods are especially useful for indus trial workpieces, where symmetry is omnipresent. "Symmetry" is interpreted in a broad sense as repeated, coplanar shape fragments. In particular, fragments that are "similar" in the mathematical sense are considered symmetric. As a general tool for the extraction and analysis of symmetries, "Arc Length Space" is proposed. In this space symmetries take on a very simple form: they correspond to straight-line segments, as suming an appropriate choice is made for the shapes' contour parameterizations. Reasoning about the possible coexistence of symmetries also becomes easier in this space. Only a restricted number of symmetry patterns can be formed. By making ap propriate choices for the contour parameters, the essential properties of Arc Length Space can be inherited for general viewpoints. Invariance to affine transformations is a key is sue. Specific results include the (informal) deduction of the five possible symmetry patterns within single connected con tour segments, the importance of rotational rather than mirror symmetries for deprojection purposes, and relations between simultaneous symmetries and critical contour points.

Minimal information to determine affine shape equivalence

Participants judged the affine equivalence of 2 simultaneously presented 4-point patterns. Performance level (d') varied between 1.5 and 2.7, depending on the information available for solving the correspondence problem (insufficient in Experiment 1a, superfluous in Experiment 1b, and minimal in Experiments 1c, 2a, 2b) and on the exposure time (unlimited in Experiments 1 and 2a and 500 ms in Experiment 2b), but it did not vary much with the complexity of the affine transformation (rotation and slant in Experiment 1 and same plus tilt in Experiment 2). Performance in Experiment 3 was lower with 3-point patterns than with 4-point patterns, whereas blocking the trials according to the affine transformation parameters had little effect. Determining affine shape equivalence with minimal-information displays is based on a fast assessment of qualitatively or quasi-invariant properties such as convexity/ concavity, parallelism, and collinearity.

Minimal information to determine affine shape equivalence

Participants judged the affine equivalence of 2 simultaneously presented 4-point patterns. Performance level (d') varied between 1.5 and 2.7, depending on the information available for solving the correspondence problem (insufficient in Experiment 1a, superfluous in Experiment 1b, and minimal in Experiments 1c, 2a, 2b) and on the exposure time (unlimited in Experiments 1 and 2a and 500 ms in Experiment 2b), but it did not vary much with the complexity of the affine transformation (rotation and slant in Experiment 1 and same plus tilt in Experiment 2). Performance in Experiment 3 was lower with 3-point patterns than with 4-point patterns, whereas blocking the trials according to the affine transformation parameters had little effect. Determining affine shape equivalence with minimal-information displays is based on a fast assessment of qualitatively or quasi-invariant properties such as convexity/ concavity, parallelism, and collinearity.

Overexpression of human poly(ADP-ribose) polymerase in transfected hamster cells leads to increased poly(ADP-ribosyl)ation and cellular sensitization to gamma irradiation

Poly(ADP-ribosyl)ation is a posttranslational modification of nuclear proteins catalyzed by poly(ADP-ribose) polymerase (PARP), an enzyme which uses NAD+ as substrate. Binding of PARP to DNA single-strand or double-strand breaks leads to enzyme activation. Inhibition of poly(ADP-ribose) formation impairs the cellular recovery from DNA damage. Here we describe stable transfectants of the Chinese hamster cell line CO60 that constitutively overexpress human PARP (COCF clones). Immunofluorescence analysis of gamma-irradiation-stimulated poly(ADP-ribose) synthesis revealed consistently larger fractions of cells positive for this polymer in the COCF clones than in control clones, which failed to express human PARP. HPLC-based quantitative determination of in vivo levels of poly(ADP-ribose) confirmed this result and revealed that the basal polymer levels of undamaged cells were significantly higher in the COCF clones. The COCF clones were sensitized to the cytotoxic effects of gamma irradiation compared with control transfectants and parental cells. This effect could not be explained by depletion of cellular NAD+ or ATP pools. Together with the well-known cellular sensitization by inhibition of poly(ADP-ribosyl)ation, our data lead us to hypothesize that an optimal level of cellular poly(ADP-ribose) accumulation exists for the cellular recovery from DNA damage.

Qualitative cues in the discrimination of affine-transformed minimal patterns

An important factor in judging whether two retinal images arise from the same object viewed from different positions may be the presence of certain properties or cues that are 'qualitative invariants' with respect to the natural transformations, particularly affine transformations, associated with changes in viewpoint. To test whether observers use certain affine qualitative cues such as concavity, convexity, collinearity, and parallelism of the image elements, a 'same-different' discrimination experiment was carried out with planar patterns that were defined by four points either connected by straight line segments (line patterns) or marked by dots (dot patterns). The first three points of each pattern were generated randomly; the fourth point fell on their diagonal bisector. According to the position of that point, the patterns were concave, triangular (three points being collinear), convex, or parallel sided. In a 'same' trial, an affine transformation was applied to one of two identical patterns; in a 'different' trial, the affine transformation was applied after the point lying on the diagonal bisector was perturbed a short, fixed distance along the bisector, inwards for one pattern and outwards for the other. Observers' ability to discriminate 'same' from 'different' pairs of patterns depended strongly on the position of the fourth, displaced, point: performance varied rapidly when the position of the displaced point was such that the patterns were nearly triangular or nearly parallel sided, consistent with observers using the hypothesised qualitative cues. The experimental data were fitted with a simple probabilistic model of discrimination performance that used a combination of these qualitative cues and a single quantitative cue.

Higher-order structure in regularity detection

In three experiments a simple Euclidean transformation (reflection, translation, rotation) was applied to collections of twelve dots in such a way that they contained equal lower-order structure, defined on the pairwise grouping of elements with their partner following transformation (e.g. parallel virtual lines), but differed in the presence vs absence of higher-order structure, defined on pairs of pairwise groupings (e.g. virtual quadrangles with correlated angles). Based on the much better performance levels (d') in the case of additional higher-order structure, we conclude that global regularities are easier to detect when the local correspondences are supported by higher-order ones formed between them. These enable the lower-order groupings to spread out across the whole pattern very rapidly (called bootstrapping). As a preliminary attempt to specify these principles, we proposed a working model with two basic components: first, a function expressing the cost of a perceptual grouping or the lack of regularity, and, secondly, an algorithm based on simulated annealing to minimize the cost function. The simulation results obtained with our current implementation of these principles showed satisfactory qualitative agreement with human regularity detection performance. Finally, the theory was shown to capture the essence of a large number of grouping phenomena taken from diverse domains such as detection of symmetry in dot patterns, global structure in Glass and vector patterns, correspondence in stereoscopic transparency and apparent motion. Therefore, we are convinced that, in principle, the mechanism used by the human visual system to detect regularity incorporates something like bootstrapping based on higher-order structure. We regard this as a promising step towards unraveling the intriguing mechanisms of classic Gestalt phenomena.

Detection of symmetry in tachistoscopically presented dot patterns: effects of multiple axes and skewing

We examined the effects of multiple axes and skewing on the detectability of symmetry in tachistoscopically presented (100-msec) dot patterns to test the role of normal grouping processes based on higher order regularities in element positions. In addition to the first-order regularities of orientational uniformity and midpoint collinearity (Jenkins, 1983), bilateral symmetry (BS) gives rise to second-order relations between two pairs of symmetric elements (represented by correlation quadrangles). We suggest that they allow the regularity (i.e., BS) to emerge simply as a result of the position-based grouping that takes place normally, so that no special symmetry-detection mechanism has to be postulated. In combination with previously investigated variables--number and orientation of axes--we introduced skewing (resulting from orthographic projection of BS) to manipulate the kind and number of higher order regularities. In agreement with our predictions, the data show that the effect of skewing angle (varied at three 15 degrees steps, clockwise and counterclockwise) on the preattentive detectability of symmetry (measured with d') increases as the number of axes decreases. On the basis of some more specific findings, we suggest that it is not as much the number of correlation quadrangles that determines the saliency of a regularity as it is the degree to which they facilitate or "bootstrap" each other.