Activity 10 - Processing Text

Processing of text is important, especially, texts containing handwritings. These can have significant applications for the analysis of documents, such as forged signatures and anonymous letters. In this activity, image processing techniques tackled previously in the activities are used to extract different letters of a handwritten text. These are labeled to identify each letter from the others.

Using a cropped portion of the document with handwritten texts, filtering with some weight was done to remove lines in the text. It also enhances enhances the details of the letters.

Original

Removal of lines and other weighted filtering

For segmenting the image, thresholding is applied to obtain the signals from the letters. This is to process each blob of the image as a letter.

Resulting binarized image

Since there are still unwanted artifacts (such as lines and dots), morphological operations were applied to clean the image. In this case, the opening, or the application of erosion and dilation morphological operations in sequence, was used.

Cleaning of image by opening operation (dilation after erosion)

Initial labeling of the letters was done to determine whether each letter was detected. This also tests whether each letter is well segmented from the background and from other letters.

Labeled image of each letter
Each letter ideally has a unique color from the other letters

To provide the basis form of the letters, thinning morphological operation was applied. The result of this shows the shape and structure of the letters, for example, of a handwritten text. The curvatures and length are some characterizations that can be made for each letter upon analysis of the results.

Thinning and labeling of the letters

The results obtained were not very satisfactory. One major problem is that the filtering of lines initially performed also removed some of the details of the letters. Some letters were even cut and no longer has the same form.
There are some morphological operations that might alleviate some of this problems. However, the basic operations applied here were insufficient, or probably, there are structuring elements that are more appropriate.
Furthermore, it is important from the start that the document image to be processed have relatively defined edges of the letters. Some blurring upon capturing the image of the document can really affect the segmentation because letters would be hard to isolate from adjacent letters. There is also a better way of providing this by using, for example, a deblurring filter (or deconvolution), that would help increase the details, producing a better resolution of the image.

Cropped image document page.


Instances of the word description highlighted by the peak points of this image

Using a sample image of the word DESCRIPTION from the filtering and binarized results previously obtained, the instances of the word is highlighted in the cropped image document shown above. Correlation was applied to detect the instance of the word with the sample DESCRIPTION image as the basis template.

For this activity, I would like to give myself a grade of 9 because, even though there are significant applications of the image processing techniques previously studied, the results obtained were somewhat unacceptable. It is important, however, that the techniques studied were implemented in this activity in the proper manner (and right instance), to some extent.

I would like to acknowledge Winsome Chloe Rara, Jay Samuel Combinido, Mark Jayson Villangca and Miguel Sison for some discussions regarding this activity. I would also like to thank Dr. Maricor Soriano for the guidance, as our professor.

Activity 12 - Color Image Segmentation

From the early activities, thresholding an image is one way of segmenting the region of interest from the background. However, this cannot be easily applied for colored images with an obtained grayscale that is almost uniform. That is, the region of interest or the object of study has the same brightness as the background.

One way to solve this problem is to exploit the color information contained in the image. Many objects of important study can have distinct colors that can separate itself from its surroundings. However, since three-dimensional objects can have shading variations of its color, images are usually rendered in their normalized chromaticity coordinates, which describe its pure color information. Normalized chromaticity coordinates are obtained for each channel (RGB) by dividing the value for that channel with the sum of the values in three channels. As a consequence, the image can be represented by just two coordinates, usually r and g. An example of a normalized chromaticity coordinate space is shown below, with r and g as the x and y coordinates, respectively.

Normalized chromaticity coordinate space

From the normalized chromaticity coordinates, color image segmentation is applied by obtaining the probability in which a pixel or a point in the image belongs to the color distribution of the region or object of interest. With this in mind, there are two methods, which basically generates the histogram (distribution) of color of the region of interest: Parametric and Non-parametric probability estimation.

Parametric probability estimation assumes a Gaussian distribution of the color of the region of interest. By obtaining the statistics from the region of interest (mean and standard deviation), the probability of a pixel falling into this distribution is easily calculated. Of course, the total probability is based on the probability of both falling into the distribution of r and g. The joint probability is taken which is the product of the probability for r and the probability for g.

On the other hand, non-parametric probability estimation uses a more accurate description of the distribution of the color of the region of interest. The actual (2D) histogram of the r and g values of the region of interest is obtained. Given the values of the r and g values of a pixel, the pixel is replaced with the value in the histogram corresponding to the r and g values. This histogram backprojection method basically uses the histogram of the region of interest as a "look-up table."

Segmentation can have significant applications for studies regarding colored objects. One major study is about corals. As a demonstration, sample images are obtained and segmented with the region of interest as the corals. This can help in studying coral growth and, possibly, in connections with specific marine species studies.

Original and its binary image


Parametric probability estimation results and its binary image


Non-parametric probability estimation results and its binary image

This image shows a set of corals displayed. The coral that chosen to be segmented here is the red one. A patch from the image of the red coral was taken as the basis for which the normalized chromaticity coordinates distribution for both parametric and non-parametric probability estimation would be obtained. Notice from the binary image of the original that thresholding is not sufficient to highlight and segment only the red coral. Applying color image segmentation, both methods showed high detection of the red coral. This demonstrates a good segmentation result. However, there are still portions of the orange coral that is unintentionally segmented. The non-parametric method result suggests that it has a more robust method. It provides lower segmentation for the orange coral.

Original and its binary image


Parametric probability estimation results and its binary image


Non-parametric probability estimation results and its binary image

Color image segmentation is now demonstrated here for an image of a site of corals in the sea. In wanting to study the areas where the orange coral grows, a patch was chosen from this coral. Segmentation was done and from the results shown above, it is very effective in isolating the coral from the background. The non-parametric probability estimation result is also verified to be better than the parametric results, because it is much more specific in segmenting the orange coral.

For this activity, I would like to give myself a grade of 10 I obtained very desirable results. I also thought of, somehow, a significant object of study to be processed (corals).
I acknowledge our professor, Dr. Maricor Soriano, and my classmates, Winsome Chloe Rara and Mark Jayson Villangca for discussions regarding this activity.

Activity 11 - Color Camera Processing

Color signals detected by colored cameras depend on the light source illuminating the scene or the objects. Images captured by camera are usually applied with corrections for objects to appear in their natural color. This is usually done by white balancing. Various white balancing algorithms have already been established. Colored cameras apply white balancing, sometimes automatically. Cameras also provide manual settings for white balancing, which is usually described by common light sources or settings such as sunlight, fluorescent lamps, etc.

In this activity, two automatic white balancing algorithms are implemented: white patch and gray world algorithms. The white patch algorithm basically takes the RGB values of a known white object in the image and subtract these values to each corresponding channel of the image. In effect, the white patched object would be rendered white and the whole image would be white balanced. On the other hand, the gray world algorithm takes the average RGB values of the whole image, and divide the whole image channels using these values. These algorithms make sense because the spectrum or color of light illuminating the scene is basically cancelled out upon division.

The two algorithms were used to white balance a set of images captured using a Motorola V3 camera set to different lighting conditions (white balancing setting; assumed incorrect), namely, cloudy, home, night, office and sunny. Two set of images were taken: (1) for a set of objects with bright colors representing the major hues (red, orange, yellow, green, blue and violet) and (2) for a set objects with the same hue, but different values and saturation (red). A white object was placed with the two sets, which would serve as the white patch. It is important the images are captured with a fixed light source illuminating the objects. Furthermore, images obtained should not be saturated. It can even be taken with very low exposures and it would still contain most , if not all, of the color information. (M. Soriano, AP 186 Activity 11 Manual, 2009)

Resulting white balanced images were obtained and shown below. The original images show that the white object no longer appears white because of the incorrectly set lighting condition of the camera used for capturing. It actually appears with the same color as the assumed setting from the lighting condition. This represents the reflection of the white object of the spectrum of the light source. The white balanced images resulting from the two algorithms correctly rendered the white object white. This should also ensure that the colors of the other objects are accurately obtained and displayed.

***The results are presented with the original unbalanced image at the top and the white patch algorithm result (left) and gray world algorithm result (right) at the bottom.

Major Hues Representation

Cloudy

Home




Night




Office




Sunny

Single Hue Representation

Cloudy




Home




Night




Office




Sunny

Intuitively, the white patch algorithm provides a better white balancing compared to the gray world algorithm. This is because the white patch algorithm is much more specific such that the white object is to be made white. The difference is much more evident in the results for the single hue set of objects. Gray world algorithm takes the average of the whole image, which makes it suffer if there is a dominant color in the image. The spectrum of the light source would not have a significant effect on the average because the majority of detected signal belongs to the dominant color. This would not be a problem for the white patch algorithm because there is a reference white patch or object that is much more specific. It would not be affected by any dominant color in the image because it still is based on the white patch.

However, as can be seen in the results for the set of images of the major hues, the gray world algorithm is sufficient enough for white balancing without any dominant color. Without the dominant color, the light source color would be the dominant signal that would account for the average RGB values of the image.

The significance of the gray world algorithm is that it doesn't need any reference white patch. It can be applied to any image directly. The white patch algorithm needs the reference white object making it more inflexible.

For this activity, I would like to give myself a grade of 10 for having relatively pleasing results. Moreover, I think I have provided an extensive analysis of this results.

I would like to thank our professor, Dr. Maricor Soriano for the discussions regarding this activity, and some of my classmates for preliminary images taken (camera and laptop use).