1 optimal halftoning for tactile imagingwebstaff.itn.liu.se/~sasgo26/digital_halftoning/...1 optimal...
Post on 18-Jun-2020
6 Views
Preview:
TRANSCRIPT
1
Optimal Halftoning for Tactile ImagingAmit Nayak,Student Member, IEEEand Kenneth E. Barner,Senior Member,
IEEE
Department of Electrical and Computer EngineeringUniversity of Delaware, Newark, Delaware 19716, USA
nayak@ece.udel.edu, barner@ece.udel.edu
Abstract
Reading of text and understanding images by touch is an important alternative and additional source of information
when sight is absent or lost. Tactile graphics and models such as edge maps, binary output etc. are the solution for
simple access to images for blind persons. This paper introduces an approach to model the human tactile system based
on the responses produced by stimuli on micro-capsule paper. This system is utilized for the purpose of generating
optimum halftone patterns on micro-capsule paper that can be utilized for the effective generation of tactile graphics.
I. INTRODUCTION
Vision plays a pivotal role in information gathering for human beings. For instance, much of
the sciences involve the visualization of materials and concepts. Valuable information in text and
images is thus gathered by humans through the visual sense. This reliance on vision is hampered,
or lost, in individuals with visual impairments. In such cases, it is necessary for the information
display to include alternative modalities.
The main system of written communication utilized by blind people throughout the world is the
Braille system. Invented by Louise Braille in the early nineteenth century, this system proved to
be better than the earlier, difficult and cumbersome, method of embossed letter reading. The use
of Braille method for reading and writing, has been researched at depth for its utility by various
researchers [1]. This method, however, allows tactile access to only text based information.
Tactile imaging is the process of converting a visual image into a touchable binary raised ver-
sion. Applications of tactile graphics have been researched on in several places. Tactile pattern
generation received a boost with the invention of Optacon and the subsequent modification [2],
2
[3]. Loomis [4] and Craig [5] initiated a discussion on the tactile pattern perception and the fac-
tors which could affect this perception. One of the earliest groups to look in to the sensory aspects
of skin in the human finger, Johnsonet al. [6], [7], [8] worked on the the discrimination and
detection of points, gaps, gratings and letters by the human finger. Martinet al. [9] have worked
in the presentation of business graphics for blind people. Pie charts, bar charts and other form
of representation of information in image format have been successfully converted into relevant
form. The data on which the graphical information is based is extracted from the image and then
the tabulated version of this data is conveyed to the user either by speech, sound and/or tactile
form. Speech-synthesizer, using acoustically different frequency tones are some ways and in the
case of bar charts the most favored way is using a 2D-Braille-Display, wherein raised dots are used
to depict the length of the actual bar in the y-axis and the x-axis is coupled as a Braille output.
Klatzky et al. [10], in 1985, discussed the identification of objects by touch and have subse-
quently compared the amount of information garnered by the visual and the haptic interface [11]
in 1987. Similarly the same time investigators had attempted to quantify the information channel
capacity of the human sensory organs and Kokjer [12] suggested an order-of-magnitude upper
limit on this property for the vibrotactile stimulation of human fingertip as 100 bit/s. This, when
compared with corresponding limits in the human ear and eye, a progression of102:104:106 re-
spectively, was found. Thus the capacity of visual sense to receive and perceive information is the
highest, as compared to hearing and touch.
Of the various studies and research done in the field of generation of tactile graphics, of particu-
lar interest are that of [13], [14], which successfully developed software algorithms for automatic
generation of tactile graphics. Meaningful information is identified and extracted from original
image data incorporating a multi-step procedure. These include edge and boundary information
extraction from the original image and representing them in tactile format. Edge and boundary
information refers to identification of regions in an image where a rapid change of intensity takes
place. Also important in the procedures are the successful segmentation of the original image.
These image processing methods were further developed by Hernandezet al. [15], [16], [17] who
3
used the edge information to generate graphics that rendered image boundary in tactile form.
Edge information on its own is, however, not enough to present the complete effect. A tactile
image should contain different textures [18] to help distinguish and discriminate between various
shapes and colors present in an image. We, therefore, propose optimal tactile imaging textures.
The underlying idea is that the various regions in segmented images can now be easily identified
by touch if they are given a separate textures. To accomplish this, we use digital halftoning methods
to create patterns for the images which can then be suitably enhanced to create the feel of various
textures.
Digital Halftoning is the process of generating a binary pixel patterns that create the illusion of
a continuous-tone image. This technique is important for display of gray-scale images in printing
process, where the direct rendition of the gray tones is not possible. In recent years a number
of comprehensive methods have been designed to carry out the halftoning process. The primary
methods are reviewed in [19], [20]. Optimum algorithms [21], [22] were found to be those that
exploit the dot-printer models and take into account the low pass nature of the human vision.
We extend this idea of model-based halftoning to the domain of the human tactile system. As
used extensively in the human visual system (HVS) modeled halftoning system, a transfer function
for the sensory system is required along with an understanding of the printing process. In an anal-
ogous fashion, we develop a model for the human tactile system and the tactile printing process.
In the case considered, we utilize the Tactile Image Enhancer, Fig. 1, operating on microcapsule
paper [13], [14], [23] as the tactile output. The output characteristics of the TIE were examined.
We develop a model for the surface generated from the TIE. We also devise a model of the human
tactile perception based on the spatial frequency response of the human tactile sense. These models
are integrated into the process of generating halftones best suited for a given gray scale.
The remainder of this paper is organized as follows. The working of the TIE and the measure-
ments of the output characteristics and resulting enhancement model are presented in Section 2. In
Section 3, we discuss the work carried out earlier on the characterization and specification of the
human finger pad. New experiments are carried out to find how sensitive the human finger is to
4
Fig. 1. Tactile Image Enhancer (TIE is manufactured by Repro-tronics Inc.).
the surface of the tactile paper, the results of which are reported in Section 4. Optimal halftoning
methods are discussed in Section 5. Finally, the conclusions and further research are presented in
Section 6.
II. I MAGE ENHANCEMENT PROCESS
Tactile images can be formed by enhancing, or raising, the surface of a special paper called
tactile or micro-capsule paper through use of the TIE, Fig. 1. The TIE is manufactured by the
Repro-tronics Inc. The first prototype was introduced in the market in 1993 and have been in use
since then. As of now there is a smaller, more portable version of TIE available in the market. The
TIE works on a very simple principle. It has a motor-driven roller that passes the tactile paper, face
up, underneath a tubular light bulb, Fig. 2. The light bulb acts as a heat source and the rollers help
in guiding the paper through the direct contact of the bulb.
The type of tactile paper which we intend to focus on is manufactured by the Matsumoto Kosan
Company Ltd. in Japan. This is also known as the micro-capsule paper. As the name suggests it
is a white paper coated with microscopic polystyrene capsules. The Micro-capsule coating on the
paper is heat reactive. Black lines or images printed on the paper, when exposed to a heat source,
absorb more heat than the surrounding areas. This causes the underlying capsules to grow. The
result is the generation of raised lines, areas, and symbols,i.e., tactile graphics. A temperature
range of 120-125�C (248-257�F) is appropriate for the heating of the papers. In the present study
5
Image (printed toner particles)
Polystyrene Microcapsules Polyethylene medium
(a)Heating element
Paper path
Transport rollers
(b)Expanded Capsules
(c)
Fig. 2. (a) Micro-capsule paper with image toner particles photocopied on it.(b) Inside the TIE.(c) Raised micro-capsules after the paper is passed through the TIE
we have preferred to use 124�C(255.2�F).
In order to understand and model the changes occurring in the enhancement process, a simple
experiment was carried out. An array of test samples of dots were used. The dot were square
pixels printed on a white paper at a resolution of 100 dots per inch (dpi). The size of the dots was
uniformly increased from a1 � 1 to a20 � 20 dot cluster. The resulting outputs were examined
for their change in dimension. Images were prepared, comprising of black square dots on a white
background and black dots with square gaps simulating white square dots on black background,
Fig. 3. These images were photocopied onto the tactile paper. The tactile paper was then passed
through the TIE.
6
(a) (b)
Fig. 3. (a) Black Dots on white background. (b) White dots on black background.
The structures such generated through this test have profiles that range between -0.5 and 0.5
mm in height. A Scanning Electron Microscope(SEM) was used to view the vertical growth of the
samples. The profiles of the samples were captured and recorded. The profile of one of the black
dot on white background, with a dot size of8 � 8, is shown in Fig 4. Simple image processing
techniques were used to extract the edge information and these were superimposed to see the
growth of the dot clusters as a function of cluster size.
Fig. 5 shows one-dimensional profiles for squares of sizes 1� 1, 2� 2,... etc which are printed at
a resolution of 100 dpi. The inverse case, which we can observe as white dots in black background,
follow a complementary pattern. The growth in both directions were recorded. The average height
or depth of the dot profiles were found to follow the curves shown in Fig. 6.
The clusters of dots are characterized by the way the height/depth and width of the dots behave
after passing through the TIE. The curves of the profiles are modeled as follows:
Fu = Au +Buln(i) (1)
Fd = Ad(e�(i�Bd)
2=Cd � 1) (2)
whereFu andFd denote the height and depth parameters respectively, and the constants were
determined experimentally to beAu=0.026,Bu = 0:16 andAd = 0:5,Bd = 0:75, Cd = 1:596 and
i denotes the size of the cluster of dots. Thus, the height and depth parameter of the dot clusters
7
Fig. 4. The Test pattern corresponding to the 8 x 8 pattern as seen through the SEM.
−0.08 −0.06 −0.04 −0.02 0 0.02 0.04 0.06
0
0.005
0.01
0.015
0.02
0.025
Width of clusters of dots (inches)
Hei
ght (
inch
es)
(a)
−0.04 −0.02 0 0.02 0.04−0.025
−0.02
−0.015
−0.01
−0.005
0
0.005
Width of clusters of dots (inches)
De
pth
(in
che
s)
(b)
Fig. 5. Edge variation for the (a) black dots on a white background, (b) white dots on a black background.
have been modeled either as exponential or logarithmic curves.
8
0 1 2 3 4 50
0.1
0.2
0.3
0.4
0.5
Width of dots (mm)
Hei
ght o
f dot
s (m
m)
Experimental dataLogarithmic curve fit
(a)
1 2 3 4 5
−0.5
−0.4
−0.3
−0.2
−0.1
0
Width of dots (mm)
De
pth
of
do
ts (
mm
)
Curve fitExperimental data
(b)
Fig. 6. (a) Height variation of black dots on white background. (b) Depth variation of white dots on black background.
Consider next the profiles of the dot clusters. On a simple suitable observation, we find that the
profile of a single pixel is approximated by a Gaussian surface as shown in the Fig. 7:
G1;1 = Ae�x2=B (3)
where the constants were determined experimentally to beA = 0:00134, B = 0:02752 andx
represents the distance along the width. Larger clusters are modeled as linear combinations of
dots. Thus an� n cluster is modeled through an amplitude adjusted convolution model:
Fn;n = n;n(G1;1 In;n) (4)
9
−0.015 −0.01 −0.005 0 0.005 0.01 0.0150
5
10
15x 10
−4
Width of dot (inches)
He
igh
t (in
che
s)
Fig. 7. Profile and the model for a unit impulse comprising of dot of size 1x1.
−0.06 −0.04 −0.02 0 0.02 0.04 0.06
0
5
10
15x 10
−3
Width of dot−cluster (inches)
Heig
ht (i
nche
s)
Experimental dataModel fit
Fig. 8. Model fitted curves for the black dots on white background
whereFn;n is the model suitable for an enhanced profile of ann� n cluster of dots andn;n is the
average attainable height/depth of the profiles for ann � n cluster dot, as shown in Fig. 5(a) and
Fig. 5(b).In;n is the collection of impulses representing a cluster of dot of sizen� n. These have
been plotted in Fig. 6. The resulting modeled curves are found to follow the profiles as predicted.
The modeled curves superimposed with the measured curves are seen in Fig. 8. This kind of
modeling can also be achieved for the white dots on black background.
10
III. TACTILE RESOLUTION
Images that are enhanced using the micro-capsule paper processed by the TIE are ready to be
used as tactile graphics. It is necessary, however, to understand how human subjects respond
to the structures on the micro-capsule paper. A group of experiments were carried out to arrive
at a mathematical model for the human tactile system and its response to the structures on the
micro-capsule paper. The spatial resolution of the dots and the ability to produce a dot profile of
substantial distinction as well as the frequency response of the tactile system is investigated in the
experiments.
A. Review of previous work
The discussion on understanding the tactile pattern perception and the factors affecting this
perception was started by Loomis [4] and Craig [5]. Johnsonet al. [6], [7], [8] began the
important exploration of the sensory aspects of the human skin. More recently, investigation was
carried out by Pawluket al. [24], [25] to develop an engineering model describing the response
of the human peripheral tactile system when an object(a flat indentor) dynamically contacts the
finger-pad. Here the system was measured at both mechanically, at the surface of the skin, and
neurophysiologically, as the resulting nerve signal heads towards the brain.
The investigation made into the spatial neural mechanisms underlying the tactile sensation by
Johnsonet al. [6], [7], [8] is divided into three parts. In the first part, specifications of the dis-
crimination behavior that depends strictly on spatial neural mechanism is discussed. Experiments
are carried out in this regard to define which aspects of the spatial discrimination are based on spa-
tial information in the neural discharge patterns. It was found that human subjects could reliably
discriminate between one and two 0.5 mm (0.019 in.) diameter points even when there was no
separation between the two points. A high level of spatial resolution was thus demonstrated.
Another set of experiments were carried out in which the subjects were required to discriminate
between stimuli with and without gaps. A success criteria of75% was used and it was found on this
basis that gap size at the threshold of edge detection was 0.87 mm(0.034 in.). Subjects were also
11
required to feel square-wave gratings that were oriented along or across the axis of the finger. It
was found that grating orientation was difficult to decipher until the gaps exceeded 0.5 mm(0.019
in.). The gap size, which confirmed with the success criteria of75% accuracy, was found to be
0.84 mm(0.033 in.).
In the second part of the experiments a similar study was carried out to find the neural responses
in the monkey finger pad skin. In the third series of experiments a mechanistic model of skin is
developed to predict the stress and strain at mechanoreceptor terminals within the skin.
A parallel approach made by Lamotte and Whitehouse [26], discusses the capabilities of humans
to detect the presence of a single raised dot of550�m(0.021 in.) diameter on a smooth plate and
to judge the magnitude of evoked sensation. The corresponding stimuli on three different kind of
mechanoreceptive peripheral nerve fibers of the monkey finger-pad (namely the slowly adapting
(SA), the rapidly adapting(RA)and the pacinian (PC) mechanoreceptive peripheral nerve fibers )
was studied for dots of different heights.
The mean detection threshold was found to be about 2.1�0.3 �m (9.4�10�5 in). Also it has
been found that a height of 1�m (3.937�10�5 in ) could be detected if the diameter of dot was
600�m (0.0236 in) or wider. In similar fashion a height of 6�m (2.36�10�4 in) was required
for detection if the dot diameter was reduced to 40�m (1.5748�10�3 in). In another experiment,
stroke velocities of about 10 mm/s (0.393 in/s) were used and the dot detection thresholds were
found out to be between 1(3.937�10�5 in) and 3�m (1.181�10�4 in) for all human observers.
This again was different for different mechanoreceptive nerve fibers.
It was reported that the magnitude of sensation evoked in humans increased with an increase
in dot height above the dot detection threshold as defined in the preceding discussion. It was
found that the impulses evoked in monkey RAs increased with dot height as did the widths of RA
receptive fields. Also, it was hypothesized that the mechanical event responsible for the activating
the RA was the lateral deformation of elevated regions of skin. This was found to be true because
the number of impulses evoked in RAs by a dot was greater when the dot was stroked across, as
opposed to along, the papillary ridges. It is conclusively proved that the responses of RAs alone
12
account for the sensory capacity to detect dot of minimal height on a smooth surface with the finger
pad.
In all these experiments described above the object to be felt by the finger-pad is an ideal system
with flat surfaces and definite structures. The enhanced micro-capsule paper does not suit an ideal
system features and hence the results cannot be directly used in its case.
B. Discrimination of dots
This experiment simulated the work carried out by and Whitehouse [26] on the micro-capsule
paper. Test patterns of individual dots of sizes ranging between a single dot and a cluster of20�20
dots were created as shown in Fig. 3. The human subjects were required to feel the patterns and
respond on the detection of the presence or absence of any structures. Sufficient time was provided
to each subject to feel the structures and then define the response. The micro-capsule paper and the
enhanced structures on it were shielded from the subject by blindfolds. The tactile paper sample
containing these test material were made accessible to the human subjects in random order.
The data were derived from experiments conducted on 8 subjects aged between 22 and 30 years
of age. The discrimination of dots experiment tested the ability of subjects to distinguish the
presence or absence of stimuli on the surface of the micro-capsule paper. Both black on white and
white on black dot patterns were used.
Psychophysics is extensively used to answer questions regarding the way human beings react
to external stimuli, such as variation in specified characteristics of environmental stimulation. Ex-
periments usually reveal typical S-shaped curves [27]. The results for the conducted experiments
show that the detection curves follow typical S-shaped curves, Fig. 9.
The threshold value is typically taken as the stimulus value corresponding to the detection re-
sponse of50%. The threshold for the black dot detection in this study was found to be 0.4 mm. The
detection of white dots in black background was found to be around 1.9 mm. Hence there exists
a difference in the detection threshold of the dots in different background. These are attributed to
characteristics and resolution of the tactile paper. The gaps in black background, which are seen
as the white dots, do not produce noticeable depth for small dots. This is because the black dots
13
0 0.5 1 1.5 2 2.5 30
20
40
60
80
100
Stimulus dimension (mm)
Pro
ba
bili
ty o
f fe
elin
g d
ots
(%
)
Black dotsWhite dots
Fig. 9. Detection curves for dots
cause an enhancement that raises the material inside the square block, thereby negating any effect
of depth. However for bigger clusters of white dots, the problem is not as pronounced.
C. Frequency Response
The frequency response of the human tactile system is another way of understanding the way
human subjects react to the raised structures on the tactile paper. There has been substantial re-
search in the designing and conducting of psychophysical experiments [28], [29] to analyze the
human visual system. The basis of the study was to understand what happens to the spatial infor-
mation1 after they are received by the eye. Since the goal is to understand the processing of data
flow within the tactile system, the visual experiments provide us with a good model to carry out
the tactile system experiments.
Visual perception is studied by recording and analyzing the response to sine wave gratings. Spa-
tial sine wave gratings (whose luminance varies sinusoidally across space) are simple building1Spatial information refers to the brightness and constant attributes across the space
14
(a) (b)
Fig. 10. Psychophysical experiments using gratings. (a)Gratings for human visual system tests; (b)Gratings for thehuman tactile system tests
blocks for complex stimuli and they are influenced by frequency, contrast, phase and orientation.
To determine a psychophysical measurement of the sensitivity of humans to the sine wave grat-
ings, human subjects are presented with a gratings of a given spatial frequency, Fig. 10(a). The
frequency range is between 3 and 48 cycles per degree of visual angle [29]. Initially the subject
cannot see a grating as the image of the grating is below threshold. The contrast of the grating is
increased till the subject reports the sighting of the grating. The sensitivity for the particular spatial
frequency is the inverse of the contrast threshold. This procedure is repeated for a large number of
frequencies. The resulting curve plotting contrast sensitivity as a function of spatial frequency is
known as the contrast sensitivity function (CSF).
A number of models have been discussed [30] to model the CSF. The dependence of the contrast
sensitivityShvs on the radial spatial frequency,fhvs was developed by Campbellet al. [31], [32].
It can be written as follows:
Shvs= �[e�2��fhvs � e�2��fhvs] (5)
where�, � and� are constants. The constant� is proportional to the average illumination. The
constants� and� are set depending on the type of model chosen during the course of the experi-
ment.
A similar approach was carried out in the current study to determine the tactile sensitivity func-
15
0 10 20 30 40 50 60 70 80 900
10
20
30
40
50
60
70
80
90
100
Spatial Frequency (cycles/paper)
Ta
ctile
Se
nsi
tivity
Experimental dataModel fit
Fig. 11. Tactile Sensitivity Curve for human beings.
tion (TSF). Gratings such as that shown in Fig. 10(b) were utilized. Non-blind subjects were
blindfolded and presented with square-wave gratings, which were sufficiently raised after passing
through the TIE. These gratings were at a fixed spatial frequency. The spatial frequency used were
in the range between 1 to 30 cycles per unit of space. Noise was introduced in the square gratings
to simulate the change in signal to noise ratio or the contrast of the gratings. The distribution of
the noise varies exponentially with space. At one edge the noise power is very high, so that the
subjects could not feel the gratings. The noise level was reduced until a point is reached where the
subject reports feeling the gratings. The corresponding noise level was recorded and treated as the
sensitivity value for the particular spatial frequency. This experiment was repeated for a number of
different spatial frequencies. The resulting spatial sensitivity values are plotted against the width
of the gratings, as shown in Fig. 11. The measured human TSF is a band-pass function, as it shows
a peak and decreasing sensitivity on either side of this peak.
A model for the tactile system utilizing the TSF data generated from the experiments is given
16
0
5
10
15
20 05
1015
20
40
45
50
55
60
65
70
75
Tac
tile
Sen
sitiv
ity
Fig. 12. Human Tactile Sensitivity model. The x-axis is the horizontal spatial frequency (cycles/mm) and the y-axisis the vertical spatial frequency (cycles/mm)
by:
Shts= (qfhts=C)� (e�fhts=D) (6)
where the constantsC andD are calculated from the empirical data to be4:65 and43:245, respec-
tively. Also, fhts is the radial spatial frequency in cycles per unit of space. The resulting model is
shown in Fig. 12.
IV. TACTILE HALFTONE STRUCTURES
In printing devices where the direct reproduction of all the gray tones is impossible, patterns of
binary pixels or halftones are substituted for the original image, in the process creating a visual
illusion of similarity. This is possible due to the low-pass nature of the human eye, which averages
black dots and white space to perceive a particular gray level. A similar band-pass response of the
human touch sense needs to be taken into account when we are trying to devise a tactile imaging
scheme based on halftones. In conventional halftoning using model-based techniques, the CSF of
the human visual system is utilized to compare the halftoned image and the original continuous
tone image. Our intention is to effectively map this idea to the tactile domain.
17
Image
Process Model
Halftoned Image
TactileTIE
TIE
Error
Human Tactile
System
TactileHuman
System
Halftone
Image
Fig. 13. Block Diagram for the optimal tactile imaging process.
Figure 13 shows the implementation of the proposed model based halftoning process for tactile
imaging. This model is similar to the model-based halftoning method reported in [21], [22], [20].
We have substituted the human tactile system and the TIE for the human visual system and the
printer, respectively.
The model details the algorithm to produce the halftone patterns that are optimum for a par-
ticular halftone process. The original image is passed through an initial halftoning process. The
model based halftoning requires a quantitative comparison between the tactile-perceived original
image and the tactile-perceived halftoned image. The tactile-perceived original image is obtained
by filtering the original image with a band-pass filter modeled according to the tactile sensitivity
function, Equation ( 6) in Section III-C. The halftoned original image is passed through a model
simulating the TIE. This transforms the distribution of dots to the simulated output of the TIE by
taking into consideration the Gaussian surface model (Eq. 4) of the TIE. At this stage, the mini-
mum criteria for recognition of the black and the white dots, as discussed in Section III-B, is used
to screen the halftoned image. The halftoned original image is converted to raised halftone struc-
tures with the necessary constraints on dots characteristics. Finally, the tactile-perceived halftoned
image is the result of output after filtering the raised halftone structures with the tactile sensitivity
function band-pass filter model. The comparison of the two perceived images produces an error
that is then superimposed on the halftoned image to arrive at an optimal dot arrangement.
18
The successful implementation of the model-based halftoning and the production of the opti-
mum halftone structures requires a choice on the type of halftoning method. The following section
discuss the advantages and shortfalls of the common halftoning techniques available. These meth-
ods are then compared with each other through a set of experiments.
A. Halftoning methods
Over the years a number of effective methods have been proposed in the field of halftoning.
Ordered dither, or amplitude modulated (AM) halftoning [19], is one of the simplest methods.
This method of creating an illusion of a continuous tone image is produced by varying the size
of the dots that are printed along a regular lattice. The number of dots, or frequency of the dots,
remains fixed. The original continuous-tone image is divided into blocks of smaller sizes that are
then compared to a threshold in a pixel by pixel comparison. The dither array is composed of
consecutive thresholds clustered together.
Floyd and Steinberg [33] developed a halftoning technique, known as error diffusion, that dis-
tributes the error generated at each pixel. This became the foundation of the present day frequency
modulated (FM) halftoning. In FM halftoning, the required illusion is achieved by varying the
distance between printed dots while the dots are held at a constant size. This process is an adaptive
algorithm that quantizes each pixel. The quantization error produced in each of the single pixel
operations is distributed amongst the neighborhood of yet to be processed pixels.
The pattern generated by this process was referred by Robert Ulichney as the blue-noise halftones
[19]. This name results from the fact that the dither patterns is composed exclusively of high fre-
quency spectral components. For a given gray levelg, the minority pixels are separated by an
average distance of�b, which is the principle blue noise wavelength. The relation between the two
parameters is given by:
�b =
8<: D=(
q(g)) for 0 < g � 1=2
D=(q(1� g)) for 1=2 < g � 1
(7)
where D is the minimum distance between addressable points on the display. The error diffusion
algorithm devised by Floyd and Steinberg, shown in Fig. 14, can be mathematically represented
19
X[n]
X[n]
Y[n]
Y[n]e e
+
+
+
−
B
Fig. 14. Error diffusion (Floyd and Steinberg, 1976)
as follows:
Y [n] =
(1 if (X[n]�BTY e[n]) � 00 else
(8)
whereB = [b1; b2; :::; bN ]T ,PN
i=0 bi = 1, the outputY [n] = [y[n � 1]; y[n � 2]; :::; y[n � N ]]T
andY e[n] = [ye[n � 1]; ye[n � 2]; :::; ye[n � N ]]T , which follows from the equationY e[n] =
Y [n]� (X[n]� (BTY e[n])). The input pixel under consideration in this expression isX[n].
An ideal printer outputs patterns that are composed of perfect square black dots. In high quality
printing situations, where this effect is true to certain extent, blue-noise halftoning is considered the
“optimum technique for minimizing visibility [34] and maximizing the apparent spatial resolution
[35]”, as reported in [20]. In practical considerations involving digital printing, printer distortions
are relevant. They are summarily categorized as dot-gain and dot-loss [20]. Dot-gain is the
increase in size of the printed dot relative to its intended size, and is the main distortion in ink
jet printing. The quality of paper also has a very crucial effect, with denser papers causing ink
to spread instead of absorbing it completely. Dot loss appears in higher resolution printers where
instead of creating a bigger dot, there is a difficulty in printing an isolated black dot. Model
based halftoning and clustering of dots are techniques that have been developed to address these
problems.
The halftoning process that are of particular interest in the present context of clustering are the
AM-FM hybrid stochastic halftoning techniques. These create minority pixel clusters that vary,
according to tone, in both their size and spacing. The spectral characteristics of such patterns have
20
X[n]
X[n]
X[n]
Y[n]
Y[n]e
h
e
+
+
++
+
-
H A
B
Fig. 15. Error diffusion with output-dependent feedback: a weighted sum of the previous output pixels is used to varythe threshold. H-hysteresis term, A-hysteresis filter, B-error filter, X[n]-input image, Y[n]-halftoned image.
mid-frequency components. Hence they are collectively known as green-noise halftone patterns.
In green-noise, the minority pixel clusters are distributed homogeneously. The average separation
(center-to-center) between clusters is termed as�g. Its square is inversely proportional to the
average number of minority pixel clusters per unit area. The green-noise principle wavelength
[20] is given by the following equation.
�g =
8<: D=(
q(g)= �M) for 0 < g � 1=2
D=(q(1� g)= �M) for 1=2 < g � 1
(9)
where D is the minimum distance between addressable points on the display,g is a particular gray
level and �M is the average number of minority pixels per cluster in the binary dither pattern.
The green noise halftone is generated by a error diffusion technique proposed by Levien [36].
He referred to it as the error diffusion with output dependent feedback, Fig. 15. In this algorithm a
weighted sum of the previous output pixels is used to vary the threshold. This makes the minority
pixels more likely to occur in clusters. The amount of clustering is controlled through the hysteresis
constantH. Large values of H can cause large clustering and smaller values lead to smaller clusters.
Levien’s algorithm is precisely defined as follows:
Y [n] =
(1 if (X[n] +BTY e[n] +HATY [n]) � 00 else
(10)
21
whereA = [a1; a2; :::; aN ]T , B = [b1; b2; :::; bN ]
T ,PN
i=0 ai = 1,PN
i=0 bi = 1, the outputY [n] =
[y[n�1]; y[n�2]; :::; y[n�N ]]T andY e[n] = [ye[n�1]; ye[n�2]; :::; ye[n�N ]]T , which follows
from the equationY e[n] = Y [n] � (X[n] + (BTY e[n])). The error filter and hysteresis filter can
take on a wide range of values, including special cases such as Floyd-Steinberg [33], Jarvis [37]
and Stucki [38] filter coefficients. Perturbation, in the form of noise, can also be added to these
filter weights to create desirable results.
B. Results and Evaluation
Psychophysical experiments can be used to determine the optimum tactile halftone technique.
The measure of goodness is defined as the level of effectiveness in feeling and understanding
information through the manual exploration of the structures. As detailed in Fig. 13, the halftoning
systems defined by the model transfer functions for AM, FM, and AM-FM hybrid methods are
considered.
As a first examination, a gray scale ramp test image is used as an original image and the corre-
sponding halftones are generated by each method. These are shown in Fig. 16. A sample image
having four gray levels is also processed by the three halftone techniques, Fig. 17. Upon observa-
tion of the generated halftone structures, we find certain differences that are particularly important
in the case of tactile halftoning.
First note that, as expected, the halftoned gray scale ramp produced by the AM halftoning
method has a uniform distribution of dots, but varying dot size. Due to the regular spacing of
dots, the (micro-capsule) growth of the dots is inhibited. That is, dots begin to merge once they
are of a certain size. Thus no discrimination can be made once the dots begin to merge. Also, due
to the asymmetry of the micro-capsule enhancement process, black dots and white dots merge at
different thresholds. Accordingly, the tactile printing process places significant limitations on the
use of AM halftoning.
In the case of the blue noise, or FM, halftoning technique, the dots have similar size but are
distributed at varying distance depending on the gray level. Once again, the minimum dot size
imposed by the tactile printing process imposes a constraint. Since the minimum dot size for black
22
(a)
(b)
(c)
(d)
Fig. 16. Test Image: (a) Original Ramp of gray scale; Test image halftoned using the HTS by (b)Amplitude Modula-tion (AM) halftoning (c) Frequency Modulation (FM) halftoning and (d) AM-FM hybrid halftoning
23
(a) (b)
(c) (d)
Fig. 17. Example using a test image (a) Original image (b) AM halftoned (c)FM halftoned (d)AM-FM halftoned
and white are different, both constraints cannot be met simultaneously under the FM structure.
This results in merging of white dots and, as in the AM case, leads to a degradation in the quality
of the resulting halftone.
The green noise, or AM-FM hybrid, halftoning structure allows the minimum black and white
dot size to co-exist, and the resulting halftone thus consists of dots within the experimentally
defined limits. Also, since both spacing and dot distribution are stochastic processes, this method
of generating tactile halftones is more robust to errors in either dot location or size. This results
in halftones where the dots are not merged by the printing (micro-capsule paper enhancement)
process. Thus, the produced halftones are tactilely distinct and more easily discriminated than
either the AM or FM generated halftones.
To test the validity of these visual observations, a set of experiments was conducted. The
halftone methods discussed in previous sections were utilized to create halftones for the simple
24
(a) (b)
(c) (d)
(e)
Fig. 18. Test images halftoned by the AM-FM hybrid method for the edge and region identification experiments: (a)Hexagon (b) Triangle (c) Rectangle (d) Circle (e) image of rock with two gray levels.
geometric figures of hexagon, triangle, rectangle, circle and an image of rock. The structures, with
halftoned patterns generated by the AM, FM, and AM-FM methods, constituted the set of stimuli
for the experiments. Figure 18 shows the structures generated by the AM-FM hybrid method.
25
1 2 3 4 50
1
2
3
4
5
6
7
8
9
10
Geometric Figures
Tim
e(s
ecs
)
AM halftoningFM halftoningAM−FM hybrid
Fig. 19. Timed identification of the geometric figures. The figures numbered 1 to 5 are Hexagon, Triangle, Rectangle,Circle, and an image of rock with two gray levels, respectively
The test patterns were enhanced as detailed earlier with the help of the TIE and were presented to
blindfolded human subjects.
The data were derived from experiments conducted on 8 subjects aged between 22 and 30 years
of age, as before. The micro-capsule paper and the enhanced structures on it were shielded from
the subject by blindfolds.
In the first experiment, the halftones of the geometric figures produced by the three methods
were numbered for our identification and presented to the subjects in random order. The subjects
were asked to recognize the shape of the geometric figures by manual exploration of the figures on
the tactile paper. The time taken by each subject to identify the figure, through manual exploration,
was noted and recorded.
This experiment was undertaken to understand how quickly the human subjects were able to
recognize shapes based on each of the halftoning methods. As each halftoning method results
in a different distribution of dots and subsequently different height/depth for the edge dots, this
26
1 2 3 4 50
10
20
30
40
50
60
70
Geometric Figures
Re
spo
nse
to
str
uct
ure
s(%
)
AM halftoningFM halftoningAM−FM hybrid
Fig. 20. Texture Discrimination.The figures numbered 1 to 5 are Hexagon, Triangle, Rectangle, Circle, and an imageof rock with two gray levels, respectively
recording helps in defining which halftoning method produces better recognition. The average
identification times for each method are shown in Fig. 19. The compiled results showed that, in
all cases, the AM-FM hybrid halftoning method leads to the shortest identification time. Thus
it can be noted that the more distinct structures produced by the AM-FM hybrid methods aid in
identification.
The second experiment tested how well a human subject is able to recognize the details in
the textures of the structures. This was aimed at the texture discrimination rather than the shape
identification, which was the goal in the first experiment. The human subjects were presented with
the three halftones of the same geometrical figure and were asked to make a comparison of the
quality of the structures on their surface.
As before, the subjects were asked to manually explore the figures and report which of the three
halftones presented to them, again in random order, was most distincti.e. easily identifiable by
touch. The experiment recorded which halftone method would give the most tactile perceivable
27
patterns on a surface for a given gray level. Figure 20 reports the responses recorded from this ex-
periment. It was found that, in most cases, the AM halftoning is too weak to produce the necessary
discrimination, most likely due to merging of dots. The FM halftoning, on the other hand, enables
greater discrimination than the AM halftoning. The AM-FM hybrid halftoning method however,
yields the greater discrimination.
The psychophysical experiment results are thus in alignment with the analysis of the algorithms.
The hybrid AM-FM halftoning, or green noise halftoning, is optimal for the generation of tactile
output among the methods available in the literature. The performance of the hybrid method is
likely due to its flexibility, in that it can be jointly optimized according to human tactile perception
and the characteristics of the printing (microcapsule enhancement) process. Both the AM and FM
methods produce suboptimal results, which can be attributed to identifiable shortcomings such as
dot merging.
Considerable work must still be undertaken to fully optimized and evaluate halftoning methods
for tactile representations. For instance, it is not currently known what an appropriate number of
tactile gray level representations is. It is unlikely that a user would be able to discriminate amongst,
say, the 256 gray levels in an 8–bit image, but discrimination amongst a smaller number of halfton-
ing patterns should be possible. Further studies are needed to clarify this issue. Additionally, the
methods must be evaluated on other output medium and displays, such as the TIGER embossing
printer. These issues are amongst those that will be the subject of further investigations.
V. CONCLUSIONS
This paper introduced the concept of digital halftoning for tactile imaging. To develop appro-
priate tactile halftoning methods, the characteristics of the Tactile Image Enhancer (TIE) were
studied in detail. This study led to an accurate model of the microcapsule paper enhancement
process. Additionally, experiments were conducted to determine human tactile sensitivity and fre-
quency response for the case of manual exploration of microcapsule paper. The TIE model and
human tactile response were integrated into a general tactile halftoning procedure. This procedure
includes as special cases the widely used AM and FM halftoning, as well as the more flexible
28
AM–FM hybrid green noise technique. Each of the methods were analyzed for their applicabil-
ity in tactile halftoning and sample outputs were generated. Psychophysical experiments on timed
identification and discrimination were conducted. The algorithm analysis and experiments indicate
that the AM–FM hybrid method produces the most tactilely appropriate halftoning patterns. Fur-
ther investigations will be conducted to more fully evaluate the developed methods to determine,
for instance, the number of halftoning patterns that can be discriminated by users, and additional
research will be conducted to optimize the procedures for other tactile display media.
VI. A CKNOWLEDGMENT
This work was supported by the National Science Foundation under the grants 9800175 and
9875658.
REFERENCES
[1] S. Millar, Reading by Touch. Routledge, New York, USA, 1997.[2] J. Bliss, “A relatively high-reslution reading aid for the blind,”IEEE Transactions on Man-Machine Systems, vol. 10, pp. 1–9,
1969.[3] J.G.Linvill and J. Bliss, “A direct translation reading aid for the blind,” inProceedings of the IEEE, vol. 54, pp. 40–51, 1966.[4] J. Loomis, “Tactile pattern perception,”Perception, vol. 10, 1981.[5] J. Craig and C. Sherrick, “Dynamic tactile displays,” inTactual Perception: A Sourcebook(W. Schiff and E. Foulke, eds.),
Cambridge University Press, 1982.[6] J. Phillips and K. Johnson, “Tactile spatial resolution (i): Two-point discrimination, gap detection, grating resolution, and
letter recognition,”Journal of Neurophysiology, vol. 46, pp. 1177–1191, 1981.[7] J. Phillips and K. Johnson, “Tactile spatial resolution (ii): Neural representation of bars, edges and gratings in monkey primary
afferents,”Journal of Neurophysiology, vol. 46, pp. 1192–1203, December 1981.[8] J. Phillips and K. Johnson, “Tactile spatial resolution (iii):a continuum mechanics model of skin predicting mechanorecptor
responses to bars, edges and gratings,”Journal of Neurophysiology, vol. 46, pp. 1204–1225, December 1981.[9] M. Kurze, L. Reichert, and T. Strothotte, “Access to business graphics for blind people,”Proceedings of the RESNA ’94
Annual Conference, June 17-22 1994.[10] R. Klatzky, S. Lederman, and V. Metzger, “Identifying objects by touch: An “expert system”,”Perception and Psychophysics,
vol. 37, no. 4, 1985.[11] R. Klatzky, S. Lederman, and C. Reed, “There’s more to touch than meets the eye: The salience of object attributes for haptics
with and without vision,”Journal of Experimental Psychology: General, vol. 116, 1987.[12] K. Kokjer, “The information capacity of the human fingertip,”IEEE Transactions on Systems, Man, and Cybernetics,
vol. SMC-17, no. 1, 1987.[13] T. P. Way and K. E. Barner, “Automatic visual to tactile translation, part I: Human factors, access methods and image manip-
ulation,” IEEE Transactions on Rehabilitation Engineering, vol. 5, pp. 81–94, Mar. 1997.[14] T. P. Way and K. E. Barner, “Automatic visual to tactile translation, part II: Evaluation of the tactile image creation system,”
IEEE Transactions on Rehabilitation Engineering, vol. 5, pp. 95–105, Mar. 1997.[15] S. Hernandez and K. Barner, “Joint region merging criteria for watershed-based image segm entation,” in2000 International
Conference on Image Processing, (Vancouver, BC, Canada), September 10-13 2000.[16] S. Hernandez and K. E. Barner, “Color image segmentation using watersheds and joint region merging criteria,”IEEE Trans-
actions on Image Processing, Aug. 2000. Submitted for publication.[17] S. Hernandez and K. Barner, “Tactile imaging using watershed-based image segmentation,” inASSETS 2000, (Arlington,
Virginia, USA), November 13-15 2000.[18] W. Schiff and E. Foulke,Tactual perception: a sourcebook. Cambridge University Press, 1982.[19] R. A. Ulichney,Digital Halftoning. Cambridge, MA: MIT Press, 1987.[20] D. L. Lau,Modern Digital Halftoning. PhD thesis, University of Delaware, Newark, Delaware, USA, May 1999.[21] T. N. Pappas and D. L. Neuhoff, “Model-based halftoning,” inProceedings of SPIE, Human Vision, Vision Processing and
Digital Display II (B. E. Rogowitz, M. H. Brill, and J. P. Allebach, eds.), vol. 1453, pp. 244–255, June 1991.
29
[22] T. N. Pappas, “Model-based techniques for digital halftoning,” inProceedings, International Conference on Image Processing(B. Werner, ed.), vol. II, pp. 26–30, IEEE, 1994.
[23] J. Gardner, “Tactile graphics: An overview and resource guide,” Available fromhttp://www.rit.edu/ easi/itd/itdv03n4/article2.html, 1999.
[24] D. Pawluk,A Viscoelastic Model of the Human Fingerpad and a Holistic Model of the Human Touch. PhD thesis, HarvardUniversity, Cambridge, Massachusetts, USA, May 1997.
[25] D. Pawluk and R. Howe, “A holistic model of human touch,” in5th Annual C.N.S. Meeting, 1996.[26] R. LaMotte and J. Whitehouse, “Tactile detection of a dot on a smooth surface: Peripheral neural events,”Journal of Neuro-
physiology, vol. 56, no. 4, 1986.[27] H. Schiffman,Sensation and Perception: An Itegrated Approach. John Wiley and Sons, New York, 1976.[28] T. Cornsweet,Visual Perception. New York: Academic press, 1971.[29] F. W. Campbell, “The transmission of spatial information through the visual system,”The Neurosciences Third Study Program,
pp. 95–103, 1974.[30] H. R. Kang,Digital Color Halftoning. SPIE Optical Engineering Press and IEEE Press,USA, 1999.[31] F. W. Campbell, R. H. Carpenter, and J. Levinson, “Visibility of aperiodic patterns compared with that of sinusoidal gratings,”
The Journal of Physiology, vol. 190, pp. 283–298, 1969.[32] F. W. Campbell, “The human eye as an optical filter,”Proceedings of the IEEE, vol. 56, no. 6, pp. 1009–1014, 1968.[33] R. W. Floyd and L. Steinberg, “An adaptive algorithm for spatial gray-scale,”Proceedings Society Information Display,
vol. 17, pp. 75–78, Feb. 1976.[34] J. Sullivan, L. Ray, and R. Miller, “Design of minimum visual modulation halftone patterns,”IEEE Transactions on Systems,
Man, and Cybernetics, vol. 21, pp. 33–38, Jan. 1991.[35] M. Rodriguez, “Graphic arts perspective on digital halftoning,” inProceedings of SPIE, Human Vision, Visual Processing,
and Digital Display V(B. E. Rogowitz and J. P. Allebach, eds.), vol. 2179, pp. 144–149, Feb. 1994.[36] R. Levien, “Output dependant feedback in error diffusion halftoning,” inIS&T’s Eighth International Congress on Advances
in Non-Impact Printing Technologies, (Williamsburg, Virginia, USA), pp. 280–282, October 25-30 1992.[37] J. F. Jarvis, C. N. Judice, and W. H. Ninke, “A survey of techniques for the display of continuous-tone pictures on bilevel
displays,”Computer Graphics and Image Processing, vol. 5, pp. 13–40, 1976.[38] P. Stucki, “Mecca-a multiple-error correcting computation algorithm for bilevel image hardcopy reproduction,” Tech. Rep.
RZ1060, IBM Research Laboratory, Zurich, Switzerland, 1981.
top related